feat(dataconverter): add custom DataConverter samples#288
Conversation
Add a new package with three production-ready custom implementations following the take-and-go layout: gzip compression, AES-256-GCM encryption, and an S3 / claim-check offload pattern (with a zero-config local-filesystem default and a commented AWS SDK v2 stub). One worker hosts all three on three task lists and prints per-sample stats banners on startup. Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
| try { | ||
| ByteArrayOutputStream out = new ByteArrayOutputStream(); | ||
| try (GZIPOutputStream gzip = new GZIPOutputStream(out)) { | ||
| gzip.write(jsonBytes); | ||
| } | ||
| return out.toByteArray(); | ||
| } catch (IOException e) { | ||
| throw new DataConverterException("Failed to gzip-compress JSON payload", e); | ||
| } |
There was a problem hiding this comment.
| try { | |
| ByteArrayOutputStream out = new ByteArrayOutputStream(); | |
| try (GZIPOutputStream gzip = new GZIPOutputStream(out)) { | |
| gzip.write(jsonBytes); | |
| } | |
| return out.toByteArray(); | |
| } catch (IOException e) { | |
| throw new DataConverterException("Failed to gzip-compress JSON payload", e); | |
| } | |
| ByteArrayOutputStream out = new ByteArrayOutputStream(); | |
| try (GZIPOutputStream gzip = new GZIPOutputStream(out)) { | |
| gzip.write(jsonBytes); | |
| } catch (IOException e) { | |
| throw new DataConverterException("Failed to gzip-compress JSON payload", e); | |
| } | |
| return out.toByteArray(); |
There was a problem hiding this comment.
Great suggestion and taken.
| * production, use S3 object lifecycle policies to automatically expire old blobs. | ||
| * ============================================================================= | ||
| */ | ||
| public final class S3OffloadDataConverter implements DataConverter { |
There was a problem hiding this comment.
You can use MinIO to integrate with real S3API. Otherwise, rename the example as largeblob to avoid confusion. There are users who are on GCP using GCS.
There was a problem hiding this comment.
Really good point. The implementation is already storage-agnostic (everything goes through a BlobStore interface), so the "S3" branding ends up being a bit of an apology.
I renamed the sample to something provider-neutral in this PR claimcheck since that's the formal pattern name, but happy to go with largeblob if you prefer.
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Signed-off-by: “Kevin” <kevlar_ksb@yahoo.com>
Code Review ✅ Approved 1 resolved / 1 findingsIntroduces Gzip compression, AES-256-GCM encryption, and S3-based claim-check DataConverter samples with updated documentation. The missing GZIPOutputStream import was resolved, and no other issues were found. ✅ 1 resolved✅ Bug: GZIPOutputStream removed —
|
| Auto-apply | Compact |
|
|
Was this helpful? React with 👍 / 👎 | Gitar
[Which sample(s) or area?]
dataconverter, rootREADME.md, and dataconverter tests.[What changed?]
[Why?]
[How did you test it?]
Run the targeted tests:
./gradlew test --tests 'com.uber.cadence.samples.dataconverter.*'Run the samples manually:
Start Cadence locally.
Register the sample domain if needed:
./gradlew -q execute -PmainClass=com.uber.cadence.samples.common.RegisterDomain./gradlew -q execute -PmainClass=com.uber.cadence.samples.dataconverter.DataConverterWorker./gradlew -q execute -PmainClass=com.uber.cadence.samples.dataconverter.CompressionStarter./gradlew -q execute -PmainClass=com.uber.cadence.samples.dataconverter.EncryptionStarter./gradlew -q execute -PmainClass=com.uber.cadence.samples.dataconverter.S3OffloadStarterAlso verified the final diff with:
git diff --check[Potential risks]
Low. The changes are additive and isolated to the new
dataconvertersample package plus README/test coverage. Existing samples should be unaffected; the main caveat is that users must adapt the demo encryption key and local BlobStore before using these patterns in production.[Release notes]
[Documentation Changes]
README.mdwith the new DataConverter sample entry.