chore: solver service byom security improvements - refactor to not in… - #2695
mswiderski wants to merge 1 commit into
Conversation
27b6e96 to
5282daf
Compare
5282daf to
d05a2d1
Compare
| * The per data set lock is taken by the operations of the storage service, and the composite operations take it | ||
| * around several of them, so it has to be reentrant without ever letting a second thread in. | ||
| */ | ||
| class AbstractStorageServiceLockTest { |
There was a problem hiding this comment.
Is it worth having a test that checks internal locking?
| Field field = AbstractStorageService.class.getDeclaredField("locks"); | ||
| field.setAccessible(true); |
There was a problem hiding this comment.
Especially if it needs to hack private fields?
| Azure(AZURE_STORAGE), | ||
| InMemory(INMEMORY_STORAGE), | ||
| FileSystem(FILESYSTEM_STORAGE); | ||
| FileSystem(FILESYSTEM_STORAGE), |
There was a problem hiding this comment.
Indeed, this is unusual naming for enum entries. Better to change all of them.
There was a problem hiding this comment.
I removed this enum from solver service as most of the storages were removed as well. No point for keeping it here.
| * Produces StorageObjectMapper instances with storage-specific (de)serialization settings. | ||
| */ | ||
| @ApplicationScoped | ||
| public class StorageObjectMapperProducer { |
There was a problem hiding this comment.
No beans in the definition module, please. It can silently alternate the behavior in any module that depends on it.
There was a problem hiding this comment.
removed ApplicationScoped from the class.
| /** | ||
| * What was last read, replaced as a whole so that a reader never sees a half-written one. | ||
| */ | ||
| private volatile Reading reading; |
There was a problem hiding this comment.
The SonarCloud comment is correct, but more importantly, what's the point of volatile here?
There was a problem hiding this comment.
don't think it is correct, the Reading is immutable object so it won't change by different thread but the reference can be changed when reloaded and that's why volatile is used, to immediately propagate that to other threads.
Service account token is rotated every hour so it needs to be periodically reloaded.
| @@ -0,0 +1,109 @@ | |||
| package ai.timefold.solver.service.definition.internal.platform; | |||
There was a problem hiding this comment.
Is this class going to be imported by the platform?
There was a problem hiding this comment.
it is used also by one component of the platform. That's why in the package, I could make a copy of it if you prefer that?
…clude storages into models and intorduce single proxy component instead
d05a2d1 to
b715230
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Cache correctness, metadata loss, compression detection, and mapper configuration defects must be resolved.
Review effort: Balanced
Findings: 3
Open (5)
What changed in this PR
Refactors solver-service storage into a model-agnostic byte-stream layer and adds authenticated service communication.
Changes:
- Centralizes serialization, compression, caching, and retries in
AbstractStorageService. - Moves in-memory storage into a dedicated module and simplifies Quarkus generation.
- Adds Kubernetes service-account authentication for map-service calls.
| File | Description |
|---|---|
service/worker/.../TestdataStorageService.java |
Adapts test storage service to the new API. |
service/worker/.../TestdataStorage.java |
Uses the relocated in-memory storage. |
service/worker/.../DefaultSolverWorkerFacadeTest.java |
Configures serialization for storage tests. |
service/worker/pom.xml |
Adds the in-memory test dependency. |
service/storage-inmemory/.../InMemoryStorage.java |
Implements model-agnostic in-memory storage. |
service/storage-inmemory/pom.xml |
Defines the new storage module. |
service/quarkus/runtime/pom.xml |
Includes default in-memory storage at runtime. |
service/quarkus/integration-tests/.../ModelsExtensionTest.java |
Updates storage injection expectations. |
service/quarkus/deployment/.../ModelsExtensionStorageGenerationTest.java |
Updates generated-storage tests. |
service/quarkus/deployment/.../TimefoldStorageProcessor.java |
Removes typed storage generation. |
service/quarkus/deployment/pom.xml |
Adds in-memory storage for tests. |
service/pom.xml |
Registers the new module. |
service/maps/service-client/.../DummyStorageService.java |
Implements the output-type hook. |
service/maps/service-client/.../ServiceAccountTokenHeadersFactoryTest.java |
Tests bearer-header behavior. |
service/maps/service-client/.../ServiceAccountTokenHeadersFactory.java |
Adds service-account authorization headers. |
service/maps/service-client/.../MapServiceClient.java |
Registers the header factory. |
service/facade/service-parent/pom.xml |
Removes direct enterprise storage dependencies. |
service/definition/.../AbstractStorageServiceRetryTest.java |
Tests retry behavior. |
service/definition/.../ServiceAccountTokenTest.java |
Tests token loading and refresh. |
service/definition/.../CompressionUtilsTest.java |
Tests compression utilities. |
service/definition/.../SupportedStorages.java |
Removes obsolete storage identifiers. |
service/definition/.../StorageItem.java |
Introduces listed-storage metadata. |
service/definition/.../StorageContent.java |
Introduces streamed storage content. |
service/definition/.../Storage.java |
Reworks storage around raw streams. |
service/definition/.../AbstractStorageService.java |
Adds serialization, caching, locking, and retries. |
service/definition/.../ServiceAccountToken.java |
Loads rotating Kubernetes credentials. |
service/definition/.../StorageObjectMapperWrapper.java |
Wraps the storage-specific mapper. |
service/definition/.../StorageObjectMapperProducer.java |
Produces the configured mapper. |
service/definition/.../inmemory/InMemoryStorage.java |
Removes the old typed implementation. |
service/definition/.../CompressionUtils.java |
Adds gzip stream utilities. |
build/bom/pom.xml |
Publishes the new storage module. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| public static InputStream decompressIfNeeded(InputStream source) throws IOException { | ||
| var pushback = new PushbackInputStream(source, GZIP_MAGIC_LENGTH); | ||
| var magic = new byte[GZIP_MAGIC_LENGTH]; | ||
| int read = pushback.read(magic); |
| * 2. Enable case-insensitive property matching as some cloud storages are case-insensitive. | ||
| */ | ||
| objectMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); | ||
| objectMapper.getDeserializationConfig().with(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true); |
| if (hasSolverStatus(item.attributes())) { | ||
| return mapper().convertValue(item.attributes(), Metadata.class); |
| /** | ||
| * The value of the {@value #AUTHORIZATION_HEADER} header to send, empty when there is no token to send. | ||
| */ | ||
| public Optional<String> authorization() { |
| /** | ||
| * Property that turns the caching of the model output and the metadata on and off. | ||
| */ | ||
| public static final String USE_CACHE_PROPERTY = "timefold.storage.use-cache"; |


…clude storages into models and intorduce single proxy component instead