feat(helm): add optional OASF-SDK extractor server subchart - #2073
feat(helm): add optional OASF-SDK extractor server subchart#2073akijakya wants to merge 2 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Signed-off-by: András Jáky <ajaky@cisco.com>
Signed-off-by: András Jáky <ajaky@cisco.com>
d441098 to
f68bc94
Compare
There was a problem hiding this comment.
Pull request overview
Adds an optional OASF-SDK extractor subchart for clustered natural-language search.
Changes:
- Adds and configures the pinned OASF-SDK Helm dependency.
- Derives the extractor’s cluster DNS address while respecting explicit overrides.
- Enables the extractor in Kind and documents deployment.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
tests/e2e/local/testenv/kind/dir-chart-values.yaml |
Enables the extractor in Kind. |
install/charts/dir/values.yaml |
Exposes umbrella-chart defaults. |
install/charts/dir/apiserver/values.yaml |
Defines extractor and subchart settings. |
install/charts/dir/apiserver/templates/configmap.yaml |
Injects the resolved extractor address. |
install/charts/dir/apiserver/templates/_helpers.tpl |
Resolves Service name and address. |
install/charts/dir/apiserver/Chart.yaml |
Adds the optional dependency. |
install/charts/dir/apiserver/Chart.lock |
Locks OASF-SDK v1.3.0. |
docs/content/dir/dir-deployment-kubernetes.md |
Documents extractor deployment and operation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| oasf-sdk: | ||
| enabled: true |
There was a problem hiding this comment.
Both observations check out, and the first one is worse than described.
15_nl_search_test.go does not skip only in Kind — it skips everywhere, always. The guard looks for the manifest at ~/.agntcy/oasf-sdk/manifest.json, but dirctl init provisions into ~/.agntcy/oasf-sdk/extractor/ (extractor.DefaultAssetDir, utils/extractor/config.go:31), which is the path 16_extractor_enricher_test.go:38 checks. On a machine that has run dirctl init the real manifest exists and the one the guard looks for does not, so the suite has been silently skipping in e2e:test:local:local too, since #1770. Filed as #2090, and worth its own PR rather than folding in here: re-enabling a suite that has not executed in months will surface whatever has drifted, and that deserves separate review.
The CI point is also correct — every entry in the reusable-test-e2e.yaml matrix is a local variant, so e2e:test:local:kind never runs. Filed as #2091 along with the missing test itself.
Not adding that Kind test path in this PR, though. #1907 scopes it as a separate follow-up ("Needed by the remote-mode e2e"), and it needs a new CI job, a test that drives the gateway rather than dirctl, and a runner that can reach the OASF schema URL — that is a change of a different shape to chart wiring.
Keeping the three lines here, with no claim that they are coverage. They make the deployment reachable for manual verification, which is how this PR was validated end to end: the apiserver logs OASF extractor resolved for gateway backend=remote address=dir-oasf-sdk.dir.svc.cluster.local:31234, and POST /v1/extract returns ranked OASF skills and domains through the gateway. The cost is near zero now that v1.3.0 embeds the model, so the environment only needs the OASF schema URL.
Problem Statement
The directory chart has no extractor, so the gateway's natural-language endpoints (
POST /v1/search,POST /v1/extract) have nothing to turn free text into OASF classes.dirctl initprovisions those assets locally, but there is nowhere to run it in a cluster.Proposed Changes
Adds
oasf-sdkv1.3.0 as an optional apiserver subchart, gatedcondition: oasf-sdk.enabledand defaulted off, mirroring the existing optionaloasfsubchart. When enabled,config.extractor.remote_addris derived from the rendered Service DNS name; an explicitly configured address always wins. Nothing is written toserver.config.ymlunless an address resolves, so the default render is unchanged. Also enables it in the local kind testenv and documents it in the Kubernetes deployment guide.The chart sets
OASF_SDK_EXTRACTOR_OASF_URL, which is required: the OASF-SDK server registersExtractorServiceonly when it is set, and without it the pod starts, passes its health check, and fails every call withunknown service agntcy.oasfsdk.extractor.v1.ExtractorService.Review notes
The values key is
oasf-sdk, notoasfSdkas the issue specified. Helm turns a dependency alias into the subchart's.Chart.Name, and the upstream chart names its container after it, so aliasing renders- name: oasfSdk— invalid, since container names must be lowercase DNS labels.v1.3.0 is pinned deliberately. It ships the embedding model inside the image, so the server no longer downloads it from
huggingface.coat startup and runs where that host is unreachable — at the cost of a ~90 MB pull. v1.2.0 is the hard floor regardless, since earlier chart versions always rendernodePort, which Kubernetes rejects on aClusterIPService (agntcy/oasf-sdk#179).The pod still needs the OASF schema URL reachable on every start, to fetch and embed the taxonomy before going ready — roughly 40 seconds. A new OASF release is picked up by restarting the pod.
Verification
Default render is byte-identical to before the change. With the subchart enabled the Service is
ClusterIPwith nonodePortand the apiserver config points at it. Verified end to end on the local kind testenv:POST /v1/extractreturns ranked OASF skills and domains through the gateway.task lint,task test:helmandtask test:unitpass.Closes #1907