Feat/standalone module configuration - #223
Draft
wyli wants to merge 18 commits into
Draft
Conversation
Generate launcher-free Modules with an exact requirements-cli.txt contract, activate metadata-derived project defaults before CLI imports, and reject host or container version drift before lifecycle work.\n\nSupport safe missing, empty, and Git-only creation targets, package the canonical wheelhouse/Docker flow, and cover installed-artifact creation. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
…e-create Signed-off-by: Wenqi Li <wenqil@nvidia.com> # Conflicts: # src/holoscan_cli/setup_scripts/requirements.template.txt
Add a versioned [tool.holoscan] table so a source project states its own
lifecycle policy instead of exporting it from a repository shell wrapper. The
5.0 early-access samples use this to delete ./holohub without requiring every
future Holoscan repository to become wrapperless.
Schema version 1 covers project identity prefixes, metadata search paths, CMake
build type, the project CTest script, the CUDA line, and an SDK block: exact
version, architecture-keyed base images, relative search hints carrying an
{arch} token, and parent-search and read-only mount capabilities. Unknown keys,
wrong types, unsupported schema versions, absolute or escaping paths, and
unsupported placeholders fail before any lifecycle work starts, so this stays
capability configuration rather than a general-purpose task runner.
Resolution order is command option, typed environment override, committed
value, metadata-derived default, then CLI default. The resolver owns
architecture normalization so a project never writes $(uname -m) in a path,
HOLOSCAN_SDK_ROOT remains the single machine-local escape hatch and now fails
closed instead of silently resolving another tree, and container recursion
prefers the CLI-owned /workspace/holoscan-sdk mount over host-side hints that
do not describe the container.
Add --project-root as a global option, and report the resolved project profile
with the provenance of each value through `holoscan version --json`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
The generated Module's quick start creates .venv at the repository root and
installs holoscan-cli into it. That wheel ships the unrendered cookiecutter
module template, so the metadata validator walked into site-packages and
validated template files: placeholder versions fail the version pattern, and
files carrying Jinja control blocks are not valid JSON at all. Following the
quick start therefore broke the module's own lint.
Add .venv and venv to the excluded directory names, alongside .git, which the
sample repository's validator already excluded.
Verified by generating a Module, creating .venv per its quick start, installing
the CLI wheel into it, and running the generated validator: it reported four
template metadata.json files under .venv before the change and validates only
the module's own five files after it.
Note the template is rendered through Jinja, so `{{ ... }}` is executable even
inside comments; the explanatory comment here deliberately avoids it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Both mechanisms existed to run an unpublished CLI build, which is a CLI-maintenance task rather than something Holoscan developers do, so neither earns its place in every generated Module. The generated Dockerfile bind-mounted a .holoscan-cli-wheelhouse directory for the install step and passed --find-links, requiring the directory, a committed .gitignore inside it, wheel-content assertions, and matching instructions in the Module README and CI workflow. It now installs the exact pin from the configured package index like any other requirement. local_source_build_context_args() emitted `--build-context holoscan-cli-src=...` when HOLOSCAN_CLI_SOURCE was set. Nothing consumed that context: no Dockerfile in the template or in any consuming repository mounted it, so the flag was passed to every docker build and ignored. HOLOSCAN_CLI_SOURCE itself is unchanged; it remains a wrapper convention that env-info reports. An unpublished build is still usable. `pip install <wheel>` satisfies the exact version contract on the host, and the Dockerfile already honors PIP_INDEX_URL and PIP_EXTRA_INDEX_URL for a local or internal index, which the Module DEVELOPER.md now documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
…table Two defects had the same cause: project discovery runs before the command parser, so decisions it made could not be revised by a command option. Discovery rejected an invalid HOLOSCAN_SDK_ROOT by raising. Because that happened before --local-sdk-root was parsed, a valid command-line SDK path could not override a broken environment value, contradicting the documented order of command option, then environment, then committed configuration. The same raise also failed commands that never need an SDK. Discovery now records the problem as a warning and still refuses to fall back to a committed hint, so an explicit override is never silently replaced by a different tree. Validation moves to the point of use, where both sources are visible at once. get_local_sdk_options() previously logged an advisory line when the path was not an SDK installation and mounted it anyway, so an invalid --local-sdk-root became a bind mount of a nonexistent directory. It now validates whichever source it selected, names that source in the error, and fails closed. `--version` no longer performs project discovery at all: it answers which CLI is installed, and an unreadable project or a schema newer than this CLI must not make that unanswerable. `version` still reports the project, but a resolution failure becomes a warning plus a project_error field rather than exit 2, which matters most for an older CLI meeting a newer project. Lifecycle commands remain fail-closed. Verified: an invalid environment plus a valid --local-sdk-root now mounts the flag's path; an invalid --local-sdk-root exits 1 instead of mounting it; an invalid environment alone still exits 1 when an SDK is required; and both --version and version report 5.0.0a14 against a project declaring schema-version 99. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
--project-root rejected a directory that is not a recognized source-project or Module root, but the environment form accepted any directory that merely existed. Pointing it at an arbitrary path therefore produced an empty project whose requirements-cli.txt and [tool.holoscan] were never read, so the Module version contract and all project configuration were silently skipped rather than enforced. Apply the same recognition check to the environment root. The two forms still differ in remedy, deliberately: an explicit --project-root is an assertion and fails, while HOLOSCAN_CLI_ROOT is ambient and warns before falling back to discovery from the current directory, matching the existing behavior for a root that does not exist. Verified that HOLOSCAN_CLI_ROOT=/tmp now warns and falls back, and that the HoloHub, Isaac OS, and early-access sample roots are still accepted without warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
The generated Dockerfile installed pytest-timeout, build, and scikit-build-core
with the system pip, but every command that uses them resolves them from the
CLI environment instead.
holoscan build and holoscan package configure CMake with
-DPython3_EXECUTABLE=<sys.executable>, which is /opt/holoscan-cli/bin/python in
the generated image. The Module's tests/CMakeLists.txt runs
${Python3_EXECUTABLE} -m pytest, and package --pkg-generator WHEEL runs
<that python> -m build after checking for the module with importlib. Both
therefore looked in an environment where nothing had been installed, so wheel
packaging failed on a missing build frontend and the pytest target had no test
runner.
Install all three into /opt/holoscan-cli. The early-access sample keeps its
model-export dependencies on the system interpreter, which is correct there
because prepare_model.sh invokes python3 directly rather than through CMake.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
The generated Dockerfile created /opt/holoscan-cli, installed the pin there, symlinked the console script onto PATH, re-derived the expected version from requirements-cli.txt in an inline Python one-liner, and asserted it matched -- ten lines and an extra apt package for python3-venv. The isolation it bought does not exist. holoscan-cli declares no runtime dependencies on Python 3.11+, so installing it into the image interpreter adds nothing else to the image. The separation was also never real: build and package configure CMake with -DPython3_EXECUTABLE=<the CLI's python>, so the venv was already the interpreter that compiled the project, ran pytest, and built wheels. That split is what made the packaging tooling land in an interpreter nothing used. Install with the image's own pip. The console script lands on PATH by itself, so the symlink and the `command -v` check go away, and pip already guarantees the pinned version that the removed assertion re-checked -- as does the CLI's own runtime contract. Verified against the pinned SDK image: holoscan resolves to /usr/local/bin/holoscan with a /usr/bin/python3 shebang, matching sys.executable, and `pip show holoscan-cli` reports no requirements. PEP 668 does not apply here. The marker covers the distro tree at /usr/lib/python3.12, while the image's pip targets /usr/local, so no --break-system-packages is needed; a real build confirms it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
The generated Dockerfile declared ARG PIP_INDEX_URL, PIP_EXTRA_INDEX_URL, and PIP_NO_INDEX so a caller could redirect pip during the image build. Nothing in the normal path needs them: holoscan-cli is published to PyPI, so the pinned requirement resolves from the default index with no configuration, and PIP_NO_INDEX existed only for the removed local wheelhouse. Declaring an ARG is what makes `--build-arg` reach the build environment, so removing them also removes the container-side index override. The host-side `--extra-index-url` guidance stays, since that is where a pre-release actually needs it, and the generated requirements-cli.txt still carries that hint for a pre-release pin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Several comments and template docs explained the reasoning at more length than the code needs. Keep the fact and drop the argument. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
…onfig
A project could only supply persistent docker arguments through a wrapper that
exported HOLOSCAN_CLI_DEFAULT_DOCKER_BUILD_ARGS and _RUN_ARGS. Those values are
usually static -- capability flags, ulimits, namespace sharing, a secret
reference -- so they belong in committed configuration where the CLI can report
them.
Add three keys to [tool.holoscan]:
docker-build-args = ["--secret", "id=token,env=GITLAB_ACCESS_TOKEN"]
docker-run-args = ["--privileged", "--pid=host", "--ulimit", "rtprio=10"]
forward-env = ["IS_CI_BUILD", "SCCACHE_MEMCACHED_ENDPOINT"]
The argument arrays feed the existing environment variables, so nothing
downstream changes. forward-env names host variables to pass into the container,
and only those that are actually set are forwarded. All three appear in
`holoscan version --json`, so a reader -- or an agent -- can see what a project
adds to its docker commands without reading a shell script.
This covers values that already exist in the environment. Computing a value
still needs a script; that remains outside committed configuration.
Also make schema-version optional. It was required boilerplate in every project
while the strict unknown-key check already rejects a newer schema. Declaring it
still pins the version, so an older CLI rejects schema-version 2 by name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Add layered project configuration, safe effective-value reporting, additive option resets, and the clean-SDK standalone Module workflow. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Automatically add the bundled Xvfb setup layer when container tests build, preserve the --no-xvfb opt-out, and report an actionable error when a reused image lacks xvfb-run. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Pass the actual test working directory to CTest and let the bundled driver preserve it, so standalone Modules do not configure the CLI installation directory. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Give the setup layer a valid fallback base image for BuildKit checks and report unnamed project selection as an informational repository-wide default. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Wenqi Li <wenqil@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Dependency ReviewThe following issues were found:
License Issuespyproject.toml
src/holoscan_cli/setup_scripts/requirements.template.txt
OpenSSF Scorecard
Scanned Files
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.