Skip to content

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rok Github Banner

Rok

Rok is a Rust Offline Kit builder.

It packages the Rust/Cargo pieces a disconnected machine or coding agent normally tries to download later: locked dependency sources, the active Rust toolchain, installed target libraries, optional Cargo subcommands, explicitly requested native build executables and their runtime-library closure, and Python-side Rust build tools such as Maturin.

The default upload profile deliberately strips toolchain material such as HTML documentation, man pages, and rust-src that ordinary compile/check work does not require. The point is to make a kit small enough to transfer while still useful enough to run real Rust tooling offline.

Quick start

From a Cargo repository or workspace:

./rok .

That creates:

<workspace>-rok.tar.gz

Extract the kit on the offline machine and point it at the separately supplied source repository:

./bin/rok-check /path/to/repository

Or run arbitrary Cargo commands through the offline wrapper:

cd /path/to/repository
/path/to/kit/bin/rok-cargo test --locked
/path/to/kit/bin/rok-cargo clippy --locked --all-targets
/path/to/kit/bin/rok-cargo build --release --locked

What v0.2 captures

Rok v0.2 keeps the Rust-specific closure as the project boundary and starts closing the native host side of Rust builds:

  • Cargo dependencies pinned by Cargo.lock;
  • crates.io and vendored git sources handled by cargo vendor;
  • Cargo's generated source mappings;
  • the active Rust toolchain/sysroot;
  • every target standard library already installed in that sysroot;
  • detected Rust components/tools carried by that toolchain;
  • explicitly requested external Cargo subcommands;
  • explicitly requested native executables used by build scripts/tooling;
  • explicitly requested pkg-config development packages with transitive package closure;
  • exact original .pc metadata plus relocatable kit-local .pc projections;
  • package-owned development headers when host package ownership can identify the exact subset;
  • concrete -L/-l link-library capture with static/shared distinction;
  • kit-local PKG_CONFIG_PATH, PKG_CONFIG_LIBDIR, CPATH, LIBRARY_PATH, LD_LIBRARY_PATH, and CMAKE_PREFIX_PATH;
  • Linux ldd discovery of those executables' shared-library closure;
  • capture of non-core shared libraries with loader-visible symlinks preserved;
  • explicit host-runtime classification for the C runtime/dynamic loader that Rok does not transplant;
  • automatically detected Maturin requirements from pyproject.toml;
  • a local-platform Python wheelhouse for captured Python-side Rust build tools;
  • offline pip-install verification when a usable wheel is captured;
  • Python version, ABI, and platform provenance for that wheelhouse;
  • enough provenance to identify the source lockfile and toolchain used to build the kit.

It also reports likely boundaries that are outside the current closure, such as undeclared native packages, git submodules, uncaptured external commands launched from build.rs, and obvious build-script network cues.

Rok does not claim those remaining boundaries are automatically captured. See ROADMAP.md. A future Lean/Lake driver is documented separately in docs/FUTURE_LEAN_DRIVER.md, but is deliberately not part of the current native-work release.

Commands

Build

rok [build] [OPTIONS] [REPO]

The build word is optional, so these are equivalent:

rok .
rok build .

Inspect

rok inspect .

Shows the resolved workspace, lockfile status, active toolchain, host triple, installed Rust targets, detected components, workspace build.rs count, and declared git submodules.

Verify a generated kit

rok verify ./my-project-rok.tar.gz

You can also verify an already extracted kit directory.

Profiles

upload — default

rok . --profile upload

Bundles vendored dependencies and the active toolchain while stripping:

  • share/doc
  • share/man
  • lib/rustlib/src

This is the profile intended for transfer-constrained workflows such as giving a repository and its Rust environment to an offline coding agent.

dev

rok . --profile dev

Same base closure, but keeps rust-src for richer tooling.

deps

rok . --profile deps

Vendors dependency sources without copying the Rust toolchain. The receiving environment must already have a compatible Rust/Cargo installation.

Rust targets

The copied sysroot already carries every target standard library installed in that toolchain. Rok inventories those targets in MANIFEST.txt.

If a repository requires a particular target, turn that assumption into a gate:

rok . --require-target wasm32-unknown-unknown

Rok refuses to package the kit if the required target is absent instead of producing an incomplete bundle that fails later.

Repeat the option for multiple targets.

Rust components

Rok detects common installed components/tools including:

  • clippy
  • rustfmt
  • rust-src
  • rust-analyzer
  • LLVM tools

Require one explicitly:

rok . --require-component clippy
rok . --require-component rustfmt

When rustup is available, unknown component names can also be checked against the active installation.

The full sysroot is copied before upload-profile pruning, so installed executable components inside the sysroot travel with the toolchain automatically.

Cargo subcommands

External Cargo tools live outside the Rust sysroot in many installations. Capture the ones a repository workflow actually needs:

rok . \
  --cargo-tool nextest \
  --cargo-tool llvm-cov

nextest and cargo-nextest are equivalent names here.

To capture every external cargo-* executable currently visible on PATH:

rok . --cargo-tools-all

That can make the kit significantly larger, so explicit tools are preferable for upload-sized kits.

Toolchain-native commands such as cargo-clippy or cargo-fmt are not duplicated when they already exist in the copied sysroot.

Native build executables

Cargo source closure is not enough when build.rs invokes host tools such as protoc, cmake, ninja, assemblers, or code generators.

Rok v0.2 can explicitly capture those executables:

rok . \
  --native-tool protoc \
  --native-tool cmake

A tool may also be supplied by path:

rok . --native-tool /opt/toolchain/bin/protoc

For every requested executable Rok records:

  • requested name/path;
  • resolved host path;
  • captured kit path;
  • SHA-256;
  • a best-effort --version line;
  • its shared-library closure when ldd can inspect it.

On Linux, non-core shared libraries are copied into:

native/lib/

while the host C runtime and dynamic loader are recorded as host-runtime requirements rather than naively copying glibc into the archive. Loader-visible symlink names are recreated with relative links so the payload remains relocatable.

Captured tools live under:

native/bin/

and are automatically added to PATH by rok-cargo. Their captured libraries are exposed through LD_LIBRARY_PATH for the command.

They can also be invoked directly through:

/path/to/kit/bin/rok-native protoc --version

This is intentionally explicit in v0.2. Rok's static build.rs scan can show candidate executables, but it does not automatically copy every command string it sees. The later execution-probe work will distinguish commands a real build actually needs from dead code, optional paths, or platform-specific branches.

Two provenance tables describe the payload:

NATIVE_TOOLS.tsv
NATIVE_LIBS.tsv

NATIVE_LIBS.tsv distinguishes captured shared objects from libraries that remain requirements on a compatible host runtime.

Native development packages

When a Rust crate needs headers, linker inputs, or .pc metadata, capture the package explicitly by its pkg-config name:

rok . --pkg openssl
rok . --pkg zlib --pkg sqlite3

--pkg treats the creating host's pkg-config result as the package-resolution authority. Rok records the exact package version and original .pc bytes, creating-host prefix/include/lib paths, exact dynamic/static flag outputs, and public/private dependency constraints. It captures public and static-private link libraries plus headers without copying an entire system include tree when ownership metadata can identify the development package's files.

On Debian-family hosts Rok uses dpkg-query ownership when available; RPM ownership is also supported. If a package resolves only to a broad include root such as /usr/include and no ownership provider can identify the package-specific header subset, Rok refuses to claim closure instead of copying the whole host SDK.

The extracted kit contains:

native/pkgconfig/original/    exact source .pc metadata
native/pkgconfig/relocated/   generated relocatable .pc projections
native/include-roots/         captured header roots
native/lib/                   development + runtime libraries
NATIVE_PACKAGES.tsv
NATIVE_PACKAGE_FLAGS.tsv
NATIVE_PACKAGE_DEPS.tsv
NATIVE_INCLUDE_ROOTS.tsv
NATIVE_HEADERS.tsv
NATIVE_LINK_LIBS.tsv

Query the relocated metadata directly:

/path/to/kit/bin/rok-pkg-config --cflags --libs openssl

rok-cargo automatically exposes the native development environment through PKG_CONFIG_PATH, PKG_CONFIG_LIBDIR, CPATH, LIBRARY_PATH, LD_LIBRARY_PATH, CMAKE_PREFIX_PATH, and PATH. Original host paths remain provenance; the generated execution projection points at the extracted kit.

The shared C runtime and toolchain-level system libraries remain explicit host compatibility requirements rather than being silently transplanted.

Maturin / Python-Rust bridge

Rok treats Maturin as part of the Rust build boundary rather than requiring the receiving environment to fetch it later.

If a workspace contains a pyproject.toml with a Maturin build requirement, Maturin build backend, or [tool.maturin] section, the default build automatically captures it:

[build-system]
requires = ["maturin==1.15.0"]
build-backend = "maturin"
rok .

Rok asks pip to download the requested package and its Python dependency closure into:

python/wheels/

When the requested tool itself is available as a wheel, Rok immediately proves that the wheelhouse can install it without an index or network access. If pip can only provide a source distribution, Rok keeps the artifact but explicitly marks the tool sdist-only instead of pretending the offline install is proven.

You can also request Maturin explicitly when it is not declared in the repository:

rok . --python-tool maturin==1.15.0

--python-tool is repeatable. Automatic detection in v0.1 is intentionally limited to Maturin so Rok does not start guessing which arbitrary Python packages belong to the Rust build boundary.

Choose a particular Python interpreter with:

rok . --python /path/to/python3

Or disable Maturin auto-capture deliberately:

rok . --no-python-auto

On the receiving machine, install only from the captured wheelhouse:

/path/to/kit/bin/rok-pip install maturin

rok-pip sets PIP_NO_INDEX=1 and PIP_FIND_LINKS to the kit wheelhouse. It never turns the wheelhouse into a claim that Python itself was bundled: the receiving machine still needs a compatible Python interpreter and pip. The creating interpreter's version, cache tag, SOABI, and platform are recorded in MANIFEST.txt.

Validation

By default Rok performs an offline Cargo metadata check using the relocated kit.

Choose a stronger gate:

rok . --validate check
rok . --validate test

Or skip execution validation entirely:

rok . --validate none

Validation happens through the generated rok-cargo wrapper, so Cargo is already operating with the vendored sources and offline mode that the receiving environment will use.

Transfer-size gate

Rok reports staged payload, vendored dependency, toolchain, Cargo-tool, native-tool/library, and captured Python-tool sizes.

If you know the transfer channel has a practical limit, enforce your own budget:

rok . --max-mib 500

Rok refuses archive creation when the staged payload exceeds the chosen limit. There is intentionally no baked-in universal upload limit because receiving systems differ.

Generated kit

A normal upload-profile kit looks like:

.
├── README.txt
├── MANIFEST.txt
├── RUST_CLOSURE_REPORT.txt
├── CARGO_TOOLS.tsv
├── NATIVE_TOOLS.tsv
├── NATIVE_LIBS.tsv
├── NATIVE_PACKAGES.tsv
├── NATIVE_PACKAGE_FLAGS.tsv
├── NATIVE_PACKAGE_DEPS.tsv
├── NATIVE_INCLUDE_ROOTS.tsv
├── NATIVE_HEADERS.tsv
├── NATIVE_LINK_LIBS.tsv
├── PYTHON_TOOLS.tsv
├── vendor-config-snippet.toml
├── vendor/
│   └── ...
├── toolchain/
│   └── ...
├── tools/
│   └── cargo/bin/...
├── native/
│   ├── bin/...
│   ├── lib/...
│   ├── include-roots/...
│   └── pkgconfig/
│       ├── original/...
│       └── relocated/...
├── env/
│   └── native-env.sh
├── python/
│   └── wheels/...
└── bin/
    ├── rok-cargo
    ├── rok-check
    ├── rok-test
    ├── rok-native
    ├── rok-pkg-config
    └── rok-pip

The source repository itself is intentionally not copied. This keeps the environment kit reusable with an independently supplied source snapshot and avoids silently changing what source was reviewed.

Why preserve Cargo's generated vendor config?

The original one-off script that inspired Rok replaced the vendor configuration with a simple crates.io mapping. That works for simple dependency graphs, but a general tool should not assume every project has only the default registry source.

Rok preserves the configuration emitted by cargo vendor and changes only the vendored directory at runtime so the extracted kit remains relocatable. This preserves source-specific mappings Cargo decided were necessary.

Closure report

Every kit contains RUST_CLOSURE_REPORT.txt.

The report is deliberately conservative. It identifies signals such as:

  • workspace and dependency build.rs files;
  • links = declarations;
  • git submodules;
  • literal Command::new("...") executables in build scripts;
  • obvious URL/curl/wget/git-clone cues in build scripts.

Those are not automatically errors. They are places where a Rust build can depend on something not represented by vendored Cargo source and the copied Rust toolchain.

Rok v0.2.1 marks explicitly captured native tools and requested pkg-config development packages as closed only for the artifacts it actually captured. It distinguishes bundled libraries from host-runtime requirements and continues reporting external boundaries it has not closed yet.

Portability

A full Rok kit is host-specific because it contains compiled Rust and Cargo binaries.

For example, a Linux x86-64 kit should be expected to require a compatible Linux x86-64 runtime. Copying a target standard library such as wasm32-unknown-unknown does not make the host compiler itself portable to another operating system or CPU architecture.

Use --profile deps if only the vendored Rust source closure needs to move between machines and the receiving host will supply its own compatible compiler.

Requirements

To create a full kit:

  • Bash
  • rustc
  • Cargo
  • tar
  • mktemp
  • sha256sum or shasum
  • Python + pip when Maturin is auto-detected or --python-tool is used
  • ldd on Linux for shared-library closure inspection when --native-tool is used (capture still records an uninspected boundary if unavailable)

The Cargo workspace must have a Cargo.lock.

cargo vendor may use the network while creating the kit when locked dependencies are not already cached locally. The generated kit is what is designed to run offline.

Tests

bash -n rok
bash tests/test_smoke.sh
bash tests/test_failures.sh

The tests use fake Rust/Cargo/pip/ldd behavior and synthetic Rust/Maturin/native-tool fixtures, so Rok's packaging, relocation, component/target gates, Cargo-tool capture, native executable/library closure, Maturin auto-detection, offline wheelhouse verification, and failure behavior can be exercised without downloading crates or Python packages.

License

MIT.

Releases

Packages

Used by

Contributors

Languages