docs(inference): add llama.cpp operator guide

[ericksoa]

Summary

Adds the first dedicated llama.cpp operator guide for NemoClaw.
The guide separates authenticated existing-server attachment from the experimental declarative DGX Spark managed path without changing runtime behavior or claiming support activation.

Related Issue

Related to #8144

Changes

• Publish one shared llama.cpp setup page for OpenClaw, Hermes, and LangChain Deep Agents Code.
• Document exact existing-server fingerprint, credential, fixed-port, and generic-compatible fallback behavior.
• Move the managed DGX Spark procedure into the dedicated page and document YAML authority, NVIDIA Nemotron defaults, shared Hugging Face cache use, security boundaries, diagnostics, recovery, cleanup, and the current absence of managed upgrade or rollback support.
• Add experimental llama.cpp provider rows to the canonical platform matrix and regenerate its provider tables.
• Preserve the former managed-guide anchor as a migration pointer.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification: Documentation-only change with no runtime behavior modification.
• Docs updated for user-facing behavior changes
• Docs not applicable — justification:
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent documentation review checked the guide against current main, the declarative recipe and preset, provider selection, credential handling, runtime-provider boundaries, lifecycle behavior, and deferred activation claims; no blocking findings remain.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: docs-updated
• Evidence: docs/inference/set-up-llama-cpp.mdx, docs/inference/choose-local-inference-server.mdx, docs/inference/choose-inference-provider.mdx, docs/reference/platform-support.mdx, and docs/index.yml were reviewed against current behavior, the NemoClaw writing rules, and the documentation style guide. ci/platform-matrix.json remains the generated provider-table authority.
• Agent: Codex Desktop

DGX Station Hardware Evidence

• Tested on DGX Station
• Tested commit:
• Station profile/scenario:
• Result:
• Supporting evidence:

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: python3 scripts/generate-platform-docs.py --check, npm run docs:check-agent-variants, npm run docs:check-routes, and git diff --check passed.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

npm run docs completed with zero errors.
Fern reported only the existing unauthenticated redirect-check and light-theme contrast warnings.

---

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Summary by CodeRabbit

• New Features
  • Added experimental support for connecting to an existing authenticated local llama.cpp server.
  • Added an experimental managed llama.cpp deployment option for DGX Spark.
• Documentation
  • Added a complete llama.cpp setup guide covering installation, authentication, model configuration, verification, recovery, upgrades, rollback, and cleanup.
  • Updated provider selection, local inference comparisons, onboarding navigation, and platform support documentation with the new options and requirements.

[coderabbitai]

📝 Walkthrough

Walkthrough

The change adds experimental llama.cpp provider metadata for authenticated loopback servers and managed DGX Spark deployments. It adds a dedicated setup guide, navigation entries, provider-selection guidance, and platform-support documentation.

Changes

llama.cpp support

Layer / File(s)	Summary
Provider support declarations ci/platform-matrix.json, docs/inference/choose-inference-provider.mdx, docs/reference/platform-support.mdx	Added experimental records for authenticated loopback llama.cpp servers on port 8081 and managed llama.cpp deployments on qualified DGX Spark hosts.
llama.cpp setup guide docs/inference/set-up-llama-cpp.mdx, docs/index.yml	Added setup instructions for existing servers and managed deployments. Added navigation entries for OpenClaw, Deep Agents, and Hermes.
Inference selection guidance docs/inference/choose-local-inference-server.mdx, docs/inference/choose-inference-provider.mdx	Added llama.cpp options to local-provider tables and linked managed installation guidance to the dedicated setup page.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• NVIDIA/NemoClaw#8511: Documents the experimental managed llama.cpp deployment on DGX Spark, including loopback runtime and lifecycle details.

Suggested labels: area: docs, v0.0.105

Suggested reviewers: cv, aasthajh

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: adding a dedicated llama.cpp operator guide to the inference documentation.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/llamacpp-managed-guide-8144

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit e5988b1 in the docs/llamacpp-manage... branch remains at 96%, unchanged from commit 9f7c278 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit e5988b1 in the docs/llamacpp-manage... branch remains at 81%, unchanged from commit 9f7c278 in the main branch.

Show a code coverage summary of the most impacted files.

File	main 9f7c278	docs/llamacpp-manage... e5988b1	+/-
src/lib/onboard...shared-state.ts	90%	73%	-17%
src/lib/onboard.../docker-spec.ts	74%	65%	-9%
src/lib/onboard...strap/docker.ts	72%	64%	-8%
src/lib/onboard...ne-authority.ts	91%	84%	-7%
src/lib/onboard...cker-journal.ts	75%	70%	-5%
src/lib/onboard...trap/adapter.ts	70%	68%	-2%
src/lib/onboard...mo-lifecycle.ts	86%	84%	-2%
src/lib/policy/index.ts	63%	62%	-1%
src/lib/onboard...an-lifecycle.ts	79%	95%	+16%
src/lib/policy/...ateway-state.ts	56%	73%	+17%

---

_{Updated August 08, 2026 06:10 UTC}

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-8610.docs.buildwithfern.com/nemoclaw

[github-actions]

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.

5 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

• llama.cpp recipe at docs/inference/set-up-llama-cpp.mdx:80: selected only by the second-opinion lane as established.
• operator-managed at ci/platform-matrix.json:140: selected only by the second-opinion lane as established.
• managed llama.cpp at docs/inference/set-up-llama-cpp.mdx:8: selected only by the second-opinion lane as established.
• cooperative native at ci/platform-matrix.json:140: selected only by the second-opinion lane as define.
• fingerprint contract at docs/inference/set-up-llama-cpp.mdx:20: selected only by the second-opinion lane as define.

1 additional E2E selection from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

• spark-install: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

• define — cooperative fingerprinting at docs/inference/choose-local-inference-server.mdx:27: Define “cooperative fingerprinting” at first use, or replace it with a short description of the native evidence checks.
• established — Docker authority at docs/inference/set-up-llama-cpp.mdx:107: Keep the established term.
• established — managed llama.cpp at docs/inference/set-up-llama-cpp.mdx:25: Keep the established term to distinguish the managed path from the existing-server path.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-onboard

1 optional E2E recommendation

• llama-cpp-dgx-spark-qualification

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.