Skip to content

docs(research): DeepSeek Harness — adopt an enforcement self-check - #253

Closed
praxagent wants to merge 8 commits into
mainfrom
docs/deepseek-harness
Closed

praxagent wants to merge 8 commits into
mainfrom
docs/deepseek-harness

Conversation

@praxagent

Copy link
Copy Markdown
Owner

Assessment of DeepSeek Harness (dsh; MIT, about 241k stars, developer preview). It's based on the repository at 639ed01, because the landing page sits behind a WAF challenge.

Stacked on #252, then #251 and #246, because all of them edit the same README list and tracker.

Verdict: document + adopt one idea, bank two; don't adopt the harness.

  • Adopt: enforcement self-check. dsh sandboxes report whether their enforcement is full or partial. Prax's containment (the loopback-only drop-in, no Docker, the proxy as the only way out) is installed by hand and never checked from inside, so a missed daemon-reload passes silently.
  • Bank: scrub credentials from spawned commands' environments. Prax makes 69 host subprocess calls and none sets env=. Exposure is unverified: I couldn't read the live process's environment.
  • Bank: an advisory reminder before the hard turn limits from feat: runaway turns stop and report, whatever made them run away #243.
  • Confirmation: dsh's monotonic, deny-only guards are the same shape as hard floors (feat: hard floors — some actions always need a person's decision #247).
  • Not adopted: the harness. It has no network policy ("outside this vocabulary"), plugins run with full access, and its own safety notice says it is not audited.

Document + adopt two ideas; confirmation of the Muse-parity containment.
- An out-of-band wire record in the secrets proxy — the tool calls the model
  returned, hashed, append-only, where Prax can't write — checked against
  Prax's own trace, so a compromised Prax can't hide activity by editing it.
- A diff of newly granted access for egress-policy changes and timed grants.
The DPU hardware is out of reach; OpenShell (0.1.x) is a peer to watch.
OpenWorker (Andrew Ng et al., MIT): the closest peer to Prax's governance
stance. Adopt hard floors — a declared set enforced after every rule that can
lower risk; Prax's earned trust can lower two login steps from HIGH to MEDIUM
on self-reported success today — and parked approvals for unattended runs
instead of refusing and losing the work. Plus approval provenance per call.

OpenShell's product page adds per-program network policy and a policy prover
with an access ceiling: queue the ceiling, and a time-boxed evaluation of
OpenShell as prax-sandbox's runtime.
Not ruled out: Prax should be highly competitive with OpenShell. Candidate
routes recorded — per-program proxy identity inside the sandbox, cgroup/eBPF
attribution, or OpenShell's supervisor after the evaluation.
…nnel-held identity

Google's CNCF sandbox application (cncf/sandbox#523). Its egress design is the
closest published match to the secrets proxy. Adopted: never inject into
cleartext (ours did; fixed in prax-secrets-proxy #7). Queued: a trusted tunnel
client holds the sandbox's proxy identity, so the program can't read it. The
platform is a scale non-goal; its DNS bypass matches our documented gap.
…or-all means audit the key

Meta RAM's agent-built benchmarks saturate unaided; detailed human specs halve
solver scores. Adopt: difficulty of LLM-authored cases measured on a solver
from another provider, and a case every solver fails gets its answer key
audited — lowest-score selection also selects wrong keys.
dsh sandboxes report full/partial enforcement; Prax's hand-installed
containment is never checked from inside. Bank: scrubbed child environments
(69 subprocess calls, none sets env=; exposure unverified) and an advisory
reminder before the hard turn limits. Monotonic deny-only guards confirm
hard floors.
…ing it, and correct it

The September README-only note said dsh has no governance layer. The code
has approvals, deny-only guards and a process sandbox; what it lacks is
network policy and audit. One page, one index entry, the correction stated.
@praxagent
praxagent force-pushed the docs/deepseek-harness branch from 3eb07f9 to be254e3 Compare October 2, 2026 05:34
@praxagent

Copy link
Copy Markdown
Owner Author

Folded into #256, which contains every commit from this branch (the five research PRs were a stack, and stacked PRs re-conflict after every squash merge). Merge #256.

@praxagent praxagent closed this Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant