feat: add verify_logic builtin tool for deterministic boolean checking#94
Open
Shrivastava-Aditya wants to merge 1 commit into
Open
Conversation
Small LLMs hallucinate on boolean satisfiability at ~20% error rates. This tool wraps boolean-algebra-engine to give the agent a zero-hallucination verification step for generated conditionals — same philosophy as the TDD governor, applied to boolean logic. - New src/tools/builtin/verify_logic.js: detects contradictions, tautologies, and satisfiability via deterministic evaluation (not LLM inference) - Registered under the 'plan' routing category alongside bone_check - Gracefully degrades if boolean-algebra-engine is not installed - README: added to the tools table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Small LLMs — your primary target — hallucinate on boolean satisfiability at around 20% error rates (empirically benchmarked across 7 models, including 70B). When a small model generates complex conditional logic — routing rules, access control, if-else chains — there's no deterministic check catching dead branches or contradictions before they land in code.
Solution
This PR adds
verify_logic, a new builtin tool that wrapsboolean-algebra-engineto give the agent a zero-hallucination verification step. It's the same philosophy as your TDD governor — don't trust the model to self-verify; use a deterministic layer instead.```js
// Agent calls this before committing generated conditional logic:
verify_logic({ expression: "(A.B).(!A)" })
// → { verdict: "CONTRADICTION — condition can never be true; this branch is dead code" }
```
Changes
Graceful degradation
Follows the same pattern as `web_browse.js` — if `boolean-algebra-engine` is not installed, the tool returns a clear error + install hint rather than crashing the agent loop.
Why this fits SmallCode's architecture
From your own ARCHITECTURE.md: "extract reliable work from small, forgetful models." Boolean logic is one of the clearest failure modes for small models. This tool closes that gap with a sub-10ms deterministic check — zero tokens, zero hallucination.