Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .claude/rules/capability-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,11 @@ Before building any of these, check if it already exists. Use the existing imple
| Pareto scoring | engine/scoring/pareto.py | multi-objective optimization |
| Convergence loop | engine/feedback/convergence.py | `ConvergenceLoop` |
| Label sanitization | engine/utils/security.py | `sanitize_label()` |
| CEG → Gate → EIE egress (the ONLY peer egress) | engine/gate_egress.py | `request_enrichment()`, `emit_graph_inference_result()` |
| Inference outputs → EIE wire shape | engine/gate_egress.py | `build_inference_outputs()` (0.55 floor, matches EIE) |
| Domain database provisioning | engine/graph/driver.py | `GraphDriver.ensure_database()` (flag: `auto_create_domain_database`) |
| Database-name validation for DDL | engine/graph/driver.py | `_DATABASE_NAME_RE` — `sanitize_label()` forbids the dashes domain ids use |
| Inference rule execution | engine/inference_rule_registry.py | `execute_rule()`, `list_registered_rules()` |

Do not add a second outbound client. `engine/gate_egress.py` is the only
CEG → peer egress; Gate resolves the destination from the action.
9 changes: 9 additions & 0 deletions .claude/rules/feature-flags.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,15 @@ All in Settings class, controllable via env vars. Contract 21: every behavioral
| strict_tenant_database | False | Require explicit `database=` on GraphDriver calls; no implicit 'neo4j' fallback (W7-01) |
| require_sdk_chassis_in_prod | False | Fail startup if `L9_CHASSIS != sdk` when `l9_env == prod` (W7-02) |

## Constellation Seam (EIE ↔ Gate ↔ CEG)
| Flag | Default | Purpose |
|------|---------|---------|
| auto_enrich_via_gate | False | Dispatch `enrich` to EIE through Gate (spends EIE budget) |
| graph_inference_feedback_enabled | False | Emit `graph-inference-result` to EIE through Gate (EIE-008 / CEG-006) |
| auto_create_domain_database | False | Provision the tenant domain database on first use (CEG-008; Enterprise-only CREATE DATABASE) |
| health_api_enabled | False | Expose engine/health/api.py via admin health_* subactions (CEG-006) |
| unvalidated_domain_packs_enabled | False | Serve the 5 migrated packs whose gates do not compile to executable Cypher (CEG-009) |

## Entity Resolution
| Flag | Default | Purpose |
|------|---------|---------|
Expand Down
9 changes: 9 additions & 0 deletions .claude/rules/subsystems.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ All handlers: `async def handle_*(tenant: str, payload: dict) -> dict`
| calibration_run | Score calibration vs expected ranges |
| score_feedback | Compute weight adjustment proposal |
| apply_weight_proposal | Apply proposed weight change |
| health_assess | AI-readiness assessment for one entity (`engine/health/api.py`) |
| health_batch_assess | Incremental batch readiness scan with cost ceilings |
| health_report | Health report for an entity (Seed tier) |
| emit_inference_feedback | Run inference rules and send `graph-inference-result` to EIE via Gate (flag: `graph_inference_feedback_enabled`) |

CEG-006: the three `health_*` subactions are what makes `engine/health/api.py`
reachable. It was imported by nothing, so `trigger_reenrichment_v2` — and with
it the whole CEG → Gate → EIE enrichment request — had no trigger any inbound
packet could reach.

## Dependency Map
```
Expand Down
20 changes: 14 additions & 6 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=l9-dev-password
NEO4J_POOL_SIZE=20

── Redis Cache ─────────────────────────────────────────────
REDIS_URL=redis://redis:6379/0

── API Configuration ───────────────────────────────────────
API_KEY=dev-key-sha256-not-for-production
LOG_LEVEL=debug
Expand All @@ -47,7 +44,6 @@ LOCAL DEVELOPMENT ACCESS (from host machine, not Docker)
API: http://localhost:8000/v1/health
Neo4j Browser: http://localhost:7475
Neo4j Bolt: bolt://localhost:7688
Redis: localhost:6379
Neo4j Credentials:
User: neo4j
Password: l9-dev-password
Expand All @@ -68,11 +64,16 @@ KGE_EMBEDDING_DIM=300
GATE_URL=http://gate:9000
GATE_ADMIN_TOKEN=your-gate-admin-token-here
L9_NODE_NAME=graph
L9_NODE_SPEC_PATH=engine/spec.yaml
GATE_REGISTRATION_ENABLED=true
GATE_REGISTER_OVERWRITE=true
GATE_REGISTER_RETRIES=3
GATE_CLIENT_TIMEOUT_SECONDS=30.0
# CEG-003: this is the name the SDK reads (gate/config.py:
# spec_path=os.getenv("GATE_NODE_SPEC_PATH", "engine/spec.yaml")). An
# L9_NODE_SPEC_PATH line sat above it with no reader at all, so an operator
# relocating the spec edited the variable that does nothing and registration
# kept using the default until that path stopped existing — at which point
# register_with_gate swallows FileNotFoundError and returns False.
GATE_NODE_SPEC_PATH=engine/spec.yaml

# ── SDK chassis (L9_CHASSIS=sdk) ─────────────────────────────
Expand Down Expand Up @@ -104,7 +105,14 @@ L9_SIGNING_ALGORITHM=hmac-sha256
L9_SIGNING_KEY=change-me-shared-hmac-secret
L9_SIGNING_KEY_ID=graph-engine-v1
# ed25519 alternative: L9_SIGNING_ALGORITHM=ed25519 + L9_SIGNING_PRIVATE_KEY
# L9_VERIFYING_KEYS_JSON={"gate-v1":"<public-key>"}
#
# CEG-002: REQUIRED whenever L9_REQUIRE_SIGNATURE=true, not optional. The same
# flag turns on verification of the responses Gate signs, and Gate signs with
# *its* key id, which CEG cannot resolve from its own signing key. Leave this
# unset and every signed Gate response is rejected with "no verifying key
# available for transport signature verification". Under hmac-sha256 the value
# is the shared secret; under ed25519 it is the peer's public key.
L9_VERIFYING_KEYS_JSON={"gate-v1":"change-me-shared-hmac-secret"}

# Allowed actions — engine.handlers.ACTION_HANDLERS minus `enrich`: Gate owns
# the `enrich` name for EIE, so CEG must never accept or advertise it.
Expand Down
11 changes: 1 addition & 10 deletions .github/workflows/ci-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,15 +182,7 @@ jobs:
--health-timeout 5s
--health-retries 5

redis:
image: redis:7-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
# CEG-007: redis service container removed with the dependency.

steps:
- name: Checkout code
Expand All @@ -215,7 +207,6 @@ jobs:
- name: Run tests with coverage
env:
DATABASE_URL: postgresql://${{ vars.TEST_DB_USER || 'test' }}:${{ vars.TEST_DB_PASSWORD || 'test' }}@localhost:5432/${{ vars.TEST_DB_NAME || 'test_db' }}
REDIS_URL: redis://localhost:6379
TESTING: "true"
run: PYTHONPATH=. pytest tests/ -v --cov=engine --cov-report=xml --cov-report=term --ignore=tests/e2e

Expand Down
13 changes: 3 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -258,15 +258,9 @@ jobs:
--health-retries 5
# Service only runs if POSTGRES_ENABLED=true (GitHub ignores services with falsy conditions)

redis:
image: ${{ vars.REDIS_IMAGE || 'redis:7-alpine' }}
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
# CEG-007: the redis service container is gone with the dependency. No
# code under engine/ or chassis/ imports redis, so it started a container
# every run for nothing.

steps:
- name: Checkout Repository
Expand All @@ -291,7 +285,6 @@ jobs:
- name: Run Tests with Coverage
env:
DATABASE_URL: postgresql://${{ vars.POSTGRES_USER || 'test_user' }}:test_password@localhost:5432/${{ vars.POSTGRES_DB || 'test_db' }}
REDIS_URL: redis://localhost:6379/0
run: |
echo "Running test suite..."
PYTHONPATH=. pytest ${{ env.TEST_DIR }} \
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Cross-tool agent instructions for the CEG repository. Read by Claude Code, Codex

```bash
make setup # Install deps, pre-commit hooks, verify Neo4j
make dev # docker-compose (app + Neo4j + Redis + Prometheus + Grafana)
make dev # docker-compose (app + Neo4j + Postgres)
make test # Full pytest suite (unit + integration + compliance)
make test-unit # Gate compilation, scoring math, parameter resolution
make test-integration # testcontainers-neo4j full pipeline
Expand Down
13 changes: 2 additions & 11 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,6 @@ health:Check all service health
@curl -sf http://localhost:8000/v1/health | python -m json.tool || echo "API: DOWN"
@echo "── Neo4j ──"
@docker exec l9-graph-neo4j cypher-shell -u neo4j -p l9-dev-password "RETURN 'ok'" 2>/dev/null || echo "Neo4j: DOWN"
@echo "── Redis ──"
@docker exec l9-graph-redis redis-cli ping || echo "Redis: DOWN"

# ── Testing ────────────────────────────────────────────────

Expand All @@ -72,21 +70,17 @@ shell:Python shell inside API container
neo4j-shell:Cypher shell into Neo4j
docker exec -it l9-graph-neo4j cypher-shell -u neo4j -p l9-dev-password

redis-shell:Redis CLI
docker exec -it l9-graph-redis redis-cli

# ── Local Dev (API outside Docker, DBs in Docker) ─────────

local-dbs:Start only Neo4j + Redis
docker compose up -d neo4j redis
local-dbs:Start only Neo4j + Postgres
docker compose up -d neo4j postgres

local-api:Run API locally against Dockerized DBs (SDK chassis; alias of local-api-sdk)
$(MAKE) local-api-sdk

local-api-legacy:Run the legacy dict chassis locally (permitted in L9_ENV=dev|local|test)
PLASTICOS_NEO4J_URI=bolt://localhost:7687 \
PLASTICOS_NEO4J_PASSWORD=l9-dev-password \
PLASTICOS_REDIS_URL=redis://localhost:6379/0 \
PLASTICOS_LOG_LEVEL=debug \
L9_LIFECYCLE_HOOK=engine.boot:GraphLifecycle \
L9_CHASSIS=legacy \
Expand All @@ -95,7 +89,6 @@ local-api-legacy:Run the legacy dict chassis locally (permitted in L9_ENV=dev|lo
local-api-sdk:Run API locally on the SDK chassis (L9_CHASSIS=sdk)
PLASTICOS_NEO4J_URI=bolt://localhost:7687 \
PLASTICOS_NEO4J_PASSWORD=l9-dev-password \
PLASTICOS_REDIS_URL=redis://localhost:6379/0 \
PLASTICOS_LOG_LEVEL=debug \
L9_LIFECYCLE_HOOK=engine.boot:GraphLifecycle \
L9_CHASSIS=sdk \
Expand Down Expand Up @@ -199,8 +192,6 @@ deploy-health: guard-vps-host ## Remote healthcheck over SSH (VPS ports may be f
@ssh $(SSH_OPTS) $(SSH_TARGET) "curl -sf http://localhost:8000/v1/health && echo" || echo "API: DOWN"
@echo "── Neo4j ──"
@ssh $(SSH_OPTS) $(SSH_TARGET) "curl -sf http://localhost:7474 >/dev/null" && echo "Neo4j: UP" || echo "Neo4j: DOWN"
@echo "── Redis ──"
@ssh $(SSH_OPTS) $(SSH_TARGET) "docker exec l9-redis-prod redis-cli ping" || echo "Redis: DOWN"

# ── Cleanup ────────────────────────────────────────────────

Expand Down
11 changes: 4 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ One engine. Any vertical. No custom code per domain.
git clone <repo-url> && cd l9-engine
./scripts/setup.sh

# 2. Start local stack (Neo4j + Redis + API)
# 2. Start local stack (Neo4j + Postgres + API)
./scripts/dev.sh

# 3. Seed sample data
Expand Down Expand Up @@ -81,8 +81,8 @@ All intelligence features are disabled by default and activated per-domain via Y
│ Domain Spec Loader │
│ YAML → Pydantic → Compiled Cypher │
├──────────────────────────────────────────────────────┤
│ Neo4j (multi-database) │ Redis (cache/scheduler)
└─────────────────────────────────────────────────────┘
Neo4j (multi-database)
└─────────────────────────────────────────────────────┘
```

### Core Concepts
Expand Down Expand Up @@ -121,8 +121,7 @@ l9-engine/
│ │ ├── pii.py # PII hash/encrypt/redact
│ │ └── audit.py # Audit logging
│ └── db/ # Database layer
│ ├── neo4j.py # Async Neo4j driver pool
│ └── redis.py # Redis connection
│ └── neo4j.py # Async Neo4j driver pool
├── domains/ # Domain specification packs
│ ├── plasticos/
│ ├── mortgage-brokerage/
Expand Down Expand Up @@ -153,7 +152,6 @@ l9-engine/
| Component | Version | Required Plugins |
|-----------|---------|------------------|
| Neo4j | 5.15+ Enterprise | APOC, Graph Data Science (GDS) |
| Redis | 7.x | — |
| Python | 3.12+ | — |

**Neo4j Plugins:**
Expand All @@ -174,7 +172,6 @@ Consistent across all L9 repos. Set in `.env` (local) or SSM Parameter Store (pr
| `NEO4J_USERNAME` | `neo4j` | Neo4j username |
| `NEO4J_PASSWORD` | — | Neo4j password (SSM in prod) |
| `NEO4J_DATABASE` | `neo4j` | Default database |
| `REDIS_URL` | `redis://localhost:6379/0` | Redis connection |
| `API_PORT` | `8000` | API listen port |
| `API_WORKERS` | `4` | Uvicorn workers |
| `DOMAINS_ROOT` | `./domains` | Path to domain specs |
Expand Down
2 changes: 0 additions & 2 deletions Readme-Requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,13 @@ status: active

- `neo4j_uri`, `neo4j_username`, `neo4j_password` → consumed by `dependencies.py` to init `GraphDriver`[^1]
- `domains_root` → consumed by `dependencies.py` to init `DomainPackLoader`[^2]
- `redis_url` → lazy-init Redis client
- Scoring weights (`w_structural`, `w_geo`, etc.) → match the spec's config reference[^3]
- Decay half-lives → match the temporal decay system in `engine-core-modules.py`[^1]

**`dependencies.py`** provides the `Depends()` functions that `match.py`, `sync.py`, and `health.py` already import:[^4]

- `get_graph_driver()` → returns the shared async Neo4j driver
- `get_domain_loader()` → returns the cached domain pack loader
- `get_redis()` → lazy Redis with graceful degradation
- `startup()` / `shutdown()` → called from `create_app()` lifespan

**`requirements.txt`** pins every dep from `pyproject.toml` with `>=X,<Y` bounds — works for Docker builds and non-Poetry envs .
Expand Down
5 changes: 3 additions & 2 deletions chassis/auth/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ class Settings(BaseSettings):
neo4j_max_connection_lifetime: int = 3600
neo4j_connection_acquisition_timeout: int = 60

# --- Redis ---
redis_url: str = "redis://localhost:6379/0"
# CEG-007: `redis_url` was declared here and never read — nothing under
# engine/ or chassis/ imports redis. Removed with the service and the
# dependency rather than left as a knob for a component that does not exist.

# --- API ---
api_port: int = 8000
Expand Down
35 changes: 12 additions & 23 deletions docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,21 +36,29 @@ services:
GATE_ADMIN_TOKEN: ${GATE_ADMIN_TOKEN:?GATE_ADMIN_TOKEN must be set for production}
GATE_REGISTRATION_ENABLED: "true"
L9_NODE_NAME: graph
L9_NODE_SPEC_PATH: engine/spec.yaml
# CEG-003: GATE_NODE_SPEC_PATH is the name the SDK reads. L9_NODE_SPEC_PATH
# had no reader; registration survived only on the SDK's default.
GATE_NODE_SPEC_PATH: engine/spec.yaml
L9_ENFORCE_GATE_ONLY_INGRESS: "true"
L9_GATE_NODE_NAME: gate
L9_REQUIRE_SIGNATURE: "true"
L9_SIGNING_ALGORITHM: hmac-sha256
L9_SIGNING_KEY: ${L9_SIGNING_KEY:?L9_SIGNING_KEY must be set for production}
L9_SIGNING_KEY_ID: ${L9_SIGNING_KEY_ID:-graph-engine-v1}
# CEG-002: L9_REQUIRE_SIGNATURE=true also verifies the responses Gate
# signs, and Gate signs with its own key id. Without this map CEG rejects
# every signed Gate response ("no verifying key available for transport
# signature verification"). Required, not defaulted: a wrong key here is a
# silent trust failure, so production must state it — JSON object mapping
# key id to key material, e.g. {"gate-v1":"<gate-key>"}.
L9_VERIFYING_KEYS_JSON: ${L9_VERIFYING_KEYS_JSON:?L9_VERIFYING_KEYS_JSON must be set when L9_REQUIRE_SIGNATURE is true}
L9_ALLOWED_ACTIONS: match,sync,admin,outcomes,resolve,health,healthcheck
L9_MAX_ATTACHMENTS: "0"
L9_MAX_ATTACHMENT_SIZE_BYTES: "0"
NEO4J_URI: bolt://neo4j:7687
NEO4J_USERNAME: ${NEO4J_USERNAME:-neo4j}
NEO4J_PASSWORD: ${NEO4J_PASSWORD:?NEO4J_PASSWORD must be set for production}
NEO4J_POOL_SIZE: "50"
REDIS_URL: redis://redis:6379/0
API_SECRET_KEY: ${API_SECRET_KEY:?API_SECRET_KEY must be set for production}
LOG_LEVEL: info
CORS_ORIGINS: ${CORS_ORIGINS:-[]}
Expand All @@ -59,11 +67,11 @@ services:
KGE_CONFIDENCE_THRESHOLD: "0.3"
KGE_EMBEDDING_DIM: "300"
PARETO_ENABLED: "true"
# CEG-007: redis was a depends_on: service_healthy here for a dependency
# no code imports.
depends_on:
neo4j:
condition: service_healthy
redis:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/v1/health')"]
interval: 30s
Expand Down Expand Up @@ -106,28 +114,9 @@ services:
retries: 10
restart: always

# ── Redis 7.x ───────────────────────────────────────────
redis:
image: redis:7-alpine
container_name: l9-redis-prod
ports:
- "6379:6379"
command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
volumes:
- redis-data:/data
networks:
- l9-net
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 3s
retries: 5
restart: always

volumes:
neo4j-data:
neo4j-logs:
redis-data:

networks:
l9-net:
Expand Down
Loading
Loading