Session Mining Lab

Deterministic tooling for mining coding-agent sessions for reusable engineering lessons and editorial leads.

This repository is staged beside an article while the method evolves. It is generic enough to use with any local coding-agent session store when the caller supplies the input path.

What it does

The pipeline:

1. reads JSONL session stores supplied on the command line;
2. extracts user, assistant, and tool messages by role;
3. removes common harness envelopes and redacts obvious private material;
4. stores redacted normalized records in SQLite;
5. scores topic, evidence, artifact, action, and problem-to-verification signals;
6. writes explainable shortlist and editorial-lead reports.

The selector is deterministic. It does not use a language model to decide which sessions are valuable. Language generation may be used after selection to draft an article from reviewed evidence.

Quick start

Requires Python 3.10 or newer and the standard library only.

python3 -m session_mining.cli \
  --input-root ./tests/fixtures \
  --output-dir ./var/example-run
python3 -m unittest discover -s tests -v

The input root may contain any number of .jsonl files. Paths are supplied by the caller and are never embedded in the code or reports as machine-specific defaults.

Evidence boundary

The score identifies editorial leads. It does not prove that a session contains a durable lesson or that a proposed technique works. Review the source record, confirm the claim, and record what was observed, measured, assumed, unknown, and still requires validation.

The default SQLite and Markdown outputs contain redacted text, but pattern redaction is not a secret scanner. Run against a deliberately selected input root, inspect outputs before sharing, and use synthetic fixtures or a separately reviewed redacted export for public examples.

Build notes

The implementation uses Python's standard library and has no runtime network or shell execution path. Reproduce the example from the repository root:

python3 -m venv .venv
. .venv/bin/activate
PYTHONPATH=src python3 -m session_mining.cli \
  --input-root ./tests/fixtures \
  --output-dir ./var/example-run
PYTHONPATH=src python3 -m unittest discover -s tests -v

For a local session store, replace ./tests/fixtures with the intended directory. Keep the output directory outside the input tree when practical. The tool indexes only regular JSONL files up to its configured size limit and skips symlinked inputs.

Codex setup prompt

Use this repository as a local session-mining utility. Read its README.md and
SECURITY.md first. Install it only in an isolated environment, then run the
standard-library tests. To scan a session store, pass its explicit directory as
--input-root and a separate scratch directory as --output-dir. Do not scan a
broader directory than necessary. Inspect corpus.sqlite and selection.md for
private data before sharing them. Treat the deterministic score as an editorial
lead, not proof of a durable lesson. Use a separate review step to validate
claims before drafting or publishing content.

Layout

src/session_mining/   parser, normalization, scoring, and CLI
tests/                standard-library tests and synthetic fixtures
docs/                 design and reproducibility notes
experiments/          repeatable experiment manifests
skill/                session-end article extraction skill

See docs/architecture.md, SECURITY.md, and skill/SKILL.md before adapting the pipeline.