Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,17 @@ Rules, not description. Code is truth — implementation is discoverable, so it
### Threads
- **Thread is a record.** Follow-up adds an answer, never rewrites one. Correction is a new question.
- **One language per thread** — its first question's. Everything a person reads follows it, replays included. The record decides, not the request.
- **Answer prompt gets the previous QUESTION only**, never the previous answer — prose beside sources it was NOT written from is how a claim acquires a citation it was never read from. One turn back, never the thread.
- **Rework ("summarize", "as a table") answers from the previous answer and ITS OWN sources, no search.** First turn is never a rework. A basis missing even ONE chunk is refused — never summarised from survivors, never searched afresh; a fresh answer to "summarize" is a different answer dressed as a summary. Not "translate".
- **Rework ("summarize", "as a table", "zeichne ein Diagramm davon") answers from the previous answer and ITS OWN sources, no search.** First turn is never a rework. A basis missing even ONE source is refused — never summarised from survivors, never searched afresh; a fresh answer to "summarize" is a different answer dressed as a summary. Not "translate".
- **A turn's basis is recorded by repo, path, commit, lines — re-read from git, never by chunk row.** A poll re-inserts every chunk of a touched file under new ids; keyed on them, a minute-old answer's diagram was refused as "no longer indexed". Missing means git cannot produce it: purged repo, replaced snapshot, file the index now skips — a file deleted or renamed since is still read at its commit. A window bigger than a chunk (split overlong line) is never read back whole. Text read only once the turn is a rework; every follow-up reads refs, not files.
- **Previous answer reaches the answer prompt as context, never a source** — markers stripped, fenced off, never cited, its claims never restated as fact. "The flow" is the flow it described. Changes and release lanes get the question only.
- **Recall reaches the model as ONE user message**, never a user/assistant pair — a prose assistant turn makes the model continue the conversation instead of returning JSON.
- **Thread is a funnel: narrows, never widens.** Pin is a ceiling; "in all repos" under a pin is not honoured. A named repo the pin excludes is reported as outside — silent refusal to widen is the same quiet drop the rule exists to stop.
- **Empty repo restriction means the whole corpus** — so a pin or chosen repo that no longer exists must FAIL the turn, never search. Has bitten twice.
- **Thread addressed by public id, never row number.** That id is not a share token: a share is revocable authorisation, an address is not.

### Memory
- **Standing instruction kept the turn it is said, in English, per reader.** About the READER only — never a claim about code, never language, audience or repo pin, never a one-off.
- **Kept only on the reader's word, checked in Go** — the gate quotes it (`memory_marker`), a quote the question lacks is dropped (`standingOnly`). The model's judgement alone filed "zeichne ein diagramm des ablaufs" as "Draw a diagram for the answer." and every later answer drew one. Forgetting is not gated. Same text twice is one rule.
- **The block outranks the prompt**: overrides form, length, diagrams, what to mention. Never citing, inventing, "nothing found", language, audience. Empty memory leaves the prompt byte-identical — that is the eval baseline. Contradiction REPLACES, a rule never expires, one past the cap is refused, never fitted in by dropping one. Never on a shared page.

### Routing
Expand Down
6 changes: 4 additions & 2 deletions backend/cmd/rongo/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -389,12 +389,14 @@ func main() {

// The viewer and the answer pipeline read files through the same service:
// the viewer shows a citation, the pipeline reads a process model whose
// nodes were cited, both at the indexed commit under the same rules.
// nodes were cited, both at the indexed commit under the same rules. A
// thread reads its answers' sources back through it too, at the commit
// each was read at, so a re-index since the answer changes nothing.
source := sourceview.New(db, gitClient, cfg.IndexMaxFileBytes).WithCommits(gitClient)
deps := httpapi.Deps{
Auth: authSvc,
Repos: repostatus.New(db, moduleOpts(cfg)),
Threads: threads.NewStore(db),
Threads: threads.NewStore(db).WithEvidence(source),
Source: source,
Commit: source,
OIDCAdminGroup: cfg.OIDCAdminGroup,
Expand Down
71 changes: 50 additions & 21 deletions backend/internal/ask/answer.go
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,8 @@ type Answer struct {

// PromptParts is the answer prompt by section, in estimated tokens. System is
// every rule the audience, language and scope assembled — and the thread's
// previous question with it, because a follow-up is written into the rules
// (answerFollowUp) rather than into the message the reader typed. Sources is
// previous question and answer with it, because a follow-up is written into
// the rules (answerFollowUp) rather than into the message the reader typed. Sources is
// the code in front of the model, headers and separators included. Question
// is what was asked, and only that.
type PromptParts struct {
Expand Down Expand Up @@ -370,23 +370,50 @@ alone and that a new thread can answer across the whole corpus, then answer for
them. Make no claim of any kind about any other repository - not a guess, not a
comparison, not "presumably".`

// FollowUp is the turn a follow-up continues: the previous question and the
// answer it got. Zero on a first turn, and then the prompt carries nothing
// about a follow-up at all.
type FollowUp struct {
Question string
Answer string
}

// followUpOf is what a follow-up of t carries into the answer prompt.
func followUpOf(t Thread) FollowUp {
return FollowUp{Question: t.Question, Answer: t.Answer}
}

// answerFollowUp is added when the turn continues a thread that already
// answered something. Its one format argument is the PREVIOUS QUESTION.
//
// The previous answer's text is not here and must not be: the sources are what
// a claim rests on, and a model handed its own earlier prose alongside sources
// it was NOT written from ends up restating it and citing the new sources for
// it. The question is enough for both things this rule is for — telling the
// model what a pronoun points at, and telling it not to write the same answer
// again with a picture on top. A rework is the exception, and it has its own
// block: answerRework.
const answerFollowUp = `

This is a follow-up to an earlier question in the same thread: %s. Answer the
NEW question. Where it points at something without naming it ("that", "this",
"it"), it points at the subject of that earlier question. Do not restate what
was already explained - the reader has it directly above.`

// answerFollowUpAnswer follows answerFollowUp when the previous turn
// answered. Its one format argument is that answer, markers stripped.
//
// "The flow" in a follow-up is the flow the previous answer described, and
// the question alone cannot say which that was. The answer is prose the
// model wrote itself, not something read from code, so it goes in as
// context and never as a source: handed back beside sources it was NOT
// written from, a claim of it would otherwise be restated and cited to a
// source that never said it.
const answerFollowUpAnswer = `

Your previous answer, which the reader has directly above, was:

<<<
%s
>>>

It is context, never a source: it tells you what the new question refers to.
Never cite it and never restate its claims as fact. Every claim you make rests
on the numbered sources below. Where they say something it did not, or
contradict it, the sources win.`

// answerRework replaces answerFollowUp on a turn that asks for the previous
// answer in another form. Its one format argument is the reader's
// instruction. The previous text stands in the user message beside the
Expand Down Expand Up @@ -1205,11 +1232,10 @@ const structureIsConfiguration = "\nThis is configuration, not code. It says whi
// the model. A model handed only a question and a system prompt answers it
// fluently from its own training, and that answer would be about some other
// codebase — the single most expensive failure this product can produce.
// followingUp is the question this thread asked last, empty when there is
// none. Only the question: see answerFollowUp for why the previous answer's
// text stays out of here.
// followingUp is the turn this one continues, zero when there is none; its
// answer goes in as context, never as a source (answerFollowUpAnswer).
func (a *Answerer) Answer(ctx context.Context, question string, audience Audience, lang Language,
sources []Source, scope Scope, followingUp string, onToken func(string)) (Answer, error) {
sources []Source, scope Scope, followingUp FollowUp, onToken func(string)) (Answer, error) {

if len(sources) == 0 {
return Answer{Text: NothingFound(lang, nil)}, nil
Expand Down Expand Up @@ -1237,8 +1263,8 @@ func (a *Answerer) Answer(ctx context.Context, question string, audience Audienc
// Rework writes the previous answer again in the form the instruction asks
// for, from that answer's own sources. The previous text goes into the user
// message beside them — the one place in the product where prose of the
// model's own reaches the answering prompt, and it is safe here because the
// sources next to it are the ones it was written from.
// model's own is material to rework rather than context, and it is safe here
// because the sources next to it are the ones it was written from.
//
// No follow-up rule: "do not restate what was already explained" would forbid
// the one thing this call exists to do, the same as Reexplain.
Expand All @@ -1249,7 +1275,7 @@ func (a *Answerer) Rework(ctx context.Context, instruction string, audience Audi
return Answer{}, fmt.Errorf("rework: no sources to rework from")
}
rules := memory.Applying(memory.From(ctx).Rows(), scope.Known)
system := systemPrompt(audience, lang, t.Sources, scope, "", fmt.Sprintf(answerRework, instruction), memory.Block(rules, scope.Known))
system := systemPrompt(audience, lang, t.Sources, scope, FollowUp{}, fmt.Sprintf(answerRework, instruction), memory.Block(rules, scope.Known))
user := renderRework(instruction, t, scope.Stages)
// The sources are measured on their own here: the user message also
// carries the previous turn, which is neither the question nor the code
Expand All @@ -1267,11 +1293,11 @@ func (a *Answerer) Rework(ctx context.Context, instruction string, audience Audi
}

// systemPrompt assembles the answering rules for one turn. followingUp is
// the previous question of an ordinary follow-up, rework the rendered rework
// the previous turn of an ordinary follow-up, rework the rendered rework
// block; at most one of them is set. memories is the reader's standing
// instructions as memory.Block rendered them, empty for a reader with none,
// and then the prompt is byte for byte what it was before memory existed.
func systemPrompt(audience Audience, lang Language, sources []Source, scope Scope, followingUp, rework, memories string) string {
func systemPrompt(audience Audience, lang Language, sources []Source, scope Scope, followingUp FollowUp, rework, memories string) string {
name := languageName(lang)
system := fmt.Sprintf(answerCommon, name)
if audience == AudienceDev {
Expand Down Expand Up @@ -1350,8 +1376,11 @@ func systemPrompt(audience Audience, lang Language, sources []Source, scope Scop
if scope.AllDenied && len(scope.Known) > 0 {
system += fmt.Sprintf(answerAllDenied, strings.Join(scope.Known, ", "))
}
if followingUp != "" {
system += fmt.Sprintf(answerFollowUp, followingUp)
if followingUp.Question != "" {
system += fmt.Sprintf(answerFollowUp, followingUp.Question)
if followingUp.Answer != "" {
system += fmt.Sprintf(answerFollowUpAnswer, strings.TrimSpace(stripMarkersOutsideFences(followingUp.Answer)))
}
}
system += rework
// Computed from the sources rather than read off the scope, so this block
Expand Down
6 changes: 3 additions & 3 deletions backend/internal/ask/answer_retry_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ func breakAfter(t *testing.T, tokens ...string) (*llm.Client, *atomic.Int32) {
func TestAnswer_aRetriedAnswerReportsTwoAttemptsInItsDetail(t *testing.T) {
c, calls := failThenStream(t, "Stored in ", "store.go [1].")

got, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, "", nil)
got, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, FollowUp{}, nil)
if err != nil {
t.Fatalf("Answer: %v", err)
}
Expand All @@ -71,7 +71,7 @@ func TestAnswer_aRetriedAnswerReportsTwoAttemptsInItsDetail(t *testing.T) {
func TestAnswer_aStreamThatBrokeAfterTextFailsTheTurn(t *testing.T) {
c, calls := breakAfter(t, "Stored [2] and ", "then")

_, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, "", nil)
_, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, FollowUp{}, nil)

if err == nil {
t.Fatal("err = nil, want the turn to fail rather than store a fragment")
Expand All @@ -86,7 +86,7 @@ func TestAnswer_aStreamThatBrokeAfterTextFailsTheTurn(t *testing.T) {
func TestAnswer_anOrdinaryTurnSaysNothingAboutAttempts(t *testing.T) {
c, _, _ := streamUpstream(t, "Stored in store.go [1].")

got, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, "", nil)
got, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, FollowUp{}, nil)
if err != nil {
t.Fatalf("Answer: %v", err)
}
Expand Down
8 changes: 4 additions & 4 deletions backend/internal/ask/answer_stages_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ func stagedSources() []Source {
func TestAnswer_stagedSourcesAreLabelledAndTheRuleSaysReportEveryStage(t *testing.T) {
c, prompt, _ := streamUpstream(t, "x")
scope := Scope{Known: []string{"acme-service", "acme-infra"}, Stages: declaredStages}
if _, err := NewAnswerer(c).Answer(context.Background(), "How often?", AudienceBA, LanguageEN, stagedSources(), scope, "", nil); err != nil {
if _, err := NewAnswerer(c).Answer(context.Background(), "How often?", AudienceBA, LanguageEN, stagedSources(), scope, FollowUp{}, nil); err != nil {
t.Fatalf("Answer: %v", err)
}
p := *prompt
Expand All @@ -51,7 +51,7 @@ func TestAnswer_anAskedStageRulesOutTheOthers(t *testing.T) {
scope := Scope{Known: []string{"acme-service", "acme-infra"}, Stages: declaredStages, Stage: "prod"}
sources := stagedSources()
sources = append(sources[:2], sources[3])
if _, err := NewAnswerer(c).Answer(context.Background(), "How often in production?", AudienceBA, LanguageEN, sources, scope, "", nil); err != nil {
if _, err := NewAnswerer(c).Answer(context.Background(), "How often in production?", AudienceBA, LanguageEN, sources, scope, FollowUp{}, nil); err != nil {
t.Fatalf("Answer: %v", err)
}
p := *prompt
Expand All @@ -66,7 +66,7 @@ func TestAnswer_anAskedStageRulesOutTheOthers(t *testing.T) {
func TestAnswer_noStagedSourceMeansNoStageRule(t *testing.T) {
c, prompt, _ := streamUpstream(t, "x")
scope := Scope{Known: []string{"peeq"}, Stages: declaredStages}
if _, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), scope, "", nil); err != nil {
if _, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), scope, FollowUp{}, nil); err != nil {
t.Fatalf("Answer: %v", err)
}
if strings.Contains(*prompt, "deployed configuration") || strings.Contains(*prompt, "(stage ") {
Expand All @@ -76,7 +76,7 @@ func TestAnswer_noStagedSourceMeansNoStageRule(t *testing.T) {

func TestAnswer_redactedMarkerIsExplainedToTheModel(t *testing.T) {
c, prompt, _ := streamUpstream(t, "x")
if _, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, "", nil); err != nil {
if _, err := NewAnswerer(c).Answer(context.Background(), "How?", AudienceBA, LanguageEN, twoSources(), Scope{}, FollowUp{}, nil); err != nil {
t.Fatalf("Answer: %v", err)
}
if !strings.Contains(*prompt, "<redacted>") {
Expand Down
Loading
Loading