Read the firmware's meta.json from a crash bundle, and its chip and sensor when the log has lost them

[widgetii]

OpenIPC/firmware#2556 makes S98crashlog write a meta.json into each crash bundle: the firmware build, kernel, command line, chip, sensor, majestic version, and the pipeline's settings without secrets. OpenIPC/majestic-webui#647 sends it as the meta field. A bundle downloaded from the WebUI and sent from /club, though, carries meta.json only inside the archive, and the server ignored it.

Why it matters

A pstore record is the tail of the kernel's log. A gk7205v300 + imx335 whose log was flooded with sensor i2c aborts sent a crash with no chip or sensor in it at all: the boot lines naming them were long gone from the record. meta.json still names them.

What changes

• Unpack keeps meta.json. When no meta field is sent, the server uses the bundle's copy, redacted like the field.
• The SoC and sensor come from meta.json when neither the camera's form fields nor the log name them. They pass the same chip-name check as the form fields; what the camera sends outright still wins.
• CRASH.md documents it.

Tests

TestMetaInTheBundle:

• a bundle carrying meta.json is stored with it;
• the chip and sensor come from it;
• an address in it is redacted;
• a form field still wins over it.

service/run.sh test ./internal/crashes/ passes. On dev, a real bundle with meta.json from a gk7205v300 + imx335, sent with a meta field, was stored with the build, chip and 44 configuration lines.

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Use bundled crash metadata when chip and sensor are missing

✨ Enhancement 🧪 Tests 📝 Documentation 🕐 10-20 Minutes

AI Description

• Read bundled meta.json when an upload has no separate meta field.
• Recover missing chip and sensor names from metadata without overriding form fields or logs.
• Document and test metadata storage, redaction, and hardware precedence.

Diagram

graph TD
  Request["Crash submission"] --> Unpack["Unpack bundle"] --> Parse["Parse crash log"] --> Defaults["Form or log hardware"] --> Select["Field else bundle"] --> Redact["Redact and validate"] --> Fallback["Fill missing hardware"] --> Store["Crash event store"]
  Request -->|"meta field"| Select
  Unpack -->|"meta.json"| Select

Loading

High-Level Assessment

The focused fallback fits the existing upload pipeline: it reuses JSON redaction and hardware-name validation while preserving form and log precedence. A separate metadata ingestion path would add complexity without a clear benefit.

Files changed (4) +52 / -2

Enhancement (2) +18 / -1

api.goUse redacted metadata to fill missing hardware names +17/-0

Use redacted metadata to fill missing hardware names

• Falls back to size-limited bundled meta.json when no meta field was sent. Redacts and validates the selected JSON, then uses valid chip and sensor names only when the form and log did not supply them.

service/internal/crashes/api.go

parse.goRetain meta.json while unpacking crash bundles +1/-1

Retain meta.json while unpacking crash bundles

• Adds meta.json to the archive entries Unpack keeps, making bundled metadata available to the upload pipeline.

service/internal/crashes/parse.go

Tests (1) +33 / -0

store_test.goTest bundled metadata storage and precedence +33/-0

Test bundled metadata storage and precedence

• Adds an integration test confirming that bundled metadata supplies chip and sensor names, is stored with addresses redacted, and does not override an explicit SoC form field.

service/internal/crashes/store_test.go

Documentation (1) +1 / -1

CRASH.mdDocument bundled metadata and hardware fallback +1/-1

Document bundled metadata and hardware fallback

• Explains what the firmware puts in meta.json, when the server uses it instead of a meta field, and when its chip and sensor names apply.

service/internal/crashes/CRASH.md

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Bundles with 16 logs are rejected ✓ Resolved

Description

Unpack now admits meta.json into the same map whose length enforces the 16-record limit. A
bundle with 16 dmesg records and the new metadata file therefore returns “more than 16 records,”
although Parse counts only the dmesg files as records.

Code

service/internal/crashes/parse.go[197]

+			if !strings.HasPrefix(name, "dmesg-") && name != "failsafe" && name != "pending" && name != "meta.json" {

Evidence

maxRecords is 16; the new allowlist includes meta.json, and the following check rejects an entry
when the map already has 16 keys. Parse counts only dmesg-* keys, while Submit returns an
unpacking error to the uploader.

service/internal/crashes/parse.go[24-33]
service/internal/crashes/parse.go[196-205]
service/internal/crashes/parse.go[218-229]
service/internal/crashes/api.go[114-120]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Adding `meta.json` to the unpacked map makes it consume a slot in the 16-record limit, rejecting bundles that previously fit.
## Fix Focus Areas
- service/internal/crashes/parse.go[196-205]
## Recommended Fix
Enforce the record limit on crash records independently of `meta.json`, and add a test with 16 dmesg files plus metadata.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Malformed bundle metadata blocks crash uploads ✓ Resolved

Description

Submit copies an embedded meta.json into in.meta and applies the existing fatal JSON
validation to it. When that optional file contains malformed JSON and no meta form field was sent,
an otherwise parseable crash is refused with HTTP 400 instead of being stored.

Code

service/internal/crashes/api.go[R139-140]

+	if in.meta == "" && len(files["meta.json"]) <= maxMeta {
+		in.meta = strings.TrimSpace(files["meta.json"])

Evidence

Unpack admits meta.json without validating its contents. The new fallback selects it, after
which Submit rejects invalid JSON before reaching Store.Insert; without that fallback, the
parsed crash proceeds to insertion.

service/internal/crashes/parse.go[186-205]
service/internal/crashes/api.go[114-120]
service/internal/crashes/api.go[139-159]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Automatically discovered, malformed `meta.json` causes an otherwise valid crash upload to fail.
## Fix Focus Areas
- service/internal/crashes/api.go[139-148]
## Recommended Fix
Validate embedded metadata separately and skip it when invalid, while retaining the existing error for an invalid explicit `meta` field. Test that a valid crash with malformed embedded metadata is accepted.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can keep summaries lean with Findings visible per group, which tucks the rest behind a View link

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[widgetii]

Both findings addressed in 0a2540c:

1. 16 records rejected. The limit now counts dmesg-* records only; failsafe, pending and meta.json don't take a record's place. There is a test with sixteen records plus meta.json.
2. Malformed embedded metadata. A meta.json found in the bundle that isn't JSON is left out, and the crash is stored without it. One sent explicitly as the meta field must still be JSON (400 otherwise). There is a test where a broken embedded meta.json still gets a 201.

[widgetii]

Rebased onto master. My commit fixing the daily-limit clock is dropped: #414 fixed the same thing on master (an event is stamped and counted on one clock). The test lookup by event id is kept, as its own commit. service/run.sh test passes on top of master.