From 3746c59486c779e983e11b7cd74b90ba3a41b594 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Wed, 29 Jul 2026 09:27:15 +0100 Subject: [PATCH] feat: raise the message cap to fit the grown record MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three SD-ELEMENTs put the record at 245 bytes, but sequenceId and sysUpTime are both 32-bit counters: at full width the same record is 261 bytes, past the 256-byte cap and into truncation. The cap doubles to 512. Flash +11,980 B (+8 on the previous stage) RAM +9,440 B (+2,304) Log stack +704 B (+256) Service +992 B (+256) The cap is the largest single demand the formatter makes of whichever task calls it, so both stacks rise by exactly the difference — 568 to 824 on the log task, 788 to 1,044 on the service task. Both seams were sized at 2 KiB when the send moved, so they absorb it and no RAM moves with them. The RAM that does move accounts for itself exactly. The ring is eight records at the cap plus a two-byte length prefix each, so it grows 2,048. The store keeps one record buffer of cap plus integrity trailer plus framing, so it grows 256. That is the whole 2,304. The store is otherwise unaffected. SOLIDSYSLOG_FILE_DEFAULT_BLOCK_SIZE must hold one worst-case record — cap plus integrity trailer plus five framing bytes, 549 here — and at 8192 it has room to spare. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 15 +++++++----- app/config/solid_syslog_tunables.h | 10 ++++---- measurements/cap-rise.csv | 13 ++++++++++ measurements/stages.tsv | 1 + run-report.md | 38 +++++++++++++++--------------- 5 files changed, 46 insertions(+), 31 deletions(-) create mode 100644 measurements/cap-rise.csv diff --git a/README.md b/README.md index abb079d..11c53c5 100644 --- a/README.md +++ b/README.md @@ -10,16 +10,18 @@ It builds on a baseline that simulates the sort of device you might be adding th measures itself: see [docs/baseline.md](docs/baseline.md) for what the baseline is, how the figures are made, and how to run it. -## This stage — Origin +## This stage — Larger cap -The record carries an `origin` SD-ELEMENT naming the software, its version and the vendor's IANA -enterprise number, so it identifies its sender instead of leaving the collector to infer that from -the source address — which matters once the store replays records hours after the event. The `ip` -PARAM is left out: the address the collector sees is still the one that reached it. +The message cap doubles to 512 bytes. Three SD-ELEMENTs had already taken a short record close to +the old ceiling, and the two counters it carries are 32-bit — at full width the same record would +cross it and start truncating. + +The cost is almost all RAM, because the cap sizes three things at once: the ring, the store's +record buffer, and the deepest single demand the formatter makes of whichever task calls it. -**Cost above baseline: Flash +11,972 B, RAM +7,136 B.** +**Cost above baseline: Flash +11,980 B, RAM +9,440 B.** @@ -45,6 +47,7 @@ committed as [`run-report.md`](run-report.md), and rewritten by every stage. | Time quality | a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap | +7,612 | +5,880 | | File store | records that survive a failed send, spooled to disk with a checksum at rest | +11,576 | +7,092 | | Origin | the device named in the record itself, not inferred from the source address | +11,972 | +7,136 | +| Larger cap | headroom for the grown record, so full-width counters cannot push it into truncation | +11,980 | +9,440 | *Deltas are bytes above the baseline, which is itself Flash 350,308 B, RAM 111,192 B.* diff --git a/app/config/solid_syslog_tunables.h b/app/config/solid_syslog_tunables.h index fc5ca6f..0142a7a 100644 --- a/app/config/solid_syslog_tunables.h +++ b/app/config/solid_syslog_tunables.h @@ -4,12 +4,10 @@ #ifndef SOLID_SYSLOG_TUNABLES_H #define SOLID_SYSLOG_TUNABLES_H -/* The longest record this device will emit. The library defaults to 2048, the - * size RFC 5424 section 6.1 says a receiver should accept; over UDP, RFC 5426 - * section 3.2 guarantees only that 480 will be accepted. This device's records are - * far shorter, so 256 stays well inside every guarantee and anything longer is - * truncated rather than dropped. */ -#define SOLIDSYSLOG_MAX_MESSAGE_SIZE 256U +/* The longest record this device will emit; anything longer is truncated rather + * than dropped. Sized to this device's worst-case record, and within what RFC 5424 + * section 6.1 asks a receiver to accept. */ +#define SOLIDSYSLOG_MAX_MESSAGE_SIZE 512U /* One sender, so one destination address. The default of 3 suits a device * running UDP, plain TCP and TLS at once. */ diff --git a/measurements/cap-rise.csv b/measurements/cap-rise.csv new file mode 100644 index 0000000..5d1d6f5 --- /dev/null +++ b/measurements/cap-rise.csv @@ -0,0 +1,13 @@ +# cap-rise figures (bytes) — captured by scripts/run.sh (CAPTURE=1). +# The device reads measurements/Baseline.csv as its frozen baseline and reports current-minus-Baseline. +flash_text,361648 +flash_data,640 +static_bss,119992 +heap_used,4440 +mbedtls_peak,21332 +mbedtls_free,11436 +lwip_mem_free,7576 +lwip_pbufs_free,14 +stack_log,824 +stack_service,1044 +stack_harness,2848 diff --git a/measurements/stages.tsv b/measurements/stages.tsv index e4752e3..1cf998b 100644 --- a/measurements/stages.tsv +++ b/measurements/stages.tsv @@ -22,3 +22,4 @@ tcp TCP records the network retransmits instead of dropping, and a send that fai time-quality Time quality a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap file-store File store records that survive a failed send, spooled to disk with a checksum at rest origin Origin the device named in the record itself, not inferred from the source address +cap-rise Larger cap headroom for the grown record, so full-width counters cannot push it into truncation diff --git a/run-report.md b/run-report.md index 6b50e16..922f6f2 100644 --- a/run-report.md +++ b/run-report.md @@ -1,4 +1,4 @@ -# solid-syslog-example — run (origin) +# solid-syslog-example — run (cap-rise) ## Device (self-measured) @@ -10,16 +10,16 @@ [device] first record logged: yes [report] --- SolidSyslog cost above baseline (simulated existing application) --- [report] key,current,baseline,used_above_baseline -[report] flash_text,361640,349992,11648 +[report] flash_text,361648,349992,11656 [report] flash_data,640,316,324 -[report] static_bss,117688,110876,6812 +[report] static_bss,119992,110876,9116 [report] heap_used,4440,4440,0 -[report] mbedtls_peak,21336,21332,4 -[report] mbedtls_free,11432,11436,-4 +[report] mbedtls_peak,21332,21332,0 +[report] mbedtls_free,11436,11436,0 [report] lwip_mem_free,7576,7576,0 -[report] lwip_pbufs_free,13,14,-1 -[report] stack_log,568,120,448 -[report] stack_service,788,52,736 +[report] lwip_pbufs_free,14,14,0 +[report] stack_log,824,120,704 +[report] stack_service,1044,52,992 [report] stack_harness,2848,2840,8 [report] --- end --- [device] ready @@ -29,7 +29,7 @@ ```text text data bss dec hex filename - 361632 648 117688 479968 752e0 /w/build/baseline-cross/baseline.elf + 361640 648 119992 482280 75be8 /w/build/baseline-cross/baseline.elf ``` ## Listeners (proved before the device ran) @@ -47,23 +47,23 @@ ## Collector (syslog-ng) received ```text -wire <134>1 2026-07-29T08:21:38.360000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="236"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] device started -parsed PRIORITY=134 TIMESTAMP=2026-07-29T08:21:38+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="236"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] MSG=device started +wire <134>1 2026-07-29T08:25:48.280000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="228"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] device started +parsed PRIORITY=134 TIMESTAMP=2026-07-29T08:25:48+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="228"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] MSG=device started ``` -## Self-check (vs measurements/origin.csv) +## Self-check (vs measurements/cap-rise.csv) ```text - OK flash_text: 361640 (expected 361640, Δ0) + OK flash_text: 361648 (expected 361648, Δ0) OK flash_data: 640 (expected 640, Δ0) - OK static_bss: 117688 (expected 117688, Δ0) + OK static_bss: 119992 (expected 119992, Δ0) OK heap_used: 4440 (expected 4440, Δ0) - OK mbedtls_peak: 21336 (expected 21336, Δ0) - OK mbedtls_free: 11432 (expected 11432, Δ0) + OK mbedtls_peak: 21332 (expected 21332, Δ0) + OK mbedtls_free: 11436 (expected 11436, Δ0) OK lwip_mem_free: 7576 (expected 7576, Δ0) - OK lwip_pbufs_free: 13 (expected 13, Δ0) - OK stack_log: 568 (expected 568, Δ0) - OK stack_service: 788 (expected 788, Δ0) + OK lwip_pbufs_free: 14 (expected 14, Δ0) + OK stack_log: 824 (expected 824, Δ0) + OK stack_service: 1044 (expected 1044, Δ0) OK stack_harness: 2848 (expected 2848, Δ0) ```