From b108ac776f9406c94fde5482cce94bd07ad3b0b3 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Mon, 20 Jul 2026 09:56:50 +0100 Subject: [PATCH] Datastreaming histogram/event ADR following discussions with Mantid/Scientists Updated the status to 'Accepted' and clarified the decision regarding data collection modes for HRPD-x. Added details about the consequences of using event mode. --- .../datastreaming/ADRs/001_histograms.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/specific_iocs/datastreaming/ADRs/001_histograms.md b/doc/specific_iocs/datastreaming/ADRs/001_histograms.md index 08caccb08..782a5bf1f 100644 --- a/doc/specific_iocs/datastreaming/ADRs/001_histograms.md +++ b/doc/specific_iocs/datastreaming/ADRs/001_histograms.md @@ -3,7 +3,7 @@ ## Status -Pending discussion with HRPD-X interested parties (including instrument scientists & Mantid). +Accepted ## Context @@ -18,15 +18,19 @@ In event mode, over the course of a run, each individual neutron event's detecti Event mode data can be later binned to form a histogram, but a histogram cannot be recovered to individual events. In other words, histogramming is lossy. The advantage of histogram mode is that it typically produces smaller data volumes. -Histogram mode has historically been used due to hardware limitations in many cases. +Histogram mode has historically been used because: +- Hardware limitations in DAE2/3 preventing event-mode acquisition at high rates +- Analysis routines in Mantid which may perform poorly on event-mode data ## Decision -For HRPD-x, we will collect all data, including data from neutron monitors, in event mode only. HRPD-x will not support -histogram mode. +For HRPD-x, we will initially stream all data, including data from neutron monitors, in event mode. The Kafka streams will only contain event-mode data. + +We will write event-mode Nexus files, however HRPD-X also require histogram-mode Nexus files (not containing neutron events) for their analysis routines. ## Consequences - Data volumes on HRPD-x will be higher running in event mode compared to histogram mode. This includes both data in-flight -during networking and Kafka processing, as well as final Nexus file sizes. +during networking and Kafka processing, as well as final event-mode Nexus file sizes. The histogram file size is independent of number of events. - Only considering events will simplify components of the HRPD-x data streaming implementation. +- There may be two separate files generated per run, `_event.nxs` and `_histogram.nxs`. The exact naming is to be determined.