Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions doc/specific_iocs/datastreaming/ADRs/001_histograms.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

## Status

Pending discussion with HRPD-X interested parties (including instrument scientists & Mantid).
Accepted

## Context

Expand All @@ -18,15 +18,19 @@ In event mode, over the course of a run, each individual neutron event's detecti
Event mode data can be later binned to form a histogram, but a histogram cannot be recovered to individual events. In
other words, histogramming is lossy. The advantage of histogram mode is that it typically produces smaller data volumes.

Histogram mode has historically been used due to hardware limitations in many cases.
Histogram mode has historically been used because:
- Hardware limitations in DAE2/3 preventing event-mode acquisition at high rates
- Analysis routines in Mantid which may perform poorly on event-mode data

## Decision

For HRPD-x, we will collect all data, including data from neutron monitors, in event mode only. HRPD-x will not support
histogram mode.
For HRPD-x, we will initially stream all data, including data from neutron monitors, in event mode. The Kafka streams will only contain event-mode data.

We will write event-mode Nexus files, however HRPD-X also require histogram-mode Nexus files (not containing neutron events) for their analysis routines.

## Consequences

- Data volumes on HRPD-x will be higher running in event mode compared to histogram mode. This includes both data in-flight
during networking and Kafka processing, as well as final Nexus file sizes.
during networking and Kafka processing, as well as final event-mode Nexus file sizes. The histogram file size is independent of number of events.
- Only considering events will simplify components of the HRPD-x data streaming implementation.
- There may be two separate files generated per run, `_event.nxs` and `_histogram.nxs`. The exact naming is to be determined.