Skip to content

Stages are outside the first release and tie every request to an unreleased sciline #35

Description

@SimonHeybrock

The first release has no interactive tuning and combines no runs (README, Goal). Stages trace only to an Assumed item, "changing a binning or a mask shows the new result within about a second" (users), which is still to be measured on LoKI files. One LoKI reduction is assumed to take seconds (sans).

Every request goes through the stage protocol, binding.stage(values, ())(). PipelineBinding always builds a sciline.Stage (pipeline.py), which exists only on the sciline branch map-reduce-outside-the-graph (scipp/sciline#245, open). So no real workflow can run on a released sciline.

Proposal

  • A binding is a function of all parameters, binding(**params) -> dict, and PipelineBinding computes with pipeline.compute.
  • Stages are removed until interactive tuning is in scope and ADR 0003: Stages outside the graph sciline#245 is merged.
  • Accumulators keep the part of a stage that pays off: an accumulator opens from a template and computes once what does not depend on its rows (ADR 0003, being rewritten). Decide this issue after that change lands.

Also found

A trace of the code against the requirements found more code that no first-release mode needs. Decide each part when its code is next touched:

  • references to pending records (stories S4 and C1 only);
  • the restart of pending records (story H2, now Assumed in the requirements);
  • datasets.watch and client.as_completed (story D7 only).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions