diff --git a/CHANGELOG.md b/CHANGELOG.md index e97d9b7..d8d8f25 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,80 @@ and each [release on GitHub](https://github.com/s4mstruthers/openprocess/release uses it as its description. Write for the people using the app: what they will notice, not how it was done. Versions before 0.7.0 were released as CPNpy. +## 0.10.5 + +- **Object-centric event logs.** *Open object-centric log* reads OCEL 2.0 + JSON (`.jsonocel`) and the older JSON-OCEL: events that each involve + several objects, by type, with no case id invented. *Flatten on an object + type* makes a classical log with one case per object of a type, for the + miners and checks that need one, and says what it copied. *Object-centric + map* draws the object-centric directly-follows graph: the activities + once, one set of arrows per object type in its own colour, with a + legend and the counts per type. A `.jsonocel` in the folder is listed + under LOGS, drops on the canvas, and appears in the Summary and the + report like any other result. + +## 0.10.4 + +- **A command palette.** ⌘K (Ctrl+K), or *View ▸ Command Palette…*: type a + few letters and the open analyses, models and logs, the folder's files, + every box (added to the current analysis), the three spaces, Connections + and every menu action are found at once; Enter runs the best match, ↑ ↓ + pick another. Nothing new on the window; everything in it a few + keystrokes away. + +## 0.10.3 + +- **Runs: every finished run, kept.** An analysis has *Canvas | Summary | + Runs*. Each finished run is a snapshot with the time, what changed before + it ("Inductive Miner: noise 0 → 0.2", "+ Check fit", "Run again, same + results") and every box's key figure. Pick A and B and the boxes line up + side by side, those whose result differs marked. *Use A's settings* puts + that run's settings back on the canvas and runs. The last twenty travel + with the workflow file. + +## 0.10.2 + +- **Export report.** ⋯ ▸ *Export report…* on an analysis writes one HTML + page, with nothing to install: the reproducibility status, the key figure + of every box, each box's result (nets, maps and trees drawn; tables and + scores as tables), how it got there, the code it ran and the papers it + follows, what it read with fingerprints, and how to cite. For a + supervisor, a reviewer, a hand-in. It opens in your browser when written. + +## 0.10.1 + +- **Step through an algorithm.** A box's *How* tab has *Step through ▸*: the + report plays one moment at a time (a note, a step of the derivation, a + value shown), with ◀ ▶, Play and the arrow keys, and the box's result drawn + above it. Each step lights up on the drawing what it names and dims the + rest, so the α-algorithm's eight steps become eight pictures: all the + activities, the start ones, the end ones, the candidate pairs, the maximal + pairs, the places. Nothing new is computed: the moments are the box's own + report, in order. + +## 0.10.0 + +Two things an academic looks for first: is this reproducible, and how do I +cite it. + +- **A reproducibility badge beside every analysis's title.** "Not recorded + yet"; "Recorded · this run reproduces it" when every result matches the + record's fingerprints and nothing the analysis depends on has changed; + "Recorded · n changes since" when an input file, a box file, a seed or a + version differs (hover for which); "This run differs from the record" + when the same inputs gave another result. Click it for the record. In a + folder the record follows each run by itself. +- **Cite.** *Help ▸ Cite OpenProcess…* gives the app's BibTeX and a one-line + citation. On a box's Code tab, each algorithm has *Cite* with BibTeX for + the paper it follows, generated from the reference list. An exported + experiment includes `CITATIONS.bib` and a *How to cite* section. The + repository has a `CITATION.cff`, so GitHub offers *Cite this repository*. +- **Fixed: an input file edited on disk was served from the cache.** A box + reading a file is now keyed by the file as it is (size and time), so a + re-run after editing the log computes again instead of repeating the old + result. + ## 0.9.5 - **The log page no longer discovers models by itself.** Its *Discover* tab diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 0000000..4ab990d --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,24 @@ +cff-version: 1.2.0 +message: If you use OpenProcess in teaching or research, please cite it as below. +type: software +title: OpenProcess: process mining, Petri nets and workflows in one open app +version: 0.10.5 +date-released: 2026-10-10 +repository-code: https://github.com/s4mstruthers/openprocess +url: https://github.com/s4mstruthers/openprocess +license: MIT +keywords: + - process mining + - Petri nets + - workflow nets + - conformance checking + - teaching + - reproducible research +authors: + - family-names: Struthers + given-names: Sam +abstract: >- + OpenProcess is a desktop app for process mining and Petri net modelling in which + every algorithm is a function whose code and source paper are one click away, and + every analysis keeps a record of its inputs, versions and seeds so that its numbers + can be checked again. diff --git a/README.md b/README.md index 93b4e9d..11e735a 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ tools (and, until version 0.7, went by the name CPNpy): | | What you can do | Instead of | |---|---|---| | **Petri nets & WF-nets** | Draw nets the way the lectures do. Get a soundness verdict with a counterexample for every violation, and replay it in the token game. Also: behavioural properties, P- and T-invariants, the footprint matrix, the reachability graph, PNML import and export. | WoPeD, ProM, pen and paper | -| **Process mining** | Import XES or CSV logs, or type textbook logs like `[^3, ^2]`. Filter them, explore variants, the dotted chart and the process map. Discover models (α-algorithm, Inductive Miner, Heuristics Miner, state-based regions). Check conformance (token replay, alignments, precision…) and compare logs. | ProM, Disco | +| **Process mining** | Import XES or CSV logs, OCEL 2.0 object-centric logs, or type textbook logs like `[^3, ^2]`. Filter them, explore variants, the dotted chart and the process map. Discover models (α-algorithm, Inductive Miner, Heuristics Miner, state-based regions). Check conformance (token replay, alignments, precision…) and compare logs. | ProM, Disco | | **Coloured Petri nets** | Open, edit and save CPN Tools models (`.cpn`), hierarchical ones included. Step through or simulate them, compute the state space, and export a simulation as an event log to mine. | CPN Tools / CPN IDE | | **Workflows** | Boxes on a canvas: a log, a miner, a fitness check, a comparison, a sweep over a setting, a prediction pipeline. Click a box to see its result, *how* it got there (the α-algorithm's eight steps, the replay per variant), its code and its settings. Change a setting and only what follows runs again. Saved as a `.cpnflow` file with everything needed to get the same numbers back. Your own algorithm is one Python function in a `boxes/` folder. | RapidProM | | **Learn** | A mode of its own for worksheets, built on the app: answer in boxes on the sheet (sets, markings, matrices, cuts and trees, alignments, a net in the editor or a workflow on the canvas beside it) and press **Check**. Most answers are checked automatically, often against answers worked out from the given log or net. Packs are plain Markdown, can be exams with a clock and points, and a past exam imports as a skeleton pack. Demo exercises included. | Answer sheets, a notes app and guesswork | @@ -741,8 +741,9 @@ extra) and *Predict the next activity*. Or start empty. one click adds the box that reads the file, already set. Type to search them all: "alpha" finds the α-algorithm, "boarding" the log; Enter adds the first match. -- **Files flow into the canvas.** Drop a log, a PNML file, a .cpn or a - ts.txt on the canvas and it becomes its input box at that spot; an input +- **Files flow into the canvas.** Drop a log, an OCEL 2.0 object-centric log + (`.jsonocel`), a PNML file, a .cpn or a ts.txt on the canvas and it becomes + its input box at that spot; an input box names its file ("INPUT · boarding.xes"). *Open log…* and *Log from notation…* in the sidebar land as boxes on the current analysis (or a new one named after the file). In a box's **Settings**, *Choose…* picks the diff --git a/openprocess/__init__.py b/openprocess/__init__.py index 9842f92..d7bdb66 100644 --- a/openprocess/__init__.py +++ b/openprocess/__init__.py @@ -48,7 +48,7 @@ #: The one place the version number is written (pyproject.toml, the update #: check and packaging/build.py read it from here). A release is tagged #: "v" + this, e.g. v0.2.0. -__version__ = "0.9.5" +__version__ = "0.10.5" __all__ = [ "Arc", "BindingElement", "CPNet", "Marking", "Multiset", "Page", "Place", diff --git a/openprocess/citation.py b/openprocess/citation.py new file mode 100644 index 0000000..9e4b951 --- /dev/null +++ b/openprocess/citation.py @@ -0,0 +1,193 @@ +"""Citations: BibTeX for the app, and for the works each algorithm follows. + +The reference list (:mod:`openprocess.references`) is written for people, one +line per work. :func:`entry` turns such a line into a BibTeX entry, well +enough for a bibliography: the authors before the title, the title between +asterisks, the venue after it, the year at the end, and the DOI or URL when +the reference has one. :func:`app_bibtex` cites the app itself, with its +version; ``CITATION.cff`` at the top of the repository says the same for +GitHub's *Cite this repository*. +""" + +from __future__ import annotations + +import datetime as _dt +import re +from dataclasses import dataclass, field + +from . import references as _references + +APP_TITLE = "OpenProcess: process mining, Petri nets and workflows in one open app" +APP_URL = "https://github.com/s4mstruthers/openprocess" +APP_AUTHORS = (("Struthers", "Sam"),) + + +def app_version() -> str: + from . import __version__ + return __version__ + + +# --------------------------------------------------------------------------- +# One entry +# --------------------------------------------------------------------------- +@dataclass(frozen=True) +class Entry: + """A BibTeX entry: its kind (``article``, ``book``, …), key and fields, in order.""" + + kind: str + key: str + fields: dict[str, str] = field(default_factory=dict) + + def bibtex(self) -> str: + width = max((len(name) for name in self.fields), default=0) + lines = [f"@{self.kind}{{{self.key},"] + for name, value in self.fields.items(): + if value: + lines.append(f" {name.ljust(width)} = {{{_escape(value)}}},") + lines.append("}") + return "\n".join(lines) + + +def _escape(value: str) -> str: + return value.replace("&", r"\&").replace("%", r"\%").replace("#", r"\#") + + +_TITLE = re.compile(r"\*(.+?)\*") +_YEAR = re.compile(r"\b(19|20)\d{2}\b") +_VOLUME = re.compile(r"\s*(\d+)\((\d+)\):(\d+[–-]\d+)") +_PAGES = re.compile(r"pp\.\s*(\d+[–-]\d+)") +_PROCEEDINGS = ("LNCS", "LNBIP", "Proceedings", "Workshop", "Conference", "Demo Track", "CEUR", "Symposium", + "ICPM", "Models of Concurrency") + + +def entry(reference: _references.Reference) -> Entry: + """The reference as a BibTeX entry (its kind guessed from the venue).""" + text = reference.citation.strip() + year_match = None + for year_match in _YEAR.finditer(text): + pass + year = year_match.group(0) if year_match else "" + link = reference.link or "" + doi, url = (link, "") if link.startswith("10.") else ("", link) + title_match = _TITLE.search(text) + if title_match is None: # a standard or a tool, named in one line + return Entry("misc", reference.key, {"title": text.rstrip("."), "year": year, "doi": doi, "url": url}) + authors = text[:title_match.start()].strip().rstrip(".,").strip() + title = title_match.group(1).strip().rstrip(",") + rest = text[title_match.end():].strip().lstrip(".,").strip().rstrip(".") + rest = re.sub(r",?\s*" + re.escape(year) + r"\s*$", "", rest).strip().rstrip(",.") if year else rest + fields: dict[str, str] = {} + if any(ch.isdigit() for ch in authors) or " " not in authors: + fields["organization"] = authors # "ISO/IEC 15909-2:2011" + else: + fields["author"] = " and ".join(part.strip() for part in re.split(r",\s*|\s+and\s+", authors) if part.strip()) + fields["title"] = title + lower = text.lower() + volume = _VOLUME.search(rest) + if "thesis" in lower: + kind = "phdthesis" + fields["school"] = re.sub(r"^phd thesis,?\s*", "", rest, flags=re.I) + elif "working paper" in lower or "technical report" in lower: + kind = "techreport" + fields["institution"] = rest + elif volume is not None: + kind = "article" + fields["journal"] = rest[:volume.start()].strip().rstrip(",") + fields["volume"], fields["number"], fields["pages"] = volume.group(1), volume.group(2), volume.group(3) + elif any(word in rest for word in _PROCEEDINGS): + kind = "inproceedings" + booktitle, publisher = rest, "" + if ". Springer" in rest or rest.endswith("Springer"): + booktitle, _, publisher = rest.rpartition(". ") + publisher = publisher.strip() or "Springer" + pages = _PAGES.search(booktitle) + if pages: + fields["pages"] = pages.group(1) + booktitle = _PAGES.sub("", booktitle).strip().rstrip(",") + fields["booktitle"] = booktitle + fields["publisher"] = publisher + elif "edition" in rest or rest.endswith("Press") or rest.startswith("Springer") or rest.startswith("Cambridge"): + kind = "book" + edition = re.search(r"(\d+)(st|nd|rd|th) edition", rest) + if edition: + fields["edition"] = edition.group(1) + rest = re.sub(r",?\s*\d+(st|nd|rd|th) edition\.?\s*", " ", rest).strip().rstrip(".,") + fields["publisher"] = rest + else: + kind = "misc" + fields["howpublished"] = rest + fields["year"] = year + fields["doi"] = doi + fields["url"] = url + return Entry(kind, reference.key, fields) + + +def entries_for_module(module: str) -> list[Entry]: + """The works the code in ``module`` follows (see :func:`references.topics_for_module`).""" + seen, out = set(), [] + for topic in _references.topics_for_module(module): + for key in topic.sources: + reference = _references.reference(key) + if reference is not None and key not in seen: + seen.add(key) + out.append(entry(reference)) + return out + + +# --------------------------------------------------------------------------- +# The app itself +# --------------------------------------------------------------------------- +def app_entry(version: str | None = None, year: int | None = None) -> Entry: + version = version or app_version() + year = year or _dt.date.today().year + return Entry("software", "openprocess", { + "author": " and ".join(f"{last}, {first}" for last, first in APP_AUTHORS), + "title": APP_TITLE, + "version": version, + "year": str(year), + "url": APP_URL, + "note": "Every algorithm shows its code and the paper it follows; every analysis keeps a record " + "that runs again", + }) + + +def app_bibtex(version: str | None = None, year: int | None = None) -> str: + return app_entry(version, year).bibtex() + + +def app_text(version: str | None = None, year: int | None = None) -> str: + """The citation in one line, as a reference list would have it.""" + version = version or app_version() + year = year or _dt.date.today().year + authors = ", ".join(f"{first} {last}" for last, first in APP_AUTHORS) + return f"{authors}. {APP_TITLE} (version {version}) [software], {year}. {APP_URL}" + + +def cff_text(version: str | None = None, released: _dt.date | None = None) -> str: + """The contents of ``CITATION.cff`` (Citation File Format 1.2.0) for ``version``.""" + version = version or app_version() + released = released or _dt.date.today() + authors = "\n".join(f" - family-names: {last}\n given-names: {first}" for last, first in APP_AUTHORS) + return ( + "cff-version: 1.2.0\n" + "message: If you use OpenProcess in teaching or research, please cite it as below.\n" + "type: software\n" + f"title: {APP_TITLE}\n" + f"version: {version}\n" + f"date-released: {released.isoformat()}\n" + f"repository-code: {APP_URL}\n" + f"url: {APP_URL}\n" + "license: MIT\n" + "keywords:\n - process mining\n - Petri nets\n - workflow nets\n - conformance checking\n" + " - teaching\n - reproducible research\n" + f"authors:\n{authors}\n" + "abstract: >-\n" + " OpenProcess is a desktop app for process mining and Petri net modelling in which\n" + " every algorithm is a function whose code and source paper are one click away, and\n" + " every analysis keeps a record of its inputs, versions and seeds so that its numbers\n" + " can be checked again.\n" + ) + + +__all__ = ["Entry", "entry", "entries_for_module", "app_entry", "app_bibtex", "app_text", "cff_text", + "APP_TITLE", "APP_URL"] diff --git a/openprocess/flow/boxes/discover.py b/openprocess/flow/boxes/discover.py index f9afe41..7bd53bf 100644 --- a/openprocess/flow/boxes/discover.py +++ b/openprocess/flow/boxes/discover.py @@ -16,7 +16,7 @@ from ...mining.processtree import ProcessTree, to_petri_net from ...mining.transition_system import TransitionSystem, transition_system_from_log from ..box import box -from ..types import Table +from ..types import OCDFG, OCEL, Table @box(name="α-algorithm", group="Discover") @@ -110,6 +110,20 @@ def regions_to_net(ts: TransitionSystem) -> PetriNet: return result.net +@box(name="Object-centric map", group="Discover") +def object_centric_map(ocel: OCEL) -> OCDFG: + """The object-centric directly-follows graph: every object followed + through its events, so each object type gets its own directly-follows + counts, drawn in its own colour on the shared activities. No case id is + invented, so nothing converges or diverges that did not.""" + from ...mining.ocel import object_centric_dfg + graph = object_centric_dfg(ocel) + for object_type in graph.object_types: + flow.note(f"{object_type}: {len(graph.edges[object_type])} paths, " + f"starts at {', '.join(sorted(graph.starts[object_type])) or '–'}") + return graph + + @box(name="Directly-follows graph", group="Discover") def directly_follows(log: EventLog) -> DFG: """The process map: which activity directly follows which, how often.""" diff --git a/openprocess/flow/boxes/input.py b/openprocess/flow/boxes/input.py index f23d63c..66e9128 100644 --- a/openprocess/flow/boxes/input.py +++ b/openprocess/flow/boxes/input.py @@ -12,6 +12,7 @@ from ...mining.playout import play_out from ...mining.transition_system import TransitionSystem, parse_transition_system from ...model.net import CPNet +from ...mining.ocel import OCEL from ..box import box @@ -50,6 +51,38 @@ def typed_log(text: str = "[^3, ^2, ]", name: str = "Ty return EventLog.from_simple_log(parse_simple_log(text), name) +@box(name="Open object-centric log", group="Input") +def open_ocel(file: Path) -> OCEL: + """Reads an object-centric event log (OCEL 2.0 JSON, ``.jsonocel``, or the + older JSON-OCEL): events that each involve several objects, by type. + + file: the OCEL file. Choose… picks it; a name alone means a file in the workflow's folder + """ + from ...mining.ocel import read_ocel + log = read_ocel(_resolve(file)) + flow.note(f"{Path(file).name}: {log.summary()}") + flow.note("Object types: " + ", ".join(f"{t} ({n})" for t, n in log.counts_by_type().items())) + return log + + +@box(name="Flatten on an object type", group="Input") +def flatten_ocel(ocel: OCEL, object_type: str = "") -> EventLog: + """A classical event log with one case per object of one type, for the + miners and checks that need a case: an event that involves several + objects of the type is copied into each of their cases (that is what + flattening does, and why the object-centric map exists). + + object_type: which type is the case (empty: the type with most objects) + """ + from ...mining.ocel import flatten + log = flatten(ocel, object_type or None) + chosen = log.name.rsplit(" · ", 1)[-1] + flow.note(f"Flattened on {chosen}: {len(log)} cases, {log.event_count} events " + f"(the log has {len(ocel)} events: the rest is copies)") + flow.note("Other types: " + ", ".join(t for t in ocel.object_types if t != chosen)) + return log + + @box(name="Open net", group="Input") def open_net(file: Path) -> PetriNet: """Opens a Petri net: a PNML file, or a net drawn in the editor. diff --git a/openprocess/flow/figures.py b/openprocess/flow/figures.py new file mode 100644 index 0000000..246c2ee --- /dev/null +++ b/openprocess/flow/figures.py @@ -0,0 +1,54 @@ +"""The key figure of a result: one number and a line, for a tile or a report. + +Shared by the Summary (:mod:`openprocess.gui.flow.summary`) and the HTML +report (:mod:`openprocess.flow.report`), so both say the same thing about a +result: a score's first metric, a table's size, a log's cases, a net's +places and transitions, a map's activities and paths. +""" + +from __future__ import annotations + +from .types import DFG, OCDFG, OCEL, EventLog, Figure, PetriNet, Scores, Table, Text + + +def number(value) -> str: + if isinstance(value, bool): + return "yes" if value else "no" + if isinstance(value, float): + return f"{value:.2f}" if abs(value) < 1000 else f"{value:,.0f}" + if isinstance(value, int): + return f"{value:,}" + return str(value) + + +def tile_parts(value) -> tuple[str, str] | None: + """(the big figure, the line under it) for a result, or None for a result + that has no figure to show (a trace, a prediction model, …).""" + if isinstance(value, Scores): + items = [(k, v) for k, v in value.metrics.items() if not isinstance(v, (dict, list))] + if not items: + return None + first, rest = items[0], items[1:4] + return (f"{number(first[1])} {first[0]}".strip(), + " · ".join(f"{k} {number(v)}" for k, v in rest) or value.model) + if isinstance(value, Table): + return f"{len(value.rows):,} × {len(value.columns)}", f"rows × columns · {value.name}" + if isinstance(value, Figure): + return "figure", value.caption or value.name + if isinstance(value, EventLog): + return f"{len(value):,} cases", f"{value.event_count:,} events" + if isinstance(value, PetriNet): + return f"{len(value.places)} · {len(value.transitions)}", "places · transitions" + if isinstance(value, DFG): + return f"{len(value.activities)} · {len(value.all_edges())}", "activities · paths" + if isinstance(value, OCEL): + return f"{len(value):,} events", f"{len(value.objects):,} objects of {len(value.object_types)} types" + if isinstance(value, OCDFG): + return f"{len(value.activities)} · {len(value.object_types)}", "activities · object types" + if isinstance(value, Text): + first = str(value).strip().splitlines()[0] if str(value).strip() else "" + return first[:40], value.name + return None + + +__all__ = ["number", "tile_parts"] diff --git a/openprocess/flow/record.py b/openprocess/flow/record.py index 2d9cea2..5f04aac 100644 --- a/openprocess/flow/record.py +++ b/openprocess/flow/record.py @@ -161,18 +161,20 @@ class Record: #: ``node id`` -> result fingerprint (variant 0), from the last run. results: dict[str, str] = field(default_factory=dict) saved: str = "" + #: The last runs, as :mod:`openprocess.flow.snapshots` keeps them (dicts). + history: list[dict] = field(default_factory=list) def to_dict(self) -> dict: return {"versions": self.versions, "environment": self.environment, "inputs": self.inputs, "custom_boxes": self.custom_boxes, "seeds": self.seeds, "results": self.results, - "saved": self.saved} + "saved": self.saved, "history": self.history} @classmethod def from_dict(cls, data: dict | None) -> "Record": data = data or {} return cls(data.get("versions", {}), data.get("environment", {}), data.get("inputs", []), data.get("custom_boxes", []), data.get("seeds", {}), data.get("results", {}), - data.get("saved", "")) + data.get("saved", ""), [d for d in data.get("history", []) if isinstance(d, dict)]) def make_record(workflow: Workflow, run: Run | None = None, folder: str | Path | None = None, @@ -252,11 +254,13 @@ def differences(record: Record, workflow: Workflow, folder: str | Path | None = # Files # --------------------------------------------------------------------------- def save(workflow: Workflow, path: str | Path, run: Run | None = None, - folder: str | Path | None = None, lock: bool = True) -> Record: - """Write the workflow and its record to ``path``; returns the record.""" + folder: str | Path | None = None, lock: bool = True, history: list[dict] | None = None) -> Record: + """Write the workflow and its record to ``path``; returns the record. + ``history``: the analysis's snapshots (see :mod:`.snapshots`), kept in the file.""" path = Path(path) folder = folder if folder is not None else path.parent record = make_record(workflow, run, folder, lock) + record.history = list(history or []) data = {"format": FORMAT, **workflow.to_dict(), "record": record.to_dict()} text = json.dumps(data, indent=2, ensure_ascii=False, default=str) temporary = path.with_name(path.name + ".tmp") @@ -318,9 +322,20 @@ def export_experiment(workflow: Workflow, run: Run | None, target: str | Path, verdict = " → " + _describe(result.value) lines.append(f"- **{workflow.title(node)}** (`{spec.id}`): {settings}; {fed}{verdict}") lines += ["", "## Inputs", ""] + [f"- `{i['file']}` sha256 `{i['sha256']}`" for i in record.inputs] + ["", "## Your boxes", ""] + [f"- `{b['file']}` sha256 `{b['sha256']}`" for b in record.custom_boxes] + from .. import citation + from .box import algorithm_calls + entries = [citation.app_entry()] + for node in workflow.order(): + for called in algorithm_calls(workflow.spec(node)): + for item in citation.entries_for_module(called.module): + if all(item.key != e.key for e in entries): + entries.append(item) + lines += ["", "## How to cite", "", citation.app_text(), "", + "`CITATIONS.bib` has this entry and the papers the algorithms follow, as BibTeX."] with zipfile.ZipFile(target, "w", zipfile.ZIP_DEFLATED) as zipped: zipped.writestr(f"{name}.cpnflow", json.dumps(data, indent=2, ensure_ascii=False, default=str)) zipped.writestr("requirements.lock", requirements_lock(record)) + zipped.writestr("CITATIONS.bib", "\n\n".join(e.bibtex() for e in entries) + "\n") zipped.writestr("README.md", "\n".join(lines) + "\n") for item in record.inputs: path = _resolve(item["file"], folder) diff --git a/openprocess/flow/report.py b/openprocess/flow/report.py new file mode 100644 index 0000000..eb06cda --- /dev/null +++ b/openprocess/flow/report.py @@ -0,0 +1,270 @@ +"""An analysis as one HTML page: for a supervisor, a reviewer, a hand-in. + +:func:`html_report` writes everything a reader needs and nothing they have +to install: the key figure of every box, each box's result (a drawing when +the app made one, a table, a score), how it got there (the box's own +report), the code it ran and the papers it follows, what it read (with +fingerprints), whether it is reproducible, and how to cite. One file, no +external assets, readable in any browser and printable. + +The page is built from the workflow and its run alone; drawings (SVG) are +passed in by the app, which has the canvas to render them. So the same +report can be made without the app (``openprocess report``), minus the +pictures. +""" + +from __future__ import annotations + +import base64 +import datetime as _dt +from html import escape +from pathlib import Path + +from .. import citation +from .box import algorithm_calls +from .figures import number, tile_parts +from .record import Record, make_record +from .runner import DONE, Run +from .types import DFG, OCDFG, OCEL, EventLog, Figure, Footprint, PetriNet, ProcessTree, Scores, Table, Text +from .workflow import Workflow + +CSS = """ +:root { --ink: #1d1d1f; --muted: #6e6e73; --line: #e1e1e6; --soft: #f5f5f7; --accent: #0a84ff; + --good: #0ca30c; --warn: #c77800; --bad: #d12b2b; } +* { box-sizing: border-box; } +body { margin: 0; background: #fff; color: var(--ink); font: 15px/1.55 -apple-system, "Helvetica Neue", + Helvetica, Arial, sans-serif; } +main { max-width: 900px; margin: 0 auto; padding: 36px 24px 72px; } +h1 { font-size: 28px; margin: 0 0 4px; } h2 { font-size: 20px; margin: 40px 0 8px; } +h3 { font-size: 16px; margin: 24px 0 6px; } p { margin: 6px 0; } .muted { color: var(--muted); } +.caption { color: var(--muted); font-size: 11px; font-weight: 700; letter-spacing: .07em; text-transform: uppercase; } +.badge { display: inline-block; font-size: 12px; font-weight: 600; padding: 2px 10px; border-radius: 10px; + color: #fff; background: var(--muted); vertical-align: middle; } +.badge.good { background: var(--good); } .badge.warning { background: var(--warn); color: #1d1d1f; } +.badge.critical { background: var(--bad); } +.tiles { display: flex; flex-wrap: wrap; gap: 10px; margin: 12px 0 4px; } +.tile { border: 1px solid var(--line); border-radius: 10px; padding: 10px 14px; min-width: 160px; } +.tile .big { font-size: 22px; font-weight: 600; } +.box { border: 1px solid var(--line); border-radius: 12px; padding: 16px 18px; margin: 16px 0; } +.box h3 { margin-top: 0; } .eyebrow { color: var(--muted); font-size: 11px; font-weight: 700; letter-spacing: .07em; } +table { border-collapse: collapse; font-size: 13px; margin: 8px 0; max-width: 100%; } +th, td { text-align: left; padding: 4px 10px 4px 0; border-bottom: 1px solid var(--line); vertical-align: top; } +th { color: var(--muted); font-weight: 600; } +pre { background: var(--soft); border-radius: 8px; padding: 12px 14px; overflow-x: auto; font-size: 12px; + line-height: 1.45; } +code { font-size: 12.5px; background: var(--soft); padding: 1px 4px; border-radius: 4px; } +svg { max-width: 100%; height: auto; } +details { margin: 10px 0; } summary { cursor: pointer; color: var(--accent); font-weight: 500; } +.steps td:first-child { white-space: nowrap; color: var(--muted); font-weight: 600; } +.mono { font-family: ui-monospace, Menlo, monospace; font-size: 12px; } +@media print { main { padding: 0; } details { break-inside: avoid; } .box { break-inside: avoid; } } +""" + + +def html_report(workflow: Workflow, run: Run | None, folder=None, drawings: dict[str, str] | None = None, + record: Record | None = None, status=None, title: str | None = None) -> str: + """The analysis as a self-contained HTML page. + + ``drawings``: SVG text per box id (the app renders nets, maps and trees); + ``status``: a :class:`~openprocess.flow.reproducibility.Status`, when known. + """ + drawings = drawings or {} + record = record or make_record(workflow, run, folder, lock=False) + name = title or workflow.name or "Analysis" + when = _dt.datetime.now().strftime("%Y-%m-%d %H:%M") + parts = [f"\n" + f"" + f"{escape(name)} — OpenProcess report
", + f"
OpenProcess report

{escape(name)}

", + f"

{len(workflow.nodes)} box{'es' if len(workflow.nodes) != 1 else ''} · " + f"made {escape(when)} with OpenProcess {escape(record.versions.get('openprocess', ''))}, " + f"Python {escape(record.versions.get('python', ''))}.

"] + if status is not None: + parts.append(f"

{escape(status.headline)} " + f"{escape(' '.join(status.details))}

") + parts.append(_summary(workflow, run)) + parts.append("

The boxes, in the order they run

") + for node in workflow.order(): + parts.append(_box_section(workflow, run, node, drawings.get(node.id))) + parts.append(_inputs_section(record)) + parts.append(_cite_section(workflow)) + parts.append("
") + return "\n".join(parts) + + +# --------------------------------------------------------------------------- +# Sections +# --------------------------------------------------------------------------- +def _summary(workflow: Workflow, run: Run | None) -> str: + tiles = [] + for node in workflow.order(): + result = run.result(node) if run is not None else None + if result is None or result.status != DONE: + continue + parts = tile_parts(result.value) + if parts is None: + continue + big, sub = parts + tiles.append(f"
{escape(workflow.spec(node).group.upper())}
" + f"
{escape(workflow.title(node.id))}
{escape(big)}
" + f"
{escape(sub)}
") + if not tiles: + return "

No results: the analysis has not run.

" + return ("

Summary

The key figure of every box, as the box reported it.

" + f"
{''.join(tiles)}
") + + +def _box_section(workflow: Workflow, run: Run | None, node, drawing: str | None) -> str: + spec = workflow.spec(node) + result = run.result(node) if run is not None else None + settings = ", ".join(f"{k} = {escape(str(v))}" for k, v in node.settings.items()) or "no settings" + fed = ", ".join(f"{escape(port)} ← {escape(workflow.title(src))}" + for port, sources in workflow.inputs_of(node).items() for src, _ in sources) or "no inputs" + out = [f"
", + f"
{escape(spec.group.upper())}

{escape(workflow.title(node.id))}

", + f"

{escape(spec.help.split(chr(10))[0]) if spec.help else ''}

", + f"

Settings: {settings}. Inputs: {fed}.

"] + if result is None: + out.append("

Not run.

") + elif result.status != DONE: + out.append(f"

{escape(result.status)} " + f"{escape(result.error or result.message or '')}

") + else: + out.append("
Result
") + if drawing: + out.append(drawing) + out.append(_value_html(result.value)) + explanation = getattr(result, "explanation", None) + if explanation is not None and not explanation.empty: + out.append("
How it got there" + _explanation_html(explanation) + "
") + out.append("
The code it ran") + out.append(f"

The box: {escape(spec.id)}

{escape(spec.source or '')}
") + for called in algorithm_calls(spec): + out.append(f"

The algorithm: {escape(called.module)}.{escape(called.name)} " + f"({escape(called.where)})

{escape(called.source or '')}
") + entries = citation.entries_for_module(called.module) + if entries: + out.append("

Follows: " + "; ".join( + escape(_one_line(e)) for e in entries) + "

") + out.append("
") + return "\n".join(out) + + +def _one_line(entry: citation.Entry) -> str: + f = entry.fields + who = f.get("author") or f.get("organization") or "" + venue = f.get("journal") or f.get("booktitle") or f.get("publisher") or f.get("school") or f.get("institution") or "" + return f"{who}. {f.get('title', '')}. {venue}, {f.get('year', '')}.".replace(" .", ".").replace(", .", ".") + + +def _value_html(value) -> str: + if isinstance(value, Scores): + rows = "".join(f"{escape(str(k))}{escape(number(v))}" + for k, v in value.metrics.items() if not isinstance(v, (dict, list))) + note = f"

{escape(value.note)}

" if value.note else "" + return f"{rows}
{escape(value.model)}
{note}" + if isinstance(value, Table): + head = "".join(f"{escape(str(c))}" for c in value.columns) + body = "".join("" + "".join(f"{escape(number(cell) if isinstance(cell, (int, float)) else str(cell))}" + for cell in row) + "" for row in value.rows[:200]) + more = f"

… {len(value.rows) - 200:,} more rows.

" if len(value.rows) > 200 else "" + return f"{head}{body}
{more}" + if isinstance(value, Figure): + if value.svg: + return value.svg + (f"

{escape(value.caption)}

" if value.caption else "") + if value.png: + data = base64.b64encode(value.png).decode("ascii") + return f"{escape(value.name)}" + return "" + if isinstance(value, EventLog): + from ..mining.stats import summarise + summary = summarise(value) + rows = "".join(f"⟨{escape(', '.join(v.sequence))}⟩{v.count:,}" + for v in summary.variants[:20]) + more = (f"

… {summary.variant_count - 20:,} more variants.

" + if summary.variant_count > 20 else "") + return (f"

{summary.case_count:,} cases · {summary.event_count:,} events · " + f"{summary.variant_count:,} variants · {summary.activity_count} activities

" + f"{rows}
VariantCases
{more}") + if isinstance(value, PetriNet): + return f"

{escape(value.summary())}

" + if isinstance(value, ProcessTree): + return f"

{escape(str(value))}

" + if isinstance(value, DFG): + return f"

{len(value.activities)} activities · {len(value.all_edges())} paths

" + if isinstance(value, OCEL): + rows = "".join(f"{escape(t)}{n:,}" for t, n in value.counts_by_type().items()) + return f"

{escape(value.summary())}

{rows}
Object typeObjects
" + if isinstance(value, OCDFG): + rows = "".join(f"{escape(t)}{escape(a)}{escape(b)}{n:,}" + for t in value.object_types for (a, b), n in value.edges[t].most_common()) + return (f"

{escape(value.summary())}

" + f"{rows}
Object typeFromToObjects
") + if isinstance(value, Footprint): + matrix = value.matrix() + head = "".join(f"{escape(a)}" for a in value.activities) + body = "".join(f"{escape(a)}" + "".join(f"{escape(cell)}" for cell in row) + "" + for a, row in zip(value.activities, matrix)) + return f"{head}{body}
" + if isinstance(value, Text): + return f"
{escape(str(value))}
" + if value is None: + return "

Nothing (the box writes a file, or has no output).

" + return f"

{escape(type(value).__name__)}

" + + +def _explanation_html(explanation) -> str: + out = [] + for entry in explanation.entries: + if entry[0] == "note": + out.append(f"

· {escape(entry[1])}

") + elif entry[0] == "steps": + rows = "".join(f"{escape(title)}{escape(content)}" + for title, content in entry[1]) + out.append(f"{rows}
") + elif entry[0] == "show": + if entry[1]: + out.append(f"
{escape(str(entry[1]))}
") + out.append(_value_html(entry[2])) + return "\n".join(out) + + +def _inputs_section(record: Record) -> str: + out = ["

What it read

"] + if record.inputs: + rows = "".join(f"{escape(i['file'])}{escape(i['sha256'][:16])}…" + for i in record.inputs) + out.append(f"{rows}
Filesha256
") + else: + out.append("

No files: every input was typed or simulated.

") + if record.custom_boxes: + rows = "".join(f"{escape(b['file'])}{escape(b['sha256'][:16])}…" + for b in record.custom_boxes) + out.append(f"

Your own boxes:

{rows}
Filesha256
") + if record.seeds: + out.append("

Seeds: " + ", ".join(f"{escape(str(k))} = {escape(str(v))}" + for k, v in record.seeds.items()) + "

") + return "\n".join(out) + + +def _cite_section(workflow: Workflow) -> str: + entries = [citation.app_entry()] + for node in workflow.order(): + for called in algorithm_calls(workflow.spec(node)): + for item in citation.entries_for_module(called.module): + if all(item.key != e.key for e in entries): + entries.append(item) + bib = "\n\n".join(e.bibtex() for e in entries) + return ("

How to cite

" + f"

{escape(citation.app_text())}

" + f"
BibTeX, with the papers the algorithms follow
{escape(bib)}
") + + +def write_report(path: str | Path, **kwargs) -> Path: + """Write :func:`html_report` to ``path``.""" + path = Path(path) + path.write_text(html_report(**kwargs), encoding="utf-8") + return path + + +__all__ = ["html_report", "write_report", "CSS"] diff --git a/openprocess/flow/reproducibility.py b/openprocess/flow/reproducibility.py new file mode 100644 index 0000000..8097f43 --- /dev/null +++ b/openprocess/flow/reproducibility.py @@ -0,0 +1,62 @@ +"""Is this analysis reproducible, right now? One status for the header. + +The record (:mod:`openprocess.flow.record`) is a file; this turns it into a +visible promise. :func:`status` compares the record with the world now +(the inputs' fingerprints, the box files, the seeds, the versions) and with +the latest run (every box's result against its recorded fingerprint), and +says one of: + +* **Not recorded yet** -- the analysis has never been saved with a run; +* **Recorded · this run reproduces it** -- every result matches the record and + nothing it depends on has changed; +* **Recorded · n changes since** -- an input file, a box file, a seed or a + version differs from the record (a re-run may give other numbers); +* **This run differs from the record** -- the same inputs gave another result. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +from .record import Record, check, differences +from .runner import DONE, Run +from .workflow import Workflow + + +@dataclass +class Status: + #: "unrecorded", "reproduces", "unchecked", "changed" or "differs". + state: str + headline: str + details: list[str] = field(default_factory=list) + + @property + def tone(self) -> str: + return {"reproduces": "good", "changed": "warning", "differs": "critical"}.get(self.state, "muted") + + +def status(workflow: Workflow, record: Record | None, run: Run | None, folder=None) -> Status: + """The analysis's reproducibility, from its record and its latest run.""" + if record is None or not record.results: + return Status("unrecorded", "Not recorded yet", [ + "Save the analysis (it is saved as you work in a folder): the record keeps the inputs' " + "fingerprints, the versions and the seeds, so the numbers can be checked again later."]) + changed = differences(record, workflow, folder) + mismatches = check(workflow, record, run) if run is not None else [] + compared = [node_id for node_id in record.results + if run is not None and run.result(node_id, 0) is not None + and run.result(node_id, 0).status == DONE] + if mismatches: + return Status("differs", "This run differs from the record", mismatches + changed) + if changed: + count = len(changed) + return Status("changed", f"Recorded · {count} change{'s' if count != 1 else ''} since", changed) + if not compared: + return Status("unchecked", "Recorded · not run since", [ + "Run ▶ and every result is checked against its recorded fingerprint."]) + return Status("reproduces", "Recorded · this run reproduces it", [ + f"{len(compared)} result{'s' if len(compared) != 1 else ''} match the record's fingerprints, and " + "nothing the analysis depends on has changed."]) + + +__all__ = ["Status", "status"] diff --git a/openprocess/flow/runner.py b/openprocess/flow/runner.py index 018395b..de2dacd 100644 --- a/openprocess/flow/runner.py +++ b/openprocess/flow/runner.py @@ -123,6 +123,16 @@ def clear(self) -> None: self.entries.clear() +def _file_signature(value) -> str: + """``size:mtime_ns`` of a file setting's file, or ``missing``.""" + from pathlib import Path + try: + stat = Path(value).stat() + except OSError: + return "missing" + return f"{stat.st_size}:{stat.st_mtime_ns}" + + def settings_key(settings: dict[str, Any]) -> str: return json.dumps({k: (str(v) if not isinstance(v, (int, float, bool, str, type(None), list, dict)) else v) for k, v in sorted(settings.items())}, sort_keys=True, default=str) @@ -267,6 +277,12 @@ def _run_node(self, workflow: Workflow, node: Node, run: Run, variant: int, result.status = IDLE result.message = f"Choose the {setting.name} in Settings" return result + # A file setting is keyed by the file as it is now (size and time), not + # just its name: an input edited on disk gets computed again, never + # served from the cache as if nothing had happened. + for setting in spec.settings: + if setting.kind == "path" and settings.get(setting.name) not in (None, ""): + keys.append(f"{setting.name}={_file_signature(settings[setting.name])}") result.key = hashlib.sha256( (spec.fingerprint + "|" + settings_key(settings) + "|" + "|".join(keys)).encode()).hexdigest() cached = self.cache.get(result.key) diff --git a/openprocess/flow/snapshots.py b/openprocess/flow/snapshots.py new file mode 100644 index 0000000..7780efd --- /dev/null +++ b/openprocess/flow/snapshots.py @@ -0,0 +1,179 @@ +"""Snapshots of an analysis: every finished run, kept, so two can be compared. + +A sweep shows the effect of one setting over a range. Across sessions the +question is wider: what did I change, and what did it do? A +:class:`Snapshot` keeps, for one finished run, every box's settings and key +figure and a fingerprint of its result; a :class:`History` keeps the last +few snapshots with the workflow file. :func:`describe_change` says what +changed between two runs ("Inductive Miner: noise 0 → 0.2"), and +:func:`compare` lines the two up box by box. A snapshot's settings can be +put back on the workflow, so a run can be returned to. +""" + +from __future__ import annotations + +import datetime as _dt +from dataclasses import dataclass, field +from typing import Any + +from .figures import tile_parts +from .record import fingerprint +from .runner import DONE, Run +from .sweep import Sweep, is_sweep +from .workflow import Workflow, _json_value + +#: How many snapshots a history keeps. +LIMIT = 20 + + +@dataclass +class Snapshot: + taken: str # ISO time + label: str = "" # what changed before this run + #: box id -> {setting: JSON value} (a sweep as {"sweep": [...]}) + settings: dict[str, dict[str, Any]] = field(default_factory=dict) + titles: dict[str, str] = field(default_factory=dict) + statuses: dict[str, str] = field(default_factory=dict) + #: box id -> (key figure, line under it), for boxes that have one + figures: dict[str, tuple[str, str]] = field(default_factory=dict) + #: box id -> result fingerprint (done boxes only) + fingerprints: dict[str, str] = field(default_factory=dict) + + def to_dict(self) -> dict: + return {"taken": self.taken, "label": self.label, "settings": self.settings, "titles": self.titles, + "statuses": self.statuses, "figures": {k: list(v) for k, v in self.figures.items()}, + "fingerprints": self.fingerprints} + + @classmethod + def from_dict(cls, data: dict) -> "Snapshot": + return cls(str(data.get("taken", "")), str(data.get("label", "")), dict(data.get("settings", {})), + dict(data.get("titles", {})), dict(data.get("statuses", {})), + {k: (str(v[0]), str(v[1])) for k, v in data.get("figures", {}).items() + if isinstance(v, (list, tuple)) and len(v) == 2}, + dict(data.get("fingerprints", {}))) + + @property + def done(self) -> int: + return sum(1 for s in self.statuses.values() if s == DONE) + + +def _settings_json(workflow: Workflow) -> dict[str, dict[str, Any]]: + return {node.id: {k: (v.to_json() if is_sweep(v) else _json_value(v)) for k, v in node.settings.items()} + for node in workflow.nodes.values()} + + +def take(workflow: Workflow, run: Run | None, previous: "Snapshot | None" = None) -> Snapshot: + """A snapshot of ``run``; its label says what changed since ``previous``.""" + snapshot = Snapshot(_dt.datetime.now().isoformat(timespec="seconds"), settings=_settings_json(workflow), + titles={node.id: workflow.title(node.id) for node in workflow.nodes.values()}) + for node in workflow.nodes.values(): + result = run.result(node) if run is not None else None + snapshot.statuses[node.id] = result.status if result is not None else "waiting" + if result is not None and result.status == DONE: + parts = tile_parts(result.value) + if parts is not None: + snapshot.figures[node.id] = parts + try: + snapshot.fingerprints[node.id] = fingerprint(result.value) + except Exception: # noqa: BLE001 - a value that cannot be fingerprinted is left out + pass + snapshot.label = describe_change(previous, snapshot) + return snapshot + + +def _show(value) -> str: + if isinstance(value, dict) and "sweep" in value: + return "sweep " + ", ".join(str(v) for v in value["sweep"][:4]) + ("…" if len(value["sweep"]) > 4 else "") + return str(value) + + +def setting_changes(before: Snapshot | None, after: Snapshot) -> list[str]: + """"Inductive Miner: noise 0 → 0.2", one per setting that differs.""" + if before is None: + return [] + out = [] + for node_id, settings in after.settings.items(): + then = before.settings.get(node_id) + if then is None: + continue + for name, value in settings.items(): + if name in then and then[name] != value: + out.append(f"{after.titles.get(node_id, node_id)}: {name} {_show(then[name])} → {_show(value)}") + return out + + +def describe_change(before: Snapshot | None, after: Snapshot) -> str: + """What changed between two runs, in a few words.""" + if before is None: + return "First run" + parts = setting_changes(before, after) + added = [after.titles.get(n, n) for n in after.settings if n not in before.settings] + removed = [before.titles.get(n, n) for n in before.settings if n not in after.settings] + parts += [f"+ {name}" for name in added] + [f"− {name}" for name in removed] + if parts: + return "; ".join(parts) + same = all(after.fingerprints.get(n) == before.fingerprints.get(n) for n in after.fingerprints) + return "Run again, same results" if same and after.fingerprints else "Run again" + + +@dataclass +class Row: + node_id: str + title: str + a: str # A's key figure ("" when A had no such box) + b: str + changed: bool + changes: list[str] = field(default_factory=list) + + +def compare(a: Snapshot, b: Snapshot) -> list[Row]: + """Box by box: the key figure in A and in B, and whether the result differs.""" + rows = [] + order = list(a.settings) + [n for n in b.settings if n not in a.settings] + for node_id in order: + title = b.titles.get(node_id) or a.titles.get(node_id, node_id) + fa, fb = a.figures.get(node_id), b.figures.get(node_id) + in_both = node_id in a.fingerprints and node_id in b.fingerprints + changed = (a.fingerprints.get(node_id) != b.fingerprints.get(node_id)) if in_both else (fa != fb) + changes = [c for c in setting_changes(a, b) if c.startswith(title + ":")] + rows.append(Row(node_id, title, fa[0] if fa else "", fb[0] if fb else "", changed, changes)) + return rows + + +@dataclass +class History: + snapshots: list[Snapshot] = field(default_factory=list) # oldest first + + @property + def latest(self) -> Snapshot | None: + return self.snapshots[-1] if self.snapshots else None + + def add(self, snapshot: Snapshot) -> bool: + """Keep ``snapshot`` unless it repeats the latest (same settings, same + results); at most :data:`LIMIT`. Returns whether it was kept.""" + latest = self.latest + if latest is not None and latest.settings == snapshot.settings \ + and latest.fingerprints == snapshot.fingerprints and set(latest.statuses) == set(snapshot.statuses): + return False + self.snapshots.append(snapshot) + del self.snapshots[:-LIMIT] + return True + + def to_dict(self) -> list[dict]: + return [s.to_dict() for s in self.snapshots] + + @classmethod + def from_dict(cls, data) -> "History": + return cls([Snapshot.from_dict(d) for d in (data or []) if isinstance(d, dict)][-LIMIT:]) + + +def settings_to_apply(snapshot: Snapshot, node_id: str) -> dict[str, Any]: + """A snapshot's settings for one box, as :meth:`Workflow.set` takes them.""" + out = {} + for name, value in snapshot.settings.get(node_id, {}).items(): + out[name] = Sweep(tuple(value["sweep"])) if isinstance(value, dict) and "sweep" in value else value + return out + + +__all__ = ["Snapshot", "History", "Row", "take", "compare", "describe_change", "setting_changes", + "settings_to_apply", "LIMIT"] diff --git a/openprocess/flow/types.py b/openprocess/flow/types.py index 4246d80..0ec49ea 100644 --- a/openprocess/flow/types.py +++ b/openprocess/flow/types.py @@ -28,6 +28,7 @@ class Embedding: ... from ..mining.conformance.alignments import AlignmentResult from ..mining.conformance.token_replay import ReplayResult from ..mining.dfg import DFG +from ..mining.ocel import OCDFG, OCEL from ..mining.footprint import Footprint from ..mining.log import EventLog, SimpleLog from ..mining.petrinet import Marking, PetriNet @@ -268,6 +269,8 @@ def is_known(cls) -> bool: (ProcessTree, "tree", "Process tree", "#2e9e4f"), (TransitionSystem, "ts", "Transition system", "#b0430f"), (DFG, "dfg", "Directly-follows graph", "#b0430f"), + (OCEL, "ocel", "Object-centric log", "#0f6e9e"), + (OCDFG, "ocdfg", "Object-centric map", "#b0430f"), (Footprint, "footprint", "Footprint", "#b0430f"), (ReplayResult, "replay", "Replay", "#8250c8"), (AlignmentResult, "alignments", "Alignments", "#8250c8"), @@ -283,7 +286,7 @@ def is_known(cls) -> bool: register_type(_cls, _key, _name, _colour) __all__ = [ - "AlignmentResult", "Any", "CPNet", "DFG", "Dataset", "EventLog", "Figure", "Footprint", + "AlignmentResult", "Any", "CPNet", "DFG", "Dataset", "EventLog", "Figure", "Footprint", "OCDFG", "OCEL", "Marking", "PetriNet", "Predictions", "Predictor", "ProcessTree", "Regions", "ReplayResult", "Scores", "SimpleLog", "Table", "Text", "TransitionSystem", "TypeInfo", "TYPES", "is_known", "register_type", "type_by_key", "type_info", diff --git a/openprocess/gui/flow/page.py b/openprocess/gui/flow/page.py index ad5c92f..62e7355 100644 --- a/openprocess/gui/flow/page.py +++ b/openprocess/gui/flow/page.py @@ -34,6 +34,7 @@ ) from ...flow import record as records +from ...flow import reproducibility, snapshots from ...flow.library import Library from ...flow.runner import BLOCKED, DONE, FAILED, IDLE, RUNNING, WAITING, Cache, Result, Run, Runner from ...flow.types import EventLog, Figure, PetriNet, Scores, Table @@ -44,6 +45,7 @@ from ..studio.workers import run_in_background from .canvas import WorkflowScene, WorkflowView from .picker import BoxPicker, input_box_for +from .runs import RunsWidget from .summary import SummaryWidget from .viewers import SettingsWidget, code_widget, how_widget, pop_out, result_widget, status_text @@ -93,6 +95,8 @@ def brief(value) -> str: class WorkflowPage(QWidget): status = Signal(str) saved = Signal() + #: A run finished (not stopped): the record wants updating (the window autosaves in a folder). + ran = Signal() #: Something changed that should be saved (the window autosaves in a folder). edited = Signal() #: A result should open as a page of its own (a LogDocument, ModelDocument, CpnDocument). @@ -141,10 +145,15 @@ def __init__(self, document, library: Library, folder: str | Path | None = None, root.setContentsMargins(0, 0, 0, 0) root.setSpacing(0) self.header = PageHeader(self.workflow.name, self._subtitle()) + #: Reproducibility, as last computed (a run finishing, a save): see refresh_title. + self._repro = reproducibility.status(self.workflow, getattr(document, "record", None), None, self.folder) # Canvas: the boxes and wires. Summary: every result's figure, and the process map. - self.summary_switch = SegmentedControl(["Canvas", "Summary"], compact=True) + self.summary_switch = SegmentedControl(["Canvas", "Summary", "Runs"], compact=True) self.summary_switch.setToolTip("Canvas: the boxes and wires.\nSummary: every result's key " - "figure, and the process map of the first log.") + "figure, and the process map.\nRuns: every finished run, two compared.") + record = getattr(document, "record", None) + #: The last runs, with what changed before each (kept in the workflow file). + self.history = snapshots.History.from_dict(record.history if record is not None else []) self.summary_switch.changed.connect(self.show_view) self.header.actions.addWidget(self.summary_switch) self.keep_button = button("Keep", self.keep_requested.emit, kind="primary", @@ -165,9 +174,12 @@ def __init__(self, document, library: Library, folder: str | Path | None = None, self.more_menu.addAction("Export experiment…", self.export_experiment).setToolTip( "A zip with the workflow, its inputs, your boxes, every result and a README: " "supplementary material for a paper") + self.more_menu.addAction("Export report…", self.export_report).setToolTip( + "One HTML page: every box's result, how it got there, the code it ran, the papers it " + "follows, what it read, and how to cite. For a supervisor, a reviewer, a hand-in") self.more_menu.addSeparator() self.more_menu.addAction("Save", self.save).setToolTip("Save the workflow and its record") - self.more_button = button("⋯", self._show_more_menu, tooltip="Re-run, Record, Export experiment, Save") + self.more_button = button("⋯", self._show_more_menu, tooltip="Re-run, Record, Export experiment, Export report, Save") self.more_button.setObjectName("moreButton") self.more_button.setFixedWidth(40) self.header.actions.addWidget(self.more_button) @@ -220,9 +232,13 @@ def __init__(self, document, library: Library, folder: str | Path | None = None, self.summary.tile_clicked.connect(self._tile_clicked) summary_wrapper = QWidget() summary_wrapper.setLayout(vbox(scroll(self.summary), margins=(20, 0, 20, 16))) + self.runs = RunsWidget(self) + runs_wrapper = QWidget() + runs_wrapper.setLayout(vbox(scroll(self.runs), margins=(20, 0, 20, 16))) self.body = QStackedWidget() self.body.addWidget(wrapper) self.body.addWidget(summary_wrapper) + self.body.addWidget(runs_wrapper) root.addWidget(self.body, 1) self._picker: BoxPicker | None = None @@ -254,7 +270,7 @@ def close_panel(self) -> None: self.select(None) def show_view(self, index: int) -> None: - """Canvas (0) or Summary (1); the Summary is rebuilt from the run when shown.""" + """Canvas (0), Summary (1) or Runs (2); the latter two are rebuilt when shown.""" if self.body.currentIndex() != index: self.body.setCurrentIndex(index) motion.lift(self.body) @@ -264,6 +280,33 @@ def show_view(self, index: int) -> None: self.summary_switch.blockSignals(False) if index == 1: self.summary.refresh() + elif index == 2: + self.runs.refresh() + + def apply_snapshot(self, snapshot: snapshots.Snapshot) -> None: + """Put a run's settings back on the boxes that still exist, and run.""" + changed = [] + for node_id in list(self.workflow.nodes): + values = snapshots.settings_to_apply(snapshot, node_id) + if not values: + continue + before = dict(self.workflow.nodes[node_id].settings) + try: + self.workflow.set(node_id, **values) + except (ValueError, TypeError) as error: + self.status.emit(str(error)) + continue + if self.workflow.nodes[node_id].settings != before: + changed.append(node_id) + if not changed: + self.status.emit("Those are the settings already") + return + self._mark_edited() + self._render_panel() + self.show_view(0) + self.status.emit(f"Settings of the run at {snapshot.taken.replace('T', ' ')} are back on " + f"{len(changed)} box{'es' if len(changed) != 1 else ''}; running") + self.run_from(changed) def _tile_clicked(self, node_id: str) -> None: """A Summary tile: that box's Result, on the canvas.""" @@ -297,6 +340,18 @@ def _subtitle(self) -> str: def refresh_title(self) -> None: self.header.set_text(self.workflow.name, self._subtitle()) + status = self._repro + self.header.set_badge(status.headline, status.tone, + "\n".join([status.headline] + ["· " + d for d in status.details] + + ["", "Click for the record."]), + self.show_record) + + def refresh_reproducibility(self) -> None: + """Compare the record with the inputs on disk and the latest run (a run + finished, the analysis was saved).""" + self._repro = reproducibility.status(self.workflow, getattr(self.document, "record", None), + self.run, self.folder) + self.refresh_title() # -- adding boxes: the picker --------------------------------------------------------------- def picker(self) -> BoxPicker: @@ -508,8 +563,13 @@ def _run_finished(self, outcome) -> None: self.status.emit(outcome.summary()) else: self.status.emit(f"The run stopped: {outcome}") - self.refresh_title() + self.refresh_reproducibility() self._render_panel() + if isinstance(outcome, Run) and not outcome.stopped and not self._has_queue: + if self.history.add(snapshots.take(self.workflow, outcome, self.history.latest)) \ + and self.body.currentIndex() == 2: + self.runs.refresh() + self.ran.emit() if self.body.currentIndex() == 1: self.summary.refresh() if self._has_queue or (isinstance(outcome, Run) and outcome.stopped): @@ -816,7 +876,7 @@ def _tab_widget(self, node, spec, result: Result | None, index: int) -> QWidget: if index == 1: if result is None or result.status not in (DONE, FAILED): return label("Runs first, then shows how it got there.", "muted", wrap=True) - return how_widget(result.explanation, self) + return how_widget(result.explanation, self, result.value) if index == 2: return code_widget(spec, self) settings = SettingsWidget(spec, node.settings, self.folder) @@ -920,7 +980,11 @@ def show_record(self) -> None: file_text.setReadOnly(True) file_text.setFont(theme.mono_font(11)) from PySide6.QtWidgets import QApplication + status = self._repro + repro = label(f"{status.headline}. " + " ".join(status.details), "muted", wrap=True) + repro.setTextFormat(Qt.RichText) dialog.setLayout(vbox( + label("REPRODUCIBILITY", "sectionLabel"), repro, label("Everything needed to run this experiment again and get the same numbers. The Python and the " "canvas are two views of the same workflow.", "muted", wrap=True), label("AS PYTHON", "sectionLabel"), code, label("THE WORKFLOW FILE", "sectionLabel"), file_text, @@ -934,10 +998,11 @@ def write_to(self, path: str, quiet: bool = False) -> None: """Save the workflow and its record to ``path`` (the window's Keep / Save As).""" target = Path(path) self.workflow.name = target.stem if target.stem else self.workflow.name - self.document.record = records.save(self.workflow, target, self.run, self.folder or target.parent) + self.document.record = records.save(self.workflow, target, self.run, self.folder or target.parent, + history=self.history.to_dict()) self.document.path = str(target) self.document.dirty = False - self.refresh_title() + self.refresh_reproducibility() if not quiet: self.status.emit(f"Saved {target.name}") self.saved.emit() @@ -966,6 +1031,65 @@ def export(self) -> None: path += ".cpnflow" self.write_to(path) + def drawings(self) -> dict[str, str]: + """SVG per box whose result the canvas can draw: nets, maps, trees (for the report).""" + from ...flow.types import DFG, OCDFG, PetriNet, ProcessTree + from ..studio.graph_builders import dfg_specs, ocdfg_specs, petri_net_specs + from ..studio.graph_view import GraphView + out: dict[str, str] = {} + if self.run is None: + return out + for node in self.workflow.order(): + result = self.run.result(node) + if result is None or result.status != DONE: + continue + value = result.value + view = GraphView() + if isinstance(value, PetriNet): + nodes, edges = petri_net_specs(value, show_place_names=len(value.places) <= 40) + positions = None + ids = {n.id for n in nodes} + if node.layout and ids <= set(node.layout): + positions = {k: v for k, v in node.layout.items() if k in ids} + view.graph.populate(nodes, edges, positions) + elif isinstance(value, DFG): + nodes, edges = dfg_specs(value, "frequency") + view.graph.populate(nodes, edges, layer_gap=64) + elif isinstance(value, OCDFG): + nodes, edges, _colours = ocdfg_specs(value) + view.graph.populate(nodes, edges, layer_gap=64) + elif isinstance(value, ProcessTree): + from ..studio.derivation_view import tree_specs + nodes, edges, positions = tree_specs(value) + view.graph.populate(nodes, edges, positions, layer_gap=40) + else: + continue + try: + out[node.id] = view.svg_text() + except Exception: # noqa: BLE001 - a drawing that fails is left out, the report still comes + continue + return out + + def export_report(self) -> None: + """⋯ ▸ Export report…: the analysis as one HTML page.""" + from ...flow import report as reports + from ..studio.widgets import suggested_path + path, _ = QFileDialog.getSaveFileName(self, "Export report", suggested_path(f"{self.workflow.name}.html"), + "Web page (*.html)") + if not path: + return + try: + reports.write_report(path, workflow=self.workflow, run=self.run, folder=self.folder, + drawings=self.drawings(), record=getattr(self.document, "record", None), + status=self._repro) + except OSError as error: + QMessageBox.warning(self, "Could not export", str(error)) + return + self.status.emit(f"Exported the report to {Path(path).name}") + from PySide6.QtGui import QDesktopServices + from PySide6.QtCore import QUrl + QDesktopServices.openUrl(QUrl.fromLocalFile(path)) + def export_experiment(self) -> None: from ..studio.widgets import suggested_path path, _ = QFileDialog.getSaveFileName(self, "Export experiment", suggested_path(f"{self.workflow.name}.zip"), diff --git a/openprocess/gui/flow/picker.py b/openprocess/gui/flow/picker.py index 7d8eb88..198b084 100644 --- a/openprocess/gui/flow/picker.py +++ b/openprocess/gui/flow/picker.py @@ -29,7 +29,7 @@ #: How many of the folder's files the Input group lists before "… more". FILE_LIMIT = 6 #: The input box for each kind of file in the folder (see workspace.file_kind). -INPUT_BOXES = {"log": "open_log", "petri": "open_net", "cpn": "open_cpn", +INPUT_BOXES = {"log": "open_log", "ocel": "open_ocel", "petri": "open_net", "cpn": "open_cpn", "ts": "open_transition_system", "typed": "typed_log"} @@ -223,7 +223,7 @@ def _add_files(self, rows: QVBoxLayout) -> None: rows.addSpacing(6) rows.addWidget(label("IN THIS FOLDER", "sectionLabel")) names = [Path(relative).name for relative, _kind in files] - icons = {"log": "log", "cpn": "cpn", "ts": "ts", "petri": "model"} + icons = {"log": "log", "ocel": "log", "cpn": "cpn", "ts": "ts", "petri": "model"} for relative, kind in files: name = Path(relative).name text = display_name(name) diff --git a/openprocess/gui/flow/runs.py b/openprocess/gui/flow/runs.py new file mode 100644 index 0000000..6744e2e --- /dev/null +++ b/openprocess/gui/flow/runs.py @@ -0,0 +1,130 @@ +"""The *Runs* view of an analysis: its snapshots, two of them compared. + +Every finished run is kept (the last twenty) with what changed before it +and every box's key figure. Pick A and B and the boxes line up side by +side, the ones whose result differs marked; *Use these settings* puts a +run's settings back on the canvas, so a run can be returned to. +""" + +from __future__ import annotations + +from PySide6.QtCore import Qt +from PySide6.QtWidgets import (QAbstractItemView, QButtonGroup, QHeaderView, QRadioButton, QTableWidget, + QTableWidgetItem, QVBoxLayout, QWidget) + +from ...flow.snapshots import Snapshot, compare +from ..studio import style +from ..studio.widgets import Card, button, hbox, label + + +class RunsWidget(QWidget): + """Snapshots on top, the comparison of A and B under them; :meth:`refresh` rebuilds both.""" + + def __init__(self, page, parent=None) -> None: + super().__init__(parent) + self.page = page + layout = QVBoxLayout(self) + layout.setContentsMargins(0, 0, 0, 0) + layout.setSpacing(14) + layout.addWidget(label( + "Every finished run is kept here (the last twenty), with what changed before it and every box's " + "key figure. Pick two, A and B, to see them side by side; Use these settings goes back to a run.", + "muted", wrap=True)) + self.table = QTableWidget(0, 5) + self.table.setHorizontalHeaderLabels(["A", "B", "When", "What changed", "Boxes"]) + self.table.verticalHeader().setVisible(False) + self.table.setEditTriggers(QAbstractItemView.NoEditTriggers) + self.table.setSelectionMode(QAbstractItemView.NoSelection) + self.table.setShowGrid(False) + header = self.table.horizontalHeader() + for column in (0, 1, 2, 4): + header.setSectionResizeMode(column, QHeaderView.ResizeToContents) + header.setSectionResizeMode(3, QHeaderView.Stretch) + self.table.setMinimumHeight(140) + layout.addWidget(self.table) + self.group_a = QButtonGroup(self) + self.group_b = QButtonGroup(self) + self.group_a.idClicked.connect(lambda _i: self._compare()) + self.group_b.idClicked.connect(lambda _i: self._compare()) + self.comparison = Card("A and B, box by box", "Run the analysis: each finished run appears above.") + self.diff = QTableWidget(0, 4) + self.diff.setHorizontalHeaderLabels(["Box", "A", "B", "Changed"]) + self.diff.verticalHeader().setVisible(False) + self.diff.setEditTriggers(QAbstractItemView.NoEditTriggers) + self.diff.setSelectionMode(QAbstractItemView.NoSelection) + self.diff.setShowGrid(False) + diff_header = self.diff.horizontalHeader() + for column in (0, 1, 2): + diff_header.setSectionResizeMode(column, QHeaderView.ResizeToContents) + diff_header.setSectionResizeMode(3, QHeaderView.Stretch) + self.diff.setMinimumHeight(120) + self.comparison.add(self.diff) + self.use_a = button("Use A's settings", lambda: self._use(self.a)) + self.use_b = button("Use B's settings", lambda: self._use(self.b)) + self.comparison.add(hbox(self.use_a, self.use_b, None)) + layout.addWidget(self.comparison) + layout.addStretch(1) + self.snapshots: list[Snapshot] = [] + self.a: Snapshot | None = None + self.b: Snapshot | None = None + + def refresh(self) -> None: + """Rebuild from the page's history: newest first, A the run before the latest, B the latest.""" + for group in (self.group_a, self.group_b): + for radio in group.buttons(): + group.removeButton(radio) + self.snapshots = list(reversed(self.page.history.snapshots)) + self.table.setRowCount(len(self.snapshots)) + for row, snapshot in enumerate(self.snapshots): + for column, group in ((0, self.group_a), (1, self.group_b)): + radio = QRadioButton() + radio.setToolTip("A: the run to compare from" if column == 0 else "B: the run to compare to") + group.addButton(radio, row) + holder = QWidget() + holder.setLayout(hbox(radio, margins=(10, 0, 4, 0))) + self.table.setCellWidget(row, column, holder) + when = snapshot.taken.replace("T", " ") + self.table.setItem(row, 2, QTableWidgetItem(when)) + self.table.setItem(row, 3, QTableWidgetItem(snapshot.label)) + self.table.setItem(row, 4, QTableWidgetItem(f"{snapshot.done} of {len(snapshot.statuses)} done")) + if self.snapshots: + self.group_b.button(0).setChecked(True) + self.group_a.button(1 if len(self.snapshots) > 1 else 0).setChecked(True) + self._compare() + + def _picked(self, group: QButtonGroup) -> Snapshot | None: + index = group.checkedId() + return self.snapshots[index] if 0 <= index < len(self.snapshots) else None + + def _compare(self) -> None: + self.a, self.b = self._picked(self.group_a), self._picked(self.group_b) + self.diff.setRowCount(0) + if self.a is None or self.b is None: + self.comparison.caption_label.setText("Run the analysis: each finished run appears above.") + self.use_a.setEnabled(False) + self.use_b.setEnabled(False) + return + rows = compare(self.a, self.b) + changed = sum(1 for r in rows if r.changed) + self.comparison.caption_label.setText( + f"A: {self.a.taken.replace('T', ' ')} ({self.a.label}). B: {self.b.taken.replace('T', ' ')} " + f"({self.b.label}). {changed} of {len(rows)} boxes give a different result.") + self.diff.setRowCount(len(rows)) + for index, row in enumerate(rows): + self.diff.setItem(index, 0, QTableWidgetItem(row.title)) + self.diff.setItem(index, 1, QTableWidgetItem(row.a or "–")) + self.diff.setItem(index, 2, QTableWidgetItem(row.b or "–")) + what = ("; ".join(row.changes) if row.changes else ("yes" if row.changed else "no")) + item = QTableWidgetItem(what) + if row.changed: + item.setForeground(style.qc(style.STATUS["warning"])) + self.diff.setItem(index, 3, item) + self.use_a.setEnabled(True) + self.use_b.setEnabled(True) + + def _use(self, snapshot: Snapshot | None) -> None: + if snapshot is not None: + self.page.apply_snapshot(snapshot) + + +__all__ = ["RunsWidget"] diff --git a/openprocess/gui/flow/stepper.py b/openprocess/gui/flow/stepper.py new file mode 100644 index 0000000..afec8c2 --- /dev/null +++ b/openprocess/gui/flow/stepper.py @@ -0,0 +1,240 @@ +"""Step through an algorithm: the How tab, one moment at a time. + +The How tab lists everything a box reported: notes, the derivation's steps, +intermediate values. For learning, seeing it all at once is too much and +too little: too much text, and nothing moves. :class:`StepThrough` plays +the same report as a sequence of *moments*, one step row or one note or one +shown value each, with ◀ ▶ and a Play button, and keeps the box's result (a +net or a process tree) drawn above, lighting up the places, transitions or +activities the current step names and dimming the rest. So the eight steps +of the α-algorithm become eight pictures: all activities, the start ones, the +end ones, the candidate pairs, the maximal pairs, the places… + +Nothing is computed here. The moments are the box's own report, in order. +""" + +from __future__ import annotations + +import re + +from PySide6.QtCore import Qt, QTimer, Signal +from PySide6.QtWidgets import QLabel, QSizePolicy, QVBoxLayout, QWidget + +from ...flow.explain import Explanation +from ...flow.types import PetriNet, ProcessTree +from .. import theme +from ..studio import style +from ..studio.graph_builders import petri_net_specs +from ..studio.graph_view import GraphView, NodeSpec +from ..studio.widgets import button, hbox, label + +#: How long Play rests on each moment. +PLAY_MS = 1400 + + +def moments_of(explanation: Explanation) -> list[tuple]: + """The report as a flat sequence: ("note", text), ("step", title, content), ("show", label, value).""" + out: list[tuple] = [] + for entry in explanation.entries: + if entry[0] == "note": + out.append(("note", entry[1])) + elif entry[0] == "steps": + for title, content in entry[1]: + out.append(("step", title, content)) + elif entry[0] == "show": + out.append(("show", entry[1], entry[2])) + return out + + +_PLACE = re.compile(r"[io]_L|p\(\{[^()]*\},\{[^()]*\}\)") +_WORD = re.compile(r"[^\W\d_][\w\-']*", re.UNICODE) + + +def named_in(moment: tuple) -> set[str]: + """The element names a moment mentions: words (activities, transitions) + and α-style place names such as ``p({a},{b})``.""" + if moment[0] == "step": + text = f"{moment[1]} {moment[2]}" + elif moment[0] == "note": + text = moment[1] + else: + text = str(moment[1]) + names = set(_WORD.findall(text)) + names |= set(_PLACE.findall(text.replace(" ", ""))) + return names + + +class StepThrough(QWidget): + """The How tab as a player; :attr:`changed` carries the current moment's index.""" + + changed = Signal(int) + + def __init__(self, explanation: Explanation, value=None, page=None, parent=None) -> None: + super().__init__(parent) + self.moments = moments_of(explanation) + self.value = value + self.page = page + self.index = 0 + self.setFocusPolicy(Qt.StrongFocus) + layout = QVBoxLayout(self) + layout.setContentsMargins(0, 0, 0, 0) + layout.setSpacing(10) + # The controls. + self.back_button = button("◀", self.previous, tooltip="The step before (←)") + self.next_button = button("▶", self.next, tooltip="The next step (→)") + self.play_button = button("Play", self.toggle_play, tooltip="One step every second and a half") + self.counter = label("", "muted") + self.title = QLabel("") + self.title.setObjectName("cardTitle") + self.title.setWordWrap(True) + layout.addLayout(hbox(self.back_button, self.next_button, self.play_button, 10, self.counter, None)) + layout.addWidget(self.title) + # The result, drawn once; its nodes light up per step. + self.view: GraphView | None = None + self._base: dict[str, NodeSpec] = {} + self._names: dict[str, str] = {} # node id -> the name a step would use + if isinstance(value, PetriNet): + nodes, edges = petri_net_specs(value, show_place_names=len(value.places) <= 40) + positions = None + if all(p.position for p in value.places.values()) and all(t.position for t in value.transitions.values()): + positions = {p.id: p.position for p in value.places.values()} + positions |= {t.id: t.position for t in value.transitions.values()} + self._names = {p.id: p.name for p in value.places.values()} + self._names |= {t.id: t.name for t in value.transitions.values()} + self._draw(nodes, edges, positions) + elif isinstance(value, ProcessTree): + from ..studio.derivation_view import tree_specs + nodes, edges, positions = tree_specs(value) + self._names = {n.id: n.text for n in nodes} + self._draw(nodes, edges, positions, layer_gap=40) + # The moments shown so far. + self.body = QWidget() + self.body.setObjectName("plain") + self.body_layout = QVBoxLayout(self.body) + self.body_layout.setContentsMargins(0, 0, 0, 0) + self.body_layout.setSpacing(8) + layout.addWidget(self.body) + layout.addStretch(1) + self.timer = QTimer(self) + self.timer.setInterval(PLAY_MS) + self.timer.timeout.connect(self._tick) + self.show_moment(0) + + # -- drawing ----------------------------------------------------------------------------- + def _draw(self, nodes, edges, positions, layer_gap: float = 48.0) -> None: + self.view = GraphView() + self.view.setMinimumHeight(220) + self.view.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Fixed) + self.view.graph.populate(nodes, edges, positions, layer_gap=layer_gap) + self._base = {n.id: n for n in nodes} + self.layout().addWidget(self.view) + QTimer.singleShot(0, self.view.fit) + + def highlighted(self, index: int) -> set[str]: + """The ids of the drawn nodes the moment at ``index`` names.""" + if not self._names or not (0 <= index < len(self.moments)): + return set() + names = named_in(self.moments[index]) + compact = {name.replace(" ", "") for name in names} + return {node_id for node_id, name in self._names.items() + if name and (name in names or name.replace(" ", "") in compact)} + + def _light(self, index: int) -> None: + if self.view is None: + return + t = style.tokens() + lit = self.highlighted(index) + for node_id, base in self._base.items(): + spec = NodeSpec(**{**base.__dict__}) + if lit: + if node_id in lit: + spec.fill = t.accent_soft + spec.stroke = t.accent + spec.emphasis = True + else: + spec.dimmed = True + self.view.graph.update_node(spec) + + # -- stepping ---------------------------------------------------------------------------- + def show_moment(self, index: int) -> None: + """Show the moments up to ``index`` and light up what the last one names.""" + count = len(self.moments) + index = max(0, min(index, count - 1)) if count else 0 + self.index = index + while self.body_layout.count(): + item = self.body_layout.takeAt(0) + if item.widget() is not None: + item.widget().hide() + item.widget().deleteLater() + t = style.tokens() + for position, moment in enumerate(self.moments[:index + 1]): + current = position == index + self.body_layout.addWidget(self._render(moment, current, t)) + moment = self.moments[index] if count else None + self.counter.setText(f"Step {index + 1} of {count}" if count else "Nothing to step through") + self.title.setText(moment[1] if moment and moment[0] == "step" else + ("Note" if moment and moment[0] == "note" else (moment[1] if moment else ""))) + self.back_button.setEnabled(index > 0) + self.next_button.setEnabled(index < count - 1) + self._light(index) + self.changed.emit(index) + + def _render(self, moment: tuple, current: bool, t) -> QWidget: + from .viewers import result_widget + if moment[0] == "note": + text = label("· " + moment[1], None if current else "muted", wrap=True, selectable=True) + return text + if moment[0] == "step": + title, content = moment[1], moment[2] + colour = t.text if current else t.text_muted + html = (f"" + f"
{title}{content}
") + row = QLabel(html) + row.setTextFormat(Qt.RichText) + row.setWordWrap(True) + row.setTextInteractionFlags(Qt.TextSelectableByMouse) + return row + host = QWidget() + host.setObjectName("plain") + column = QVBoxLayout(host) + column.setContentsMargins(0, 0, 0, 0) + if moment[1]: + column.addWidget(label(str(moment[1]).upper(), "sectionLabel")) + column.addWidget(result_widget(moment[2], self.page)) + return host + + def next(self) -> None: + self.show_moment(self.index + 1) + + def previous(self) -> None: + self.show_moment(self.index - 1) + + def toggle_play(self) -> None: + if self.timer.isActive(): + self.timer.stop() + self.play_button.setText("Play") + return + if self.index >= len(self.moments) - 1: + self.show_moment(0) + self.play_button.setText("Pause") + self.timer.start() + + def _tick(self) -> None: + if self.index >= len(self.moments) - 1: + self.timer.stop() + self.play_button.setText("Play") + return + self.next() + + def keyPressEvent(self, event) -> None: # noqa: N802 + if event.key() in (Qt.Key_Right, Qt.Key_Space): + self.next() + elif event.key() == Qt.Key_Left: + self.previous() + else: + super().keyPressEvent(event) + + +__all__ = ["StepThrough", "moments_of", "named_in", "PLAY_MS"] diff --git a/openprocess/gui/flow/summary.py b/openprocess/gui/flow/summary.py index 252e97f..b3cd46a 100644 --- a/openprocess/gui/flow/summary.py +++ b/openprocess/gui/flow/summary.py @@ -14,7 +14,8 @@ from PySide6.QtCore import Qt, QTimer, Signal from PySide6.QtWidgets import QFrame, QLabel, QSlider, QSizePolicy, QVBoxLayout, QWidget -from ...flow.types import DFG, EventLog, Figure, PetriNet, Scores, Table, Text +from ...flow.figures import tile_parts +from ...flow.types import DFG, EventLog from ..studio import style from ..studio.graph_builders import dfg_specs from ..studio.graph_view import GraphView @@ -24,42 +25,6 @@ TILE_LIMIT = 24 -def _number(value) -> str: - if isinstance(value, bool): - return "yes" if value else "no" - if isinstance(value, float): - return f"{value:.2f}" if abs(value) < 1000 else f"{value:,.0f}" - if isinstance(value, int): - return f"{value:,}" - return str(value) - - -def tile_parts(value) -> tuple[str, str] | None: - """(the big figure, the line under it) for a result, or None for a result - that has no figure to show (a trace, a prediction model, …).""" - if isinstance(value, Scores): - items = [(k, v) for k, v in value.metrics.items() if not isinstance(v, (dict, list))] - if not items: - return None - first, rest = items[0], items[1:4] - return (f"{_number(first[1])} {first[0]}".strip(), - " · ".join(f"{k} {_number(v)}" for k, v in rest) or value.model) - if isinstance(value, Table): - return f"{len(value.rows):,} × {len(value.columns)}", f"rows × columns · {value.name}" - if isinstance(value, Figure): - return "figure", value.caption or value.name - if isinstance(value, EventLog): - return f"{len(value):,} cases", f"{value.event_count:,} events" - if isinstance(value, PetriNet): - return f"{len(value.places)} · {len(value.transitions)}", "places · transitions" - if isinstance(value, DFG): - return f"{len(value.activities)} · {len(value.all_edges())}", "activities · paths" - if isinstance(value, Text): - first = str(value).strip().splitlines()[0] if str(value).strip() else "" - return first[:40], value.name - return None - - class SummaryTile(QFrame): """One box's figure: the box (group and name), the figure, a line under it. Clickable.""" diff --git a/openprocess/gui/flow/viewers.py b/openprocess/gui/flow/viewers.py index c8d461c..f08420d 100644 --- a/openprocess/gui/flow/viewers.py +++ b/openprocess/gui/flow/viewers.py @@ -33,7 +33,7 @@ from ...flow.explain import Explanation from ...flow.runner import Result from ...flow.sweep import Sweep, is_sweep -from ...flow.types import (AlignmentResult, CPNet, DFG, Dataset, EventLog, Figure, Footprint, PetriNet, +from ...flow.types import (OCDFG, OCEL, AlignmentResult, CPNet, DFG, Dataset, EventLog, Figure, Footprint, PetriNet, Predictions, Predictor, ProcessTree, Regions, ReplayResult, Scores, SimpleLog, Table, Text, TransitionSystem) from .. import theme @@ -229,6 +229,32 @@ def tidied(_id: str, _x: float, _y: float, graph=view.graph, n=node) -> None: formula = label(tree_formula(value), wrap=True) formula.setTextFormat(Qt.RichText) add(formula) + elif isinstance(value, OCEL): + counts = value.counts_by_type() + tiles = hbox(StatTile("Events", f"{len(value):,}"), StatTile("Activities", f"{len(value.activities):,}"), + StatTile("Objects", f"{len(value.objects):,}"), + StatTile("Object types", f"{len(value.object_types)}"), spacing=8) + tiles_host = QWidget() + tiles_host.setLayout(tiles) + add(tiles_host) + add(label("An object-centric log: every event names the objects it involves, by type. " + "Flatten on an object type for a classical log, or draw the object-centric map.", + "muted", wrap=True)) + add(_table(["object type", "objects"], [[t, n] for t, n in counts.items()])) + add(_table(["activity", "events"], [[a, n] for a, n in value.activities.most_common()])) + elif isinstance(value, OCDFG): + from ..studio.graph_builders import ocdfg_specs + from ..studio.widgets import Legend, LegendSwatch + nodes, edges, colours = ocdfg_specs(value) + add(_graph(nodes, edges, layer_gap=64, height=300)) + legend = Legend() + legend.set_items([(LegendSwatch("line", colour, 2.5), f"{object_type}: its own directly-follows " + f"counts") for object_type, colour in colours.items()]) + add(legend) + add(label(value.summary() + ". Each object type is followed through its own events, so no case id " + "is invented.", "muted", wrap=True)) + rows = [[t, a, b, n] for t in value.object_types for (a, b), n in value.edges[t].most_common()] + add(_table(["object type", "from", "to", "objects"], rows)) elif isinstance(value, DFG): nodes, edges = dfg_specs(value) add(_graph(nodes, edges, layer_gap=90)) @@ -345,7 +371,10 @@ def tidied(_id: str, _x: float, _y: float, graph=view.graph, n=node) -> None: # --------------------------------------------------------------------------- # How # --------------------------------------------------------------------------- -def how_widget(explanation: Explanation, page=None) -> QWidget: +def how_widget(explanation: Explanation, page=None, value=None) -> QWidget: + """The box's report in full, with *Step through* to play it one moment at + a time over a drawing of ``value`` (the box's result).""" + from .stepper import StepThrough, moments_of host = QWidget() layout = QVBoxLayout(host) layout.setContentsMargins(0, 0, 0, 0) @@ -354,13 +383,43 @@ def how_widget(explanation: Explanation, page=None) -> QWidget: layout.addWidget(label("This box has nothing to add: its result is all there is.", "muted", wrap=True)) return host t = style.tokens() + full = QWidget() + full.setObjectName("plain") + full_layout = QVBoxLayout(full) + full_layout.setContentsMargins(0, 0, 0, 0) + full_layout.setSpacing(10) + if len(moments_of(explanation)) >= 2: + stepper: list = [] + + def toggle(_checked=False) -> None: + if stepper: + widget = stepper.pop() + widget.hide() + widget.setParent(None) + widget.deleteLater() + full.setVisible(True) + step_button.setText("Step through ▸") + return + widget = StepThrough(explanation, value, page) + stepper.append(widget) + layout.insertWidget(1, widget) + full.setVisible(False) + step_button.setText("◂ All at once") + widget.setFocus() + step_button = button("Step through ▸", toggle, + tooltip="Play the derivation one step at a time, with the result lighting up " + "what each step names") + step_button.setObjectName("stepThrough") + layout.addLayout(hbox(label(f"{len(moments_of(explanation))} moments: notes, steps and values, in the " + "order the box reported them.", "muted", wrap=True), step_button)) + layout.addWidget(full) for entry in explanation.entries: if entry[0] == "note": text = label("· " + entry[1], "muted" if not entry[1].startswith("Warning") else None, wrap=True, selectable=True) if entry[1].startswith("Warning"): text.setStyleSheet(f"color: {style.STATUS['warning']};") - layout.addWidget(text) + full_layout.addWidget(text) elif entry[0] == "steps": rows = [] for title, content in entry[1]: @@ -372,12 +431,12 @@ def how_widget(explanation: Explanation, page=None) -> QWidget: table.setTextFormat(Qt.RichText) table.setWordWrap(True) table.setTextInteractionFlags(Qt.TextSelectableByMouse) - layout.addWidget(table) + full_layout.addWidget(table) elif entry[0] == "show": - caption, value = entry[1], entry[2] + caption = entry[1] if caption: - layout.addWidget(label(caption.upper(), "sectionLabel")) - layout.addWidget(result_widget(value, page)) + full_layout.addWidget(label(caption.upper(), "sectionLabel")) + full_layout.addWidget(result_widget(entry[2], page)) return host @@ -553,7 +612,10 @@ def code_widget(spec: BoxSpec, page=None, compact: bool = True) -> QWidget: whole = button("Whole file ⤢", lambda _checked=False, c=called: show_file(c, host.window()), tooltip="The whole algorithm: the file this function lives in, in a window of its own") whole.setObjectName("algorithmFile") - algorithm_card.add(hbox(title, None, whole)) + cite = button("Cite", lambda _checked=False, c=called: _cite(c, host.window()), + tooltip="BibTeX for the paper(s) this algorithm follows") + cite.setObjectName("algorithmCite") + algorithm_card.add(hbox(title, None, cite, whole)) follows = _follows(called) if follows: algorithm_card.add(follows) @@ -565,6 +627,11 @@ def code_widget(spec: BoxSpec, page=None, compact: bool = True) -> QWidget: return host +def _cite(called: Called, parent=None): + from ..studio.cite import cite_module + return cite_module(called.module, called.name, parent) + + def _extent(called: Called) -> str: """"alpha_miner is 24 of alpha.py's 310 lines; the other 286 are the algorithm's helpers" — so it is plain that the snippet is one function @@ -771,7 +838,7 @@ def _kinds(self) -> tuple[str, ...]: and so on (nothing for a box author to declare). Empty: any kind.""" kinds = [] for port in self.spec.outputs: - for cls, kind in ((EventLog, "log"), (PetriNet, "petri"), (CPNet, "cpn"), + for cls, kind in ((EventLog, "log"), (OCEL, "ocel"), (PetriNet, "petri"), (CPNet, "cpn"), (TransitionSystem, "ts")): if port.type is cls: kinds.append(kind) @@ -798,6 +865,7 @@ def _files(self) -> list[str]: def _dialog_filter(self) -> str: """The Choose… dialog's file types, matching :meth:`_files`.""" by_kind = {"log": "Event logs (*.xes *.xes.gz *.gz *.csv *.txt)", + "ocel": "Object-centric logs (*.jsonocel *.json)", "petri": "Petri nets (*.pnml)", "cpn": "Coloured Petri nets (*.cpn)", "ts": "Transition systems (*.txt)"} kinds = self._kinds() or tuple(by_kind) diff --git a/openprocess/gui/studio/app.py b/openprocess/gui/studio/app.py index c91cd29..c5c0b2f 100644 --- a/openprocess/gui/studio/app.py +++ b/openprocess/gui/studio/app.py @@ -56,7 +56,7 @@ from pathlib import Path from PySide6.QtCore import ( - QEvent, QFile, QFileSystemWatcher, QRect, QRectF, QSettings, QSize, Qt, QTimer, QUrl, Signal, + QEvent, QFile, QFileSystemWatcher, QPoint, QRect, QRectF, QSettings, QSize, Qt, QTimer, QUrl, Signal, ) from PySide6.QtGui import ( QAction, QColor, QDesktopServices, QIcon, QKeySequence, QPainter, QPen, QPixmap, QShortcut, @@ -916,6 +916,73 @@ def done(result) -> None: run_in_background(lambda: PIP_RUNNER(command), done, lambda message: done((1, str(message)))) + # -- the command palette: ⌘K ------------------------------------------------------------ + def palette_entries(self) -> list: + """Everything the palette can run: documents, the folder's files, boxes + for the current analysis, the spaces and Connections, every menu action.""" + from .command_palette import Entry + from .documents import TransitionSystemDocument + entries: list[Entry] = [] + kinds = {WorkflowDocument: "Analysis", CpnDocument: "Model", ModelDocument: "Model", LogDocument: "Log", + ComparisonDocument: "Comparison", TransitionSystemDocument: "Transition system"} + for document in self.documents: + kind = next((k for cls, k in kinds.items() if isinstance(document, cls)), "Document") + where = Path(document.path).name if document.path else "not saved" + entries.append(Entry(kind, document.name, where, lambda d=document: self._select_document(d), + keywords="open")) + if self._tree is not None: + open_paths = {self._key(d.path) for d in self.documents if d.path} + for folder in self._tree.walk(): + for file in folder.files: + if self._key(file.path) in open_paths: + continue + entries.append(Entry("File", file.name, file.relative, lambda p=str(file.path): self.open_path(p), + keywords=file.kind, rank=1)) + page = self.current_page() + library = self.library() + for group, specs in library.by_group().items(): + for spec in specs: + if isinstance(page, WorkflowPage): + run = lambda s=spec, pg=page: pg.add_box_in_view(s.id) + detail = f"{group} · add to {page.workflow.name}" + else: + run = lambda s=spec: self.analysis_for("Untitled").add_box_in_view(s.id) + detail = f"{group} · add to a new analysis" + entries.append(Entry("Box", spec.name, detail, run, keywords=f"{spec.id} {group}", rank=2)) + for space in SPACES: + entries.append(Entry("Space", LABELS[space], {MINE: "event logs and analyses", MODEL: "Petri nets and " + "coloured nets", LEARN: "exercises"}[space], lambda sp=space: self.set_space(sp))) + entries.append(Entry("Space", "Connections", "what flows in and out, and the tools that plug in", + self.show_connections)) + for top in self.menuBar().actions(): + menu = top.menu() + if menu is None: + continue + path = top.text().replace("&", "") + stack = [(menu, path)] + while stack: + current, trail = stack.pop(0) + for action in current.actions(): + if action.isSeparator() or not action.text(): + continue + if action.menu() is not None: + stack.append((action.menu(), f"{trail} ▸ {action.text().replace('&', '')}")) + continue + shortcut = action.shortcut().toString() + entries.append(Entry("Action", action.text().replace("&", ""), + trail + (f" · {shortcut}" if shortcut else ""), + lambda a=action: a.trigger(), rank=3)) + return entries + + def show_command_palette(self) -> None: + """⌘K: find and run anything by typing.""" + from .command_palette import CommandPalette + if self.in_learn: + self.leave_learn() + self.command_palette = CommandPalette(self.palette_entries(), self) + centre = self.mapToGlobal(QPoint(self.width() // 2 - 280, 120)) + self.command_palette.open_at(centre) + def show_connections(self) -> None: """The Connections view: what flows in and out, and the tools that plug in.""" if self.in_learn: @@ -1284,6 +1351,7 @@ def _build_menus(self) -> None: self._action(view_menu, LABELS[space], f"Ctrl+Alt+{index + 1}", lambda _on=False, sp=space: self.set_space(sp)) self._action(view_menu, "Connections", None, self.show_connections) + self._action(view_menu, "Command Palette…", "Ctrl+K", self.show_command_palette) view_menu.addSeparator() self._action(view_menu, "Show Welcome Page", "Ctrl+1", lambda: (self.in_learn and self.leave_learn(), @@ -1325,6 +1393,7 @@ def _build_menus(self) -> None: self._action(help_menu, "References", None, lambda: self.show_guide("references")) self._action(help_menu, "Writing Exercise Packs", None, lambda: self.show_guide("exercise-packs")) + self._action(help_menu, "Cite OpenProcess…", None, self.cite_app) self._action(help_menu, "Check for Updates…", None, lambda: self.check_for_updates(manual=True)) self._action(help_menu, f"About {APPLICATION_NAME}", None, self._about) @@ -1347,6 +1416,11 @@ def _action(self, menu, text, shortcut, slot) -> QAction: menu.addAction(action) return action + def cite_app(self) -> None: + """Help ▸ Cite OpenProcess…: the app's BibTeX and a one-line citation.""" + from .cite import cite_app + self.cite_dialog = cite_app(self) + def _definitions(self) -> None: """Every analysis property, defined mathematically (docs/definitions.md).""" from .definition_view import show_reference @@ -1442,6 +1516,7 @@ def _make_page(self, document) -> QWidget: page.edit_requested.connect(lambda net, origin, doc=document: self.edit_petri_net(net, origin, doc)) page.keep_requested.connect(lambda doc=document: self.keep_workflow(doc)) page.edited.connect(lambda doc=document: (self._refresh_item(doc), self._schedule_autosave(doc))) + page.ran.connect(lambda doc=document: self._schedule_autosave(doc)) # the record follows the run if self.workspace is not None: page.custom_allowed = bool(self.workspace.settings().get("boxes_allowed")) page._refresh_custom_bar() @@ -2318,6 +2393,9 @@ def open_path(self, path: str, csv_mapping: dict | None = None) -> None: if problems else ""), 10000) elif lower.endswith(".txt") and file_kind(Path(path)) in ("log", "ts"): self.add_document(self._read_text_file(path)) + elif file_kind(Path(path)) == "ocel": + # An object-centric log has no page of its own: it is a box on an analysis. + self.boxes_for_files([path]) elif lower.endswith(".csv"): if csv_mapping is not None: # reopening: reuse the saved mapping mapping = ColumnMapping(**csv_mapping) @@ -3136,7 +3214,7 @@ def _document_row(self, document) -> QTreeWidgetItem: def _file_row(self, file, text: str) -> QTreeWidgetItem: """A file of the folder that is not open: lighter, opens with a click.""" - kind = {"log": "log", "cpn": "cpn", "ts": "ts", "workflow": "workflow"}.get(file.kind, "model") + kind = {"log": "log", "ocel": "log", "cpn": "cpn", "ts": "ts", "workflow": "workflow"}.get(file.kind, "model") key = self._key(file.path) if file.in_cloud: text += " ☁" @@ -3567,9 +3645,10 @@ def _autosave(self, document) -> None: def _flush_autosaves(self, ids: list[int] | None = None) -> None: """Save now what autosave would save in a moment (switching, closing, quitting).""" for document in list(self.documents): + timer = self._autosave_timers.get(document.id) + pending = timer is not None and timer.isActive() # a run waiting to be recorded if (ids is None or document.id in ids) and getattr(document, "autosave", False) \ - and document.dirty: - timer = self._autosave_timers.get(document.id) + and (document.dirty or pending): if timer is not None: timer.stop() self._autosave(document) diff --git a/openprocess/gui/studio/cite.py b/openprocess/gui/studio/cite.py new file mode 100644 index 0000000..426346b --- /dev/null +++ b/openprocess/gui/studio/cite.py @@ -0,0 +1,58 @@ +"""*Cite*: BibTeX for the app and for the works an algorithm follows, to copy.""" + +from __future__ import annotations + +from PySide6.QtWidgets import QApplication, QDialog, QPlainTextEdit + +from ... import citation +from .. import theme +from .widgets import button, hbox, label, vbox + + +def cite_dialog(title: str, intro: str, entries: list[citation.Entry], parent=None, + text_line: str = "") -> QDialog: + """A window with the BibTeX of ``entries`` (and a one-line citation when + given), each with a Copy button.""" + dialog = QDialog(parent) + dialog.setWindowTitle(title) + dialog.resize(720, 560) + bibtex = "\n\n".join(entry.bibtex() for entry in entries) + code = QPlainTextEdit(bibtex) + code.setReadOnly(True) + code.setFont(theme.mono_font(11)) + widgets = [label(intro, "muted", wrap=True)] + if text_line: + line = QPlainTextEdit(text_line) + line.setReadOnly(True) + line.setMaximumHeight(64) + widgets += [label("IN A SENTENCE", "sectionLabel"), line] + widgets += [label("BIBTEX", "sectionLabel"), code] + buttons = [button("Copy BibTeX", lambda: QApplication.clipboard().setText(bibtex), kind="primary")] + if text_line: + buttons.append(button("Copy the sentence", lambda: QApplication.clipboard().setText(text_line))) + widgets.append(hbox(*buttons, None, button("Close", dialog.close))) + dialog.setLayout(vbox(*widgets, margins=(16, 16, 16, 16))) + dialog.bibtex = bibtex + dialog.show() + return dialog + + +def cite_app(parent=None) -> QDialog: + """Help ▸ Cite OpenProcess…""" + return cite_dialog( + "Cite OpenProcess", + "If OpenProcess helped with a course, a thesis or a paper, cite it like this. Each algorithm's Code " + "tab has a Cite button for the paper it follows.", + [citation.app_entry()], parent, citation.app_text()) + + +def cite_module(module: str, name: str, parent=None) -> QDialog: + """The Code tab's Cite: the works the function's module follows.""" + entries = citation.entries_for_module(module) + intro = (f"The works that {name} follows, as BibTeX." if entries else + f"{name} has no reference on record: see Help ▸ References for the list, and cite OpenProcess " + "itself for the implementation.") + return cite_dialog(f"Cite: {name}", intro, entries or [citation.app_entry()], parent) + + +__all__ = ["cite_dialog", "cite_app", "cite_module"] diff --git a/openprocess/gui/studio/command_palette.py b/openprocess/gui/studio/command_palette.py new file mode 100644 index 0000000..2c0f573 --- /dev/null +++ b/openprocess/gui/studio/command_palette.py @@ -0,0 +1,162 @@ +"""The command palette: one key, type, Enter. + +⌘K (Ctrl+K) opens a small window with a search field. Typing finds, all at +once, the open analyses, models and logs, the folder's files, every box (to +add to the current analysis), the three spaces and Connections, and every +menu action by its name. Enter runs the first match; ↑ ↓ pick another. It +fits the calm design better than more menus: nothing is added to the +window, and anything in it is a few keystrokes away. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Callable + +from PySide6.QtCore import QPoint, Qt, Signal +from PySide6.QtWidgets import (QFrame, QGraphicsDropShadowEffect, QLineEdit, QListWidget, QListWidgetItem, + QVBoxLayout, QWidget) +from PySide6.QtGui import QColor + +from .. import theme +from . import motion, style +from .widgets import label, vbox + +SHADOW_MARGIN = 24 +LIMIT = 40 +#: The order kinds come in when nothing is typed, and their captions. +KINDS = ("Analysis", "Model", "Log", "Comparison", "Transition system", "File", "Space", "Box", "Action") + + +@dataclass +class Entry: + kind: str + title: str + detail: str = "" + run: Callable[[], None] = lambda: None + keywords: str = "" + #: Lower is earlier when nothing is typed. + rank: int = field(default=0) + + @property + def haystack(self) -> str: + return f"{self.title} {self.detail} {self.keywords} {self.kind}".lower() + + +def match(entries: list[Entry], query: str, limit: int = LIMIT) -> list[Entry]: + """The entries every word of ``query`` is in, best first: a title that + starts with the query, then a title that contains it, then the rest.""" + words = query.lower().split() + if not words: + return sorted(entries, key=lambda e: (KINDS.index(e.kind) if e.kind in KINDS else len(KINDS), e.rank, + e.title.lower()))[:limit] + scored = [] + for entry in entries: + hay = entry.haystack + if not all(word in hay for word in words): + continue + title = entry.title.lower() + score = 0 if title.startswith(query.lower()) else 1 if query.lower() in title else \ + 2 if all(word in title for word in words) else 3 + scored.append((score, KINDS.index(entry.kind) if entry.kind in KINDS else len(KINDS), entry.rank, + title, entry)) + scored.sort(key=lambda item: item[:4]) + return [item[4] for item in scored[:limit]] + + +class CommandPalette(QWidget): + """A popup: a search field and the matches; emits :attr:`ran` with the entry it ran.""" + + ran = Signal(object) + + def __init__(self, entries: list[Entry], parent: QWidget | None = None) -> None: + super().__init__(parent, Qt.Popup | Qt.FramelessWindowHint | Qt.NoDropShadowWindowHint) + self.setAttribute(Qt.WA_TranslucentBackground) + self.entries = entries + self.card = QFrame() + self.card.setObjectName("boxPicker") + shadow = QGraphicsDropShadowEffect(self.card) + shadow.setBlurRadius(SHADOW_MARGIN + 8) + shadow.setOffset(0, 6) + shadow.setColor(QColor(0, 0, 0, 110 if theme.is_dark() else 50)) + self.card.setGraphicsEffect(shadow) + self.search = QLineEdit() + self.search.setObjectName("boxSearch") + self.search.setPlaceholderText("Type to find an analysis, a file, a box, a space or an action") + self.search.setClearButtonEnabled(True) + self.search.textChanged.connect(lambda _t: self.refill()) + self.search.returnPressed.connect(self.run_current) + self.search.installEventFilter(self) + self.list = QListWidget() + self.list.setObjectName("paletteList") + self.list.setUniformItemSizes(True) + self.list.itemActivated.connect(lambda _item: self.run_current()) + self.list.itemClicked.connect(lambda _item: self.run_current()) + self.hint = label("↑ ↓ to pick · Enter to run · Esc to close", "muted") + self.card.setLayout(vbox(self.search, self.list, self.hint, spacing=10, margins=(14, 14, 14, 12))) + self.card.setMinimumWidth(560) + self.setLayout(vbox(self.card, margins=(SHADOW_MARGIN,) * 4)) + self.matches: list[Entry] = [] + self.refill() + + def refill(self) -> None: + self.matches = match(self.entries, self.search.text()) + self.list.clear() + t = style.tokens() + for entry in self.matches: + item = QListWidgetItem(f"{entry.title}") + detail = f"{entry.kind}" + (f" · {entry.detail}" if entry.detail else "") + item.setText(f"{entry.title} {detail}") + item.setToolTip(detail) + item.setForeground(QColor(t.text)) + self.list.addItem(item) + if not self.matches: + item = QListWidgetItem("Nothing matches. Try another word.") + item.setFlags(Qt.NoItemFlags) + item.setForeground(QColor(t.text_muted)) + self.list.addItem(item) + elif self.list.count(): + self.list.setCurrentRow(0) + rows = max(1, min(len(self.matches) or 1, 12)) + self.list.setFixedHeight(rows * 28 + 8) + + def run_current(self) -> None: + row = self.list.currentRow() + if not self.matches: + return + entry = self.matches[row if 0 <= row < len(self.matches) else 0] + self.close() + entry.run() + self.ran.emit(entry) + + def eventFilter(self, watched, event) -> bool: # noqa: N802 + from PySide6.QtCore import QEvent + if watched is self.search and event.type() == QEvent.KeyPress: + key = event.key() + if key in (Qt.Key_Down, Qt.Key_Up): + row = self.list.currentRow() + (1 if key == Qt.Key_Down else -1) + if 0 <= row < self.list.count(): + self.list.setCurrentRow(row) + return True + if key == Qt.Key_Escape: + self.close() + return True + return super().eventFilter(watched, event) + + def open_at(self, point: QPoint) -> None: + """Show with the card's top-left at ``point`` (global), kept on screen.""" + self.adjustSize() + screen = self.screen().availableGeometry() if self.screen() else None + hint = self.sizeHint() + width, height = hint.width() - 2 * SHADOW_MARGIN, hint.height() - 2 * SHADOW_MARGIN + x, y = point.x(), point.y() + if screen is not None: + x = max(screen.left(), min(x, screen.right() - width)) + y = max(screen.top(), min(y, screen.bottom() - height)) + self.move(x - SHADOW_MARGIN, y - SHADOW_MARGIN) + self.show() + motion.fade_in(self) + self.search.setFocus() + + +__all__ = ["CommandPalette", "Entry", "match", "KINDS"] diff --git a/openprocess/gui/studio/connections.py b/openprocess/gui/studio/connections.py index dffb667..876d9c9 100644 --- a/openprocess/gui/studio/connections.py +++ b/openprocess/gui/studio/connections.py @@ -23,6 +23,7 @@ #: (title, what it is, the input boxes' function names) INPUTS = [ ("Event logs", "XES (also gzipped), CSV, or a .txt in the course notation", ("open_log",)), + ("Object-centric logs", "OCEL 2.0 JSON (.jsonocel), or the older JSON-OCEL", ("open_ocel", "flatten_ocel")), ("Petri nets", "PNML files, from any tool", ("open_net",)), ("Coloured nets", "CPN Tools .cpn files", ("open_cpn",)), ("Transition systems", "ts.txt, typed as s0 -a-> s1", ("open_transition_system",)), diff --git a/openprocess/gui/studio/graph_builders.py b/openprocess/gui/studio/graph_builders.py index 78cd771..5a0459a 100644 --- a/openprocess/gui/studio/graph_builders.py +++ b/openprocess/gui/studio/graph_builders.py @@ -81,6 +81,30 @@ def petri_net_specs(net: PetriNet, marking: Marking | None = None, # --------------------------------------------------------------------------- # Process maps # --------------------------------------------------------------------------- +def ocdfg_specs(graph) -> tuple[list[NodeSpec], list[EdgeSpec], dict[str, str]]: + """The object-centric map: the activities once, and one set of arrows per + object type in that type's colour, side by side where types share a path. + Returns the nodes, the edges and ``{object type: colour}`` for a legend.""" + t = style.tokens() + colours = {object_type: style.categorical(index) for index, object_type in enumerate(graph.object_types)} + nodes = [NodeSpec(activity, "activity", text=activity, subtext=f"{count:,} events", + fill=t.surface, stroke=t.border, + tooltip=f"{activity}: {count:,} events\n" + "\n".join( + f"{object_type}: {graph.objects_at.get((activity, object_type), 0)} objects" + for object_type in graph.object_types)) + for activity, count in graph.activities.most_common()] + edges: list[EdgeSpec] = [] + busiest = max((n for counts in graph.edges.values() for n in counts.values()), default=1) + count_types = len(graph.object_types) or 1 + for index, object_type in enumerate(graph.object_types): + offset = (index - (count_types - 1) / 2) * 12.0 + for (a, b), n in graph.edges[object_type].items(): + edges.append(EdgeSpec(a, b, text=str(n), width=1.2 + 2.6 * n / busiest, colour=colours[object_type], + tooltip=f"{object_type}: {n} object{'s' if n != 1 else ''} went {a} → {b}", + bend=offset if count_types > 1 else 0.0)) + return nodes, edges, colours + + def dfg_specs(dfg: DFG, mode: str = "frequency") -> tuple[list[NodeSpec], list[EdgeSpec]]: """Frequency mode: node fill = how often; edge width = how often. Performance mode: edge labels are mean durations; the slowest edges are diff --git a/openprocess/gui/studio/graph_view.py b/openprocess/gui/studio/graph_view.py index e043fd1..40d36de 100644 --- a/openprocess/gui/studio/graph_view.py +++ b/openprocess/gui/studio/graph_view.py @@ -64,6 +64,7 @@ class EdgeSpec: dashed: bool = False tooltip: str = "" arrow: bool = True # False: a plain line (a tree's "contains") + bend: float = 0.0 # sideways offset, so parallel edges (object types) stay apart def _font(size: float = 11.5, bold: bool = False) -> QFont: @@ -460,7 +461,11 @@ def populate(self, nodes: list[NodeSpec], edges: list[EdgeSpec], if len(route) < 2: route = [self.nodes[spec.source].pos(), self.nodes[spec.target].pos()] item = EdgeItem(spec, self.nodes[spec.source], self.nodes[spec.target], route) - if len(route) == 2 and (spec.target, spec.source) in pairs: + if spec.bend: + item.route = [self.nodes[spec.source].pos(), self.nodes[spec.target].pos()] + item.bend = spec.bend + item.rebuild() + elif len(route) == 2 and (spec.target, spec.source) in pairs: item.bend = 16.0 item.rebuild() self.addItem(item) @@ -697,6 +702,25 @@ def export_png(self, path: str, scale: float = 2.0) -> None: painter.end() image.save(path) + def svg_text(self) -> str: + """The drawing as SVG text (for a report).""" + from PySide6.QtCore import QBuffer, QByteArray + from PySide6.QtSvg import QSvgGenerator + rect = self.graph.sceneRect() + data = QByteArray() + buffer = QBuffer(data) + buffer.open(QBuffer.WriteOnly) + generator = QSvgGenerator() + generator.setOutputDevice(buffer) + generator.setSize(rect.size().toSize()) + generator.setViewBox(QRectF(0, 0, rect.width(), rect.height())) + painter = QPainter(generator) + self.graph.render(painter, QRectF(0, 0, rect.width(), rect.height()), rect) + painter.end() + buffer.close() + text = bytes(data).decode("utf-8", errors="replace") + return text[text.index(" None: from PySide6.QtSvg import QSvgGenerator rect = self.graph.sceneRect() diff --git a/openprocess/gui/studio/spaces.py b/openprocess/gui/studio/spaces.py index f8a8ce8..cd4a0bf 100644 --- a/openprocess/gui/studio/spaces.py +++ b/openprocess/gui/studio/spaces.py @@ -27,7 +27,7 @@ LABELS = {MINE: "Mine", MODEL: "Model", LEARN: "Learn"} #: The folder's file kinds (see :func:`~.workspace.file_kind`) each space lists. -FILE_KINDS = {MINE: ("workflow", "log", "ts"), MODEL: ("petri", "cpn")} +FILE_KINDS = {MINE: ("workflow", "log", "ocel", "ts"), MODEL: ("petri", "cpn")} #: The file kinds a space's main section is made of; the rest are its material. MAIN_KINDS = {MINE: ("workflow",), MODEL: ("petri", "cpn")} diff --git a/openprocess/gui/studio/style.py b/openprocess/gui/studio/style.py index b21bc2f..f3dd6e8 100644 --- a/openprocess/gui/studio/style.py +++ b/openprocess/gui/studio/style.py @@ -348,6 +348,15 @@ def stylesheet() -> str: #revealRow QLabel {{ background: transparent; }} #revealTitle {{ font-weight: 600; }} + /* The command palette's list. */ + QListWidget#paletteList {{ background: transparent; border: none; outline: none; font-size: 13px; }} + QListWidget#paletteList::item {{ padding: 5px 8px; border-radius: 6px; }} + QListWidget#paletteList::item:selected {{ background: {t.accent_soft}; color: {t.text}; }} + + /* The reproducibility badge beside an analysis's title. */ + QPushButton#reproBadge {{ border: none; border-radius: 9px; padding: 2px 9px; font-size: 11px; + font-weight: 600; }} + /* The Summary's tiles and the Model palette. */ #summaryValue {{ font-size: 22px; font-weight: 600; }} QPushButton#paletteTool {{ background: {t.surface}; border: 1px solid {t.border}; border-radius: 7px; diff --git a/openprocess/gui/studio/widgets.py b/openprocess/gui/studio/widgets.py index 81a510b..0d608cf 100644 --- a/openprocess/gui/studio/widgets.py +++ b/openprocess/gui/studio/widgets.py @@ -528,15 +528,47 @@ def __init__(self, title: str = "", subtitle: str = "", parent=None) -> None: text.setSpacing(2) self.title = ElidedLabel(title, "pageTitle") self.subtitle = ElidedLabel(subtitle, "pageSubtitle") - text.addWidget(self.title) + title_row = QHBoxLayout() + title_row.setSpacing(10) + title_row.addWidget(self.title) + title_row.addStretch(1) + text.addLayout(title_row) text.addWidget(self.subtitle) self._text = text + self._title_row = title_row self.back: QPushButton | None = None + self.badge: QPushButton | None = None layout.addLayout(text, 1) self.actions = QHBoxLayout() self.actions.setSpacing(8) layout.addLayout(self.actions) + def set_badge(self, text: str, tone: str = "muted", tooltip: str = "", slot=None) -> None: + """A small chip beside the title ("Recorded · this run reproduces it"). + ``tone``: good, warning, critical or muted. Empty text hides it.""" + if self.badge is None: + self.badge = QPushButton() + self.badge.setObjectName("reproBadge") + self.badge.setCursor(Qt.PointingHandCursor) + self.badge.setFocusPolicy(Qt.NoFocus) + self._title_row.insertWidget(1, self.badge, 0, Qt.AlignVCenter) + self._badge_slot = None + from . import style + colours = {"good": (style.STATUS["good"], "#ffffff"), + "warning": (style.STATUS["warning"], "#1d1d1f"), + "critical": (style.STATUS["critical"], "#ffffff"), + "muted": (style.qc(style.tokens().text, 0.10).name(QColor.HexArgb), style.tokens().text_secondary)} + background, ink = colours.get(tone, colours["muted"]) + self.badge.setStyleSheet(f"QPushButton#reproBadge {{ background: {background}; color: {ink}; }}") + self.badge.setText(text) + self.badge.setToolTip(tooltip) + self.badge.setVisible(bool(text)) + if self._badge_slot is not None: + self.badge.clicked.disconnect(self._badge_slot) + self._badge_slot = slot + if slot is not None: + self.badge.clicked.connect(slot) + def set_back(self, text: str, slot) -> None: """A link above the title, back to where this page was opened from ("‹ Back to Discover and check").""" diff --git a/openprocess/gui/studio/workspace.py b/openprocess/gui/studio/workspace.py index e87ee8a..18572c4 100644 --- a/openprocess/gui/studio/workspace.py +++ b/openprocess/gui/studio/workspace.py @@ -40,6 +40,8 @@ ".pnml": "petri", ".cpn": "cpn", ".cpnflow": "workflow", + ".jsonocel": "ocel", + ".json": "ocel", } #: Text files the app opens, by the end of their name: a log in textbook diff --git a/openprocess/mining/ocel.py b/openprocess/mining/ocel.py new file mode 100644 index 0000000..4f17b10 --- /dev/null +++ b/openprocess/mining/ocel.py @@ -0,0 +1,262 @@ +"""Object-centric event logs (OCEL): events that touch several objects at once. + +A classical event log has one case per trace. In an order-to-cash process an +event such as *pack items* belongs to one order, three items and one +package at the same time; forcing a case id flattens that and invents +convergence and divergence. OCEL keeps the objects: every event names the +objects it involves, by type. This module reads the two JSON forms of the +standard (OCEL 2.0, and the older JSON-OCEL 1.0), flattens a log onto one +object type when a classical algorithm needs a case, and builds the +**object-centric directly-follows graph**: one directly-follows relation per +object type, drawn in that type's colour on the shared activities. + +Follows: W.M.P. van der Aalst and A. Berti, *Discovering Object-Centric Petri +Nets* (Fundamenta Informaticae 175, 2020) and the OCEL 2.0 specification +(ocel-standard.org). +""" + +from __future__ import annotations + +import json +from collections import Counter, defaultdict +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +from .log import KEY_NAME, KEY_TIME, Event, EventLog, Trace + + +@dataclass +class OCEvent: + id: str + activity: str + time: datetime | None + #: object type -> the ids of the objects of that type this event involves + objects: dict[str, list[str]] = field(default_factory=dict) + attributes: dict[str, Any] = field(default_factory=dict) + + @property + def object_ids(self) -> list[str]: + return [o for ids in self.objects.values() for o in ids] + + +@dataclass +class OCObject: + id: str + type: str + attributes: dict[str, Any] = field(default_factory=dict) + + +@dataclass +class OCEL: + """An object-centric event log: events in time order, and the objects they touch.""" + + name: str = "Object-centric log" + events: list[OCEvent] = field(default_factory=list) + objects: dict[str, OCObject] = field(default_factory=dict) + object_types: list[str] = field(default_factory=list) + source_path: str | None = None + + def __len__(self) -> int: + return len(self.events) + + @property + def activities(self) -> Counter: + return Counter(e.activity for e in self.events) + + def objects_of_type(self, object_type: str) -> list[OCObject]: + return [o for o in self.objects.values() if o.type == object_type] + + def counts_by_type(self) -> dict[str, int]: + return {t: sum(1 for o in self.objects.values() if o.type == t) for t in self.object_types} + + def events_of(self, object_id: str) -> list[OCEvent]: + return [e for e in self.events if object_id in e.object_ids] + + def summary(self) -> str: + types = ", ".join(f"{n} {t}" for t, n in self.counts_by_type().items()) + return (f"{len(self.events):,} events · {len(self.activities)} activities · " + f"{len(self.objects):,} objects ({types})") + + +# --------------------------------------------------------------------------- +# Reading +# --------------------------------------------------------------------------- +def _time(value) -> datetime | None: + if value is None or value == "": + return None + if isinstance(value, (int, float)): + return datetime.fromtimestamp(value, tz=timezone.utc) + text = str(value).strip() + if text.endswith("Z"): + text = text[:-1] + "+00:00" + try: + return datetime.fromisoformat(text) + except ValueError: + for pattern in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%dT%H:%M:%S", "%Y-%m-%d"): + try: + return datetime.strptime(text, pattern) + except ValueError: + continue + return None + + +def read_ocel(path: str | Path) -> OCEL: + """Read an OCEL 2.0 JSON file (``.jsonocel`` or ``.json``), or a JSON-OCEL 1.0 one.""" + path = Path(path) + data = json.loads(path.read_text(encoding="utf-8")) + if "objectTypes" in data or "eventTypes" in data: + log = _read_ocel2(data) + elif "ocel:events" in data: + log = _read_ocel1(data) + else: + raise ValueError(f"{path.name} is not an object-centric event log (no objectTypes, no ocel:events)") + log.name = path.name.split(".")[0] + log.source_path = str(path) + return log + + +def _read_ocel2(data: dict) -> OCEL: + log = OCEL() + log.object_types = [t["name"] for t in data.get("objectTypes", []) if isinstance(t, dict) and t.get("name")] + for item in data.get("objects", []): + object_id, object_type = str(item.get("id")), str(item.get("type", "")) + attributes = {} + for attribute in item.get("attributes", []) or []: + if isinstance(attribute, dict) and "name" in attribute: + attributes[attribute["name"]] = attribute.get("value") # the latest value wins + log.objects[object_id] = OCObject(object_id, object_type, attributes) + if object_type and object_type not in log.object_types: + log.object_types.append(object_type) + for item in data.get("events", []): + objects: dict[str, list[str]] = defaultdict(list) + for relation in item.get("relationships", []) or []: + object_id = str(relation.get("objectId")) + kind = log.objects[object_id].type if object_id in log.objects else "object" + objects[kind].append(object_id) + attributes = {} + for attribute in item.get("attributes", []) or []: + if isinstance(attribute, dict) and "name" in attribute: + attributes[attribute["name"]] = attribute.get("value") + log.events.append(OCEvent(str(item.get("id")), str(item.get("type", "")), _time(item.get("time")), + dict(objects), attributes)) + _finish(log) + return log + + +def _read_ocel1(data: dict) -> OCEL: + log = OCEL() + globals_ = data.get("ocel:global-log", {}) or {} + log.object_types = list(globals_.get("ocel:object-types", []) or []) + for object_id, item in (data.get("ocel:objects", {}) or {}).items(): + object_type = str(item.get("ocel:type", "")) + log.objects[str(object_id)] = OCObject(str(object_id), object_type, dict(item.get("ocel:ovmap", {}) or {})) + if object_type and object_type not in log.object_types: + log.object_types.append(object_type) + for event_id, item in (data.get("ocel:events", {}) or {}).items(): + objects: dict[str, list[str]] = defaultdict(list) + for object_id in item.get("ocel:omap", []) or []: + kind = log.objects[str(object_id)].type if str(object_id) in log.objects else "object" + objects[kind].append(str(object_id)) + log.events.append(OCEvent(str(event_id), str(item.get("ocel:activity", "")), + _time(item.get("ocel:timestamp")), dict(objects), + dict(item.get("ocel:vmap", {}) or {}))) + _finish(log) + return log + + +def _finish(log: OCEL) -> None: + """Events in time order (ties keep the file's order); object types in a stable order.""" + log.events.sort(key=lambda e: (e.time is None, e.time or datetime.min.replace(tzinfo=None)) if e.time is None + or e.time.tzinfo is None else (False, e.time)) + seen = set() + types = [] + for t in log.object_types + [o.type for o in log.objects.values()]: + if t and t not in seen: + seen.add(t) + types.append(t) + log.object_types = types + + +# --------------------------------------------------------------------------- +# Flattening: one object type becomes the case notion +# --------------------------------------------------------------------------- +def flatten(log: OCEL, object_type: str | None = None) -> EventLog: + """A classical event log with one case per object of ``object_type`` (the + type with most objects when None). An event that involves several + objects of the type is copied into each of their cases: that is what + flattening does, and why the object-centric map exists.""" + object_type = object_type or max(log.counts_by_type().items(), key=lambda kv: kv[1], default=("", 0))[0] + if object_type not in log.object_types: + raise ValueError(f"No object type {object_type!r}: this log has {', '.join(log.object_types) or 'none'}") + traces = [] + for obj in log.objects_of_type(object_type): + events = [e for e in log.events if obj.id in e.objects.get(object_type, ())] + if not events: + continue + trace = Trace({KEY_NAME: obj.id, "ocel:type": object_type, **obj.attributes}) + for event in events: + attributes = {KEY_NAME: event.activity, "ocel:event": event.id, **event.attributes} + if event.time is not None: + attributes[KEY_TIME] = event.time + for other_type, ids in event.objects.items(): + if other_type != object_type: + attributes[f"ocel:{other_type}"] = ", ".join(ids) + trace.events.append(Event(attributes)) + traces.append(trace) + return EventLog(traces=traces, attributes={KEY_NAME: f"{log.name} · {object_type}"}, source_path=log.source_path) + + +# --------------------------------------------------------------------------- +# The object-centric directly-follows graph +# --------------------------------------------------------------------------- +@dataclass +class OCDFG: + """One directly-follows relation per object type, over shared activities.""" + + activities: Counter = field(default_factory=Counter) # activity -> events + object_types: list[str] = field(default_factory=list) + #: object type -> Counter of (a, b): how many objects of the type went a then b + edges: dict[str, Counter] = field(default_factory=dict) + starts: dict[str, Counter] = field(default_factory=dict) # type -> first activities + ends: dict[str, Counter] = field(default_factory=dict) # type -> last activities + #: (activity, type) -> how many objects of the type the activity's events involve + objects_at: dict[tuple[str, str], int] = field(default_factory=dict) + name: str = "Object-centric map" + + def summary(self) -> str: + paths = sum(len(c) for c in self.edges.values()) + return f"{len(self.activities)} activities · {paths} paths over {len(self.object_types)} object types" + + +def object_centric_dfg(log: OCEL) -> OCDFG: + """Follow every object through its events: each type gets its own + directly-follows counts, drawn later in its own colour.""" + graph = OCDFG(activities=log.activities, object_types=list(log.object_types), name=f"Map of {log.name}") + per_object: dict[str, list[OCEvent]] = defaultdict(list) + for event in log.events: + for ids in event.objects.values(): + for object_id in ids: + per_object[object_id].append(event) + objects_at: Counter = Counter() + for object_type in log.object_types: + edges: Counter = Counter() + starts: Counter = Counter() + ends: Counter = Counter() + for obj in log.objects_of_type(object_type): + events = per_object.get(obj.id, []) + if not events: + continue + starts[events[0].activity] += 1 + ends[events[-1].activity] += 1 + for a, b in zip(events, events[1:]): + edges[(a.activity, b.activity)] += 1 + for event in events: + objects_at[(event.activity, object_type)] += 1 + graph.edges[object_type], graph.starts[object_type], graph.ends[object_type] = edges, starts, ends + graph.objects_at = dict(objects_at) + return graph + + +__all__ = ["OCEL", "OCDFG", "OCEvent", "OCObject", "read_ocel", "flatten", "object_centric_dfg"] diff --git a/tests/data/orders.jsonocel b/tests/data/orders.jsonocel new file mode 100644 index 0000000..fa2ec14 --- /dev/null +++ b/tests/data/orders.jsonocel @@ -0,0 +1,42 @@ +{ + "objectTypes": [ + {"name": "order", "attributes": [{"name": "price", "type": "float"}]}, + {"name": "item", "attributes": [{"name": "product", "type": "string"}]}, + {"name": "package", "attributes": []} + ], + "eventTypes": [ + {"name": "place order", "attributes": []}, + {"name": "pick item", "attributes": []}, + {"name": "pack items", "attributes": []}, + {"name": "ship package", "attributes": []}, + {"name": "pay order", "attributes": []} + ], + "objects": [ + {"id": "o1", "type": "order", "attributes": [{"name": "price", "time": "2024-03-01T09:00:00Z", "value": 120.0}]}, + {"id": "o2", "type": "order", "attributes": [{"name": "price", "time": "2024-03-01T09:30:00Z", "value": 45.0}]}, + {"id": "i1", "type": "item", "attributes": [{"name": "product", "time": "2024-03-01T09:00:00Z", "value": "lamp"}]}, + {"id": "i2", "type": "item", "attributes": [{"name": "product", "time": "2024-03-01T09:00:00Z", "value": "chair"}]}, + {"id": "i3", "type": "item", "attributes": [{"name": "product", "time": "2024-03-01T09:30:00Z", "value": "mug"}]}, + {"id": "p1", "type": "package", "attributes": []} + ], + "events": [ + {"id": "e1", "type": "place order", "time": "2024-03-01T09:00:00Z", "attributes": [], + "relationships": [{"objectId": "o1", "qualifier": "order"}, {"objectId": "i1", "qualifier": "item"}, {"objectId": "i2", "qualifier": "item"}]}, + {"id": "e2", "type": "place order", "time": "2024-03-01T09:30:00Z", "attributes": [], + "relationships": [{"objectId": "o2", "qualifier": "order"}, {"objectId": "i3", "qualifier": "item"}]}, + {"id": "e3", "type": "pick item", "time": "2024-03-01T10:00:00Z", "attributes": [], + "relationships": [{"objectId": "i1", "qualifier": "item"}]}, + {"id": "e4", "type": "pick item", "time": "2024-03-01T10:05:00Z", "attributes": [], + "relationships": [{"objectId": "i2", "qualifier": "item"}]}, + {"id": "e5", "type": "pick item", "time": "2024-03-01T10:10:00Z", "attributes": [], + "relationships": [{"objectId": "i3", "qualifier": "item"}]}, + {"id": "e6", "type": "pack items", "time": "2024-03-01T11:00:00Z", "attributes": [], + "relationships": [{"objectId": "i1", "qualifier": "item"}, {"objectId": "i2", "qualifier": "item"}, {"objectId": "i3", "qualifier": "item"}, {"objectId": "p1", "qualifier": "package"}]}, + {"id": "e7", "type": "ship package", "time": "2024-03-01T12:00:00Z", "attributes": [], + "relationships": [{"objectId": "p1", "qualifier": "package"}]}, + {"id": "e8", "type": "pay order", "time": "2024-03-02T09:00:00Z", "attributes": [], + "relationships": [{"objectId": "o1", "qualifier": "order"}]}, + {"id": "e9", "type": "pay order", "time": "2024-03-02T10:00:00Z", "attributes": [], + "relationships": [{"objectId": "o2", "qualifier": "order"}]} + ] +} diff --git a/tests/test_citation.py b/tests/test_citation.py new file mode 100644 index 0000000..9751e07 --- /dev/null +++ b/tests/test_citation.py @@ -0,0 +1,55 @@ +"""Citations: BibTeX from the reference list, the app's own entry, CITATION.cff.""" + +from __future__ import annotations + +import datetime +from pathlib import Path + +import openprocess +from openprocess import citation, references + + +def test_every_reference_becomes_a_bibtex_entry(): + for reference in references.REFERENCES: + entry = citation.entry(reference) + assert entry.key == reference.key and entry.fields["title"] + assert entry.kind in ("article", "inproceedings", "book", "phdthesis", "techreport", "misc") + text = entry.bibtex() + assert text.startswith(f"@{entry.kind}{{{reference.key},") and text.endswith("}") + if reference.key != "cpntools": # a tool, undated + assert entry.fields["year"].isdigit() + + +def test_the_kinds_and_fields_follow_the_citation(): + article = citation.entry(references.reference("aalst2004")) + assert article.kind == "article" and article.fields["journal"] == "IEEE Transactions on Knowledge and Data Engineering" + assert (article.fields["volume"], article.fields["number"], article.fields["pages"]) == ("16", "9", "1128–1142") + assert article.fields["author"] == "W.M.P. van der Aalst and A.J.M.M. Weijters and L. Maruster" + assert article.fields["doi"] == "10.1109/TKDE.2004.47" and article.fields["year"] == "2004" + paper = citation.entry(references.reference("leemans2013")) + assert paper.kind == "inproceedings" and paper.fields["publisher"] == "Springer" and paper.fields["pages"] == "311–329" + book = citation.entry(references.reference("aalst2016")) + assert book.kind == "book" and book.fields["edition"] == "2" and book.fields["publisher"] == "Springer" + thesis = citation.entry(references.reference("adriansyah2014")) + assert thesis.kind == "phdthesis" and "Eindhoven" in thesis.fields["school"] + standard = citation.entry(references.reference("pnml")) + assert standard.fields["organization"] == "ISO/IEC 15909-2:2011" + assert "\\&" in citation.entry(references.reference("leemans2019")).bibtex() # escaped for BibTeX + + +def test_the_algorithms_cite_the_works_they_follow(): + keys = [e.key for e in citation.entries_for_module("openprocess.mining.discovery.alpha")] + assert "aalst2004" in keys + assert citation.entries_for_module("openprocess.nowhere") == [] + + +def test_the_app_cites_itself_and_the_cff_file_is_current(): + text = citation.app_bibtex("0.10.0", 2026) + assert text.startswith("@software{openprocess,") and "version = {0.10.0}" in text and "Struthers, Sam" in text + assert "0.10.0" in citation.app_text("0.10.0", 2026) and citation.APP_URL in citation.app_text("0.10.0", 2026) + cff = Path(__file__).resolve().parents[1] / "CITATION.cff" + assert cff.exists() + assert f"version: {openprocess.__version__}" in cff.read_text(encoding="utf-8"), \ + "CITATION.cff names another version than openprocess.__version__: regenerate it with citation.cff_text" + generated = citation.cff_text("1.2.3", datetime.date(2030, 1, 2)) + assert "cff-version: 1.2.0" in generated and "date-released: 2030-01-02" in generated diff --git a/tests/test_flow.py b/tests/test_flow.py index e9939c6..2ff11c8 100644 --- a/tests/test_flow.py +++ b/tests/test_flow.py @@ -557,3 +557,46 @@ def test_a_tidied_layout_is_kept_with_the_box(tmp_path): assert again.nodes[miner.id].layout == {"p_start": (0.0, 10.0), "t_a": (120.0, 10.5)} assert again.nodes[log.id].layout == {} assert "layout" not in next(b for b in wf.to_dict()["boxes"] if b["id"] == log.id) + + +def test_reproducibility_status_follows_the_record_and_the_run(tmp_path): + """Not recorded → reproduces → changed (an input file edited) → differs (another result).""" + from openprocess.flow import reproducibility + from openprocess.flow.record import make_record + + (tmp_path / "orders.log.txt").write_text("[^3, ^2, ]", encoding="utf-8") + wf = Workflow("Repro") + log = wf.add(open_log, {"file": "orders.log.txt"}) + miner = wf.add(alpha_miner) + wf.connect(log, miner) + runner = Runner(standard_library(), folder=tmp_path) + run = runner.run(wf) + assert run.ok + assert reproducibility.status(wf, None, run, tmp_path).state == "unrecorded" + record = make_record(wf, run, tmp_path, lock=False) + status = reproducibility.status(wf, record, run, tmp_path) + assert status.state == "reproduces" and status.tone == "good" and "2 results" in status.details[0] + assert reproducibility.status(wf, record, None, tmp_path).state == "unchecked" + (tmp_path / "orders.log.txt").write_text("[^9]", encoding="utf-8") + status = reproducibility.status(wf, record, run, tmp_path) + assert status.state == "changed" and status.headline == "Recorded · 1 change since" and "has changed" in status.details[0] + again = runner.run(wf) # the changed log gives another model + status = reproducibility.status(wf, record, again, tmp_path) + assert status.state == "differs" and status.tone == "critical" + assert any("different result" in d for d in status.details) + + +def test_an_edited_input_file_is_not_served_from_the_cache(tmp_path): + """The cache keys a file setting by the file as it is, not by its name.""" + import os + (tmp_path / "orders.log.txt").write_text("[^2]", encoding="utf-8") + wf = Workflow("Fresh") + log = wf.add(open_log, {"file": "orders.log.txt"}) + runner = Runner(standard_library(), folder=tmp_path) + first = runner.run(wf) + assert len(first.value(log)) == 2 + (tmp_path / "orders.log.txt").write_text("[^5]", encoding="utf-8") + now = (tmp_path / "orders.log.txt").stat() + os.utime(tmp_path / "orders.log.txt", ns=(now.st_atime_ns, now.st_mtime_ns + 1_000_000)) + second = runner.run(wf, previous=first) + assert len(second.value(log)) == 5 and second.result(log).key != first.result(log).key diff --git a/tests/test_flow_gui.py b/tests/test_flow_gui.py index 59d926a..e523c77 100644 --- a/tests/test_flow_gui.py +++ b/tests/test_flow_gui.py @@ -321,9 +321,11 @@ def test_the_page_opens_calm_and_the_panel_comes_with_a_click(app): window.action_new_workflow(TEMPLATES[0][1]) page = window.current_page() _wait_run(app, page) - header = [b.text() for b in page.header.findChildren(QPushButton) if b.isVisible()] - assert header == ["Canvas", "Summary", "+ Add box", "Run ▶", "⋯"] - assert [a.text() for a in page.more_menu.actions() if a.text()] == ["Re-run", "Record…", "Export experiment…", "Save"] + header = [b.text() for b in page.header.findChildren(QPushButton) + if b.isVisible() and b.objectName() != "reproBadge"] + assert header == ["Canvas", "Summary", "Runs", "+ Add box", "Run ▶", "⋯"] + assert [a.text() for a in page.more_menu.actions() if a.text()] == [ + "Re-run", "Record…", "Export experiment…", "Export report…", "Save"] assert not page.panel.isVisible() and not hasattr(page, "box_tree") log, miner = page.workflow.order()[:2] page.select(log.id) @@ -932,3 +934,225 @@ def test_motion_settles_the_window_and_can_be_turned_off(app, monkeypatch): assert motion.lift(window.modes) is None and motion.fade_in(window) is None page.document.dirty = False window.close() + + +def test_the_header_says_whether_the_analysis_is_reproducible(app, tmp_path): + """A badge beside the title: not recorded, then recorded and reproducing + once the run is saved, then "changes since" when an input file is edited.""" + from openprocess.gui.flow.templates import TEMPLATES + from openprocess.gui.studio.app import StudioWindow + + week = tmp_path / "Week 2" + week.mkdir() + (week / "orders.log.txt").write_text("[^3, ^2, ]", encoding="utf-8") + window = StudioWindow() + window.resize(1400, 900) + window.show() + assert window.open_workspace(str(week)) + window.action_new_workflow(TEMPLATES[0][1]) # on the folder's first log + page = window.current_page() + assert page.header.badge is not None and page.header.badge.text() == "Not recorded yet" + _wait_run(app, page) + window._flush_autosaves() # the run is recorded on save + assert page.header.badge.text() == "Recorded · this run reproduces it" + assert page.document.record is not None and page.document.record.results + (week / "orders.log.txt").write_text("[^9]", encoding="utf-8") + page.refresh_reproducibility() + assert page.header.badge.text() == "Recorded · 1 change since" + assert "has changed" in page.header.badge.toolTip() + page.header.badge.click() # the record, with the status on top + _pump(app, 0.2) + from PySide6.QtWidgets import QDialog, QLabel + dialog = next(d for d in window.findChildren(QDialog) if "the record" in d.windowTitle()) + assert any("1 change since" in w.text() for w in dialog.findChildren(QLabel)) + dialog.close() + page.document.dirty = False + window.close() + + +def test_cite_the_app_and_an_algorithm(app): + """Help ▸ Cite OpenProcess… gives the app's BibTeX; an algorithm's Code tab + has Cite for the papers it follows.""" + from PySide6.QtWidgets import QPushButton + + from openprocess.gui.flow.templates import TEMPLATES + from openprocess.gui.studio.app import StudioWindow + + window = StudioWindow() + window.resize(1400, 900) + window.show() + window.cite_app() + assert window.cite_dialog.bibtex.startswith("@software{openprocess,") + assert "version = {" in window.cite_dialog.bibtex + window.cite_dialog.close() + window.action_new_workflow(TEMPLATES[0][1]) + page = window.current_page() + _wait_run(app, page) + page.select(page.workflow.order()[1].id, 2) # the miner's Code tab + _pump(app, 0.2) + cite = next(b for b in page.panel_host.findChildren(QPushButton) if b.objectName() == "algorithmCite") + cite.click() + _pump(app, 0.2) + from PySide6.QtWidgets import QDialog + dialog = next(d for d in window.findChildren(QDialog) if d.windowTitle().startswith("Cite:")) + assert "@inproceedings{leemans2013," in dialog.bibtex + dialog.close() + page.document.dirty = False + window.close() + + +def test_step_through_plays_the_derivation_over_the_result(app): + """The α-algorithm's How tab can be stepped: each step lights up on the net + what it names (all activities, the start ones, the places…), the moments + accumulate, and Play advances by itself.""" + from PySide6.QtWidgets import QPushButton + + from openprocess.gui.flow import stepper as stepping + from openprocess.gui.flow.stepper import StepThrough + from openprocess.gui.flow.templates import TEMPLATES + from openprocess.gui.studio.app import StudioWindow + + window = StudioWindow() + window.resize(1400, 900) + window.show() + window.action_new_workflow(TEMPLATES[1][1]) # Compare discovery: it has the α-algorithm + page = window.current_page() + _wait_run(app, page) + alpha = _node(page, "alpha_miner") + page.select(alpha.id, 1) # How + _pump(app, 0.2) + toggle = next(b for b in page.panel_host.findChildren(QPushButton) if b.objectName() == "stepThrough") + toggle.click() + _pump(app, 0.2) + player = page.panel_host.findChildren(StepThrough)[0] + net = page.run.result(alpha).value + titles = [m[1] for m in player.moments if m[0] == "step"] + assert titles[0].startswith("1. T_L") and titles[-1].startswith("8.") + first_step = next(i for i, m in enumerate(player.moments) if m[0] == "step") + player.show_moment(first_step) # all activities light up + assert player.highlighted(first_step) == {t.id for t in net.transitions.values()} + player.next() # the start activities only + lit = {net.transitions[i].name for i in player.highlighted(player.index)} + assert lit == {"a"} and player.body_layout.count() == first_step + 2 + places = next(i for i, m in enumerate(player.moments) if m[0] == "step" and m[1].startswith("6.")) + player.show_moment(places) + assert player.highlighted(places) >= {p.id for p in net.places.values()} # every place is named + assert not player.next_button.isEnabled() if places == len(player.moments) - 1 else True + stepping.PLAY_MS = 60 + player.timer.setInterval(60) + player.show_moment(0) + player.toggle_play() + assert player.play_button.text() == "Pause" + _pump(app, 1.2) + assert player.index == len(player.moments) - 1 and player.play_button.text() == "Play" + toggle.click() # back to the full report + _pump(app, 0.1) + assert not page.panel_host.findChildren(StepThrough) + page.document.dirty = False + window.close() + + +def test_export_report_writes_one_page_with_the_drawings(app, tmp_path, monkeypatch): + """⋯ ▸ Export report…: an HTML file with the net and the map drawn as SVG.""" + from PySide6.QtGui import QDesktopServices + + from openprocess.gui.flow import page as flow_page + from openprocess.gui.flow.templates import TEMPLATES + from openprocess.gui.studio.app import StudioWindow + + window = StudioWindow() + window.resize(1400, 900) + window.show() + window.action_new_workflow(TEMPLATES[0][1]) + page = window.current_page() + _wait_run(app, page) + target = tmp_path / "report.html" + monkeypatch.setattr(flow_page.QFileDialog, "getSaveFileName", staticmethod(lambda *a, **k: (str(target), ""))) + opened = [] + monkeypatch.setattr(QDesktopServices, "openUrl", staticmethod(lambda url: opened.append(url.toLocalFile()) or True)) + assert any(a.text() == "Export report…" for a in page.more_menu.actions()) + page.export_report() + text = target.read_text(encoding="utf-8") + assert "^3, ^2, ]"}) + miner = wf.add(alpha_miner) + fit = wf.add(check_fit) + dfg = wf.add(directly_follows) + wf.connect(log, miner) + wf.connect(miner, fit) + wf.connect(log, fit) + wf.connect(log, dfg) + run = Runner(standard_library(), folder=tmp_path).run(wf) + assert run.ok, [r.error for r in run.failed()] + return wf, run + + +def test_the_report_has_everything_a_reader_needs(tmp_path): + wf, run = _analysis(tmp_path) + record = make_record(wf, run, tmp_path, lock=False) + page = html_report(wf, run, tmp_path, drawings={run.result(wf.order()[1]).node: ""}, + record=record, status=status(wf, record, run, tmp_path)) + assert page.startswith("") and "" in page # the drawing the app made + assert "1. T_L" in page and "def alpha_miner" in page and "W.M.P. van der Aalst" in page + assert "Fitness" in page or "fitness" in page + assert "activities · paths" in page # the map's tile + # What it read, and how to cite. + assert "What it read" in page and "How to cite" in page and "@software{openprocess," in page + assert "@article{aalst2004," in page + + +def test_the_report_is_written_as_a_file_and_survives_a_run_that_has_not_happened(tmp_path): + wf, run = _analysis(tmp_path) + target = write_report(tmp_path / "out.html", workflow=wf, run=run, folder=tmp_path) + assert target.exists() and target.read_text(encoding="utf-8").count("
^3, ^2, ]"}) + miner = wf.add(inductive_miner) + wf.connect(log, miner) + history = History() + first = take(wf, _run(wf), None) + assert first.label == "First run" and first.figures[log.id][0] == "6 cases" and history.add(first) + assert not history.add(take(wf, _run(wf), history.latest)) # the same again: not kept + wf.set(miner, noise=0.3) + second = take(wf, _run(wf), history.latest) + assert second.label == "Inductive Miner: noise 0.0 → 0.3" and history.add(second) + by_title = {r.title: r for r in compare(first, second)} + assert not by_title["Typed log"].changed + assert by_title["Inductive Miner"].changes == ["Inductive Miner: noise 0.0 → 0.3"] + assert settings_to_apply(first, miner.id)["noise"] == 0.0 + check = wf.add(soundness) + wf.connect(miner, check) + third = take(wf, _run(wf), history.latest) + assert third.label == "+ Soundness" + for _ in range(LIMIT + 5): + history.add(Snapshot(f"t{_}", settings={"n": {"x": _}})) + assert len(history.snapshots) == LIMIT + + +def test_the_history_travels_with_the_workflow_file(tmp_path): + wf = Workflow("Kept") + log = wf.add(typed_log, {"text": "[^2]"}) + run = _run(wf) + history = History() + history.add(take(wf, run, None)) + path = tmp_path / "kept.cpnflow" + save(wf, path, run, tmp_path, lock=False, history=history.to_dict()) + again, record = load(path, wf.library) + restored = History.from_dict(record.history) + assert len(restored.snapshots) == 1 and restored.latest.label == "First run" + assert restored.latest.figures[log.id] == ("2 cases", "4 events") diff --git a/tests/test_studio_gui.py b/tests/test_studio_gui.py index 0c4fc26..73b3b18 100644 --- a/tests/test_studio_gui.py +++ b/tests/test_studio_gui.py @@ -759,7 +759,10 @@ def test_hover_arrow_draws_arcs(app): _pump(app, 0.05) p1 = next(i for i in scene.place_items.values() if i.place.name == "p1") - # Hover just right of p1: the arrow appears on that side. + # Hover just right of p1: the arrow appears on that side. (First a move + # elsewhere: Qt sends no move event when the cursor is already there.) + QTest.mouseMove(port, view.mapFromScene(p1.pos() + QPointF(-200, -200))) + _pump(app, 0.02) QTest.mouseMove(port, view.mapFromScene(p1.pos() + QPointF(30, 0))) _pump(app, 0.05) handle = scene._handle @@ -828,6 +831,8 @@ def test_dragging_the_arrow_to_empty_space_adds_the_other_node(app): net_page = page.net.pages[0] def drag_from(node, end: QPointF) -> None: + QTest.mouseMove(port, view.mapFromScene(node.pos() + QPointF(-200, -200))) # a real move next + _pump(app, 0.02) QTest.mouseMove(port, view.mapFromScene(node.pos() + QPointF(node.rect().width() / 2 + 12, 0))) _pump(app, 0.05) @@ -2500,3 +2505,55 @@ def test_every_computed_card_links_to_its_code(app): for document in window.documents: document.dirty = False window.close() + + +def test_the_command_palette_finds_and_runs_anything(app, tmp_path): + """⌘K: documents, the folder's files, boxes, spaces and menu actions, all + by typing; Enter runs the best match.""" + from PySide6.QtCore import Qt + from PySide6.QtTest import QTest + + from openprocess.gui.studio.app import StudioWindow + from openprocess.gui.studio.command_palette import match + + week = tmp_path / "Week 2" + week.mkdir() + (week / "orders.log.txt").write_text("[^3, ^2, ]", encoding="utf-8") + window = StudioWindow() + window.resize(1400, 900) + window.show() + assert window.open_workspace(str(week)) + window.action_new_workflow(None) + page = window.current_page() + entries = window.palette_entries() + kinds = {e.kind for e in entries} + assert {"Analysis", "File", "Box", "Space", "Action"} <= kinds + assert [e.title for e in match(entries, "alpha")][0] == "α-algorithm" + assert any(e.title == "orders" and e.kind == "File" for e in match(entries, "orders")) + assert match(entries, "cite openprocess")[0].kind == "Action" + assert match(entries, "zzzz") == [] + # Typing and Enter: a box lands on the current analysis. + window.show_command_palette() + palette = window.command_palette + assert palette.isVisible() + QTest.keyClicks(palette.search, "alpha") + assert palette.matches[0].title == "α-algorithm" + QTest.keyClick(palette.search, Qt.Key_Return) + _pump(app, 0.2) + assert not palette.isVisible() + assert [page.workflow.spec(n).name for n in page.workflow.order()] == ["α-algorithm"] + # A space, by name. + window.show_command_palette() + QTest.keyClicks(window.command_palette.search, "model") + QTest.keyClick(window.command_palette.search, Qt.Key_Return) + _pump(app, 0.2) + assert window.space == "model" + # A file: it opens. + window.show_command_palette() + QTest.keyClicks(window.command_palette.search, "orders") + QTest.keyClick(window.command_palette.search, Qt.Key_Return) + _pump(app, 0.3) + assert any(d.name == "orders" or (d.path or "").endswith("orders.log.txt") for d in window.documents) + for document in window.documents: + document.dirty = False + window.close()