diff --git a/.agents/donna/intro.donna.md b/.agents/donna/intro.donna.md deleted file mode 100644 index 8a34945d..00000000 --- a/.agents/donna/intro.donna.md +++ /dev/null @@ -1,48 +0,0 @@ -# Introduction to the Donna tool - -```toml donna -kind = "donna.lib.specification" -``` -This document provides an introduction to the Donna — a CLI tool that helps manage the work of AI agents like Codex. - -## Overview - -Donna is designed to to invert control flow: instead of agents deciding what to do next, the `donna` tells agents what to do next by following predefined workflows. - -The core idea is that most of high-level workflows are more algorithmic than it may seem at first glance. For example, it may be difficult to fix a particular type issue in the codebase, but the overall process of polishing the codebase is quite linear: - -1. Ensure all tests pass. -2. Ensure the code is formatted correctly. -3. Ensure there are no linting errors. -4. Go to the step 1 if you changed something in the process. -5. Finish. - -We may need coding agents on the each step of the process, but there no reason for agents to manage the whole grooming loop by themselves — it take longer time, spends tokens and may lead to confusion of agents. - -## Artifact Tags - -To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. The synced Donna artifacts addressed under `**` use the next set of tags. - -Artifact type tags: - -- `workflow` — workflow artifact — is set automatically by Donna. -- `specification` — specification artifact — is set automatically by Donna. - -## Instructions - -1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool `{{ donna.lib.view("./usage/cli.donna.md") }}`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. -2. If you need to perform a work with Donna, you **MUST** select an appropriate Donna workflow to perform the work and run it. -3. If there is no appropriate workflow, ask the developer for a precise instructions on what to do. -4. If you are executing a workflow operation and need to perform a complex action or changes, you SHOULD search for an appropriate workflow and run it as a child workflow — it is the intended way to use Donna. -5. Run to list all workflows: `{{ donna.lib.list("**", predicate='"workflow" in section.tags') }}` -6. Run to list all specifications: `{{ donna.lib.list("**", predicate='"specification" in section.tags') }}` - -## Journaling - -Donna creates internal journal records for important workflow events, according to the description in `{{ donna.lib.view("./usage/cli.donna.md") }}`. - -Journal records can be forwarded to a third-party tool by configuring `[journal].cmd` in `/donna.toml`. - -The configured command is a list of command arguments. Arguments whose first and last characters are `{` and `}` are replaced with attributes of `JournalRecord`. - -If `[journal].cmd` is omitted, Donna treats it as `None` and performs no journal writing. diff --git a/donna/fixtures/specs/intro.donna.md b/.agents/donna/intro.md similarity index 69% rename from donna/fixtures/specs/intro.donna.md rename to .agents/donna/intro.md index ede43a66..5a0980ba 100644 --- a/donna/fixtures/specs/intro.donna.md +++ b/.agents/donna/intro.md @@ -1,8 +1,4 @@ # Introduction to the Donna tool - -```toml donna -kind = "donna.lib.specification" -``` This document provides an introduction to the Donna — a CLI tool that helps manage the work of AI agents like Codex. ## Overview @@ -19,27 +15,18 @@ The core idea is that most of high-level workflows are more algorithmic than it We may need coding agents on the each step of the process, but there no reason for agents to manage the whole grooming loop by themselves — it take longer time, spends tokens and may lead to confusion of agents. -## Artifact Tags - -To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. The synced Donna artifacts addressed under `**` use the next set of tags. - -Artifact type tags: - -- `workflow` — workflow artifact — is set automatically by Donna. -- `specification` — specification artifact — is set automatically by Donna. - ## Instructions -1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool by running `donna skill usage`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. +1. On start of the YOUR session you **MUST** read and understand instructions on using the Donna tool in the `./usage/cli.md` file. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. 2. If you need to perform a work with Donna, you **MUST** select an appropriate Donna workflow to perform the work and run it. 3. If there is no appropriate workflow, ask the developer for a precise instructions on what to do. 4. If you are executing a workflow operation and need to perform a complex action or changes, you SHOULD search for an appropriate workflow and run it as a child workflow — it is the intended way to use Donna. -5. Run to list all workflows: `{{ donna.lib.list("**", predicate='"workflow" in section.tags') }}` -6. Run to list all specifications: `{{ donna.lib.list("**", predicate='"specification" in section.tags') }}` +5. To find workflows, read the following files: `**/*.donna.md`. +6. To find documentation, read the following files: `**/*.md`. ## Journaling -Donna creates internal journal records for important workflow events, according to the description in `donna skill usage`. +Donna creates internal journal records for important workflow events, according to the description in the `./usage/cli.md` file. Journal records can be forwarded to a third-party tool by configuring `[journal].cmd` in `/donna.toml`. diff --git a/.agents/donna/research/specs/report.donna.md b/.agents/donna/research/specs/report.donna.md deleted file mode 100644 index 59358fa2..00000000 --- a/.agents/donna/research/specs/report.donna.md +++ /dev/null @@ -1,163 +0,0 @@ -# Format of the Research Report document - -```toml donna -kind = "donna.lib.specification" -``` - -This document describes the format and structure of a Research Report document used by Donna workflows from `../**` namespace. - -## Overview - -Donna introduces a group of workflows located in `../**` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. - -Session-related research artifacts MUST be stored as `@/.session/donna/research/.donna.md`, unless the developer or parent workflow specifies a different location. The `` MUST be unique within the session. - -The agent (via workflows) creates the artifact and updates it iteratively as the research process progresses. - -## Research report structure - -The research report is a Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - -- **Primary section** -- title and short description of the research problem. -- **Original problem description** -- original problem statement from the developer or parent workflow. -- **Formalized problem description** -- formalized version of the problem statement. -- **Goals** -- list of goals the research should achieve. -- **Desired form of final solution** -- description of the expected form and constraints for the final solution. -- **Solution space** -- description of analysis axes and synthesis dimensions. -- **Information to collect** -- list of information required to research the problem. -- **Information sources** -- list of sources that can provide the required information. -- **Collected information** -- gathered information with source references. -- **Analysis** -- analysis of collected information along the specified axes. -- **Synthesized solutions** -- synthesized solution options along the specified dimensions. -- **Evaluation** -- evaluation of synthesized solutions against the goals. -- **Final solution** -- final solution in the desired form. - -## General language and format - -- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. -- You MUST follow the structure specified in this document. - -### List format - -- If a section is described as a list, it MUST contain only a single markdown list. -- Each list item MUST be concise and clear. -- Each list item SHOULD be atomic and focused on a single aspect. -- Reviewer MUST be able to tell if the list item statement is true or false by inspecting the resulting artifacts and behavior. - -Common approaches to improve list items: - -- Split a single item with an enumeration into multiple items with a single focus. -- Transform an abstract item into a concrete one by referencing specific artifacts, measurable criteria, verifiable conditions, etc. - -### Trusted inputs - -Some sections of the research report MUST be based on trusted inputs. Trusted inputs are: - -- Original problem description from the developer or parent workflow. -- Statements from the research report itself. -- Existing project documentation, code, and artifacts. -- External standards, when they define constraints or best practices for the project domain. -- Documentation of third-party libraries, frameworks, or tools when they describe constraints or best practices. -- Primary research sources (datasets, reports, official publications) were used to collect the required information. - -## `Primary` section - -- Title MUST be concise and reflect the essence of the research problem. -- Description MUST provide a brief overview of the problem, its purpose, and why research is needed. - -## `Original problem description` section - -- This section MUST contain the original problem description from the developer or from the parent workflow. -- The problem description MUST NOT be modified by agents. - -## `Formalized problem description` section - -- The section MUST contain a clear professional high-level description of the problem based on the original description. -- The section MUST be limited to a single paragraph with a few sentences. -- The section MUST explain what someone gains after the problem is solved and how they can see it working. - -## `Goals` section - -- This section MUST contain a list of goals that the research should achieve. -- Each goal MUST be grounded in the formalized problem description. - -Goal quality criteria: - -- A goal MUST be a desired end state, outcome, or result. -- A goal MUST define what ultimately should be true, not how to achieve it. - -Examples: - -- Bad: `- Investigate database options.` -- Good: `- Identify a database option that meets the project's scalability requirements.` - -## `Desired form of final solution` section - -- The section MUST be grounded in the formalized problem description and the goals. -- The section MUST describe the expected form of the final solution (for example: recommendation, decision matrix, implementation plan, or specification). -- The section MUST specify any required structure, formatting, or constraints on the final solution. -- The section SHOULD be a short list or a short paragraph, whichever is clearer for the problem. - -## `Solution space` section - -- The section MUST describe the axes along which collected information will be analyzed and the dimensions along which solutions will be synthesized. -- The section MUST contain two subsections: **Analysis axes** and **Synthesis dimensions**. -- Each axis or dimension MUST be grounded in the goals or the formalized problem description. - -### `Analysis axes` subsection - -- The subsection MUST contain a list of analysis axes. -- Each axis MUST describe a single perspective or criterion used to analyze information. -- Each axis SHOULD be phrased so it is clear how to apply it to collected information. - -### `Synthesis dimensions` subsection - -- The subsection MUST contain a list of synthesis dimensions. -- Each dimension MUST describe a single perspective or criterion used to synthesize solution options. -- Each dimension SHOULD make the comparison between options easier. - -## `Information to collect` section - -- The section MUST contain a list of information items required to research the problem. -- Each item MUST be specific enough to be collected or verified from sources. -- Each item MUST be grounded in the formalized problem description or the goals. - -## `Information sources` section - -- The section MUST contain a list of sources that can provide the required information. -- Each source entry MUST include a short identifier and a brief description of the source. -- Each source entry SHOULD include access method, scope, and reliability notes if relevant. -- Each source MUST be relevant to at least one item from **Information to collect**. - -## `Collected information` section - -- The section MUST contain the collected information mapped to the items from **Information to collect**. -- Each collected information item MUST reference one or more source identifiers from **Information sources**. -- The section MUST make it clear which information items are satisfied and which are missing. -- If a required information item cannot be collected, the section MUST state that explicitly and explain why. - -## `Analysis` section - -- The section MUST analyze the collected information along the **Analysis axes**. -- The analysis MUST be organized so each axis can be reviewed independently. -- The analysis MUST clearly separate observed facts from inferences or assumptions. -- The analysis format MUST fit the **Desired form of final solution** and should make downstream synthesis straightforward. - -## `Synthesized solutions` section - -- The section MUST present synthesized solutions or options in a format consistent with the **Synthesis dimensions**. -- Each solution SHOULD reference the analysis items that justify it. -- The synthesis format MUST fit the **Desired form of final solution**. - -## `Evaluation` section - -- The section MUST evaluate each synthesized solution against the **Goals**. -- The evaluation MUST make trade-offs explicit and identify risks or uncertainties. -- The evaluation MUST result in a clear comparison between solutions. - -## `Final solution` section - -- The section MUST present the final solution in the form specified in **Desired form of final solution**. -- The final solution MUST be justified by the evaluation results. -- If the evaluation does not allow a confident final solution, the section MUST state the remaining uncertainties and what additional information would resolve them. diff --git a/.agents/donna/research/work/research.donna.md b/.agents/donna/research/work/research.donna.md deleted file mode 100644 index 79b42f92..00000000 --- a/.agents/donna/research/work/research.donna.md +++ /dev/null @@ -1,198 +0,0 @@ -# Research workflow - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - -Workflow for performing research on a given question/problem/situation. Collects relevant information, analyzes it, synthesizes possible options, and produces an answer/solution/deliverables. - -The purpose of this workflow is to provide an information for making decisions or producing solutions based on researched data. It can be used: - -- When the developer explicitly asks to conduct research. -- When other workflows need to perform research as a subtask. - -## Start - -```toml donna -id = "start" -kind = "donna.lib.request_action" -fsm_mode = "start" -``` - -1. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("../specs/report.donna.md") }}` if you haven't done it yet. -3. `{{ donna.lib.goto("ensure_problem_description_exists") }}` - -## Ensure problem description exists - -```toml donna -id = "ensure_problem_description_exists" -kind = "donna.lib.request_action" -``` - -At this point, you SHOULD have a clear description of the problem in your context. I.e., you know what you need to do in this workflow. - -1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `@/.session/donna/**` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. -3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_artifact") }}`. - -## Prepare research artifact - -```toml donna -id = "prepare_artifact" -kind = "donna.lib.request_action" -``` - -1. Based on the problem description you have, suggest an artifact name in the format `@/.session/donna/research/.donna.md`. `` MUST be unique within the session. -{# TODO: we can add donna.lib.list('@/.session/donna/**') here as the command to list all session artifacts #} -2. Create the artifact and specify an original problem description in it. -3. `{{ donna.lib.goto("formalize_research") }}` - -## Formalize Research - -```toml donna -id = "formalize_research" -kind = "donna.lib.request_action" -``` - -1. Using your knowledge about the project and the original problem description, formulate a format description of the problem -2. Update the artifact with the formalized problem description. -3. `{{ donna.lib.goto("set_primary_section_of_the_research_artifact") }}` - -## Set primary section of the research artifact - -```toml donna -id = "set_primary_section_of_the_research_artifact" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description, update the primary (h1) section of the research artifact to reflect the problem being researched. So the agents can effectively find and understand the research artifact using Donna. -2. `{{ donna.lib.goto("formulate_goals") }}` - -## Formulate Goals - -```toml donna -id = "formulate_goals" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description, formulate a list of goals that the problem solution should achieve. -2. Update the artifact with the list of goals. -3. `{{ donna.lib.goto("fomalize_form_of_final_solution") }}` - -## Formalize form of final solution - -```toml donna -id = "fomalize_form_of_final_solution" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description and the list of goals, formulate the desired form of the final solution. -2. Update the artifact with the desired form of the final solution. -3. `{{ donna.lib.goto("describe_solution_space") }}` - -## Describe solution space - -```toml donna -id = "describe_solution_space" -kind = "donna.lib.request_action" -``` - -To produce a solution, we should understand how to analyze the data and how to synthesize results into the final solution. - -For this, we need to list the axes along which we will analyze the data and the dimensions along which we will synthesize the results. - -1. List the analysis axes. -2. List the synthesis dimensions. -3. Update the artifact with the description of the solution space. -4. `{{ donna.lib.goto("describe_information_to_collect") }}` - -## Describe information to collect - -```toml donna -id = "describe_information_to_collect" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description and the list of goals, list the information to be collected to research the problem. -2. Update the artifact with the description of the information to collect. -3. `{{ donna.lib.goto("list_information_sources") }}` - -## List information sources - -```toml donna -id = "list_information_sources" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description and the list of goals, formulate a list of information sources that can help to research the problem. -2. Update the artifact with the list of information sources. -3. `{{ donna.lib.goto("collect_information") }}` - -## Collect information - -```toml donna -id = "collect_information" -kind = "donna.lib.request_action" -``` - -1. For each information source from the list: - 1. For each piece of required information from the description: - 1. If the source can not provide this piece of information, skip it. - 2. Access the source and collect the required information. - 3. Update the artifact with the collected information. -2. `{{ donna.lib.goto("analyze_information") }}` - -## Analyze information - -```toml donna -id = "analyze_information" -kind = "donna.lib.request_action" -``` - -1. Using the analysis axes from the solution space description, analyze the collected information. Choose the format that best represents the analysis results and suits the required solution form. -2. Update the artifact with the analysis results. -3. `{{ donna.lib.goto("synthesize_solutions") }}` - -## Synthesize solutions - -```toml donna -id = "synthesize_solutions" -kind = "donna.lib.request_action" -``` - -1. Using the synthesis dimensions from the solution space description, synthesize possible solutions based on the analysis results. Choose the format that best represents the synthesized solutions and suits the required solution form. -2. Update the artifact with the synthesized solutions. -3. `{{ donna.lib.goto("evaluate_solutions") }}` - -## Evaluate solutions - -```toml donna -id = "evaluate_solutions" -kind = "donna.lib.request_action" -``` - -1. Evaluate the synthesized solutions against the goals from the formalized problem description. -2. Update the artifact with the evaluation results. -3. `{{ donna.lib.goto("produce_final_solution") }}` - -## Produce final solution - -```toml donna -id = "produce_final_solution" -kind = "donna.lib.request_action" -``` - -1. Based on the evaluation results, produce the final solution in the desired form. -2. Update the artifact with the final solution. -3. `{{ donna.lib.goto("finish") }}` - -## Finish - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -Research workflow completed. Provide the final research output to the developer. diff --git a/.agents/donna/rfc/specs/design.donna.md b/.agents/donna/rfc/specs/design.donna.md deleted file mode 100644 index e5e7f19c..00000000 --- a/.agents/donna/rfc/specs/design.donna.md +++ /dev/null @@ -1,141 +0,0 @@ -# Format of the Design document - -```toml donna -kind = "donna.lib.specification" -``` - -This document describes the format and structure of a Design document used to design changes to a project proposed in a Request for Change (RFC). This document is an input for planning and will be treated as strong recommendations for the implementation of the proposed change. - -## Overview - -Donna introduces a group of workflows located in `../**` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. - -You create a Design document to explicitly describe the exact changes you want to make to the project in order to implement the RFC. - -If not otherwise specified, Design documents for the session MUST be stored as `@/.session/donna/design/.donna.md` artifacts under `/.session/donna`. - -**The Design document MUST list exact changes to the project that will be implemented.** E.g. concrete function names and signatures, file paths, data structures, etc. - -**The Design document MAY skip implementation details.** E.g. it may skip the exact implementation of a function body, but it should specify the function signature and its purpose; It may use pseudocode to describe the logic of a function, but it should not skip describing the logic at all. - -The Design document MUST NOT be a high-level description of the problem and solution. A high-level description of the problem and solution should be provided in the RFC document. - -**Identifiers in this document MUST follow project-specific naming conventions.** Before working on a Design document, you should read the artifacts with project guidelines for naming and code style, if any exist. - -## Design document structure - -The RFC document is Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - -- **Primary section** — title and short description of the proposed change. -- **Inputs** — list of input documents that are relevant for the proposed change, starting from the RFC document. -- **Architecture changes** — list of high-level architectural changes that MUST be implemented. -- **Highlevel code changes** — list of high-level code changes that MUST be implemented: modules to add/modify, functions to add/modify, classes to add/modify, etc. This section should not include low-level implementation details, but it should include enough details to understand the scope of the change and its impact on the project. -- **Data structure changes** — list of changes to data structures that MUST be implemented. -- **Interface changes** — list of changes to interfaces (e.g. function signatures, API endpoints, etc.) that MUST be implemented. -- **Tests changes** — list of autotests that MUST be implemented/updated/removed. Only if the project already uses autotests. -- **Documentation changes** — list of changes in the documentation. Only if the project already has documentation. -- **Other changes** — list of other changes that do not fit into the previous sections, but are still relevant for the proposed change. -- **Order of implementation** — a proposed order of implementation of the changes listed in the previous sections. This section should be treated as a recommendation (from the author of the Design document), not a strict requirement. - -## General language and format - -- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. -- You MUST follow the structure specified in this document. - -### List format - -- If a section is described as a list, it MUST contain only a single markdown list. -- Each list item MUST be concise and clear. -- Each list item SHOULD be atomic and focused on a single aspect. -- Reviewer MUST be able to tell whether the list item statement is true or false by inspecting the resulting artifacts and behavior. - -Examples: - -- Bad: `- Implement a new module.` -- Bad: `- The interface MUST include button A, button B, and dropdown C.` -- Good: `- Implement a module "a.b.c" that is responsible for X, Y, and Z.` -- Good: `- Add a button class "ButtonA" to the module "a.b.c"` -- Good: `- "ButtonA" class MUST have a method "click()" that performs action X when called.` - -Common approaches to improve list items: - -- Split a single item with an enumeration into multiple items with a single focus. -- Transform an abstract item into a concrete one by specifying measurable criteria or user-visible behavior. - -## `Primary` section - -- Title MUST be concise and reflect the essence of the proposed change. Derive it from the RFC. -- Description MUST provide an essence of the proposed change. - -## `Inputs` section - -- The section MUST contain a list of documents that are relevant to the proposed change. -- The first item in the list MUST be the original RFC document that describes the problem and the proposed solution. -- Other items in the list MAY include other documentation, code files, external links, etc., that are relevant for the proposed change. - -## `Architecture changes` section - -- The section MUST contain a free-form but precise and grounded description of the high-level architectural changes that MUST be implemented. -- Along the text, you may add code snippets, diagrams, and other visual aids to make the description clearer and more precise. - -## `High-level code changes` section - -- The section MUST contain a list of high-level changes in the code. -- The level of abstraction: `add a module A`, `remove the class B`, `change the implementation of a function C`. -- The section MUST list only the most important changes that are significant cornerstones of the proposed change. -- The section MAY omit low-level details, such as the small or utilitarian functions, minor refactorings, etc. - -## `Data structure changes` section - -- The section MUST list exact changes to data structures that MUST be implemented. -- Each change MUST be accompanied by a description of the purpose of the change and its impact on the project. - -Examples of statements about structure changes: - -- Bad: `- Change the data structure of the project.` -- Bad: `- Update the class A.` -- Good: `- Add a field "x" to the class "A".` -- Good: `- Change the type of the field "y" in the class "B" from "int" to "str".` -- Good: `- Add structure "C" with fields "a", "b", and "c".` - -## `Interface changes` section - -- The section MUST list exact changes to interfaces that MUST be implemented. -- Each change MUST be accompanied by a description of the purpose of the change and its impact on the project. - -Examples of statements about interface changes: - -- Bad: `- Change the interface of functions in the project.` -- Bad: `- Update the API endpoints.` -- Good: `- Add a new API endpoint "/api/v1/resource" that accepts POST requests with JSON body containing fields "a", "b", and "c".` -- Good: `- Change the signature of the function "foo" from "foo(x: int) -> str" to "foo(x: int, y: str) -> str".` - -## `Tests changes` section - -- If the project does not use autotests, this section MUST contain a statement `No changes in tests are required, since the project does not use autotests.` -- If the project uses autotests, this section MUST contain a list of autotests that MUST be implemented/updated/removed. -- Each changes piece of logic MUST have at least one corresponding autotest that verifies its correctness and prevents regressions in the future. -- Each added/updated branch of logic MUST have at least one corresponding autotest that verifies its correctness and prevents regressions in the future. - -Examples of statements about test changes: - -- Bad: `- Add tests for the new module.` -- Bad: `- Update tests for the changed function.` -- Good: `- Add a test "test_foo_with_valid_input" that verifies that the function "foo" returns the expected result when called with valid input.` -- Good: `- Add a test "test_foo_success_path" that verifies that the function "foo" returns the expected result when called with input that follows the success path.` - -## `Documentation changes` section - -- If the project does not have documentation, this section MUST contain a statement `No changes in documentation are required, since the project does not have documentation.` -- If the project has documentation, this section MUST contain a list of changes in the documentation that MUST be implemented. - -## `Other changes` section - -- The section MAY contain a list of other changes that do not fit into the previous sections, but are still relevant for the proposed change. -- The section MUST be a single statement: `No other changes are required, since all relevant changes are covered in the previous sections.` if there are no other changes to mention. - -## `Order of implementation` section - -- The section MUST contain a proposed order of implementation of the changes listed in the previous sections. -- The section MUST refer only to identifiers mentioned in the previous sections, and it MUST NOT introduce new identifiers or entities that are not mentioned in the previous sections. diff --git a/.agents/donna/rfc/specs/request_for_change.donna.md b/.agents/donna/rfc/specs/request_for_change.donna.md deleted file mode 100644 index b5e1e92f..00000000 --- a/.agents/donna/rfc/specs/request_for_change.donna.md +++ /dev/null @@ -1,266 +0,0 @@ -# Format of the Request for Change document - -```toml donna -kind = "donna.lib.specification" -``` - -This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `../**` namespace. This document is an input for a Design document creation. - -## Overview - -Donna introduces a group of workflows located in `../**` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. - -You create RFC documents to propose changes to the project. - -If not otherwise specified, RFC documents for the session MUST be stored as `@/.session/donna/rfc/.donna.md` artifacts under `/.session/donna`. - -## RFC structure - -The RFC document is Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - -- **Primary section** — title and short description of the proposed change. -- **Original description** — original description of the requested changes from the developer or parent workflow. -- **Formal description** — formal description of the requested changes. -- **Goals** — list of goals that the proposed change aims to achieve. -- **Objectives** — list of objectives that need to be accomplished to achieve the goals. -- **Constraints** — list of constraints that must be considered while implementing the proposed change. -- **Requirements** — list of requirements that the proposed change must fulfill. -- **Acceptance criteria** — list of criteria that define when the proposed change is considered complete and successful. -- **Solution** — list of statements about how the final result should look like. -- **Verification** — list of statements about how to verify each objective, constraint, requirement, and acceptance criterion. -- **Deliverables** — list of deliverables that should be produced as part of the proposed change. -- **Action items** — unordered list of atomic actions/changes that must be performed to implement the proposed change. - -## General language and format - -- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. -- You MUST follow the structure specified in this document. - -### List format - -- If a section is described as a list, it MUST contain only a single markdown list. -- Each list item MUST be concise and clear. -- Each list item SHOULD be atomic and focused on a single aspect. -- Reviewer MUST be able to tell whether the list item statement is true or false by inspecting the resulting artifacts and behavior. - -Examples: - -- Bad: `- Improve performance and fix bugs.` -- Bad: `- The interface MUST include button A, button B, and dropdown C.` -- Good: `- Performance test MUST show at least 20% improvement in response time under load.` -- Good: `- Fix bug A causing a crash when input is empty.` -- Good: `- The interface MUST include button A that triggers action X.` -- Good: `- The interface MUST include button B that triggers action Y.` - -Common approaches to improve list items: - -- Split a single item with an enumeration into multiple items with a single focus. -- Transform an abstract item into a concrete one by specifying measurable criteria or user-visible behavior. - -### Trusted inputs - -Some sections of the RFC document MUST be based on trusted inputs. Trusted inputs are: - -- Original description from the developer or parent workflow. -- Statements from the RFC document itself. -- Existing project documentation, code, and artifacts. -- External standards, when they define constraints or best practices for the project domain. -- Documentation of third-party libraries, frameworks, or tools when they describe constraints or best practices. - -## `Primary` section - -- Title MUST be concise and reflect the essence of the proposed change. -- Description MUST provide a brief overview of the proposed change, its purpose, and its expected impact. - -## `Original description` section - -- This section MUST contain the original request from the developer or from the parent workflow. -- The request MUST NOT be modified by agents. - -## `Formal description` section - -- The section MUST contain a clear professional high-level description of the work to be done based on the developer's request. -- The section MUST be limited to a single paragraph with a few sentences. -- The section MUST explain what someone gains after these changes and how they can see it working. - State the user-visible behavior the change will enable. - -## `Goals` section - -- This section MUST contain a list of goals that the proposed change aims to achieve. - -The goal quality criteria: - -- A goal MUST be a desired end state, outcome, or result. -- A goal MUST define what ultimately should be true, not how to achieve it. - -Examples: - -- Bad: `- Implement authentication system.` -- Good: `- Protect user data from unauthorized access.` - -## `Objectives` section - -- The section MUST contain a list of objectives that need to be completed to achieve the goals. -- Each goal MUST have a set of objectives that, when all achieved, ensure the goal is met. -- Each goal MUST have 2–6 objectives, unless the goal is demonstrably trivial (≤1 artifact, no dependencies). - -Objective quality criteria: - -- An objective MUST be a specific, measurable condition that must be true for a goal to be satisfied. -- An objective MUST describe an achieved state, capability, artifact, or behavior. - -Examples: - -- Bad: `- Create user authentication and authorization system.` -- Bad: `- Design user authentication flow.` -- Good: `- Specification for user authentication flow exists.` -- Good: `- User is able to log in using email and password.` - -## `Constraints` section - -- The section MUST contain a list of constraints that the changes MUST respect. -- The section MAY be empty only if no constraints are explicitly known. -- A constraint MUST be derived from trusted inputs. Agents MUST NOT invent constraints. - -Constraint quality criteria: - -- A constraint MUST be a hard limit on the solution space. -- A constraint MUST be externally imposed by technology, policy, compatibility, time, budget, etc. -- A constraint MUST NOT describe desired behavior or outcomes. -- A constraint MUST be non-negotiable within the scope of the RFC. - -Examples: - -- Bad: `- The system should be easy to maintain.` -- Bad: `- Use clean architecture.` -- Good: `- The solution MUST be compatible with Python 3.12.` -- Good: `- The solution MUST NOT introduce new runtime dependencies.` -- Good: `- The solution MUST follow the specification ../../../../specs/abc.donna.md` -- Good: `MUST not change public CLI flags` - -## `Requirements` section - -- The section MUST contain a list of requirements that the proposed change MUST fulfill. - -Requirement quality criteria: - -- A requirement MUST be a single atomic condition, capability, or feature that the system MUST meet, provide, or exhibit after the change is implemented. -- A requirement MUST be directly testable. -- A requirement MUST be stated independently of implementation details. -- A requirement MUST NOT restate goals or objectives verbatim. - -Examples: - -- Bad: `- Improve security.` -- Bad: `- Implement OAuth.` -- Good: `- The system MUST reject authentication attempts with invalid credentials.` -- Good: `- The system MUST log all failed authentication attempts with timestamp and user identifier.` - -## `Acceptance criteria` section - -- The section MUST contain a list of acceptance criteria used to determine whether the proposed change is complete and successful. -- An acceptance criterion MUST be derived explicitly from statements in the RFC document. Agents MUST NOT invent acceptance criteria. - -Acceptance criteria quality criteria: - -- An acceptance criterion MUST be a single atomic check that results in a pass/fail outcome. -- An acceptance criterion check MUST be about a single artifact: a single file exists, a single test passes, a single behavior observed, etc. -- An acceptance criterion MUST NOT describe implementation steps, internal design decisions, or "how" to achieve the result. - -Examples: - -- Bad: `- All requirements are implemented.` -- Bad: `- The feature works as expected.` -- Good: `- The artifact with documentation for X exists.` -- Good: `- The autotest for requirement Y exists.` -- Good: `- All autotests pass without errors.` -- Good: `- The tool can run on Python 3.12 without errors.` - -## `Solution` section - -- The section MUST contain a list of statements describing how the system should look like/behave after the proposed changes are implemented. -- The section MUST NOT establish an order of implementation steps. -- The full solution MUST ensure the truth/validity of all statements in the RFC. - -Solution statement quality criteria: - -- A solution statement MUST be a specific change in a specific artifact, behavior, or capability. -- A solution statement MUST NOT prescribe implementation steps. -- A solution statement SHOULD be phrased in the present tense, describing an achieved state. - -Examples: - -- Bad: `- First implement the database schema, then add API endpoints.` -- Bad: `- Use framework X to handle authentication.` -- Good: `- The system exposes an authentication API that accepts credentials and returns an access token on success.` -- Good: `- User-facing documentation describes how to configure and use the authentication feature.` - -## `Verification` section - -- The section MUST contain a list of checks that MUST be passed to prove that the work is complete and correct. -- Each verification statement MUST map to a single item from **Objectives**, **Constraints**, **Requirements**, or **Acceptance criteria**. -- Each **Objective**, **Constraint**, **Requirement**, or **Acceptance criterion** MUST map to a single verification statement. -- Each verification statement MUST describe *how* the corresponding item can be verified. - -Verification quality criteria: - -- A verification statement MUST be a concrete verification step or check. -- A verification statement MUST be automatable if it is possible. -- A verification statement SHOULD be verifiable by agents without human intervention. -- A verification statement MUST result in a boolean outcome (verified / not verified). -- A verification statement MUST reference specific artifacts, commands, tests, or observable behavior. - -Examples: - -- Bad: `- Verify that authentication works correctly.` -- Bad: `- Review the implementation manually.` -- Good: `- Run test suite `tests/auth/test_login.py`; all tests MUST pass.` -- Good: `- Inspect artifact `../../../../specs/authentication.donna.md`; it MUST exist and contain section "Login flow".` -- Good: `- Execute CLI command `tool login` with invalid credentials; command MUST exit with non-zero code.` - -## `Deliverables` section - -- The section MUST contain a list of concrete artifacts that MUST exist after the changes are implemented. -- A deliverable MUST be derived from trusted inputs. Agents MUST NOT invent deliverables. - -Deliverable quality criteria: - -- A deliverable MUST be a tangible, single artifact or a single part of an artifact, not an activity or process. -- A deliverable MUST be independently identifiable and verifiable. -- Each deliverable MUST be phrased as an existence statement using normative - language: "MUST include …", "MUST exist …", etc. -- Explicitly add source files as deliverables only when the task is specifically - about creating or modifying those files (e.g., "MUST add docs/cli.md …"). - -Examples: - -- Bad: `- Implement authentication code` -- Bad: `- Refactor auth module.` -- Good: `- Module app/auth/authentication.py exists.` -- Good: `- Donna artifact ../../../../specs/authentication.donna.md exists.` -- Good: `- Test suite tests/auth/ exists.` - -## `Action items` section - -- The section MUST contain an unordered list of atomic actions. - -Action item quality criteria: - -- An action item MUST be a single change that can be applied or executed. -- An action item MUST be small enough to be completed without further decomposition. -- An action item MUST be phrased as an imperative sentence. -- An action item MUST affect a specific artifact, file, config, etc. -- An action item MUST reference concrete paths, identifiers, or commands. -- An action item MUST be actionable by an agent. - -Examples: - -- Bad: `- Work on authentication.` -- Bad: `- Improve security everywhere.` -- Bad: `- Fix the bugs A` -- Good: `- Create an artifact ../../../../specs/authentication.donna.md with sections "Login flow" and "Token lifecycle".` -- Good: `- Add test file tests/auth/test_login.py covering invalid credential cases.` -- Good: `- Implement test tests/auth/test_login.py:TestLogin:test_invalid_credentials.` -- Good: `- Update CLI help text to include login command description.` -- Good: `- Implement function app/auth/authentication.py:authenticate_user().` diff --git a/.agents/donna/rfc/work/design.donna.md b/.agents/donna/rfc/work/design.donna.md deleted file mode 100644 index db356d55..00000000 --- a/.agents/donna/rfc/work/design.donna.md +++ /dev/null @@ -1,121 +0,0 @@ -# Create a Design document - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - -This workflow creates a Design document artifact based on an RFC and aligned with `../specs/design.donna.md`. - -## Start Work - -```toml donna -id = "start" -kind = "donna.lib.request_action" -fsm_mode = "start" -``` - -1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. -3. `{{ donna.lib.goto("ensure_rfc_artifact_exists") }}` - -## Ensure RFC artifact exists - -```toml donna -id = "ensure_rfc_artifact_exists" -kind = "donna.lib.request_action" -``` - -At this point, you SHOULD have a clear RFC to design. - -1. If you have an RFC artifact id in your context, view it and `{{ donna.lib.goto("prepare_design_artifact") }}`. -2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("@/.session/donna/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. -3. If you have no RFC artifact id in your context, and you don't know where it is, ask the developer to provide the RFC artifact id or to create a new RFC. After you get it and view the artifact, `{{ donna.lib.goto("prepare_design_artifact") }}`. - -## Prepare Design artifact - -```toml donna -id = "prepare_design_artifact" -kind = "donna.lib.request_action" -``` - -1. If the name of the artifact is not specified explicitly, assume it to be `@/.session/donna/design/.donna.md`, where `` SHOULD correspond to the RFC slug. -2. Save the next template into the artifact, replace `` with appropriate values. - -~~~ -# - -```toml donna -kind = "donna.lib.specification" -``` - -<short description of the proposed design> - -## Inputs - -- <RFC artifact id> - -## Architecture changes - -## High-level code changes - -## Data structure changes - -## Interface changes - -## Tests changes - -## Documentation changes - -## Other changes - -## Order of implementation -~~~ - -3. `{{ donna.lib.goto("initial_fill") }}` - -## Initial Fill - -```toml donna -id = "initial_fill" -kind = "donna.lib.request_action" -``` - -1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. -2. Read the RFC artifact selected in the previous step if you haven't done it yet. -3. Analyze the project if needed to understand the requested change context. -4. Fill in all sections of the Design draft artifact. -5. Ensure the first item in `Inputs` section is the RFC artifact id. -6. `{{ donna.lib.goto("review_design_format") }}` - -## Review Design Format - -```toml donna -id = "review_design_format" -kind = "donna.lib.request_action" -``` - -1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("../specs/design.donna.md") }}`. -2. For each mismatch, make necessary edits to the Design draft artifact to ensure compliance. -3. `{{ donna.lib.goto("review_design_content") }}` - -## Review Design Content - -```toml donna -id = "review_design_content" -kind = "donna.lib.request_action" -``` - -1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. -2. Make necessary edits to the Design draft artifact to address identified issues. -3. If there were changes made on this step or the previous `review_design_format` step `{{ donna.lib.goto("review_design_format") }}`. -4. If no changes were made, `{{ donna.lib.goto("finish") }}`. - -## Complete Draft - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -Design draft is complete and ready for implementation planning. diff --git a/.agents/donna/rfc/work/do.donna.md b/.agents/donna/rfc/work/do.donna.md deleted file mode 100644 index b97cf0ff..00000000 --- a/.agents/donna/rfc/work/do.donna.md +++ /dev/null @@ -1,121 +0,0 @@ - -# Do Request For Change work - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "estimate_complexity" -``` - -This workflow uses a description of a problem or changes required from the developer or parent workflow to: - -1. Create a Request for Change (RFC) artifact. -2. Create a Design document artifact based on the RFC. -3. Plan the work required to implement the designed changes. -4. Execute the planned work. - -## Estimate complexity - -```toml donna -id = "estimate_complexity" -kind = "donna.lib.request_action" -``` - -1. If the developer or parent workflow explicitly specified that the work is complex, `{{ donna.lib.goto("create_rfc") }}`. -2. If the developer or parent workflow explicitly specified that the work is simple, `{{ donna.lib.goto("fast_planning") }}`. -3. Make a rough estimate of the complexity of the requested changes. -4. If the complexity is small, `{{ donna.lib.goto("fast_planning") }}`. -5. Otherwise, `{{ donna.lib.goto("create_rfc") }}`. - -Examples of changes with small complexity: - -- A single file edit with clear instructions. -- A small addition to an existing function or class. -- A minor configuration change. -- A small documentation update. -- A simple bug fix with a known solution. - -## Fast planning - -```toml donna -id = "fast_planning" -kind = "donna.lib.request_action" -``` - -1. Draft a simple plan to implement the requested changes directly, without creating a formal RFC artifact. -2. `{{ donna.lib.goto("plan_rfc_work") }}`. - -## Create RFC - -```toml donna -id = "create_rfc" -kind = "donna.lib.request_action" -fsm_mode = "start" -``` - -1. Choose a workflow to create an RFC artifact. -2. Run the chosen workflow. -3. After completing the workflow `{{ donna.lib.goto("design") }}`. - -## Design - -```toml donna -id = "design" -kind = "donna.lib.request_action" -``` - -1. Choose a workflow to create a Design document artifact based on the RFC created in the previous step. -2. Run the chosen workflow. -3. After completing the workflow `{{ donna.lib.goto("plan_rfc_work") }}`. - -## Plan RFC work - -```toml donna -id = "plan_rfc_work" -kind = "donna.lib.request_action" -``` - -1. Choose the workflow to plan the work. If you created a Design document in the previous step, use it as a basis. -2. Run the chosen workflow. -3. Ensure you know the workflow id created in the previous step (default is `@/.session/donna/execute_rfc.donna.md` if not specified). -4. After completing the workflow `{{ donna.lib.goto("execute_rfc_work") }}`. - -## Execute RFC work - -```toml donna -id = "execute_rfc_work" -kind = "donna.lib.request_action" -``` - -1. Run the workflow created by the plan step (default: `@/.session/donna/execute_rfc.donna.md`) and complete it. -2. After completing the workflow `{{ donna.lib.goto("polish_changes") }}`. - -## Polish changes - -```toml donna -id = "polish_changes" -kind = "donna.lib.request_action" -``` - -1. Find the workflow to polish the changes made. -2. Run the chosen workflow if you found one. -3. `{{ donna.lib.goto("log_changes") }}`. - -## Log changes - -```toml donna -id = "log_changes" -kind = "donna.lib.request_action" -``` - -1. Find the workflow to log the changes made in the scope of this RFC. -2. Run the chosen workflow if you found one. -3. `{{ donna.lib.goto("finish") }}`. - -## Finish - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -RFC workflow execution completed. Report the outcome to the developer. diff --git a/.agents/donna/rfc/work/plan.donna.md b/.agents/donna/rfc/work/plan.donna.md deleted file mode 100644 index 20044c33..00000000 --- a/.agents/donna/rfc/work/plan.donna.md +++ /dev/null @@ -1,76 +0,0 @@ - -# Plan work on base of Design & Request for Change documents - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - -This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow stored as a `@/.session/donna/**` artifact under `<project-root>/.session/donna` with detailed steps to implement the designed changes. - -## Start Work - -```toml donna -id = "start" -kind = "donna.lib.request_action" -fsm_mode = "start" -``` - -1. Read the Design document that the developer or parent workflow wants you to implement. -2. Read the RFC document that the developer or parent workflow wants you to implement, if it exists. -3. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. -4. `{{ donna.lib.goto("prepare_workflow_artifact") }}` - -## Prepare workflow artifact - -```toml donna -id = "prepare_workflow_artifact" -kind = "donna.lib.request_action" -``` - -1. If the name of the artifact is not specified explicitly, assume it to `@/.session/donna/plans/<short-id-equal-to-design-slug>.donna.md`. -2. Create a workflow with the next operations: - - Start - - A step for each action point in the RFC document and each item in the `Order of implementation` in Design document with the goal to minimize dependencies between steps and introduce changes incrementally. - - Add additional steps if the design requires it. - - (`id="review_changes"`) A gate step to start reviewing the changes. - - A gate step to start reviewing the deliverables. - - A step per deliverable to check if it exists and is correct. If the deliverable is missing or incorrect, the step MUST specify how to fix it and `goto` to the `review_changes` step. - - A gate step to start verification. - - A step for each verification statement to check it. If the statement is not satisfied, the step MUST specify how to fix it and `goto` to the `review_changes` step. - - A finish step. -3. `{{ donna.lib.goto("review_workflow") }}` - -## Review workflow - -```toml donna -id = "review_workflow" -kind = "donna.lib.request_action" -``` - -For each of the steps in the workflow you created: - -1. If the step can fail, add instructions on how to fix the issue and `goto` to the appropriate step. -2. If the step requires both research and implementation, split it into two steps: research step and implementation step. -3. If the step required editing multiple artifacts (multiple files, multiple functions, etc.), split it into multiple steps, one per change required. -4. If the step is too big or complex, split it into multiple smaller steps. -5. Fux & improve naming and IDs in the step's title and body. - -Naming rules: - -- A title of operation MUST always have meaning without knowing context: no sequential numbers, no generic titles like `Plan part 1`. -- If you mention an ID from the RFC or the Design document, you MUST include a human-readable name of the entity you refer. E.g. not `O1` but `O1 (Implement a new module "a.b.c")` - -After all steps are reviewed: - -1. If the changes were introduced, `{{ donna.lib.goto("review_workflow") }}`. -2. If no changes were introduced, `{{ donna.lib.goto("finish") }}`. - -## Finish - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -Work plan finalized. Ready to execute the planned workflow. diff --git a/.agents/donna/rfc/work/request.donna.md b/.agents/donna/rfc/work/request.donna.md deleted file mode 100644 index b5f2a0f1..00000000 --- a/.agents/donna/rfc/work/request.donna.md +++ /dev/null @@ -1,124 +0,0 @@ - -# Create a Request for Change - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - -This workflow creates a Request for Change (RFC) document based on a description of the problem or the required changes. - -## Start Work - -```toml donna -id = "start" -kind = "donna.lib.request_action" -fsm_mode = "start" -``` - -1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. -3. `{{ donna.lib.goto("ensure_work_description_exists") }}` - -## Ensure work description exists - -```toml donna -id = "ensure_work_description_exists" -kind = "donna.lib.request_action" -``` - -At this point, you SHOULD have a clear description of the problem in your context. I.e. you know what you need to do in this workflow. - -1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("@/.session/donna/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. -3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. - -## Prepare RFC artifact - -```toml donna -id = "prepare_rfc_artifact" -kind = "donna.lib.request_action" -``` - -1. If the name of the artifact is not specified explicitly, assume it to be `@/.session/donna/rfc/<short-problem-related-identifier>.donna.md`, where `<short-problem-related-identifier>` MUST be unique within the session. -2. Save the next template into the artifact, replace `<variables>` with appropriate values. - -~~~ -# <Title> - -```toml donna -kind = "donna.lib.specification" -``` - -<short description of the proposed change> - -## Original description - -<original description of the requested changes from the developer or parent workflow> - -## Formal description - -## Goals - -## Objectives - -## Constraints - -## Requirements - -## Acceptance criteria - -## Solution - -## Verification - -## Deliverables - -## Action items -~~~ - -3. `{{ donna.lib.goto("initial_fill") }}` - -## Initial Fill - -```toml donna -id = "initial_fill" -kind = "donna.lib.request_action" -``` - -1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. -2. Analyze the project if needed to understand the context of the requested change. -3. Based on the problem description you have, fill in all sections of the RFC draft artifact. -4. `{{ donna.lib.goto("review_rfc_format") }}` - -## Review RFC Format - -```toml donna -id = "review_rfc_format" -kind = "donna.lib.request_action" -``` - -1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}`. -2. For each mismatch, make necessary edits to the RFC draft artifact to ensure compliance with the RFC specification. -3. `{{ donna.lib.goto("review_rfc_content") }}` - -## Review RFC Content - -```toml donna -id = "review_rfc_content" -kind = "donna.lib.request_action" -``` - -1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. -2. Make necessary edits to the RFC draft artifact to address identified issues. -3. If there were changes made on this step or the previous `review_rfc_format` step `{{ donna.lib.goto("review_rfc_format") }}`. -4. If no changes were made, `{{ donna.lib.goto("finish") }}`. - -## Complete Draft - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -RFC draft is complete and ready for planning or review. diff --git a/.agents/donna/usage/artifacts.donna.md b/.agents/donna/usage/artifacts.md similarity index 85% rename from .agents/donna/usage/artifacts.donna.md rename to .agents/donna/usage/artifacts.md index ac7181ba..ace1f00c 100644 --- a/.agents/donna/usage/artifacts.donna.md +++ b/.agents/donna/usage/artifacts.md @@ -1,9 +1,5 @@ # Default Text Artifacts Behavior -```toml donna -kind = "donna.lib.specification" -``` - This document describes the default format and behavior of Donna's text artifacts. This format and behavior is what should be expected by default from an artifact if not specified otherwise. @@ -22,7 +18,7 @@ To get information from the artifact, developers, agents and Donna view one of i **If you need an information from the artifact, you MUST view its representation**. Artifact sources are only for editing. -Read the specification `{{ donna.lib.view("./cli.donna.md") }}` to learn how to work with artifacts via Donna CLI. +Read the `./cli.md` file to learn how to work with artifacts via Donna CLI. ## Source Format and Rendering @@ -32,12 +28,10 @@ When rendering the artifact, Donna processes the Jinja2 template with a predefin **Artifact source should not use Jinja2 inheritance features** like `{{ "{% extends %}" }}` and `{{ "{% block %}" }}`. -Donna provides a set of special directives that can and MUST be used in the artifact source to enhance its behavior. Some of these directives are valid for all artifacts, some are valid only for specific section kinds. +Donna provides a set of special directives that can and MUST be used in workflow artifact sources to enhance their behavior. Some of these directives are valid for all artifacts, some are valid only for specific section kinds. Here are some examples: -- `{{ "{{ donna.lib.view(<artifact-id-pattern>) }}" }}` — references another artifact (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to view the artifact; in `analysis` mode it renders a `$$donna ... $$` marker used for internal parsing. -- `{{ "{{ donna.lib.list(<artifact-id-pattern>) }}" }}` — references artifact listing by pattern (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to list artifacts; in `analysis` mode it renders a `$$donna ... $$` marker used for internal parsing. - `{{ "{{ donna.lib.goto(<workflow-operation-id>) }}" }}` — references the next workflow operation to execute. In `view`/`execute` modes it renders an exact CLI command to advance the workflow; in `analysis` mode it renders a `$$donna goto ... $$` marker used to extract workflow transitions. ## Jinja2 rendering @@ -117,27 +111,24 @@ Artifacts can include semantic tags via a `tags` field in the section configurat Tags are used for deterministic artifact filtering and discovery (for example, via `donna -p <protocol> artifacts list ... --predicate '"workflow" in section.tags'`). Tags are typically attached to the primary section and describe the artifact as a whole. -The canonical list of standard tags is documented in `../intro.donna.md`. +The canonical list of standard tags is documented in `../intro.md`. ## Section Kinds, Their Formats and Behaviors ### Header section -Header section MUST contain a config block with a `kind` property. The `kind` MUST be a full Python import path pointing to the primary section kind instance. +Donna artifact header sections MUST contain a config block with a `kind` property. The `kind` MUST be a full Python import path pointing to the primary section kind instance. Example (`donna` keyword skipped for examples): ```toml -kind = "donna.lib.specification" +kind = "donna.lib.workflow" +start_operation_id = "start_operation" ``` Header section MUST also contain short human-readable description of the artifact outside of the config block. -### Kind: Specification - -Specification artifacts describe various aspects of the project in a structured way. - -Currently there is no additional structure or semantics for this kind of artifact. +Plain Markdown documentation does not need a Donna config block and should use the `.md` extension. ### Kind: Workflow @@ -265,7 +256,5 @@ Donna provides multiple directives that MUST be used in the artifact source to e Here they are: -1. `{{ "{{ donna.lib.view(<artifact-id-pattern>) }}" }}` — references another artifact (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to view the artifact; in `analysis` mode it renders a `$$donna ... $$` marker. -2. `{{ "{{ donna.lib.list(<artifact-id-pattern>) }}" }}` — references artifact listing by pattern (supports `*` and `**` wildcard patterns). In `view`/`execute` modes it renders an exact CLI command to list artifacts; in `analysis` mode, it renders a `$$donna ... $$` marker. -3. `{{ "{{ donna.lib.goto(<workflow-operation-id>) }}" }}` — references the next workflow operation to execute. In `view`/`execute` modes it renders an exact CLI command to advance the workflow; in `analysis` mode, it renders a `$$donna goto ... $$` marker used for transition extraction. -4. `{{ "{{ donna.lib.task_variable(<variable_name>) }}" }}` — in `view` mode renders a placeholder note about task-variable substitution, in `execute` mode renders the actual task-context value (or an explicit error marker if missing), and in `analysis` mode renders a `$$donna task_variable ... $$` marker. +1. `{{ "{{ donna.lib.goto(<workflow-operation-id>) }}" }}` — references the next workflow operation to execute. In `view`/`execute` modes it renders an exact CLI command to advance the workflow; in `analysis` mode, it renders a `$$donna goto ... $$` marker used for transition extraction. +2. `{{ "{{ donna.lib.task_variable(<variable_name>) }}" }}` — in `view` mode renders a placeholder note about task-variable substitution, in `execute` mode renders the actual task-context value (or an explicit error marker if missing), and in `analysis` mode renders a `$$donna task_variable ... $$` marker. diff --git a/.agents/donna/usage/cli.donna.md b/.agents/donna/usage/cli.md similarity index 95% rename from .agents/donna/usage/cli.donna.md rename to .agents/donna/usage/cli.md index d1527174..22854b50 100644 --- a/.agents/donna/usage/cli.donna.md +++ b/.agents/donna/usage/cli.md @@ -1,9 +1,5 @@ # Donna Usage Instructions -```toml donna -kind = "donna.lib.specification" -``` - This document describes how agents MUST use Donna CLI to manage and perform their workflows. **Agents MUST follow the instructions and guidelines outlined in this document precisely.** @@ -72,7 +68,7 @@ Donna renders cells differently, depending on the protocol used. There are three sets of commands: -- `donna -p <protocol> workspaces …` — initializes and refreshes Donna project files. Most likely it will be used once per project. +- `donna -p <protocol> workspaces …` — initializes Donna project files. Most likely it will be used once per project. - `donna -p <protocol> sessions …` — manages sessions. You will use these commands to start, push forward, and manage your work. - `donna -p <protocol> artifacts …` — manages artifact discovery, reading, and validation. @@ -85,7 +81,7 @@ Use: Run `donna -p <protocol> workspaces init [<directory-path>]` to initialize Donna in the given directory. If `<directory-path>` is omitted, Donna will initialize the current working directory. -It is a one time operation you need to perform once per project to create `donna.toml`, the configured session directory, and synced Donna fixtures. +It is a one time operation you need to perform once per project to create `donna.toml` and the configured session directory. ### Starting sessions @@ -152,11 +148,11 @@ The format of `<artifact-pattern>` is as follows: - full artifact identifier: `@/...` - `*` — single wildcard matches a single level in the rooted artifact path. Examples: - - `*/intro.donna.md` — matches all artifacts with filename `intro.donna.md` exactly one directory below the project root. - - `@/*/intro.donna.md` — equivalent full form. + - `*/work.donna.md` — matches all artifacts with filename `work.donna.md` exactly one directory below the project root. + - `@/*/work.donna.md` — equivalent full form. - `**` — double wildcard matches multiple levels in the rooted artifact path. Examples: - `**/name.donna.md` — matches all artifacts with filename `name.donna.md` anywhere in the Donna project. - - `@/**/intro.donna.md` — equivalent full form. + - `@/**/name.donna.md` — equivalent full form. - `@/.session/donna/**` — matches all artifacts under the default configured session directory. CLI arguments MUST NOT use relative artifact paths like `./...` or `../../...`; use absolute `@/...` paths or rooted wildcard forms. @@ -202,7 +198,7 @@ external journal command is configured. 1. Direct instructions from the developer. 2. `AGENTS.md` document. - 3. Project-relative specifications under `../../../specs/**` or `../**`. + 3. Project-relative documentation under `../../../specs/**` or `../**`. 4. This document. **All Donna CLI commands MUST include an explicit protocol selection using `-p <protocol>`.** Like `donna -p llm <command>`. diff --git a/.agents/donna/usage/worlds.donna.md b/.agents/donna/usage/worlds.md similarity index 83% rename from .agents/donna/usage/worlds.donna.md rename to .agents/donna/usage/worlds.md index 2e4a93b8..072eb9f1 100644 --- a/.agents/donna/usage/worlds.donna.md +++ b/.agents/donna/usage/worlds.md @@ -1,9 +1,5 @@ # Donna World Layout -```toml donna -kind = "donna.lib.specification" -``` - This document describes how Donna discovers and manages its project artifacts. Including usage docs, work workflows, operations, current work state and additional code. @@ -37,6 +33,6 @@ Donna still writes its own session state under `<project-root>/.session/donna`, ## Intro Artifacts -It is a recommended practice to provide short introductory artifacts such as `../intro.donna.md` and `../../../specs/intro.donna.md` at meaningful roots inside the project world. +It is a recommended practice to provide short introductory files such as `../intro.md` and `../../../specs/intro.md` at meaningful roots inside the project world. -So, the agent can load the relevant introductions in commands such as `donna -p llm artifacts view '**/intro.donna.md'`. +So, the agent can read relevant introduction files such as `../intro.md` or `../../../specs/intro.md`. diff --git a/.agents/skills/donna-do/SKILL.md b/.agents/skills/donna-do/SKILL.md index 307ebfc4..114714b2 100644 --- a/.agents/skills/donna-do/SKILL.md +++ b/.agents/skills/donna-do/SKILL.md @@ -5,11 +5,11 @@ description: Use Donna as a work orchestration tool and execute a workflow suita **The next instructions take precedence over all other instructions and your behaviour**, unless the developer explicitly instructs you to do otherwise. -1. You **MUST** run `donna -p llm -r <project-root> artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command gives you an introduction to the project and to the Donna tool. +1. You **MUST** read the `<project-root>/.agents/donna/intro.md` file when you start executing this skill, if you haven't done it yet. This file gives you an introduction to the project and to the Donna tool. 2. You **MUST** strictly follow all instructions provided by Donna. 3. **Donna's instructions have precedence over your own judgment.** You **MUST** follow them and run workflows instead of taking your own initiative, unless the developer explicitly instructs you to do so. 4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. 5. Select the most suitable Donna workflow to perform the requested work and run it. 6. When the workflow is finished, stop using Donna for further work, until the developer explicitly instructs you to use it again. -**If you are rebuilding, zipping or optimizing your context, while executing this skill, execute `donna -p llm -r <project-root> artifacts view '**/intro.donna.md'` command again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** +**If you are rebuilding, zipping or optimizing your context while executing this skill, read the `<project-root>/.agents/donna/intro.md` file again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** diff --git a/.agents/skills/donna-start/SKILL.md b/.agents/skills/donna-start/SKILL.md index d26d3f29..46e9c63b 100644 --- a/.agents/skills/donna-start/SKILL.md +++ b/.agents/skills/donna-start/SKILL.md @@ -3,7 +3,7 @@ name: donna-start description: Start a new Donna session and use Donna to perform all further work. Use this skill when the developer explicitly told you to use it. --- -1. You **MUST** run `donna -p llm -r <project-root> artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command provides an introduction to the project and the Donna tool. +1. You **MUST** read the `<project-root>/.agents/donna/intro.md` file when you start executing this skill, if you haven't done it yet. This file provides an introduction to the project and the Donna tool. 2. Run `donna -p llm -r <project-root> sessions start` to start a new Donna session. 3. Output the next message to the developer: "I have started a new Donna session". 4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. diff --git a/AGENTS.md b/AGENTS.md index f0ba107c..df47c7d7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,7 @@ Since this is the repository that contains the Donna project itself, you have di In all commands that use `donna`, you MUST replace `donna` with `./bin/donna.sh` when you run the command. -For example, instead of `donna artifacts view '*:intro'` you MUST run `./bin/donna.sh artifacts view '*:intro'`. +For example, instead of `donna artifacts list` you MUST run `./bin/donna.sh artifacts list`. ## Top priority tools diff --git a/README.md b/README.md index 0b58edfb..beb3ba55 100644 --- a/README.md +++ b/README.md @@ -34,15 +34,15 @@ Donna executes such loops for the agents, thereby saving time, context, and toke - **Deterministic workflows** — define fixed & validated control flow for agents to follow. - **Saves context, tokens and time** — agents do not need to think when thinking is not required. -- **Readable artifacts** — all workflows and specifications are pure Markdown files with some [Jinja2](https://github.com/pallets/jinja) templating. +- **Readable artifacts** — all workflows and documentation are pure Markdown files with some [Jinja2](https://github.com/pallets/jinja) templating. - **Artifact management** — non-fuzzy navigation and smart agent-focused rendering of artifacts. - **Agent-centric behavior** — Donna guides agents through workflows, helps them be on the path, and fixes mistakes. -- **Extensible architecture** — implement your own operations, validators, renderers, and artifact sources. -- **Batteries included** — Donna goes with a set of pre-defined workflows, so you can start using it right away. +- **Extensible architecture** — implement your own operations, validators, renderers, and directives. +- **Project-local workflows** — keep reusable work procedures in your repository and adjust them to your team. ## Example -Donna is developed via Donna itself. You can find real-life examples of workflows and specifications in the [specs](./specs) folder of this repository. +Donna is developed via Donna itself. You can find real-life workflow examples in [workflows](./workflows) and documentation in [specs](./specs). The example below is a simplified version of the polishing workflow that formats code, runs linters, and fixes found problems until all checks pass. It uses the single operation type `donna.lib.request_action` to ask the agent to perform specific instructions. @@ -114,14 +114,14 @@ What you may notice: Directives, like `{{ goto("operation_id") }}`, render itself depending on the context: -- For the agent, they render an exact CLI command to run, such as `donna -p llm sessions action-request-completed <action-request-id> '@/specs/work/polish.donna.md:finish'`. +- For the agent, they render an exact CLI command to run, such as `donna -p llm sessions action-request-completed <action-request-id> '@/workflows/polish.donna.md:finish'`. - For Donna, they render a specific marker that can be extracted and used to analyze an artifact. For example, Donna uses `goto` directives to build an FSM of the workflow and validate it before running: does each operation exist, can the workflow be completed, are there unreachable operations, etc. Generally speaking, **all you need is `donna.lib.request_action` operation** — it is enough to achieve a great deal of automation by delegating some decisions to the agent. However, there are some more specific operations that simplify things and make workflows more agile or performant. </details> -You can find a more complex implementation of the same workflow in the [polish.donna.md](./specs/work/polish.donna.md) file. It demonstrates other Donna operations, such as running scripts directly and branching. +You can find a more complex implementation of the same workflow in the [polish.donna.md](./workflows/polish.donna.md) file. It demonstrates other Donna operations, such as running scripts directly and branching. ## Installation @@ -143,10 +143,6 @@ donna workspaces init Donna will: - Create `donna.toml` in your project root and the configured session directory. -- Sync bundled Donna specs into `.agents/donna/`. -- Install skills into `.agents/skills/` folder. - -If you upgrade Donna later, run `donna workspaces update` to refresh `.agents/donna/` and `.agents/skills/`. 3. Ask your agent to do something like `$donna-do Add a button that …`. The agent will discover the appropriate workflow and execute it. @@ -164,7 +160,7 @@ Commands you may need: - `donna workspaces init` — Initialize Donna in your project. - `donna sessions start` — start a new working session, remove everything from the previous session. -- `donna artifacts list <pattern>` — list artifacts with short descriptions. +- `donna artifacts list` — list workflows with short descriptions. Donna can send internal journal records to a third-party tool. Configure it in `donna.toml`: @@ -201,7 +197,7 @@ The documentation below covers aspects important to humans and partially duplica ## Batteries Included -Donna comes with built-in workflows and specifications that empower agents to work in a smart way. +Donna comes with documentation that helps agents work in a smart way, while workflows live in project-local `workflows/` directories. However, **I encourage you to experiment and implement your own workflows**. Meta-programming is fun; specialized workflows are more efficient. @@ -215,26 +211,25 @@ Additionally, Donna will: - choose fast or slow route depending on the complexity of the changes required; - find and run (if any) polishing workflow to ensure the codebase is in a good state after the changes; -- find and run (if any) workflow to update your changelog. -Note that the default Donna workflows are designed to be reliable and useful for a wide range of projects. They may not be optimal in terms of token usage or speed for your particular project. The intended use of Donna is to implement your own workflows that account for your project's specifics. +Note that example Donna workflows are designed to be reliable and useful for a wide range of projects. They may not be optimal in terms of token usage or speed for your particular project. The intended use of Donna is to implement your own workflows that account for your project's specifics. Points of interest: -- [@/.agents/donna/rfc/specs/request_for_change.donna.md](./.agents/donna/rfc/specs/request_for_change.donna.md) — specification of the RFC document. -- [@/.agents/donna/rfc/work/request.donna.md](./.agents/donna/rfc/work/request.donna.md) — workflow to create a RFC document. -- [@/.agents/donna/rfc/work/plan.donna.md](./.agents/donna/rfc/work/plan.donna.md) — workflow to plan work on an RFC and create a new workflow. -- [@/.agents/donna/rfc/work/do.donna.md](./.agents/donna/rfc/work/do.donna.md) — meta workflow to automate the whole work from a developer request to a changelog update. +- [@/workflows/rfc/specs/request_for_change.md](./workflows/rfc/specs/request_for_change.md) — documentation for the RFC document. +- [@/workflows/rfc/request.donna.md](./workflows/rfc/request.donna.md) — workflow to create a RFC document. +- [@/workflows/rfc/plan.donna.md](./workflows/rfc/plan.donna.md) — workflow to plan work on an RFC and create a new workflow. +- [@/workflows/rfc/do.donna.md](./workflows/rfc/do.donna.md) — meta workflow to automate the whole work from a developer request to final verification. ## Artifacts on Filesystem -- Artifacts are text files Donna reads and validates. In practice they are usually Markdown workflows and specifications stored as `.donna.md` files. -- Donna discovers artifacts directly in the project filesystem and limits what is visible via `donna.toml:file_filters`. +- Artifacts are text files Donna reads and validates. In practice they are usually Markdown workflows stored as `.donna.md` files. General documentation uses normal `.md` files. +- Donna discovers workflow artifacts by recursively scanning the directories listed in `donna.toml:workflow_dirs`. By default, Donna uses these artifact areas: -- `specs/` — project-owned artifacts. -- `.agents/donna/` — bundled Donna specs and workflows synced by `donna workspaces init` or `donna workspaces update`. +- `workflows/` — project-owned workflows. +- `.agents/donna/` — project-local Donna documentation, when present. - `.session/donna/` — session artifacts and Donna runtime state. ### Rendering @@ -249,41 +244,19 @@ Artifact ids are project-relative filepaths prefixed with `@/`. Section ids appe Examples: -- `@/specs/work/polish.donna.md` -- `@/.agents/donna/work/polish.donna.md` +- `@/workflows/polish.donna.md` +- `@/workflows/rfc/request.donna.md` - `@/.session/donna/execute_rfc.donna.md:review_changes` -You and agents can `list`, `view`, and `validate` artifacts. - -- `donna -p llm artifacts list [<pattern>]` — shows a short description from its h1 section. -- `donna -p llm artifacts view <pattern>` — shows the full content of the artifact with proper rendering. -- `donna -p llm artifacts validate [<pattern>]` — validates matching artifacts. - -Commands accept both precise artifact ids and glob patterns. Patterns may use absolute ids like `@/...` or rooted wildcard forms like `**/intro.donna.md`. - -Patterns allow using: - -- `*` — matches zero or more characters inside one path level. -- `?` — matches exactly one character inside one path level. -- `[]` — matches a single character class inside one path level. -- `**` — matches zero or more path levels. +You and agents can `list` workflow artifacts and `validate` Donna artifacts. -Examples: - -- `@/*.donna.md` — matches all artifacts directly under the project root. -- `@/**/intro.donna.md` — matches all artifacts named `intro.donna.md`. -- `@/.session/donna/**` — matches all session artifacts. -- `**/test_*.donna.md` — matches artifact filenames that start with `test_`. - -Commands that accept an artifact pattern also support `--predicate '<python-expression>'` to filter by primary section properties available as `section`. +- `donna -p llm artifacts list` — shows workflow descriptions from their h1 sections. +- `donna -p llm artifacts validate <artifact-id>...` — validates one or more artifacts. +- `donna -p llm artifacts validate --all` — validates every discovered artifact. -Currently, Donna supports two artifact tags: +Artifact ids use absolute project-root form like `@/workflows/polish.donna.md`. -- `workflow` — marks a workflow artifact — is set automatically by Donna. -- `specification` — marks a specification artifact — is set automatically by Donna. - -You can find all workflows with the command -`donna -p llm artifacts list --predicate '"workflow" in section.tags'`. +You can find all workflows with `donna -p llm artifacts list`. ## Sessions @@ -334,15 +307,15 @@ Donna can detect errors (in artifacts, in execution, etc). If an error can be fi <summary><strong>An example of error message from Donna</strong></summary> ```bash -$ donna -p llm sessions run @/specs/work/polish.donna.md +$ donna -p llm sessions run @/workflows/polish.donna.md kind=artifact_validation_error media_type=text/markdown -artifact_id=@/specs/work/polish.donna.md +artifact_id=@/workflows/polish.donna.md error_code=donna.artifacts.section_not_found section_id=run_autoflake_scriptx -Error in artifact '@/specs/work/polish.donna.md', section 'run_autoflake_scriptx': Section `run_autoflake_scriptx` is not available in artifact `@/specs/work/polish.donna.md`. +Error in artifact '@/workflows/polish.donna.md', section 'run_autoflake_scriptx': Section `run_autoflake_scriptx` is not available in artifact `@/workflows/polish.donna.md`. Ways to fix: @@ -366,11 +339,11 @@ The simplest example of such generation is currently used as a primary way for D ### Discovering workflows -If you want to run a child workflow from an operation, you can just instruct an agent like `Run the workflow @/specs/work/my-cool-workflow.donna.md` and the agent will find it and run. +If you want to run a child workflow from an operation, you can just instruct an agent like `Run the workflow @/workflows/my-cool-workflow.donna.md` and the agent will find it and run. However, it is not very agile. Instead, I suggest you describe the desired outcome and let the agent find the most suitable workflow. In that case, you'll be able to define customized workflows for specific types of changes and let the agent choose the best one for the current situation. -For example, you can have two workflows `@/specs/work/write-backend-test.donna.md` and `@/specs/work/write-frontend-test.donna.md`, and your operation can say `Run the workflow that will write a test for the current change`, and the agent will choose the most suitable workflow based on the context and the workflow descriptions. +For example, you can have two workflows `@/workflows/write-backend-test.donna.md` and `@/workflows/write-frontend-test.donna.md`, and your operation can say `Run the workflow that will write a test for the current change`, and the agent will choose the most suitable workflow based on the context and the workflow descriptions. ## Jinja2 rendering @@ -396,16 +369,12 @@ You can find detailed documentation of all built-in directives in `donna skill a Here they are: -1. `donna.lib.view` — references another artifact. In `view`/`execute` modes, it renders an exact CLI command to view the artifact. In analysis mode, it will be used to validate the existence of the artifact and track dependencies between artifacts. -2. `donna.lib.list` — references artifact listing. In `view`/`execute` modes, it renders an exact CLI command to list artifacts. In analysis mode, it will be used to validate the existence of the artifacts and track dependencies between artifacts. -3. `donna.lib.goto` — references the next workflow operation to execute. In `view`/`execute` modes, it renders an exact CLI command to advance the workflow. In `analysis` mode, it is used to construct and validate an FSM for the workflow. -4. `donna.lib.task_variable` — in `view` mode renders a placeholder with a note about task-variable substitution, in `execute` mode renders the actual task-context value. In `analysis`, it will be used to control a set of variables used in the artifact. - -## Specifications +1. `donna.lib.goto` — references the next workflow operation to execute. In `view`/`execute` modes, it renders an exact CLI command to advance the workflow. In `analysis` mode, it is used to construct and validate an FSM for the workflow. +2. `donna.lib.task_variable` — in `view` mode renders a placeholder with a note about task-variable substitution, in `execute` mode renders the actual task-context value. In `analysis`, it will be used to control a set of variables used in the artifact. -Specifications can be perceived just as documentation. There is no special handling for them. +## Documentation -Later, they will be validated like workflows, but currently, it is just docs. +Specifications are plain Markdown documentation. Donna does not treat them as artifacts or validate them as workflows. ## Extending Donna @@ -416,11 +385,10 @@ All Donna logic is referenced by Python import paths. That means: What you can implement: -- Custom sections (including operations) for Donna artifacts. Check [./donna/primitives/artifacts](./donna/primitives/artifacts) and [./donna/primitives/operations](./donna/primitives/operations) subpackages for examples. +- Custom sections (including operations) for Donna artifacts. Check [./donna/primitives/artifacts](./donna/primitives/artifacts) and [./donna/primitives/sections](./donna/primitives/sections) subpackages for examples. - Custom rendering directives. Check [./donna/primitives/directives](./donna/primitives/directives) subpackage for examples. -- Custom artifact parsers. Check [./donna/workspaces/sources](./donna/workspaces/sources) subpackage for examples. -Sources and file filters are configured in the `donna.toml` file of your project. +Donna workflow artifacts are Markdown files ending with `.donna.md` inside directories configured by `workflow_dirs` in your project `donna.toml`. Sections and directives are used directly in artifacts by their Python import paths. diff --git a/donna.toml b/donna.toml index 582172e4..df0e30b2 100644 --- a/donna.toml +++ b/donna.toml @@ -1,29 +1,9 @@ -session = ".session/donna" +session_dir = ".session/donna" +default_section_kind = "donna.lib.text" +default_primary_section_kind = "donna.lib.workflow" +default_primary_section_id = "primary" cache_lifetime = 1.0 - -[[sources]] -kind = "donna.lib.sources.markdown" -extension = ".donna.md" - -[[file_filters]] -mode = "include" -pattern = "@/.session/donna/**/*.donna.md" - -[[file_filters]] -mode = "include" -pattern = "@/.agents/**/*.donna.md" - -[[file_filters]] -mode = "ignore" -pattern = ".*/**" - -[[file_filters]] -mode = "include" -pattern = "**/*.donna.md" - -[[file_filters]] -mode = "ignore" -pattern = "**" +workflow_dirs = ["./workflows", "./.session/donna"] [journal] cmd = ["./bin/taskwarior.sh", "log", "+journal", "+donna", "kind:event", "{message}"] diff --git a/donna/cli/commands/artifacts.py b/donna/cli/commands/artifacts.py index 559e4bad..1e55efff 100644 --- a/donna/cli/commands/artifacts.py +++ b/donna/cli/commands/artifacts.py @@ -1,86 +1,80 @@ +import sys +from typing import Annotated + +import click import typer from donna.cli.application import app -from donna.cli.types import ArtifactIdPatternArgument, PredicateOption, validate_supported_artifact_pattern +from donna.cli.types import ArtifactIdArgument, ArtifactIdsArgument, RenderModeOption from donna.cli.utils import command_context from donna.context.context import context -from donna.domain.artifact_ids import ArtifactIdPattern from donna.machine import journal as machine_journal from donna.protocol.cell_shortcuts import operation_succeeded -from donna.workspaces.artifacts import RENDER_CONTEXT_VIEW +from donna.workspaces.artifacts import RENDER_CONTEXT_VIEW, ArtifactRenderContext, fetch_artifact_bytes +from donna.workspaces.templates import render as render_template artifacts_cli = typer.Typer() -DEFAULT_ARTIFACT_PATTERN = ArtifactIdPattern.parse("**").unwrap() - def _log_artifact_operation(message: str) -> None: machine_journal.add(message=message) -def _log_operation_on_artifacts( - message: str, - pattern: ArtifactIdPattern, - predicate: PredicateOption | None, -) -> None: - if predicate is None: - return _log_artifact_operation(f"{message} `{pattern}`") - - return _log_artifact_operation(f"{message} `{pattern}` with predicate `{predicate.source}`") - - -@artifacts_cli.command( - help=( - "List artifacts matching a pattern with a supported source extension " - "and show their status summaries. Lists all artifacts by default." - ) -) -def list( +@artifacts_cli.command(name="list", help="List available workflow artifacts and show their status summaries.") +def list_( typer_context: typer.Context, - pattern: ArtifactIdPatternArgument = DEFAULT_ARTIFACT_PATTERN, - predicate: PredicateOption = None, ) -> None: with command_context(typer_context) as command: - validate_supported_artifact_pattern(pattern) - _log_operation_on_artifacts("List artifacts", pattern, predicate) + _log_artifact_operation("List artifacts") - artifacts = context().artifacts.list(pattern, RENDER_CONTEXT_VIEW, predicate=predicate).unwrap() + artifacts = context().artifacts.list(RENDER_CONTEXT_VIEW).unwrap() command.write_cells(artifact.node().status() for artifact in artifacts) -@artifacts_cli.command( - help="Display artifacts matching a pattern or specific id that uses a supported source extension." -) -def view( +@artifacts_cli.command(help="Render an artifact with the selected render mode and write the markdown to stdout.") +def render( typer_context: typer.Context, - pattern: ArtifactIdPatternArgument, - predicate: PredicateOption = None, + artifact_id: ArtifactIdArgument, + mode: RenderModeOption, ) -> None: - with command_context(typer_context) as command: - validate_supported_artifact_pattern(pattern) - _log_operation_on_artifacts("View artifacts", pattern, predicate) + with command_context(typer_context): + _log_artifact_operation(f"Render artifact `{artifact_id}` in `{mode.value}` mode") - artifacts = context().artifacts.list(pattern, RENDER_CONTEXT_VIEW, predicate=predicate).unwrap() - command.write_cells(artifact.node().info() for artifact in artifacts) + content = fetch_artifact_bytes(artifact_id).unwrap().decode("utf-8") + render_context = ArtifactRenderContext(primary_mode=mode) + rendered = render_template(artifact_id, content, render_context).unwrap() + sys.stdout.write(rendered) -@artifacts_cli.command( - help=( - "Validate artifacts matching a pattern with a supported source extension " - "(defaults to all artifacts) and return any errors." - ) -) -def validate( + +@artifacts_cli.command(help="Validate the given artifact ids, or validate every discovered artifact with --all.") +def validate( # noqa: CCR001 typer_context: typer.Context, - pattern: ArtifactIdPatternArgument = DEFAULT_ARTIFACT_PATTERN, - predicate: PredicateOption = None, -) -> None: # noqa: CCR001 + artifact_ids: ArtifactIdsArgument = None, + all_artifacts: Annotated[ + bool, + typer.Option("--all", help="Validate every discovered artifact."), + ] = False, +) -> None: with command_context(typer_context) as command: - validate_supported_artifact_pattern(pattern) - _log_operation_on_artifacts("Validate artifacts", pattern, predicate) - - artifacts = context().artifacts.list(pattern, RENDER_CONTEXT_VIEW, predicate=predicate).unwrap() + if all_artifacts and artifact_ids: + raise click.UsageError("Pass artifact ids or --all, not both.") + + if not all_artifacts and not artifact_ids: + raise click.UsageError("Pass artifact ids or --all.") + + if all_artifacts: + _log_artifact_operation("Validate all artifacts") + artifacts = context().artifacts.list(RENDER_CONTEXT_VIEW).unwrap() + else: + assert artifact_ids is not None + _log_artifact_operation( + f"Validate artifacts {', '.join(f'`{artifact_id}`' for artifact_id in artifact_ids)}" + ) + artifacts = [ + context().artifacts.load(artifact_id, RENDER_CONTEXT_VIEW).unwrap() for artifact_id in artifact_ids + ] errors = [] diff --git a/donna/cli/commands/sessions.py b/donna/cli/commands/sessions.py index 3d47a7a6..a5cb799e 100644 --- a/donna/cli/commands/sessions.py +++ b/donna/cli/commands/sessions.py @@ -5,8 +5,6 @@ ActionRequestIdArgument, ArtifactIdArgument, ArtifactSectionIdArgument, - validate_supported_artifact_id, - validate_supported_artifact_section_id, ) from donna.cli.utils import command_context from donna.machine import sessions @@ -50,7 +48,6 @@ def details(context: typer.Context) -> None: @sessions_cli.command(help="Run a workflow from an artifact to drive the current session forward.") def run(context: typer.Context, workflow_id: ArtifactIdArgument) -> None: with command_context(context) as command: - validate_supported_artifact_id(workflow_id) command.write_cells(sessions.start_workflow(workflow_id).unwrap()) @@ -63,7 +60,6 @@ def action_request_completed( next_operation_id: ArtifactSectionIdArgument, ) -> None: with command_context(context) as command: - validate_supported_artifact_section_id(next_operation_id) command.write_cells(sessions.complete_action_request(request_id, next_operation_id).unwrap()) diff --git a/donna/cli/commands/workspaces.py b/donna/cli/commands/workspaces.py index 07bf515d..0c29f05b 100644 --- a/donna/cli/commands/workspaces.py +++ b/donna/cli/commands/workspaces.py @@ -1,34 +1,23 @@ import typer from donna.cli.application import app -from donna.cli.types import SkillsOption, SpecsOption from donna.cli.utils import command_context from donna.protocol.cell_shortcuts import operation_succeeded -from donna.workspaces.initialization import initialize_workspace, update_workspace +from donna.workspaces.initialization import initialize_workspace workspaces_cli = typer.Typer() @workspaces_cli.command(help="Initialize Donna project config.") -def init(context: typer.Context, skills: SkillsOption = True, specs: SpecsOption = True) -> None: +def init(context: typer.Context) -> None: with command_context(context, load_environment=False) as command: target_dir = command.target_dir() - initialize_workspace(target_dir, install_skills=skills, install_specs=specs).unwrap() + initialize_workspace(target_dir).unwrap() command.write_cells([operation_succeeded("Donna project initialized successfully")]) -@workspaces_cli.command(help="Update Donna project files.") -def update(context: typer.Context, skills: SkillsOption = True, specs: SpecsOption = True) -> None: - with command_context(context) as command: - target_dir = command.target_dir() - - update_workspace(target_dir, install_skills=skills, install_specs=specs).unwrap() - - command.write_cells([operation_succeeded("Donna project updated successfully")]) - - app.add_typer( workspaces_cli, name="workspaces", diff --git a/donna/cli/types.py b/donna/cli/types.py index 1e040d0d..6f9dad45 100644 --- a/donna/cli/types.py +++ b/donna/cli/types.py @@ -6,11 +6,12 @@ from donna.cli.utils import output_cells from donna.core.errors import ErrorsList from donna.domain import errors as domain_errors -from donna.domain.artifact_ids import ArtifactId, ArtifactIdPattern, ArtifactSectionId +from donna.domain.artifact_ids import ArtifactId, ArtifactSectionId +from donna.domain.constants import DONNA_ARTIFACT_EXTENSION from donna.domain.internal_ids import ActionRequestId -from donna.machine.artifacts import ArtifactPredicate from donna.protocol.modes import Mode -from donna.workspaces.config import config as workspace_config +from donna.workspaces.artifacts import has_donna_artifact_extension +from donna.workspaces.templates import RenderMode def _exit_with_errors(errors: ErrorsList) -> NoReturn: @@ -40,80 +41,24 @@ def _absolute_artifact_section_id_or_exit(value: str) -> str: return value -def _absolute_artifact_pattern_or_exit(value: str | ArtifactIdPattern) -> str: - if isinstance(value, ArtifactIdPattern): - return str(value) - - first_part = value.split(ArtifactId.delimiter, maxsplit=1)[0] - if any(character in first_part for character in "*?[]"): - return f"{ArtifactId.prefix}{value}" - - if not value.startswith(ArtifactId.prefix): - _exit_with_errors([domain_errors.InvalidIdPattern(id_type=ArtifactIdPattern.__name__, value=value)]) - - return value - - def _parse_artifact_id(value: str) -> ArtifactId: result = ArtifactId.parse(_absolute_artifact_id_or_exit(value)) return _parse_result_or_exit(result.ok(), result.err()) -def _parse_artifact_id_pattern(value: str | ArtifactIdPattern) -> ArtifactIdPattern: - if isinstance(value, ArtifactIdPattern): - return value - - result = ArtifactIdPattern.parse(_absolute_artifact_pattern_or_exit(value)) - return _parse_result_or_exit(result.ok(), result.err()) - - def _parse_artifact_section_id(value: str) -> ArtifactSectionId: result = ArtifactSectionId.parse(_absolute_artifact_section_id_or_exit(value)) return _parse_result_or_exit(result.ok(), result.err()) -def _match_supported_extension(filename: str) -> str | None: - supported_extensions = workspace_config().supported_extensions() - normalized = filename.strip().lower() - - for extension in sorted(supported_extensions, key=len, reverse=True): - if normalized.endswith(extension): - return extension - - return None - - def _artifact_filename(value: str) -> str: return pathlib.PurePosixPath(value.split(ArtifactSectionId.delimiter, maxsplit=1)[0]).name -def _pattern_filename(value: str) -> str | None: - last_part = pathlib.PurePosixPath(value).name - if last_part in {"*", "**"}: - return None - - dot_index = last_part.find(".") - if dot_index == -1: - return None - - return f"placeholder{last_part[dot_index:]}" - - def validate_supported_artifact_id(artifact_id: ArtifactId) -> None: - if _match_supported_extension(_artifact_filename(str(artifact_id))) is None: + if not has_donna_artifact_extension(_artifact_filename(str(artifact_id))): raise typer.BadParameter( - f"Unsupported artifact extension for '{artifact_id}'. Use a filename extension supported by the sources." - ) - - -def validate_supported_artifact_pattern(pattern: ArtifactIdPattern) -> None: - filename = _pattern_filename(str(pattern)) - if filename is None: - return - - if _match_supported_extension(filename) is None: - raise typer.BadParameter( - f"Unsupported artifact extension for '{pattern}'. Use a filename extension supported by the sources." + f"Unsupported artifact extension for '{artifact_id}'. Use '*{DONNA_ARTIFACT_EXTENSION}'." ) @@ -121,13 +66,16 @@ def validate_supported_artifact_section_id(section_id: ArtifactSectionId) -> Non validate_supported_artifact_id(section_id.artifact_id) -def _parse_artifact_predicate(value: str) -> ArtifactPredicate: - result = ArtifactPredicate.parse(value) - errors = result.err() - if errors is not None: - _exit_with_errors(errors) +def _parse_supported_artifact_id(value: str) -> ArtifactId: + artifact_id = _parse_artifact_id(value) + validate_supported_artifact_id(artifact_id) + return artifact_id - return result.unwrap() + +def _parse_supported_artifact_section_id(value: str) -> ArtifactSectionId: + section_id = _parse_artifact_section_id(value) + validate_supported_artifact_section_id(section_id) + return section_id def _parse_action_request_id(value: str) -> ActionRequestId: @@ -173,42 +121,31 @@ def _parse_input_path(value: str) -> pathlib.Path: ArtifactIdArgument = Annotated[ ArtifactId, typer.Argument( - parser=_parse_artifact_id, + parser=_parse_supported_artifact_id, help=( - "Artifact ID in absolute project-root form with a supported source " - "extension (e.g., '@/specs/intro.donna.md')." + "Artifact ID in absolute project-root form with the Donna artifact " + "extension (e.g., '@/workflows/polish.donna.md')." ), ), ] -ArtifactIdPatternArgument = Annotated[ - ArtifactIdPattern, +ArtifactIdsArgument = Annotated[ + list[ArtifactId] | None, typer.Argument( - parser=_parse_artifact_id_pattern, + parser=_parse_supported_artifact_id, help=( - "Artifact pattern in absolute form '@/...' or rooted wildcard form like " - "'*/x.donna.md' and '**/x.donna.md'. Patterns that name a file " - "extension must use a supported source extension." + "Artifact IDs in absolute project-root form with the Donna artifact " + "extension (e.g., '@/workflows/polish.donna.md')." ), ), ] -PredicateOption = Annotated[ - ArtifactPredicate | None, - typer.Option( - "--predicate", - "-p", - parser=_parse_artifact_predicate, - help="Filter artifacts by predicate expression evaluated with `section` global.", - ), -] - ArtifactSectionIdArgument = Annotated[ ArtifactSectionId, typer.Argument( - parser=_parse_artifact_section_id, + parser=_parse_supported_artifact_section_id, help=( "Artifact section ID in absolute project-root form 'artifact:section' " "(e.g. '@/.session/donna/plans/artifact_id_filepaths.donna.md:finish')." @@ -227,6 +164,14 @@ def _parse_input_path(value: str) -> pathlib.Path: ), ] +RenderModeOption = Annotated[ + RenderMode, + typer.Option( + "--mode", + help="Artifact render mode to use. Examples: --mode=view, --mode=execute, --mode=analysis.", + ), +] + RootOption = Annotated[ pathlib.Path | None, @@ -244,23 +189,6 @@ def _parse_input_path(value: str) -> pathlib.Path: ), ] -SkillsOption = Annotated[ - bool, - typer.Option( - "--skills/--no-skills", - help="Enable or disable skills updates in `.agents/skills`.", - ), -] - -SpecsOption = Annotated[ - bool, - typer.Option( - "--specs/--no-specs", - help="Enable or disable Donna specs updates in `.agents/donna`.", - ), -] - - InputPathArgument = Annotated[ pathlib.Path, typer.Argument( diff --git a/donna/context/artifacts.py b/donna/context/artifacts.py index b79b69f9..56134dd6 100644 --- a/donna/context/artifacts.py +++ b/donna/context/artifacts.py @@ -3,9 +3,9 @@ from donna.context.entity_cache import TimedCache, TimedCacheValue from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.artifact_ids import ArtifactId, ArtifactIdPattern, ArtifactSectionId +from donna.domain.artifact_ids import ArtifactId from donna.domain.types import Milliseconds -from donna.machine.artifacts import Artifact, ArtifactPredicate, ArtifactSection +from donna.machine.artifacts import Artifact from donna.workspaces.templates import RenderMode if TYPE_CHECKING: @@ -102,61 +102,24 @@ def load( # noqa: CCR001 return Ok(artifact) - @unwrap_to_error - def resolve_section( - self, - target_id: ArtifactSectionId, - render_context: "ArtifactRenderContext", - ) -> Result[ArtifactSection, ErrorsList]: - artifact = self.load(target_id.artifact_id, render_context).unwrap() - return Ok(artifact.get_section(target_id.local_id).unwrap()) - - @unwrap_to_error - def _list_artifact_if_matches( - self, - artifact_id: ArtifactId, - render_context: "ArtifactRenderContext", - predicate: ArtifactPredicate | None, - ) -> Result[Artifact | None, ErrorsList]: - artifact = self.load(artifact_id, render_context).unwrap() - - if predicate is None: - return Ok(artifact) - - section = artifact.primary_section().unwrap() - - predicate_result = predicate.evaluate(section) - if predicate_result.is_err(): - return Ok(None) - - if not predicate_result.unwrap(): - return Ok(None) - - return Ok(artifact) - @unwrap_to_error def list( # noqa: CCR001 self, - pattern: ArtifactIdPattern, render_context: "ArtifactRenderContext", - predicate: ArtifactPredicate | None = None, ) -> Result[list[Artifact], ErrorsList]: from donna.workspaces.artifacts import list_artifact_ids artifacts: list[Artifact] = [] errors: ErrorsList = [] - for artifact_id in list_artifact_ids(pattern): - artifact_result = self._list_artifact_if_matches(artifact_id, render_context, predicate) + for artifact_id in list_artifact_ids(): + artifact_result = self.load(artifact_id, render_context) if artifact_result.is_err(): errors.extend(artifact_result.unwrap_err()) continue artifact = artifact_result.unwrap() - if artifact is None: - continue - artifacts.append(artifact) if errors: diff --git a/donna/domain/artifact_ids.py b/donna/domain/artifact_ids.py index a9d414df..37a9ffef 100644 --- a/donna/domain/artifact_ids.py +++ b/donna/domain/artifact_ids.py @@ -1,42 +1,15 @@ import pathlib from typing import Sequence -from donna.domain.id_paths import ( - IdPath, - IdPathPattern, - IdPathSegmentLiteralMatcher, - IdPathSegmentMatcher, - IdPathSegmentRecursiveMatcher, - IdPathSegmentSingleMatcher, - NormalizedRawIdPath, -) +from donna.domain.constants import ARTIFACT_ID_PREFIX +from donna.domain.id_paths import IdPath, NormalizedRawIdPath from donna.domain.ids import SectionId, _is_artifact_slug_part -ARTIFACT_ID_PREFIX = "@/" -_ARTIFACT_PATTERN_EXTRA_CHARACTERS = set("*?[]") - - -def _is_artifact_pattern_part(part: str) -> bool: - if not part: - return False - - if part == "**": - return True - - allowed_characters = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-") - allowed_characters.update(_ARTIFACT_PATTERN_EXTRA_CHARACTERS) - - if any(character not in allowed_characters for character in part): - return False - - return any(character not in ".-" for character in part) - def normalize_path( # noqa: CCR001 text: str, *, relative_to: "ArtifactId | None" = None, - allow_wildcards: bool, ) -> NormalizedRawIdPath | None: if not isinstance(text, str) or not text: return None @@ -67,10 +40,7 @@ def normalize_path( # noqa: CCR001 normalized_parts.pop() continue - if allow_wildcards: - if not _is_artifact_pattern_part(part): - return None - elif not _is_artifact_slug_part(part): + if not _is_artifact_slug_part(part): return None normalized_parts.append(part) @@ -79,9 +49,8 @@ def normalize_path( # noqa: CCR001 return None last_part = normalized_parts[-1] - if not allow_wildcards or last_part not in {"*", "**"}: - if not pathlib.PurePosixPath(last_part).suffix: - return None + if not pathlib.PurePosixPath(last_part).suffix: + return None return NormalizedRawIdPath("/".join(normalized_parts)) @@ -95,7 +64,7 @@ def normalize_artifact_section_id(text: str, *, relative_to: "ArtifactId | None" except ValueError: return None - normalized_artifact_id = normalize_path(artifact_part, relative_to=relative_to, allow_wildcards=False) + normalized_artifact_id = normalize_path(artifact_part, relative_to=relative_to) if normalized_artifact_id is None or not SectionId.validate(local_part): return None @@ -122,31 +91,6 @@ def to_full_local(self, local_id: SectionId) -> "ArtifactSectionId": return ArtifactSectionId(NormalizedRawIdPath(f"{self.raw_value}:{local_id}")) -class ArtifactIdPattern(IdPathPattern["ArtifactId"]): - __slots__ = () - id_class = ArtifactId - - def __str__(self) -> str: - rendered = self.id_class.delimiter.join(str(part) for part in self) - if self and not isinstance(self[0], IdPathSegmentLiteralMatcher): - return rendered - - return f"{self.id_class.prefix}{rendered}" - - @classmethod - def _parse_pattern_part(cls, part: str) -> IdPathSegmentMatcher | None: - if part == "**": - return IdPathSegmentRecursiveMatcher(part) - - if not _is_artifact_pattern_part(part): - return None - - if any(char in part for char in _ARTIFACT_PATTERN_EXTRA_CHARACTERS): - return IdPathSegmentSingleMatcher(part) - - return IdPathSegmentLiteralMatcher(part) - - class ArtifactSectionId(IdPath): __slots__ = () prefix = ARTIFACT_ID_PREFIX diff --git a/donna/domain/constants.py b/donna/domain/constants.py new file mode 100644 index 00000000..87baef3c --- /dev/null +++ b/donna/domain/constants.py @@ -0,0 +1,8 @@ +import pathlib + +ARTIFACT_ID_PREFIX = "@/" +DONNA_CONFIG_NAME = "donna.toml" +DONNA_ARTIFACT_EXTENSION = ".donna.md" +DONNA_DEFAULT_SESSION_DIR = pathlib.Path(".session") / "donna" +DONNA_DEFAULT_WORKFLOW_DIR = pathlib.Path("workflows") +STATE_FILE_NAME = "state.json" diff --git a/donna/domain/errors.py b/donna/domain/errors.py index 729098e6..bc04b6a2 100644 --- a/donna/domain/errors.py +++ b/donna/domain/errors.py @@ -35,13 +35,3 @@ class InvalidIdFormat(EnvironmentError): ] id_type: str value: str - - -class InvalidIdPattern(EnvironmentError): - code: str = "donna.domain.invalid_id_pattern" - message: str = "Invalid {error.id_type}: '{error.value}'." - ways_to_fix: list[str] = [ - "Use identifiers or '*'/'**' tokens separated by the expected delimiter.", - ] - id_type: str - value: str diff --git a/donna/domain/id_paths.py b/donna/domain/id_paths.py index cafbd269..c3d03056 100644 --- a/donna/domain/id_paths.py +++ b/donna/domain/id_paths.py @@ -1,7 +1,5 @@ -import fnmatch -import re -from functools import lru_cache, total_ordering -from typing import Any, Generic, Self, Sequence, TypeVar +from functools import total_ordering +from typing import Any, Self, Sequence, TypeVar from pydantic_core import PydanticCustomError, core_schema @@ -10,56 +8,6 @@ from donna.domain import errors as domain_errors -def _match_pattern_parts( # noqa: CCR001 - pattern_parts: Sequence["IdPathSegmentMatcher"], value_parts: Sequence[str] -) -> bool: - def match_at(p_index: int, v_index: int) -> bool: # noqa: CCR001 - while True: - if p_index >= len(pattern_parts): - return v_index >= len(value_parts) - - matcher = pattern_parts[p_index] - - if matcher.is_recursive(): - for next_index in range(v_index, len(value_parts) + 1): - if match_at(p_index + 1, next_index): - return True - return False - - if v_index >= len(value_parts): - return False - - if not matcher.matches_segment(value_parts[v_index]): - return False - - p_index += 1 - v_index += 1 - - return match_at(0, 0) - - -def _match_pattern_prefix(pattern_parts: Sequence["IdPathSegmentMatcher"], prefix_parts: Sequence[str]) -> bool: - @lru_cache(maxsize=None) - def match_at(p_index: int, v_index: int) -> bool: # noqa: CCR001 - if v_index >= len(prefix_parts): - return True - - if p_index >= len(pattern_parts): - return False - - matcher = pattern_parts[p_index] - - if matcher.is_recursive(): - return match_at(p_index + 1, v_index) or match_at(p_index, v_index + 1) - - if matcher.matches_segment(prefix_parts[v_index]): - return match_at(p_index + 1, v_index + 1) - - return False - - return match_at(0, 0) - - def _stringify_value(value: Any) -> str: if isinstance(value, str): return value @@ -89,72 +37,10 @@ def _invalid_format(id_type: str, value: Any) -> Result[TParsed, ErrorsList]: return Err([domain_errors.InvalidIdFormat(id_type=id_type, value=_stringify_value(value))]) -def _invalid_pattern(id_type: str, value: Any) -> Result[TParsed, ErrorsList]: - return Err([domain_errors.InvalidIdPattern(id_type=id_type, value=_stringify_value(value))]) - - class NormalizedRawIdPath(str): __slots__ = () -class IdPathSegmentMatcher: - __slots__ = ("pattern_text",) - pattern_text: str - - def __init__(self, pattern_text: str) -> None: - self.pattern_text = pattern_text - - def __str__(self) -> str: - return self.pattern_text - - def __repr__(self) -> str: - return f"{type(self).__name__}({self.pattern_text!r})" - - def __hash__(self) -> int: - return hash((type(self), self.pattern_text)) - - def __eq__(self, other: object) -> bool: - if not isinstance(other, type(self)): - return NotImplemented - - return self.pattern_text == other.pattern_text - - def matches_segment(self, value: str) -> bool: - raise NotImplementedError - - def is_recursive(self) -> bool: - return False - - -class IdPathSegmentLiteralMatcher(IdPathSegmentMatcher): - __slots__ = () - - def matches_segment(self, value: str) -> bool: - return self.pattern_text == value - - -class IdPathSegmentSingleMatcher(IdPathSegmentMatcher): - __slots__ = ("_regex",) - _regex: re.Pattern[str] - - def __init__(self, pattern_text: str) -> None: - super().__init__(pattern_text) - self._regex = re.compile(fnmatch.translate(pattern_text)) - - def matches_segment(self, value: str) -> bool: - return self._regex.fullmatch(value) is not None - - -class IdPathSegmentRecursiveMatcher(IdPathSegmentMatcher): - __slots__ = () - - def matches_segment(self, value: str) -> bool: - return True - - def is_recursive(self) -> bool: - return True - - @total_ordering class IdPath: __slots__ = ("parts",) @@ -287,96 +173,3 @@ def validate(v: Any) -> "IdPath": return cls(normalized) return cls._build_pydantic_schema(validate) - - -TIdPath = TypeVar("TIdPath", bound="IdPath") -TIdPathPattern = TypeVar("TIdPathPattern", bound="IdPathPattern[Any]") - - -class IdPathPattern(tuple[IdPathSegmentMatcher, ...], Generic[TIdPath]): - __slots__ = () - id_class: type[TIdPath] - - def __str__(self) -> str: - return self.id_class.delimiter.join(str(part) for part in self) - - @classmethod - def _parse_pattern_part(cls, part: str) -> IdPathSegmentMatcher | None: - if part == "**": - return IdPathSegmentRecursiveMatcher(part) - - if part == "": - return None - - if any(char in part for char in "*?[]"): - return IdPathSegmentSingleMatcher(part) - - if not part.isidentifier(): - return None - - return IdPathSegmentLiteralMatcher(part) - - @classmethod - def parse(cls: type[TIdPathPattern], text: str) -> Result[TIdPathPattern, ErrorsList]: # noqa: CCR001 - if not isinstance(text, str) or not text: - return _invalid_pattern(cls.__name__, text) - - if not cls.id_class.delimiter: - return _invalid_pattern(cls.__name__, text) - - if cls.id_class.prefix and text.startswith(cls.id_class.prefix): - text = text.removeprefix(cls.id_class.prefix) - - parts = text.split(cls.id_class.delimiter) - - if any(part == "" for part in parts): - return _invalid_pattern(cls.__name__, text) - - compiled_parts: list[IdPathSegmentMatcher] = [] - - for part in parts: - matcher = cls._parse_pattern_part(part) - if matcher is None: - return _invalid_pattern(cls.__name__, text) - compiled_parts.append(matcher) - - return Ok(cls(compiled_parts)) - - def matches(self, value: TIdPath) -> bool: - return _match_pattern_parts(self, value.parts) - - def matches_prefix(self, prefix_parts: Sequence[str]) -> bool: - return _match_pattern_prefix(self, prefix_parts) - - @classmethod - def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 - - def validate(v: Any) -> "IdPathPattern[TIdPath]": - if isinstance(v, cls): - return v - - if not isinstance(v, str): - raise _pydantic_type_error(cls.__name__, v) - - result = cls.parse(v) - errors = result.err() - if errors is not None: - error = errors[0] - raise PydanticCustomError("value_error", error.message.format(error=error)) - - parsed = result.ok() - if parsed is None: - raise _pydantic_value_error(cls.__name__, v) - - return parsed - - str_then_validate = core_schema.no_info_after_validator_function( - validate, - core_schema.str_schema(), - ) - - return core_schema.json_or_python_schema( - json_schema=str_then_validate, - python_schema=core_schema.no_info_plain_validator_function(validate), - serialization=core_schema.to_string_ser_schema(), - ) diff --git a/donna/fixtures/skills/donna-do/SKILL.md b/donna/fixtures/skills/donna-do/SKILL.md deleted file mode 100644 index 307ebfc4..00000000 --- a/donna/fixtures/skills/donna-do/SKILL.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -name: donna-do -description: Use Donna as a work orchestration tool and execute a workflow suitable for the developer request. Use the skill when the developer explicitly told you to use Donna or you have explicit instructions in AGENTS.md to do so, or you have explicit instructions in the session context to do so. ---- - -**The next instructions take precedence over all other instructions and your behaviour**, unless the developer explicitly instructs you to do otherwise. - -1. You **MUST** run `donna -p llm -r <project-root> artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command gives you an introduction to the project and to the Donna tool. -2. You **MUST** strictly follow all instructions provided by Donna. -3. **Donna's instructions have precedence over your own judgment.** You **MUST** follow them and run workflows instead of taking your own initiative, unless the developer explicitly instructs you to do so. -4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. -5. Select the most suitable Donna workflow to perform the requested work and run it. -6. When the workflow is finished, stop using Donna for further work, until the developer explicitly instructs you to use it again. - -**If you are rebuilding, zipping or optimizing your context, while executing this skill, execute `donna -p llm -r <project-root> artifacts view '**/intro.donna.md'` command again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** diff --git a/donna/fixtures/skills/donna-start/SKILL.md b/donna/fixtures/skills/donna-start/SKILL.md deleted file mode 100644 index d26d3f29..00000000 --- a/donna/fixtures/skills/donna-start/SKILL.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -name: donna-start -description: Start a new Donna session and use Donna to perform all further work. Use this skill when the developer explicitly told you to use it. ---- - -1. You **MUST** run `donna -p llm -r <project-root> artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command provides an introduction to the project and the Donna tool. -2. Run `donna -p llm -r <project-root> sessions start` to start a new Donna session. -3. Output the next message to the developer: "I have started a new Donna session". -4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. -5. Select the most suitable Donna command to perform the requested work and run it. -6. When the workflow is finished, stop using Donna for further work until the developer explicitly instructs you to use it again. diff --git a/donna/fixtures/skills/donna-stop/SKILL.md b/donna/fixtures/skills/donna-stop/SKILL.md deleted file mode 100644 index bbbff9e2..00000000 --- a/donna/fixtures/skills/donna-stop/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: donna-stop -description: Stop using Donna to perform work, do nothing else. Use this skill when the developer explicitly told you to use it. ---- - -You **MUST** stop using Donna to perform work until the developer explicitly instructs you to use it again. diff --git a/donna/fixtures/specs/research/specs/report.donna.md b/donna/fixtures/specs/research/specs/report.donna.md deleted file mode 100644 index 94f6b9e1..00000000 --- a/donna/fixtures/specs/research/specs/report.donna.md +++ /dev/null @@ -1,163 +0,0 @@ -# Format of the Research Report document - -```toml donna -kind = "donna.lib.specification" -``` - -This document describes the format and structure of a Research Report document used by Donna workflows from `../**` namespace. - -## Overview - -Donna introduces a group of workflows located in `../**` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. - -Session-related research artifacts MUST be stored as `@/.session/donna/research/<short-problem-related-identifier>.donna.md`, unless the developer or parent workflow specifies a different location. The `<short-problem-related-identifier>` MUST be unique within the session. - -The agent (via workflows) creates the artifact and updates it iteratively as the research process progresses. - -## Research report structure - -The research report is a Donna artifact (check `donna skill artifacts`) with the next structure: - -- **Primary section** -- title and short description of the research problem. -- **Original problem description** -- original problem statement from the developer or parent workflow. -- **Formalized problem description** -- formalized version of the problem statement. -- **Goals** -- list of goals the research should achieve. -- **Desired form of final solution** -- description of the expected form and constraints for the final solution. -- **Solution space** -- description of analysis axes and synthesis dimensions. -- **Information to collect** -- list of information required to research the problem. -- **Information sources** -- list of sources that can provide the required information. -- **Collected information** -- gathered information with source references. -- **Analysis** -- analysis of collected information along the specified axes. -- **Synthesized solutions** -- synthesized solution options along the specified dimensions. -- **Evaluation** -- evaluation of synthesized solutions against the goals. -- **Final solution** -- final solution in the desired form. - -## General language and format - -- You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `donna skill artifacts`. -- You MUST follow the structure specified in this document. - -### List format - -- If a section is described as a list, it MUST contain only a single markdown list. -- Each list item MUST be concise and clear. -- Each list item SHOULD be atomic and focused on a single aspect. -- Reviewer MUST be able to tell if the list item statement is true or false by inspecting the resulting artifacts and behavior. - -Common approaches to improve list items: - -- Split a single item with an enumeration into multiple items with a single focus. -- Transform an abstract item into a concrete one by referencing specific artifacts, measurable criteria, verifiable conditions, etc. - -### Trusted inputs - -Some sections of the research report MUST be based on trusted inputs. Trusted inputs are: - -- Original problem description from the developer or parent workflow. -- Statements from the research report itself. -- Existing project documentation, code, and artifacts. -- External standards, when they define constraints or best practices for the project domain. -- Documentation of third-party libraries, frameworks, or tools when they describe constraints or best practices. -- Primary research sources (datasets, reports, official publications) were used to collect the required information. - -## `Primary` section - -- Title MUST be concise and reflect the essence of the research problem. -- Description MUST provide a brief overview of the problem, its purpose, and why research is needed. - -## `Original problem description` section - -- This section MUST contain the original problem description from the developer or from the parent workflow. -- The problem description MUST NOT be modified by agents. - -## `Formalized problem description` section - -- The section MUST contain a clear professional high-level description of the problem based on the original description. -- The section MUST be limited to a single paragraph with a few sentences. -- The section MUST explain what someone gains after the problem is solved and how they can see it working. - -## `Goals` section - -- This section MUST contain a list of goals that the research should achieve. -- Each goal MUST be grounded in the formalized problem description. - -Goal quality criteria: - -- A goal MUST be a desired end state, outcome, or result. -- A goal MUST define what ultimately should be true, not how to achieve it. - -Examples: - -- Bad: `- Investigate database options.` -- Good: `- Identify a database option that meets the project's scalability requirements.` - -## `Desired form of final solution` section - -- The section MUST be grounded in the formalized problem description and the goals. -- The section MUST describe the expected form of the final solution (for example: recommendation, decision matrix, implementation plan, or specification). -- The section MUST specify any required structure, formatting, or constraints on the final solution. -- The section SHOULD be a short list or a short paragraph, whichever is clearer for the problem. - -## `Solution space` section - -- The section MUST describe the axes along which collected information will be analyzed and the dimensions along which solutions will be synthesized. -- The section MUST contain two subsections: **Analysis axes** and **Synthesis dimensions**. -- Each axis or dimension MUST be grounded in the goals or the formalized problem description. - -### `Analysis axes` subsection - -- The subsection MUST contain a list of analysis axes. -- Each axis MUST describe a single perspective or criterion used to analyze information. -- Each axis SHOULD be phrased so it is clear how to apply it to collected information. - -### `Synthesis dimensions` subsection - -- The subsection MUST contain a list of synthesis dimensions. -- Each dimension MUST describe a single perspective or criterion used to synthesize solution options. -- Each dimension SHOULD make the comparison between options easier. - -## `Information to collect` section - -- The section MUST contain a list of information items required to research the problem. -- Each item MUST be specific enough to be collected or verified from sources. -- Each item MUST be grounded in the formalized problem description or the goals. - -## `Information sources` section - -- The section MUST contain a list of sources that can provide the required information. -- Each source entry MUST include a short identifier and a brief description of the source. -- Each source entry SHOULD include access method, scope, and reliability notes if relevant. -- Each source MUST be relevant to at least one item from **Information to collect**. - -## `Collected information` section - -- The section MUST contain the collected information mapped to the items from **Information to collect**. -- Each collected information item MUST reference one or more source identifiers from **Information sources**. -- The section MUST make it clear which information items are satisfied and which are missing. -- If a required information item cannot be collected, the section MUST state that explicitly and explain why. - -## `Analysis` section - -- The section MUST analyze the collected information along the **Analysis axes**. -- The analysis MUST be organized so each axis can be reviewed independently. -- The analysis MUST clearly separate observed facts from inferences or assumptions. -- The analysis format MUST fit the **Desired form of final solution** and should make downstream synthesis straightforward. - -## `Synthesized solutions` section - -- The section MUST present synthesized solutions or options in a format consistent with the **Synthesis dimensions**. -- Each solution SHOULD reference the analysis items that justify it. -- The synthesis format MUST fit the **Desired form of final solution**. - -## `Evaluation` section - -- The section MUST evaluate each synthesized solution against the **Goals**. -- The evaluation MUST make trade-offs explicit and identify risks or uncertainties. -- The evaluation MUST result in a clear comparison between solutions. - -## `Final solution` section - -- The section MUST present the final solution in the form specified in **Desired form of final solution**. -- The final solution MUST be justified by the evaluation results. -- If the evaluation does not allow a confident final solution, the section MUST state the remaining uncertainties and what additional information would resolve them. diff --git a/donna/fixtures/specs/research/work/research.donna.md b/donna/fixtures/specs/research/work/research.donna.md deleted file mode 100644 index a2f9c181..00000000 --- a/donna/fixtures/specs/research/work/research.donna.md +++ /dev/null @@ -1,198 +0,0 @@ -# Research workflow - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - -Workflow for performing research on a given question/problem/situation. Collects relevant information, analyzes it, synthesizes possible options, and produces an answer/solution/deliverables. - -The purpose of this workflow is to provide an information for making decisions or producing solutions based on researched data. It can be used: - -- When the developer explicitly asks to conduct research. -- When other workflows need to perform research as a subtask. - -## Start - -```toml donna -id = "start" -kind = "donna.lib.request_action" -fsm_mode = "start" -``` - -1. Read the artifact instructions by running `donna skill artifacts` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("../specs/report.donna.md") }}` if you haven't done it yet. -3. `{{ donna.lib.goto("ensure_problem_description_exists") }}` - -## Ensure problem description exists - -```toml donna -id = "ensure_problem_description_exists" -kind = "donna.lib.request_action" -``` - -At this point, you SHOULD have a clear description of the problem in your context. I.e., you know what you need to do in this workflow. - -1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `@/.session/donna/**` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. -3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_artifact") }}`. - -## Prepare research artifact - -```toml donna -id = "prepare_artifact" -kind = "donna.lib.request_action" -``` - -1. Based on the problem description you have, suggest an artifact name in the format `@/.session/donna/research/<short-problem-related-identifier>.donna.md`. `<short-problem-related-identifier>` MUST be unique within the session. -{# TODO: we can add donna.lib.list('@/.session/donna/**') here as the command to list all session artifacts #} -2. Create the artifact and specify an original problem description in it. -3. `{{ donna.lib.goto("formalize_research") }}` - -## Formalize Research - -```toml donna -id = "formalize_research" -kind = "donna.lib.request_action" -``` - -1. Using your knowledge about the project and the original problem description, formulate a format description of the problem -2. Update the artifact with the formalized problem description. -3. `{{ donna.lib.goto("set_primary_section_of_the_research_artifact") }}` - -## Set primary section of the research artifact - -```toml donna -id = "set_primary_section_of_the_research_artifact" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description, update the primary (h1) section of the research artifact to reflect the problem being researched. So the agents can effectively find and understand the research artifact using Donna. -2. `{{ donna.lib.goto("formulate_goals") }}` - -## Formulate Goals - -```toml donna -id = "formulate_goals" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description, formulate a list of goals that the problem solution should achieve. -2. Update the artifact with the list of goals. -3. `{{ donna.lib.goto("fomalize_form_of_final_solution") }}` - -## Formalize form of final solution - -```toml donna -id = "fomalize_form_of_final_solution" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description and the list of goals, formulate the desired form of the final solution. -2. Update the artifact with the desired form of the final solution. -3. `{{ donna.lib.goto("describe_solution_space") }}` - -## Describe solution space - -```toml donna -id = "describe_solution_space" -kind = "donna.lib.request_action" -``` - -To produce a solution, we should understand how to analyze the data and how to synthesize results into the final solution. - -For this, we need to list the axes along which we will analyze the data and the dimensions along which we will synthesize the results. - -1. List the analysis axes. -2. List the synthesis dimensions. -3. Update the artifact with the description of the solution space. -4. `{{ donna.lib.goto("describe_information_to_collect") }}` - -## Describe information to collect - -```toml donna -id = "describe_information_to_collect" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description and the list of goals, list the information to be collected to research the problem. -2. Update the artifact with the description of the information to collect. -3. `{{ donna.lib.goto("list_information_sources") }}` - -## List information sources - -```toml donna -id = "list_information_sources" -kind = "donna.lib.request_action" -``` - -1. Based on the formalized problem description and the list of goals, formulate a list of information sources that can help to research the problem. -2. Update the artifact with the list of information sources. -3. `{{ donna.lib.goto("collect_information") }}` - -## Collect information - -```toml donna -id = "collect_information" -kind = "donna.lib.request_action" -``` - -1. For each information source from the list: - 1. For each piece of required information from the description: - 1. If the source can not provide this piece of information, skip it. - 2. Access the source and collect the required information. - 3. Update the artifact with the collected information. -2. `{{ donna.lib.goto("analyze_information") }}` - -## Analyze information - -```toml donna -id = "analyze_information" -kind = "donna.lib.request_action" -``` - -1. Using the analysis axes from the solution space description, analyze the collected information. Choose the format that best represents the analysis results and suits the required solution form. -2. Update the artifact with the analysis results. -3. `{{ donna.lib.goto("synthesize_solutions") }}` - -## Synthesize solutions - -```toml donna -id = "synthesize_solutions" -kind = "donna.lib.request_action" -``` - -1. Using the synthesis dimensions from the solution space description, synthesize possible solutions based on the analysis results. Choose the format that best represents the synthesized solutions and suits the required solution form. -2. Update the artifact with the synthesized solutions. -3. `{{ donna.lib.goto("evaluate_solutions") }}` - -## Evaluate solutions - -```toml donna -id = "evaluate_solutions" -kind = "donna.lib.request_action" -``` - -1. Evaluate the synthesized solutions against the goals from the formalized problem description. -2. Update the artifact with the evaluation results. -3. `{{ donna.lib.goto("produce_final_solution") }}` - -## Produce final solution - -```toml donna -id = "produce_final_solution" -kind = "donna.lib.request_action" -``` - -1. Based on the evaluation results, produce the final solution in the desired form. -2. Update the artifact with the final solution. -3. `{{ donna.lib.goto("finish") }}` - -## Finish - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -Research workflow completed. Provide the final research output to the developer. diff --git a/donna/lib/__init__.py b/donna/lib/__init__.py index b7045897..9d5bc56e 100644 --- a/donna/lib/__init__.py +++ b/donna/lib/__init__.py @@ -1,17 +1,14 @@ """Shared instances for standard library kind definitions.""" -from donna.primitives.artifacts.specification import Specification, Text from donna.primitives.artifacts.workflow import Workflow from donna.primitives.directives.goto import GoTo -from donna.primitives.directives.list import List from donna.primitives.directives.task_variable import TaskVariable -from donna.primitives.directives.view import View -from donna.primitives.operations.finish_workflow import FinishWorkflow -from donna.primitives.operations.output import Output -from donna.primitives.operations.request_action import RequestAction -from donna.primitives.operations.run_script import RunScript +from donna.primitives.sections.finish_workflow import FinishWorkflow +from donna.primitives.sections.output import Output +from donna.primitives.sections.request_action import RequestAction +from donna.primitives.sections.run_script import RunScript +from donna.primitives.sections.text import Text -specification = Specification() workflow = Workflow() text = Text() request_action = RequestAction() @@ -19,7 +16,5 @@ output = Output() run_script = RunScript() -view = View(analyze_id="view") -list = List(analyze_id="list") goto = GoTo(analyze_id="goto") task_variable = TaskVariable(analyze_id="task_variable") diff --git a/donna/lib/sources.py b/donna/lib/sources.py deleted file mode 100644 index 2e916323..00000000 --- a/donna/lib/sources.py +++ /dev/null @@ -1,5 +0,0 @@ -"""Shared source constructor instances for default configuration.""" - -from donna.workspaces.sources.markdown import MarkdownSourceConstructor - -markdown = MarkdownSourceConstructor() diff --git a/donna/machine/artifacts.py b/donna/machine/artifacts.py index 068d16c8..9a081ab3 100644 --- a/donna/machine/artifacts.py +++ b/donna/machine/artifacts.py @@ -1,8 +1,5 @@ -import ast from typing import Any -import pydantic - from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error @@ -17,7 +14,6 @@ class ArtifactSectionConfig(BaseEntity): id: SectionId kind: PythonPath - tags: list[str] = pydantic.Field(default_factory=list) class ArtifactSectionMeta(BaseEntity): @@ -31,7 +27,6 @@ class ArtifactSection(BaseEntity): kind: PythonPath title: str description: str - tags: list[str] = pydantic.Field(default_factory=list) primary: bool = False meta: ArtifactSectionMeta @@ -133,71 +128,6 @@ def markdown_blocks(self) -> Result[list[str], ErrorsList]: return Ok(blocks) -class ArtifactPredicate(BaseEntity): - source: str - - @classmethod - def parse(cls, source: str) -> Result["ArtifactPredicate", ErrorsList]: - from donna.machine import errors as machine_errors - - normalized = source.strip() - if not normalized: - return Err( - [ - machine_errors.InvalidArtifactPredicateExpression( - predicate=source, - reason="Predicate must not be empty.", - ) - ] - ) - - try: - tree = ast.parse(normalized, mode="eval") - except SyntaxError as exc: - return Err( - [ - machine_errors.InvalidArtifactPredicateExpression( - predicate=normalized, - reason=f"Invalid Python expression: {exc.msg}", - ) - ] - ) - - for node in ast.walk(tree): - if isinstance(node, ast.Call): - return Err( - [ - machine_errors.InvalidArtifactPredicateExpression( - predicate=normalized, - reason="Function calls are not allowed in artifact predicates.", - ) - ] - ) - - return Ok(cls(source=normalized)) - - def evaluate(self, section: Any) -> Result[bool, ErrorsList]: - from donna.machine import errors as machine_errors - - try: - result = eval( # noqa: S307 - compile(self.source, "<artifact-predicate>", "eval"), - {"__builtins__": {}}, - {"section": section}, - ) - except Exception as exc: - return Err( - [ - machine_errors.ArtifactPredicateEvaluationFailed( - predicate=self.source, - reason=str(exc), - ) - ] - ) - - return Ok(bool(result)) - - class ArtifactNode(Node): __slots__ = ("_artifact",) diff --git a/donna/machine/errors.py b/donna/machine/errors.py index c9689b39..c9da94e1 100644 --- a/donna/machine/errors.py +++ b/donna/machine/errors.py @@ -106,29 +106,6 @@ class PrimitiveNotPrimitive(EnvironmentError): import_path: str -class InvalidArtifactPredicateExpression(EnvironmentError): - code: str = "donna.artifacts.invalid_predicate_expression" - message: str = "Artifact predicate `{error.predicate}` is invalid: {error.reason}" - ways_to_fix: list[str] = [ - "Use a valid Python expression.", - "Use `section` as the artifact primary-section variable.", - "Do not use function calls in predicate expressions.", - ] - predicate: str - reason: str - - -class ArtifactPredicateEvaluationFailed(EnvironmentError): - code: str = "donna.artifacts.predicate_evaluation_failed" - message: str = "Artifact predicate `{error.predicate}` evaluation failed: {error.reason}" - ways_to_fix: list[str] = [ - "Ensure the predicate expression uses available variables (for example: `section`).", - "Use valid operations for values in artifact primary section.", - ] - predicate: str - reason: str - - class ArtifactValidationError(EnvironmentError): cell_kind: str = "artifact_validation_error" artifact_id: ArtifactId diff --git a/donna/machine/primitives.py b/donna/machine/primitives.py index f5c940c3..f5748db7 100644 --- a/donna/machine/primitives.py +++ b/donna/machine/primitives.py @@ -12,11 +12,9 @@ from donna.machine.artifacts import ArtifactSectionConfig if TYPE_CHECKING: - from donna.machine.artifacts import Artifact, ArtifactSection + from donna.machine.artifacts import Artifact from donna.machine.changes import Change from donna.machine.tasks import Task, WorkUnit - from donna.workspaces.config import SourceConfig as SourceConfigModel - from donna.workspaces.sources.base import SourceConfig as SourceConfigValue # TODO: Currently it is a kind of God interface. It is convenient for now. @@ -28,7 +26,7 @@ def validate_section(self, artifact: "Artifact", section_id: SectionId) -> Resul return Ok(None) def execute_section( - self, task: "Task", unit: "WorkUnit", section: "ArtifactSection" + self, task: "Task", unit: "WorkUnit", artifact: "Artifact", section_id: SectionId ) -> Result[list["Change"], ErrorsList]: raise machine_errors.PrimitiveMethodUnsupported( primitive_name=self.__class__.__name__, method_name="execute_section()" @@ -39,11 +37,6 @@ def apply_directive(self, context: Context, *argv: Any, **kwargs: Any) -> Result primitive_name=self.__class__.__name__, method_name="apply_directive()" ) - def construct_source(self, config: "SourceConfigModel") -> "SourceConfigValue": - raise machine_errors.PrimitiveMethodUnsupported( - primitive_name=self.__class__.__name__, method_name="construct_source()" - ) - @unwrap_to_error def resolve_primitive(primitive_id: PythonPath | str) -> Result[Primitive, ErrorsList]: # noqa: CCR001 diff --git a/donna/machine/state.py b/donna/machine/state.py index 5f41bfbe..c3574d6c 100644 --- a/donna/machine/state.py +++ b/donna/machine/state.py @@ -175,7 +175,8 @@ def complete_action_request( @unwrap_to_error def start_workflow(self, full_operation_id: ArtifactSectionId) -> Result[None, ErrorsList]: - workflow = context().artifacts.resolve_section(full_operation_id, RENDER_CONTEXT_VIEW).unwrap() + artifact = context().artifacts.load(full_operation_id.artifact_id, RENDER_CONTEXT_VIEW).unwrap() + workflow = artifact.get_section(full_operation_id.local_id).unwrap() machine_journal.add( message=f"Start workflow `{workflow.title}`", @@ -188,7 +189,8 @@ def start_workflow(self, full_operation_id: ArtifactSectionId) -> Result[None, E def finish_workflow(self, task_id: TaskId) -> None: task = self.current_task assert task is not None - workflow = context().artifacts.resolve_section(task.workflow_id, RENDER_CONTEXT_VIEW).unwrap() + artifact = context().artifacts.load(task.workflow_id.artifact_id, RENDER_CONTEXT_VIEW).unwrap() + workflow = artifact.get_section(task.workflow_id.local_id).unwrap() machine_journal.add( message=f"Finish workflow `{workflow.title}`", diff --git a/donna/machine/tasks.py b/donna/machine/tasks.py index 538b7d3d..1ee18cb2 100644 --- a/donna/machine/tasks.py +++ b/donna/machine/tasks.py @@ -66,7 +66,8 @@ def run(self, task: Task) -> Result[list["Change"], ErrorsList]: ) ctx = context() with ctx.current_operation_id.scope(self.operation_id): - operation = ctx.artifacts.resolve_section(self.operation_id, render_context).unwrap() + artifact = ctx.artifacts.load(self.operation_id.artifact_id, render_context).unwrap() + operation = artifact.get_section(self.operation_id.local_id).unwrap() operation_kind = ctx.primitives.resolve(operation.kind).unwrap() machine_journal.add( @@ -74,6 +75,6 @@ def run(self, task: Task) -> Result[list["Change"], ErrorsList]: message=operation.title, ).unwrap() - changes = operation_kind.execute_section(task, self, operation).unwrap() + changes = operation_kind.execute_section(task, self, artifact, operation.id).unwrap() return Ok(changes) diff --git a/donna/primitives/artifacts/specification.py b/donna/primitives/artifacts/specification.py deleted file mode 100644 index f903e700..00000000 --- a/donna/primitives/artifacts/specification.py +++ /dev/null @@ -1,31 +0,0 @@ -from typing import TYPE_CHECKING, ClassVar - -import pydantic - -from donna.machine.artifacts import ArtifactSectionConfig -from donna.machine.primitives import Primitive -from donna.workspaces.sources.markdown import MarkdownSectionMixin - -if TYPE_CHECKING: - pass - - -class TextConfig(ArtifactSectionConfig): - pass - - -class Text(MarkdownSectionMixin, Primitive): - config_class: ClassVar[type[TextConfig]] = TextConfig - - -class SpecificationConfig(ArtifactSectionConfig): - @pydantic.field_validator("tags", mode="after") - @classmethod - def ensure_specification_tag(cls, value: list[str]) -> list[str]: - if "specification" in value: - return value - return [*value, "specification"] - - -class Specification(MarkdownSectionMixin, Primitive): - config_class: ClassVar[type[SpecificationConfig]] = SpecificationConfig diff --git a/donna/primitives/artifacts/workflow.py b/donna/primitives/artifacts/workflow.py index 8c8f6b03..4db59536 100644 --- a/donna/primitives/artifacts/workflow.py +++ b/donna/primitives/artifacts/workflow.py @@ -1,7 +1,5 @@ from typing import TYPE_CHECKING, ClassVar, cast -import pydantic - from donna.core import errors as core_errors from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error @@ -12,7 +10,7 @@ from donna.machine.operations import FsmMode, OperationMeta from donna.machine.primitives import Primitive from donna.workspaces import markdown -from donna.workspaces.sources.markdown import MarkdownSectionMixin +from donna.workspaces.markdown_parser import MarkdownSectionMixin if TYPE_CHECKING: from donna.machine.changes import Change @@ -30,6 +28,15 @@ class WrongStartOperation(ArtifactValidationError): start_operation_id: SectionId +class StartOperationMissing(ArtifactValidationError): + code: str = "donna.workflows.start_operation_missing" + message: str = "Workflow section `{error.section_id}` does not define a start operation and has no tail sections." + ways_to_fix: list[str] = [ + "Add at least one H2 operation section.", + "Set `start_operation_id` explicitly in the workflow H1 config block.", + ] + + class SectionIsNotAnOperation(ArtifactValidationError): code: str = "donna.workflows.section_is_not_an_operation" message: str = "Section `{error.workflow_section_id}` is not an operation and cannot be part of the workflow." @@ -94,20 +101,15 @@ def find_workflow_sections(start_operation_id: SectionId, artifact: Artifact) -> class WorkflowConfig(ArtifactSectionConfig): - start_operation_id: SectionId - - @pydantic.field_validator("tags", mode="after") - @classmethod - def ensure_workflow_tag(cls, value: list[str]) -> list[str]: - if "workflow" in value: - return value - return [*value, "workflow"] + start_operation_id: SectionId | None = None class WorkflowMeta(ArtifactSectionMeta): - start_operation_id: SectionId + start_operation_id: SectionId | None = None def cells_meta(self) -> dict[str, object]: + if self.start_operation_id is None: + return {} return {"start_operation_id": str(self.start_operation_id)} @@ -127,27 +129,41 @@ def markdown_construct_meta( @unwrap_to_error def execute_section( - self, task: "Task", unit: "WorkUnit", section: ArtifactSection + self, task: "Task", unit: "WorkUnit", artifact: Artifact, section_id: SectionId ) -> Result[list["Change"], ErrorsList]: from donna.machine.changes import ChangeAddWorkUnit + section = artifact.get_section(section_id).unwrap() + start_operation_id = self._resolve_start_operation_id(artifact, section).unwrap() + full_id = artifact.id.to_full_local(start_operation_id) + + return Ok([ChangeAddWorkUnit(task_id=task.id, operation_id=full_id)]) + + def _resolve_start_operation_id( + self, + artifact: Artifact, + section: ArtifactSection, + ) -> Result[SectionId, ErrorsList]: if not isinstance(section.meta, WorkflowMeta): - return Err([WorkflowSectionNotWorkflow(artifact_id=section.artifact_id, section_id=section.id)]) + return Err([WorkflowSectionNotWorkflow(artifact_id=artifact.id, section_id=section.id)]) - full_id = section.artifact_id.to_full_local(section.meta.start_operation_id) + if section.meta.start_operation_id is not None: + return Ok(section.meta.start_operation_id) - return Ok([ChangeAddWorkUnit(task_id=task.id, operation_id=full_id)]) + for tail_section in artifact.sections: + if tail_section.primary: + continue + + return Ok(tail_section.id) + + return Err([StartOperationMissing(artifact_id=artifact.id, section_id=section.id)]) @unwrap_to_error def validate_section( # noqa: CCR001, CFQ001 self, artifact: Artifact, section_id: SectionId ) -> Result[None, ErrorsList]: section = artifact.get_section(section_id).unwrap() - - if not isinstance(section.meta, WorkflowMeta): - return Err([WorkflowSectionNotWorkflow(artifact_id=artifact.id, section_id=section_id)]) - - start_operation_id = section.meta.start_operation_id + start_operation_id = self._resolve_start_operation_id(artifact, section).unwrap() errors: ErrorsList = [] diff --git a/donna/primitives/directives/list.py b/donna/primitives/directives/list.py deleted file mode 100644 index a81007c7..00000000 --- a/donna/primitives/directives/list.py +++ /dev/null @@ -1,95 +0,0 @@ -from typing import Any - -from jinja2.runtime import Context - -from donna.core import errors as core_errors -from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain import errors as domain_errors -from donna.domain.artifact_ids import ArtifactIdPattern, normalize_path -from donna.machine.artifacts import ArtifactPredicate -from donna.machine.templates import Directive, PreparedDirectiveResult -from donna.workspaces import config as workspace_config - - -class EnvironmentError(core_errors.EnvironmentError): - cell_kind: str = "directive_error" - - -class ListInvalidArguments(EnvironmentError): - code: str = "donna.directives.list.invalid_arguments" - message: str = ( - "List directive requires exactly one positional argument: artifact_id_pattern (got {error.provided_count})." - ) - ways_to_fix: list[str] = ["Provide exactly one argument: artifact_id_pattern."] - provided_count: int - - -class ListInvalidKeyword(EnvironmentError): - code: str = "donna.directives.list.invalid_keyword" - message: str = "List directive accepts only the `predicate` keyword argument (got {error.keyword})." - ways_to_fix: list[str] = ["Remove unsupported keyword arguments."] - keyword: str - - -class ListInvalidPredicate(EnvironmentError): - code: str = "donna.directives.list.invalid_predicate" - message: str = "List directive `predicate` must be a string." - ways_to_fix: list[str] = ["Provide predicate as a string, e.g. predicate='section.kind == \"...\"'."] - - -class List(Directive): - @unwrap_to_error - def _prepare_arguments( # noqa: CCR001 - self, - context: Context, - *argv: Any, - **kwargs: Any, - ) -> PreparedDirectiveResult: - if argv is None or len(argv) != 1: - return Err([ListInvalidArguments(provided_count=0 if argv is None else len(argv))]) - - for keyword in kwargs: - if keyword != "predicate": - return Err([ListInvalidKeyword(keyword=keyword)]) - - artifact_pattern_text = str(argv[0]) - normalized = normalize_path(artifact_pattern_text, relative_to=context["artifact_id"], allow_wildcards=True) - if normalized is None: - return Err( - [domain_errors.InvalidIdPattern(id_type=ArtifactIdPattern.__name__, value=artifact_pattern_text)] - ) - - artifact_pattern = ArtifactIdPattern.parse(str(normalized)).unwrap() - - predicate = kwargs.get("predicate") - if predicate is None: - parsed_predicate: ArtifactPredicate | None = None - elif isinstance(predicate, str): - parsed_predicate = ArtifactPredicate.parse(predicate).unwrap() - else: - return Err([ListInvalidPredicate()]) - - return Ok((artifact_pattern, parsed_predicate)) - - def render_view( - self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None - ) -> Result[Any, ErrorsList]: - protocol = workspace_config.protocol().value - root_dir = workspace_config.project_dir() - predicate_suffix = "" - if predicate is not None: - escaped = predicate.source.replace("'", "'\"'\"'") - predicate_suffix = f" --predicate '{escaped}'" - return Ok( - f"{artifact_pattern} (donna -p {protocol} -r '{root_dir}' " - f"artifacts list '{artifact_pattern}'{predicate_suffix})" - ) - - def render_analyze( - self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None - ) -> Result[Any, ErrorsList]: - if predicate is None: - return Ok(f"$$donna {self.analyze_id} {artifact_pattern} donna$$") - - return Ok(f"$$donna {self.analyze_id} {artifact_pattern} predicate={predicate.source!r} donna$$") diff --git a/donna/primitives/directives/view.py b/donna/primitives/directives/view.py deleted file mode 100644 index 8637df9c..00000000 --- a/donna/primitives/directives/view.py +++ /dev/null @@ -1,95 +0,0 @@ -from typing import Any - -from jinja2.runtime import Context - -from donna.core import errors as core_errors -from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain import errors as domain_errors -from donna.domain.artifact_ids import ArtifactIdPattern, normalize_path -from donna.machine.artifacts import ArtifactPredicate -from donna.machine.templates import Directive, PreparedDirectiveResult -from donna.workspaces import config as workspace_config - - -class EnvironmentError(core_errors.EnvironmentError): - cell_kind: str = "directive_error" - - -class ViewInvalidArguments(EnvironmentError): - code: str = "donna.directives.view.invalid_arguments" - message: str = ( - "View directive requires exactly one positional argument: artifact_id_pattern (got {error.provided_count})." - ) - ways_to_fix: list[str] = ["Provide exactly one argument: artifact_id_pattern."] - provided_count: int - - -class ViewInvalidKeyword(EnvironmentError): - code: str = "donna.directives.view.invalid_keyword" - message: str = "View directive accepts only the `predicate` keyword argument (got {error.keyword})." - ways_to_fix: list[str] = ["Remove unsupported keyword arguments."] - keyword: str - - -class ViewInvalidPredicate(EnvironmentError): - code: str = "donna.directives.view.invalid_predicate" - message: str = "View directive `predicate` must be a string." - ways_to_fix: list[str] = ["Provide predicate as a string, e.g. predicate='section.kind == \"...\"'."] - - -class View(Directive): - @unwrap_to_error - def _prepare_arguments( # noqa: CCR001 - self, - context: Context, - *argv: Any, - **kwargs: Any, - ) -> PreparedDirectiveResult: - if argv is None or len(argv) != 1: - return Err([ViewInvalidArguments(provided_count=0 if argv is None else len(argv))]) - - for keyword in kwargs: - if keyword != "predicate": - return Err([ViewInvalidKeyword(keyword=keyword)]) - - artifact_pattern_text = str(argv[0]) - normalized = normalize_path(artifact_pattern_text, relative_to=context["artifact_id"], allow_wildcards=True) - if normalized is None: - return Err( - [domain_errors.InvalidIdPattern(id_type=ArtifactIdPattern.__name__, value=artifact_pattern_text)] - ) - - artifact_pattern = ArtifactIdPattern.parse(str(normalized)).unwrap() - - predicate = kwargs.get("predicate") - if predicate is None: - parsed_predicate: ArtifactPredicate | None = None - elif isinstance(predicate, str): - parsed_predicate = ArtifactPredicate.parse(predicate).unwrap() - else: - return Err([ViewInvalidPredicate()]) - - return Ok((artifact_pattern, parsed_predicate)) - - def render_view( - self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None - ) -> Result[Any, ErrorsList]: - protocol = workspace_config.protocol().value - root_dir = workspace_config.project_dir() - predicate_suffix = "" - if predicate is not None: - escaped = predicate.source.replace("'", "'\"'\"'") - predicate_suffix = f" --predicate '{escaped}'" - return Ok( - f"{artifact_pattern} (donna -p {protocol} -r '{root_dir}' " - f"artifacts view '{artifact_pattern}'{predicate_suffix})" - ) - - def render_analyze( - self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None - ) -> Result[Any, ErrorsList]: - if predicate is None: - return Ok(f"$$donna {self.analyze_id} {artifact_pattern} donna$$") - - return Ok(f"$$donna {self.analyze_id} {artifact_pattern} predicate={predicate.source!r} donna$$") diff --git a/donna/primitives/operations/__init__.py b/donna/primitives/sections/__init__.py similarity index 100% rename from donna/primitives/operations/__init__.py rename to donna/primitives/sections/__init__.py diff --git a/donna/primitives/operations/finish_workflow.py b/donna/primitives/sections/finish_workflow.py similarity index 81% rename from donna/primitives/operations/finish_workflow.py rename to donna/primitives/sections/finish_workflow.py index c551f3c8..45c76d8b 100644 --- a/donna/primitives/operations/finish_workflow.py +++ b/donna/primitives/sections/finish_workflow.py @@ -3,12 +3,13 @@ from donna.core.errors import ErrorsList from donna.core.result import Ok, Result, unwrap_to_error from donna.domain.artifact_ids import ArtifactId -from donna.machine.artifacts import ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta +from donna.domain.ids import SectionId +from donna.machine.artifacts import Artifact, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.operations import FsmMode, OperationConfig, OperationKind, OperationMeta from donna.protocol import cell_shortcuts from donna.protocol.utils import instant_output_cell from donna.workspaces import markdown -from donna.workspaces.sources.markdown import MarkdownSectionMixin +from donna.workspaces.markdown_parser import MarkdownSectionMixin if TYPE_CHECKING: from donna.machine.changes import Change @@ -22,10 +23,11 @@ class FinishWorkflowConfig(OperationConfig): class FinishWorkflow(MarkdownSectionMixin, OperationKind): @unwrap_to_error def execute_section( - self, task: "Task", unit: "WorkUnit", operation: ArtifactSection + self, task: "Task", unit: "WorkUnit", artifact: Artifact, section_id: SectionId ) -> Result[list["Change"], ErrorsList]: from donna.machine.changes import ChangeFinishTask + operation = artifact.get_section(section_id).unwrap() info = cell_shortcuts.info(operation.description) instant_output_cell(info) diff --git a/donna/primitives/operations/output.py b/donna/primitives/sections/output.py similarity index 90% rename from donna/primitives/operations/output.py rename to donna/primitives/sections/output.py index 6fff8800..a4e54334 100644 --- a/donna/primitives/operations/output.py +++ b/donna/primitives/sections/output.py @@ -4,13 +4,13 @@ from donna.core.result import Err, Ok, Result, unwrap_to_error from donna.domain.artifact_ids import ArtifactId from donna.domain.ids import SectionId -from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta +from donna.machine.artifacts import Artifact, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.errors import ArtifactValidationError from donna.machine.operations import OperationConfig, OperationKind, OperationMeta from donna.protocol import cell_shortcuts from donna.protocol.utils import instant_output_cell from donna.workspaces import markdown -from donna.workspaces.sources.markdown import MarkdownSectionMixin +from donna.workspaces.markdown_parser import MarkdownSectionMixin if TYPE_CHECKING: from donna.machine.changes import Change @@ -60,10 +60,11 @@ def markdown_construct_meta( @unwrap_to_error def execute_section( - self, task: "Task", unit: "WorkUnit", operation: ArtifactSection + self, task: "Task", unit: "WorkUnit", artifact: Artifact, section_id: SectionId ) -> Result[list["Change"], ErrorsList]: from donna.machine.changes import ChangeAddWorkUnit + operation = artifact.get_section(section_id).unwrap() meta = cast(OutputMeta, operation.meta) info = cell_shortcuts.info(operation.description) diff --git a/donna/primitives/operations/request_action.py b/donna/primitives/sections/request_action.py similarity index 89% rename from donna/primitives/operations/request_action.py rename to donna/primitives/sections/request_action.py index a3ffd9ef..dbf1aa70 100644 --- a/donna/primitives/operations/request_action.py +++ b/donna/primitives/sections/request_action.py @@ -9,10 +9,10 @@ from donna.domain.artifact_ids import ArtifactId from donna.domain.ids import SectionId from donna.machine.action_requests import ActionRequest -from donna.machine.artifacts import ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta +from donna.machine.artifacts import Artifact, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.operations import FsmMode, OperationConfig, OperationKind, OperationMeta from donna.workspaces import markdown -from donna.workspaces.sources.markdown import MarkdownSectionMixin +from donna.workspaces.markdown_parser import MarkdownSectionMixin if TYPE_CHECKING: from donna.machine.changes import Change @@ -74,10 +74,11 @@ def markdown_construct_meta( @unwrap_to_error def execute_section( - self, task: "Task", unit: "WorkUnit", operation: ArtifactSection + self, task: "Task", unit: "WorkUnit", artifact: Artifact, section_id: SectionId ) -> Result[list["Change"], ErrorsList]: from donna.machine.changes import ChangeAddActionRequest + operation = artifact.get_section(section_id).unwrap() request_text = operation.description full_operation_id = unit.operation_id diff --git a/donna/primitives/operations/run_script.py b/donna/primitives/sections/run_script.py similarity index 95% rename from donna/primitives/operations/run_script.py rename to donna/primitives/sections/run_script.py index 2296cbd1..38d1d383 100644 --- a/donna/primitives/operations/run_script.py +++ b/donna/primitives/sections/run_script.py @@ -12,12 +12,12 @@ from donna.domain.artifact_ids import ArtifactId from donna.domain.ids import SectionId from donna.machine import journal as machine_journal -from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta +from donna.machine.artifacts import Artifact, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.errors import ArtifactValidationError from donna.machine.operations import OperationConfig, OperationKind, OperationMeta from donna.workspaces import config as workspace_config from donna.workspaces import markdown -from donna.workspaces.sources.markdown import MarkdownSectionMixin +from donna.workspaces.markdown_parser import MarkdownSectionMixin if TYPE_CHECKING: from donna.machine.changes import Change @@ -25,7 +25,7 @@ class InternalError(core_errors.InternalError): - """Base class for internal errors in donna.primitives.operations.run_script.""" + """Base class for internal errors in donna.primitives.sections.run_script.""" class RunScriptMissingGotoOnSuccess(ArtifactValidationError): @@ -142,10 +142,11 @@ def markdown_construct_meta( @unwrap_to_error def execute_section( - self, task: "Task", unit: "WorkUnit", operation: ArtifactSection + self, task: "Task", unit: "WorkUnit", artifact: Artifact, section_id: SectionId ) -> Result[list["Change"], ErrorsList]: from donna.machine.changes import ChangeAddWorkUnit, ChangeSetTaskContext + operation = artifact.get_section(section_id).unwrap() meta = cast(RunScriptMeta, operation.meta) script = meta.script diff --git a/donna/primitives/sections/text.py b/donna/primitives/sections/text.py new file mode 100644 index 00000000..f1b547ec --- /dev/null +++ b/donna/primitives/sections/text.py @@ -0,0 +1,13 @@ +from typing import ClassVar + +from donna.machine.artifacts import ArtifactSectionConfig +from donna.machine.primitives import Primitive +from donna.workspaces.markdown_parser import MarkdownSectionMixin + + +class TextConfig(ArtifactSectionConfig): + pass + + +class Text(MarkdownSectionMixin, Primitive): + config_class: ClassVar[type[TextConfig]] = TextConfig diff --git a/donna/skills/fixtures/artifacts.md b/donna/skills/fixtures/artifacts.md index fe6c1437..07cec236 100644 --- a/donna/skills/fixtures/artifacts.md +++ b/donna/skills/fixtures/artifacts.md @@ -2,137 +2,89 @@ Donna artifacts are project files that Donna can discover, render, validate, and execute as workflow input. They are usually Markdown files with the `.donna.md` extension. -Donna reads artifacts from the project filesystem. It does not mutate project artifacts through `artifacts` commands. Developers and agents edit files directly, then ask Donna to list, view, or validate them. +Donna reads artifacts from the project filesystem. It does not mutate project artifacts through `artifacts` commands. Developers and agents edit files directly, then ask Donna to list workflows or validate artifacts. ## Artifact Locations The common artifact areas are: -- `<project-root>/specs`: project-owned specifications and workflows. -- `<project-root>/.agents/donna`: synced built-in Donna specs and workflows. +- `<project-root>/workflows`: project-owned workflows. +- `<project-root>/.agents/donna`: project-local Donna documentation, when present. - `<project-root>/.session/donna`: session artifacts and active workflow state. Example: ```text -specs/intro.donna.md -.agents/donna/work/polish.donna.md +workflows/polish.donna.md +workflows/rfc/request.donna.md .session/donna/current_task.donna.md ``` -Donna sees only files allowed by `donna.toml:file_filters`. +Donna sees only `.donna.md` files under directories listed in `donna.toml:workflow_dirs`. -## List Artifacts - -List all visible artifacts: - -```bash -donna -p llm artifacts list '**' -``` - -List introductions: - -```bash -donna -p llm artifacts list '**/intro.donna.md' -``` +## List Workflows List workflow artifacts: ```bash -donna -p llm artifacts list '**' --predicate '"workflow" in section.tags' -``` - -## View Artifacts - -View rendered artifact content when you need information from an artifact: - -```bash -donna -p llm artifacts view '@/specs/intro.donna.md' -``` - -View all matching introductions: - -```bash -donna -p llm artifacts view '**/intro.donna.md' +donna -p llm artifacts list ``` -Agents should prefer rendered views for reading. Read source files directly only when editing the artifact or investigating rendering problems. +Read artifact source files directly when you need details beyond the workflow introduction. ## Validate Artifacts Validate one artifact: ```bash -donna -p llm artifacts validate '@/specs/intro.donna.md' +donna -p llm artifacts validate '@/workflows/polish.donna.md' ``` Validate all visible artifacts: ```bash -donna -p llm artifacts validate '**' +donna -p llm artifacts validate --all ``` Run validation after creating or editing Donna artifacts. -## Artifact Patterns +## Artifact IDs -Use `@/` for project-root paths: +Use `@/` for project-root artifact ids: ```bash -donna -p llm artifacts view '@/specs/core/top_level_architecture.donna.md' +donna -p llm artifacts validate '@/workflows/polish.donna.md' ``` -Use recursive wildcards when the exact location is unknown: +Validate multiple specific artifacts by passing multiple ids: ```bash -donna -p llm artifacts list '**/*.donna.md' +donna -p llm artifacts validate '@/workflows/polish.donna.md' '@/workflows/rfc/request.donna.md' ``` -Pattern examples: - -- `@/*.donna.md`: Donna Markdown artifacts directly under the project root. -- `@/**/intro.donna.md`: any introduction artifact. -- `@/.agents/donna/**`: synced Donna artifacts. -- `@/.session/donna/**`: session artifacts. - -Do not pass relative filesystem paths such as `./specs/intro.donna.md`. Use `@/specs/intro.donna.md`. +Do not pass relative filesystem paths such as `./workflows/polish.donna.md`. Use `@/workflows/polish.donna.md`. ## Creating Artifacts -Create artifacts as source files in an included location. A minimal specification artifact: +Create normal documentation as plain Markdown files. A minimal documentation file: ````markdown # Example Specification -```toml donna -kind = "donna.lib.specification" -tags = ["specification"] -``` - -This artifact documents one stable project rule. +This document describes one stable project rule. ```` -After creating it, validate: - -```bash -donna -p llm artifacts validate '@/specs/example.donna.md' -``` +Documentation files are not Donna artifacts and do not need artifact validation. ## Creating Workflows -A workflow artifact defines a finite-state machine. The head section declares workflow metadata and the start operation. Each H2 section declares one operation. +A workflow artifact defines a finite-state machine. The H1 section is a workflow by default. Each H2 section declares one operation. When the H1 config omits `start_operation_id`, Donna starts from the first H2 section. Minimal workflow: ````markdown # Example Workflow -```toml donna -kind = "donna.lib.workflow" -tags = ["workflow"] -start_operation_id = "ask_agent" -``` - This workflow asks the agent to do one thing and finish. ## Do The Work @@ -159,38 +111,32 @@ The workflow is complete. Validate the workflow before running it: ```bash -donna -p llm artifacts validate '@/specs/work/example.donna.md' +donna -p llm artifacts validate '@/workflows/example.donna.md' ``` Run it: ```bash -donna -p llm sessions run '@/specs/work/example.donna.md' +donna -p llm sessions run '@/workflows/example.donna.md' ``` ## Managing Artifacts -Use direct file edits to create, update, move, or delete artifact sources. Then use Donna to inspect the result. +Use direct file edits to create, update, move, or delete artifact files. Then use Donna to inspect the result. Recommended loop: 1. Edit the artifact source file. -2. View the rendered artifact: - -```bash -donna -p llm artifacts view '@/specs/example.donna.md' -``` - -3. Validate the artifact: +2. Validate the artifact: ```bash donna -p llm artifacts validate '@/specs/example.donna.md' ``` -4. If it is a workflow, list it with the workflow predicate: +3. If it is a workflow, list it: ```bash -donna -p llm artifacts list '**' --predicate '"workflow" in section.tags' +donna -p llm artifacts list ``` Keep artifact files concise. Put project-wide explanations in specifications and operational step-by-step instructions in workflows. diff --git a/donna/skills/fixtures/configuration.md b/donna/skills/fixtures/configuration.md index f73d73ac..a5b5729d 100644 --- a/donna/skills/fixtures/configuration.md +++ b/donna/skills/fixtures/configuration.md @@ -6,116 +6,77 @@ Donna project configuration lives at: <project-root>/donna.toml ``` -The file is created by `donna -p llm workspaces init`. Edit it when the project needs custom artifact sources, artifact visibility rules, cache behavior, or journal forwarding. +The file is created by `donna -p llm workspaces init`. Edit it when the project needs workflow source directories, default section settings, cache behavior, or journal forwarding. ## Minimal Configuration A default project can use the generated configuration without manual edits. The effective defaults are: ```toml -session = ".session/donna" +session_dir = ".session/donna" +default_section_kind = "donna.lib.text" +default_primary_section_kind = "donna.lib.workflow" +default_primary_section_id = "primary" cache_lifetime = 1.0 - -[[sources]] -kind = "donna.lib.sources.markdown" -extension = ".donna.md" - -[[file_filters]] -mode = "include" -pattern = "@/.session/donna/**/*.donna.md" - -[[file_filters]] -mode = "include" -pattern = "@/.agents/**/*.donna.md" - -[[file_filters]] -mode = "ignore" -pattern = ".*/**" - -[[file_filters]] -mode = "include" -pattern = "**/*.donna.md" - -[[file_filters]] -mode = "ignore" -pattern = "**" +workflow_dirs = ["./workflows", "./.session/donna"] [journal] ``` ## Session Directory -`session` points to Donna's temporary session directory. +`session_dir` points to Donna's temporary session directory. Donna stores runtime state, action requests, and session-created artifacts there. Default: ```toml -session = ".session/donna" +session_dir = ".session/donna" ``` Relative paths are resolved from the project root; absolute paths are used as configured. Use a directory ignored by version control unless a project intentionally tracks session artifacts. -## Sources +## Default Sections -`sources` tell Donna how to load artifacts with specific filename extensions. +Donna loads artifacts only from `*.donna.md` Markdown files. -Default Markdown source: +When a non-primary section omits `kind`, Donna uses `default_section_kind`. When the primary section omits `kind`, Donna uses `default_primary_section_kind`. When the primary section omits `id`, Donna uses `default_primary_section_id`. ```toml -[[sources]] -kind = "donna.lib.sources.markdown" -extension = ".donna.md" +default_section_kind = "donna.lib.text" +default_primary_section_kind = "donna.lib.workflow" +default_primary_section_id = "primary" ``` Fields: -- `kind`: full Python path to a Donna source constructor. -- `extension`: filename suffix handled by that source. +- `default_section_kind`: full Python path to the primitive used for sections without an explicit `kind`. +- `default_primary_section_kind`: full Python path to the primitive used for the primary H1 section without an explicit `kind`. +- `default_primary_section_id`: section id assigned to the primary H1 section when it omits `id`. -Add a source only when Donna has a source implementation for that artifact format. Keep `.donna.md` configured unless the project intentionally disables default Markdown artifacts. +## Workflow Directories -Example with an additional custom source: +`workflow_dirs` controls where Donna searches for workflow artifacts. Donna recursively scans each configured directory and recognizes only files ending with `.donna.md`. -```toml -[[sources]] -kind = "donna.lib.sources.markdown" -extension = ".donna.md" +Default: -[[sources]] -kind = "project.donna_sources.yaml" -extension = ".donna.yaml" +```toml +workflow_dirs = ["./workflows", "./.session/donna"] ``` -## File Filters - -`file_filters` control which project files Donna can see as artifacts. Filters are evaluated in order. The first matching rule decides whether a file is included, ignored, or required. - -Modes: - -- `include`: admit matching files when they exist. -- `ignore`: hide matching files. -- `required`: admit matching files and treat missing expected files as an error when relevant. - -Patterns are Donna artifact patterns rooted at the project. Use `@/` for explicit project-root paths and `**` for recursive matching. +Paths are relative to the Donna project root. Missing directories are ignored, so a project can keep the default list before all of those directories exist. Example: ```toml -[[file_filters]] -mode = "include" -pattern = "@/specs/**/*.donna.md" - -[[file_filters]] -mode = "ignore" -pattern = "@/specs/archive/**" - -[[file_filters]] -mode = "ignore" -pattern = "**" +workflow_dirs = [ + "./workflows", + "./.session/donna", + "./team-workflows", +] ``` -Put narrow rules before broad rules. Keep a final `ignore "**"` rule when you want an allow-list. +Use narrower directories when you want Donna to ignore unrelated `.donna.md` files elsewhere in the project. ## Journal Forwarding @@ -173,8 +134,8 @@ Use a smaller value when artifacts are edited rapidly by external tools. Use the After editing `donna.toml`, run: ```bash -donna -p llm artifacts list '**' -donna -p llm artifacts validate '**' +donna -p llm artifacts list +donna -p llm artifacts validate --all ``` -If Donna cannot load the project config, inspect the reported configuration error and fix the TOML or unsupported source path before continuing workflow work. +If Donna cannot load the project config, inspect the reported configuration error and fix the TOML before continuing workflow work. diff --git a/donna/skills/fixtures/initialization.md b/donna/skills/fixtures/initialization.md index 58e99631..efec8480 100644 --- a/donna/skills/fixtures/initialization.md +++ b/donna/skills/fixtures/initialization.md @@ -1,8 +1,8 @@ # `donna` Initialization -Initialization creates the Donna project config and optionally installs built-in Donna skills and specs into the project. +Initialization creates the Donna project config and session directory. -Use this document when a project has no `donna.toml`, when built-in Donna fixtures are missing, or when synced fixture files need to be refreshed. +Use this document when a project has no `donna.toml`. ## What Initialization Creates @@ -11,11 +11,9 @@ Use this document when a project has no `donna.toml`, when built-in Donna fixtur ```text <project-root>/donna.toml <project-root>/.session/donna/ -<project-root>/.agents/skills/ -<project-root>/.agents/donna/ ``` -`donna.toml` stores configuration. The configured session directory stores Donna runtime state and session artifacts. The `.agents/skills` and `.agents/donna` directories contain built-in agent-facing Donna skills, workflows, and specifications. +`donna.toml` stores configuration. The configured session directory stores Donna runtime state and session artifacts. ## Initialize The Current Directory @@ -37,44 +35,6 @@ donna -p llm --root /path/to/project workspaces init The target directory must already exist. Donna creates `donna.toml` and the configured session directory inside it. -## Install Only Part Of The Fixtures - -Skip built-in skills: - -```bash -donna -p llm workspaces init --no-skills -``` - -Skip synced Donna specs and workflows: - -```bash -donna -p llm workspaces init --no-specs -``` - -Use these options only when the project deliberately manages those files another way. - -## Refresh Existing Fixtures - -Use `update`, not `init`, for an existing Donna project: - -```bash -donna -p llm workspaces update -``` - -Refresh only built-in skills: - -```bash -donna -p llm workspaces update --no-specs -``` - -Refresh only synced Donna specs and workflows: - -```bash -donna -p llm workspaces update --no-skills -``` - -`update` requires an existing `donna.toml`. - ## First Checks After Initialization Verify the project config can load: @@ -83,26 +43,20 @@ Verify the project config can load: donna -p llm sessions status ``` -List available artifacts: - -```bash -donna -p llm artifacts list '**' -``` - List available workflows: ```bash -donna -p llm artifacts list '**' --predicate '"workflow" in section.tags' +donna -p llm artifacts list ``` Validate artifacts: ```bash -donna -p llm artifacts validate '**' +donna -p llm artifacts validate --all ``` ## Agent Guidance Initialize Donna only when the developer asks for it or when the task explicitly requires Donna and no `donna.toml` exists. -Do not overwrite project-owned workflows or specifications by hand. Use `workspaces update` for built-in fixtures, and edit project-owned artifacts directly when the developer asks for project-specific behavior changes. +Edit project-owned artifacts directly when the developer asks for project-specific behavior changes. diff --git a/donna/skills/fixtures/usage.md b/donna/skills/fixtures/usage.md index 00951617..116f1704 100644 --- a/donna/skills/fixtures/usage.md +++ b/donna/skills/fixtures/usage.md @@ -5,7 +5,7 @@ Donna is a CLI tool for orchestrating AI-agent work with project-local workflows Use this document as the first reference for command usage. For narrower topics, use: - `donna skill configuration` for `donna.toml`. -- `donna skill initialization` for creating or refreshing Donna project files. +- `donna skill initialization` for creating or checking Donna project files. - `donna skill artifacts` for artifact layout, discovery, and authoring rules. - `donna skill usage` for this command overview. @@ -38,7 +38,7 @@ donna -p llm sessions status The root option goes before the command: ```bash -donna -p llm --root /path/to/project artifacts list '**' +donna -p llm --root /path/to/project artifacts list ``` ## Skill Documents @@ -58,7 +58,7 @@ Use these documents when an agent needs stable instructions before `donna.toml` ## Project Commands -Workspace commands create or refresh Donna-owned project files. +Workspace commands create or check Donna project configuration. Initialize Donna in the current directory: @@ -72,19 +72,6 @@ Initialize Donna in an explicit existing directory: donna -p llm --root /path/to/project workspaces init ``` -Refresh synced Donna skills and specifications in an existing Donna project: - -```bash -donna -p llm workspaces update -``` - -Use `--no-skills` or `--no-specs` when only one fixture family should be changed: - -```bash -donna -p llm workspaces update --no-specs -donna -p llm workspaces update --no-skills -``` - ## Session Commands All workflow execution happens in the active session. Session state lives under the configured session directory, `.session/donna` by default. @@ -118,7 +105,7 @@ donna -p llm sessions continue Run a workflow artifact: ```bash -donna -p llm sessions run @/.agents/donna/work/polish.donna.md +donna -p llm sessions run @/workflows/polish.donna.md ``` Complete an action request by passing its id and the next operation id exactly as Donna instructed: @@ -129,36 +116,24 @@ donna -p llm sessions action-request-completed AR-12-x @/.session/donna/workflow ## Artifact Commands -Artifacts are project files admitted by Donna's configured sources and file filters. Agents use artifacts to discover workflows, read specifications, and validate Donna-readable files. - -List all visible artifacts: - -```bash -donna -p llm artifacts list '**' -``` +Artifacts are `*.donna.md` project files under Donna's configured `workflow_dirs`. Agents use artifacts to discover workflows, read documentation, and validate Donna-readable files. List workflows: ```bash -donna -p llm artifacts list '**' --predicate '"workflow" in section.tags' -``` - -View an artifact: - -```bash -donna -p llm artifacts view '@/specs/intro.donna.md' +donna -p llm artifacts list ``` Validate all visible artifacts: ```bash -donna -p llm artifacts validate '**' +donna -p llm artifacts validate --all ``` -Artifact patterns use `@/` for project-root paths. Recursive `**` patterns are allowed: +Validate specific artifacts by absolute project-root id: ```bash -donna -p llm artifacts view '**/intro.donna.md' +donna -p llm artifacts validate '@/workflows/polish.donna.md' ``` ## Normal Agent Flow @@ -173,13 +148,13 @@ donna -p llm sessions status 3. If there is no active work and a workflow is needed, list workflows: ```bash -donna -p llm artifacts list '**' --predicate '"workflow" in section.tags' +donna -p llm artifacts list ``` 4. Start the selected workflow: ```bash -donna -p llm sessions run @/.agents/donna/work/polish.donna.md +donna -p llm sessions run @/workflows/polish.donna.md ``` 5. Execute Donna action requests exactly. diff --git a/donna/workspaces/artifacts.py b/donna/workspaces/artifacts.py index 10599a9a..08685831 100644 --- a/donna/workspaces/artifacts.py +++ b/donna/workspaces/artifacts.py @@ -4,7 +4,8 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.artifact_ids import ArtifactId, ArtifactIdPattern +from donna.domain.artifact_ids import ArtifactId +from donna.domain.constants import DONNA_ARTIFACT_EXTENSION from donna.domain.id_paths import NormalizedRawIdPath from donna.domain.types import Milliseconds from donna.machine.tasks import Task, WorkUnit @@ -13,7 +14,6 @@ if TYPE_CHECKING: from donna.machine.artifacts import Artifact - from donna.workspaces import config as workspace_config class ArtifactRenderContext(BaseEntity): @@ -26,7 +26,6 @@ class ArtifactRenderContext(BaseEntity): class FilesystemRawArtifact(BaseEntity): - source_id: str path: pathlib.Path def get_bytes(self) -> bytes: @@ -34,128 +33,100 @@ def get_bytes(self) -> bytes: @unwrap_to_error def render(self, artifact_id: ArtifactId, render_context: ArtifactRenderContext) -> Result["Artifact", ErrorsList]: - return Ok(render_artifact_from_source(artifact_id, self.source_id, self.get_bytes(), render_context).unwrap()) - + return Ok(render_markdown_artifact(artifact_id, self.get_bytes(), render_context).unwrap()) -def _match_supported_extension(path: pathlib.Path, supported_extensions: set[str]) -> str | None: - name = path.name.lower() - for extension in sorted(supported_extensions, key=len, reverse=True): - if name.endswith(extension): - return extension +def has_donna_artifact_extension(path: pathlib.Path | str) -> bool: + return pathlib.PurePath(path).name.lower().endswith(DONNA_ARTIFACT_EXTENSION) - return None +def _artifact_id_from_parts(parts: Sequence[str]) -> ArtifactId | None: + artifact_name = "/".join(parts) + if not ArtifactId.validate(artifact_name): + return None -def _required_filters_match_prefix( - prefix_parts: Sequence[str], filters: Sequence["workspace_config.FileFilter"] -) -> bool: - from donna.workspaces import config as workspace_config - - for file_filter in filters: - if file_filter.mode != workspace_config.FileFilterMode.required: - continue + return ArtifactId(NormalizedRawIdPath(artifact_name)) - if not file_filter.pattern.matches_prefix(prefix_parts): - return False - return True +def _workflow_dir_parts(path: pathlib.Path) -> tuple[str, ...]: + return pathlib.PurePosixPath(path.as_posix()).parts -def _is_artifact_visible( # noqa: CCR001 - artifact_id: ArtifactId, filters: Sequence["workspace_config.FileFilter"] -) -> bool: - from donna.workspaces import config as workspace_config +def _artifact_is_in_workflow_dirs(artifact_id: ArtifactId, workflow_dirs: Sequence[pathlib.Path]) -> bool: + for workflow_dir in workflow_dirs: + workflow_parts = _workflow_dir_parts(workflow_dir) + if len(artifact_id.parts) <= len(workflow_parts): + continue - included = False + if artifact_id.parts[: len(workflow_parts)] == workflow_parts: + return True - for file_filter in filters: - if file_filter.mode == workspace_config.FileFilterMode.required: - if not file_filter.pattern.matches(artifact_id): - return False + return False - continue - if included: - continue - - matches = file_filter.pattern.matches(artifact_id) - if not matches: - continue +def _artifact_is_visible_in_workspace(artifact_id: ArtifactId) -> bool: + from donna.workspaces import config as workspace_config - if file_filter.mode == workspace_config.FileFilterMode.ignore: - return False + return _artifact_is_in_workflow_dirs(artifact_id, workspace_config.config().workflow_dirs) - included = True - return True +def _artifact_path_from_filesystem_entry(entry: pathlib.Path, parts: list[str]) -> pathlib.Path | None: + if not entry.is_file(): + return None + if not has_donna_artifact_extension(entry): + return None -def _artifact_id_from_parts(parts: Sequence[str]) -> ArtifactId | None: - artifact_name = "/".join(parts) - if not ArtifactId.validate(artifact_name): + artifact_parts = parts + [entry.name] + if _artifact_id_from_parts(artifact_parts) is None: return None - return ArtifactId(NormalizedRawIdPath(artifact_name)) + return pathlib.Path(*artifact_parts) -def _artifact_is_visible_in_workspace(artifact_id: ArtifactId) -> bool: - from donna.workspaces import config as workspace_config +def _walk_workflow_dir(node: pathlib.Path, parts: list[str]) -> Iterator[pathlib.Path]: + for entry in sorted(node.iterdir(), key=lambda item: item.name): + if entry.is_dir(): + yield from _walk_workflow_dir(entry, parts + [entry.name]) + continue - return _is_artifact_visible(artifact_id, workspace_config.config().file_filters) + artifact_path = _artifact_path_from_filesystem_entry(entry, parts) + if artifact_path is not None: + yield artifact_path -def walk_filesystem(filters: list["workspace_config.FileFilter"]) -> Iterator[pathlib.Path]: # noqa: CCR001 - from donna.workspaces.config import config, project_dir +def walk_filesystem(workflow_dirs: Sequence[pathlib.Path]) -> Iterator[pathlib.Path]: + from donna.workspaces.config import project_dir root = project_dir() if not root.exists() or not root.is_dir(): return - supported_extensions = config().supported_extensions() - - def walk(node: pathlib.Path, parts: list[str]) -> Iterator[pathlib.Path]: # noqa: CCR001 - for entry in sorted(node.iterdir(), key=lambda item: item.name): - if entry.is_dir(): - next_parts = parts + [entry.name] - if not _required_filters_match_prefix(next_parts, filters): - continue - - yield from walk(entry, next_parts) - continue - - if not entry.is_file(): - continue - - if _match_supported_extension(entry, supported_extensions) is None: - continue - - artifact_parts = parts + [entry.name] - artifact_id = _artifact_id_from_parts(artifact_parts) - if artifact_id is None or not _is_artifact_visible(artifact_id, filters): - continue - - yield pathlib.Path(*artifact_parts) + for workflow_dir in workflow_dirs: + workflow_parts = _workflow_dir_parts(workflow_dir) + workflow_path = root.joinpath(*workflow_parts) + if not workflow_path.exists() or not workflow_path.is_dir(): + continue - yield from walk(root, []) + yield from _walk_workflow_dir(workflow_path, list(workflow_parts)) -def list_artifact_ids(pattern: ArtifactIdPattern) -> list[ArtifactId]: # noqa: CCR001 +def list_artifact_ids() -> list[ArtifactId]: from donna.workspaces import config as workspace_config - filters = [ - *workspace_config.config().file_filters, - workspace_config.FileFilter(mode=workspace_config.FileFilterMode.required, pattern=pattern), - ] - artifacts: list[ArtifactId] = [] + seen: set[ArtifactId] = set() - for relative_path in walk_filesystem(filters): + for relative_path in walk_filesystem(workspace_config.config().workflow_dirs): artifact_id = _artifact_id_from_parts(relative_path.parts) if artifact_id is None: continue + if artifact_id in seen: + continue + artifacts.append(artifact_id) + seen.add(artifact_id) return artifacts @@ -174,15 +145,13 @@ def resolve_artifact_path(artifact_id: ArtifactId) -> Result[pathlib.Path | None @unwrap_to_error -def fetch_artifact_bytes(artifact_id: ArtifactId) -> Result[tuple[str, bytes], ErrorsList]: +def fetch_artifact_bytes(artifact_id: ArtifactId) -> Result[bytes, ErrorsList]: raw_artifact = fetch_raw_artifact(artifact_id).unwrap() - return Ok((raw_artifact.source_id, raw_artifact.get_bytes())) + return Ok(raw_artifact.get_bytes()) @unwrap_to_error def fetch_raw_artifact(artifact_id: ArtifactId) -> Result[FilesystemRawArtifact, ErrorsList]: - from donna.workspaces.config import config - if not _artifact_is_visible_in_workspace(artifact_id): return Err([world_errors.ArtifactNotFound(artifact_id=artifact_id)]) @@ -190,47 +159,43 @@ def fetch_raw_artifact(artifact_id: ArtifactId) -> Result[FilesystemRawArtifact, if artifact_path is None: return Err([world_errors.ArtifactNotFound(artifact_id=artifact_id)]) - supported_extension = _match_supported_extension(artifact_path, config().supported_extensions()) - if supported_extension is None: + if not has_donna_artifact_extension(artifact_path): return Err( [ - world_errors.UnsupportedArtifactSourceExtension( + world_errors.UnsupportedArtifactExtension( artifact_id=artifact_id, extension="".join(artifact_path.suffixes).lower() or artifact_path.suffix.lower(), ) ] ) - source_config = config().find_source_for_extension(supported_extension) - if source_config is None: - return Err( - [ - world_errors.UnsupportedArtifactSourceExtension( - artifact_id=artifact_id, - extension=supported_extension, - ) - ] - ) - return Ok( FilesystemRawArtifact( - source_id=source_config.kind, path=artifact_path, ) ) @unwrap_to_error -def render_artifact_from_source( +def render_markdown_artifact( artifact_id: ArtifactId, - source_id: str, content: bytes, render_context: ArtifactRenderContext, ) -> Result["Artifact", ErrorsList]: from donna.workspaces.config import config + from donna.workspaces.markdown_parser import construct_artifact_from_bytes - source_config = config().get_source_config(source_id).unwrap() - return Ok(source_config.construct_artifact_from_bytes(artifact_id, content, render_context).unwrap()) + workspace_config = config() + return Ok( + construct_artifact_from_bytes( + artifact_id, + content, + render_context, + default_section_kind=workspace_config.default_section_kind, + default_primary_section_kind=workspace_config.default_primary_section_kind, + default_primary_section_id=workspace_config.default_primary_section_id, + ).unwrap() + ) @unwrap_to_error diff --git a/donna/workspaces/config.py b/donna/workspaces/config.py index 4d19c540..d58cb4c9 100644 --- a/donna/workspaces/config.py +++ b/donna/workspaces/config.py @@ -2,44 +2,20 @@ import enum import pathlib -from typing import TYPE_CHECKING, Any +from typing import TYPE_CHECKING import pydantic from donna.core.entities import BaseEntity -from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result -from donna.domain.artifact_ids import ArtifactIdPattern +from donna.domain.constants import DONNA_DEFAULT_SESSION_DIR, DONNA_DEFAULT_WORKFLOW_DIR +from donna.domain.id_paths import NormalizedRawIdPath +from donna.domain.ids import SectionId from donna.domain.python_path import PythonPath -from donna.machine.primitives import resolve_primitive from donna.workspaces import errors as world_errors -from donna.workspaces.sources.base import SourceConfig as SourceConfigValue -from donna.workspaces.sources.base import SourceConstructor if TYPE_CHECKING: from donna.protocol.modes import Mode -DONNA_CONFIG_NAME = "donna.toml" -DONNA_DEFAULT_SESSION_DIR = pathlib.Path(".session") / "donna" - - -class SourceConfig(BaseEntity): - kind: PythonPath - extension: str - - model_config = pydantic.ConfigDict(extra="allow") - - -class FileFilterMode(str, enum.Enum): - ignore = "ignore" - include = "include" - required = "required" - - -class FileFilter(BaseEntity): - mode: FileFilterMode - pattern: ArtifactIdPattern - class JournalRecordAttribute(str, enum.Enum): timestamp = "timestamp" @@ -84,95 +60,54 @@ def validate_cmd(cls, value: list[str] | None) -> list[str] | None: return value -def _default_sources() -> list[SourceConfig]: - return [ - SourceConfig.model_validate( - { - "kind": "donna.lib.sources.markdown", - "extension": ".donna.md", - } - ), - ] - - -def _default_file_filters() -> list[FileFilter]: +def _default_workflow_dirs() -> list[pathlib.Path]: return [ - FileFilter(mode=FileFilterMode.include, pattern=ArtifactIdPattern.parse("@/.agents/**/*.donna.md").unwrap()), - FileFilter(mode=FileFilterMode.ignore, pattern=ArtifactIdPattern.parse(".*/**").unwrap()), - FileFilter(mode=FileFilterMode.include, pattern=ArtifactIdPattern.parse("**/*.donna.md").unwrap()), - FileFilter(mode=FileFilterMode.ignore, pattern=ArtifactIdPattern.parse("**").unwrap()), + DONNA_DEFAULT_WORKFLOW_DIR, + DONNA_DEFAULT_SESSION_DIR, ] -class Config(BaseEntity): - session: pathlib.Path = DONNA_DEFAULT_SESSION_DIR - sources: list[SourceConfig] = pydantic.Field(default_factory=_default_sources) - file_filters: list[FileFilter] = pydantic.Field(default_factory=_default_file_filters) - journal: JournalConfig = pydantic.Field(default_factory=JournalConfig) - _sources_instances: list[SourceConfigValue] = pydantic.PrivateAttr(default_factory=list) - - cache_lifetime: float = 1.0 - - def model_post_init(self, __context: Any) -> None: # noqa: CCR001 - sources: list[SourceConfigValue] = [] - - for source_config in self.sources: - primitive_result = resolve_primitive(source_config.kind) - if primitive_result.is_err(): - error = primitive_result.unwrap_err()[0] - raise ValueError(error.message.format(error=error)) - - primitive = primitive_result.unwrap() - - if not isinstance(primitive, SourceConstructor): - raise ValueError(f"Source constructor '{source_config.kind}' is not supported") +def _serialize_workflow_dir(path: pathlib.Path) -> str: + return f"./{path.as_posix()}" - sources.append(primitive.construct_source(source_config)) - object.__setattr__(self, "_sources_instances", sources) - session_filter = FileFilter(mode=FileFilterMode.include, pattern=self.session_artifact_pattern()) - if session_filter not in self.file_filters: - object.__setattr__(self, "file_filters", [session_filter, *self.file_filters]) +def _validate_workflow_dir(path: pathlib.Path) -> pathlib.Path: + if path.is_absolute(): + raise ValueError("Workflow directories must be relative to the Donna project root.") - def session_artifact_pattern(self) -> ArtifactIdPattern: - session_path = self.session.as_posix().strip("/") - if session_path in {"", "."}: - return ArtifactIdPattern.parse("@/**/*.donna.md").unwrap() + if any(part == ".." for part in path.parts): + raise ValueError("Workflow directories must not contain parent-directory references.") - return ArtifactIdPattern.parse(f"@/{session_path}/**/*.donna.md").unwrap() + return path - @property - def sources_instances(self) -> list[SourceConfigValue]: - return list(self._sources_instances) - def get_source_config(self, kind: str) -> Result[SourceConfigValue, ErrorsList]: - for source in self._sources_instances: - if source.kind == kind: - return Ok(source) +class Config(BaseEntity): + session_dir: pathlib.Path = DONNA_DEFAULT_SESSION_DIR + default_section_kind: PythonPath = PythonPath(NormalizedRawIdPath("donna.lib.text")) + default_primary_section_kind: PythonPath = PythonPath(NormalizedRawIdPath("donna.lib.workflow")) + default_primary_section_id: SectionId = SectionId("primary") + workflow_dirs: list[pathlib.Path] = pydantic.Field(default_factory=_default_workflow_dirs) + journal: JournalConfig = pydantic.Field(default_factory=JournalConfig) - return Err( - [ - world_errors.SourceConfigNotConfigured( - source_id=kind, - kind=kind, - ) - ] - ) + cache_lifetime: float = 1.0 - def find_source_for_extension(self, extension: str) -> SourceConfigValue | None: - for source in self._sources_instances: - if source.supports_extension(extension): - return source + @pydantic.field_validator("workflow_dirs", mode="after") + @classmethod + def validate_workflow_dirs(cls, value: list[pathlib.Path]) -> list[pathlib.Path]: + workflow_dirs: list[pathlib.Path] = [] - return None + for path in value: + path = _validate_workflow_dir(path) + if path in workflow_dirs: + continue - def supported_extensions(self) -> set[str]: - extensions: set[str] = set() + workflow_dirs.append(path) - for source in self._sources_instances: - extensions.add(source.extension) + return workflow_dirs - return extensions + @pydantic.field_serializer("workflow_dirs") + def serialize_workflow_dirs(self, value: list[pathlib.Path]) -> list[str]: + return [_serialize_workflow_dir(path) for path in value] class Workspace(BaseEntity): diff --git a/donna/workspaces/errors.py b/donna/workspaces/errors.py index 7cf7cf16..6c0f3b2d 100644 --- a/donna/workspaces/errors.py +++ b/donna/workspaces/errors.py @@ -2,6 +2,7 @@ from donna.core import errors as core_errors from donna.domain.artifact_ids import ArtifactId +from donna.domain.constants import DONNA_ARTIFACT_EXTENSION class InternalError(core_errors.InternalError): @@ -68,17 +69,6 @@ class JournalCommandFailed(WorkspaceError): details: str -class SourceError(WorkspaceError): - cell_kind: str = "source_error" - source_id: str - - -class SourceConfigNotConfigured(SourceError): - code: str = "donna.workspaces.source_config_not_configured" - message: str = "Source config `{error.kind}` is not configured" - kind: str - - class ArtifactError(WorkspaceError): cell_kind: str = "artifact_error" artifact_id: ArtifactId @@ -104,11 +94,11 @@ class ArtifactMultipleFiles(ArtifactError): ] -class UnsupportedArtifactSourceExtension(ArtifactError): - code: str = "donna.workspaces.unsupported_artifact_source_extension" - message: str = "Unsupported artifact source extension `{error.extension}` for `{error.artifact_id}`" +class UnsupportedArtifactExtension(ArtifactError): + code: str = "donna.workspaces.unsupported_artifact_extension" + message: str = "Unsupported artifact extension `{error.extension}` for `{error.artifact_id}`" ways_to_fix: list[str] = [ - "Use a supported extension for the configured sources.", + f"Use the Donna artifact extension: `*{DONNA_ARTIFACT_EXTENSION}`.", ] extension: str diff --git a/donna/workspaces/initialization.py b/donna/workspaces/initialization.py index daf741a7..0d30efb0 100644 --- a/donna/workspaces/initialization.py +++ b/donna/workspaces/initialization.py @@ -1,6 +1,4 @@ -import importlib.resources import pathlib -import shutil import tomllib import tomli_w @@ -8,58 +6,24 @@ from donna.core import errors as core_errors from donna.core import utils from donna.core.result import Err, Ok, Result, unwrap_to_error +from donna.domain.constants import DONNA_CONFIG_NAME from donna.protocol.modes import Mode from donna.workspaces import config from donna.workspaces import errors as world_errors from donna.workspaces import sessions as workspace_sessions -SKILLS_ROOT_DIR = pathlib.Path(".agents") / "skills" -DONNA_SKILL_FIXTURE_DIR = pathlib.Path("fixtures") / "skills" -DONNA_SPECS_ROOT_DIR = pathlib.Path(".agents") / "donna" -DONNA_SPECS_FIXTURE_DIR = pathlib.Path("fixtures") / "specs" - -# this list must only increase in size, -# do not remove old items from it, since users may upgrade from older versions of Donna -# where these skills were installed -DONNA_SKILL_CLEANUP_LIST = ["donna-do", "donna-start", "donna-stop"] - - -def _sync_donna_skill(project_dir: pathlib.Path) -> None: - source = importlib.resources.files("donna").joinpath(*DONNA_SKILL_FIXTURE_DIR.parts) - - # cleanup - for skill_id in DONNA_SKILL_CLEANUP_LIST: - target_dir = project_dir / SKILLS_ROOT_DIR / skill_id - if target_dir.exists(): - shutil.rmtree(target_dir) - - # copy all content of fixtures/skills to the skills directory - with importlib.resources.as_file(source) as source_dir: - shutil.copytree(source_dir, project_dir / SKILLS_ROOT_DIR, dirs_exist_ok=True) - - -def _sync_donna_specs(project_dir: pathlib.Path) -> None: - source = importlib.resources.files("donna").joinpath(*DONNA_SPECS_FIXTURE_DIR.parts) - target_dir = project_dir / DONNA_SPECS_ROOT_DIR - - if target_dir.exists(): - shutil.rmtree(target_dir) - - with importlib.resources.as_file(source) as source_dir: - shutil.copytree(source_dir, target_dir) - @unwrap_to_error def load_workspace(root_dir: pathlib.Path | None = None) -> Result[config.Workspace, core_errors.ErrorsList]: """Load workspace configuration without mutating process-global state.""" if root_dir is None: - project_dir = utils.discover_project_dir(config.DONNA_CONFIG_NAME).unwrap() + project_dir = utils.discover_project_dir(DONNA_CONFIG_NAME).unwrap() else: project_dir = root_dir.resolve() - if not (project_dir / config.DONNA_CONFIG_NAME).is_file(): - return Err([core_errors.ProjectDirNotFound(config_name=config.DONNA_CONFIG_NAME)]) + if not (project_dir / DONNA_CONFIG_NAME).is_file(): + return Err([core_errors.ProjectDirNotFound(config_name=DONNA_CONFIG_NAME)]) - config_path = project_dir / config.DONNA_CONFIG_NAME + config_path = project_dir / DONNA_CONFIG_NAME if not config_path.exists(): return Ok(config.Workspace(root=project_dir, config=config.Config())) @@ -96,14 +60,10 @@ def initialize_runtime( @unwrap_to_error -def initialize_workspace( - project_dir: pathlib.Path, - install_skills: bool = True, - install_specs: bool = True, -) -> Result[config.Workspace, core_errors.ErrorsList]: +def initialize_workspace(project_dir: pathlib.Path) -> Result[config.Workspace, core_errors.ErrorsList]: """Initialize Donna project configuration.""" project_dir = project_dir.resolve() - config_path = project_dir / config.DONNA_CONFIG_NAME + config_path = project_dir / DONNA_CONFIG_NAME if config_path.exists(): return Err([world_errors.WorkspaceAlreadyInitialized(config_path=config_path)]) @@ -119,31 +79,4 @@ def initialize_workspace( workspace_sessions.ensure_dir() - if install_skills: - _sync_donna_skill(project_dir) - - if install_specs: - _sync_donna_specs(project_dir) - return Ok(workspace) - - -@unwrap_to_error -def update_workspace( - project_dir: pathlib.Path, - install_skills: bool = True, - install_specs: bool = True, -) -> Result[None, core_errors.ErrorsList]: - project_dir = project_dir.resolve() - config_path = project_dir / config.DONNA_CONFIG_NAME - - if not config_path.exists(): - return Err([core_errors.ProjectDirNotFound(config_name=config.DONNA_CONFIG_NAME)]) - - if install_skills: - _sync_donna_skill(project_dir) - - if install_specs: - _sync_donna_specs(project_dir) - - return Ok(None) diff --git a/donna/workspaces/sources/markdown.py b/donna/workspaces/markdown_parser.py similarity index 84% rename from donna/workspaces/sources/markdown.py rename to donna/workspaces/markdown_parser.py index 40927f31..39b1df6c 100644 --- a/donna/workspaces/sources/markdown.py +++ b/donna/workspaces/markdown_parser.py @@ -1,10 +1,9 @@ import uuid -from typing import TYPE_CHECKING, Any, ClassVar, Literal, Protocol, cast +from typing import Any, ClassVar, Protocol, cast from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error from donna.domain.artifact_ids import ArtifactId -from donna.domain.id_paths import NormalizedRawIdPath from donna.domain.ids import SectionId from donna.domain.python_path import PythonPath from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta @@ -12,7 +11,6 @@ from donna.workspaces import errors as world_errors from donna.workspaces import markdown from donna.workspaces.artifacts import ArtifactRenderContext -from donna.workspaces.sources.base import SourceConfig, SourceConstructor from donna.workspaces.templates import RenderMode, render @@ -27,25 +25,6 @@ def markdown_construct_section( pass -class Config(SourceConfig): - kind: Literal["markdown"] = "markdown" - extension: str = ".donna.md" - default_section_kind: PythonPath = PythonPath(NormalizedRawIdPath("donna.lib.text")) - default_primary_section_id: SectionId = SectionId("primary") - - def construct_artifact_from_bytes( - self, artifact_id: ArtifactId, content: bytes, render_context: ArtifactRenderContext - ) -> Result[Artifact, ErrorsList]: - return construct_artifact_from_bytes(artifact_id, content, render_context, self) - - -class MarkdownSourceConstructor(SourceConstructor): - def construct_source(self, config: "SourceConfigModel") -> Config: - data = config.model_dump() - data.pop("kind", None) - return Config.model_validate(data) - - class MarkdownSectionMixin: config_class: ClassVar[type[ArtifactSectionConfig]] @@ -114,7 +93,6 @@ def markdown_construct_section( # noqa: CCR001 kind=section_config.kind, title=title, description=description, - tags=section_config.tags, primary=primary, meta=meta, ) @@ -154,17 +132,38 @@ def parse_artifact_content( def construct_artifact_from_bytes( - artifact_id: ArtifactId, content: bytes, render_context: ArtifactRenderContext, config: Config + artifact_id: ArtifactId, + content: bytes, + render_context: ArtifactRenderContext, + default_section_kind: PythonPath, + default_primary_section_kind: PythonPath, + default_primary_section_id: SectionId, ) -> Result[Artifact, ErrorsList]: - return construct_artifact_from_markdown_source(artifact_id, content.decode("utf-8"), render_context, config) + return construct_artifact_from_markdown_source( + artifact_id, + content.decode("utf-8"), + render_context, + default_section_kind=default_section_kind, + default_primary_section_kind=default_primary_section_kind, + default_primary_section_id=default_primary_section_id, + ) @unwrap_to_error def construct_artifact_from_markdown_source( # noqa: CCR001 - artifact_id: ArtifactId, content: str, render_context: ArtifactRenderContext, config: Config + artifact_id: ArtifactId, + content: str, + render_context: ArtifactRenderContext, + default_section_kind: PythonPath, + default_primary_section_kind: PythonPath, + default_primary_section_id: SectionId, ) -> Result[Artifact, ErrorsList]: original_sections = parse_artifact_content(artifact_id, content, render_context).unwrap() head_config = dict(original_sections[0].config().unwrap()) + + if "kind" not in head_config or head_config["kind"] is None: + head_config["kind"] = default_primary_section_kind + head_kind_value = head_config["kind"] if isinstance(head_kind_value, PythonPath): head_kind = head_kind_value @@ -172,7 +171,7 @@ def construct_artifact_from_markdown_source( # noqa: CCR001 head_kind = PythonPath.parse(head_kind_value).unwrap() if "id" not in head_config or head_config["id"] is None: - head_config["id"] = config.default_primary_section_id + head_config["id"] = default_primary_section_id primary_primitive = resolve_primitive(head_kind).unwrap() _ensure_markdown_constructible(primary_primitive, head_kind).unwrap() @@ -191,7 +190,7 @@ def construct_artifact_from_markdown_source( # noqa: CCR001 sections = construct_sections_from_markdown( artifact_id=artifact_id, sections=original_sections[1:], - default_section_kind=config.default_section_kind, + default_section_kind=default_section_kind, ).unwrap() sections = [primary_section, *sections] return Ok(Artifact(id=artifact_id, sections=sections)) @@ -258,7 +257,3 @@ def _ensure_markdown_constructible( kind_label = f"'{primitive_id}'" if primitive_id is not None else repr(primitive) return Err([world_errors.PrimitiveDoesNotSupportMarkdown(primitive_id=kind_label)]) - - -if TYPE_CHECKING: - from donna.workspaces.config import SourceConfig as SourceConfigModel diff --git a/donna/workspaces/sessions.py b/donna/workspaces/sessions.py index 75d43f58..4da1268b 100644 --- a/donna/workspaces/sessions.py +++ b/donna/workspaces/sessions.py @@ -1,19 +1,26 @@ import pathlib import shutil +from donna.domain.constants import STATE_FILE_NAME from donna.workspaces.config import config, project_dir +def _path() -> pathlib.Path: + return project_dir() / config().session_dir + + def dir() -> pathlib.Path: - return project_dir() / config().session + session_dir = _path() + session_dir.mkdir(parents=True, exist_ok=True) + return session_dir def ensure_dir() -> None: - dir().mkdir(parents=True, exist_ok=True) + dir() def reset_dir() -> None: - session_dir = dir() + session_dir = _path() if session_dir.exists(): shutil.rmtree(session_dir) @@ -21,7 +28,7 @@ def reset_dir() -> None: def read_state() -> bytes | None: - path = dir() / "state.json" + path = dir() / STATE_FILE_NAME if not path.exists(): return None @@ -29,6 +36,5 @@ def read_state() -> bytes | None: def write_state(content: bytes) -> None: - path = dir() / "state.json" - ensure_dir() + path = dir() / STATE_FILE_NAME path.write_bytes(content) diff --git a/donna/workspaces/sources/__init__.py b/donna/workspaces/sources/__init__.py deleted file mode 100644 index a4746aa8..00000000 --- a/donna/workspaces/sources/__init__.py +++ /dev/null @@ -1 +0,0 @@ -"""Helpers for constructing artifacts from different source types.""" diff --git a/donna/workspaces/sources/base.py b/donna/workspaces/sources/base.py deleted file mode 100644 index 41bab356..00000000 --- a/donna/workspaces/sources/base.py +++ /dev/null @@ -1,55 +0,0 @@ -from __future__ import annotations - -from abc import ABC, abstractmethod -from typing import TYPE_CHECKING - -import pydantic - -from donna.core.entities import BaseEntity -from donna.core.errors import ErrorsList -from donna.core.result import Result -from donna.machine.primitives import Primitive - -if TYPE_CHECKING: - from donna.domain.artifact_ids import ArtifactId - from donna.machine.artifacts import Artifact - from donna.workspaces.artifacts import ArtifactRenderContext - from donna.workspaces.config import SourceConfig as SourceConfigModel - - -class SourceConfig(BaseEntity, ABC): - kind: str - extension: str - - @classmethod - def normalize_extension(cls, extension: str) -> str: - normalized = str(extension).strip().lower() - - if not normalized: - raise ValueError("Extension must not be empty") - - if not normalized.startswith("."): - normalized = f".{normalized}" - - if normalized == ".": - raise ValueError("Extension must include characters after '.'") - - return normalized - - @pydantic.field_validator("extension") - @classmethod - def _normalize_extension(cls, value: str) -> str: - return cls.normalize_extension(value) - - def supports_extension(self, extension: str) -> bool: - return self.normalize_extension(extension) == self.extension - - @abstractmethod - def construct_artifact_from_bytes( # noqa: E704 - self, artifact_id: "ArtifactId", content: bytes, render_context: "ArtifactRenderContext" - ) -> Result["Artifact", ErrorsList]: ... # noqa: E704 - - -class SourceConstructor(Primitive, ABC): - @abstractmethod - def construct_source(self, config: SourceConfigModel) -> SourceConfig: ... # noqa: E704 diff --git a/specs/core/error_handling.donna.md b/specs/core/error_handling.md similarity index 99% rename from specs/core/error_handling.donna.md rename to specs/core/error_handling.md index f309699f..d79673ff 100644 --- a/specs/core/error_handling.donna.md +++ b/specs/core/error_handling.md @@ -1,10 +1,6 @@ # Error handling -```toml donna -kind = "donna.lib.specification" -``` - This document describes how Donna handles errors that may occur during its operation, including error propagation, logging, and recovery strategies. ## Error kinds diff --git a/specs/core/top_level_architecture.donna.md b/specs/core/top_level_architecture.md similarity index 85% rename from specs/core/top_level_architecture.donna.md rename to specs/core/top_level_architecture.md index f2aa3609..f83e8749 100644 --- a/specs/core/top_level_architecture.donna.md +++ b/specs/core/top_level_architecture.md @@ -1,10 +1,6 @@ # Top-level architecture -```toml donna -kind = "donna.lib.specification" -``` - This document describes the top-level architecture of the Donna project, providing an overview of its main components and their interactions. ## Basic statements @@ -26,8 +22,7 @@ The code is separated by layers/subsystems into subpackages: - `donna.cli` — code that implements the `donna` CLI tool, its commands, arguments parsing, etc. - `donna.primitives` — code that implements basic building blocks for Donna's behavior: concrete implementations of various classes from the `donna.machine`. - `donna.lib` — module that contains constructed primitives to be used in donna artifacts by referencing them by python import path. Like `donna.lib.workflow`, `donna.lib.goto`, etc. -- `donna.fixtures.skills` — bundled skills that are distributed with Donna and synced into projects under `.agents/skills`. -- `donna.fixtures.specs` — bundled Donna specifications and workflows that are distributed with Donna and synced into projects under `.agents/donna`, where they are addressed as `@/.agents/donna/**` artifacts. +- `donna.skills` — code and built-in text documents used by the `donna skill` command. ## Data structures diff --git a/specs/intro.donna.md b/specs/intro.md similarity index 71% rename from specs/intro.donna.md rename to specs/intro.md index 10df4bb0..2030c729 100644 --- a/specs/intro.donna.md +++ b/specs/intro.md @@ -1,9 +1,5 @@ # Introduction to the Donna development - -```toml donna -kind = "donna.lib.specification" -``` This document provides an introduction to the Donna project for agents and developers who want to understand how to work with the Donna codebase. ## Project overview @@ -28,14 +24,13 @@ We may need coding agents on each step of the process, but there is no reason fo - **Artifact** — any text or binary document managed by Donna in the project filesystem; text artifacts are typically Markdown templates with metadata and are the primary units of knowledge and instructions. - **Artifact Section** — a part of a text artifact separated by markdown headers, has its own configuration block and semantics depending on section kind. - **Configuration block** — a fenced code block with the `donna` keyword (preferably TOML) that configures an artifact or its section. -- **Directive** — a Jinja2 helper like `donna.lib.view(...)` or `donna.lib.goto(...)` that adds meta information or special behavior to an artifact. +- **Directive** — a Jinja2 helper like `donna.lib.goto(...)` that adds meta information or special behavior to an artifact. - **Environment error** — a structured, user-facing error describing problem in the environment Donna operates in (e.g., missing artifact, invalid config). These errors are expected to be handled by agents or users. - **Head section** — the H1 section of a markdown artifact (before the first H2) that contains the primary description and mandatory config block. - **Internal error** — an error caused by a bug or unexpected state in Donna itself. These errors are not expected to be handled by agents or users. - **Protocol** — the output/interaction mode for Donna (e.g., `llm`) that governs CLI behavior and rendering. - **Session** — the active unit of work tracked by Donna; its state and artifacts live under `<project-root>/.session/donna`. -- **Source** — the entity that implements logic of building an artifact from its raw data (text or binary). -- **Specification** — a text artifact of kind `donna.lib.specification` that documents behavior, rules, or project guidance. +- **Specification** — a normal Markdown document that describes behavior, rules, or project guidance. - **Story** — a semantically consistent scope of work within a session; a conceptual unit not directly represented in the tool. - **Tail section** — each H2 section of an artifact. - **Workspace** — the Donna project rooted at the directory containing `donna.toml`; runtime state lives under the configured session directory. @@ -45,17 +40,19 @@ We may need coding agents on each step of the process, but there is no reason fo ## Points of interest - `./donna/` — a directory containing source code of project — `donna` CLI tool. -- `./specs/` — a directory containing project-specific donna artifacts that is used to manage the work of AI agents on this project. +- `./specs/` — a directory containing project-specific documentation. +- `./workflows/` — a directory containing project-specific Donna workflows used to manage the work of AI agents on this project. - `./.session/donna/` — the configured temporary session directory used by Donna for runtime state and session artifacts. -## Specifications of interest +## Documentation of interest Since this is the repository that contains the Donna project itself, you MUST pay additional attention to which project-scoped artifact ids you are viewing. -- `@/.agents/donna/**` contains synced Donna specifications and workflows related to the Donna tool behavior. You access them when you need to use Donna itself. You change the source fixtures when you make changes to Donna behavior. -- `@/specs/**` contains project-specific specifications and workflows for developing the Donna codebase. You access them when you need to understand how to introduce changes to this repository. You change them when you change the development processes or documentation of the Donna project as a software project. +- `@/.agents/donna/**` contains local Donna documentation for this repository. +- `@/specs/**` contains project-specific documentation for developing the Donna codebase. You access it when you need to understand how to introduce changes to this repository. +- `@/workflows/**` contains project-specific workflows for developing the Donna codebase. You change them when you change the development processes of the Donna project as a software project. -Check the next specifications: +Check the next documents: -- `{{ donna.lib.view("@/specs/core/top_level_architecture.donna.md") }}` when you need to introduce any changes in Donna or to research its code. -- `{{ donna.lib.view("@/specs/core/error_handling.donna.md") }}` when you need to implement any new feature in Donna that may produce, process or propagate errors. +- Read the `@/specs/core/top_level_architecture.md` file when you need to introduce any changes in Donna or to research its code. +- Read the `@/specs/core/error_handling.md` file when you need to implement any new feature in Donna that may produce, process or propagate errors. diff --git a/specs/work/log_changes.donna.md b/specs/work/log_changes.donna.md deleted file mode 100644 index c13a46a6..00000000 --- a/specs/work/log_changes.donna.md +++ /dev/null @@ -1,175 +0,0 @@ -# Log Changes Workflow - -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "determine_scope" -``` - -Log new unreleased changes in the Changy-managed changelog. - -## Determine change scope - -```toml donna -id = "determine_scope" -kind = "donna.lib.request_action" -``` - -1. If this workflow was started by a parent workflow and it provided a specific scope, analyze changes only in that scope. -2. If this workflow was started directly by a developer with no additional scope, analyze all changes in the current branch. -3. If you are analyzing full branch changes, `{{ donna.lib.goto("analyze_full_branch_changes") }}`. -4. If you are analyzing a scoped set of changes, `{{ donna.lib.goto("analyze_scoped_changes") }}`. - -## Analyze full branch changes - -```toml donna -id = "analyze_full_branch_changes" -kind = "donna.lib.request_action" -``` - -1. Determine the base branch (typically `main`) and compare against it to identify changes introduced by this branch. -2. Collect the change summary using git commands such as: - - `git status -sb` - - `git diff --stat main...HEAD` - - `git log --oneline --decorate main..HEAD` -3. Summarize the main changes across the entire branch to use for the changelog entry. -4. `{{ donna.lib.goto("analyze_branch_name") }}` - -## Analyze scoped changes - -```toml donna -id = "analyze_scoped_changes" -kind = "donna.lib.request_action" -``` - -1. Focus on changes in the `@/.session/donna/**` artifacts provided by the parent workflow. -2. Summarize the main changes within that scoped set to use for the changelog entry. -3. Only after the scoped analysis, check the git state to confirm the summary reflects the current working tree. -4. `{{ donna.lib.goto("analyze_branch_name") }}` - -## Analyze branch name - -```toml donna -id = "analyze_branch_name" -kind = "donna.lib.request_action" -``` - -1. Determine the current branch name with `git rev-parse --abbrev-ref HEAD`. -2. Extract task id and short description from the branch name: - - Task id: first token matching `<letters>-<digits>` (examples: `abc-123`, `xyz-456`). - - Short description: the remainder of the branch name after the task id, with `-` converted to spaces (examples: `new api`, `fix crash`). -3. If no meaningful branch description exists, derive a concise description from the change summary. -4. `{{ donna.lib.goto("locate_unreleased_file") }}` - -## Locate unreleased changes file - -```toml donna -id = "locate_unreleased_file" -kind = "donna.lib.request_action" -``` - -1. Locate the Changy unreleased changes file (typically `changes/unreleased.md`). -2. If unsure, search with `rg --files -g 'unreleased.md' changes` or `find changes -name 'unreleased.md'`. -3. `{{ donna.lib.goto("update_changes_section") }}` - -## Update changes section - -```toml donna -id = "update_changes_section" -kind = "donna.lib.request_action" -``` - -1. Add a new entry under the `### Changes` section (create the section if missing). -2. Format the main entry: - - With task id: `- <short-id> <short-changes-description>` - - Without task id: `- <short-changes-description>` -3. Add sub-items for all major changes in behavior, architecture, or code. -4. `{{ donna.lib.goto("update_breaking_changes_section") }}` - -Notes: - -- Use past tense (`Added …`, `Fixed …`, etc.) -- Be concise. - -## Update breaking changes section - -```toml donna -id = "update_breaking_changes_section" -kind = "donna.lib.request_action" -``` - -1. If there are breaking changes, add or update the `### Breaking Changes` section with the relevant entries. -2. If there are no breaking changes, do not add the section. -3. `{{ donna.lib.goto("update_migration_section") }}` - -Notes: - -- Use past tense (`Added …`, `Fixed …`) or present tense (`X is now …`). -- Be concise. - -## Update migration section - -```toml donna -id = "update_migration_section" -kind = "donna.lib.request_action" -``` - -1. If migrations are needed, add or update the `### Migration` section with the relevant entries. -2. If no migrations are needed, do not add the section. -3. `{{ donna.lib.goto("update_deprecations_section") }}` - -Notes: - -- Use past tense (`Added …`, `Fixed …`) or present tense (`X is now …`). -- Be concise. - -## Update deprecations section - -```toml donna -id = "update_deprecations_section" -kind = "donna.lib.request_action" -``` - -1. If deprecations are introduced, add or update the `### Deprecations` section with the relevant entries. -2. If no deprecations are introduced, do not add the section. -3. `{{ donna.lib.goto("update_removals_section") }}` - -Notes: - -- Use past tense (`Added …`, `Fixed …`) or present tense (`X is now …`). -- Be concise. - -## Update removals section - -```toml donna -id = "update_removals_section" -kind = "donna.lib.request_action" -``` - -1. If functionality removals occur, add or update the `### Removals` section with the relevant entries. -2. If no removals occur, do not add the section. -3. `{{ donna.lib.goto("validate_changelog") }}` - -Notes: - -- Use past tense (`Added …`, `Fixed …`) or present tense (`X is now …`). -- Be concise. - -## Validate changelog - -```toml donna -id = "validate_changelog" -kind = "donna.lib.request_action" -``` - -1. Ensure the changelog remains well-structured and readable. -2. Confirm the updated sections are in the expected order and formatting. -3. `{{ donna.lib.goto("finish") }}` - -## Finish - -```toml donna -id = "finish" -kind = "donna.lib.finish" -``` - -Log changes workflow completed. diff --git a/specs/work/polish.donna.md b/workflows/polish.donna.md similarity index 98% rename from specs/work/polish.donna.md rename to workflows/polish.donna.md index 3214fe85..69929b1f 100644 --- a/specs/work/polish.donna.md +++ b/workflows/polish.donna.md @@ -1,10 +1,5 @@ # Polishing Workflow -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "run_autoflake_script" -``` - Initiate operations to polish and refine the donna codebase: running & fixing tests, formatting code, fixing type annotations, etc. This workflow MUST NOT be used to introduce new logic into the project or refactor it — only to fix existing issues. ## Run Autoflake diff --git a/donna/fixtures/specs/rfc/work/design.donna.md b/workflows/rfc/design.donna.md similarity index 69% rename from donna/fixtures/specs/rfc/work/design.donna.md rename to workflows/rfc/design.donna.md index d4c5cbcc..40370f0b 100644 --- a/donna/fixtures/specs/rfc/work/design.donna.md +++ b/workflows/rfc/design.donna.md @@ -1,11 +1,6 @@ # Create a Design document -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - -This workflow creates a Design document artifact based on an RFC and aligned with `../specs/design.donna.md`. +This workflow creates a Design document artifact based on an RFC and aligned with `workflows/rfc/specs/design.md`. ## Start Work @@ -15,7 +10,7 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. +1. Read the `workflows/rfc/specs/design.md` file if you haven't done it yet. 2. Read the artifact instructions by running `donna skill artifacts` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_rfc_artifact_exists") }}` @@ -28,14 +23,14 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear RFC to design. -1. If you have an RFC artifact id in your context, view it and `{{ donna.lib.goto("prepare_design_artifact") }}`. -2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("@/.session/donna/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. -3. If you have no RFC artifact id in your context, and you don't know where it is, ask the developer to provide the RFC artifact id or to create a new RFC. After you get it and view the artifact, `{{ donna.lib.goto("prepare_design_artifact") }}`. +1. If you have an RFC artifact id in your context, view it and `{{ donna.lib.goto("prepare_design_document") }}`. +2. If you have no RFC artifact id in your context, but you know it is in one of the files matching `@/.session/donna/**`, find and view it. Then `{{ donna.lib.goto("prepare_design_document") }}`. +3. If you have no RFC artifact id in your context, and you don't know where it is, ask the developer to provide the RFC artifact id or to create a new RFC. After you get it and view the artifact, `{{ donna.lib.goto("prepare_design_document") }}`. -## Prepare Design artifact +## Prepare Design document ```toml donna -id = "prepare_design_artifact" +id = "prepare_design_document" kind = "donna.lib.request_action" ``` @@ -45,10 +40,6 @@ kind = "donna.lib.request_action" ~~~ # <Title> -```toml donna -kind = "donna.lib.specification" -``` - <short description of the proposed design> ## Inputs @@ -81,10 +72,10 @@ id = "initial_fill" kind = "donna.lib.request_action" ``` -1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. +1. Read the `workflows/rfc/specs/design.md` file if you haven't done it yet. 2. Read the RFC artifact selected in the previous step if you haven't done it yet. 3. Analyze the project if needed to understand the requested change context. -4. Fill in all sections of the Design draft artifact. +4. Fill in all sections of the Design draft document. 5. Ensure the first item in `Inputs` section is the RFC artifact id. 6. `{{ donna.lib.goto("review_design_format") }}` @@ -95,8 +86,8 @@ id = "review_design_format" kind = "donna.lib.request_action" ``` -1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("../specs/design.donna.md") }}`. -2. For each mismatch, make necessary edits to the Design draft artifact to ensure compliance. +1. List mismatches between the Design artifact and the specification in the `workflows/rfc/specs/design.md` file. +2. For each mismatch, make necessary edits to the Design draft document to ensure compliance. 3. `{{ donna.lib.goto("review_design_content") }}` ## Review Design Content @@ -106,8 +97,8 @@ id = "review_design_content" kind = "donna.lib.request_action" ``` -1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. -2. Make necessary edits to the Design draft artifact to address identified issues. +1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. +2. Make necessary edits to the Design draft document to address identified issues. 3. If there were changes made on this step or the previous `review_design_format` step `{{ donna.lib.goto("review_design_format") }}`. 4. If no changes were made, `{{ donna.lib.goto("finish") }}`. diff --git a/donna/fixtures/specs/rfc/work/do.donna.md b/workflows/rfc/do.donna.md similarity index 89% rename from donna/fixtures/specs/rfc/work/do.donna.md rename to workflows/rfc/do.donna.md index b97cf0ff..f661111c 100644 --- a/donna/fixtures/specs/rfc/work/do.donna.md +++ b/workflows/rfc/do.donna.md @@ -1,11 +1,6 @@ # Do Request For Change work -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "estimate_complexity" -``` - This workflow uses a description of a problem or changes required from the developer or parent workflow to: 1. Create a Request for Change (RFC) artifact. @@ -98,17 +93,6 @@ kind = "donna.lib.request_action" 1. Find the workflow to polish the changes made. 2. Run the chosen workflow if you found one. -3. `{{ donna.lib.goto("log_changes") }}`. - -## Log changes - -```toml donna -id = "log_changes" -kind = "donna.lib.request_action" -``` - -1. Find the workflow to log the changes made in the scope of this RFC. -2. Run the chosen workflow if you found one. 3. `{{ donna.lib.goto("finish") }}`. ## Finish diff --git a/donna/fixtures/specs/rfc/work/plan.donna.md b/workflows/rfc/plan.donna.md similarity index 97% rename from donna/fixtures/specs/rfc/work/plan.donna.md rename to workflows/rfc/plan.donna.md index 19a6017f..d5eb3e39 100644 --- a/donna/fixtures/specs/rfc/work/plan.donna.md +++ b/workflows/rfc/plan.donna.md @@ -1,11 +1,6 @@ # Plan work on base of Design & Request for Change documents -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow stored as a `@/.session/donna/**` artifact under `<project-root>/.session/donna` with detailed steps to implement the designed changes. ## Start Work diff --git a/donna/fixtures/specs/rfc/work/request.donna.md b/workflows/rfc/request.donna.md similarity index 69% rename from donna/fixtures/specs/rfc/work/request.donna.md rename to workflows/rfc/request.donna.md index e31a5532..26409b8f 100644 --- a/donna/fixtures/specs/rfc/work/request.donna.md +++ b/workflows/rfc/request.donna.md @@ -1,11 +1,6 @@ # Create a Request for Change -```toml donna -kind = "donna.lib.workflow" -start_operation_id = "start" -``` - This workflow creates a Request for Change (RFC) document based on a description of the problem or the required changes. ## Start Work @@ -16,7 +11,7 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. +1. Read the `workflows/rfc/specs/request_for_change.md` file if you haven't done it yet. 2. Read the artifact instructions by running `donna skill artifacts` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_work_description_exists") }}` @@ -29,14 +24,14 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear description of the problem in your context. I.e. you know what you need to do in this workflow. -1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("@/.session/donna/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. -3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. +1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_rfc_document") }}`. +2. If you have no problem description in your context, but you know it is in one of the files matching `@/.session/donna/**`, find and view it. Then `{{ donna.lib.goto("prepare_rfc_document") }}`. +3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_rfc_document") }}`. -## Prepare RFC artifact +## Prepare RFC document ```toml donna -id = "prepare_rfc_artifact" +id = "prepare_rfc_document" kind = "donna.lib.request_action" ``` @@ -46,10 +41,6 @@ kind = "donna.lib.request_action" ~~~ # <Title> -```toml donna -kind = "donna.lib.specification" -``` - <short description of the proposed change> ## Original description @@ -86,9 +77,9 @@ id = "initial_fill" kind = "donna.lib.request_action" ``` -1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. +1. Read the `workflows/rfc/specs/request_for_change.md` file if you haven't done it yet. 2. Analyze the project if needed to understand the context of the requested change. -3. Based on the problem description you have, fill in all sections of the RFC draft artifact. +3. Based on the problem description you have, fill in all sections of the RFC draft document. 4. `{{ donna.lib.goto("review_rfc_format") }}` ## Review RFC Format @@ -98,8 +89,8 @@ id = "review_rfc_format" kind = "donna.lib.request_action" ``` -1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}`. -2. For each mismatch, make necessary edits to the RFC draft artifact to ensure compliance with the RFC specification. +1. List mismatches between the RFC artifact and the specification in the `workflows/rfc/specs/request_for_change.md` file. +2. For each mismatch, make necessary edits to the RFC draft document to ensure compliance with the RFC specification. 3. `{{ donna.lib.goto("review_rfc_content") }}` ## Review RFC Content @@ -109,8 +100,8 @@ id = "review_rfc_content" kind = "donna.lib.request_action" ``` -1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. -2. Make necessary edits to the RFC draft artifact to address identified issues. +1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. +2. Make necessary edits to the RFC draft document to address identified issues. 3. If there were changes made on this step or the previous `review_rfc_format` step `{{ donna.lib.goto("review_rfc_format") }}`. 4. If no changes were made, `{{ donna.lib.goto("finish") }}`. diff --git a/donna/fixtures/specs/rfc/specs/design.donna.md b/workflows/rfc/specs/design.md similarity index 97% rename from donna/fixtures/specs/rfc/specs/design.donna.md rename to workflows/rfc/specs/design.md index 71237d1b..56ba5d97 100644 --- a/donna/fixtures/specs/rfc/specs/design.donna.md +++ b/workflows/rfc/specs/design.md @@ -1,9 +1,5 @@ # Format of the Design document -```toml donna -kind = "donna.lib.specification" -``` - This document describes the format and structure of a Design document used to design changes to a project proposed in a Request for Change (RFC). This document is an input for planning and will be treated as strong recommendations for the implementation of the proposed change. ## Overview @@ -12,7 +8,7 @@ Donna introduces a group of workflows located in `../**` namespace that organize You create a Design document to explicitly describe the exact changes you want to make to the project in order to implement the RFC. -If not otherwise specified, Design documents for the session MUST be stored as `@/.session/donna/design/<short-problem-related-identifier>.donna.md` artifacts under `<project-root>/.session/donna`. +If not otherwise specified, Design documents for the session MUST be stored as `@/.session/donna/design/<short-problem-related-identifier>.donna.md` files under `<project-root>/.session/donna`. **The Design document MUST list exact changes to the project that will be implemented.** E.g. concrete function names and signatures, file paths, data structures, etc. @@ -24,7 +20,7 @@ The Design document MUST NOT be a high-level description of the problem and solu ## Design document structure -The RFC document is Donna artifact (check `donna skill artifacts`) with the next structure: +The RFC document is a Markdown file with the next structure: - **Primary section** — title and short description of the proposed change. - **Inputs** — list of input documents that are relevant for the proposed change, starting from the RFC document. diff --git a/donna/fixtures/specs/rfc/specs/request_for_change.donna.md b/workflows/rfc/specs/request_for_change.md similarity index 96% rename from donna/fixtures/specs/rfc/specs/request_for_change.donna.md rename to workflows/rfc/specs/request_for_change.md index 7d85548d..f5f93355 100644 --- a/donna/fixtures/specs/rfc/specs/request_for_change.donna.md +++ b/workflows/rfc/specs/request_for_change.md @@ -1,9 +1,5 @@ # Format of the Request for Change document -```toml donna -kind = "donna.lib.specification" -``` - This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `../**` namespace. This document is an input for a Design document creation. ## Overview @@ -12,11 +8,11 @@ Donna introduces a group of workflows located in `../**` namespace that organize You create RFC documents to propose changes to the project. -If not otherwise specified, RFC documents for the session MUST be stored as `@/.session/donna/rfc/<short-problem-related-identifier>.donna.md` artifacts under `<project-root>/.session/donna`. +If not otherwise specified, RFC documents for the session MUST be stored as `@/.session/donna/rfc/<short-problem-related-identifier>.donna.md` files under `<project-root>/.session/donna`. ## RFC structure -The RFC document is Donna artifact (check `donna skill artifacts`) with the next structure: +The RFC document is a Markdown file with the next structure: - **Primary section** — title and short description of the proposed change. - **Original description** — original description of the requested changes from the developer or parent workflow. @@ -136,7 +132,7 @@ Examples: - Bad: `- Use clean architecture.` - Good: `- The solution MUST be compatible with Python 3.12.` - Good: `- The solution MUST NOT introduce new runtime dependencies.` -- Good: `- The solution MUST follow the specification ../../../../specs/abc.donna.md` +- Good: `- The solution MUST follow the specification ../../../../specs/abc.md` - Good: `MUST not change public CLI flags` ## `Requirements` section @@ -216,7 +212,7 @@ Examples: - Bad: `- Verify that authentication works correctly.` - Bad: `- Review the implementation manually.` - Good: `- Run test suite `tests/auth/test_login.py`; all tests MUST pass.` -- Good: `- Inspect artifact `../../../../specs/authentication.donna.md`; it MUST exist and contain section "Login flow".` +- Good: `- Inspect artifact `../../../../specs/authentication.md`; it MUST exist and contain section "Login flow".` - Good: `- Execute CLI command `tool login` with invalid credentials; command MUST exit with non-zero code.` ## `Deliverables` section @@ -238,7 +234,7 @@ Examples: - Bad: `- Implement authentication code` - Bad: `- Refactor auth module.` - Good: `- Module app/auth/authentication.py exists.` -- Good: `- Donna artifact ../../../../specs/authentication.donna.md exists.` +- Good: `- Markdown document ../../../../specs/authentication.md exists.` - Good: `- Test suite tests/auth/ exists.` ## `Action items` section