diff --git a/.apm/instructions/backend.instructions.md b/.apm/instructions/backend.instructions.md index 70bfe99518..b1741d9ee7 100644 --- a/.apm/instructions/backend.instructions.md +++ b/.apm/instructions/backend.instructions.md @@ -3,13 +3,23 @@ description: "Go backend coding standards for Sippy" applyTo: "**/*.go" --- -* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Follow idiomatic Go practices. +* Choose names carefully using concise but appropriately descriptive words; in the scope of a + package, a name need only describe its function relative to that package. Provide docstring for + every package-level name explaining _why_ it exists and describing any parameter whose purpose is + not completely obvious from name and context. +* Keep packages, structs, and methods focused on a single clear conceptual "chunk": + a package should represent one cohesive concept, a struct should represent a single entity + in the scope of its package, and a method should operate on a single level of abstraction. + Prefer descriptive names that evoke a specific concept (e.g., "TestResultAggregator" not + "Manager"). Avoid generic names like "Manager", "Handler", "Util" without specific context, + and method names that don't indicate what they do (e.g., "Process", "Handle", "Do"). + Structs with more than about 7 top-level fields should be refactored into focused sub-types. +* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Use `k8s.io/apimachinery/pkg/util/sets` (e.g. `sets.New[string]()`) to deduplicate or collect unique strings. Do not use `map[string]bool` as a hand-rolled set. * Prefer structured logging where it makes sense; particularly for names and IDs, and often for counts, a log.WithField() call is preferred over formatting values into a string. -* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. * When modifying any data provider (BigQuery or PostgreSQL), ensure **parity between both implementations**. Changes to query logic, filtering, or returned data in one provider must be reflected in the other. * **Timestamps and dates**: Use proper types, never epoch integers. - PostgreSQL columns: use `TIMESTAMP WITH TIME ZONE` for timestamps and `DATE` for date-only values. All timestamps are UTC. @@ -17,3 +27,16 @@ applyTo: "**/*.go" - API responses: timestamps serialize as RFC 3339 strings, dates as `YYYY-MM-DD` strings. Never return epoch millisecond integers. Never manually format with `time.Format()` into a string field. - Materialized views: project timestamp columns directly from the source table. Do not convert to epoch with `EXTRACT(epoch FROM ...)`. - GORM model tags: include `gorm:"type:date"` on date-only columns so GORM and migrations use the correct PostgreSQL type. +* Check `pkg/util/` for existing helper functions before adding inline utility logic. + Avoid calling the same utility function multiple times with identical arguments in the + same code path. +* Never ignore returned errors as `_` without clear justification. Errors should be wrapped + with context using `fmt.Errorf` with `%w`. Avoid `panic()` except in `init()` or fatal + conditions. Check for nil before dereferencing pointers. +* **Never** concatenate or format SQL queries with values directly from user input. Always use + placeholders for parameters in queries, preferably named (`@Name`). +* Structs used with GORM that have fields not backed by database columns (such as computed or + API-only fields) must include the `gorm:"-"` tag to explicitly exclude them from GORM operations. + BigQuery struct fields must have `bigquery:"column_name"` tags that exactly match the BigQuery + query result schema names, whether from table columns or aliases (give computed fields aliases). +* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. diff --git a/.apm/instructions/docs.instructions.md b/.apm/instructions/docs.instructions.md index 27bc4bd340..58b5456622 100644 --- a/.apm/instructions/docs.instructions.md +++ b/.apm/instructions/docs.instructions.md @@ -21,3 +21,7 @@ applyTo: "**" * Documentation and code belong in the same PR; never treat a docs update as a follow-up task. * Do not use em dashes when writing docs. Use commas, parentheses, or periods instead. +* Files under `pkg/**/jobrunscan**/`, `pkg/**/jobrunannotator**/`, `pkg/api/jobartifacts/**`, and + `sippy-ng/src/component_readiness/JobArtifactQuery.js` are part of the symptoms feature documented + in `docs/features/job-analysis-symptoms.md`. Update this document when there are changes to data + models, API surface, or data flow in the symptoms feature. diff --git a/.apm/instructions/frontend.instructions.md b/.apm/instructions/frontend.instructions.md index 332b8da9a2..95cce74382 100644 --- a/.apm/instructions/frontend.instructions.md +++ b/.apm/instructions/frontend.instructions.md @@ -3,21 +3,33 @@ description: "React/Material-UI frontend guidelines for Sippy" applyTo: "sippy-ng/**" --- +* The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. * After making changes, always run formatting and linting to maintain consistency: -```bash -npx eslint . --fix -npx prettier --write . -``` + ```bash + npx eslint . --fix + npx prettier --write . + ``` * Prefer functional components and React hooks over class components. * Keep UI elements consistent with Material-UI standards. - -The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. - +* Avoid nested ternary expressions in JSX. When there are more than two + branches, use if/else if chains, early returns, or a lookup object instead. + A single ternary is fine; nesting ternaries makes code hard to follow. +* Before adding date/time formatting, duration calculations, or string + utilities inline, check `sippy-ng/src/helpers.js` for existing functions + like `relativeDuration`, `safeEncodeURIComponent`, etc. Prefer reusing + existing helpers over reimplementing similar logic. +* React components with extensive inline CSS should use the `useStyles` pattern. + Inline style objects with more than 3-4 properties should be extracted to + `useStyles()` or styled components. Inline styles are acceptable for simple, + dynamic values (e.g., width based on props). * **Timestamps and dates from the API**: - Timestamps arrive as RFC 3339 strings (e.g., `"2024-06-27T15:30:00Z"`), not epoch millisecond integers. Use `new Date(value)` or `Temporal.Instant.from(value)` to parse them. - Dates arrive as `YYYY-MM-DD` strings (e.g., `"2024-06-27"`). Use `Temporal.PlainDate.from(value)` for date arithmetic. - For MUI DataGrid timestamp columns, use `type: 'date'` with a `valueGetter` that returns a `Date` object. Do not return epoch milliseconds from `valueGetter`. - For filter values sent to the API, use ISO 8601 strings (e.g., `new Date(...).toISOString()`), not epoch millisecond integers. - For day-level bucketing or date arithmetic, prefer `Temporal.PlainDate` over `Date` with manual millisecond math. +* Non-trivial modifications to frontend logic must include unit test coverage. + If a function is hard to test, consider refactoring to separate pure logic + from side effects. diff --git a/.apm/instructions/general.instructions.md b/.apm/instructions/general.instructions.md index 60f01aa32d..5d68a28dbb 100644 --- a/.apm/instructions/general.instructions.md +++ b/.apm/instructions/general.instructions.md @@ -20,5 +20,6 @@ The system consists of: * A **Go-based API backend**. * A **React/Material-UI frontend** (located in `sippy-ng`). * Data sources including **PostgreSQL**, and **BigQuery** +* A **headless daemon** for asynchronous processing Favor clarity and maintainability over cleverness. Comments should be minimal, helpful, and explain the "why" not the "what". diff --git a/.apm/instructions/query.instructions.md b/.apm/instructions/query.instructions.md new file mode 100644 index 0000000000..404834b244 --- /dev/null +++ b/.apm/instructions/query.instructions.md @@ -0,0 +1,14 @@ +--- +description: "Guidelines for BigQuery and SQL query-building code" +applyTo: "pkg/**/query/**" +--- + +* Abbreviations should be expanded on first use, for example Common Table Expression (CTE) and + Materialized View (matview). "SQL" is exempt from this rule. +* BigQuery and SQL query-building code should have inline comments explaining the + purpose of each major query section (CTEs, JOINs, window functions, WHERE clauses). +* Functions that construct queries must stay under 200 lines; extract sub-queries or + CTEs into helper functions when they grow beyond that. +* When constructing SQL queries, prefer using a format string and/or multi-line string rather than + concatenation of short strings; and always use placeholders for parameters. + diff --git a/.apm/instructions/testing.instructions.md b/.apm/instructions/testing.instructions.md index fb8307a8fe..a85506920e 100644 --- a/.apm/instructions/testing.instructions.md +++ b/.apm/instructions/testing.instructions.md @@ -22,3 +22,11 @@ applyTo: "**/*_test.go" interface. * Prefer **table-driven tests** with descriptive case names. Search the same package for existing test patterns before writing new ones. +* New or modified functionality must include test coverage: + new Go functions and methods should have corresponding unit tests, + bug fixes should include a regression test that fails without the fix, + and pure functions (no DB/external dependencies) should always be tested. + If a function is hard to test, consider refactoring it into simpler functions + (disentangle separate conceptual chunks). + Exceptions: trivial changes (renaming, formatting, comments), generated code or + configuration-only changes, and refactors already covered by existing tests. diff --git a/.claude/rules/backend.md b/.claude/rules/backend.md index 5842594eb1..ed53e07e2c 100644 --- a/.claude/rules/backend.md +++ b/.claude/rules/backend.md @@ -3,13 +3,23 @@ paths: - "**/*.go" --- -* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Follow idiomatic Go practices. +* Choose names carefully using concise but appropriately descriptive words; in the scope of a + package, a name need only describe its function relative to that package. Provide docstring for + every package-level name explaining _why_ it exists and describing any parameter whose purpose is + not completely obvious from name and context. +* Keep packages, structs, and methods focused on a single clear conceptual "chunk": + a package should represent one cohesive concept, a struct should represent a single entity + in the scope of its package, and a method should operate on a single level of abstraction. + Prefer descriptive names that evoke a specific concept (e.g., "TestResultAggregator" not + "Manager"). Avoid generic names like "Manager", "Handler", "Util" without specific context, + and method names that don't indicate what they do (e.g., "Process", "Handle", "Do"). + Structs with more than about 7 top-level fields should be refactored into focused sub-types. +* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Use `k8s.io/apimachinery/pkg/util/sets` (e.g. `sets.New[string]()`) to deduplicate or collect unique strings. Do not use `map[string]bool` as a hand-rolled set. * Prefer structured logging where it makes sense; particularly for names and IDs, and often for counts, a log.WithField() call is preferred over formatting values into a string. -* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. * When modifying any data provider (BigQuery or PostgreSQL), ensure **parity between both implementations**. Changes to query logic, filtering, or returned data in one provider must be reflected in the other. * **Timestamps and dates**: Use proper types, never epoch integers. - PostgreSQL columns: use `TIMESTAMP WITH TIME ZONE` for timestamps and `DATE` for date-only values. All timestamps are UTC. @@ -17,3 +27,16 @@ paths: - API responses: timestamps serialize as RFC 3339 strings, dates as `YYYY-MM-DD` strings. Never return epoch millisecond integers. Never manually format with `time.Format()` into a string field. - Materialized views: project timestamp columns directly from the source table. Do not convert to epoch with `EXTRACT(epoch FROM ...)`. - GORM model tags: include `gorm:"type:date"` on date-only columns so GORM and migrations use the correct PostgreSQL type. +* Check `pkg/util/` for existing helper functions before adding inline utility logic. + Avoid calling the same utility function multiple times with identical arguments in the + same code path. +* Never ignore returned errors as `_` without clear justification. Errors should be wrapped + with context using `fmt.Errorf` with `%w`. Avoid `panic()` except in `init()` or fatal + conditions. Check for nil before dereferencing pointers. +* **Never** concatenate or format SQL queries with values directly from user input. Always use + placeholders for parameters in queries, preferably named (`@Name`). +* Structs used with GORM that have fields not backed by database columns (such as computed or + API-only fields) must include the `gorm:"-"` tag to explicitly exclude them from GORM operations. + BigQuery struct fields must have `bigquery:"column_name"` tags that exactly match the BigQuery + query result schema names, whether from table columns or aliases (give computed fields aliases). +* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. diff --git a/.claude/rules/frontend.md b/.claude/rules/frontend.md index 0beefa4ba6..c8a74665e4 100644 --- a/.claude/rules/frontend.md +++ b/.claude/rules/frontend.md @@ -3,21 +3,33 @@ paths: - "sippy-ng/**" --- +* The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. * After making changes, always run formatting and linting to maintain consistency: -```bash -npx eslint . --fix -npx prettier --write . -``` + ```bash + npx eslint . --fix + npx prettier --write . + ``` * Prefer functional components and React hooks over class components. * Keep UI elements consistent with Material-UI standards. - -The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. - +* Avoid nested ternary expressions in JSX. When there are more than two + branches, use if/else if chains, early returns, or a lookup object instead. + A single ternary is fine; nesting ternaries makes code hard to follow. +* Before adding date/time formatting, duration calculations, or string + utilities inline, check `sippy-ng/src/helpers.js` for existing functions + like `relativeDuration`, `safeEncodeURIComponent`, etc. Prefer reusing + existing helpers over reimplementing similar logic. +* React components with extensive inline CSS should use the `useStyles` pattern. + Inline style objects with more than 3-4 properties should be extracted to + `useStyles()` or styled components. Inline styles are acceptable for simple, + dynamic values (e.g., width based on props). * **Timestamps and dates from the API**: - Timestamps arrive as RFC 3339 strings (e.g., `"2024-06-27T15:30:00Z"`), not epoch millisecond integers. Use `new Date(value)` or `Temporal.Instant.from(value)` to parse them. - Dates arrive as `YYYY-MM-DD` strings (e.g., `"2024-06-27"`). Use `Temporal.PlainDate.from(value)` for date arithmetic. - For MUI DataGrid timestamp columns, use `type: 'date'` with a `valueGetter` that returns a `Date` object. Do not return epoch milliseconds from `valueGetter`. - For filter values sent to the API, use ISO 8601 strings (e.g., `new Date(...).toISOString()`), not epoch millisecond integers. - For day-level bucketing or date arithmetic, prefer `Temporal.PlainDate` over `Date` with manual millisecond math. +* Non-trivial modifications to frontend logic must include unit test coverage. + If a function is hard to test, consider refactoring to separate pure logic + from side effects. diff --git a/.claude/rules/general.md b/.claude/rules/general.md index 7504893386..45f5a5c358 100644 --- a/.claude/rules/general.md +++ b/.claude/rules/general.md @@ -20,5 +20,6 @@ The system consists of: * A **Go-based API backend**. * A **React/Material-UI frontend** (located in `sippy-ng`). * Data sources including **PostgreSQL**, and **BigQuery** +* A **headless daemon** for asynchronous processing Favor clarity and maintainability over cleverness. Comments should be minimal, helpful, and explain the "why" not the "what". diff --git a/.claude/rules/query.md b/.claude/rules/query.md new file mode 100644 index 0000000000..e6539d1422 --- /dev/null +++ b/.claude/rules/query.md @@ -0,0 +1,10 @@ +--- +paths: + - "pkg/**/query/**" +--- + +* BigQuery and SQL query-building code should have inline comments explaining the + purpose of each major query section (CTEs, JOINs, window functions, WHERE clauses). +* Abbreviations like CTE, matview, etc. should be expanded on first use. +* Functions that construct queries must stay under 200 lines; extract sub-queries or + CTEs into helper functions when they grow beyond that. diff --git a/.claude/rules/testing.md b/.claude/rules/testing.md index c084f25b4d..4ecf210618 100644 --- a/.claude/rules/testing.md +++ b/.claude/rules/testing.md @@ -22,3 +22,11 @@ paths: interface. * Prefer **table-driven tests** with descriptive case names. Search the same package for existing test patterns before writing new ones. +* New or modified functionality must include test coverage: + new Go functions and methods should have corresponding unit tests, + bug fixes should include a regression test that fails without the fix, + and pure functions (no DB/external dependencies) should always be tested. + If a function is hard to test, consider refactoring it into simpler functions + (disentangle separate conceptual chunks). + Exceptions: trivial changes (renaming, formatting, comments), generated code or + configuration-only changes, and refactors already covered by existing tests. diff --git a/.cursor/rules/backend.mdc b/.cursor/rules/backend.mdc index d94b2e3bb9..e0afc6d33c 100644 --- a/.cursor/rules/backend.mdc +++ b/.cursor/rules/backend.mdc @@ -3,13 +3,23 @@ description: Go backend coding standards for Sippy globs: "**/*.go" --- -* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Follow idiomatic Go practices. +* Choose names carefully using concise but appropriately descriptive words; in the scope of a + package, a name need only describe its function relative to that package. Provide docstring for + every package-level name explaining _why_ it exists and describing any parameter whose purpose is + not completely obvious from name and context. +* Keep packages, structs, and methods focused on a single clear conceptual "chunk": + a package should represent one cohesive concept, a struct should represent a single entity + in the scope of its package, and a method should operate on a single level of abstraction. + Prefer descriptive names that evoke a specific concept (e.g., "TestResultAggregator" not + "Manager"). Avoid generic names like "Manager", "Handler", "Util" without specific context, + and method names that don't indicate what they do (e.g., "Process", "Handle", "Do"). + Structs with more than about 7 top-level fields should be refactored into focused sub-types. +* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Use `k8s.io/apimachinery/pkg/util/sets` (e.g. `sets.New[string]()`) to deduplicate or collect unique strings. Do not use `map[string]bool` as a hand-rolled set. * Prefer structured logging where it makes sense; particularly for names and IDs, and often for counts, a log.WithField() call is preferred over formatting values into a string. -* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. * When modifying any data provider (BigQuery or PostgreSQL), ensure **parity between both implementations**. Changes to query logic, filtering, or returned data in one provider must be reflected in the other. * **Timestamps and dates**: Use proper types, never epoch integers. - PostgreSQL columns: use `TIMESTAMP WITH TIME ZONE` for timestamps and `DATE` for date-only values. All timestamps are UTC. @@ -17,3 +27,16 @@ globs: "**/*.go" - API responses: timestamps serialize as RFC 3339 strings, dates as `YYYY-MM-DD` strings. Never return epoch millisecond integers. Never manually format with `time.Format()` into a string field. - Materialized views: project timestamp columns directly from the source table. Do not convert to epoch with `EXTRACT(epoch FROM ...)`. - GORM model tags: include `gorm:"type:date"` on date-only columns so GORM and migrations use the correct PostgreSQL type. +* Check `pkg/util/` for existing helper functions before adding inline utility logic. + Avoid calling the same utility function multiple times with identical arguments in the + same code path. +* Never ignore returned errors as `_` without clear justification. Errors should be wrapped + with context using `fmt.Errorf` with `%w`. Avoid `panic()` except in `init()` or fatal + conditions. Check for nil before dereferencing pointers. +* **Never** concatenate or format SQL queries with values directly from user input. Always use + placeholders for parameters in queries, preferably named (`@Name`). +* Structs used with GORM that have fields not backed by database columns (such as computed or + API-only fields) must include the `gorm:"-"` tag to explicitly exclude them from GORM operations. + BigQuery struct fields must have `bigquery:"column_name"` tags that exactly match the BigQuery + query result schema names, whether from table columns or aliases (give computed fields aliases). +* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. diff --git a/.cursor/rules/frontend.mdc b/.cursor/rules/frontend.mdc index f56ba9c834..cd47bb17ad 100644 --- a/.cursor/rules/frontend.mdc +++ b/.cursor/rules/frontend.mdc @@ -3,21 +3,33 @@ description: React/Material-UI frontend guidelines for Sippy globs: "sippy-ng/**" --- +* The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. * After making changes, always run formatting and linting to maintain consistency: -```bash -npx eslint . --fix -npx prettier --write . -``` + ```bash + npx eslint . --fix + npx prettier --write . + ``` * Prefer functional components and React hooks over class components. * Keep UI elements consistent with Material-UI standards. - -The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. - +* Avoid nested ternary expressions in JSX. When there are more than two + branches, use if/else if chains, early returns, or a lookup object instead. + A single ternary is fine; nesting ternaries makes code hard to follow. +* Before adding date/time formatting, duration calculations, or string + utilities inline, check `sippy-ng/src/helpers.js` for existing functions + like `relativeDuration`, `safeEncodeURIComponent`, etc. Prefer reusing + existing helpers over reimplementing similar logic. +* React components with extensive inline CSS should use the `useStyles` pattern. + Inline style objects with more than 3-4 properties should be extracted to + `useStyles()` or styled components. Inline styles are acceptable for simple, + dynamic values (e.g., width based on props). * **Timestamps and dates from the API**: - Timestamps arrive as RFC 3339 strings (e.g., `"2024-06-27T15:30:00Z"`), not epoch millisecond integers. Use `new Date(value)` or `Temporal.Instant.from(value)` to parse them. - Dates arrive as `YYYY-MM-DD` strings (e.g., `"2024-06-27"`). Use `Temporal.PlainDate.from(value)` for date arithmetic. - For MUI DataGrid timestamp columns, use `type: 'date'` with a `valueGetter` that returns a `Date` object. Do not return epoch milliseconds from `valueGetter`. - For filter values sent to the API, use ISO 8601 strings (e.g., `new Date(...).toISOString()`), not epoch millisecond integers. - For day-level bucketing or date arithmetic, prefer `Temporal.PlainDate` over `Date` with manual millisecond math. +* Non-trivial modifications to frontend logic must include unit test coverage. + If a function is hard to test, consider refactoring to separate pure logic + from side effects. diff --git a/.cursor/rules/general.mdc b/.cursor/rules/general.mdc index 0b5e0c6255..983de364ec 100644 --- a/.cursor/rules/general.mdc +++ b/.cursor/rules/general.mdc @@ -20,5 +20,6 @@ The system consists of: * A **Go-based API backend**. * A **React/Material-UI frontend** (located in `sippy-ng`). * Data sources including **PostgreSQL**, and **BigQuery** +* A **headless daemon** for asynchronous processing Favor clarity and maintainability over cleverness. Comments should be minimal, helpful, and explain the "why" not the "what". diff --git a/.cursor/rules/query.mdc b/.cursor/rules/query.mdc new file mode 100644 index 0000000000..1461040e16 --- /dev/null +++ b/.cursor/rules/query.mdc @@ -0,0 +1,10 @@ +--- +description: Guidelines for BigQuery and SQL query-building code +globs: "pkg/**/query/**" +--- + +* BigQuery and SQL query-building code should have inline comments explaining the + purpose of each major query section (CTEs, JOINs, window functions, WHERE clauses). +* Abbreviations like CTE, matview, etc. should be expanded on first use. +* Functions that construct queries must stay under 200 lines; extract sub-queries or + CTEs into helper functions when they grow beyond that. diff --git a/.cursor/rules/testing.mdc b/.cursor/rules/testing.mdc index 729d392272..e0e056c31a 100644 --- a/.cursor/rules/testing.mdc +++ b/.cursor/rules/testing.mdc @@ -22,3 +22,11 @@ globs: "**/*_test.go" interface. * Prefer **table-driven tests** with descriptive case names. Search the same package for existing test patterns before writing new ones. +* New or modified functionality must include test coverage: + new Go functions and methods should have corresponding unit tests, + bug fixes should include a regression test that fails without the fix, + and pure functions (no DB/external dependencies) should always be tested. + If a function is hard to test, consider refactoring it into simpler functions + (disentangle separate conceptual chunks). + Exceptions: trivial changes (renaming, formatting, comments), generated code or + configuration-only changes, and refactors already covered by existing tests. diff --git a/AGENTS.md b/AGENTS.md index d34890c1b6..466ef071dc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # AGENTS.md - + @@ -55,6 +55,10 @@ This runs SQL-level tests against a real PostgreSQL instance using testcontainer * Documentation and code belong in the same PR; never treat a docs update as a follow-up task. * Do not use em dashes when writing docs. Use commas, parentheses, or periods instead. +* Files under `pkg/**/jobrunscan**/`, `pkg/**/jobrunannotator**/`, `pkg/api/jobartifacts/**`, and + `sippy-ng/src/component_readiness/JobArtifactQuery.js` are part of the symptoms feature documented + in `docs/features/job-analysis-symptoms.md`. Update this document when there are changes to data + models, API surface, or data flow in the symptoms feature. ### APM context generation @@ -74,19 +78,30 @@ The system consists of: * A **Go-based API backend**. * A **React/Material-UI frontend** (located in `sippy-ng`). * Data sources including **PostgreSQL**, and **BigQuery** +* A **headless daemon** for asynchronous processing Favor clarity and maintainability over cleverness. Comments should be minimal, helpful, and explain the "why" not the "what". ## Files matching `**/*.go` -* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Follow idiomatic Go practices. +* Choose names carefully using concise but appropriately descriptive words; in the scope of a + package, a name need only describe its function relative to that package. Provide docstring for + every package-level name explaining _why_ it exists and describing any parameter whose purpose is + not completely obvious from name and context. +* Keep packages, structs, and methods focused on a single clear conceptual "chunk": + a package should represent one cohesive concept, a struct should represent a single entity + in the scope of its package, and a method should operate on a single level of abstraction. + Prefer descriptive names that evoke a specific concept (e.g., "TestResultAggregator" not + "Manager"). Avoid generic names like "Manager", "Handler", "Util" without specific context, + and method names that don't indicate what they do (e.g., "Process", "Handle", "Do"). + Structs with more than about 7 top-level fields should be refactored into focused sub-types. +* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Use `k8s.io/apimachinery/pkg/util/sets` (e.g. `sets.New[string]()`) to deduplicate or collect unique strings. Do not use `map[string]bool` as a hand-rolled set. * Prefer structured logging where it makes sense; particularly for names and IDs, and often for counts, a log.WithField() call is preferred over formatting values into a string. -* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. * When modifying any data provider (BigQuery or PostgreSQL), ensure **parity between both implementations**. Changes to query logic, filtering, or returned data in one provider must be reflected in the other. * **Timestamps and dates**: Use proper types, never epoch integers. - PostgreSQL columns: use `TIMESTAMP WITH TIME ZONE` for timestamps and `DATE` for date-only values. All timestamps are UTC. @@ -94,6 +109,19 @@ Favor clarity and maintainability over cleverness. Comments should be minimal, h - API responses: timestamps serialize as RFC 3339 strings, dates as `YYYY-MM-DD` strings. Never return epoch millisecond integers. Never manually format with `time.Format()` into a string field. - Materialized views: project timestamp columns directly from the source table. Do not convert to epoch with `EXTRACT(epoch FROM ...)`. - GORM model tags: include `gorm:"type:date"` on date-only columns so GORM and migrations use the correct PostgreSQL type. +* Check `pkg/util/` for existing helper functions before adding inline utility logic. + Avoid calling the same utility function multiple times with identical arguments in the + same code path. +* Never ignore returned errors as `_` without clear justification. Errors should be wrapped + with context using `fmt.Errorf` with `%w`. Avoid `panic()` except in `init()` or fatal + conditions. Check for nil before dereferencing pointers. +* **Never** concatenate or format SQL queries with values directly from user input. Always use + placeholders for parameters in queries, preferably named (`@Name`). +* Structs used with GORM that have fields not backed by database columns (such as computed or + API-only fields) must include the `gorm:"-"` tag to explicitly exclude them from GORM operations. + BigQuery struct fields must have `bigquery:"column_name"` tags that exactly match the BigQuery + query result schema names, whether from table columns or aliases (give computed fields aliases). +* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. ## Files matching `**/*_test.go` @@ -117,6 +145,14 @@ Favor clarity and maintainability over cleverness. Comments should be minimal, h interface. * Prefer **table-driven tests** with descriptive case names. Search the same package for existing test patterns before writing new ones. +* New or modified functionality must include test coverage: + new Go functions and methods should have corresponding unit tests, + bug fixes should include a regression test that fails without the fix, + and pure functions (no DB/external dependencies) should always be tested. + If a function is hard to test, consider refactoring it into simpler functions + (disentangle separate conceptual chunks). + Exceptions: trivial changes (renaming, formatting, comments), generated code or + configuration-only changes, and refactors already covered by existing tests. --- *This file was generated by APM CLI. Do not edit manually.* diff --git a/CLAUDE.md b/CLAUDE.md index 03e3105d09..ee7caf561f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # CLAUDE.md - + # Project Standards @@ -56,6 +56,10 @@ This runs SQL-level tests against a real PostgreSQL instance using testcontainer * Documentation and code belong in the same PR; never treat a docs update as a follow-up task. * Do not use em dashes when writing docs. Use commas, parentheses, or periods instead. +* Files under `pkg/**/jobrunscan**/`, `pkg/**/jobrunannotator**/`, `pkg/api/jobartifacts/**`, and + `sippy-ng/src/component_readiness/JobArtifactQuery.js` are part of the symptoms feature documented + in `docs/features/job-analysis-symptoms.md`. Update this document when there are changes to data + models, API surface, or data flow in the symptoms feature. ### APM context generation @@ -75,19 +79,30 @@ The system consists of: * A **Go-based API backend**. * A **React/Material-UI frontend** (located in `sippy-ng`). * Data sources including **PostgreSQL**, and **BigQuery** +* A **headless daemon** for asynchronous processing Favor clarity and maintainability over cleverness. Comments should be minimal, helpful, and explain the "why" not the "what". ## Files matching `**/*.go` -* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Follow idiomatic Go practices. +* Choose names carefully using concise but appropriately descriptive words; in the scope of a + package, a name need only describe its function relative to that package. Provide docstring for + every package-level name explaining _why_ it exists and describing any parameter whose purpose is + not completely obvious from name and context. +* Keep packages, structs, and methods focused on a single clear conceptual "chunk": + a package should represent one cohesive concept, a struct should represent a single entity + in the scope of its package, and a method should operate on a single level of abstraction. + Prefer descriptive names that evoke a specific concept (e.g., "TestResultAggregator" not + "Manager"). Avoid generic names like "Manager", "Handler", "Util" without specific context, + and method names that don't indicate what they do (e.g., "Process", "Handle", "Do"). + Structs with more than about 7 top-level fields should be refactored into focused sub-types. +* When adding or updating APIs, **use HATEOAS** in responses to support discoverability and consistent client interaction. * Use `k8s.io/apimachinery/pkg/util/sets` (e.g. `sets.New[string]()`) to deduplicate or collect unique strings. Do not use `map[string]bool` as a hand-rolled set. * Prefer structured logging where it makes sense; particularly for names and IDs, and often for counts, a log.WithField() call is preferred over formatting values into a string. -* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. * When modifying any data provider (BigQuery or PostgreSQL), ensure **parity between both implementations**. Changes to query logic, filtering, or returned data in one provider must be reflected in the other. * **Timestamps and dates**: Use proper types, never epoch integers. - PostgreSQL columns: use `TIMESTAMP WITH TIME ZONE` for timestamps and `DATE` for date-only values. All timestamps are UTC. @@ -95,6 +110,19 @@ Favor clarity and maintainability over cleverness. Comments should be minimal, h - API responses: timestamps serialize as RFC 3339 strings, dates as `YYYY-MM-DD` strings. Never return epoch millisecond integers. Never manually format with `time.Format()` into a string field. - Materialized views: project timestamp columns directly from the source table. Do not convert to epoch with `EXTRACT(epoch FROM ...)`. - GORM model tags: include `gorm:"type:date"` on date-only columns so GORM and migrations use the correct PostgreSQL type. +* Check `pkg/util/` for existing helper functions before adding inline utility logic. + Avoid calling the same utility function multiple times with identical arguments in the + same code path. +* Never ignore returned errors as `_` without clear justification. Errors should be wrapped + with context using `fmt.Errorf` with `%w`. Avoid `panic()` except in `init()` or fatal + conditions. Check for nil before dereferencing pointers. +* **Never** concatenate or format SQL queries with values directly from user input. Always use + placeholders for parameters in queries, preferably named (`@Name`). +* Structs used with GORM that have fields not backed by database columns (such as computed or + API-only fields) must include the `gorm:"-"` tag to explicitly exclude them from GORM operations. + BigQuery struct fields must have `bigquery:"column_name"` tags that exactly match the BigQuery + query result schema names, whether from table columns or aliases (give computed fields aliases). +* After making changes, always run `gofmt -w` on modified files to ensure proper formatting. ## Files matching `**/*_test.go` @@ -118,6 +146,14 @@ Favor clarity and maintainability over cleverness. Comments should be minimal, h interface. * Prefer **table-driven tests** with descriptive case names. Search the same package for existing test patterns before writing new ones. +* New or modified functionality must include test coverage: + new Go functions and methods should have corresponding unit tests, + bug fixes should include a regression test that fails without the fix, + and pure functions (no DB/external dependencies) should always be tested. + If a function is hard to test, consider refactoring it into simpler functions + (disentangle separate conceptual chunks). + Exceptions: trivial changes (renaming, formatting, comments), generated code or + configuration-only changes, and refactors already covered by existing tests. --- *This file was generated by APM CLI. Do not edit manually.* diff --git a/Makefile b/Makefile index 4ad66d7981..75d970489e 100644 --- a/Makefile +++ b/Makefile @@ -79,10 +79,12 @@ apm: verify-migrations: ./hack/verify-migrations.sh +APM_GENERATED_FILES := .claude .cursor .gemini .opencode AGENTS.md CLAUDE.md GEMINI.md sippy-ng/AGENTS.md sippy-ng/CLAUDE.md mcp/AGENTS.md mcp/CLAUDE.md pkg/db/query/AGENTS.md pkg/db/query/CLAUDE.md + verify-apm: apm - @if [ -n "$$(git status --porcelain -- .claude .cursor .gemini .opencode AGENTS.md CLAUDE.md GEMINI.md sippy-ng/AGENTS.md sippy-ng/CLAUDE.md mcp/AGENTS.md mcp/CLAUDE.md)" ]; then \ + @if [ -n "$$(git status --porcelain -- $(APM_GENERATED_FILES))" ]; then \ echo "ERROR: Generated APM files are out of date. Run 'make apm' and commit the results."; \ - git status --short -- .claude .cursor .gemini .opencode AGENTS.md CLAUDE.md GEMINI.md sippy-ng/AGENTS.md sippy-ng/CLAUDE.md mcp/AGENTS.md mcp/CLAUDE.md; \ + git status --short -- $(APM_GENERATED_FILES); \ exit 1; \ fi diff --git a/apm.lock.yaml b/apm.lock.yaml index a2d0912ef5..3de62a0000 100644 --- a/apm.lock.yaml +++ b/apm.lock.yaml @@ -94,13 +94,13 @@ local_deployed_file_hashes: .claude/commands/sippy-generate-release-views.md: sha256:eb4c9eeeea2ab2a90e8a8839147d8a1a309ea6ce3dafd397c6d2485c93068a9a .claude/commands/sippy-update-ga-release-views.md: sha256:4a5589bacc05127e427a2de4d34a8f13e05e297bdf6ebf7473c9e71f47a6b4f4 .claude/commands/sippy-update-job-variant.md: sha256:f88742dddeec5024931959a8330fdce362ffdd9b8825e808830ac346605cbd16 - .claude/rules/backend.md: sha256:5a5dc512c2429362c7db75f52e9407a5a1576864b59361c4df5d4d8601d17de8 + .claude/rules/backend.md: sha256:60f9f7f5315eb4c1f9bc84af897529796e38abb343056c2ed6605cb0ddcf3401 .claude/rules/config.md: sha256:96c5e42c039230f1e4e7f9ba56d04e6f76f23deeecdd858634c440d6029a6a83 .claude/rules/dev-commands.md: sha256:171b806b2f75a71f20126b8a9ceabe88592c11cede3fe510498e855d53d43b9e - .claude/rules/frontend.md: sha256:cdfc2cdc3981c43d91dcf9583d0fff2d3f0b4f355ded4237aa264211fe42600c - .claude/rules/general.md: sha256:997f68e86cb43485ec5f108be3417f9bbb43ae1faffd660d598f18260f5df3ce + .claude/rules/frontend.md: sha256:3abcacd88759b5d833d36457a89b66b6a0a53c7df29c847d17775db1f879300b + .claude/rules/general.md: sha256:e6dff95b03237ef5756e873afac2851c8f658d860b7853eb063c7766738533c0 .claude/rules/mcp.md: sha256:ddfe965e7cf8cddbba1374c6ae582a20ac0af17c958bf10e1a4edff6ff2ad0b8 - .claude/rules/testing.md: sha256:a28be642547f1093d2f39d3a29e114dc11b480a1e130b3dfd4a19cd8ffe348de + .claude/rules/testing.md: sha256:aef755feb8a6829ddb1d2609b98b17380c3c457cc86fd09158c2ec1d7432a893 .cursor/commands/sippy-dev-app.md: sha256:656276ed961940c137dde32ecdb0501427d4d811502a27125ba073adc770d266 .cursor/commands/sippy-dev-frontend.md: sha256:42eae4b3bc610c9fcb43533a6fed229a6d1c409d279f3d6f93672986ede62e3a .cursor/commands/sippy-dev-migrate.md: sha256:80160e88e0cc0fc09ab3dd9cc6fc496fe87dd8873800eb65d700868034d59da2 @@ -112,13 +112,13 @@ local_deployed_file_hashes: .cursor/commands/sippy-generate-release-views.md: sha256:eb4c9eeeea2ab2a90e8a8839147d8a1a309ea6ce3dafd397c6d2485c93068a9a .cursor/commands/sippy-update-ga-release-views.md: sha256:4a5589bacc05127e427a2de4d34a8f13e05e297bdf6ebf7473c9e71f47a6b4f4 .cursor/commands/sippy-update-job-variant.md: sha256:f88742dddeec5024931959a8330fdce362ffdd9b8825e808830ac346605cbd16 - .cursor/rules/backend.mdc: sha256:5355475617074d8212962b19c94649c9609f3565d2e844caa6f4730c619c0dea + .cursor/rules/backend.mdc: sha256:c89283fab05858564980b107fd0d0e99fd33e49409b4935ab2401999ed6e5c26 .cursor/rules/config.mdc: sha256:d6e2195399bbb26a3fef7e54bd01862ffce39d89dce8aabdb0b89c89028192eb .cursor/rules/dev-commands.mdc: sha256:7e9635959af4dd2bf54b348dfdf41e3cf2b77c01aa3496a2e42a273c372ff9c0 - .cursor/rules/frontend.mdc: sha256:6cbbb94363b782dc6763344a72f5a996fe25a4bd5db08862e578b611ed083612 - .cursor/rules/general.mdc: sha256:5bc6e1e12d53d85656248c9dc1239c74bcc0df29d5987f3b08e3d79e3df413b7 + .cursor/rules/frontend.mdc: sha256:1c2e1bd63f186d5794bf7dcf5e07bac7e46544857e8b032e380fdf15ed297f83 + .cursor/rules/general.mdc: sha256:9baa2a69757a9a5de30c901fabb48ae1d894e403cde7ae60170d24d25f52664e .cursor/rules/mcp.mdc: sha256:c02472afd46e4c89f71d4487dcd5da98b0c1bcbcf7f9cbc4d7ed4e7d3a206ec1 - .cursor/rules/testing.mdc: sha256:5192be9724566f57a7423dbb65780b54d99b79d3f11e99d2225257e705f99a96 + .cursor/rules/testing.mdc: sha256:06eef9186dd1cec1bcdf067344e5e2e83515da66f8df4102c2b1e85641a21b1f .gemini/commands/sippy-dev-app.toml: sha256:fc28174eeab4e440694a823bd838d429241997a018d8a13f32e0f67ca4d973c5 .gemini/commands/sippy-dev-frontend.toml: sha256:ec4ab5e1fb7581f09473e33b3ed4f53ce40509f23aac581e3b100ad1f59de5e5 .gemini/commands/sippy-dev-migrate.toml: sha256:25c98ba4bfdb95270dfcb4238ae688f9a66f0e645d8a4a6c5e03b1cf8db5cb7e diff --git a/pkg/db/query/AGENTS.md b/pkg/db/query/AGENTS.md new file mode 100644 index 0000000000..38f7b18e78 --- /dev/null +++ b/pkg/db/query/AGENTS.md @@ -0,0 +1,21 @@ +# AGENTS.md + + + + + +## Files matching `pkg/**/query/**` + + +* Abbreviations should be expanded on first use, for example Common Table Expression (CTE) and + Materialized View (matview). "SQL" is exempt from this rule. +* BigQuery and SQL query-building code should have inline comments explaining the + purpose of each major query section (CTEs, JOINs, window functions, WHERE clauses). +* Functions that construct queries must stay under 200 lines; extract sub-queries or + CTEs into helper functions when they grow beyond that. +* When constructing SQL queries, prefer using a format string and/or multi-line string rather than + concatenation of short strings; and always use placeholders for parameters. + +--- +*This file was generated by APM CLI. Do not edit manually.* +*To regenerate: `apm compile`* diff --git a/pkg/db/query/CLAUDE.md b/pkg/db/query/CLAUDE.md new file mode 100644 index 0000000000..bb7f27cd36 --- /dev/null +++ b/pkg/db/query/CLAUDE.md @@ -0,0 +1,22 @@ +# CLAUDE.md + + + + +# Project Standards + +## Files matching `pkg/**/query/**` + + +* Abbreviations should be expanded on first use, for example Common Table Expression (CTE) and + Materialized View (matview). "SQL" is exempt from this rule. +* BigQuery and SQL query-building code should have inline comments explaining the + purpose of each major query section (CTEs, JOINs, window functions, WHERE clauses). +* Functions that construct queries must stay under 200 lines; extract sub-queries or + CTEs into helper functions when they grow beyond that. +* When constructing SQL queries, prefer using a format string and/or multi-line string rather than + concatenation of short strings; and always use placeholders for parameters. + +--- +*This file was generated by APM CLI. Do not edit manually.* +*To regenerate: `apm compile`* diff --git a/sippy-ng/AGENTS.md b/sippy-ng/AGENTS.md index a73b497d08..17d696f140 100644 --- a/sippy-ng/AGENTS.md +++ b/sippy-ng/AGENTS.md @@ -1,30 +1,42 @@ # AGENTS.md - + ## Files matching `sippy-ng/**` +* The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. * After making changes, always run formatting and linting to maintain consistency: -```bash -npx eslint . --fix -npx prettier --write . -``` + ```bash + npx eslint . --fix + npx prettier --write . + ``` * Prefer functional components and React hooks over class components. * Keep UI elements consistent with Material-UI standards. - -The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. - +* Avoid nested ternary expressions in JSX. When there are more than two + branches, use if/else if chains, early returns, or a lookup object instead. + A single ternary is fine; nesting ternaries makes code hard to follow. +* Before adding date/time formatting, duration calculations, or string + utilities inline, check `sippy-ng/src/helpers.js` for existing functions + like `relativeDuration`, `safeEncodeURIComponent`, etc. Prefer reusing + existing helpers over reimplementing similar logic. +* React components with extensive inline CSS should use the `useStyles` pattern. + Inline style objects with more than 3-4 properties should be extracted to + `useStyles()` or styled components. Inline styles are acceptable for simple, + dynamic values (e.g., width based on props). * **Timestamps and dates from the API**: - Timestamps arrive as RFC 3339 strings (e.g., `"2024-06-27T15:30:00Z"`), not epoch millisecond integers. Use `new Date(value)` or `Temporal.Instant.from(value)` to parse them. - Dates arrive as `YYYY-MM-DD` strings (e.g., `"2024-06-27"`). Use `Temporal.PlainDate.from(value)` for date arithmetic. - For MUI DataGrid timestamp columns, use `type: 'date'` with a `valueGetter` that returns a `Date` object. Do not return epoch milliseconds from `valueGetter`. - For filter values sent to the API, use ISO 8601 strings (e.g., `new Date(...).toISOString()`), not epoch millisecond integers. - For day-level bucketing or date arithmetic, prefer `Temporal.PlainDate` over `Date` with manual millisecond math. +* Non-trivial modifications to frontend logic must include unit test coverage. + If a function is hard to test, consider refactoring to separate pure logic + from side effects. --- *This file was generated by APM CLI. Do not edit manually.* diff --git a/sippy-ng/CLAUDE.md b/sippy-ng/CLAUDE.md index bb805987d5..95cdcacbdf 100644 --- a/sippy-ng/CLAUDE.md +++ b/sippy-ng/CLAUDE.md @@ -1,6 +1,6 @@ # CLAUDE.md - + # Project Standards @@ -8,24 +8,36 @@ ## Files matching `sippy-ng/**` +* The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. * After making changes, always run formatting and linting to maintain consistency: -```bash -npx eslint . --fix -npx prettier --write . -``` + ```bash + npx eslint . --fix + npx prettier --write . + ``` * Prefer functional components and React hooks over class components. * Keep UI elements consistent with Material-UI standards. - -The frontend uses `npm`. If you must install or update any dependencies, always use the `--ignore-scripts` flag. - +* Avoid nested ternary expressions in JSX. When there are more than two + branches, use if/else if chains, early returns, or a lookup object instead. + A single ternary is fine; nesting ternaries makes code hard to follow. +* Before adding date/time formatting, duration calculations, or string + utilities inline, check `sippy-ng/src/helpers.js` for existing functions + like `relativeDuration`, `safeEncodeURIComponent`, etc. Prefer reusing + existing helpers over reimplementing similar logic. +* React components with extensive inline CSS should use the `useStyles` pattern. + Inline style objects with more than 3-4 properties should be extracted to + `useStyles()` or styled components. Inline styles are acceptable for simple, + dynamic values (e.g., width based on props). * **Timestamps and dates from the API**: - Timestamps arrive as RFC 3339 strings (e.g., `"2024-06-27T15:30:00Z"`), not epoch millisecond integers. Use `new Date(value)` or `Temporal.Instant.from(value)` to parse them. - Dates arrive as `YYYY-MM-DD` strings (e.g., `"2024-06-27"`). Use `Temporal.PlainDate.from(value)` for date arithmetic. - For MUI DataGrid timestamp columns, use `type: 'date'` with a `valueGetter` that returns a `Date` object. Do not return epoch milliseconds from `valueGetter`. - For filter values sent to the API, use ISO 8601 strings (e.g., `new Date(...).toISOString()`), not epoch millisecond integers. - For day-level bucketing or date arithmetic, prefer `Temporal.PlainDate` over `Date` with manual millisecond math. +* Non-trivial modifications to frontend logic must include unit test coverage. + If a function is hard to test, consider refactoring to separate pure logic + from side effects. --- *This file was generated by APM CLI. Do not edit manually.*