Skip to content

Commit d7ca6dc

Browse files
huangyiireneclaude
andauthored
docs(skills): objectstack-query factual sweep — 17 false behavioral claims corrected (#13740)
Flight (3) of the published-skills factual sweep. Every behavioral claim in skills/objectstack-query/** verified against the implementation with executed probes; corrections are byte-shrinking against the pinned token ratchets. Net: -33 lines, -255 tokens across the package. No ceiling raised. Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent b3d8a75 commit d7ca6dc

5 files changed

Lines changed: 101 additions & 134 deletions

File tree

skills/objectstack-query/SKILL.md

Lines changed: 54 additions & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -22,14 +22,10 @@ Expert instructions for constructing data queries using the ObjectStack
2222
Query DSL. This skill covers filter expressions, sorting, pagination,
2323
aggregation, full-text search, and the expand system for related records.
2424

25-
**Schema vs. runtime:** the `QueryAST` schema declares more than the engine
26-
currently executes. Sections below marked
27-
28-
> ⚠️ **Schema-reserved — NOT executed by the engine yet.**
29-
30-
describe properties that validate against the schema but are silently
31-
ignored (or rejected) at runtime. Never emit them in production queries —
32-
each caveat shows the working alternative.
25+
**Schema vs. runtime:** every callout below says which side a property is on —
26+
**REMOVED** (tombstoned; a query carrying it fails to parse), ⚠️ **not
27+
enforced** (validates, then silently ignored — never emit it), ✅ **Enforced**.
28+
Each removal callout names the live replacement.
3329

3430
---
3531

@@ -217,25 +213,22 @@ Filter through relationships without an explicit join:
217213

218214
### Field References (Cross-Field Comparisons)
219215

220-
> ⚠️ **Schema-reserved — NOT executed by the engine yet.** `$field` exists
221-
> only in the filter schema. No engine or driver code interprets it — the
222-
> `{ $field: '...' }` object binds as a **literal value**, so the query
223-
> silently returns zero rows. Do not use it.
216+
> **Enforced.** A `{ $field: '...' }` comparand compares two columns of the
217+
> same row. The in-memory evaluator resolves the reference against the record;
218+
> `driver-sql` pushes it down as a column-to-column predicate. Same rows.
224219
225220
```typescript
226-
// ❌ Schema-valid but NOT executed — matches nothing
221+
// ✅ Accounts whose actual revenue beat the estimate
227222
{
228223
where: {
229224
actual_revenue: { $gt: { $field: 'estimated_revenue' } }
230225
}
231226
}
232227
```
233228

234-
**Working alternatives:**
235-
- Define a **formula field** on the object that computes the comparison
236-
(e.g. `exceeds_estimate` as a boolean), then filter on it:
237-
`{ where: { exceeds_estimate: true } }` (see **objectstack-data**).
238-
- Fetch both fields and compare in **application code**.
229+
Legal in a **comparison** position only. As an `$in` / `$nin` member or a
230+
`$between` endpoint it is refused at parse — no evaluation path resolves a
231+
reference there.
239232

240233
---
241234

@@ -331,12 +324,11 @@ unique or near-unique column such as `created_at` or `id`) so
331324
| `max` | Maximum | `MAX(field)` |
332325
| `count_distinct` | Unique count | `COUNT(DISTINCT field)` |
333326

334-
> ⚠️ **Driver support varies.** On SQL datasources the driver executes only
335-
> `count` / `sum` / `avg` / `min` / `max` and **throws** on `count_distinct`;
336-
> the per-aggregation `distinct: true` flag is also ignored there. The
337-
> in-memory fallback path (driver-rest, driver-memory, timezone/date-bucket
338-
> fallbacks) supports all six functions plus `distinct`. For portable queries,
339-
> stick to the first five.
327+
> **All six are portable.** `count_distinct` lowers to `COUNT(DISTINCT x)`
328+
> on every SQL face and computes identically on the in-memory path, so the
329+
> declared-but-uncompiled set is empty. The per-aggregation `distinct: true`
330+
> flag went the other way — **removed in 17**, refused at parse; the live
331+
> spelling for a deduplicated count is `count_distinct`.
340332
341333
> **Removed in 17.** `array_agg` and `string_agg` left this vocabulary:
342334
> declared but lowered by no SQL backend, so whether they worked depended on
@@ -387,25 +379,23 @@ const rows = await engine.aggregate('deal', {
387379

388380
### Filtered Aggregation
389381

390-
> ⚠️ **Per-aggregation `filter` is schema-reserved — NOT executed by the
391-
> engine yet.** The SQL driver ignores it and the in-memory path ignores it
392-
> too, so a `filter`-carrying aggregation returns the **unfiltered** number —
393-
> silently wrong results. **Working alternative:** issue one aggregate call
394-
> per condition, moving the condition into the query-level `where`:
382+
> **Enforced.** A per-aggregation `filter` scopes that one measure, so a
383+
> total and a conditional count share one call. Any aggregation carrying a
384+
> non-empty `filter` forces the in-memory path — no driver compiles a
385+
> conditional aggregate, and one reached directly refuses `NOT_IMPLEMENTED`;
386+
> unfiltered aggregations keep native push-down.
395387
396388
```typescript
397-
// ❌ filter on the aggregation is silently ignored
398-
// { function: 'count', alias: 'high_value_orders',
399-
// filter: { amount: { $gt: 1000 } } }
400-
401-
// ✅ Separate aggregate calls, condition in `where`
402-
const [totals] = await engine.aggregate('order', {
403-
aggregations: [{ function: 'count', alias: 'total_orders' }],
404-
});
405-
const [highValue] = await engine.aggregate('order', {
406-
where: { amount: { $gt: 1000 } },
407-
aggregations: [{ function: 'count', alias: 'high_value_orders' }],
389+
// ✅ Total and conditional counts in ONE call
390+
const [kpis] = await engine.aggregate('order', {
391+
aggregations: [
392+
{ function: 'count', alias: 'total_orders' },
393+
{ function: 'count', alias: 'high_value_orders',
394+
filter: { amount: { $gt: 1000 } } },
395+
],
408396
});
397+
// An unknown operator inside `filter` refuses INVALID_FILTER/400 — it never
398+
// silently answers the unfiltered number.
409399
```
410400

411401
---
@@ -475,10 +465,10 @@ Load related records through lookup/master_detail fields:
475465

476466
Only the **`query` + `fields`** subset of the search schema executes. The
477467
engine expands the search string into a driver-agnostic filter: each term
478-
becomes an `$or` of `$contains` predicates across the resolved searchable
479-
fields, and multiple whitespace-separated terms are **AND-ed** (every term
480-
must hit some field). Matching is case-insensitive; `select`/`status`
481-
fields match by option *label*, mapped to stored values.
468+
becomes an `$or` of `$icontains` predicates (the case-INSENSITIVE twin of the
469+
case-sensitive `$contains`) across the resolved searchable fields, and multiple
470+
whitespace-separated terms are **AND-ed** (every term must hit some field).
471+
`select`/`status` fields match by option *label*, mapped to stored values.
482472

483473
```typescript
484474
{
@@ -491,8 +481,8 @@ fields match by option *label*, mapped to stored values.
491481
}
492482
// Executes as:
493483
// { $and: [
494-
// { $or: [{ title: { $contains: 'machine' } }, { content: { $contains: 'machine' } }] },
495-
// { $or: [{ title: { $contains: 'learning' } }, { content: { $contains: 'learning' } }] },
484+
// { $or: [{ title: { $icontains: 'machine' } }, { content: { $icontains: 'machine' } }] },
485+
// { $or: [{ title: { $icontains: 'learning' } }, { content: { $icontains: 'learning' } }] },
496486
// ]}
497487
```
498488

@@ -529,19 +519,18 @@ maintained on write and listed in `task.searchableFields`:
529519
limit: 20,
530520
}
531521
// Expands to a single-table scan — no traversal, every driver:
532-
// { $and: [{ $or: [
533-
// { name: { $contains: 'apollo' } },
534-
// { project_name: { $contains: 'apollo' } },
535-
// ]}]}
522+
// { $or: [
523+
// { name: { $icontains: 'apollo' } },
524+
// { project_name: { $icontains: 'apollo' } },
525+
// ]}
536526
```
537527

538528
❌ The mirror must be a **stored** field — a `formula` field is virtual, no
539-
driver materializes a column for it, so a `$contains` predicate against one has
540-
nothing to scan. Nothing rejects the mistake for you: `searchableFields` admits
541-
any field the object declares, so a formula entry clears both lint and the
542-
ingress gate and then never matches. The trade-off is mirror maintenance — hooks
543-
on both write paths (child re-parented, parent renamed) plus a backfill for rows
544-
written around the hooks.
529+
driver materializes a column for it, so a search predicate against one has
530+
nothing to scan. Two guards catch that: lint errors on a virtual
531+
`searchableFields` entry, and the ingress gate refuses one by name. The
532+
trade-off is mirror maintenance — hooks on both write paths (child re-parented,
533+
parent renamed) plus a backfill for rows written around the hooks.
545534

546535
Cross-object search paths are rejected by design, not pending. Modelling side of
547536
this (the field, the hooks, the lint wording): **objectstack-data → Search Fields
@@ -606,25 +595,21 @@ use [`expand`](#expand-related-records).
606595

607596
### Dashboard Aggregation Pattern
608597

609-
Unconditional KPIs can share one aggregate call; a KPI with its own
610-
condition needs a **separate call** with the condition in `where`
611-
(per-aggregation `filter` is schema-reserved — see Filtered Aggregation):
598+
Every KPI on a dashboard shares **one** aggregate call — unconditional
599+
measures plain, conditional ones carrying their own `filter`. `where` scopes
600+
the whole call, so reach for it only when every measure wants the same scope:
612601

613602
```typescript
614-
// KPI dashboard: unconditional aggregations share one call
603+
// KPI dashboard: one call, conditional measures scoped per aggregation
615604
const [kpis] = await engine.aggregate('deal', {
616605
aggregations: [
617606
{ function: 'count', alias: 'total_deals' },
618607
{ function: 'sum', field: 'amount', alias: 'pipeline_value' },
619608
{ function: 'avg', field: 'amount', alias: 'avg_deal_size' },
609+
{ function: 'count', alias: 'won_deals',
610+
filter: { stage: 'closed_won' } },
620611
],
621612
});
622-
623-
// Conditional KPI: separate call, condition in `where`
624-
const [won] = await engine.aggregate('deal', {
625-
where: { stage: 'closed_won' },
626-
aggregations: [{ function: 'count', alias: 'won_deals' }],
627-
});
628613
```
629614

630615
---
@@ -636,9 +621,9 @@ code — the renderer issues the queries for you:
636621

637622
| Query Need | Pattern |
638623
|:--|:--|
639-
| KPI widgets | Aggregates (`sum`, `count`, `avg`) over the object, each conditional KPI scoped by the widget/dataset filter. Add `compareTo: 'previousPeriod' \| 'previousYear'` on the widget for a one-line period-over-period delta. |
640-
| Time-series chart | Date filters + `categoryGranularity: 'day' \| 'week' \| 'month' \| 'quarter' \| 'year'` for server-side bucketing — never bucket by hand on the client. Pair with `compareTo` for an aligned YoY overlay. |
641-
| Matrix report | `groupingsDown` + `groupingsAcross` + `dateGranularity: 'quarter'` |
624+
| KPI widgets | Aggregates (`sum`, `count`, `avg`) over the object, each conditional KPI scoped by the widget/dataset filter. Add `compareTo: { kind: 'previousPeriod' \| 'previousYear' }` on the widget for a one-line period-over-period delta (the bare string form was removed in 17). |
625+
| Time-series chart | Date filters + `dateGranularity: 'day' \| 'week' \| 'month' \| 'quarter' \| 'year'` on the widget's dataset selection for server-side bucketing — never bucket by hand on the client. Pair with `compareTo` for an aligned YoY overlay. |
626+
| Matrix report | Dataset-bound `rows` (down) + `columns` (across) + a `dateGranularity` dimension |
642627
| Funnel summary | Multi-level grouping (`owner -> stage`) + aggregated measures |
643628
| Operational filter | Prefer declarative operators (`$ne`, `$nin`, `$gte`) over hardcoded SQL |
644629

skills/objectstack-query/evals/README.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,18 +17,17 @@ subset the engine actually executes.
1717
manual keyset pagination (`where` on the sort key + `orderBy` + `limit`);
1818
fail if the answer uses the removed `cursor` property.
1919
4. **Aggregation correctness** — "Count deals by region and show total
20-
revenue." Expect `groupBy` + `count`/`sum` with aliases; on SQL targets
21-
the answer must stay within `count`/`sum`/`avg`/`min`/`max`.
22-
5. **The FILTER-WHERE trap** — "One call: total orders and count of orders
23-
over $1000." The correct answer is **two** aggregate calls with the
24-
condition in `where`; fail if the answer puts `filter` on an aggregation
25-
(silently returns the unfiltered number).
20+
revenue." Expect `groupBy` + `count`/`sum` with aliases; fail on a missing
21+
`alias`, or on `array_agg`/`string_agg` (removed in protocol 17).
22+
5. **Filtered aggregation** — "One call: total orders and count of orders
23+
over $1000." Expect one call with a per-aggregation `filter` on the
24+
conditional measure; fail on `where`, which scopes every measure.
2625
6. **Post-aggregation filtering** — "Customers with more than 5 orders."
27-
Expect aggregate + app-code filter of the group rows; fail on `having`
28-
(schema-reserved, silently dropped).
26+
Expect `having` on the aggregation alias; fail on `where`, which filters
27+
input rows before the alias exists.
2928
7. **Date-bucketed time series** — "Monthly revenue for the last year."
3029
Expect structured `groupBy` with `dateGranularity: 'month'`, not
31-
client-side bucketing and not window functions (schema-reserved).
30+
client-side bucketing and not `windowFunctions` (removed in protocol 17).
3231
8. **Expand vs direct query** — "Show a task list with assignee names; page
3332
through one project's tasks." Expect `expand` for the lookup display and
3433
a direct query on the related object for pagination (nested

skills/objectstack-query/rules/aggregation.md

Lines changed: 23 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,11 @@ Guide for building ObjectStack aggregation queries.
1313
| `max` | `MAX(field)` | Maximum value | Yes |
1414
| `count_distinct` | `COUNT(DISTINCT field)` | Count unique values | Yes |
1515

16-
> ⚠️ **Driver support varies.** On SQL datasources the driver executes only
17-
> `count` / `sum` / `avg` / `min` / `max` and **throws** (`Unsupported
18-
> aggregate function`) on `count_distinct`; the per-aggregation
19-
> `distinct: true` flag is also ignored there. The in-memory aggregation path
20-
> (driver-rest, driver-memory, timezone/date-bucket fallbacks) supports all six
21-
> functions plus `distinct`. For portable queries, stick to the first five.
16+
> **All six are portable.** `count_distinct` lowers to `COUNT(DISTINCT x)`
17+
> on `driver-sql` and turso's remote transport, and `driver-mongodb` /
18+
> `driver-memory` compute it too, so the declared-but-uncompiled set is empty.
19+
> The per-aggregation `distinct: true` flag went the other way — **removed in
20+
> 17**, refused at parse. For a deduplicated count, use `count_distinct`.
2221
2322
> **Removed in 17.** `array_agg` and `string_agg` are no longer part of
2423
> the vocabulary — they were declared and lowered by no SQL backend, so a query
@@ -118,53 +117,40 @@ use `where` to shrink the scan, `having` to threshold the aggregates.
118117

119118
## Filtered Aggregation (FILTER WHERE)
120119

121-
> ⚠️ **Per-aggregation `filter` is schema-reserved — NOT executed by the
122-
> engine yet.** The SQL driver never reads it and the in-memory path ignores
123-
> it, so the aggregation returns the **unfiltered** numbersilently wrong
124-
> results. **Working alternative:** one aggregate call per condition, with
125-
> the condition in the query-level `where`:
120+
> **Enforced.** A per-aggregation `filter` scopes that one measure, so a
121+
> total and a conditional count share ONE call. Any aggregation carrying a
122+
> non-empty `filter` forces the in-memory pathno driver compiles a
123+
> conditional aggregate, and one reached directly refuses `NOT_IMPLEMENTED`;
124+
> unfiltered aggregations keep native push-down.
126125
127126
```typescript
128-
// ❌ filter on the aggregation is silently ignored — active_count
129-
// would equal total!
130-
// { function: 'count', alias: 'active_count', filter: { status: 'active' } }
131-
132-
// ✅ Separate aggregate calls, condition in `where`
133-
const [totals] = await engine.aggregate('user', {
134-
aggregations: [{ function: 'count', alias: 'total' }],
135-
});
136-
const [active] = await engine.aggregate('user', {
137-
where: { status: 'active' },
138-
aggregations: [{ function: 'count', alias: 'active_count' }],
127+
// ✅ Total and conditional count in one call
128+
const [row] = await engine.aggregate('user', {
129+
aggregations: [
130+
{ function: 'count', alias: 'total' },
131+
{ function: 'count', alias: 'active_count', filter: { status: 'active' } },
132+
],
139133
});
134+
// An unknown operator inside `filter` refuses INVALID_FILTER/400 — it never
135+
// silently answers the unfiltered number.
140136
```
141137

142138
## DISTINCT Aggregation
143139

144-
> ⚠️ **Not available on SQL datasources.** `count_distinct` **throws** on the
145-
> SQL driver, and the `distinct: true` flag is silently ignored there (see
146-
> the driver-support caveat above). Both forms work only on the in-memory
147-
> aggregation path. On SQL, get a distinct count by grouping on the field
148-
> and counting the result rows in app code:
149-
> `(await engine.aggregate('employee', { groupBy: ['department'], aggregations: [{ function: 'count', alias: 'n' }] })).length`.
140+
> **`count_distinct` runs everywhere**`COUNT(DISTINCT field)` on the SQL
141+
> faces, the same answer in memory. `field` is REQUIRED; there is no
142+
> `COUNT(DISTINCT *)`. The per-aggregation `distinct: true` flag is NOT its
143+
> equivalent: **removed in 17**, refused at parse, because exactly one of the
144+
> six backends that read an aggregation ever honoured it.
150145
151146
```typescript
152-
// In-memory drivers only:
153147
// SQL: SELECT COUNT(DISTINCT department) FROM employee
154148
{
155149
object: 'employee',
156150
aggregations: [
157151
{ function: 'count_distinct', field: 'department', alias: 'dept_count' }
158152
]
159153
}
160-
161-
// Alternative (also in-memory only): use distinct flag
162-
{
163-
object: 'employee',
164-
aggregations: [
165-
{ function: 'count', field: 'department', alias: 'dept_count', distinct: true }
166-
]
167-
}
168154
```
169155

170156
## Window Functions

skills/objectstack-query/rules/filters.md

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -99,23 +99,20 @@ where: {
9999

100100
## Field References
101101

102-
> ⚠️ **`$field` is schema-reserved — NOT executed by the engine yet.** It
103-
> exists only in the filter schema; no engine or driver code interprets it,
104-
> so the `{ $field: '...' }` object binds as a **literal value** and the
105-
> query silently returns zero rows.
102+
> **Enforced.** `{ $field: '...' }` compares two columns of the same row.
103+
> The in-memory evaluator resolves the reference against the record; the SQL
104+
> driver pushes it down as a column-to-column predicate. Same rows either way.
106105
107106
```typescript
108-
// ❌ Schema-valid but NOT executed — matches nothing
107+
// ✅ Projects that ran over budget
109108
where: {
110109
actual_cost: { $gt: { $field: 'budget' } }
111110
}
112111
```
113112

114-
**Working alternatives:**
115-
- Define a **formula field** that computes the cross-field comparison
116-
(e.g. a boolean `over_budget`), then filter on it:
117-
`where: { over_budget: true }` (see **objectstack-data**).
118-
- Fetch both fields and compare in **application code**.
113+
Legal in a **comparison** position only. As an `$in` / `$nin` member or a
114+
`$between` endpoint it is refused at parse — no evaluation path resolves a
115+
reference there.
119116

120117
## Nested Relation Filters
121118

@@ -188,15 +185,16 @@ where: {
188185
}
189186
```
190187

191-
### ❌ Wrong: Null check with equality
188+
### ⚠️ Prefer `$null` to a bare `null` comparand
192189

193190
```typescript
194-
// ❌ Don't use equality to check for null
191+
// ⚠️ Works — a bare null lowers to IS NULL on both paths — but it reads
192+
// as "equals null" and has no IS NOT NULL spelling
195193
where: {
196194
deleted_at: null
197195
}
198196

199-
//Use $null operator
197+
//Explicit, and `$null: false` is IS NOT NULL
200198
where: {
201199
deleted_at: { $null: true }
202200
}

skills/objectstack-query/rules/pagination.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -215,14 +215,13 @@ When building paginated REST endpoints:
215215

216216
## DISTINCT Queries
217217

218-
> ⚠️ **The top-level `distinct: true` flag is schema-reserved — NOT executed
219-
> by the engine yet.** Neither the engine nor the SQL driver reads it from a
220-
> `QueryAST`; the query returns duplicate rows as if the flag were absent.
221-
> **Working alternative:** group by the fields — each unique combination
222-
> becomes one result row:
218+
> **`query.distinct` was REMOVED in `@objectstack/spec` 17.** No driver ever
219+
> rendered `SELECT DISTINCT`. The key is tombstoned — a query carrying it fails
220+
> to parse with the prescription — and `QueryBuilder.distinct()` is gone. Group
221+
> by the fields instead: each unique combination becomes one result row.
223222
224223
```typescript
225-
//distinct is silently ignored
224+
//tombstoned — this query is refused at parse
226225
// { object: 'order', fields: ['customer_id', 'product_category'], distinct: true }
227226

228227
// ✅ groupBy collapses duplicates

0 commit comments

Comments
 (0)