Add offline SQL generation for Master schema attribute indexes

CLAUDE.md

@@ -17,7 +17,7 @@ node bin/workflow.js   # run the CLI directly (same as `npm run dev` / `npm star
 ```
 
 - **`build` is a no-op** (`echo 'Build not needed for now'`) — nothing to compile.
-- **There is no test suite, linter, or formatter configured.** Do not invent `npm test`/`npm run lint` commands; they will fail. Verify changes by running the CLI against a real vNext project directory.
+- **Index SQL tests**: `node --test test/indexes.test.js` (Node 18+); real PostgreSQL tests use `VNEXT_INDEX_TEST_URL=... node --test test/indexes.postgres.test.js` against a disposable database. No npm test/lint script is configured. Also verify the CLI against a real vNext project directory.
 
 The CLI always treats `process.cwd()` as the project root and requires at least one solution file (`vnext.config.json` or `vnext.{domain}.config.json`) in that directory. To exercise it, `cd` into a vNext workspace (not this repo) before running. A two-solution testbed lives at `../vnext-example` (`core` + `partner`); copy it into a scratch directory before running write-mode commands against it.
 
@@ -63,3 +63,13 @@ Every workspace command runs the table above **once per solution, sequentially**
 - Library functions (`discover.js`, `workflow.js`, `csx.js`) take a **solution object** (`projectRoot`, `componentsRoot`, `componentTypes`, …) rather than a bare `projectRoot` or reading cwd directly; commands receive it from `runForEachSolution`. Only `config.get('PROJECT_ROOT')` (inside `solutions.loadWorkspace`) reads cwd.
 - Commands are split into a thin `xxxCommand(options)` (header, prompts that must happen once, then `runForEachSolution`) and an `xxxSolution(solution, options)` body that returns `{ success, failed, errors }` so the workspace summary can aggregate.
 - DB and API helpers swallow connection errors and return `false`/`null` rather than throwing — callers treat a missing instance as "not in DB".
+
+## Manual attribute-index SQL
+
+`wf indexes generate` is strictly offline: `src/lib/indexes/` resolves local Master
+references and emits SQL; `src/commands/indexes.js` writes immutable batches. Never add DB execution
+or automatic sync/publish hooks. DBA execution owns the maintenance window. Keep physical keys/columns
+compatible with runtime `AttributeIndexDefinition` (`v1:latest` SHA-256 first 24 hex); version matching
+follows runtime `InstanceDataVersionComparer`, including package revisions. SQL compares actual index
+structure, skips equivalent indexes and only rebuilds/removes owned indexes. Projection retirement
+requires explicit `--retire-obsolete` and a complete local inventory of active workflow versions.

README.md

@@ -171,6 +171,74 @@ wf reset
 
 All workspace commands (`check`, `csx`, `sync`, `update`, `reset`) accept the global `--domain <name>` option. Without it they run once per solution file found in the project root.
 
+### `wf indexes generate`
+
+**Purpose**: Generate offline attribute-index SQL for planned execution by your DBA team.
+
+From a domain workspace containing `vnext.config.json`:
+
+```bash
+wf indexes generate --output ./index-sql
+wf indexes generate --flow money-transfer --output ./index-sql
+```
+
+Each invocation writes a **new batch folder** containing one `.sql` per workflow, a `manifest.json`
+(source paths, versions, SHA-256 checksums, physical index definitions), and execution notes. Earlier
+batches are preserved. The command never connects to API/DB and never executes SQL; `sync`, `update`
+and Master publication do not invoke it.
+
+The generator resolves local workflow → Master references from the configured component directories.
+All local versions of a workflow contribute requirements. Full `-pkg.` revisions remain pinned; `latest`,
+artifact, minor and major selectors use runtime version ordering. Build metadata is ignored. Referenced
+schemas must exist locally; incomplete references, duplicate versions and incompatible types fail.
+
+Indexed fields use `x-indexed: true`. Nested scalar string/number/integer/boolean fields are supported;
+dates use `format: "date-time"`. Filtering/sorting permissions remain `x-filterOperators`/`x-sortable`.
+The generator preserves the runtime `v1:latest` column/key contract; readable index names include a hash
+of their physical definition.
+
+The DBA reviews and executes each SQL file in a maintenance window:
+
+```bash
+psql -X -v ON_ERROR_STOP=1 --dbname=vNext_MyDomainDb --file=index-sql/<batch>/money_transfer.sql
+```
+
+SQL takes an advisory lock plus an **ACCESS EXCLUSIVE table lock**.
+It validates current/history data, adds stored generated columns in one table rewrite, reuses structurally
+matching indexes (including legacy names), rebuilds changed owned indexes, and updates the runtime ready
+catalog atomically. Index comparison covers keys, includes, ordering, collation, access method, operator
+classes and partial predicates. Unmanaged collisions abort; no `CASCADE` is issued. Replaying an unchanged
+batch preserves index OIDs and skips data validation, rewrites and ANALYZE. Missing/invalid values are not
+silently coerced; conversion errors identify the field and roll back all changes.
+
+This is **not concurrent DDL**. Plan disk/WAL/replica capacity for generated-column rewrites; the script's
+5-second lock timeout may be adjusted during DBA review. Text-search indexes require `pg_trgm` in `public`
+and `tr-TR-x-icu`. Runtime routing uses `AttributeIndexes:Enabled`; catalog refresh defaults to 30 seconds.
+
+Obsolete projections are retained by default for other deployed versions. To explicitly retire them:
+
+```bash
+wf indexes generate --flow money-transfer --retire-obsolete
+```
+
+Include **every still-active workflow version** locally and drain readers/writers for this maintenance.
+Retirement removes obsolete expressions/owned indexes and marks catalog entries not ready; columns and
+stored values remain. This is required when an old numeric/date cast would reject values after a type
+change. The offline tool cannot verify deployed versions: compare the source manifest before execution.
+Roll back query routing through the runtime's `AttributeIndexes:DisabledFlows`; schedule physical cleanup
+separately. See the runtime [maintenance runbook](https://github.com/burgan-tech/vnext/blob/main/docs/runtime/manual-attribute-index-maintenance.md).
+
+**Tests (Node 18+ test runner):**
+
+```bash
+node --test test/indexes.test.js
+# Use only a disposable PostgreSQL database; the test owns uniquely named fixture schemas.
+VNEXT_INDEX_TEST_URL=postgresql://user:password@localhost:5432/disposable_test \
+  node --test test/indexes.postgres.test.js
+```
+
+---
+
 ### `wf check`
 
 **Purpose**: System health check
@@ -820,3 +888,36 @@ npm run dev
 ## 📝 License
 
 MIT License - see [LICENSE](LICENSE) for details.
+
+### Schema purpose for index generation
+
+Schema components use the existing `attributes.type` string. Its values are not restricted to an enum;
+legacy and custom types remain valid. Only the exact value `master` permits `x-indexed` metadata
+(including `false`) and contributes index SQL. Missing, null, blank or other values never mean master.
+The existing publication requirement for a non-empty schema type is unchanged. JSON Schema `type`
+keywords within `attributes.schema` retain their existing meaning.
+A latest reference resolving to a non-master schema does not fall back to an older master version.
+No matching masters means no SQL batch. There is no separate component root `type` field.
+
+```json
+{
+  "key": "order-master",
+  "domain": "sales",
+  "flow": "sys-schemas",
+  "version": "1.0.0",
+  "flowVersion": "1.0.0",
+  "tags": ["orders"],
+  "attributes": {
+    "type": "master",
+    "schema": {
+      "type": "object",
+      "properties": {
+        "amount": { "type": "number", "x-indexed": true, "x-filterOperators": ["gt"] }
+      }
+    }
+  }
+}
+```
+
+The schema validator reads `SchemaDefinition.Type` (`attributes.type`) during publication, including
+schema seed items. No extra envelope metadata is required.

bin/workflow.js

@@ -82,6 +82,15 @@ program
   .description('Reset workflows (force update)')
   .action(resetCommand);
 
+// Offline index maintenance artifacts; execution is owned by the DBA team.
+program.command('indexes').description('Generate DBA-reviewed attribute index SQL')
+  .command('generate')
+  .description('Read local workflow/Master definitions and write SQL files without contacting API/DB')
+  .option('--flow <key>', 'Generate for one workflow key (all its local versions)')
+  .option('-o, --output <directory>', 'Parent folder for a new immutable SQL batch', 'index-sql')
+  .option('--retire-obsolete', 'Retire obsolete projections; requires ALL active workflow versions locally')
+  .action(require('../src/commands/indexes'));
+
 // Config command
 program
   .command('config')

src/commands/indexes.js

@@ -0,0 +1,46 @@
+const fs = require('fs');
+const path = require('path');
+const { LOG } = require('../lib/ui');
+const { loadPlans, hash } = require('../lib/indexes/definitions');
+const { generateSql, indexes } = require('../lib/indexes/sql');
+
+async function generate(options, projectRoot = process.cwd()) {
+  const plans = await loadPlans(projectRoot, options.flow);
+  if (!plans.length) throw new Error("No workflows referencing attributes.type: master schemas found; no SQL files generated.");
+  // Validate/render the whole batch before creating any output; never overwrite an earlier batch.
+  const rendered = plans.map(plan => ({ plan, sql: generateSql(plan, options) }));
+  const output = path.resolve(projectRoot, options.output || 'index-sql');
+  fs.mkdirSync(output, { recursive: true });
+  const batch = fs.mkdtempSync(path.join(output, new Date().toISOString().replace(/[:.]/g, '-') + '-'));
+  const manifest = { formatVersion: 1, physicalContract: 'v1:latest', generatorVersion: require('../../package.json').version,
+    domain: plans[0].domain, retireObsolete: !!options.retireObsolete,
+    generatedAt: new Date().toISOString(), flows: [] };
+  for (const { plan, sql } of rendered) {
+    const file = plan.schema + '.sql';
+    fs.writeFileSync(path.join(batch, file), sql, { flag: 'wx' });
+    manifest.flows.push({ ...plan, projections: plan.projections.map(p => ({ ...p, indexes: indexes(p) })), file, sha256: hash(sql) });
+  }
+  fs.writeFileSync(path.join(batch, 'manifest.json'), JSON.stringify(manifest, null, 2) + '\n', { flag: 'wx' });
+  fs.writeFileSync(path.join(batch, 'README.txt'), [
+    `Domain: ${manifest.domain}. Generated offline; nothing was executed.`,
+    'DBA: review manifest/source versions and each SQL file. Execute files individually in a maintenance window.',
+    'Example: psql -X -v ON_ERROR_STOP=1 --dbname=<target-database> --file=<flow.sql>',
+    'Scripts lock InstancesData ACCESS EXCLUSIVE and use one transaction per flow. Plan disk/WAL/replica capacity for table rewrites.',
+    'A 5s lock_timeout is included; adjust it during DBA review if necessary. Re-run the SAME script after rollback on failure.',
+    'Existing correct indexes are adopted; changed CLI-owned indexes are rebuilt. Unmanaged name collisions fail.',
+    'No obsolete projection is retired unless --retire-obsolete was explicitly passed. Retirement requires all active versions locally and drained runtime readers/writers.',
+    'Stored values/columns are retained on retirement; physical cleanup remains a separate DBA operation.',
+    'Runtime reads use AttributeIndexes:Enabled and refresh the catalog after CatalogCacheSeconds (default 30).',
+    'Rollback routing with AttributeIndexes:DisabledFlows. Do not drop columns under running readers.', ''
+  ].join('\n'), { flag: 'wx' });
+  return { batch, count: rendered.length };
+}
+async function command(options) {
+  try {
+    const { batch, count } = await generate(options);
+    LOG.success(`Generated ${count} SQL file(s): ${batch}`);
+    LOG.info('No API or database connection was made. Give this batch to your DBA for review and execution.');
+  } catch (error) { LOG.error(error.message); process.exitCode = 1; }
+}
+module.exports = command;
+module.exports.generate = generate;