feat: support newer provider plugin-protocol features via targetVersions (RFC-04)

[so0k]

Implements RFC-04 — provider feature availability: support the newer provider plugin-protocol capability families with targetVersions-aware codegen and synth-time validation. Proposal, dataset, sweep tooling and interactive report live in open-constructs/cdktn-planning → RFCS/04-provider-feature-availability/ (only the merged matrix is vendored here, matching the function-availability split).

Builds directly on the #269 foundation (targetVersions + ValidateFeatureTargetSupport) and the #268 usage-registry pattern.

Guiding decision

Generate the full surface the schema offers; narrow per project at synth time via targetVersions. Generation-time filtering would fork the generated API by project configuration — impossible for prebuilt providers and hostile to caching. Constraints baked into the validations are the per-product >= ranges from the sweep dataset.

Review guide (one commit per RFC rollout item)

1. chore: vendor provider-feature availability matrix — dataset digest + README only, no behavior change.
2. feat(lib): TerraformEphemeralResource — new public base class synthesizing to the top-level ephemeral key (JSON + HCL renderer), refs as ephemeral.<type>.<id>.<attr>, no provisioners/connection/import/move. Constructor registers the target-version validation unconditionally (new API surface — only fires on use, no feature flag per RFC). Internal providerFeatureConstraints map sourced from the vendored matrix.
3. feat(provider-generator): acquire newer provider-protocol schema sections (Phase 0, a bugfix on its own) — commons types for functions / ephemeral_resource_schemas / resource_identity_schemas / write_only; sanitizer walks ephemeral schemas; fetched schemas stamped with the fetching CLI; cache key now includes CLI product+minor (a schema fetched once with an old CLI no longer poisons every later generation); fetch-time warning when the fetching binary structurally cannot emit sections the targets admit.
4. feat(provider-generator): generate ephemeral resource bindings — third schema family mirroring the data_ pipeline (EphemeralRandomPassword in ephemeral-random-password/), config extends TerraformEphemeralMetaArguments, no generateConfigForImport. Zero churn in existing snapshots (ephemeral models append after resources/data sources; class-name dedup is order-dependent).
5. feat(provider-generator): generate provider-defined function bindings — TimeProviderFunctions.rfc3339Parse(ts) → ${provider::time::rfc3339_parse(...)}; namespace defaults to the registry short name with a providerLocalName override (local names change the namespace, aliases don't). Runtime flows through one public jsii chokepoint (TerraformProviderFunction.invoke) feeding a usage registry; TerraformStack validates usage against terraform >=1.8.0 / opentofu >=1.7.0 — note the deliberate asymmetry: OpenTofu language support (1.7.0) predates its schema emission (1.8.0), so generation and validation use different boundaries.
6. feat(provider-generator): deprecate write-only attribute getters, validate usage — providers never persist write-only values (every read is null by protocol contract), so the state-backed getter is a trap: emitted @deprecated now, removal rides the next prebuilt major (JSII-breaking otherwise). Setting one (setter or constructor config) registers usage via a new protected TerraformResource.registerProviderFeatureUsage hook — generated code reaches the validation machinery only by extending base classes.
7. feat(cli): thread targetVersions into cdktn get — GetOptions.targetVersions → ConstructsMaker → readSchema, driving the Phase 0 fetch-time warning; constraints.json gains diagnostic stamps (targetVersions, fetching cli) without affecting the filterAlreadyGenerated staleness logic (targets don't change codegen output, so they must not force regeneration).

Test coverage

• Generator: snapshot tests against real terraform providers schema -json fragments from the sweep (random ephemeral, time functions incl. object returns, vault *_wo), plus a synthetic fixture for variadic/reserved-name mapping branches. Zero churn in pre-existing snapshots.
• Core: validation matrices mirroring validations.test.ts (admit/exclude per product, hint text, the OpenTofu 1.7.0 asymmetry, registry resets).
• Schema: emission-gap logic matrix, cache-key suffix behavior, sanitizer walk over ephemeral schemas; CLI stamps in network snapshots are stubbed (environment-dependent).

Known/deferred

• matchers.test.ts › toPlanSuccessfully fails in this environment before and after these changes (downloads the real docker provider; verified pre-existing via stash on the unmodified tree).
• Deferred per RFC Phase 4: resource identity consumption (schema plumbing already carried), list resources / actions / state stores (Terraform-only today; re-evaluate on Support 'query' CLI and provider 'list' resources as in Terraform 1.14 opentofu/opentofu#3787).
• Follow-ups tracked, not built: hcl2cdk conversion of ephemeral {} blocks; ephemeral entry in the edge-provider schema (cross-language compile coverage); a typescript/synth-app-style integration test gated on CI Terraform >= 1.10 (repo pins 1.7.5 via mise).

🤖 Generated with Claude Code

Review findings addressed (from the demo-harness verification comment)

• Fixed — targetVersions validation bypass: CdktfConfig.targetVersions (the runGetInDir path) now validates via commons validateTargetVersions (warn + ignore), and the emission-gap check treats invalid ranges as not-wanted instead of throwing — a malformed range previously crashed cdktn get via semver.intersects.
• Fixed — ephemeral lifecycle narrowed: new TerraformEphemeralResourceLifecycle (precondition/postcondition only) replaces the full managed-resource lifecycle on the ephemeral API, before it ships as jsii surface.
• Fixed (docs) — provider-fn self-reference cycle: generated provider-function JSDoc and TerraformProviderFunction.invoke now warn against calling a provider's functions inside that same provider's configuration block.
• Documented — ephemeral × write-only: registration deliberately skips ephemeral resources — write-only is a state concept, ephemeral resources have no state, and no schema in the sweep (incl. vault's 16 ephemeral resources) combines the two.
• Follow-up — nested write-only registration: write_only inside nested blocks gets the deprecated getter but skips usage registration; deep config scanning deserves its own PR (a miss degrades to the plan-time error, not silent breakage).
• Pre-existing / by design: the AWS barrel-import OOM predates this PR (lazy-index is the existing mitigation); Fn.ephemeralasnull + sensitive on outputs is Terraform semantics (docs candidate); list resources / actions / resource identity codegen is the RFC Phase 4 deferral noted above.

[sakul-learning]

Draft PR #296 review notes / verification results

I built and published a small reusable demo harness against this PR head so the new provider-protocol features can be exercised with real generated AWS bindings:

https://github.com/sakul-learning/cdktn-provider-features-demo

PR head used for the run: 78299b4b9ece16c45ab26a0cd28dec08e8ba57e9.

Verification performed

Using the PR-head packages directly, not npmjs:

• pnpm run assert:prhead-deps confirms cdktn resolves from the PR-head worktree and the CLI comes from the PR-head cdktn-cli bundle.
• Terraform 1.15.7:
  • captured AWS provider schema
  • generated AWS bindings
  • ran schema inspection and generated-feature assertions
  • ran pnpm exec tsc --noEmit
  • ran pnpm run synth:all
  • ran pnpm run plan:all
• OpenTofu 1.12.3:
  • reused the Terraform-generated bindings
  • ran generated-feature assertions
  • ran pnpm exec tsc --noEmit
  • ran pnpm run synth:all
  • ran pnpm run plan:all

The AWS provider schema from Terraform 1.15.7 exposed:

• provider functions: 4
• ephemeral resources: 10
• list resources: 153
• provider actions: 11
• resource identities: 437

Generated bindings confirmed:

• provider functions: yes
• ephemeral resources: yes
• list resources: no first-class generated bindings observed
• provider actions: no first-class generated bindings observed
• resource identities: no first-class generated bindings observed

OpenTofu 1.12.3 successfully planned the examples with the Terraform-generated bindings. A separate OpenTofu schema probe exposed provider functions, ephemeral resources, and resource identities, but reported list_resource_schemas: 0 and action_schemas: 0, so Terraform 1.15.7 is the better generator binary for this comprehensive AWS capability test.

UX / integration notes from the demo

These are non-blocking, but important feedback from trying to use the generated bindings in a real app:

• I switched the examples from the AWS barrel import (./.gen/providers/aws) to direct generated-module imports (provider, provider-functions, and the specific ephemeral resource module). The barrel import path pulled in thousands of generated AWS classes and hit the Node/TypeScript OOM path during synth.
• After that import change, I had to replace the remaining namespace-style references (provider.*, providerFunctions.*, and ephemeral...*) with direct class references like AwsProvider, AwsProviderFunctions, and EphemeralAwsSecretsmanagerRandomPassword.
• The ephemeral output needs both Fn.ephemeralasnull(...) and sensitive: true so Terraform and OpenTofu accept the plan. Without ephemeralasnull, the ephemeral value is used in a non-ephemeral context; without sensitive, the output still fails because it is derived from secret ephemeral data.
• I removed a trial use of AwsProviderFunctions.userAgent(...) inside AwsProvider configuration after OpenTofu reported a provider self-reference/cycle. The mistaken assumption was that provider-defined functions behave like pure static helpers that are safe to use while configuring that same provider. The generated call is still a provider-namespaced function (provider::aws::user_agent), so putting it inside the aws provider block asks Terraform/OpenTofu to evaluate an AWS provider function while the AWS provider is still being configured. The demo now exercises provider functions in outputs instead, after provider configuration exists.

Non-blocking findings / suggested follow-ups

• targetVersions validation bypass in cdktn get path
  targetVersions is validated in parseConfig (packages/@cdktn/commons/src/config.ts), but cdktn get appears to use CdktfConfig.read() plus the raw targetVersions getter (packages/@cdktn/cli-core/src/lib/cdktf-config.ts, then passed through in packages/@cdktn/cli-core/src/lib/get.ts). That path appears to bypass the parseConfig validator, so malformed targetVersions can reach generation.

• Ephemeral resources expose too much lifecycle surface
  TerraformEphemeralMetaArguments currently exposes lifecycle?: TerraformResourceLifecycle. Terraform ephemeral blocks only support lifecycle precondition / postcondition; they do not support the full managed-resource lifecycle surface such as ignore_changes, replace_triggered_by, create_before_destroy, etc. I’d introduce a narrowed lifecycle type for ephemeral resources, e.g. only precondition and postcondition, and use that in TerraformEphemeralMetaArguments.

• Nested write-only attributes do not register feature usage
  The constructor-time write-only registration only iterates top-level assignable attributes in resource.configStruct.assignableAttributes. Provider schemas can put write_only inside nested blocks, not only on top-level resource config attributes, so nested write-only usage can skip registerProviderFeatureUsage("writeOnlyAttributes").

• Generated ephemeral resources skip write-only feature usage registration
  The generator currently limits write-only registration to classes whose parent is TerraformResource; generated ephemeral resources do not get that registration path. Maybe current provider schemas do not combine ephemeral resources with write-only attributes, but since this PR is adding generalized newer-protocol support, I’d either cover the ephemeral path or explicitly document/filter why that combination cannot happen.

Overall: the core provider-function and ephemeral-resource generated bindings worked in real synth/plan flows for Terraform and OpenTofu once the UX issues above were accounted for. The largest current generated-coverage gap is that Terraform exposes AWS list resources, actions, and resource identities, but I did not see first-class generated bindings for those schema sections yet.

[vincenthsh]

not sure if useful to have this in every JSDocs - it's a very limited edge case