From 6fc6b464750941740a10d7e1b6ce2dab2e557ad8 Mon Sep 17 00:00:00 2001 From: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> Date: Thu, 3 Sep 2026 08:42:22 -0400 Subject: [PATCH 1/4] Opus analysis, contemporary with shared-context Signed-off-by: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> --- context/local/advice/platform-support.md | 15 +++-- context/local/advice/transport-development.md | 30 ++++----- context/local/standards/testing.md | 64 ++++++++++++++++++- 3 files changed, 89 insertions(+), 20 deletions(-) diff --git a/context/local/advice/platform-support.md b/context/local/advice/platform-support.md index 2b696114..0a3fb38b 100644 --- a/context/local/advice/platform-support.md +++ b/context/local/advice/platform-support.md @@ -1,6 +1,13 @@ #### Platform Support -- Consider cross-platform compatibility -- Test on multiple operating systems when applicable -- Use Train's platform detection system -- Handle platform-specific edge cases + +> **Moved.** The canonical, in-depth guide to extending Train's platform / family +> detection now lives in the shared, cross-repo product area: +> +> - **Extending platform support (worked examples):** +> [`context/shared/by-product/train/advice/platform-support.md`](../../shared/by-product/train/advice/platform-support.md) +> - **Platform detection design:** +> [`context/shared/by-product/train/design/platform-detection.md`](../../shared/by-product/train/design/platform-detection.md) +> +> This file is retained only as a pointer so existing references keep working. + diff --git a/context/local/advice/transport-development.md b/context/local/advice/transport-development.md index e8a0ee4b..74159b27 100644 --- a/context/local/advice/transport-development.md +++ b/context/local/advice/transport-development.md @@ -1,17 +1,17 @@ +### Transport Development -#### Transport Development Guidelines -- Inherit from `Train::Plugins::Transport` -- Implement required methods: `connection`, `options` -- Use Train's connection management patterns -- Handle platform-specific requirements -- Provide appropriate error handling -- Support Train's audit logging when applicable +> **Moved.** The canonical, in-depth guidance for authoring a Train transport / +> plugin now lives in the shared, cross-repo product area (it is integration +> guidance consumed by plugin authors in separate repos): +> +> - **Transport development reference & checklist:** +> [`context/shared/by-product/train/advice/transport-development.md`](../../shared/by-product/train/advice/transport-development.md) +> - **End-to-end plugin authoring hub:** +> [`context/shared/by-product/train/advice/writing-a-plugin.md`](../../shared/by-product/train/advice/writing-a-plugin.md) +> - **Connection contract:** +> [`context/shared/by-product/train/interfaces/connection-api.md`](../../shared/by-product/train/interfaces/connection-api.md) +> - **Plugin archetypes (OS-command vs API):** +> [`context/shared/by-product/train/design/plugin-archetypes.md`](../../shared/by-product/train/design/plugin-archetypes.md) +> +> This file is retained only as a pointer so existing references keep working. - -#### Transport Development -- Follow the plugin architecture pattern -- Use Train's connection management -- Implement proper platform detection -- Handle authentication securely -- Support Train's file and command interfaces -- Provide meaningful error messages diff --git a/context/local/standards/testing.md b/context/local/standards/testing.md index b95f3528..94765458 100644 --- a/context/local/standards/testing.md +++ b/context/local/standards/testing.md @@ -40,4 +40,66 @@ describe Train::Transports::MyTransport do end end end -``` \ No newline at end of file +``` + +--- + +## Testing the Train Gem Itself + +> This section covers testing **the `inspec/train` codebase** (this repo). For +> authoring and testing a **standalone Train plugin** (in its own `train-` +> gem repo), see the shared, cross-repo guide: +> [`context/shared/by-product/train/advice/testing-plugins.md`](../../shared/by-product/train/advice/testing-plugins.md). + +### Layout & runner +- Unit tests live under `test/unit/**` and end in `_test.rb`. The default Rake + task runs them: `Rake::TestTask` with pattern `test/unit/**/*_test.rb` + (see `Rakefile`; `rake` alone runs `:test`). +- Mirror the `lib/` tree in `test/unit/` — e.g. `lib/train/file/remote/windows.rb` + is covered by `test/unit/file/remote/windows_test.rb`. Existing subdirs: + `test/unit/{extras,file,platforms,plugins,transports}`. +- Shared setup is `require "helper"` (`test/helper.rb`), which wires up Minitest, + `mocha/minitest`, and SimpleCov. +- Integration/OS-specific suites are separate and gated: + `rake test:docker`, `rake test:windows`, `rake test:ssh[user@server]`, and + Test Kitchen (`test/integration`, `test/windows`). Do **not** require live + targets for unit tests. + +### What to cover +- **Options & defaults:** that `option` declarations, `:default`/`:coerce`, and + `validate_options`/`validate_audit_log_options` behave (see `lib/train/options.rb`). +- **URI/target unpacking:** `Train.unpack_target_from_uri`, `target_config`, and + `group_keys_and_keyfiles` edge cases (empty scheme `mock://`, query params, + key vs key_file splitting). +- **Connection contract:** `run_command` arity dispatch, `file` caching, `upload`/ + `download`, and error raising (`Train::ClientError`/`TransportError`). +- **Platform detection:** add/extend specs in + `test/unit/platforms/**` when touching `lib/train/platforms/**`. + +### Prefer the Mock transport over live connections +Use `Train::Transports::Mock` to exercise connection/file/command logic without a +real backend: +```ruby +require "helper" +require "train/transports/mock" + +describe "command handling" do + let(:conn) { Train.create("mock", verbose: true).connection } + + it "returns a stubbed command result" do + conn.mock_command("hostname", "web01", "", 0) + result = conn.run_command("hostname") + _(result.stdout).must_equal "web01" + _(result.exit_status).must_equal 0 + end +end +``` +Use `mocha` stubs only for third-party SDK/network boundaries — not for Train's +own classes, which should be driven through their real interfaces. + +### Coverage & style gates +- Maintain **> 80%** coverage (SimpleCov, configured in `test/helper.rb`). +- Run **ChefStyle** before submitting: `chefstyle` (auto-fix with `chefstyle -a`). + `chefstyle` is loaded in the `Rakefile`. +- Keep unit tests deterministic and offline; anything requiring sudo, SSH, WinRM, + Docker, or a cloud/API credential belongs in the gated integration tasks. From 0671973e92329f983dfed4e917af3bdc181a5d73 Mon Sep 17 00:00:00 2001 From: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> Date: Thu, 3 Sep 2026 08:56:41 -0400 Subject: [PATCH 2/4] Architecture analysis and diagrams Signed-off-by: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> --- .../diagrams/01-component-context.puml | 74 +++++ .../diagrams/01-component-context.svg | 1 + .../diagrams/02-core-classes.puml | 88 ++++++ .../architecture/diagrams/02-core-classes.svg | 1 + .../diagrams/03-transport-hierarchy.puml | 55 ++++ .../diagrams/03-transport-hierarchy.svg | 1 + .../diagrams/04-platform-detection.puml | 82 ++++++ .../diagrams/04-platform-detection.svg | 1 + .../diagrams/05-file-hierarchy.puml | 59 ++++ .../diagrams/05-file-hierarchy.svg | 1 + .../diagrams/06-error-hierarchy.puml | 46 ++++ .../diagrams/06-error-hierarchy.svg | 1 + .../diagrams/07-seq-run-command.puml | 59 ++++ .../diagrams/07-seq-run-command.svg | 1 + .../diagrams/08-seq-platform-detect.puml | 53 ++++ .../diagrams/08-seq-platform-detect.svg | 1 + .../local/architecture/diagrams/_style.puml | 68 +++++ context/local/architecture/overview.md | 253 ++++++++++++++++++ 18 files changed, 845 insertions(+) create mode 100644 context/local/architecture/diagrams/01-component-context.puml create mode 100644 context/local/architecture/diagrams/01-component-context.svg create mode 100644 context/local/architecture/diagrams/02-core-classes.puml create mode 100644 context/local/architecture/diagrams/02-core-classes.svg create mode 100644 context/local/architecture/diagrams/03-transport-hierarchy.puml create mode 100644 context/local/architecture/diagrams/03-transport-hierarchy.svg create mode 100644 context/local/architecture/diagrams/04-platform-detection.puml create mode 100644 context/local/architecture/diagrams/04-platform-detection.svg create mode 100644 context/local/architecture/diagrams/05-file-hierarchy.puml create mode 100644 context/local/architecture/diagrams/05-file-hierarchy.svg create mode 100644 context/local/architecture/diagrams/06-error-hierarchy.puml create mode 100644 context/local/architecture/diagrams/06-error-hierarchy.svg create mode 100644 context/local/architecture/diagrams/07-seq-run-command.puml create mode 100644 context/local/architecture/diagrams/07-seq-run-command.svg create mode 100644 context/local/architecture/diagrams/08-seq-platform-detect.puml create mode 100644 context/local/architecture/diagrams/08-seq-platform-detect.svg create mode 100644 context/local/architecture/diagrams/_style.puml create mode 100644 context/local/architecture/overview.md diff --git a/context/local/architecture/diagrams/01-component-context.puml b/context/local/architecture/diagrams/01-component-context.puml new file mode 100644 index 00000000..ddcff83d --- /dev/null +++ b/context/local/architecture/diagrams/01-component-context.puml @@ -0,0 +1,74 @@ +@startuml 01-component-context +!include _style.puml +title Train — Component / Context View + +' How Train sits between its consumers and its plugins. + +package "Consumers" { + package "Chef InSpec" { + [Inspec::Backend] as InspecBackend <> + [InSpec resources\n(command, file, os, ...)] as InspecResources + [Inspec::Config\n(unpack_train_credentials)] as InspecConfig + } + package "Chef Infra Client" { + [Chef target_io\n(TargetIO::File / Dir / etc.)] as ChefTargetIO + [Chef::Application] as ChefApp + } +} + +package "Train core (train gem)" { + [Train.create /\nload_transport\n(registry)] as TrainEntry <> + [Transport\n(plugin base)] as Transport <> + [BaseConnection] as BaseConnection <> + [Options\n(Class/InstanceOptions)] as Options <> + [Platforms + Detect\n(Scanner, Specifications)] as Platforms <> + [File / Stat] as File <> + [AuditLog] as AuditLog <> + [errors] as Errors <> +} + +package "Plugins" { + package "OS-command transports" <> { + [local] as PLocal <> + [ssh] as PSsh <> + [docker / podman] as PDocker <> + [train-winrm] as PWinrm <> + [train-kubernetes] as PK8s <> + } + package "API transports" <> { + [aws / azure / gcp] as PAws <> + [train-rest] as PRest <> + [vmware] as PVmware <> + } +} + +InspecConfig --> TrainEntry : credentials hash +InspecBackend --> TrainEntry : Train.create(name, opts) +InspecBackend --> BaseConnection : holds connection +InspecResources --> BaseConnection : run_command / file / os +ChefApp --> TrainEntry : Train.create +ChefTargetIO --> BaseConnection : file / run_command\n(reads transport_options[:sudo/:user]) + +TrainEntry --> Transport : resolves & instantiates +Transport --> BaseConnection : #connection +BaseConnection --> Options : merge/validate +BaseConnection --> AuditLog : log cmd/file (if enabled) +BaseConnection --> Platforms : #platform (lazy scan) +BaseConnection --> File : #file +BaseConnection ..> Errors : raises + +Transport <|-- PLocal +Transport <|-- PSsh +Transport <|-- PDocker +Transport <|-- PWinrm +Transport <|-- PK8s +Transport <|-- PAws +Transport <|-- PRest +Transport <|-- PVmware + +note bottom of Platforms + API transports skip real detection and + call force_platform! instead of scanning. +end note + +@enduml diff --git a/context/local/architecture/diagrams/01-component-context.svg b/context/local/architecture/diagrams/01-component-context.svg new file mode 100644 index 00000000..5528c795 --- /dev/null +++ b/context/local/architecture/diagrams/01-component-context.svg @@ -0,0 +1 @@ +Train — Component / Context ViewTrain — Component / Context ViewConsumersChef InSpecChef Infra ClientTrain core (train gem)PluginsOS-command transports«Rectangle»API transports«Rectangle»«core»Inspec::BackendInSpec resources(command, file, os, ...)Inspec::Config(unpack_train_credentials)Chef target_io(TargetIO::File / Dir / etc.)Chef::Application«core»Train.create /load_transport(registry)«core»Transport(plugin base)«abstract»BaseConnection«core»Options(Class/InstanceOptions)«core»Platforms + Detect(Scanner, Specifications)«core»File / Stat«core»AuditLog«core»errors«oscmd»local«oscmd»ssh«oscmd»docker / podman«oscmd»train-winrm«oscmd»train-kubernetes«api»aws / azure / gcp«api»train-rest«api»vmwareAPI transports skip real detection andcall force_platform! instead of scanning.credentials hashTrain.create(name, opts)holds connectionrun_command / file / osTrain.createfile / run_command(reads transport_options[:sudo/:user])resolves & instantiatesconnectionmerge/validatelog cmd/file (if enabled)platform (lazy scan)fileraises \ No newline at end of file diff --git a/context/local/architecture/diagrams/02-core-classes.puml b/context/local/architecture/diagrams/02-core-classes.puml new file mode 100644 index 00000000..7fbc5a92 --- /dev/null +++ b/context/local/architecture/diagrams/02-core-classes.puml @@ -0,0 +1,88 @@ +@startuml 02-core-classes +!include _style.puml +title Train — Core Classes + +top to bottom direction + +class "Train" as Train <> { + +self.create(name, *args) : Transport + +self.options(name) : Hash + +self.load_transport(name) : Class + +self.target_config(config) : Hash + +self.unpack_target_from_uri(uri, opts) : Hash + +self.validate_backend(creds, default="local") : String +} + +class "Plugins" as Plugins <> { + +self.registry : Hash + +self.plugin(version=1) : Class +} + +class "Plugins::Transport" as Transport <> { + +initialize(options={}) + +connection(_options=nil) : BaseConnection {abstract} + +self.name(name) + -- + +self.option(name, conf, &blk) + +self.default_options : Hash + +validate_options(opts) +} + +abstract class "BaseConnection" as BaseConnection <> { + +initialize(options=nil) + +run_command(cmd, opts={}, &blk) : CommandResult + +file(path, *args) : File + +platform() : Platform + +upload(locals, remote) + +download(remotes, local) + +force_platform!(name, details=nil) : Platform + +enable_cache(type) / disable_cache(type) + +close() / wait_until_ready() / login_command() + -- + #run_command_via_connection(cmd, [opts], &blk) {abstract} + #file_via_connection(path, *args) {abstract} + -- + -@cache_enabled : {file:true, command:false, api_call:false} + -@audit_log : AuditLog +} + +class "Options" as Options <> { + ClassOptions: option / default_options / include_options + InstanceOptions: merge_options / validate_options +} + +class "CommandResult" as CommandResult <> { + +stdout + +stderr + +exit_status +} + +class "File" as File <> { + DATA_FIELDS = exist? mode owner group + uid gid content mtime size selinux_label path + +to_json / type / directory? / symlink? + +md5sum / sha256sum +} + +class "AuditLog" as AuditLog <> { + +self.create(opts) : Logger + +info(entry) +} + +Train --> Plugins : registry lookup +Plugins --> Transport : plugin(1) base +Transport ..> Options : Options.attach(self) +Transport --> BaseConnection : #connection returns +BaseConnection ..> Options : include InstanceOptions +BaseConnection --> CommandResult : run_command returns +BaseConnection --> File : file returns +BaseConnection --> AuditLog : logs when enabled + +note right of BaseConnection + run_command dispatches on the + arity of run_command_via_connection + (1 vs 2) for plugin back-compat. + Caching: file=on, command/api_call=off. +end note + +@enduml diff --git a/context/local/architecture/diagrams/02-core-classes.svg b/context/local/architecture/diagrams/02-core-classes.svg new file mode 100644 index 00000000..406ad077 --- /dev/null +++ b/context/local/architecture/diagrams/02-core-classes.svg @@ -0,0 +1 @@ +Train — Core ClassesTrain — Core Classes«core»Trainself.create(name, *args) : Transportself.options(name) : Hashself.load_transport(name) : Classself.target_config(config) : Hashself.unpack_target_from_uri(uri, opts) : Hashself.validate_backend(creds, default="local") : String«core»Pluginsself.registry : Hash<String,Class>self.plugin(version=1) : Class«core»Plugins::Transportinitialize(options={})connection(_options=nil) : BaseConnectionself.name(name)self.option(name, conf, &blk)self.default_options : Hashvalidate_options(opts)«abstract»BaseConnectioninitialize(options=nil)run_command(cmd, opts={}, &blk) : CommandResultfile(path, *args) : Fileplatform() : Platformupload(locals, remote)download(remotes, local)force_platform!(name, details=nil) : Platformenable_cache(type) / disable_cache(type)close() / wait_until_ready() / login_command()run_command_via_connection(cmd, [opts], &blk)file_via_connection(path, *args)@cache_enabled : {file:true, command:false, api_call:false}@audit_log : AuditLog«core»OptionsClassOptions: option / default_options / include_optionsInstanceOptions: merge_options / validate_options«core»CommandResultstdoutstderrexit_status«abstract»FileDATA_FIELDS = exist? mode owner groupuid gid content mtime size selinux_label pathto_json / type / directory? / symlink?md5sum / sha256sum«core»AuditLogself.create(opts) : Loggerinfo(entry)run_command dispatches on thearity of run_command_via_connection(1 vs 2) for plugin back-compat.Caching: file=on, command/api_call=off.registry lookupplugin(1) baseOptions.attach(self)connection returnsinclude InstanceOptionsrun_command returnsfile returnslogs when enabled \ No newline at end of file diff --git a/context/local/architecture/diagrams/03-transport-hierarchy.puml b/context/local/architecture/diagrams/03-transport-hierarchy.puml new file mode 100644 index 00000000..d4a97458 --- /dev/null +++ b/context/local/architecture/diagrams/03-transport-hierarchy.puml @@ -0,0 +1,55 @@ +@startuml 03-transport-hierarchy +!include _style.puml +title Train — Transport / Connection Hierarchy (two archetypes) + +abstract class "BaseConnection" as Base <> { + +run_command / file / platform + #run_command_via_connection {abstract} + #file_via_connection {abstract} +} + +package "OS-command transports\n(implement run_command_via_connection + file_via_connection,\nreal platform detection)" <> { + class "Local::Connection" as Local <> + class "SSH::Connection" as Ssh <> + class "Docker::Connection" as Docker <> + class "Podman::Connection" as Podman <> + class "TrainPlugins::Winrm::Connection" as Winrm <> + class "TrainPlugins::Kubernetes::Connection" as K8s <> +} + +package "API transports\n(force_platform!, bespoke client surface,\nno file/command primitives)" <> { + class "Aws::Connection" as Aws <> + class "Azure::Connection" as Azure <> + class "Gcp::Connection" as Gcp <> + class "VMware::Connection" as Vmware <> + class "TrainPlugins::Rest::Connection" as Rest <> +} + +Base <|-- Local +Base <|-- Ssh +Base <|-- Docker +Base <|-- Podman +Base <|-- Winrm +Base <|-- K8s +Base <|-- Aws +Base <|-- Azure +Base <|-- Gcp +Base <|-- Vmware +Base <|-- Rest + +note right of Aws + API archetype exposes clients like + aws_client(klass) / azure_client; + callers must know the transport type. + Anti-patterns observed here: + - ENV export of AWS creds + - k8s connect() calling exit on error +end note + +note left of Local + OS-command archetype fulfills the + universal contract: run_command + file + work against any resource unchanged. +end note + +@enduml diff --git a/context/local/architecture/diagrams/03-transport-hierarchy.svg b/context/local/architecture/diagrams/03-transport-hierarchy.svg new file mode 100644 index 00000000..562e7e69 --- /dev/null +++ b/context/local/architecture/diagrams/03-transport-hierarchy.svg @@ -0,0 +1 @@ +Train — Transport / Connection Hierarchy (two archetypes)Train — Transport / Connection Hierarchy (two archetypes)OS-command transports(implement run_command_via_connection + file_via_connection,real platform detection)API transports(force_platform!, bespoke client surface,no file/command primitives)«oscmd»Local::Connection«oscmd»SSH::Connection«oscmd»Docker::Connection«oscmd»Podman::Connection«oscmd»TrainPlugins::Winrm::Connection«oscmd»TrainPlugins::Kubernetes::Connection«api»Aws::Connection«api»Azure::Connection«api»Gcp::Connection«api»VMware::Connection«api»TrainPlugins::Rest::Connection«abstract»BaseConnectionrun_command / file / platformrun_command_via_connectionfile_via_connectionAPI archetype exposes clients likeaws_client(klass) / azure_client;callers must know the transport type.Anti-patterns observed here:- ENV export of AWS creds- k8s connect() calling exit on errorOS-command archetype fulfills theuniversal contract: run_command + filework against any resource unchanged. \ No newline at end of file diff --git a/context/local/architecture/diagrams/04-platform-detection.puml b/context/local/architecture/diagrams/04-platform-detection.puml new file mode 100644 index 00000000..d08e3c95 --- /dev/null +++ b/context/local/architecture/diagrams/04-platform-detection.puml @@ -0,0 +1,82 @@ +@startuml 04-platform-detection +!include _style.puml +title Train — Platform Detection Subsystem + +class "Train::Platforms" as Platforms <> { + +self.name(name, cond={}) : Platform + +self.family(name, cond={}) : Family + +self.list / families + +self.top_platforms + +self.export +} + +class "Detect" as Detect <> { + +self.scan(backend) : Platform +} + +class "Detect::Scanner" as Scanner <> { + +initialize(backend) + +scan : Platform + -scan_children(parent) + -scan_family_children(plat) + -check_condition(condition) + -get_platform(plat) +} + +class "Detect::Specifications::OS" as SpecOS <> { + +self.load +} +class "Detect::Specifications::Api" as SpecApi <> { + +self.load +} + +class "Platform" as Platform <> { + +name / title / family + +backend / platform : Hash + +family_hierarchy + +find_family_hierarchy + +add_platform_methods + +uuid +} + +class "Family" as Family <> { + +name / title + +children / families +} + +class "Common" as Common <> { + +detect(&block) + +family / families(*) + +platform(name) +} + +class "Detect::UUID" as UUID <> { + +find_or_create_uuid +} + +Detect --> Scanner : new(backend).scan +Scanner --> Platforms : top_platforms +Scanner --> Platform : get_platform / add_platform_methods +Platforms --> Platform : creates / registry (list) +Platforms --> Family : creates / registry (families) +Platform ..|> Common +Family ..|> Common +SpecOS ..> Platforms : declares family/platform tree +SpecApi ..> Platforms : declares API platforms +Platform --> UUID : lazy uuid + +note bottom of Scanner + scan() instance_evals each platform's + detect block against the backend, walking + top_platforms -> children -> families. + Probes run via run_command + file on the + backend (OSCommon helpers). +end note + +note right of SpecApi + API transports register here and are + selected through force_platform!, + bypassing the command/file probing. +end note + +@enduml diff --git a/context/local/architecture/diagrams/04-platform-detection.svg b/context/local/architecture/diagrams/04-platform-detection.svg new file mode 100644 index 00000000..7db6367b --- /dev/null +++ b/context/local/architecture/diagrams/04-platform-detection.svg @@ -0,0 +1 @@ +Train — Platform Detection SubsystemTrain — Platform Detection Subsystem«core»Train::Platformsself.list / familiesself.top_platformsself.exportself.name(name, cond={}) : Platformself.family(name, cond={}) : Family«core»Detectself.scan(backend) : Platform«core»Detect::Scannerscan : Platforminitialize(backend)scan_children(parent)scan_family_children(plat)check_condition(condition)get_platform(plat)«core»Detect::Specifications::OSself.load«core»Detect::Specifications::Apiself.load«core»Platformname / title / familybackend / platform : Hashfamily_hierarchyfind_family_hierarchyadd_platform_methodsuuid«core»Familyname / titlechildren / families«core»Commondetect(&block)family / families(*)platform(name)«core»Detect::UUIDfind_or_create_uuidscan() instance_evals each platform'sdetect block against the backend, walkingtop_platforms -> children -> families.Probes run via run_command + file on thebackend (OSCommon helpers).API transports register here and areselected through force_platform!,bypassing the command/file probing.new(backend).scantop_platformsget_platform / add_platform_methodscreates / registry (list)creates / registry (families)declares family/platform treedeclares API platformslazy uuid \ No newline at end of file diff --git a/context/local/architecture/diagrams/05-file-hierarchy.puml b/context/local/architecture/diagrams/05-file-hierarchy.puml new file mode 100644 index 00000000..65138164 --- /dev/null +++ b/context/local/architecture/diagrams/05-file-hierarchy.puml @@ -0,0 +1,59 @@ +@startuml 05-file-hierarchy +!include _style.puml +title Train — File Abstraction Hierarchy + +abstract class "Train::File" as File <> { + +initialize(backend, path, follow_symlink=true) + DATA_FIELDS = exist? mode owner group uid + gid content mtime size selinux_label path + +to_json / type / source / path + +file? / directory? / symlink? / socket? ... + +md5sum / sha256sum + +mounted? + #sanitize_filename(path) +} + +class "File::Local" as Local <> +class "File::Remote" as Remote <> + +class "File::Local::Unix" as LUnix <> +class "File::Local::Windows" as LWin <> + +class "File::Remote::Unix" as RUnix <> +class "File::Remote::Windows" as RWin <> +class "File::Remote::Linux" as RLinux <> +class "File::Remote::Aix" as RAix <> +class "File::Remote::Qnx" as RQnx <> + +class "Extras::Stat" as Stat <> { + +self.stat(path, backend, follow) + +self.find_type(mode) + mode/type/owner/group/... +} + +File <|-- Local +File <|-- Remote +Local <|-- LUnix +Local <|-- LWin +Remote <|-- RUnix +Remote <|-- RWin +RUnix <|-- RLinux +RUnix <|-- RAix +RUnix <|-- RQnx + +File ..> Stat : mode/type via Stat + +note right of File + Base raises NotImplementedError for every + DATA_FIELD except path. OS/transport + subclass supplies the real implementation. + BaseConnection#file returns one of these. +end note + +note bottom of Remote + Remote subclasses shell out via the + backend (run_command) to read metadata; + Local subclasses use Ruby File/Stat directly. +end note + +@enduml diff --git a/context/local/architecture/diagrams/05-file-hierarchy.svg b/context/local/architecture/diagrams/05-file-hierarchy.svg new file mode 100644 index 00000000..6e8ab7d6 --- /dev/null +++ b/context/local/architecture/diagrams/05-file-hierarchy.svg @@ -0,0 +1 @@ +Train — File Abstraction HierarchyTrain — File Abstraction Hierarchy«abstract»Train::FileDATA_FIELDS = exist? mode owner group uidgid content mtime size selinux_label pathto_json / type / source / pathfile? / directory? / symlink? / socket? ...md5sum / sha256summounted?initialize(backend, path, follow_symlink=true)sanitize_filename(path)«core»File::Local«core»File::Remote«core»File::Local::Unix«core»File::Local::Windows«core»File::Remote::Unix«core»File::Remote::Windows«core»File::Remote::Linux«core»File::Remote::Aix«core»File::Remote::Qnx«core»Extras::Statmode/type/owner/group/...self.stat(path, backend, follow)self.find_type(mode)Base raises NotImplementedError for everyDATA_FIELD except path. OS/transportsubclass supplies the real implementation.BaseConnection#file returns one of these.Remote subclasses shell out via thebackend (run_command) to read metadata;Local subclasses use Ruby File/Stat directly.mode/type via Stat \ No newline at end of file diff --git a/context/local/architecture/diagrams/06-error-hierarchy.puml b/context/local/architecture/diagrams/06-error-hierarchy.puml new file mode 100644 index 00000000..68a856a7 --- /dev/null +++ b/context/local/architecture/diagrams/06-error-hierarchy.puml @@ -0,0 +1,46 @@ +@startuml 06-error-hierarchy +!include _style.puml +title Train — Exception Hierarchy + +class "::StandardError" as Std <> + +class "Train::Error" as Err <> { + +reason : Symbol + +initialize(message="", reason=:not_provided) +} + +class "Train::UserError" as UserErr <> +class "Train::ClientError" as ClientErr <> +class "Train::TransportError" as TransportErr <> + +class "Train::PluginLoadError" as PluginLoad <> { + +transport_name +} +class "Train::PlatformDetectionFailed" as PlatDetect <> +class "Train::PlatformUuidDetectionFailed" as PlatUuid <> +class "Train::UnknownCacheType" as CacheType <> +class "Train::CommandTimeoutReached" as CmdTimeout <> + +Std <|-- Err +Err <|-- UserErr +Err <|-- ClientErr +Err <|-- TransportErr +Err <|-- PlatDetect +Err <|-- PlatUuid +Err <|-- CacheType +Err <|-- CmdTimeout +UserErr <|-- PluginLoad + +note right of Err + Every library-raised exception derives + from Train::Error and carries a :reason + symbol for programmatic handling. +end note + +note bottom of UserErr + UserError = bad user input (creds, target) + ClientError = misuse of the API / unimpl. + TransportError = failure in the transport layer +end note + +@enduml diff --git a/context/local/architecture/diagrams/06-error-hierarchy.svg b/context/local/architecture/diagrams/06-error-hierarchy.svg new file mode 100644 index 00000000..e3968864 --- /dev/null +++ b/context/local/architecture/diagrams/06-error-hierarchy.svg @@ -0,0 +1 @@ +Train — Exception HierarchyTrain — Exception Hierarchy«abstract»::StandardError«core»Train::Errorreason : Symbolinitialize(message="", reason=:not_provided)«core»Train::UserError«core»Train::ClientError«core»Train::TransportError«core»Train::PluginLoadErrortransport_name«core»Train::PlatformDetectionFailed«core»Train::PlatformUuidDetectionFailed«core»Train::UnknownCacheType«core»Train::CommandTimeoutReachedEvery library-raised exception derivesfrom Train::Error and carries a :reasonsymbol for programmatic handling.UserError = bad user input (creds, target)ClientError = misuse of the API / unimpl.TransportError = failure in the transport layer \ No newline at end of file diff --git a/context/local/architecture/diagrams/07-seq-run-command.puml b/context/local/architecture/diagrams/07-seq-run-command.puml new file mode 100644 index 00000000..b4269aec --- /dev/null +++ b/context/local/architecture/diagrams/07-seq-run-command.puml @@ -0,0 +1,59 @@ +@startuml 07-seq-run-command +!include _style.puml +title Train — Sequence: create connection and run a command + +actor Consumer as C +participant "Train" as T <> +participant "Plugins.registry" as Reg +participant "Transport\n(subclass)" as Tr +participant "BaseConnection\n(subclass)" as Conn +participant "AuditLog" as Audit +participant "run_command_via_connection\n(plugin impl)" as Impl + +== Resolve & connect == +C -> T : create(name, opts) +activate T +T -> T : load_transport(name) +T -> Reg : registry[name] +alt registered + Reg --> T : class +else not registered + T -> T : require "train/transports/name" + note right of T + then require "train-" gem; + silent rescue LoadError -> + raise PluginLoadError + end note +end +T -> Tr : new(opts) (merge/validate options) +Tr --> C : transport +deactivate T +C -> Tr : connection +Tr -> Conn : new(options) +note right of Conn + loads OS + Api specifications, + sets cache {file:true, command:false, api_call:false}, + wires AuditLog if enabled +end note +Conn --> C : connection + +== Run a command == +C -> Conn : run_command(cmd, opts) +activate Conn +Conn -> Audit : info(type: cmd, ...) [if enabled] +Conn -> Conn : arity = run_command_via_connection.arity.abs +alt arity == 1 + Conn -> Impl : run_command_via_connection(cmd, &blk) +else arity == 2 + Conn -> Impl : run_command_via_connection(cmd, opts, &blk) +else other + Conn -->x C : NotImplementedError +end +Impl --> Conn : CommandResult(stdout, stderr, exit_status) +alt command caching enabled + Conn -> Conn : @cache[:command][cmd] ||= result +end +Conn --> C : CommandResult +deactivate Conn + +@enduml diff --git a/context/local/architecture/diagrams/07-seq-run-command.svg b/context/local/architecture/diagrams/07-seq-run-command.svg new file mode 100644 index 00000000..910d27f4 --- /dev/null +++ b/context/local/architecture/diagrams/07-seq-run-command.svg @@ -0,0 +1 @@ +Train — Sequence: create connection and run a commandTrain — Sequence: create connection and run a commandConsumerConsumer«core»Train«core»TrainPlugins.registryPlugins.registryTransport(subclass)Transport(subclass)BaseConnection(subclass)BaseConnection(subclass)AuditLogAuditLogrun_command_via_connection(plugin impl)run_command_via_connection(plugin impl)Resolve & connectcreate(name, opts)load_transport(name)registry[name]alt[registered]class[not registered]require "train/transports/name"then require "train-<name>" gem;silent rescue LoadError ->raise PluginLoadErrornew(opts) (merge/validate options)transportconnectionnew(options)loads OS + Api specifications,sets cache {file:true, command:false, api_call:false},wires AuditLog if enabledconnectionRun a commandrun_command(cmd, opts)info(type: cmd, ...) [if enabled]arity = run_command_via_connection.arity.absalt[arity == 1]run_command_via_connection(cmd, &blk)[arity == 2]run_command_via_connection(cmd, opts, &blk)[other]NotImplementedErrorCommandResult(stdout, stderr, exit_status)alt[command caching enabled]@cache[:command][cmd] ||= resultCommandResult \ No newline at end of file diff --git a/context/local/architecture/diagrams/08-seq-platform-detect.puml b/context/local/architecture/diagrams/08-seq-platform-detect.puml new file mode 100644 index 00000000..8e695d45 --- /dev/null +++ b/context/local/architecture/diagrams/08-seq-platform-detect.puml @@ -0,0 +1,53 @@ +@startuml 08-seq-platform-detect +!include _style.puml +title Train — Sequence: lazy platform detection + +actor Consumer as C +participant "BaseConnection" as Conn <> +participant "Platforms::Detect" as Detect <> +participant "Detect::Scanner" as Scan <> +participant "Train::Platforms\n(registry)" as Reg <> +participant "Platform" as Plat <> + +== OS-command transport == +C -> Conn : platform (aliased as os) +activate Conn +Conn -> Conn : @platform ||= ... +Conn -> Detect : scan(self) +Detect -> Scan : new(backend).scan +activate Scan +Scan -> Reg : top_platforms +loop each candidate platform/family + Scan -> Scan : instance_eval(&plat.detect) + note right of Scan + detect block probes the target via + backend.run_command / backend.file + (uname, /etc/os-release, lsb, files ...) + end note + alt match + Scan -> Scan : scan_children -> family hierarchy + end +end +alt found + Scan -> Plat : get_platform (backend, platform hash,\nadd_platform_methods, family_hierarchy) + Plat --> Scan : Platform +else none + Scan -->x C : PlatformDetectionFailed +end +Scan --> Detect : Platform +deactivate Scan +Detect --> Conn : Platform +Conn --> C : Platform (os.family, os.name, os.release, ...) +deactivate Conn + +== API transport (no probing) == +C -> Conn : platform +activate Conn +Conn -> Conn : force_platform!(name, details) +Conn -> Reg : Platforms.name(name) +Conn -> Plat : backend=self; find_family_hierarchy; add_platform_methods +Plat --> Conn : Platform +Conn --> C : Platform +deactivate Conn + +@enduml diff --git a/context/local/architecture/diagrams/08-seq-platform-detect.svg b/context/local/architecture/diagrams/08-seq-platform-detect.svg new file mode 100644 index 00000000..31230d31 --- /dev/null +++ b/context/local/architecture/diagrams/08-seq-platform-detect.svg @@ -0,0 +1 @@ +Train — Sequence: lazy platform detectionTrain — Sequence: lazy platform detectionConsumerConsumer«abstract»BaseConnection«abstract»BaseConnection«core»Platforms::Detect«core»Platforms::Detect«core»Detect::Scanner«core»Detect::Scanner«core»Train::Platforms(registry)«core»Train::Platforms(registry)«core»Platform«core»PlatformOS-command transportplatform (aliased as os)@platform ||= ...scan(self)new(backend).scantop_platformsloop[each candidate platform/family]instance_eval(&plat.detect)detect block probes the target viabackend.run_command / backend.file(uname, /etc/os-release, lsb, files ...)alt[match]scan_children -> family hierarchyalt[found]get_platform (backend, platform hash,add_platform_methods, family_hierarchy)Platform[none]PlatformDetectionFailedPlatformPlatformPlatform (os.family, os.name, os.release, ...)API transport (no probing)platformforce_platform!(name, details)Platforms.name(name)backend=self; find_family_hierarchy; add_platform_methodsPlatformPlatform \ No newline at end of file diff --git a/context/local/architecture/diagrams/_style.puml b/context/local/architecture/diagrams/_style.puml new file mode 100644 index 00000000..939bc6c4 --- /dev/null +++ b/context/local/architecture/diagrams/_style.puml @@ -0,0 +1,68 @@ +' Shared PlantUML skin for Train architecture diagrams. +' Included by every diagram via: !include _style.puml +' Keep this the single source of visual truth so all diagrams look consistent. + +skinparam backgroundColor #FFFFFF +skinparam shadowing false +skinparam defaultFontName Helvetica +skinparam defaultFontSize 12 +skinparam roundcorner 6 +skinparam ArrowColor #37474F +skinparam ArrowThickness 1 + +skinparam class { + BackgroundColor #FFFFFF + BorderColor #37474F + ArrowColor #37474F + BorderThickness 1 + AttributeFontColor #455A64 + FontColor #102027 +} + +skinparam package { + BackgroundColor #FAFAFA + BorderColor #90A4AE + FontColor #263238 +} + +skinparam note { + BackgroundColor #FFF8E1 + BorderColor #FFB300 + FontColor #4E342E +} + +skinparam sequence { + ArrowColor #37474F + LifeLineBorderColor #90A4AE + LifeLineBackgroundColor #ECEFF1 + ParticipantBorderColor #37474F + ParticipantBackgroundColor #FFFFFF + ParticipantFontColor #102027 + ActorBorderColor #37474F +} + +' Stereotype colors used to distinguish the two plugin archetypes. +' <> = OS-command transport (implements run_command_via_connection + file_via_connection) +' <> = API transport (force_platform!, bespoke client surface, no file/command) +' <> = shipped inside the train gem +' <> = external train-* gem +skinparam class<> { + BackgroundColor #E8F5E9 + BorderColor #2E7D32 +} +skinparam class<> { + BackgroundColor #E3F2FD + BorderColor #1565C0 +} +skinparam class<> { + BackgroundColor #FFFFFF + BorderColor #37474F +} +skinparam class<> { + BackgroundColor #F3E5F5 + BorderColor #6A1B9A +} +skinparam class<> { + BackgroundColor #FFFDE7 + BorderColor #F9A825 +} diff --git a/context/local/architecture/overview.md b/context/local/architecture/overview.md new file mode 100644 index 00000000..60ef54f5 --- /dev/null +++ b/context/local/architecture/overview.md @@ -0,0 +1,253 @@ +--- +repo: inspec/train +products: + - inspec + - chef-infra-client +divisions: + - chef +--- + +# Train — Architecture Overview (for Implementors & Designers) + +This document is a **visual, structural tour** of the Train transport library. It +is aimed at people who need to *extend* Train — write a transport/plugin, add +platform support, or reason about how a change ripples through consumers — and at +designers who need an accurate mental model of the moving parts. + +It intentionally stays lean on prose: the deep, prescriptive guidance lives in the +shared, cross-repo product area under +[`context/shared/by-product/train/`](../../shared/by-product/train/background/product.md). +This page shows **how the pieces fit together**; those docs tell you **what to do +about it**. + +> **Diagrams are rendered from source.** Every image below is generated from a +> PlantUML `.puml` file in [`diagrams/`](diagrams). See +> [Regenerating the diagrams](#regenerating-the-diagrams) at the end. + +--- + +## 1. The one-paragraph model + +Train is a **unified transport contract**. `Train.create(name, opts)` resolves a +named transport through a plugin **registry**, builds a `Transport`, and hands back +a `BaseConnection` subclass. Consumers (Chef InSpec, Chef Infra Client) then use a +tiny, stable surface on that connection — `run_command`, `file`, and `platform` +(aliased `os`) — without knowing which transport is underneath. Options handling, +lazy platform detection, optional caching, and audit logging are all provided by +the base class. + +### 1.1 Component / context + +How Train sits between its consumers and its plugins, and the single most important +structural fact — plugins split into **two archetypes**. + +![Component / context view](diagrams/01-component-context.svg) + +- **Consumers** — InSpec builds its backend and resources on the connection; + `Inspec::Config` unpacks credentials before calling `Train.create`. Chef's + `target_io` layer routes file/dir/command operations through a Train connection + and reads `transport_options[:sudo]`/`[:user]` off it. +- **Core** — the registry + `Transport` + `BaseConnection`, plus `Options`, + `Platforms`/`Detect`, `File`/`Stat`, `AuditLog`, and the error taxonomy. +- **Plugins** — see the archetype split below. + +See: [`design/architecture.md`](../../shared/by-product/train/design/architecture.md), +[`domains/integration-inspec.md`](../../shared/by-product/train/domains/integration-inspec.md), +[`domains/integration-chef.md`](../../shared/by-product/train/domains/integration-chef.md). + +--- + +## 2. The two plugin archetypes + +The single most useful lens for understanding (and correctly extending) Train: + +| | **OS-command transports** | **API transports** | +|---|---|---| +| Examples | `local`, `ssh`, `docker`, `podman`, `train-winrm`, `train-kubernetes` | `aws`, `azure`, `gcp`, `vmware`, `train-rest` | +| Implements | `run_command_via_connection` **and** `file_via_connection` | neither | +| Platform | **real detection** via the Scanner | `force_platform!` (hardcoded) | +| Consumer surface | universal — `run_command`/`file` work unchanged | **bespoke** clients (`aws_client(klass)`, REST verbs, …) | + +![Transport / connection hierarchy](diagrams/03-transport-hierarchy.svg) + +If you are writing a plugin, **decide which archetype you are** first — it +determines almost everything else. Full treatment: +[`design/plugin-archetypes.md`](../../shared/by-product/train/design/plugin-archetypes.md). + +--- + +## 3. Core building blocks + +The classes an implementor actually touches, with the methods that matter. + +![Core classes](diagrams/02-core-classes.svg) + +- **`Train` (module)** — entry points: `create`, `options`, `load_transport`, + and the target/credential helpers (`target_config`, `unpack_target_from_uri`, + `validate_backend`). +- **`Plugins` / `Plugins::Transport`** — `Plugins.registry` maps names to classes; + a transport registers itself with `name "foo"`. `Transport#connection` is the + abstract factory you override. +- **`BaseConnection`** — the heart. Public `run_command` / `file` / `platform` + wrap the private `*_via_connection` methods with **audit logging** and + **caching**; `force_platform!` is the API-archetype escape hatch. +- **`Options`** — `Options.attach(klass)` mixes in the `option`/`default_options` + DSL and `merge_options`/`validate_options`. +- **`CommandResult`** — `Struct.new(:stdout, :stderr, :exit_status)`; the universal + return of `run_command`. +- **`File`** — abstract; declares `DATA_FIELDS` and raises `NotImplementedError` + for each until a subclass supplies it. +- **`AuditLog`** — optional structured logging of commands, file access, and + uploads. + +Interface references: +[`interfaces/connection-api.md`](../../shared/by-product/train/interfaces/connection-api.md), +[`interfaces/command-primitive.md`](../../shared/by-product/train/interfaces/command-primitive.md), +[`interfaces/file-primitive.md`](../../shared/by-product/train/interfaces/file-primitive.md), +[`interfaces/options-and-targets.md`](../../shared/by-product/train/interfaces/options-and-targets.md). + +--- + +## 4. Platform detection + +Detection is **lazy** (first call to `platform`/`os`) and driven by a `Scanner` +that walks a declarative tree of `Platform`/`Family` specifications, `instance_eval`-ing +each `detect` block against the live backend. + +![Platform detection subsystem](diagrams/04-platform-detection.svg) + +Key facts for anyone adding platform support: + +- Specifications live in `Detect::Specifications::OS` and `::Api` and register into + `Train::Platforms.list` / `.families`. +- The Scanner starts from `top_platforms` and descends children/families, using + `run_command` + `file` probes (uname, `/etc/os-release`, lsb, file existence…). +- API transports **do not** run this — they call `force_platform!` and select an + entry from the API specifications directly. + +See: [`design/platform-detection.md`](../../shared/by-product/train/design/platform-detection.md), +[`advice/platform-support.md`](../../shared/by-product/train/advice/platform-support.md). + +--- + +## 5. The File abstraction + +`BaseConnection#file` returns a `Train::File` subclass. The base is deliberately +abstract; Local subclasses read metadata with Ruby directly, Remote subclasses shell +out through the backend. + +![File abstraction hierarchy](diagrams/05-file-hierarchy.svg) + +See: [`interfaces/file-primitive.md`](../../shared/by-product/train/interfaces/file-primitive.md), +[`interfaces/os-file-command-differences.md`](../../shared/by-product/train/interfaces/os-file-command-differences.md). + +--- + +## 6. Error taxonomy + +Every library-raised exception derives from `Train::Error` and carries a `:reason` +symbol, so callers can branch on cause rather than message text. + +![Exception hierarchy](diagrams/06-error-hierarchy.svg) + +- `UserError` — bad user input (credentials, target). `PluginLoadError` is a + `UserError` and carries `transport_name`. +- `ClientError` — API misuse / unimplemented method. +- `TransportError` — failures inside the transport layer. + +See: [`interfaces/error-taxonomy.md`](../../shared/by-product/train/interfaces/error-taxonomy.md). + +--- + +## 7. Request flows + +### 7.1 Create a connection and run a command + +Note the two behaviors that surprise newcomers: the **silent plugin-load fallback** +(registry → core require → `train-` gem → `PluginLoadError`), and the +**arity dispatch** in `run_command` that keeps older single-argument plugins working. + +![Sequence: run a command](diagrams/07-seq-run-command.svg) + +Caching defaults are `{ file: true, command: false, api_call: false }` — file reads +are cached by default, commands are not. + +See: [`design/connection-lifecycle.md`](../../shared/by-product/train/design/connection-lifecycle.md), +[`design/caching.md`](../../shared/by-product/train/design/caching.md), +[`advice/audit-logging.md`](../../shared/by-product/train/advice/audit-logging.md). + +### 7.2 Lazy platform detection + +![Sequence: platform detection](diagrams/08-seq-platform-detect.svg) + +--- + +## 8. Extension points & invariants (read before you build) + +- **Pick an archetype.** OS-command transports must implement *both* + `run_command_via_connection` and `file_via_connection` and support real + detection. API transports call `force_platform!` and expose their own client. +- **Return the contract types.** `run_command` must return a `CommandResult` + (`stdout`, `stderr`, `exit_status`); `file` must return a `Train::File`. +- **Respect option handling.** Declare options with `option`, rely on + `merge_options`/`validate_options`; don't mutate `@options` behind the base + class's back after `super`. +- **Never kill the host process.** Fail with a `Train::TransportError` — do not + call `exit`, and don't leak credentials into global `ENV`. +- **Prefer `transport://credset` targets.** Let `unpack_target_from_uri` do the + URL parsing; don't invent a new field mapping. +- **Honor the error taxonomy.** Raise the right `Train::Error` subclass with a + `:reason`. + +These invariants (and the concrete anti-patterns observed in shipping plugins — +ENV credential leaks, `exit`-on-error, post-`super` option mutation, silent +`rescue LoadError`, "no validation whatsoever" in `validate_backend`) are catalogued +with fixes in: + +- [`advice/writing-a-plugin.md`](../../shared/by-product/train/advice/writing-a-plugin.md) — the authoring hub +- [`advice/anti-patterns.md`](../../shared/by-product/train/advice/anti-patterns.md) +- [`advice/credentials-handling.md`](../../shared/by-product/train/advice/credentials-handling.md) +- [`advice/url-targets.md`](../../shared/by-product/train/advice/url-targets.md) +- [`advice/sudo-and-command-wrappers.md`](../../shared/by-product/train/advice/sudo-and-command-wrappers.md) +- [`advice/testing-plugins.md`](../../shared/by-product/train/advice/testing-plugins.md) +- [`advice/api-stability.md`](../../shared/by-product/train/advice/api-stability.md) +- [`advice/plugin-packaging.md`](../../shared/by-product/train/advice/plugin-packaging.md) + +For testing Train itself, see the repo-local +[`context/local/standards/testing.md`](../standards/testing.md). + +--- + +## Regenerating the diagrams + +Sources are PlantUML (`diagrams/*.puml`), sharing the skin in +`diagrams/_style.puml`. They are rendered to SVG with `plantuml.jar` and Graphviz +`dot`. + +```bash +# from the repo root; requires Java + Graphviz dot on PATH +# (plantuml.jar is not committed — fetch it into tmp/ first) +curl -sSL -o tmp/plantuml.jar \ + https://repo1.maven.org/maven2/net/sourceforge/plantuml/plantuml/1.2024.7/plantuml-1.2024.7.jar + +cd context/local/architecture/diagrams +java -jar ../../../../tmp/plantuml.jar -tsvg -nometadata *.puml +``` + +When you change Train's structure, **update the relevant `.puml` and re-render** so +the diagrams stay faithful to the code. These diagrams are drawn from the actual +implementation (arity shim, archetype split, `force_platform!`), not an idealized +design — keep them honest. + +### Diagram index + +| # | Source | Shows | +|---|--------|-------| +| 01 | [`01-component-context.puml`](diagrams/01-component-context.puml) | Consumers → core → plugins by archetype | +| 02 | [`02-core-classes.puml`](diagrams/02-core-classes.puml) | Core classes & associations | +| 03 | [`03-transport-hierarchy.puml`](diagrams/03-transport-hierarchy.puml) | Connection inheritance; two archetypes | +| 04 | [`04-platform-detection.puml`](diagrams/04-platform-detection.puml) | Detect/Scanner/Specifications/Platform/Family | +| 05 | [`05-file-hierarchy.puml`](diagrams/05-file-hierarchy.puml) | `Train::File` subclass tree + Stat | +| 06 | [`06-error-hierarchy.puml`](diagrams/06-error-hierarchy.puml) | `Train::Error` taxonomy | +| 07 | [`07-seq-run-command.puml`](diagrams/07-seq-run-command.puml) | create → connect → run_command | +| 08 | [`08-seq-platform-detect.puml`](diagrams/08-seq-platform-detect.puml) | Lazy platform detection scan | From 7529a1b5f8c0b794a7f15e0d2c9e73b6e662e84a Mon Sep 17 00:00:00 2001 From: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> Date: Thu, 3 Sep 2026 09:13:35 -0400 Subject: [PATCH 3/4] API-via-CLI considered harmful Signed-off-by: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> --- .../diagrams/01-component-context.puml | 13 ++++++++-- .../diagrams/01-component-context.svg | 2 +- .../diagrams/03-transport-hierarchy.puml | 23 +++++++++++++---- .../diagrams/03-transport-hierarchy.svg | 2 +- .../local/architecture/diagrams/_style.puml | 25 +++++++++++++++++++ context/local/architecture/overview.md | 19 +++++++++++--- 6 files changed, 72 insertions(+), 12 deletions(-) diff --git a/context/local/architecture/diagrams/01-component-context.puml b/context/local/architecture/diagrams/01-component-context.puml index ddcff83d..c71786d4 100644 --- a/context/local/architecture/diagrams/01-component-context.puml +++ b/context/local/architecture/diagrams/01-component-context.puml @@ -31,9 +31,11 @@ package "Plugins" { package "OS-command transports" <> { [local] as PLocal <> [ssh] as PSsh <> - [docker / podman] as PDocker <> [train-winrm] as PWinrm <> - [train-kubernetes] as PK8s <> + } + package "OS-command by history\n(should be API — anti-pattern)" <> { + [docker / podman] as PDocker <> + [train-kubernetes] as PK8s <> } package "API transports" <> { [aws / azure / gcp] as PAws <> @@ -71,4 +73,11 @@ note bottom of Platforms call force_platform! instead of scanning. end note +note right of PK8s + docker/podman/kubernetes shell out to a CLI + (docker/podman/kubectl exec) that fronts an API. + They are OS-command by history but should be + API transports. Lessons, not models. +end note + @enduml diff --git a/context/local/architecture/diagrams/01-component-context.svg b/context/local/architecture/diagrams/01-component-context.svg index 5528c795..712e49cd 100644 --- a/context/local/architecture/diagrams/01-component-context.svg +++ b/context/local/architecture/diagrams/01-component-context.svg @@ -1 +1 @@ -Train — Component / Context ViewTrain — Component / Context ViewConsumersChef InSpecChef Infra ClientTrain core (train gem)PluginsOS-command transports«Rectangle»API transports«Rectangle»«core»Inspec::BackendInSpec resources(command, file, os, ...)Inspec::Config(unpack_train_credentials)Chef target_io(TargetIO::File / Dir / etc.)Chef::Application«core»Train.create /load_transport(registry)«core»Transport(plugin base)«abstract»BaseConnection«core»Options(Class/InstanceOptions)«core»Platforms + Detect(Scanner, Specifications)«core»File / Stat«core»AuditLog«core»errors«oscmd»local«oscmd»ssh«oscmd»docker / podman«oscmd»train-winrm«oscmd»train-kubernetes«api»aws / azure / gcp«api»train-rest«api»vmwareAPI transports skip real detection andcall force_platform! instead of scanning.credentials hashTrain.create(name, opts)holds connectionrun_command / file / osTrain.createfile / run_command(reads transport_options[:sudo/:user])resolves & instantiatesconnectionmerge/validatelog cmd/file (if enabled)platform (lazy scan)fileraises \ No newline at end of file +Train — Component / Context ViewTrain — Component / Context ViewConsumersChef InSpecChef Infra ClientTrain core (train gem)PluginsOS-command transports«Rectangle»OS-command by history(should be API — anti-pattern)«Rectangle»API transports«Rectangle»«core»Inspec::BackendInSpec resources(command, file, os, ...)Inspec::Config(unpack_train_credentials)Chef target_io(TargetIO::File / Dir / etc.)Chef::Application«core»Train.create /load_transport(registry)«core»Transport(plugin base)«abstract»BaseConnection«core»Options(Class/InstanceOptions)«core»Platforms + Detect(Scanner, Specifications)«core»File / Stat«core»AuditLog«core»errors«oscmd»local«oscmd»ssh«oscmd»train-winrm«historical»docker / podman«historical»train-kubernetes«api»aws / azure / gcp«api»train-rest«api»vmwareAPI transports skip real detection andcall force_platform! instead of scanning.docker/podman/kubernetes shell out to a CLI(docker/podman/kubectl exec) that fronts an API.They are OS-command by history but should beAPI transports. Lessons, not models.credentials hashTrain.create(name, opts)holds connectionrun_command / file / osTrain.createfile / run_command(reads transport_options[:sudo/:user])resolves & instantiatesconnectionmerge/validatelog cmd/file (if enabled)platform (lazy scan)fileraises \ No newline at end of file diff --git a/context/local/architecture/diagrams/03-transport-hierarchy.puml b/context/local/architecture/diagrams/03-transport-hierarchy.puml index d4a97458..4ff0098a 100644 --- a/context/local/architecture/diagrams/03-transport-hierarchy.puml +++ b/context/local/architecture/diagrams/03-transport-hierarchy.puml @@ -11,10 +11,13 @@ abstract class "BaseConnection" as Base <> { package "OS-command transports\n(implement run_command_via_connection + file_via_connection,\nreal platform detection)" <> { class "Local::Connection" as Local <> class "SSH::Connection" as Ssh <> - class "Docker::Connection" as Docker <> - class "Podman::Connection" as Podman <> class "TrainPlugins::Winrm::Connection" as Winrm <> - class "TrainPlugins::Kubernetes::Connection" as K8s <> +} + +package "OS-command by history — should be API\n(archetype mismatch: shell out to a CLI that fronts an API)" <> { + class "Docker::Connection" as Docker <> + class "Podman::Connection" as Podman <> + class "TrainPlugins::Kubernetes::Connection" as K8s <> } package "API transports\n(force_platform!, bespoke client surface,\nno file/command primitives)" <> { @@ -41,15 +44,25 @@ note right of Aws API archetype exposes clients like aws_client(klass) / azure_client; callers must know the transport type. - Anti-patterns observed here: + Impl anti-pattern observed here: - ENV export of AWS creds - - k8s connect() calling exit on error end note note left of Local OS-command archetype fulfills the universal contract: run_command + file work against any resource unchanged. + Correctly archetyped: these target a + real OS shell (local/ssh/winrm). +end note + +note bottom of K8s + Mis-archetyped: docker/podman/kubernetes + target API-first systems but shell out to + docker/podman/kubectl exec. Should be API + transports. train-kubernetes also calls + exit() on connect error and swallows ENOENT. + Two axes: archetype fit + impl quality. end note @enduml diff --git a/context/local/architecture/diagrams/03-transport-hierarchy.svg b/context/local/architecture/diagrams/03-transport-hierarchy.svg index 562e7e69..182ea0a7 100644 --- a/context/local/architecture/diagrams/03-transport-hierarchy.svg +++ b/context/local/architecture/diagrams/03-transport-hierarchy.svg @@ -1 +1 @@ -Train — Transport / Connection Hierarchy (two archetypes)Train — Transport / Connection Hierarchy (two archetypes)OS-command transports(implement run_command_via_connection + file_via_connection,real platform detection)API transports(force_platform!, bespoke client surface,no file/command primitives)«oscmd»Local::Connection«oscmd»SSH::Connection«oscmd»Docker::Connection«oscmd»Podman::Connection«oscmd»TrainPlugins::Winrm::Connection«oscmd»TrainPlugins::Kubernetes::Connection«api»Aws::Connection«api»Azure::Connection«api»Gcp::Connection«api»VMware::Connection«api»TrainPlugins::Rest::Connection«abstract»BaseConnectionrun_command / file / platformrun_command_via_connectionfile_via_connectionAPI archetype exposes clients likeaws_client(klass) / azure_client;callers must know the transport type.Anti-patterns observed here:- ENV export of AWS creds- k8s connect() calling exit on errorOS-command archetype fulfills theuniversal contract: run_command + filework against any resource unchanged. \ No newline at end of file +Train — Transport / Connection Hierarchy (two archetypes)Train — Transport / Connection Hierarchy (two archetypes)OS-command transports(implement run_command_via_connection + file_via_connection,real platform detection)OS-command by history — should be API(archetype mismatch: shell out to a CLI that fronts an API)API transports(force_platform!, bespoke client surface,no file/command primitives)«oscmd»Local::Connection«oscmd»SSH::Connection«oscmd»TrainPlugins::Winrm::Connection«historical»Docker::Connection«historical»Podman::Connection«historical»TrainPlugins::Kubernetes::Connection«api»Aws::Connection«api»Azure::Connection«api»Gcp::Connection«api»VMware::Connection«api»TrainPlugins::Rest::Connection«abstract»BaseConnectionrun_command / file / platformrun_command_via_connectionfile_via_connectionAPI archetype exposes clients likeaws_client(klass) / azure_client;callers must know the transport type.Impl anti-pattern observed here:- ENV export of AWS credsOS-command archetype fulfills theuniversal contract: run_command + filework against any resource unchanged.Correctly archetyped: these target areal OS shell (local/ssh/winrm).Mis-archetyped: docker/podman/kubernetestarget API-first systems but shell out todocker/podman/kubectl exec. Should be APItransports. train-kubernetes also callsexit() on connect error and swallows ENOENT.Two axes: archetype fit + impl quality. \ No newline at end of file diff --git a/context/local/architecture/diagrams/_style.puml b/context/local/architecture/diagrams/_style.puml index 939bc6c4..d3bbb87c 100644 --- a/context/local/architecture/diagrams/_style.puml +++ b/context/local/architecture/diagrams/_style.puml @@ -66,3 +66,28 @@ skinparam class<> { BackgroundColor #FFFDE7 BorderColor #F9A825 } +' <> = built as an OS-command transport but conceptually mis-archetyped +' (targets an API-first system; should be an API transport). Cautionary, not a model. +skinparam class<> { + BackgroundColor #FFF3E0 + BorderColor #E65100 +} + +' Component-diagram variants (diagram 01 uses component nodes, which take +' their stereotype colors from skinparam component<<...>>, not class<<...>>). +skinparam component<> { + BackgroundColor #E8F5E9 + BorderColor #2E7D32 +} +skinparam component<> { + BackgroundColor #E3F2FD + BorderColor #1565C0 +} +skinparam component<> { + BackgroundColor #FFFFFF + BorderColor #37474F +} +skinparam component<> { + BackgroundColor #FFF3E0 + BorderColor #E65100 +} diff --git a/context/local/architecture/overview.md b/context/local/architecture/overview.md index 60ef54f5..5f3006b8 100644 --- a/context/local/architecture/overview.md +++ b/context/local/architecture/overview.md @@ -63,15 +63,28 @@ The single most useful lens for understanding (and correctly extending) Train: | | **OS-command transports** | **API transports** | |---|---|---| -| Examples | `local`, `ssh`, `docker`, `podman`, `train-winrm`, `train-kubernetes` | `aws`, `azure`, `gcp`, `vmware`, `train-rest` | +| Examples | `local`, `ssh`, `train-winrm` | `aws`, `azure`, `gcp`, `vmware`, `train-rest` | | Implements | `run_command_via_connection` **and** `file_via_connection` | neither | | Platform | **real detection** via the Scanner | `force_platform!` (hardcoded) | | Consumer surface | universal — `run_command`/`file` work unchanged | **bespoke** clients (`aws_client(klass)`, REST verbs, …) | +> **A third, cautionary group — historically mis-archetyped.** `docker`, `podman`, +> and `train-kubernetes` are *implemented* as OS-command transports (they shell +> out to `docker exec` / `podman exec` / `kubectl exec`) but their targets are +> **API-first systems** (Docker Engine API, Podman libpod API, Kubernetes API). +> They should have been **API** transports. Treat them as *lessons, not models* — +> the archetype choice itself is the defect, and `train-kubernetes` compounds it +> with real implementation anti-patterns (exit-on-error, swallowed `ENOENT`, +> hardcoded tty/stdin). Judge transports on **two axes**: *archetype fit* +> (conceptual) and *implementation quality* (behavioral). See +> [`advice/anti-patterns.md` #0](../../shared/by-product/train/advice/anti-patterns.md) +> and [`design/plugin-archetypes.md`](../../shared/by-product/train/design/plugin-archetypes.md). + ![Transport / connection hierarchy](diagrams/03-transport-hierarchy.svg) -If you are writing a plugin, **decide which archetype you are** first — it -determines almost everything else. Full treatment: +If you are writing a plugin, **decide which archetype you are** first — pick by +the target's *native* contract (a real shell → OS-command; anything reachable +over an HTTP/SDK API → API, *even if a CLI exists*). Full treatment: [`design/plugin-archetypes.md`](../../shared/by-product/train/design/plugin-archetypes.md). --- From 548e428ed640d3dcdd86e381101b12cc2e6d059e Mon Sep 17 00:00:00 2001 From: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> Date: Thu, 3 Sep 2026 09:44:46 -0400 Subject: [PATCH 4/4] Some clarifications Signed-off-by: Clinton Wolfe <156460+clintoncwolfe@users.noreply.github.com> --- .../diagrams/01-component-context.puml | 20 +++-- .../diagrams/01-component-context.svg | 2 +- context/local/architecture/overview.md | 8 +- context/local/background/release-and-ci.md | 84 +++++++++++++++++++ context/local/standards/coding-standards.md | 5 ++ context/local/standards/testing.md | 34 +++++--- context/local/standards/workflow.md | 24 +++++- 7 files changed, 155 insertions(+), 22 deletions(-) create mode 100644 context/local/background/release-and-ci.md diff --git a/context/local/architecture/diagrams/01-component-context.puml b/context/local/architecture/diagrams/01-component-context.puml index c71786d4..cb67dec7 100644 --- a/context/local/architecture/diagrams/01-component-context.puml +++ b/context/local/architecture/diagrams/01-component-context.puml @@ -29,21 +29,29 @@ package "Train core (train gem)" { package "Plugins" { package "OS-command transports" <> { - [local] as PLocal <> - [ssh] as PSsh <> + [local *] as PLocal <> + [ssh *] as PSsh <> [train-winrm] as PWinrm <> } package "OS-command by history\n(should be API — anti-pattern)" <> { - [docker / podman] as PDocker <> + [docker / podman *] as PDocker <> [train-kubernetes] as PK8s <> } package "API transports" <> { - [aws / azure / gcp] as PAws <> + [azure / gcp / vmware *] as PAzure <> + [train-aws] as PAws <> [train-rest] as PRest <> - [vmware] as PVmware <> } } +note top of PLocal + * = in-repo transport (ships in train / train-core). + Bare "train-*" names are **external plugin gems**, + installed separately and discovered via RubyGems. + See design/architecture.md#in-repo-transports-vs-external-plugin-gems. + (winrm is external despite the stale train-core gemspec text.) +end note + InspecConfig --> TrainEntry : credentials hash InspecBackend --> TrainEntry : Train.create(name, opts) InspecBackend --> BaseConnection : holds connection @@ -64,9 +72,9 @@ Transport <|-- PSsh Transport <|-- PDocker Transport <|-- PWinrm Transport <|-- PK8s +Transport <|-- PAzure Transport <|-- PAws Transport <|-- PRest -Transport <|-- PVmware note bottom of Platforms API transports skip real detection and diff --git a/context/local/architecture/diagrams/01-component-context.svg b/context/local/architecture/diagrams/01-component-context.svg index 712e49cd..ea0fd6c9 100644 --- a/context/local/architecture/diagrams/01-component-context.svg +++ b/context/local/architecture/diagrams/01-component-context.svg @@ -1 +1 @@ -Train — Component / Context ViewTrain — Component / Context ViewConsumersChef InSpecChef Infra ClientTrain core (train gem)PluginsOS-command transports«Rectangle»OS-command by history(should be API — anti-pattern)«Rectangle»API transports«Rectangle»«core»Inspec::BackendInSpec resources(command, file, os, ...)Inspec::Config(unpack_train_credentials)Chef target_io(TargetIO::File / Dir / etc.)Chef::Application«core»Train.create /load_transport(registry)«core»Transport(plugin base)«abstract»BaseConnection«core»Options(Class/InstanceOptions)«core»Platforms + Detect(Scanner, Specifications)«core»File / Stat«core»AuditLog«core»errors«oscmd»local«oscmd»ssh«oscmd»train-winrm«historical»docker / podman«historical»train-kubernetes«api»aws / azure / gcp«api»train-rest«api»vmwareAPI transports skip real detection andcall force_platform! instead of scanning.docker/podman/kubernetes shell out to a CLI(docker/podman/kubectl exec) that fronts an API.They are OS-command by history but should beAPI transports. Lessons, not models.credentials hashTrain.create(name, opts)holds connectionrun_command / file / osTrain.createfile / run_command(reads transport_options[:sudo/:user])resolves & instantiatesconnectionmerge/validatelog cmd/file (if enabled)platform (lazy scan)fileraises \ No newline at end of file +Train — Component / Context ViewTrain — Component / Context ViewConsumersChef InSpecChef Infra ClientTrain core (train gem)PluginsOS-command transports«Rectangle»OS-command by history(should be API — anti-pattern)«Rectangle»API transports«Rectangle»«core»Inspec::BackendInSpec resources(command, file, os, ...)Inspec::Config(unpack_train_credentials)Chef target_io(TargetIO::File / Dir / etc.)Chef::Application«core»Train.create /load_transport(registry)«core»Transport(plugin base)«abstract»BaseConnection«core»Options(Class/InstanceOptions)«core»Platforms + Detect(Scanner, Specifications)«core»File / Stat«core»AuditLog«core»errors«oscmd»local *«oscmd»ssh *«oscmd»train-winrm«historical»docker / podman *«historical»train-kubernetes«api»azure / gcp / vmware *«api»train-aws«api»train-rest= in-repo transport (ships in train / train-core).Bare "train-*" names areexternal plugin gems,installed separately and discovered via RubyGems.See design/architecture.md#in-repo-transports-vs-external-plugin-gems.(winrm is external despite the stale train-core gemspec text.)API transports skip real detection andcall force_platform! instead of scanning.docker/podman/kubernetes shell out to a CLI(docker/podman/kubectl exec) that fronts an API.They are OS-command by history but should beAPI transports. Lessons, not models.credentials hashTrain.create(name, opts)holds connectionrun_command / file / osTrain.createfile / run_command(reads transport_options[:sudo/:user])resolves & instantiatesconnectionmerge/validatelog cmd/file (if enabled)platform (lazy scan)fileraises \ No newline at end of file diff --git a/context/local/architecture/overview.md b/context/local/architecture/overview.md index 5f3006b8..f060fe82 100644 --- a/context/local/architecture/overview.md +++ b/context/local/architecture/overview.md @@ -49,7 +49,13 @@ structural fact — plugins split into **two archetypes**. and reads `transport_options[:sudo]`/`[:user]` off it. - **Core** — the registry + `Transport` + `BaseConnection`, plus `Options`, `Platforms`/`Detect`, `File`/`Stat`, `AuditLog`, and the error taxonomy. -- **Plugins** — see the archetype split below. +- **Plugins** — see the archetype split below. Nodes marked `*` in the diagram are + **in-repo** transports (they ship inside the `train`/`train-core` gems); bare + `train-*` names are **external plugin gems** installed separately and discovered + over RubyGems. This distinction is orthogonal to the archetype split — e.g. + `ssh` (in-repo) and `train-winrm` (external gem) are both OS-command transports. + `winrm` is external despite the stale `train-core` gemspec description; see the + canonical [in-repo vs external transport map](../../shared/by-product/train/design/architecture.md#in-repo-transports-vs-external-plugin-gems). See: [`design/architecture.md`](../../shared/by-product/train/design/architecture.md), [`domains/integration-inspec.md`](../../shared/by-product/train/domains/integration-inspec.md), diff --git a/context/local/background/release-and-ci.md b/context/local/background/release-and-ci.md new file mode 100644 index 00000000..11973ede --- /dev/null +++ b/context/local/background/release-and-ci.md @@ -0,0 +1,84 @@ +--- +repo: inspec/train +products: + - inspec + - chef-infra-client +divisions: + - chef +--- + +# Release, Versioning & CI (Train repo) + +Authoritative notes on how the `train` repo actually builds, versions, tests, and +ships — sourced from `.expeditor/` and the gemspecs. This supersedes any +JIRA/prompt-style workflow text under `context/local/standards/workflow.md`, which +is illustrative, not the house process. + +## Release automation — Chef Expeditor + +Releases are driven by **Chef Expeditor** (`.expeditor/config.yml`), not by manual +tagging or `gem push`. The relevant machinery: + +- **Published gems:** `train` and `train-core` (`rubygems:` in the config). +- **On every PR merged to a release branch**, Expeditor runs, in order: + 1. `built_in:bump_version` (skippable with the `Expeditor: Skip Version Bump` / + `Expeditor: Skip All` labels), + 2. `bash:.expeditor/update_version.sh` (only if the version bumped), + 3. `built_in:update_changelog`, + 4. `built_in:build_gem`, + 5. trigger the `coverage` pipeline. +- **On promotion** (`project_promoted`): `built_in:rollover_changelog` then + `built_in:publish_rubygems`. +- **Git tags:** `v{{version}}` format; PR branches are deleted on merge. +- **Release branches:** `main` (version constraint `3.*`) and `2-stable` (`2.*`). + +### Versioning (SemVer, label-driven) + +- The version lives in `lib/train/version.rb` (`Train::VERSION`, currently + **3.16.5**) and is bumped automatically. +- Default bump is a patch. Apply a PR label to bump higher: + - `Expeditor: Bump Minor Version` + - `Expeditor: Bump Major Version` +- Don't hand-edit `version.rb` in a feature PR; let Expeditor own it. + +## Continuous integration — Buildkite + +PRs are validated by the **`verify`** Buildkite pipeline +(`.expeditor/verify.pipeline.yml`; 20-min timeout, one automatic retry): + +- **Lint** — `RAKE_TASK=lint` runs **ChefStyle** on the oldest supported Ruby + (**3.1**), to catch new Ruby-isms early. +- **Tests** — the Minitest suite runs on **Ruby 3.1** and **Ruby 3.4** (Linux), + plus **Ruby 3.1 on Windows**. +- A separate **`coverage`** pipeline (`.expeditor/coverage.pipeline.yml`) generates + the coverage report; coverage is **not** a merge gate (see + [../standards/testing.md](../standards/testing.md)). + +## Supported Ruby + +- **Authoritative floor:** `required_ruby_version >= 3.1.0` in both + `train.gemspec` and `train-core.gemspec`. +- CI exercises **3.1** (floor) and **3.4**. A local `.ruby-version` file, if + present, is untracked/gitignored and is **not** the supported-version pin. + +## Gem layout (what ships where) + +- **`train-core`** — the plugin system + a minimal in-repo backend set + (`local`, `ssh`, `cisco_ios`, `mock`); light dependencies. Does **not** bundle + winrm. +- **`train`** — the "batteries included" meta-gem: `train-core` **plus** the + external **`train-winrm`** gem (`~> 0.4.0`) and the heavier in-repo cloud/container + transports (`azure`, `gcp`, `docker`, `podman`, `vmware`) with their SDKs. +- External transports (`winrm`, `aws`, `rest`, `kubernetes`, …) ship as separate + `train-*` plugin gems. See the canonical + [in-repo vs external transport map](../../shared/by-product/train/design/architecture.md#in-repo-transports-vs-external-plugin-gems) + and [../../shared/by-product/train/advice/plugin-packaging.md](../../shared/by-product/train/advice/plugin-packaging.md). + +## What this means for a contributor + +- Open a PR against `main`; keep it green on the `verify` pipeline (ChefStyle + + tests on 3.1/3.4). +- Do **not** bump the version or edit the changelog by hand — add the appropriate + `Expeditor: Bump …` label if a non-patch release is warranted. +- The gem is published automatically on promotion; there is no manual `gem push` + step. diff --git a/context/local/standards/coding-standards.md b/context/local/standards/coding-standards.md index b697c997..a9428c87 100644 --- a/context/local/standards/coding-standards.md +++ b/context/local/standards/coding-standards.md @@ -2,6 +2,11 @@ ### Code Quality Standards #### Ruby Standards +- **Supported Ruby**: the authoritative requirement is the gemspec floor — + `required_ruby_version >= 3.1.0` (both `train.gemspec` and `train-core.gemspec`). + Target `>= 3.1.0`. (Note: a local `.ruby-version` file may be present for + developer convenience, but it is untracked/gitignored and **not** authoritative — + don't treat it as the supported-version pin.) - Follow Ruby community conventions - Use proper indentation (2 spaces) - Add appropriate comments and documentation diff --git a/context/local/standards/testing.md b/context/local/standards/testing.md index 94765458..5c42c41a 100644 --- a/context/local/standards/testing.md +++ b/context/local/standards/testing.md @@ -2,23 +2,35 @@ #### Unit Testing Requirements - **Framework**: Minitest (primary testing framework) -- **Coverage**: Maintain > 80% test coverage +- **Coverage**: >80% is an *aspirational* target, **not** an enforced gate — see + "Coverage Configuration" below for what the repo actually does. - **Location**: Tests should be in `test/unit/` directories - **Naming**: Test files should end with `_test.rb` - **Mocking**: Use `mocha/minitest` for mocking external dependencies #### Coverage Configuration +Coverage is **opt-in and ungated**. The real `test/helper.rb` only starts +SimpleCov when `CI_ENABLE_COVERAGE` is set, and configures **no** +`minimum_coverage` threshold and **no** `add_group`s: + ```ruby -# Example SimpleCov configuration -SimpleCov.start do - add_filter "/test/" - add_group "Transports", ["lib/train/transports"] - add_group "Platforms", ["lib/train/platforms"] - add_group "Plugins", ["lib/train/plugins"] - minimum_coverage 80 +# test/helper.rb (actual) +if ENV["CI_ENABLE_COVERAGE"] + require "simplecov" + SimpleCov.formatters = SimpleCov::Formatter::MultiFormatter.new([ + SimpleCov::Formatter::HTMLFormatter, + ]) + SimpleCov.start do + add_filter "/test/" + end end ``` +So there is no build-breaking coverage floor; running tests without +`CI_ENABLE_COVERAGE` does not measure coverage at all. Treat >80% as a goal to +aim for, not a CI gate. (A dedicated `.expeditor/coverage.pipeline.yml` runs the +coverage job in CI.) + #### Test Structure Example ```ruby require "helper" @@ -98,8 +110,10 @@ Use `mocha` stubs only for third-party SDK/network boundaries — not for Train' own classes, which should be driven through their real interfaces. ### Coverage & style gates -- Maintain **> 80%** coverage (SimpleCov, configured in `test/helper.rb`). +- **Coverage is not gated.** SimpleCov runs only when `CI_ENABLE_COVERAGE` is set + and enforces no `minimum_coverage` (see "Coverage Configuration" above). >80% is + a goal, not a build gate. - Run **ChefStyle** before submitting: `chefstyle` (auto-fix with `chefstyle -a`). - `chefstyle` is loaded in the `Rakefile`. + `chefstyle` is loaded in the `Rakefile` — this *is* an enforced style gate. - Keep unit tests deterministic and offline; anything requiring sudo, SSH, WinRM, Docker, or a cloud/API credential belongs in the gated integration tasks. diff --git a/context/local/standards/workflow.md b/context/local/standards/workflow.md index d2fac0a1..3c041edd 100644 --- a/context/local/standards/workflow.md +++ b/context/local/standards/workflow.md @@ -1,4 +1,20 @@ +> [!IMPORTANT] +> **Correction / house process.** The content below is an *illustrative* +> JIRA-driven template; it is **not** the authoritative process for this repo. +> The real release/CI process is **Chef Expeditor + Buildkite + ChefStyle** — +> see [../background/release-and-ci.md](../background/release-and-ci.md). In +> particular: +> - **Version bumps and the changelog are automated by Expeditor** (label-driven); +> do not bump `lib/train/version.rb` or edit the changelog by hand in a feature PR. +> - **Test coverage is NOT gated.** SimpleCov runs only under `CI_ENABLE_COVERAGE` +> with no `minimum_coverage`; the ">80%" figures below are aspirational, not a +> CI threshold (see [testing.md](./testing.md)). +> - PRs are validated by the Buildkite **`verify`** pipeline (ChefStyle lint on +> Ruby 3.1; tests on Ruby 3.1 and 3.4, plus 3.1 on Windows). +> - JIRA/HTML-PR-body/prompt-confirmation steps below are optional local +> conventions, not required by the project. + ### JIRA Integration & Task Implementation Workflow When a JIRA ID is provided, follow this complete workflow: @@ -28,7 +44,7 @@ When a JIRA ID is provided, follow this complete workflow: #### 4. Unit Test Creation - **MANDATORY**: Create comprehensive unit test cases for all new code - Use Minitest framework (primary testing framework in this repo) -- Ensure test coverage is **> 80%** for the repository +- Aim for high coverage (a ~80% target is aspirational, **not** a CI gate) - Follow existing test patterns in `test/unit/` directories - Mock external dependencies appropriately using Mocha - Test both success and failure scenarios @@ -37,7 +53,7 @@ When a JIRA ID is provided, follow this complete workflow: #### 5. Test Execution & Validation - Run all unit tests to ensure they pass -- Verify test coverage meets the 80% threshold +- Review coverage locally if desired (no enforced threshold; see testing.md) - Fix any failing tests or coverage issues - Ensure no existing tests are broken by changes - Run integration tests when applicable @@ -120,7 +136,7 @@ When implementing a task, follow this prompt-based approach: - Create comprehensive unit tests - Run tests and verify coverage - Test platform compatibility when applicable - - **Prompt**: "Tests created and passing. Coverage verified > 80%. Next step: Code quality & linting. Ready to proceed? (y/n)" + - **Prompt**: "Tests created and passing. Coverage reviewed (aspirational, not gated). Next step: Code quality & linting. Ready to proceed? (y/n)" 4. **Code Quality & Linting** - Run ChefStyle linting: `chefstyle` and `chefstyle -a` @@ -215,7 +231,7 @@ Use MCP server functions to: 1. **JIRA Analysis** → Fetch and understand requirements (transport-specific) 2. **Planning** → Break down implementation approach (consider Train architecture) 3. **Implementation** → Code the solution following Train patterns -4. **Testing** → Create comprehensive tests (>80% coverage, platform compatibility) +4. **Testing** → Create comprehensive tests (aspirational ~80% coverage, not gated; platform compatibility) 5. **Code Quality** → Run ChefStyle linting and fix all issues 6. **PR Creation** → Use GitHub CLI with proper labeling 7. **Prompt-based** → Confirm each step before proceeding