diff --git a/tutorials/following_a_git_commit_to_runtime_environments.md b/tutorials/following_a_git_commit_to_runtime_environments.md index 6e604d7..1d94e65 100644 --- a/tutorials/following_a_git_commit_to_runtime_environments.md +++ b/tutorials/following_a_git_commit_to_runtime_environments.md @@ -4,7 +4,7 @@ description: "In this 5 minute tutorial you'll learn how Kosli tracks \"life aft --- We will follow an actual git commit from a CI pipeline all the way into production runtime environments. -By the end, you will have queried Kosli to see an artifact's full history — from creation through deployment to scaling and shutdown — without any access to the production environment. +By the end, you will have queried Kosli to see an artifact's history — from creation in CI through running in production to eventual shutdown — without any access to the production environment. We will use **cyber-dojo**, an open-source microservice platform whose Kosli data is public. The commit we follow fixed a misconfiguration: `runner` should run with three replicas but was accidentally running with one after a migration from GKE to ECS. @@ -38,56 +38,61 @@ kosli list flows You will see: ```plaintext -NAME DESCRIPTION VISIBILITY -creator UX for Group/Kata creation public -custom-start-points Custom exercises choices public -dashboard UX for a group practice dashboard public -differ Diff files from two traffic-lights public -exercises-start-points Exercises choices public -languages-start-points Language+TestFramework choices public -nginx Reverse proxy public -repler REPL for Python images public -runner Test runner public -saver Group/Kata model+persistence public -version-reporter UX for git+image version-reporter public -web UX for practicing TDD public +NAME DESCRIPTION VISIBILITY TAGS +creator-ci UX for Group/Kata creation private [ci=github], [repo_url=https://github.com/cyber-dojo/creator], [kind=build], [env=aws-beta] +custom-start-points-ci Custom exercises choices private [env=aws-beta], [ci=github], [repo_url=https://github.com/cyber-dojo/custom-start-points], [kind=build] +dashboard-ci UX for a group practice dashboard private [ci=github], [repo_url=https://github.com/cyber-dojo/dashboard], [kind=build], [env=aws-beta] +differ-ci Diff files from two traffic-lights private [env=aws-beta], [ci=github], [repo_url=https://github.com/cyber-dojo/differ], [kind=build] +differ-ci-tf Terraform human-readable plan and state file fingerprint private +docker-base-ci Build cyber-dojo/docker-base image private +exercises-start-points-ci Exercises choices private [ci=github], [repo_url=https://github.com/cyber-dojo/exercises-start-points], [kind=build], [env=aws-beta] +languages-start-points-ci Language+TestFramework choices private [ci=github], [repo_url=https://github.com/cyber-dojo/languages-start-points], [kind=build], [env=aws-beta] +nginx-ci Reverse proxy private [kind=build], [env=aws-beta], [ci=github], [repo_url=https://github.com/cyber-dojo/nginx] +production-promotion Promotes sets of Artifacts from aws-beta to aws-prod private [ci=github], [repo_url=https://github.com/cyber-dojo/aws-prod-co-promotion], [kind=release], [env=aws-prod] +production-server-access Flow to track production server access private +runner-ci Test runner private [ci=github], [repo_url=https://github.com/cyber-dojo/runner], [kind=build], [env=aws-beta] + +...some output elided... ``` ## Follow the artifact -The commit that fixed the replica count was [16d9990](https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4) in the `runner` repository. Fetch its full history from Kosli: +The commit that fixed the replica count was [16d9990](https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4) in the `runner` repository. Fetch its history from Kosli with `kosli search`, which accepts a git commit (full or short-form) or an artifact fingerprint: ```shell -kosli get artifact runner:16d9990 +kosli search 16d9990 ``` You will see: ```plaintext -Name: cyberdojo/runner:16d9990 -Flow: runner -Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 -Created on: Mon, 22 Aug 2022 11:35:00 CEST • 15 days ago -Git commit: 16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 -Commit URL: https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 -Build URL: https://github.com/cyber-dojo/runner/actions/runs/2902808452 -State: COMPLIANT +Search result resolved to commit 16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 +Name: cyberdojo/runner:16d9990 +Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 +Has provenance: true +Flow: runner-archived-at-1709658802 +Git commit: 16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 +Commit URL: https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 +Build URL: https://github.com/cyber-dojo/runner/actions/runs/2902808452 +Artifact URL: https://app.kosli.com/cyber-dojo/flows/runner-archived-at-1709658802/artifacts/9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 +Compliance state: COMPLIANT +Running in: [ ] +Exited from: [ aws-beta, aws-prod ] History: - Artifact created Mon, 22 Aug 2022 11:35:00 CEST - branch-coverage evidence received Mon, 22 Aug 2022 11:36:02 CEST - Deployment #18 to aws-beta environment Mon, 22 Aug 2022 11:37:17 CEST - Deployment #19 to aws-prod environment Mon, 22 Aug 2022 11:38:21 CEST - Started running in aws-beta#84 environment Mon, 22 Aug 2022 11:38:28 CEST - Started running in aws-prod#65 environment Mon, 22 Aug 2022 11:39:22 CEST - Scaled down from 3 to 2 in aws-beta#117 environment Wed, 24 Aug 2022 18:03:42 CEST - No longer running in aws-beta#119 environment Wed, 24 Aug 2022 18:05:42 CEST - Scaled down from 3 to 1 in aws-prod#94 environment Wed, 24 Aug 2022 18:10:28 CEST - No longer running in aws-prod#96 environment Wed, 24 Aug 2022 18:12:28 CEST + Artifact created Mon, 22 Aug 2022 11:35:00 CEST + Started running in aws-beta#84 environment Mon, 22 Aug 2022 11:38:28 CEST + Started running in aws-prod#65 environment Mon, 22 Aug 2022 11:39:22 CEST + No longer running in aws-beta#119 environment Wed, 24 Aug 2022 18:05:42 CEST + No longer running in aws-prod#96 environment Wed, 24 Aug 2022 18:12:28 CEST ``` -The **History** shows the artifact's complete lifecycle: created by CI, evidence attested, deployed to both environments, running with the correct 3 replicas, then eventually scaled down and replaced by a newer version. The state `COMPLIANT` means all required evidence was provided before deployment. + +When this commit was made, the runner repository reported to a flow simply named `runner`. cyber-dojo's flows have since been reorganized (today the repository reports to `runner-ci`, as the flow list above shows) and the original flows archived. Archiving a flow currently renames it by appending `-archived-at-`, which is why the historical evidence displays the longer name. + + +The **History** shows the artifact's lifecycle: created by CI, running in both environments, and eventually replaced by a newer version. `Has provenance: true` means the artifact was reported to Kosli by a CI pipeline, so its build history is known. The compliance state `COMPLIANT` means all required evidence was provided before deployment. -The same information is available in the [Kosli web interface](https://app.kosli.com/cyber-dojo/flows/runner/artifacts/9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625). +The same information is available in the [Kosli web interface](https://app.kosli.com/cyber-dojo/flows/runner-archived-at-1709658802/artifacts/9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625). ## Inspect the environment snapshot @@ -100,19 +105,19 @@ kosli get snapshot aws-prod#65 The output will be: ```plaintext -COMMIT ARTIFACT FLOW RUNNING_SINCE REPLICAS -16d9990 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:16d9990 runner 11 days ago 3 +COMMIT ARTIFACT FLOW COMPLIANCE RUNNING_SINCE REPLICAS +16d9990 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:16d9990 runner-archived-at-1709658802 COMPLIANT 2022-08-22 3 Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 -7c45272 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/shas:7c45272 shas 11 days ago 1 +7c45272 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/shas:7c45272 shas-archived-at-1705491385 COMPLIANT 2022-08-22 1 Fingerprint: 76c442c04283c4ca1af22d882750eb960cf53c0aa041bbdb2db9df2f2c1282be ...some output elided... -85d83c6 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:85d83c6 runner 13 days ago 1 +85d83c6 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:85d83c6 runner-archived-at-1709658802 COMPLIANT 2022-08-20 1 Fingerprint: eeb0cfc9ee7f69fbd9531d5b8c1e8d22a8de119e2a422344a714a868e9a8bfec -1a2b170 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/differ:1a2b170 differ 13 days ago 1 +1a2b170 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/differ:1a2b170 differ-archived-at-1707630536 COMPLIANT 2022-08-20 1 Fingerprint: d8440b94f7f9174c180324ceafd4148360d9d7c916be2b910f132c58b8a943ae ``` @@ -133,9 +138,10 @@ Only present in aws-prod#65 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:16d9990 Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 - Flow: runner + Flow: runner-archived-at-1709658802 Commit URL: https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 - Started: Mon, 22 Aug 2022 11:39:17 CEST • 15 days ago + Started: Mon, 22 Aug 2022 11:39:17 CEST • 2022-08-22 + Instances: 3 ``` This confirms that `runner:16d9990` is the only new artifact in snapshot 65 — exactly the commit that fixed the replica count. diff --git a/tutorials/tracing_a_production_incident_back_to_git_commits.md b/tutorials/tracing_a_production_incident_back_to_git_commits.md index 672967d..aee9ae0 100644 --- a/tutorials/tracing_a_production_incident_back_to_git_commits.md +++ b/tutorials/tracing_a_production_incident_back_to_git_commits.md @@ -40,48 +40,56 @@ kosli log env aws-prod --interval 176..177 You should see: ```plaintext -SNAPSHOT EVENT FLOW DEPLOYMENTS -#177 Artifact: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/creator:31dee35 creator #87 +SNAPSHOT EVENT FLOW +#177 Artifact: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/creator:31dee35 creator-archived-at-1707630496 Fingerprint: 5d1c926530213dadd5c9fcbf59c8822da56e32a04b0f9c774d7cdde3cf6ba66d - Description: 1 instance stopped running (from 1 to 0). + Description: 1 instance stopped running (from 1 to 0) Reported at: Tue, 06 Sep 2022 16:53:28 CEST -#176 Artifact: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/creator:b7a5908 creator #89 +#176 Artifact: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/creator:b7a5908 creator-archived-at-1707630496 Fingerprint: 860ad172ace5aee03e6a1e3492a88b3315ecac2a899d4f159f43ca7314290d5a - Description: 1 instance started running (from 0 to 1). + Description: 1 instance started running (from 0 to 1) Reported at: Tue, 06 Sep 2022 16:52:28 CEST ``` + +When this incident happened the flow was simply named `creator`. The flow has since been archived, and archiving a flow currently renames it by appending `-archived-at-`. The historical evidence is unchanged; only the displayed name is longer. + + These two snapshots are part of the same blue-green deployment: `creator:b7a5908` started in snapshot #176, and `creator:31dee35` stopped in snapshot #177. The new artifact arrived just before the 500 error — that is the one to investigate. ## Dig into the artifact -Get the full history of `creator:b7a5908`, using the fingerprint prefix from snapshot #176: +Get the full history of `creator:b7a5908` with `kosli search`, using the fingerprint prefix from snapshot #176: ```shell -kosli get artifact creator@860ad17 +kosli search 860ad17 ``` You should see: ```plaintext -Name: cyberdojo/creator:b7a5908 -Flow: creator -Fingerprint: 860ad172ace5aee03e6a1e3492a88b3315ecac2a899d4f159f43ca7314290d5a -Created on: Tue, 06 Sep 2022 16:48:07 CEST • 21 hours ago -Git commit: b7a590836cf140e17da3f01eadd5eca17d9efc65 -Commit URL: https://github.com/cyber-dojo/creator/commit/b7a590836cf140e17da3f01eadd5eca17d9efc65 -Build URL: https://github.com/cyber-dojo/creator/actions/runs/3001102984 -State: COMPLIANT +Search result resolved to artifact with fingerprint 860ad172ace5aee03e6a1e3492a88b3315ecac2a899d4f159f43ca7314290d5a +Name: cyberdojo/creator:b7a5908 +Fingerprint: 860ad172ace5aee03e6a1e3492a88b3315ecac2a899d4f159f43ca7314290d5a +Has provenance: true +Flow: creator-archived-at-1707630496 +Git commit: b7a590836cf140e17da3f01eadd5eca17d9efc65 +Commit URL: https://github.com/cyber-dojo/creator/commit/b7a590836cf140e17da3f01eadd5eca17d9efc65 +Build URL: https://github.com/cyber-dojo/creator/actions/runs/3001102984 +Artifact URL: https://app.kosli.com/cyber-dojo/flows/creator-archived-at-1707630496/artifacts/860ad172ace5aee03e6a1e3492a88b3315ecac2a899d4f159f43ca7314290d5a +Compliance state: COMPLIANT +Running in: [ ] +Exited from: [ aws-beta, aws-prod ] History: Artifact created Tue, 06 Sep 2022 16:48:07 CEST - Deployment #88 to aws-beta environment Tue, 06 Sep 2022 16:49:59 CEST - Deployment #89 to aws-prod environment Tue, 06 Sep 2022 16:51:12 CEST Started running in aws-beta#196 environment Tue, 06 Sep 2022 16:51:42 CEST Started running in aws-prod#176 environment Tue, 06 Sep 2022 16:52:28 CEST + No longer running in aws-beta#199 environment Tue, 06 Sep 2022 21:28:42 CEST + No longer running in aws-prod#179 environment Tue, 06 Sep 2022 21:30:28 CEST ``` -The artifact was deployed to `aws-prod` at 16:51 — right when the incident began. The output includes a direct link to the git commit. +The artifact started running in `aws-prod` at 16:52 — right when the incident began. The output includes a direct link to the git commit. (You can also see the artifact exiting both environments later that evening, once the incident was fixed by a newer commit.) ## Follow to the commit