Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 50 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,56 @@ Set workspace hosts and Unity Catalog grants per `docs/setup.md` in the rendered

If the bundle was scaffolded inside a Databricks **Git folder** in the workspace (the recommended path for Genie Code users), the workspace UI also surfaces a **Deployments panel** on the bundle that lets you pick a target and deploy with one click — no terminal required. This matches the layout produced by the workspace UI's native "Create → Bundle" flow.

> **Coming next:** production lifecycle patterns (evaluation gates, governance posture, monitoring, feedback loops) are in active design — they'll ship as Innovate-driven workflows on top of the scaffold. Subscribe to releases for updates.
---

## Production lifecycle

The scaffold generates the envelope. The **AgentOps Lifecycle** skill guides
the project through the complete Single-Account Single-Agent lifecycle — from
first data ingestion to production monitoring — across three phases and 10 steps:

| Phase | Steps | What happens |
|-------|-------|--------------|
| **Dev** | 1–5 | Data prep + Vector Search indexing, agent implementation with MLflow tracing, offline eval gate, SME calibration |
| **Staging** | 6–7 | CI gate (unit tests + bundle validate + eval gate on PR), staging deploy + integration tests |
| **Production** | 8–10 | CD deploy, batch inferencing baseline, real-time monitoring + SME feedback loop |

MLflow is the operational spine at every level: experiment tracking in dev,
eval gate in CI, trace logging in prod, user feedback via `mlflow.log_feedback()`.
The `evaluation/gate.py` pattern (generated by the scaffold) runs at three

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same path issue here. Also: main intentionally trimmed this section to a one-line "Coming next" stub three days before this branch - worth a quick confirm that re-expanding it now is the call, vs. keeping the README lean and letting the SKILL carry the detail.

points — locally in dev, in CI on every PR, and against production data before
users are admitted — so quality regressions are caught before they reach users.

### Using the lifecycle skill

Install the plugin, then ask your coding assistant to continue from where the
scaffold left off:

```bash
# Install (Claude Code / Cursor)
./plugin/skills/install_skills.sh

# Then in the assistant:
/agentops-lifecycle
# or: "walk me through the agentops lifecycle"
```

In **Genie Code**, run `install_genie_code_skills.py` as a notebook to install,
then say "walk me through the agentops lifecycle."

The skill provides step-by-step agent actions, copy-paste code examples
grounded in the generated project files, validation criteria, and common issue
resolutions for each of the 10 steps.

### Machine-readable workflow definition

`workflows/single-account-single-agent.json` is a structured definition of the
10-step lifecycle with all actions, validations, escalation hints, and phase
metadata. It can be consumed programmatically by workflow engines (e.g.,
`/innovate`) or used as a reference when building custom automation on top of
the scaffold.

---

## Documentation

Expand Down
40 changes: 29 additions & 11 deletions plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,27 @@ A portable plugin that scaffolds AgentOps Stacks projects from inside a coding a

The plugin and the [DAB template](../template/) share the same scaffold contract (`.agentops-stacks/manifest.yml`). The CLI alone is sufficient for scaffolding; the plugin is the additive on-ramp for users working inside a coding assistant.

## What this plugin ships today
## What this plugin ships

- One skill: **`agentops-stacks`** — scaffolds a new project (DAB layout, dev/staging/prod targets, UC schema and volume, MLflow experiment, CI/CD wiring for one of GitHub Actions, GitHub Actions for GHES, GitLab, or Azure DevOps).
- One command: **`/init-agentops-stacks`** — discoverability wrapper around the skill for Claude Code / Cursor users.
- Installers under `skills/` that mirror [ai-dev-kit's pattern](https://github.com/databricks-solutions/ai-dev-kit#installation):
- `install_skills.sh` — CLI installer. Lands the skill into `.claude/skills/agentops-stacks/` locally, with optional `--install-to-genie` to also upload to the workspace.
- `install_genie_code_skills.py` — Databricks notebook for in-workspace installs without a local clone.
### Skills

Production patterns (evaluation gates, governance posture, monitoring, feedback loops) are not in this plugin yet. They land as separate skills as the project matures.
| Skill | Trigger | What it does |
|-------|---------|--------------|
| **`agentops-stacks`** | `scaffold a new agentops project` | Scaffolds a new project (DAB layout, dev/staging/prod targets, UC schema and volume, MLflow experiment, CI/CD wiring). One-time use at project start. |
| **`agentops-lifecycle`** | `walk me through the agentops lifecycle` | Guides an existing scaffold through the complete Single-Account Single-Agent lifecycle — data prep, agent dev, eval gate, SME calibration, CI/CD promotion, batch eval baseline, and production monitoring. 10 steps across dev → staging → prod. |

### Commands

| Command | Skill |
|---------|-------|
| `/init-agentops-stacks` | `agentops-stacks` |
| `/agentops-lifecycle` | `agentops-lifecycle` |

### Installers

Under `skills/`, mirroring [ai-dev-kit's pattern](https://github.com/databricks-solutions/ai-dev-kit#installation):
- `install_skills.sh` — CLI installer. Lands skills into `.claude/skills/` locally, with optional `--install-to-genie` to also upload to the workspace.
- `install_genie_code_skills.py` — Databricks notebook for in-workspace installs without a local clone.

## Prerequisites

Expand Down Expand Up @@ -82,15 +94,21 @@ Prerequisites: `databricks` CLI installed, a coding assistant that loads `.claud
plugin/
├── README.md # this file
├── commands/
│ └── init-agentops-stacks.md # CC/Cursor slash command
│ ├── init-agentops-stacks.md # scaffold command
│ └── agentops-lifecycle.md # lifecycle command
└── skills/
├── install_skills.sh # local + Genie upload installer
├── install_genie_code_skills.py # in-workspace notebook installer
└── agentops-stacks/
└── SKILL.md # skill instructions for the assistant
├── agentops-stacks/
│ └── SKILL.md # scaffold skill
└── agentops-lifecycle/
└── SKILL.md # lifecycle skill (10-step dev→prod guide)
```

The skill is a single `SKILL.md`. There's no Python renderer and no vendored template tree — `databricks bundle init` reads the template directly from this repo at scaffold time.
Each skill is a single `SKILL.md`. There's no Python renderer and no vendored
template tree — `databricks bundle init` reads the template directly from this
repo at scaffold time, and the lifecycle skill references generated project
files by their scaffold-relative paths.

## Known UX notes

Expand Down
22 changes: 22 additions & 0 deletions plugin/commands/agentops-lifecycle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
description: >
Guide an agentops-stacks project through its complete production lifecycle —
data prep, agent dev, eval gates, CI/CD promotion, and production monitoring.
Run after `databricks bundle init` (or `/init-agentops-stacks`).
---

Use the `agentops-lifecycle` skill.

This command guides an existing agentops-stacks scaffold through the
Single-Account Single-Agent lifecycle:

- Steps 1–5 (Dev): data preparation, agent implementation, offline evaluation,
SME calibration
- Steps 6–7 (Staging): CI gate, integration tests
- Steps 8–10 (Production): CD deploy, batch inferencing baseline, monitoring

**Prerequisite:** `.agentops-stacks/manifest.yml` must exist. Run
`/init-agentops-stacks` first if you haven't scaffolded yet.

Defer to the skill's SKILL.md for full step-by-step guidance, code examples,
validation criteria, and common issue resolutions.
Loading