dgdao-migrate

A struct-first schema migration framework for Dgraph. Define your schema as Go structs and dgdao-migrate scaffolds your next migration from their snapshots, then applies ordered, run-once schema and data changes to a dgdao database over an explicit revision chain — with resumable phased migrations and content checksums that make applied migrations immutable.

This guide is consumer-agnostic. Mount the commands in any Kong CLI through migratecli, or call the engine functions directly.

Install

go get github.com/dgraph-io/dgdao-migrate

The engine lives at …/dgdao-migrate/migrate; the migratecli command at …/dgdao-migrate/migrate/migratecli.

The model

A migration is one run-once change with a timestamp ID. It holds an ordered list of steps; each step makes an optional schema change, then runs an optional data transform. The runner records each step as it completes, so a crashed migration resumes at its first unapplied step.

migrate.Migration{
    ID:    20260601090000,
    After: 20260528000001,
    Name:  "add_mime_category",
    Steps: []migrate.Step{
        {Name: "schema", Schema: migrate.SchemaChange{EnsureSchema: frozen}},
        {Name: "backfill", Up: backfillMime},
    },
}

Every step must be idempotent. A schema Alter auto-commits, and the runner writes the step's record separately, so re-running a step after a crash must converge to the same end state.

The revision chain

Migrations are ordered by an explicit predecessor chain, not by sorting IDs. Each migration's After names the ID of the migration it follows. Exactly one migration — the baseline — has After == 0; it is the root. The runner builds the chain by walking After from the root and applies migrations in that order.

Explicit ordering catches the mistake an ID sort hides. Two developers on parallel branches each scaffold a migration; on merge, both set After to the same predecessor. That fork is a real ambiguity, so the runner rejects it rather than guess an order from timestamps.

Chain validation runs before any migration applies and returns a typed error on the first fault:

Fault	Error
Two migrations share an ID	*ErrDuplicateID
No migration has After == 0	*ErrNoRoot
More than one migration has After == 0	*ErrMultipleRoots
An After names no registered migration	*ErrUnknownPredecessor
Two migrations share a predecessor	*ErrDivergentHistory
The After links form a loop	*ErrCycle

Resolve divergence by re-pointing one migration's After at the other, which linearizes the two into a sequence. There are no merge nodes.

After is structural, so the migration checksum covers it. Editing After on a migration already applied to a database trips ErrChecksumMismatch, the same immutability guard that protects the schema.

Schema changes

A step's SchemaChange carries exactly one of three fields:

• Ensure []any — struct templates whose derived schema applies additively. The runner re-derives both the applied schema and the checksum from the live structs at run time, so an Ensure step drifts as those structs evolve. Use it for throwaway or bootstrap schema.
• EnsureSchema string — a frozen schema string that applies additively. MarshalSchema renders the current structs to this string once; stored verbatim, it never drifts. Use it for any migration that must stay reproducible after it ships, including the baseline.
• Alter string — a raw Dgraph schema string, for changes the additive forms cannot express: rename, drop, retype.

Ensure and EnsureSchema apply the same predicates; only the freezing differs. Freeze anything that ships.

Declare each predicate consistently

A directive like @index or @reverse is a property of the predicate, not of an individual edge — Dgraph stores one definition per predicate. So when several structs declare the same predicate (for example, many types holding a tenant edge), every one of them must give it identical directives. If they disagree, there is no single correct schema to render, so MarshalSchema returns an error naming the predicate and the structs on each side rather than silently picking one. Picking one would also hide the disagreement: a later edit to the struct whose declaration "lost" would never change the output. Make the declarations agree — declare reverse on every edge of the predicate, or none.

Immutability

Each step's checksum covers its identity and its schema portion, never its Up/Down closures. Each migration's checksum covers its ID, its After, and its steps' checksums. Editing, reordering, adding, or removing a step in an already-applied migration changes the checksum, and the runner rejects it with ErrChecksumMismatch. Correct a mistake with a new migration; never edit a shipped one.

Commands

Mount migratecli.MigrateCmd in a Kong CLI and bind a Provider:

type Provider interface {
    Client() mg.Client
    Migrations() []migrate.Migration
    Models() []any
}

Runtime commands use Client and Migrations; authoring commands use Models.

Command	Purpose
up	Apply all pending migrations, resuming any partial one.
down <version>	Roll back every migration after <version>, head first.
status	Show applied, in-progress, and pending migrations.
version	Print the highest applied migration ID.
history [--tree] [--verbose]	Show the chain and flag a broken history.
create <name>	Scaffold the next migration from the current structs.
diff [--check]	Show, or check for, the drift the next migration would capture.
snapshot	Re-sync the desired-state snapshot to the current structs.
verify	Check the live schema against the current structs.

history prints even when the chain is broken — showing the fork is its point — and exits non-zero on any structural fault, so it doubles as a CI chain lint. diff --check and verify also exit non-zero on drift.

Retyping a predicate

RetypePredicate changes a predicate's scalar type without data loss. It expands a RetypeSpec into five staged, checkpointed, idempotent steps: stage the converted values under a temporary predicate, verify the counts, then swap. The operation is irreversible — every step's Down is nil — so the runner refuses any down range that includes it.

steps := migrate.RetypePredicate(migrate.RetypeSpec{
    Predicate: "size",
    To:        migrate.Int,
    Index:     "int",
    Convert:   func(old string) (any, error) { return strconv.Atoi(old) },
})

Scaffolding

Scaffolding writes the next migration by comparing the current structs against a checked-in desired-state snapshot.

The model aggregate. Tooling needs one value per schema entity type. dgdao-gen emits schema.Models() []any for exactly this; supply it through Provider.Models(). Because the generator owns the list, it cannot fall out of sync with the declared types.

The desired-state file. <migrations-dir>/schema_state.schema holds the full canonical schema as of the latest scaffold. It is checked in and tool-managed: create advances it, and snapshot re-syncs it. Seed it once at adoption with snapshot.

The diff. Both sides are canonical MarshalSchema output, so the diff is a stable set difference over predicate declarations and over each type's field list. It classifies each change:

Change	Condition	Action
Added	predicate present now, absent in state	emit into the EnsureSchema delta
Index changed	same predicate, index/directives differ	emit into the delta
Type changed	same predicate, scalar type differs	flag for RetypePredicate; never emitted
Removed	predicate in state, absent now	flag; never auto-dropped
Type added	type present now, absent in state	emit the full type definition
Type fields changed	same type, field list differs	emit the full type definition; note each field it drops
Type removed	type in state, absent now	flag; never auto-dropped

Type membership is diffed on its own because Dgraph stores each type's field list separately from the predicates: adding an existing predicate to a type is a schema change even though no predicate declaration changes. Dgraph replaces a whole type on alter, so the .schema file carries the complete current definition of every added or changed type, after the predicate lines, never a partial one. A field dropped from a type is removed from it by that definition, so the notes name it: the predicate and its data stay, but the field no longer appears in expand(_all_) for that type.

If the delta is purely flagged, the generated .schema is empty and the step is a stub carrying only the action-required notes. The scaffolder never emits an unsafe EnsureSchema.

Create. create <name> validates the chain (aborting on a broken history), diffs the structs against the snapshot, sets After to the current head, writes <id>_<name>.go and <id>_<name>.schema, advances the snapshot, and appends the new variable to the All slice. It writes nothing until validation passes, and fails with an actionable message when it runs outside a Go project root or finds no migrations directory.

Engine API.

func Scaffold(p ScaffoldParams) (ScaffoldReport, error) // create
func Snapshot(p ScaffoldParams) (string, error)         // re-sync the snapshot
func Diff(dir string, models []any) Delta               // offline drift
func Verify(ctx context.Context, c mg.Client, models []any) (Drift, error)

Drift gates

Two gates reuse the diff engine and catch different failures.

• Offline — diff --check. Exits non-zero when the structs have drifted from the snapshot, including a change to type membership alone. It needs no database and runs in make check, the gofmt -l idiom for migrations.
• Live — verify. Exits non-zero when the database lacks a predicate the structs declare, lacks a type they declare, lacks a field in one of those types, or — against a real Dgraph — defines a predicate differently. Type checks work against the embedded engine as well. Run it after up in CI. verify is one-directional: it ignores predicates, types, and type fields the database has but the structs do not. Pass a client with auto-schema disabled, so the check reflects what migrations applied rather than what an auto-schema client would re-create.

Upgrading from v0.2.4 or earlier. Those versions recorded types in the snapshot without emitting them in the migration, so a type or type field added with them may be missing from the database. verify then reports MISSING TYPE or MISSING FIELD, while diff and create see no delta. Fix it with a hand-written migration whose EnsureSchema carries the full definition of each type verify reports.

Related projects

Part of the dgdao family:

• dgdao — core typed client and DAO library for Dgraph
• dgdao-gen — code generator and wrapper-entity runtime
• dgdao-telemetry — OpenTelemetry instrumentation for the dgdao typed client