Back up Roo Code conversation history on the Mac before removing the extension

[ppat]

Context

Roo Code (RooVeterinaryInc.roo-cline) is retired in favor of Claude Code. Before it's removed anywhere, every conversation it holds needs a verified-complete backup — LLM session history is unrecoverable if lost.

The workspace side (Coder, remote ~/.vscode-server) is already done: two verified archives + a sha256 manifest live in ~/code/roo-history/ on that box, produced by the script below. This issue tracks doing the same thing on the Mac before Roo is uninstalled there, using the identical script (it takes the source directory as an argument, so it runs unchanged — see the macOS-specific notes below).

Do not skip verification. "I copied the folder" is not the bar here — the bar is "I can prove nothing was dropped or corrupted." Run the full script and read its PASS/FAIL output before deleting anything.

Two gotchas found on the workspace side — check both hold on the Mac too

1. tasks/_index.json under-counts tasks. On the Coder workspace, _index.json ({"version":.., "updatedAt":.., "entries":[...]}) listed only 23 of the 42 actual task directories under tasks/. Anything that enumerates conversations via _index.json — instead of listing tasks/ on disk — will silently miss the other 19. The export script below always lists tasks/ directly and never touches _index.json. Verify the same gap exists (or doesn't) on the Mac before trusting any tool that relies on the index.
2. No state.vscdb / hidden SQLite globalState. On the workspace side there is no state.vscdb anywhere under ~/.vscode-server — Roo's task data lives entirely as flat files under globalStorage/rooveterinaryinc.roo-cline/tasks/<task-id>/. VS Code desktop (which is what runs on the Mac) does keep a state.vscdb per profile, and Roo may keep additional state there (e.g. taskHistory in extension globalState) that has no on-disk file equivalent. This has not been checked on the Mac and can't be checked remotely — before calling the Mac backup complete, look for state.vscdb under ~/Library/Application Support/Code/User/globalStorage/ (and any profile subdirectories) and, if Roo has entries there, export them too (e.g. sqlite3 state.vscdb "select value from ItemTable where key like '%roo%'") — the script below does not do this for you.

Source path (macOS)

~/Library/Application Support/Code/User/globalStorage/rooveterinaryinc.roo-cline

(Layout should match the workspace side: tasks/<task-id>/{api_conversation_history.json, ui_messages.json, history_item.json?, task_metadata.json?, cmd-*.txt?}, plus cache/ and settings/.)

Steps followed on the workspace side (repeat on the Mac)

1. Confirmed the source directory exists and sized it (du -sh).
2. Independently verified both gotchas above with ad hoc commands (task-dir count vs. _index.json entry count; find ... -iname '*state.vscdb*') rather than assuming they still held.
3. Ran an integrity pass over every JSON file in tasks/ (json.load in a loop) — zero parse failures, zero zero-byte files.
4. Ran roo-export.sh <source-dir> (script below). It:
   • Refuses to run if the source is missing or tasks/ is empty.
   • Copies the source tree byte-for-byte into a temp dir first (cp -a), then builds everything else from that copy — the readable export never re-reads the source, so copying and rendering are cleanly separated concerns.
   • Writes roo-history-raw-<date>.tar.gz (untouched tree, including cache/ and settings/).
   • Builds a human-readable export from the copy — one Markdown + one JSONL file per task (ordered turns, roles labelled, tool calls/results rendered), plus INDEX.md covering every task directory (not just what _index.json lists) — and tars it as roo-history-readable-<date>.tar.gz.
   • Verifies, and fails loudly (non-zero exit, FAIL: lines) if any of these don't hold:
     • task-directory count matches across the live source, the copy, and the readable export;
     • every JSON file in the copy parses;
     • a sha256 manifest of every file under the source (incl. cache/, settings/) matches a manifest of the copy;
     • the raw tarball extracts and its contents match that same manifest (catches a corrupted/truncated archive, not just a corrupted copy).
   • Also writes roo-history-sha256-manifest-<date>.txt into the destination as a standalone audit trail.
5. Before trusting the script's own verdict, cross-checked it with tools outside the script: diff -rq <extracted raw tarball> <live source> → IDENTICAL, and find <source> -newer roo-export.sh → empty (confirms nothing under the source was touched).
6. Self-tested the verification logic itself against a synthetic decoy directory before running it for real: confirmed it refuses on a missing/empty source, confirmed it goes red when a JSON file is corrupted, confirmed a sha256 manifest diff catches a single flipped byte in an otherwise-identical copy, confirmed a truncated tarball fails to extract, and confirmed the task-count check would have caught the exact _index.json under-count bug from gotcha feat: setting up dotfiles - first pass #1 had the script relied on the index instead of the filesystem.

Real run against the workspace's 42-task, 60MB source: all checks PASS, raw archive ~14MB, readable archive ~11MB, 162 files in the sha256 manifest (142 JSON files across tasks/, plus cache/, settings/, _index.json).

macOS-specific gotchas to watch for

The script is already written to be portable across Linux and macOS's stock (BSD-flavored) tools, but these are the specific traps that were designed around — sanity-check them if the script errors out on the Mac:

• sha256sum doesn't exist on macOS. The script tries sha256sum first, then falls back to shasum -a 256 (stock on macOS). No action needed unless both are somehow missing.
• find -printf is GNU-only and isn't available on BSD find (macOS default). The script avoids it entirely, using -print0 + a while read -d '' loop instead, which both find implementations support.
• stat --format / stat -c (GNU) vs. stat -f (BSD) have incompatible syntax. The script avoids stat entirely — file sizes are read with wc -c, which is consistent across both.
• The space in "Application Support" needs quoting in every reference to the source path — e.g. roo-export.sh "$HOME/Library/Application Support/Code/User/globalStorage/rooveterinaryinc.roo-cline". The script itself quotes "$SRC_DIR" throughout, but the invocation is on you.
• tar differences: macOS ships bsdtar as /usr/bin/tar; the script only uses the lowest-common-denominator flags (czf, xzf, -C), which both bsdtar and GNU tar support identically — no --sort, --mtime, or other GNU-only flags are used.
• cp -a is supported by both GNU cp and BSD cp (on BSD it's equivalent to -RpP), so no change needed there.
• date +%Y%m%d uses only basic strftime specifiers, which BSD date and GNU date both support identically (no -d/--date relative-date parsing is used).
• Bash version: macOS ships bash 3.2 (last GPLv2 release; Apple won't ship GPLv3 bash). The script was deliberately written against 3.2 — no associative arrays (declare -A), no mapfile/readarray, no ${var,,} case conversion, nothing that only exists in bash 4+. If you have a newer bash from Homebrew on $PATH ahead of /bin/bash, that's fine too; the script doesn't require 3.2, it just doesn't require anything newer.
• python3 must be on PATH. The script uses it for JSON parsing and rendering (both stdlib-only: json, datetime, glob, os — no third-party packages). Recent macOS doesn't ship Python by default; if python3 isn't found the script exits immediately with a clear error rather than failing partway through. Install via Xcode Command Line Tools (xcode-select --install) or Homebrew (brew install python3) if needed.
• shellcheck-clean: the script passes shellcheck with no warnings (checked with v0.11.0). If you modify it, re-run shellcheck roo-export.sh before trusting a modified version with irreplaceable data.

Invocation on the Mac

chmod +x roo-export.sh
./roo-export.sh "$HOME/Library/Application Support/Code/User/globalStorage/rooveterinaryinc.roo-cline"
# writes into ~/code/roo-history/ by default; pass a second argument to override

Read the full PASS/FAIL summary it prints. Only proceed to uninstall Roo once every line says PASS and the script exits 0.

The script

#!/usr/bin/env bash
# roo-export.sh — back up a Roo Code (rooveterinaryinc.roo-cline) globalStorage
# directory into a raw tarball and a human-readable tarball, with verification
# that both are complete and byte-accurate against the source.
#
# Usage:
#   roo-export.sh <source-dir> [dest-dir]
#
#   <source-dir>  Roo's globalStorage directory, e.g.
#                 Linux/Coder:  ~/.vscode-server/data/User/globalStorage/rooveterinaryinc.roo-cline
#                 macOS:        ~/Library/Application Support/Code/User/globalStorage/rooveterinaryinc.roo-cline
#   [dest-dir]    Where to write the archives. Defaults to ~/code/roo-history.
#
# The script is read-only on <source-dir>. It never deletes or modifies
# anything there.
#
# What it does:
#   1. Copies the source directory tree byte-for-byte into a raw tarball
#      (roo-history-raw-<date>.tar.gz), including cache/ and settings/.
#   2. Derives a human-readable export (one Markdown + one JSONL file per
#      task, plus an INDEX.md covering every task directory) from that copy
#      — never re-reading the source — and tars it as
#      roo-history-readable-<date>.tar.gz.
#   3. Verifies: task-directory counts (source vs. copy vs. readable output),
#      that every JSON file in the copy parses, and a sha256 manifest of
#      every file under the source vs. the copy vs. the extracted raw
#      tarball. Any mismatch is a hard failure (non-zero exit).
#
# Roo's own tasks/_index.json is NOT used to enumerate tasks — on at least
# one real installation it was missing 19 of 42 task directories. This
# script always enumerates tasks/ directly from the filesystem.

set -euo pipefail

usage() {
  echo "Usage: $0 <source-dir> [dest-dir]" >&2
  echo "  <source-dir>  Roo's globalStorage/rooveterinaryinc.roo-cline directory" >&2
  echo "  [dest-dir]    default: \$HOME/code/roo-history" >&2
}

SRC_DIR="${1:-}"
if [ -z "$SRC_DIR" ]; then
  usage
  exit 1
fi
# Strip any trailing slash so basename is well-defined.
SRC_DIR="${SRC_DIR%/}"
DEST_DIR="${2:-$HOME/code/roo-history}"

if ! command -v python3 >/dev/null 2>&1; then
  echo "ERROR: python3 is required (used for JSON parsing / rendering) but was not found on PATH" >&2
  exit 1
fi

if [ ! -d "$SRC_DIR" ]; then
  echo "ERROR: source directory does not exist: $SRC_DIR" >&2
  exit 1
fi
if [ ! -d "$SRC_DIR/tasks" ]; then
  echo "ERROR: no tasks/ subdirectory under source: $SRC_DIR" >&2
  exit 1
fi

SRC_TASK_COUNT=$(find "$SRC_DIR/tasks" -mindepth 1 -maxdepth 1 -type d | wc -l | tr -d ' ')
if [ "$SRC_TASK_COUNT" -eq 0 ]; then
  echo "ERROR: source tasks/ directory is empty — refusing to export nothing" >&2
  exit 1
fi

echo "Source: $SRC_DIR ($SRC_TASK_COUNT task directories found)"

mkdir -p "$DEST_DIR"

DATE_STAMP=$(date +%Y%m%d)
SRC_BASENAME=$(basename "$SRC_DIR")

WORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/roo-export.XXXXXX")
cleanup() {
  rm -rf "$WORK_DIR"
}
trap cleanup EXIT

sha256_of() {
  if command -v sha256sum >/dev/null 2>&1; then
    sha256sum "$1" | awk '{print $1}'
  elif command -v shasum >/dev/null 2>&1; then
    shasum -a 256 "$1" | awk '{print $1}'
  else
    echo "ERROR: need sha256sum (Linux) or shasum (macOS) on PATH" >&2
    return 1
  fi
}

# generate_manifest ROOT OUTFILE
# Writes "<sha256>  <path relative to ROOT>" for every file under ROOT,
# sorted, so two manifests can be diffed directly.
generate_manifest() {
  local root="$1" outfile="$2" f rel hash
  : >"$outfile"
  while IFS= read -r -d '' f; do
    rel="${f#"$root"/}"
    hash=$(sha256_of "$f")
    printf '%s  %s\n' "$hash" "$rel" >>"$outfile"
  done < <(find "$root" -type f -print0)
  sort "$outfile" -o "$outfile"
}

echo "==> Step 1: raw copy (read-only on source)"
COPY_ROOT="$WORK_DIR/$SRC_BASENAME"
cp -a "$SRC_DIR" "$COPY_ROOT"

COPY_TASK_COUNT=$(find "$COPY_ROOT/tasks" -mindepth 1 -maxdepth 1 -type d | wc -l | tr -d ' ')

RAW_TAR="$DEST_DIR/roo-history-raw-${DATE_STAMP}.tar.gz"
tar czf "$RAW_TAR" -C "$WORK_DIR" "$SRC_BASENAME"
echo "    wrote $RAW_TAR"

echo "==> Step 2: readable export (derived from the copy, never re-reads source)"
READABLE_DIR="$WORK_DIR/readable"
mkdir -p "$READABLE_DIR/tasks"

PY_OUTPUT=$(python3 - "$COPY_ROOT" "$READABLE_DIR" <<'PYEOF'
import sys, os, json, glob, datetime

copy_root, out_dir = sys.argv[1], sys.argv[2]
tasks_dir = os.path.join(copy_root, "tasks")
tasks_out = os.path.join(out_dir, "tasks")


def iso(ts_ms):
    try:
        return datetime.datetime.fromtimestamp(ts_ms / 1000, datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
    except Exception:
        return "unknown"


def render_block(b):
    if isinstance(b, str):
        return b
    if not isinstance(b, dict):
        return str(b)
    t = b.get("type")
    if t == "text":
        return b.get("text", "")
    if t == "reasoning":
        return "_(reasoning)_\n\n" + (b.get("text", "") or b.get("reasoning", ""))
    if t == "tool_use":
        return "**Tool call: %s**\n```json\n%s\n```" % (
            b.get("name", "?"),
            json.dumps(b.get("input", {}), indent=2, ensure_ascii=False),
        )
    if t == "tool_result":
        c = b.get("content", "")
        if isinstance(c, list):
            parts = []
            for cb in c:
                if isinstance(cb, dict) and cb.get("type") == "text":
                    parts.append(cb.get("text", ""))
                elif isinstance(cb, dict) and cb.get("type") == "image":
                    parts.append("[image omitted]")
                else:
                    parts.append(str(cb))
            c = "\n".join(parts)
        return "**Tool result**\n```\n%s\n```" % c
    if t == "image":
        return "[image omitted]"
    return "```json\n%s\n```" % json.dumps(b, indent=2, ensure_ascii=False)


# Enumerate tasks directly from the filesystem copy — NOT from _index.json,
# which on the source installation this was built against covered only 23
# of 42 task directories.
task_dirs = sorted(
    d for d in os.listdir(tasks_dir) if os.path.isdir(os.path.join(tasks_dir, d))
)

entries = []
bad_json = []
total_json = 0

for task_id in task_dirs:
    tdir = os.path.join(tasks_dir, task_id)
    api_path = os.path.join(tdir, "api_conversation_history.json")
    hist_path = os.path.join(tdir, "history_item.json")

    for jf in glob.glob(os.path.join(tdir, "*.json")):
        total_json += 1
        try:
            with open(jf) as fh:
                json.load(fh)
        except Exception as e:
            bad_json.append((jf, str(e)))

    messages = []
    if os.path.exists(api_path):
        try:
            with open(api_path) as fh:
                messages = json.load(fh)
        except Exception:
            messages = []

    title = None
    ts_first = None
    workspace = None
    mode = None
    if os.path.exists(hist_path):
        try:
            with open(hist_path) as fh:
                h = json.load(fh)
            title = h.get("task")
            ts_first = h.get("ts")
            workspace = h.get("workspace")
            mode = h.get("mode")
        except Exception:
            pass

    if ts_first is None and messages:
        ts_first = messages[0].get("ts")

    if title is None and messages:
        for m in messages:
            if m.get("role") == "user":
                c = m.get("content")
                text = None
                if isinstance(c, str):
                    text = c
                elif isinstance(c, list):
                    for b in c:
                        if isinstance(b, dict) and b.get("type") == "text":
                            text = b.get("text")
                            break
                if text:
                    text = text.replace("<task>", "").replace("</task>", "").strip()
                    title = (text[:100] + "...") if len(text) > 100 else text
                break

    if title is None:
        title = "(untitled)"
    title_source = "history_item.json" if os.path.exists(hist_path) else "inferred from first message"

    turn_count = len(messages)

    md_lines = []
    md_lines.append("# Task %s" % task_id)
    md_lines.append("")
    md_lines.append("- **Title (%s):** %s" % (title_source, title))
    md_lines.append("- **Date:** %s" % (iso(ts_first) if ts_first else "unknown"))
    md_lines.append("- **Turns (messages):** %d" % turn_count)
    if workspace:
        md_lines.append("- **Workspace:** %s" % workspace)
    if mode:
        md_lines.append("- **Mode:** %s" % mode)
    md_lines.append("")
    md_lines.append("---")
    md_lines.append("")

    jsonl_lines = []
    for i, m in enumerate(messages, 1):
        role = m.get("role", "?")
        ts = m.get("ts")
        c = m.get("content")
        md_lines.append("## Turn %d — %s (%s)" % (i, role.capitalize(), iso(ts) if ts else "unknown"))
        md_lines.append("")
        if isinstance(c, str):
            md_lines.append(c)
        elif isinstance(c, list):
            for b in c:
                md_lines.append(render_block(b))
                md_lines.append("")
        else:
            md_lines.append("```json\n%s\n```" % json.dumps(c, indent=2, ensure_ascii=False))
        md_lines.append("")
        jsonl_lines.append(json.dumps(m, ensure_ascii=False))

    with open(os.path.join(tasks_out, task_id + ".md"), "w") as fh:
        fh.write("\n".join(md_lines))
    with open(os.path.join(tasks_out, task_id + ".jsonl"), "w") as fh:
        fh.write("\n".join(jsonl_lines) + ("\n" if jsonl_lines else ""))

    entries.append(
        {
            "id": task_id,
            "title": title,
            "title_source": title_source,
            "date": iso(ts_first) if ts_first else "unknown",
            "ts": ts_first or 0,
            "turns": turn_count,
            "workspace": workspace or "",
        }
    )

entries.sort(key=lambda e: e["ts"])

indexed_count = sum(1 for e in entries if e["title_source"] == "history_item.json")
index_lines = []
index_lines.append("# Roo Code Task Index")
index_lines.append("")
index_lines.append(
    "%d task directories exported (all of them — only %d had a history_item.json / were covered by Roo's own tasks/_index.json)."
    % (len(entries), indexed_count)
)
index_lines.append("")
index_lines.append("| # | Task ID | Title | Date | Turns | Title source | Workspace |")
index_lines.append("|---|---------|-------|------|-------|---------------|-----------|")
for i, e in enumerate(entries, 1):
    title_cell = e["title"].replace("|", "\\|").replace("\n", " ")
    index_lines.append(
        "| %d | %s | %s | %s | %d | %s | %s |"
        % (i, e["id"], title_cell, e["date"], e["turns"], e["title_source"], e["workspace"])
    )

with open(os.path.join(out_dir, "INDEX.md"), "w") as fh:
    fh.write("\n".join(index_lines) + "\n")

print("RESULT:task_dirs_processed=%d" % len(task_dirs))
print("RESULT:total_json_checked=%d" % total_json)
print("RESULT:bad_json_count=%d" % len(bad_json))
for jf, err in bad_json:
    print("BAD_JSON:%s:%s" % (jf, err))
PYEOF
)

echo "$PY_OUTPUT" | grep -v '^RESULT:' || true
TASK_DIRS_PROCESSED=$(echo "$PY_OUTPUT" | grep '^RESULT:task_dirs_processed=' | cut -d= -f2)
TOTAL_JSON_CHECKED=$(echo "$PY_OUTPUT" | grep '^RESULT:total_json_checked=' | cut -d= -f2)
BAD_JSON_COUNT=$(echo "$PY_OUTPUT" | grep '^RESULT:bad_json_count=' | cut -d= -f2)

READABLE_TAR="$DEST_DIR/roo-history-readable-${DATE_STAMP}.tar.gz"
tar czf "$READABLE_TAR" -C "$WORK_DIR" readable
echo "    wrote $READABLE_TAR"

echo "==> Step 3: verification"
FAIL=0

if [ "$SRC_TASK_COUNT" -ne "$COPY_TASK_COUNT" ] || [ "$SRC_TASK_COUNT" -ne "$TASK_DIRS_PROCESSED" ]; then
  echo "FAIL: task directory count mismatch — source=$SRC_TASK_COUNT copy=$COPY_TASK_COUNT processed=$TASK_DIRS_PROCESSED"
  FAIL=1
else
  echo "PASS: task directory count matches across source, copy, and readable export ($SRC_TASK_COUNT)"
fi

if [ "$BAD_JSON_COUNT" -ne 0 ]; then
  echo "FAIL: $BAD_JSON_COUNT JSON file(s) in the copy failed to parse"
  FAIL=1
else
  echo "PASS: all $TOTAL_JSON_CHECKED JSON files in the copy parse cleanly"
fi

MD_COUNT=$(find "$READABLE_DIR/tasks" -maxdepth 1 -name '*.md' -type f | wc -l | tr -d ' ')
JSONL_COUNT=$(find "$READABLE_DIR/tasks" -maxdepth 1 -name '*.jsonl' -type f | wc -l | tr -d ' ')
INDEX_ROWS=$(grep -c '^| [0-9]' "$READABLE_DIR/INDEX.md" || true)
if [ "$MD_COUNT" -eq "$SRC_TASK_COUNT" ] && [ "$JSONL_COUNT" -eq "$SRC_TASK_COUNT" ] && [ "$INDEX_ROWS" -eq "$SRC_TASK_COUNT" ]; then
  echo "PASS: readable export covers all $SRC_TASK_COUNT tasks (md=$MD_COUNT jsonl=$JSONL_COUNT index_rows=$INDEX_ROWS)"
else
  echo "FAIL: readable export incomplete (expected $SRC_TASK_COUNT; md=$MD_COUNT jsonl=$JSONL_COUNT index_rows=$INDEX_ROWS)"
  FAIL=1
fi

MANIFEST_SRC="$WORK_DIR/manifest-src.txt"
MANIFEST_COPY="$WORK_DIR/manifest-copy.txt"
generate_manifest "$SRC_DIR" "$MANIFEST_SRC"
generate_manifest "$COPY_ROOT" "$MANIFEST_COPY"
MANIFEST_FILE_COUNT=$(wc -l <"$MANIFEST_SRC" | tr -d ' ')
if diff -q "$MANIFEST_SRC" "$MANIFEST_COPY" >/dev/null 2>&1; then
  echo "PASS: sha256 manifest of copy matches source ($MANIFEST_FILE_COUNT files, incl. cache/ and settings/)"
else
  echo "FAIL: sha256 manifest mismatch between source and copy"
  diff "$MANIFEST_SRC" "$MANIFEST_COPY" | head -20
  FAIL=1
fi

EXTRACT_DIR="$WORK_DIR/extract-test"
mkdir -p "$EXTRACT_DIR"
tar xzf "$RAW_TAR" -C "$EXTRACT_DIR"
MANIFEST_EXTRACT="$WORK_DIR/manifest-extract.txt"
generate_manifest "$EXTRACT_DIR/$SRC_BASENAME" "$MANIFEST_EXTRACT"
if diff -q "$MANIFEST_SRC" "$MANIFEST_EXTRACT" >/dev/null 2>&1; then
  echo "PASS: raw tarball extracts byte-identical to source"
else
  echo "FAIL: raw tarball extraction does not match source"
  diff "$MANIFEST_SRC" "$MANIFEST_EXTRACT" | head -20
  FAIL=1
fi

MANIFEST_OUT="$DEST_DIR/roo-history-sha256-manifest-${DATE_STAMP}.txt"
cp "$MANIFEST_SRC" "$MANIFEST_OUT"

echo ""
echo "===== SUMMARY ====="
echo "Source task directories : $SRC_TASK_COUNT"
echo "Raw archive             : $RAW_TAR ($(wc -c <"$RAW_TAR" | tr -d ' ') bytes)"
echo "Readable archive        : $READABLE_TAR ($(wc -c <"$READABLE_TAR" | tr -d ' ') bytes)"
echo "SHA-256 manifest        : $MANIFEST_OUT ($MANIFEST_FILE_COUNT files)"
echo "JSON files validated    : $TOTAL_JSON_CHECKED (bad: $BAD_JSON_COUNT)"

if [ "$FAIL" -ne 0 ]; then
  echo ""
  echo "RESULT: FAILED -- do not treat this export as complete. See FAIL lines above."
  exit 1
fi

echo ""
echo "RESULT: ALL CHECKS PASSED"

Not included here

Per instruction, the actual tarballs from the workspace-side run are not attached to this issue — they stay local under ~/code/roo-history/ on the Coder workspace. This issue is purely the playbook + script for repeating the process on the Mac.