Skip to content

PROD-2311: record post-publish versionIDs via Fetch-layer poll (fixes mapping drift) #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
5PK merged 4 commits into from
Jul 20, 2026
Merged

PROD-2311: record post-publish versionIDs via Fetch-layer poll (fixes mapping drift) #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7f36ef0
PROD-2311: resolve post-publish targetVersionID via Fetch-layer poll …
5PK Jul 14, 2026
ee7800a
PROD-2311: classify unresolved items by observed-ness, not baseline===0
5PK Jul 14, 2026
d48429d
PROD-2311: resolve post-publish versionIDs via Management API instead…
5PK Jul 20, 2026
79f2c26
Merge remote-tracking branch 'origin/main' into PROD-2311-versionid-poll
5PK Jul 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 28 additions & 11 deletions src/core/push.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,18 +165,25 @@ export class Push {
autoPublishErrors = await this.executeAutoPublish(results, autoPublish);
}

// Split auto-publish outcomes: genuine publish failures (blocking) vs
// post-publish mapping/refresh bookkeeping notices (non-blocking). PROD-2311
const genuinePublishErrors = autoPublishErrors.filter(
({ type }) => type === "publish" || type === "fatal"
);
const mappingWarnings = autoPublishErrors.filter(
({ type }) => type === "mapping" || type === "refresh"
);

// PROD-2310: Fold auto-publish failures into the returned success signal so the
// entry point can exit non-zero. Real publish failures ("publish") and a fatal
// auto-publish crash ("fatal") are unambiguous failures that CI must detect; they
// previously never affected the exit code. "mapping"/"refresh" entries are
// post-publish bookkeeping that PROD-2311 may reclassify, so they are deliberately
// NOT treated as hard failures here (see PROD-2311 follow-up).
// entry point can exit non-zero. "mapping"/"refresh" entries are post-publish
// bookkeeping (PROD-2311) and are deliberately NOT treated as hard failures here.
const success = syncSuccess && !hasBlockingAutoPublishErrors(autoPublishErrors);

// Final error summary - show if there were ANY failures (sync or auto-publish)
const hasFailures = totalSyncFailures > 0 || syncErrors.length > 0 || autoPublishErrors.length > 0;
// Final error summary - show only if there were genuine (blocking) failures
const hasBlockingFailures =
totalSyncFailures > 0 || syncErrors.length > 0 || genuinePublishErrors.length > 0;

if (hasFailures) {
if (hasBlockingFailures) {
console.log(ansiColors.red("\n" + "═".repeat(50)));
console.log(ansiColors.red("⚠️ ERROR SUMMARY"));
console.log(ansiColors.red("═".repeat(50)));
Expand Down Expand Up @@ -218,16 +225,26 @@ export class Push {
});
}

// Show auto-publish errors
if (autoPublishErrors.length > 0) {
// Show genuine auto-publish failures (publish/fatal)
if (genuinePublishErrors.length > 0) {
console.log(ansiColors.red(`\n Auto-Publish Errors:`));
autoPublishErrors.forEach(({ locale, type, error }) => {
genuinePublishErrors.forEach(({ locale, type, error }) => {
const localeDisplay = locale ? `[${locale}]` : "";
console.log(ansiColors.red(` • ${localeDisplay} ${type}: ${error}`));
});
}
}

// PROD-2311: post-publish mapping/version bookkeeping notices are NOT publish
// failures — show them under a non-blocking header so they don't read as broken.
if (mappingWarnings.length > 0) {
console.log(ansiColors.yellow(`\n Mapping Update Warnings (non-blocking):`));
mappingWarnings.forEach(({ locale, type, error }) => {
const localeDisplay = locale ? `[${locale}]` : "";
console.log(ansiColors.yellow(` • ${localeDisplay} ${type}: ${error}`));
});
}

// Show log file paths at the very end
if (logFilePaths.length > 0) {
console.log(ansiColors.cyan("\n📄 Log Files:"));
Expand Down
59 changes: 19 additions & 40 deletions src/lib/mappers/mapping-version-updater.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
/**
* Mapping Version Updater
*
* After publishing, updates the mappings with the new versionIDs
* by reading the refreshed data from the filesystem using fileOperations.
* After publishing, updates the mappings with the new targetVersionIDs.
* PROD-2311: versionIDs are read directly from the Management API right after
* publish, instead of a start-of-run filesystem snapshot that misses items
* created during the same run. The Management API is strongly consistent (no
* CDN propagation delay), so a single call per item is enough — no polling.
*/

import { fileOperations } from "../../core";
import { getLogger } from "../../core/state";
import * as mgmtApi from "@agility/management-sdk";
import { getApiClient, getLogger } from "../../core/state";
import { ContentItemMapper } from "./content-item-mapper";
import { PageMapper } from "./page-mapper";
import { getContentItemsFromFileSystem } from "../getters/filesystem/get-content-items";
import { getPagesFromFileSystem } from "../getters/filesystem/get-pages";
import ansiColors from "ansi-colors";
import { MappingUpdateResult } from "../../types";

Expand Down Expand Up @@ -65,32 +66,21 @@ export async function updateContentMappingsAfterPublish(
}

try {
// Create file operations for target (we only need target data for versionID)
const targetFileOps = new fileOperations(targetGuid, locale);

// Load content items from target filesystem (refreshed after pull)
const targetContentItems = getContentItemsFromFileSystem(targetFileOps);

// Create content item mapper
const contentMapper = new ContentItemMapper(sourceGuid, targetGuid, locale);
const apiClient = getApiClient();

// Create lookup map for quick access
const targetContentMap = new Map(targetContentItems.map((item) => [item.contentID, item]));

// Update targetVersionID for each published content item
for (const targetContentId of uniqueContentIds) {
const targetItem = targetContentMap.get(targetContentId);
if (!targetItem) {
errors.push(`Target content item ${targetContentId} not found in filesystem`);
let targetItem: mgmtApi.ContentItem;
try {
targetItem = await apiClient.contentMethods.getContentItem(targetContentId, targetGuid, locale);
} catch (error: any) {
errors.push(`Target content item ${targetContentId} not found on target instance: ${error.message}`);
continue;
}

// Update only the target versionID in the mapping
const result = contentMapper.updateTargetVersionID(targetContentId, targetItem.properties.versionID);

if (result.success) {
updated++;
// Track all version updates with display info
changes.push({
id: targetContentId,
oldVersion: result.oldVersionID!,
Expand Down Expand Up @@ -134,32 +124,21 @@ export async function updatePageMappingsAfterPublish(
}

try {
// Create file operations for target (we only need target data for versionID)
const targetFileOps = new fileOperations(targetGuid, locale);

// Load pages from target filesystem (refreshed after pull)
const targetPages = getPagesFromFileSystem(targetFileOps);

// Create page mapper
const pageMapper = new PageMapper(sourceGuid, targetGuid, locale);
const apiClient = getApiClient();

// Create lookup map for quick access
const targetPageMap = new Map(targetPages.map((page) => [page.pageID, page]));

// Update targetVersionID for each published page
for (const targetPageId of uniquePageIds) {
const targetPage = targetPageMap.get(targetPageId);
if (!targetPage) {
errors.push(`Target page ${targetPageId} not found in filesystem`);
let targetPage: mgmtApi.PageItem;
try {
targetPage = await apiClient.pageMethods.getPage(targetPageId, targetGuid, locale);
} catch (error: any) {
errors.push(`Target page ${targetPageId} not found on target instance: ${error.message}`);
continue;
}

// Update only the target versionID in the mapping
const result = pageMapper.updateTargetVersionID(targetPageId, targetPage.properties.versionID);

if (result.success) {
updated++;
// Track all version updates with display info
changes.push({
id: targetPageId,
oldVersion: result.oldVersionID!,
Expand Down
73 changes: 32 additions & 41 deletions src/lib/mappers/tests/mapping-version-updater.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,20 @@ import * as fs from "fs";
import * as os from "os";
import * as path from "path";
import { resetState, setState } from "core/state";
import * as stateModule from "core/state";

// Mock the API client so no real network calls happen
const mockGetContentItem = jest.fn();
const mockGetPage = jest.fn();

// Mock the filesystem getters to avoid real I/O beyond what we control
jest.mock("lib/getters/filesystem/get-content-items", () => ({
getContentItemsFromFileSystem: jest.fn(),
}));
jest.mock("lib/getters/filesystem/get-pages", () => ({
getPagesFromFileSystem: jest.fn(),
}));

// Import after mocks are in place
import { getContentItemsFromFileSystem } from "lib/getters/filesystem/get-content-items";
import { getPagesFromFileSystem } from "lib/getters/filesystem/get-pages";
import {
updateContentMappingsAfterPublish,
updatePageMappingsAfterPublish,
updateMappingsAfterPublish,
VersionChangeDetail,
} from "lib/mappers/mapping-version-updater";
import { ContentItemMapper } from "lib/mappers/content-item-mapper";
import { PageMapper } from "lib/mappers/page-mapper";

const mockGetContentItems = getContentItemsFromFileSystem as jest.MockedFunction<typeof getContentItemsFromFileSystem>;
const mockGetPages = getPagesFromFileSystem as jest.MockedFunction<typeof getPagesFromFileSystem>;

let tmpDir: string;

beforeAll(() => {
Expand All @@ -45,8 +35,12 @@ beforeEach(() => {
jest.spyOn(console, "log").mockImplementation(() => {});
jest.spyOn(console, "warn").mockImplementation(() => {});
jest.spyOn(console, "error").mockImplementation(() => {});
mockGetContentItems.mockReturnValue([]);
mockGetPages.mockReturnValue([]);
mockGetContentItem.mockReset();
mockGetPage.mockReset();
jest.spyOn(stateModule, "getApiClient").mockReturnValue({
contentMethods: { getContentItem: mockGetContentItem },
pageMethods: { getPage: mockGetPage },
} as any);
});

afterEach(() => {
Expand Down Expand Up @@ -99,44 +93,44 @@ describe("updateContentMappingsAfterPublish", () => {
const result = await updateContentMappingsAfterPublish([], SRC, TGT, LOCALE);
expect(result.updated).toBe(0);
expect(result.errors).toHaveLength(0);
expect(mockGetContentItem).not.toHaveBeenCalled();
});

it("deduplicates published content IDs before processing", async () => {
mockGetContentItems.mockReturnValue([]);
mockGetContentItem.mockRejectedValue(new Error("not found"));
const result = await updateContentMappingsAfterPublish([100, 100, 100], SRC, TGT, LOCALE);
// 100 is not found in filesystem → one error, not three
// 100 is not found on the target → one error, not three
expect(result.errors).toHaveLength(1);
expect(mockGetContentItem).toHaveBeenCalledTimes(1);
});

it("records an error when a content item is not found in target filesystem", async () => {
mockGetContentItems.mockReturnValue([]);
it("records an error when a content item is not found on the target instance", async () => {
mockGetContentItem.mockRejectedValue(new Error("404"));
const result = await updateContentMappingsAfterPublish([999], SRC, TGT, LOCALE);
expect(result.errors.length).toBeGreaterThan(0);
expect(result.errors[0]).toMatch(/999/);
});

it("records an error when no mapping exists for a target content ID", async () => {
const targetItem = makeContentItem({ contentID: 200, properties: { versionID: 7 } });
mockGetContentItems.mockReturnValue([targetItem]);
mockGetContentItem.mockResolvedValue(makeContentItem({ contentID: 200, properties: { versionID: 7 } }));
const result = await updateContentMappingsAfterPublish([200], SRC, TGT, LOCALE);
expect(result.errors.length).toBeGreaterThan(0);
expect(result.errors[0]).toMatch(/200/);
});

it("updates the mapping successfully when item and mapping exist", async () => {
seedContentMapper(10, 20, 5);
const targetItem = makeContentItem({ contentID: 20, properties: { versionID: 9 } });
mockGetContentItems.mockReturnValue([targetItem]);
mockGetContentItem.mockResolvedValue(makeContentItem({ contentID: 20, properties: { versionID: 9 } }));
const result = await updateContentMappingsAfterPublish([20], SRC, TGT, LOCALE);
expect(result.updated).toBe(1);
expect(result.errors).toHaveLength(0);
expect(result.changes[0].newVersion).toBe(9);
expect(mockGetContentItem).toHaveBeenCalledWith(20, TGT, LOCALE);
});

it("tracks change.changed as false when versionID is already up to date", async () => {
seedContentMapper(10, 20, 5);
const targetItem = makeContentItem({ contentID: 20, properties: { versionID: 5 } });
mockGetContentItems.mockReturnValue([targetItem]);
mockGetContentItem.mockResolvedValue(makeContentItem({ contentID: 20, properties: { versionID: 5 } }));
const result = await updateContentMappingsAfterPublish([20], SRC, TGT, LOCALE);
expect(result.changes[0].changed).toBe(false);
});
Expand All @@ -149,31 +143,31 @@ describe("updatePageMappingsAfterPublish", () => {
const result = await updatePageMappingsAfterPublish([], SRC, TGT, LOCALE);
expect(result.updated).toBe(0);
expect(result.errors).toHaveLength(0);
expect(mockGetPage).not.toHaveBeenCalled();
});

it("records an error when a page is not found in target filesystem", async () => {
mockGetPages.mockReturnValue([]);
it("records an error when a page is not found on the target instance", async () => {
mockGetPage.mockRejectedValue(new Error("404"));
const result = await updatePageMappingsAfterPublish([999], SRC, TGT, LOCALE);
expect(result.errors.length).toBeGreaterThan(0);
expect(result.errors[0]).toMatch(/999/);
});

it("records an error when no mapping exists for a target page ID", async () => {
const targetPage = makePage({ pageID: 500, properties: { versionID: 3 } });
mockGetPages.mockReturnValue([targetPage]);
mockGetPage.mockResolvedValue(makePage({ pageID: 500, properties: { versionID: 3 } }));
const result = await updatePageMappingsAfterPublish([500], SRC, TGT, LOCALE);
expect(result.errors.length).toBeGreaterThan(0);
expect(result.errors[0]).toMatch(/500/);
});

it("updates the mapping successfully when page and mapping exist", async () => {
seedPageMapper(1, 50, 1);
const targetPage = makePage({ pageID: 50, properties: { versionID: 8 } });
mockGetPages.mockReturnValue([targetPage]);
mockGetPage.mockResolvedValue(makePage({ pageID: 50, properties: { versionID: 8 } }));
const result = await updatePageMappingsAfterPublish([50], SRC, TGT, LOCALE);
expect(result.updated).toBe(1);
expect(result.errors).toHaveLength(0);
expect(result.changes[0].newVersion).toBe(8);
expect(mockGetPage).toHaveBeenCalledWith(50, TGT, LOCALE);
});
});

Expand All @@ -190,21 +184,19 @@ describe("updateMappingsAfterPublish", () => {

it("processes both content and page IDs", async () => {
seedContentMapper(10, 20, 1);
const targetContent = makeContentItem({ contentID: 20, properties: { versionID: 2 } });
mockGetContentItems.mockReturnValue([targetContent]);
mockGetContentItem.mockResolvedValue(makeContentItem({ contentID: 20, properties: { versionID: 2 } }));

seedPageMapper(1, 50, 1);
const targetPage = makePage({ pageID: 50, properties: { versionID: 2 } });
mockGetPages.mockReturnValue([targetPage]);
mockGetPage.mockResolvedValue(makePage({ pageID: 50, properties: { versionID: 2 } }));

const { result } = await updateMappingsAfterPublish([20], [50], SRC, TGT, LOCALE);
expect(result.contentMappingsUpdated).toBe(1);
expect(result.pageMappingsUpdated).toBe(1);
});

it("accumulates errors from both content and page processing", async () => {
mockGetContentItems.mockReturnValue([]);
mockGetPages.mockReturnValue([]);
mockGetContentItem.mockRejectedValue(new Error("404"));
mockGetPage.mockRejectedValue(new Error("404"));
const { result } = await updateMappingsAfterPublish([999], [888], SRC, TGT, LOCALE);
expect(result.errors.length).toBeGreaterThanOrEqual(2);
});
Expand All @@ -215,8 +207,7 @@ describe("updateMappingsAfterPublish", () => {
describe("VersionChangeDetail structure", () => {
it("includes expected fields in change objects", async () => {
seedContentMapper(10, 20, 5);
const targetItem = makeContentItem({ contentID: 20, properties: { versionID: 9 } });
mockGetContentItems.mockReturnValue([targetItem]);
mockGetContentItem.mockResolvedValue(makeContentItem({ contentID: 20, properties: { versionID: 9 } }));
const result = await updateContentMappingsAfterPublish([20], SRC, TGT, LOCALE);
const change = result.changes[0];
expect(change).toMatchObject({
Expand Down
Loading