From 2523dd15449a1b2ff133937e7214986f270bae17 Mon Sep 17 00:00:00 2001 From: frathe Date: Thu, 3 Sep 2026 10:09:48 +0200 Subject: [PATCH] docs: plan measured CI test sharding docs: add measured CI sharding tickets ci: capture monolithic race baseline add plan for Microsoft Store MSIX packaging moved done task to finished_refactorings docs: record controlled CI baseline testshards: generate deterministic UI manifest testshards: enforce safe complete shard assignment test: add shared sharded race contract ci: run required checks concurrently docs: close measured CI sharding optimization --- .github/testshards/internal-ui.tsv | 577 +++++++++ .github/workflows/ci.yml | 89 +- ARCHITECTURE.md | 17 + Makefile | 76 +- .../2026-09-02-address-pr-13-review.md | 0 .../issues/01-route-swipe-input-by-reveal.md | 0 .../issues/02-preserve-right-wheel-anchor.md | 0 .../issues/03-document-review-and-verify.md | 0 .../spec.md | 0 .../2026-09-02-windows-release-signing.md | 0 .../2026-09-03-measured-ci-test-sharding.md | 994 +++++++++++++++ ...pture-controlled-monolithic-ci-baseline.md | 62 + ...2-generate-deterministic-shard-manifest.md | 59 + ...-enforce-safe-complete-shard-assignment.md | 54 + .../04-run-sharded-race-contract-locally.md | 60 + .../05-run-required-ci-work-concurrently.md | 61 + .../06-measure-assess-close-optimization.md | 70 ++ .../spec.md | 336 +++++ plans/2026-09-03-microsoft-store-msix.md | 248 ++++ qodana.yaml | 1 + scripts/testshards/main.go | 1096 +++++++++++++++++ scripts/testshards/main_test.go | 977 +++++++++++++++ todos.md | 55 +- 23 files changed, 4772 insertions(+), 60 deletions(-) create mode 100644 .github/testshards/internal-ui.tsv rename {plans => finished_refactorings}/2026-09-02-address-pr-13-review.md (100%) rename {.scratch/swipe-unlinked-pointer-routing => finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing}/issues/01-route-swipe-input-by-reveal.md (100%) rename {.scratch/swipe-unlinked-pointer-routing => finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing}/issues/02-preserve-right-wheel-anchor.md (100%) rename {.scratch/swipe-unlinked-pointer-routing => finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing}/issues/03-document-review-and-verify.md (100%) rename {.scratch/swipe-unlinked-pointer-routing => finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing}/spec.md (100%) rename {plans => finished_refactorings}/2026-09-02-windows-release-signing.md (100%) create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/01-capture-controlled-monolithic-ci-baseline.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/02-generate-deterministic-shard-manifest.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/03-enforce-safe-complete-shard-assignment.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/04-run-sharded-race-contract-locally.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/05-run-required-ci-work-concurrently.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/06-measure-assess-close-optimization.md create mode 100644 finished_refactorings/2026-09-03-measured-ci-test-sharding/spec.md create mode 100644 plans/2026-09-03-microsoft-store-msix.md create mode 100644 scripts/testshards/main.go create mode 100644 scripts/testshards/main_test.go diff --git a/.github/testshards/internal-ui.tsv b/.github/testshards/internal-ui.tsv new file mode 100644 index 0000000..ba30095 --- /dev/null +++ b/.github/testshards/internal-ui.tsv @@ -0,0 +1,577 @@ +# PicFetch test shard manifest v1 +# package: github.com/frathe/picfetch/internal/ui +# shards: 3 +# baseline-sha: fcd11103969ac8d1b5a1a220270c0730cb5e1913 +# baseline-run: 33734286005 +# baseline-attempts: 1,2,3 +# ui-1: 177 entries, 326.750s median-weight sum +# ui-2: 178 entries, 327.070s median-weight sum +# ui-3: 212 entries, 326.740s median-weight sum +# name shard +TestActionsMenu_CopyImageUsesStub ui-1 +TestActionsMenu_GridToggleDisablesRotateZoomInfo ui-1 +TestActionsMenu_HideDisabledDuringVariantsSession ui-1 +TestActionsMenu_HideDuplicatesEnablesVariantsOnPairNotUnique ui-1 +TestActionsMenu_InfoNoopsWhileGridVisible ui-1 +TestActionsMenu_ShowVariantsNoopsWhileHideOff ui-1 +TestActionsMenu_SortItemJumpsWithoutCycling ui-1 +TestActionsMenu_ToggleBrowseDuplicatesChecksVariants ui-1 +TestActionsMenu_ToggleInfoOverlay ui-1 +TestActionsMenu_TrashNoopsWithoutFiles ui-1 +TestActionsMenu_TrashOpensConfirmation ui-1 +TestActionsMenu_WallpaperNoopsWithoutImage ui-1 +TestApplySettings_DoesNotRevertLiveShortcutChanges ui-1 +TestApplySettings_RetunesImageCacheWithoutSorting ui-1 +TestApplyStagedUpdate_SavesNotesAndCallsApply ui-1 +TestAttemptLoad_CacheHitServesFileRemovedFromDisk ui-1 +TestBatchDelete_LeavesTheWindowMaximized ui-1 +TestBuildMainMenu_ExportItemOpensThePrompt ui-1 +TestBuildMainMenu_SaveChangesItemInvokesSaveRotation ui-1 +TestBuildMainMenu_SettingsItemOpensTheSettingsWindow ui-1 +TestBuildMainMenu_WindowItemsDisplayTheirAccelerators ui-1 +TestBuildViewer_SavedSessionShowsRestoreLink ui-1 +TestCanExport_TrueForAFormatWithNoEncoder ui-1 +TestCanExport_TrueForALoadedImage ui-1 +TestCanSaveRotation_FalseForAnimatedImage ui-1 +TestCanSaveRotation_FalseWithNoImage ui-1 +TestCanSetWallpaper_FalseWhileLoading ui-1 +TestCancelScan_PreservesExistingFilesInMergeMode ui-1 +TestCheckForUpdates_DefaultsToFalseOnStartup ui-1 +TestClearToDropzone_ClearsInspect ui-1 +TestClearToDropzone_ExitsPictureFrameMode ui-1 +TestCloseFilesItem_DisabledAgainAfterCloseFiles ui-1 +TestCloseFiles_ResetsLoadedFilesToWelcomeState ui-1 +TestCompareAllowedCommands_ToolbarRemainsAboveInputShield ui-1 +TestCompareAnimated_FreezesFirstDecodedFrameForEntireSession ui-1 +TestCompareCommandEntryPoints_MenuAndFeatureCallbacksAreIgnored ui-1 +TestCompareEntry_RequiresExactlyTwoExplicitSelections ui-1 +TestCompareInputLimit_FailsWithoutRemovingEitherSelectedSource ui-1 +TestCompareLinkControl_CtrlLAndButtonShareReadyGate ui-1 +TestCompareLinkToggle_CanvasOverlayOwnsPhysicalCtrlL ui-1 +TestCompareLinkToggle_ChainsHookPersistsAndRelinksOnSecondPress ui-1 +TestCompareMemory_HoldsBothFullDecodesBeyondSharedCacheBudget ui-1 +TestComparePanInputs_CanvasDragAndShiftWheelStayInComparison ui-1 +TestCompareRestoration_EscapeRevealsTheUnchangedFilteredGrid ui-1 +TestCompareShortcut_RegistersDefaultAndPhysicalControlWithoutDuplicates ui-1 +TestCompareSwapPreservesGrid_StateOrderAndSelectionAnchor ui-1 +TestCompareTransitionPreservesGrid_StateAndTitle ui-1 +TestCompareZoom_KeyboardRoutesToComparisonWithoutChangingCoveredState ui-1 +TestCopyPathToClipboard_NoFilesIsNoop ui-1 +TestCopyPathToClipboard_SetsFilePath ui-1 +TestCopySelectionActivation ui-1 +TestCopySelectionClipboardFailure ui-1 +TestCopySelectionEncodeFailure ui-1 +TestCopySelectionKeyboard ui-1 +TestCopySelectionPixels ui-1 +TestCopySelectionRotation ui-1 +TestCopySelectionSurvivesZoomPanAndResize ui-1 +TestCopySelectionUnknownKeyCancels ui-1 +TestCopy_OutsideTheGridStillCopiesTheImage ui-1 +TestCopy_ReportsAFailedFileCopy ui-1 +TestCurrentUpdateCallback_DropsEventSupersededWhileQueued ui-1 +TestDrain_WaitsUpdateDone ui-1 +TestE2E_BadDropAfterImagesClearsDisplay ui-1 +TestE2E_CopySelection ui-1 +TestE2E_DeleteConfirmationShowsWhichButtonReturnWillPress ui-1 +TestE2E_EscapeCancelsScanInsteadOfClosing ui-1 +TestE2E_HoveringDropzoneHighlightsBorderThenReverts ui-1 +TestE2E_InitialLaunchShowsWelcome ui-1 +TestE2E_TappingRestoreLinkRestoresNotFileDialog ui-1 +TestEnterPictureFrameMode_ClosesOpenGrid ui-1 +TestExifLink_HiddenWhenTheImageHasNoExifData ui-1 +TestExifLink_ShownWhenTheImageHasExifData ui-1 +TestExifLink_VisibilityFollowsNavigation ui-1 +TestExifWindow_LeftRightChangeImage ui-1 +TestExportAs_RunsSavePanelInBackground ui-1 +TestFavoriteShortcutOpensStoredFilesThroughViewer ui-1 +TestFinishLoad_PreloadsBothNeighbors ui-1 +TestGenerationTracksFileSetIdentityNotNavigation ui-1 +TestGlobalFavoriteShortcutOpensStoredFilesThroughViewer ui-1 +TestGridHighlight_TitleKeepsTheModePrefixes ui-1 +TestGridHighlight_VariantsHoverUpdatesTitleAndHidesMergePrefix ui-1 +TestHandleDropBlockedWhileCopyPendingShowsToast ui-1 +TestHandleDrop_ClosesOpenGrid ui-1 +TestHandleDrop_HidesRestoreLinkEvenWithoutUsingIt ui-1 +TestHandleDrop_MergeModeMergesIntoExistingSet ui-1 +TestHandleDrop_NaturalSortsByDefault ui-1 +TestHandleDrop_RecursesIntoNestedDirectories ui-1 +TestHandleDrop_SecondDropWithoutMergeModeReplaces ui-1 +TestHandleDrop_SupersededScanGoroutineExits ui-1 +TestHandleKeyEvent_DTogglesHideDuplicatesWhenGridClosed ui-1 +TestHandleKeyEvent_EscapeResetsFade ui-1 +TestHandleKeyEvent_GFromInspectReopensVariants ui-1 +TestHandleKeyEvent_HomeEndOnFolderSiblings ui-1 +TestHandleKeyEvent_LeftRightStillNavigateInPictureFrameMode ui-1 +TestHandleKeyEvent_LeftRightWalkFolderSiblings ui-1 +TestHandleKeyEvent_PEntersPictureFrameMode ui-1 +TestHandleKeyEvent_PlainPStillEntersPictureFrameModeAfterShiftP ui-1 +TestHandleKeyEvent_UpDownNavigateOutsidePictureFrameMode ui-1 +TestHandleKeyEvent_VNoopsInImageView ui-1 +TestHandleTypedRune_GridVisible_SwallowedByExportPrompt ui-1 +TestHandleTypedRune_IgnoredWhileAFyneDialogIsUp ui-1 +TestInfoOverlayReportsLogicalSizeNotLiveRaster ui-1 +TestInstallOpenWithHandler_FlushesADeliveryThatArrivedBeforeInstall ui-1 +TestInvalidateLoad_CancelsPriorLoadContext ui-1 +TestKey0YieldsCopySelectionAndResetsRotation ui-1 +TestLastUpdateCheckDay_ConcurrentWithCurrentPreferences ui-1 +TestManualUpdateCheck_CancelledRequestEmitsNoTerminalCallback ui-1 +TestManualUpdateFlow_CallbacksReadyThenPerformRequestsRelaunch ui-1 +TestMaxWindowSizeGetterSetter ui-1 +TestMaybeShowWhatsNew_ShowsAndClearsCache ui-1 +TestMemoryLimitSetters_FloorAtOne ui-1 +TestOpenInitialFiles_ArgvAndADeliveryBecomeOneScanWithArgvFirst ui-1 +TestOpenInitialFiles_PlainLaunchStartsNoScan ui-1 +TestOpenShortcuts_InvokeFileDialog ui-1 +TestOpenWithHandler_DeliveryWhileInstalledOpensTheFiles ui-1 +TestOverlayOrder_ConfirmationAndToastSitAboveTheGrid ui-1 +TestPerformDelete_RemovesCurrentFileAndAdvancesToTheNextOne ui-1 +TestPreloadOne_SkipsANeighborTooLargeForTheBudget ui-1 +TestRasterizeVectorCoalescesABurst ui-1 +TestRecordWindowPosition_ClockwiseSpiralDragFiresTheGesture ui-1 +TestRecordWindowPosition_StraightDragDoesNotFireTheGesture ui-1 +TestRemoveFile_PurgesCacheEntry ui-1 +TestRemoveFiles_HandlesUnsortedIndices ui-1 +TestReportChooserError_TogglesToastByOS ui-1 +TestReset_ResetsFadeLeftMidTransition ui-1 +TestSVGDisplaysAtLogicalSize ui-1 +TestSaveRotation_WritesRotatedPixelsAndResetsState ui-1 +TestSetAsWallpaper_SweepsTheFileItReplaced ui-1 +TestSetDuplicateDistance_UpdatesGetterAndCurrentPreferences ui-1 +TestSetMaxImageCacheMBRetunesTheVectorRasterCeiling ui-1 +TestSetMaxScan_FloorsAtOne ui-1 +TestSetSortMode_JumpsDirectlyRatherThanCycling ui-1 +TestSetSortMode_SafeWithNoFilesLoaded ui-1 +TestSetStaticWindowSize_UpdatesGetterAndCurrentPreferences ui-1 +TestShiftDelete_WhileGridVisiblePromptsForTheSelection ui-1 +TestShowExifWindow_OpensAndRaisesSameWindow ui-1 +TestShowImage_StartsLoadLifecycle ui-1 +TestShow_ResetsZoomOnNavigation ui-1 +TestSlideInterval_DefaultsBeforePictureFrameModeEverRuns ui-1 +TestStartWindowPosPolling_TestDriverGetsNoopStop ui-1 +TestStartup_LoadsSavedSecondaryWindowGeometry ui-1 +TestStartup_OmittedPreferencesUseShippedDefaults ui-1 +TestStaticWindowSize_LoadDoesNotResize ui-1 +TestStepImage_NoopWhileExportPromptVisible ui-1 +TestStepImage_SingleFileDropWalksFolderSiblings ui-1 +TestStripMetadata_HidesExifLinkAndShrinksReportedSize ui-1 +TestSweepUpdateBackup_SweepsWithNoFailureRecord ui-1 +TestSyncFavoritePreviews_OpeningAFavoriteWritesPreviews ui-1 +TestSyncWindowToZoom_LargeImageStaysCapped ui-1 +TestSyncWindowToZoom_ZoomInGrowsWindow ui-1 +TestThemeMode_RestoresAppliesLiveAndPersists ui-1 +TestToggleInfoOverlay_HiddenUntilAnImageIsLoaded ui-1 +TestTogglePictureFrameMode_ExitResetsFade ui-1 +TestTranslationsHaveNoUnicodeArrows ui-1 +TestUpdateCheck_RestoredTodayDoesNotCheck ui-1 +TestUpdateCheck_TurningOffInvalidatesInFlight ui-1 +TestUpdateCheck_TurningOffKeepsCompletedStage ui-1 +TestUpdateFailureContent_KeepsTheWholeMessageReachableAtTheSmallestWindow ui-1 +TestViewerReset ui-1 +TestViewerShow_AnimatesGIF ui-1 +TestViewerShow_AutoAdvancesPastBrokenFileDuringNavigation ui-1 +TestViewerShow_AutoAdvancesPastBrokenFirstFile ui-1 +TestViewerShow_LoadsAndNavigates ui-1 +TestWhatsNewCache_RoundTripAndClear ui-1 +TestWindowMenu_AfterOneJPEGDropEnablesExifGridAndPictureFrame ui-1 +TestWindowMenu_GridActionNoopsDuringPictureFrame ui-1 +TestWindowMenu_GridActionOpensAndDoesNotToggleOff ui-1 +TestWindowMenu_GridToggleGreysGridAndEnablesViewer ui-1 +TestWindowMenu_VKeyLeavesGrid ui-1 +TestWindowMenu_VKeyLeavesPictureFrame ui-1 +TestWindowMenu_ViewerLeavesGrid ui-1 +TestWindowMenu_ViewerLeavesPictureFrame ui-1 +TestWireAddFavoritesShortcut_DoesNothingWithoutFiles ui-1 +TestWireExportShortcuts_OpensPromptAndSetsWallpaperWithoutColliding ui-1 +TestWireManageFavoritesShortcut_DoesNothingWhileTheExportPromptIsUp ui-1 +TestWireSaveShortcut_SavesTheCurrentRotation ui-1 +TestYieldingMenuCallbacksWrapsEveryField ui-1 +TestActionsMenu_CopyPathWritesClipboard ui-2 +TestActionsMenu_HideNoopsWithoutFiles ui-2 +TestActionsMenu_InfoToggleChecksItem ui-2 +TestActionsMenu_MergeToggleChecksItem ui-2 +TestActionsMenu_PictureFrameLeavesRotateEnabled ui-2 +TestActionsMenu_RotateNoopsWhileGridVisible ui-2 +TestActionsMenu_RotateNoopsWithoutImage ui-2 +TestActionsMenu_SetSortModeWithNoFiles ui-2 +TestActionsMenu_ShowVariantsSecondClickLeavesBrowse ui-2 +TestActionsMenu_SortItemNoopWhenAlreadySelected ui-2 +TestActionsMenu_ToggleMergeMode ui-2 +TestActionsMenu_WallpaperSetsDesktop ui-2 +TestAdvance_ShuffleOnNeverRepeatsCurrentIndex ui-2 +TestAppState_RemoveFileEvictsCacheWithoutCallerAsking ui-2 +TestApplySettings_ChangesSortWhenAsked ui-2 +TestApplyStagedUpdate_SameVersionRemovesWithoutApply ui-2 +TestAttemptLoad_DisplaysAnImageLargerThanTheWholeCacheBudget ui-2 +TestAttemptLoad_ReportsAFileTooLargeToOpen ui-2 +TestAttemptLoad_ToastsAndFallsBackToAStaticFrameForAnOversizedAnimation ui-2 +TestBatchDelete_ClosesTheGridWhenNothingIsLeft ui-2 +TestBuildMainMenu_ActionsItemsDisplayTheirAccelerators ui-2 +TestBuildMainMenu_CloseFilesItemResetsToWelcomeState ui-2 +TestBuildMainMenu_OpenFilesItemInvokesTheNativeChooser ui-2 +TestBuildMainMenu_Structure ui-2 +TestCanExport_FalseWhileLoading ui-2 +TestCanExport_FalseWithNoImage ui-2 +TestCanExport_TrueForAnAnimatedImage ui-2 +TestCanSaveRotation_FalseForRAW ui-2 +TestCanSaveRotation_FalseWithZeroRotation ui-2 +TestCanSetWallpaper_TrueForAFormatWithNoEncoder ui-2 +TestCancelScan_NoOpWhenNotScanning ui-2 +TestClearToDropzone_FinishesInFlightScan ui-2 +TestClearToDropzone_HidesInfoCardButKeepsThePreference ui-2 +TestCloseFilesItem_EnabledAfterFilesLoaded ui-2 +TestCompareDividerKeys_ViewerRoutesShiftWithoutChangingCoveredState ui-2 +TestCompareExit_BackToGridLeavesTheGridSelectionAndTitleUntouched ui-2 +TestCompareGridLeak_PointerGesturesCannotScrollCoveredGrid ui-2 +TestCompareInputIsolation_TypedKeysAndRunesCannotReachCoveredGrid ui-2 +TestCompareLinkToggle_RequiresExactPhysicalControlL ui-2 +TestCompareMenuState_DisablesOrdinaryCommandsAndRestoresThemOnExit ui-2 +TestCompareOrientation_UsesCanonicalEXIFPixelsAndIgnoresViewerRotation ui-2 +TestCompareSelection_HiddenHostIndicesDetermineLeftAndRight ui-2 +TestCompareShortcut_PhysicalControlOpensComparison ui-2 +TestCompareSwipePointer_CanvasRoutesDividerAndPaneDrag ui-2 +TestCompareTitle_OwnsTheWindowUntilComparisonCloses ui-2 +TestCompareTitle_TracksIdentityOrderAndRestoresGridTitle ui-2 +TestCopyImageMenuActionCopiesActiveSelection ui-2 +TestCopyImageToClipboard_DispatchFailureShowsToast ui-2 +TestCopyImageToClipboard_NoImageIsNoop ui-2 +TestCopySelectionAnimatedFrame ui-2 +TestCopySelectionBusy ui-2 +TestCopySelectionCancel ui-2 +TestCopySelectionCancelsBeforeOtherCommands ui-2 +TestCopySelectionCommandKeyDoesNotCancel ui-2 +TestCopySelectionFocusLoss ui-2 +TestCopySelectionInfoOverlay ui-2 +TestCopySelectionSuccess ui-2 +TestCopy_WhileGridVisibleCopiesTheSelectionAsFileReferences ui-2 +TestCopy_WhileGridVisibleFallsBackToTheHighlightedCell ui-2 +TestCurrentPreferences_CarriesSecondaryWindowGeometry ui-2 +TestCurrentPreferences_DefaultFavoritePreviewCacheIsTrue ui-2 +TestE2E_LaunchWithSavedSessionShowsRestoreLink ui-2 +TestE2E_SuccessfulDropShowsImage ui-2 +TestExifLink_OpensExifWindow ui-2 +TestExportAs_AppendsTheExtensionOfTheFormatActuallyPicked ui-2 +TestExportAs_CancelWritesNothing ui-2 +TestExportAs_ReportsAFailedWrite ui-2 +TestFavoritePreviewCache_DefaultsToTrueOnStartup ui-2 +TestFavoritesMenuYieldsCopySelection ui-2 +TestHandleDrop_AcceptsPNGAndGIF ui-2 +TestHandleDrop_CancelsAnyPendingDeleteConfirmation ui-2 +TestHandleDrop_EmptyDrop ui-2 +TestHandleDrop_MergeModeDropWithNothingSupportedKeepsExistingSet ui-2 +TestHandleDrop_SingleFileDoesNotRecurse ui-2 +TestHandleDrop_SingleFileExpandsSiblingsAndKeepsOpened ui-2 +TestHandleDrop_TwoFilesInSameDirDoNotExpand ui-2 +TestHandleDrop_UnsupportedSingleFileDoesNotExpandFolder ui-2 +TestHandleKeyEvent_ArrowAfterEndWhileInspectingReturnsToGroup ui-2 +TestHandleKeyEvent_EscapeFromInspectReopensVariantsThenHideGrid ui-2 +TestHandleKeyEvent_EscapeLeavesPictureFrameModeWithoutResetting ui-2 +TestHandleKeyEvent_EscapeWithoutInspectStillResets ui-2 +TestHandleKeyEvent_ExportPromptSwallowsNavigationButRespondsToItsOwnKeys ui-2 +TestHandleKeyEvent_GTogglesGrid ui-2 +TestHandleKeyEvent_GridVisible_SwallowsNavigation ui-2 +TestHandleKeyEvent_HomeEndSkipHiddenExtras ui-2 +TestHandleKeyEvent_HomeEndWhileInspectingUseWholeSet ui-2 +TestHandleKeyEvent_IgnoredWhileAFyneDialogIsUp ui-2 +TestHandleKeyEvent_PlainDStillHidesWhenGridClosed ui-2 +TestHandleKeyEvent_ShiftDNoopOnUniqueDoesNotOpenGrid ui-2 +TestHandleKeyEvent_ShiftPTogglesShuffleAndPrefixesTitle ui-2 +TestHandleKeyEvent_UpDownAdjustIntervalInsteadOfNavigating ui-2 +TestHandleKeyEvent_VFromInspectLeavesInspectOn ui-2 +TestHandleKeyEvent_ZoomInThenOutClearsGrabCursor ui-2 +TestHandleKeyEvent_ZoomShortcuts ui-2 +TestHandleTypedRune_GridIsNotACanvasOverlay ui-2 +TestInvalidateLoad_WakesAnimateImmediately ui-2 +TestInvalidateSortCancelsAndFinalizesCurrentProgress ui-2 +TestJumpIfHiddenExtra_JumpsAgainAfterClearInspect ui-2 +TestJumpIfHiddenExtra_MovesToRepresentative ui-2 +TestManualUpdateCheck_ReadyCallbackUsesVersionString ui-2 +TestMaybeShowUpdateFailure_NoRecordShowsNothing ui-2 +TestMaybeShowUpdateFailure_ReportsWhateverVersionFailed ui-2 +TestMaybeShowWhatsNew_EmptyCacheDoesNotShow ui-2 +TestMemoryLimitGettersAndSetters ui-2 +TestNavigationDoesNotInvalidateScan ui-2 +TestOpenReleasesPage_HandsTheReleasesURLToTheApp ui-2 +TestOpenReleasesPage_LogsAnAppThatCannotOpenTheURL ui-2 +TestOpenWithHandler_DeliveryClosesTheGridAndCancelsAPendingDelete ui-2 +TestPerformDelete_OnLastImageOfMultipleAdvancesWithoutPanicking ui-2 +TestPerformUpdate_InvalidMissingOrSameStageDoesNotQuit ui-2 +TestPromptExport_DoesNothingWhileTheDeleteCardIsUp ui-2 +TestPromptExport_EachChoiceExportsItsOwnFormat ui-2 +TestPromptExport_ReopeningDoesNotResetAnAlreadyMadeChoice ui-2 +TestRecordWindowPosition_StillFeedsThePositionTracker ui-2 +TestRemoveFiles_DropsEveryIndexAndPurgesTheirCacheEntries ui-2 +TestReportClipboardError_ShowsToast ui-2 +TestResetRotation_AfterClearToDropzoneDoesNotPanic ui-2 +TestRotateBy_ResetsZoomFit ui-2 +TestRotateBy_StaticSizeDoesNotResize ui-2 +TestSVGReRenderNeverMutatesTheCachedEntry ui-2 +TestSVGReRendersAfterRotation ui-2 +TestSVGReRendersAtHigherDensityOnZoom ui-2 +TestSVGRotationSwapsLogicalAxes ui-2 +TestSVGRotationSwapsTheLogicalSizeZoomMeasuresAgainst ui-2 +TestSVGThenRasterClearsVectorState ui-2 +TestSaveItem_DisabledAfterCloseFiles ui-2 +TestSaveItem_DisabledInitially ui-2 +TestSaveRotation_PreservesJPEGExif ui-2 +TestSelectAllShortcut_IgnoredOutsideTheGrid ui-2 +TestSetAsWallpaper_DoesNothingWithoutAnImage ui-2 +TestSetAsWallpaper_ReportsAFailure ui-2 +TestSetCheckForUpdates_UpdatesGetterAndCurrentPreferences ui-2 +TestSetFavoritePreviewCacheOffCancelsAnInFlightPass ui-2 +TestSetFavoritePreviewCache_UpdatesGetterAndCurrentPreferences ui-2 +TestSetSlideInterval_ClampsToMinimum ui-2 +TestShiftDelete_OutsideTheGridStillPromptsForTheCurrentFile ui-2 +TestShowExifWindow_ContentAndRefreshOnNavigation ui-2 +TestShowExifWindow_NoopWithNothingLoaded ui-2 +TestShowImage_InPictureFrameModeEndsFullyOpaque ui-2 +TestShutdownClosesActiveComparisonBeforeEventLoopStops ui-2 +TestStartViewerRuntime_DefaultOffDoesNotAssignClient ui-2 +TestStartup_LoadsSavedPreferencesIntoViewer ui-2 +TestStartup_RestoresLastUpdateCheckDay ui-2 +TestStepImageBlockedWhileCopyPending ui-2 +TestStepImage_HideDuplicatesShowsHighestResolution ui-2 +TestStepImage_InspectLoopsVariantsNotUniques ui-2 +TestStepImage_NoopWhileDeleteConfirmVisible ui-2 +TestStepImage_NoopWhileLoading ui-2 +TestStepImage_SkipsHiddenExtras ui-2 +TestSweepUpdateBackup_KeepsTheBackupAfterAFailedRestore ui-2 +TestSweepUpdateBackup_KeepsTheBackupWhenTheRecordCannotBeRead ui-2 +TestSweepUpdateBackup_SweepsAfterAFailureThatLeftTheExecutableAlone ui-2 +TestSyncFavoritePreviews_EmptyListSweepsStalePreviews ui-2 +TestSyncFavoritePreviews_PreferenceOffWritesNothing ui-2 +TestSyncFavoritePreviews_SecondOpenAddsNothing ui-2 +TestSyncWindowToZoom_FitToWindowRestoresWindow ui-2 +TestSyncWindowToZoom_SlideshowActiveDoesNotResize ui-2 +TestSyncWindowToZoom_StaticSizeDoesNotResize ui-2 +TestToggleInfoOverlay_ZoomLineTracksZoomChanges ui-2 +TestToggleMergeMode_PrefixesTitleAndPersistsAcrossDrops ui-2 +TestUpdateCheck_RemovesStaleStage ui-2 +TestUpdateCheck_SettingOffNeverCallsHTTP ui-2 +TestUpdateFailureContent_NeedsNoScrollingOnARoomyWindow ui-2 +TestVectorRasterLandsWhenUIHopIsAsync ui-2 +TestViewerIndexStaysValidAcrossFileStateTransitions ui-2 +TestViewerReset_DoesNotReshowRestoreLinkOnceConsumed ui-2 +TestViewerShow_NavigatingAwayStopsAnimation ui-2 +TestViewerShow_RAWPreviewMarksTheTitle ui-2 +TestViewerShow_RejectsAbsurdHeaderDimensions ui-2 +TestWindowMenu_ExifActionOpensWhenAFileIsDisplayed ui-2 +TestWindowMenu_ExifDisabledWhileOpenAndEnabledAfterClose ui-2 +TestWindowMenu_GKeyWhileGridClosedMatchesToggle ui-2 +TestWindowMenu_GridActionNoopsWithoutFiles ui-2 +TestWindowMenu_PictureFrameActionEntersAndDoesNotToggleOff ui-2 +TestWindowMenu_PictureFrameDisabledDuringVariantsSession ui-2 +TestWireGlobalShortcuts_CopySelectionWithCopyShortcut ui-2 +TestZoomKeysDriveVectorRerenders ui-2 +TestZoomWidget_LayoutFillsViewportAtFit ui-2 +TestActionsMenu_AfterOneJPEGDrop ui-3 +TestActionsMenu_CloseFiles ui-3 +TestActionsMenu_FreshViewer ui-3 +TestActionsMenu_HideTogglesLikeD ui-3 +TestActionsMenu_RotateTurnsImageClockwise ui-3 +TestActionsMenu_SWithTwoFilesChecksCaptureDate ui-3 +TestActionsMenu_ShowVariantsNoopOnUnique ui-3 +TestActionsMenu_ShowVariantsNoopsDuringPictureFrame ui-3 +TestActionsMenu_ShowVariantsOpensGridOnPairAfterHide ui-3 +TestActionsMenu_ZoomInThenOutChangesPercent ui-3 +TestActionsMenu_ZoomNoopsWhileGridVisible ui-3 +TestAdvance_SingleFileDropWalksSiblings ui-3 +TestAdvance_WrapsAroundAtTheEnd ui-3 +TestAppStateClearFilesResetsFileState ui-3 +TestAppStateModelPreferences ui-3 +TestAppStateRemoveFileRemovesOneMatchingUnsortedDuplicate ui-3 +TestAppStateReplaceFilesCopiesAndResetsIndex ui-3 +TestApplySettings_AppliesOnlyChangedFields ui-3 +TestApplyStagedUpdate_RelaunchFailureRetainsNotesAndStage ui-3 +TestApplyStagedUpdate_UnverifiedStageNeverApplies ui-3 +TestAutomaticCheckDoesNotBlockOrSupersedeManualPreparation ui-3 +TestBatchDelete_LeavesTheWindowMaximizedOnAColdReload ui-3 +TestBatchDelete_RemovesEverySelectedFileAndLeavesTheGridOpen ui-3 +TestBuildMainMenu_ExportItemDisplaysItsAccelerator ui-3 +TestBuildMainMenu_ManualOpenedObserverSyncsWindowHelp ui-3 +TestBuildViewer_NoSavedSessionHidesRestoreLink ui-3 +TestBuildViewer_RegistersAllEightFeatures ui-3 +TestCanSaveRotation_FalseForUnsupportedFormat ui-3 +TestCanSaveRotation_FalseWhileLoading ui-3 +TestCanSaveRotation_TrueAfterRotatingAnEncodableFormat ui-3 +TestCanSetWallpaper_FalseWithNoImage ui-3 +TestCanSetWallpaper_TrueForALoadedImage ui-3 +TestCancelScan_CancelsInFlightScanWithNoFilesYet ui-3 +TestCaptureRegionCopySourceReleasesPauseWhenCaptureFails ui-3 +TestChooserErrorDetail_FallsBackToErrorString ui-3 +TestChooserErrorDetail_PrefersStderr ui-3 +TestClearToDropzone_PurgesTheImageCache ui-3 +TestCloseFilesClearsVector ui-3 +TestCloseFilesItem_DisabledWithNoFilesLoaded ui-3 +TestCloseFiles_CancelsScanInProgress ui-3 +TestCloseFiles_NeverClosesTheWindow ui-3 +TestCompareCancel_EscapeStopsPendingLoadsAndPreservesSelection ui-3 +TestCompareCommandIsolation_ShortcutsAreIgnored ui-3 +TestCompareDividerKeys_RemainActiveWhilePanesAreUnlinked ui-3 +TestCompareFailure_ReturnsToGridWithoutRemovingEitherFile ui-3 +TestCompareHelp_F1OpensManualWithoutLeavingComparison ui-3 +TestCompareLinkToggle_ControlAloneDoesNotChangeMode ui-3 +TestCompareLinkToggle_ZoomsOnlyTheLastHoveredPaneWithoutHeldModifier ui-3 +TestCompareOpenRefusal_DropDialogShortcutAndOpenWithAreDiscarded ui-3 +TestCompareRAW_UsesEmbeddedPreviewFromCanonicalLoader ui-3 +TestCompareRestoration_PreservesANonzeroGridScrollPosition ui-3 +TestCompareSelection_DuplicateFilterKeepsHiddenSelectedHostFile ui-3 +TestCopyImageToClipboard_DispatchesEncodedPNG ui-3 +TestCopySelectionAvailability ui-3 +TestCopySelectionBusyBlocksOtherCommands ui-3 +TestCopySelectionRAWPreview ui-3 +TestCopySelectionRepeatedActivation ui-3 +TestCopySelectionSVG ui-3 +TestCopySelectionTransparency ui-3 +TestCopySelectionZoomPanResize ui-3 +TestDupeCompute_ConcurrentWithFileRemoval ui-3 +TestE2E_BadDropWithNothingLoadedShowsPlaceholder ui-3 +TestE2E_EscapeQuitsWhenNothingLoaded ui-3 +TestE2E_EscapeResetsAfterImagesLoaded ui-3 +TestE2E_TappingDropzoneArtOpensFileDialog ui-3 +TestExportAs_AppendsTheFormatExtensionWhenThePickedNameCannotBeEncoded ui-3 +TestExportAs_ExportsAFormatThatHasNoEncoderOfItsOwn ui-3 +TestExportAs_HonorsAnEncodableExtensionTheUserTyped ui-3 +TestExportAs_JPEGSourceKeepsGPSExif ui-3 +TestExportAs_NoOpWithNothingLoaded ui-3 +TestExportAs_SuggestsTheSourceNameWithTheNewExtensionInItsOwnFolder ui-3 +TestExportAs_WritesTheDisplayedFrameToThePickedPath ui-3 +TestExportItem_DisabledInitiallyAndEnabledOnceAnImageLoads ui-3 +TestFavoritesMenuItemOpensStoredFilesThroughViewer ui-3 +TestFileSnapshot_KeysAndGenerationMoveTogether ui-3 +TestFinishLoad_ResetsRotationOnNavigation ui-3 +TestGridHighlight_NamesTheHighlightedFileInTheTitle ui-3 +TestGridHighlight_VariantsTitleUsesSizeAndPath ui-3 +TestHandleDrop_AcceptsRAW ui-3 +TestHandleDrop_ErrorAfterImagesClearsDisplay ui-3 +TestHandleDrop_FiltersUnsupportedFiles ui-3 +TestHandleDrop_MergeSingleFileDoesNotExpandSiblings ui-3 +TestHandleDrop_NoSupportedImages ui-3 +TestHandleDrop_SiblingScanTruncationToast ui-3 +TestHandleDrop_TruncatedScanToastNamesTheCap ui-3 +TestHandleKeyEvent_DNoopWhileInspecting ui-3 +TestHandleKeyEvent_DeleteCardIsNotACanvasOverlay ui-3 +TestHandleKeyEvent_DeleteConfirmSwallowsNavigationButRespondsToItsOwnKeys ui-3 +TestHandleKeyEvent_DialogSwallowsTheAppsOtherKeysToo ui-3 +TestHandleKeyEvent_EOpensExifWindow ui-3 +TestHandleKeyEvent_EscapeDuringFirstDropReorderDoesNotCloseWindow ui-3 +TestHandleKeyEvent_EscapeDuringResortOfExistingFilesDoesNotClearThem ui-3 +TestHandleKeyEvent_GIsIgnoredDuringPictureFrameMode ui-3 +TestHandleKeyEvent_GridVisible_ReturnNavigatesAndCloses ui-3 +TestHandleKeyEvent_LeftRightUseStepImage ui-3 +TestHandleKeyEvent_PNoopWhileInspecting ui-3 +TestHandleKeyEvent_ShiftDOpensGridOnCurrentGroup ui-3 +TestHandleKeyEvent_VLeavesPictureFrameMode ui-3 +TestHandleTypedRune_GridVisible_SwallowedByDeleteConfirmation ui-3 +TestHandleTypedRune_OnlyReachesTheGridWhileItIsUp ui-3 +TestInvalidateLoad_ZeroValueIsSafe ui-3 +TestJumpIfHiddenExtra_NoopWhileInspecting ui-3 +TestManualUpdateCheck_BypassesSettingAndDailyGate ui-3 +TestManualUpdateCheck_PreparationFailureArrivesThroughCallback ui-3 +TestMaxScanGetterSetter ui-3 +TestMaybeShowUpdateFailure_ClearsRecord ui-3 +TestMaybeShowWhatsNew_VersionMismatchDoesNotShow ui-3 +TestMergeModeGetterSetter ui-3 +TestNormalizePreferenceDefaults ui-3 +TestOpenFileDialog_RunsChooserInBackground ui-3 +TestOpenInitialFiles_OpensPendingWhenNothingWasDelivered ui-3 +TestOpenWithHandler_DeliveryMergesWhenMergeModeIsOn ui-3 +TestPerformDelete_LastFileReturnsToEmptyDropzone ui-3 +TestPerformDelete_OSFailureKeepsTheFileAndToastsAnError ui-3 +TestPerformUpdate_RecordsRelaunchThenQuits ui-3 +TestRandomOtherIndex_CoversEveryOtherIndex ui-3 +TestRandomOtherIndex_EmptyOrSingleReturnsCurrentUnchanged ui-3 +TestRandomOtherIndex_NeverReturnsCurrent ui-3 +TestRasterFormatKeepsNoVectorState ui-3 +TestRasterizeVectorStopsOnShutdownSignal ui-3 +TestRecordWindowPosition_CounterClockwiseSpiralDragFiresTheOtherWay ui-3 +TestRefreshMenus_FavoriteChangeRepublishesTheBarThroughTheViewer ui-3 +TestRequestDelete_DoesNothingWhileTheExportPromptIsUp ui-3 +TestRequestLifecycleBeginCancelsAndSupersedesPrevious ui-3 +TestRequestLifecycleConcurrentBeginLeavesOneCurrentToken ui-3 +TestRequestLifecycleInvalidateCancelsWithoutReplacement ui-3 +TestRequestTokenCancelDoesNotSupersedeNewerRequest ui-3 +TestResetRotation_Key0ClearsRotationAndZoom ui-3 +TestResizeToImage ui-3 +TestRestoreSession_LoadsSavedFilesAndHidesLink ui-3 +TestRevisionZeroValueAdvancesAndMatches ui-3 +TestRotateBy_CounterClockwiseWraps ui-3 +TestRotateBy_FourStepsReturnsToStart ui-3 +TestRotateBy_NoImageIsNoOp ui-3 +TestRotateBy_SwapsBoundsAndResizesWindow ui-3 +TestRotatedNonSquareSVGKeepsItsAspectRatio ui-3 +TestRotatingAZoomedSVGSizesTheWindowFromItsLogicalSize ui-3 +TestRunFileChooser_CancelLeavesStateUntouched ui-3 +TestRunFileChooser_LoadsSelectedImage ui-3 +TestSVGThumbnailsInTheGridOverview ui-3 +TestSaveItem_EnabledAfterRotatingAndDisabledAfterNavigatingAway ui-3 +TestSaveRotation_FailedWriteLeavesRotationUnchanged ui-3 +TestSaveRotation_NoOpWhenNothingToSave ui-3 +TestSelectAllShortcut_PicksEveryCellWhileTheGridIsUp ui-3 +TestSetAsWallpaper_PointsTheOSAtACopyOfTheFrameOnScreen ui-3 +TestSetFavoritePreviewCacheOnLeavesAPassAlone ui-3 +TestSetMaxWindowSize_FloorsAtTheDropZoneSize ui-3 +TestSetSlideInterval_TakesEffect ui-3 +TestSetSortMode_SnapshotDoesNotAliasUnsortedFiles ui-3 +TestSettingsState_SubstitutesSlideIntervalDefault ui-3 +TestShiftDelete_IgnoredWithNothingLoaded ui-3 +TestShiftDelete_WhileGridVisibleFallsBackToTheHighlightedCell ui-3 +TestShowWindowGrid_FromInspectReopensVariants ui-3 +TestShowWindowPictureFrame_NoopsDuringVariantsSession ui-3 +TestShow_TracksAnimatedGIFLoopDuration ui-3 +TestSlideShuffleGetterSetter ui-3 +TestStaleFileStateCompletionsDoNotOverwriteNewerState ui-3 +TestStartViewerRuntime_ReplacesConstructionStopAfterGeometryRestoration ui-3 +TestStartWindowPosPolling_PanicsWithoutConstructedSlideshow ui-3 +TestStaticWindowSize_DefaultsToFalseOnStartup ui-3 +TestStaticWindowSize_ResetKeepsWindowSize ui-3 +TestStepImageYieldsIdleCopySelection ui-3 +TestStepImage_NextAndPrevWrap ui-3 +TestStepImage_NoopWhileFyneDialogIsUp ui-3 +TestStepImage_NoopWithOneFile ui-3 +TestSuggestedExportPath ui-3 +TestSyncFavoritePreviews_WarmsTheGridThumbnailCache ui-3 +TestSyncMenus_KeepsFavoritesAddItemInStep ui-3 +TestSyncMenus_TracksTheWallpaperItem ui-3 +TestSyncWindowToZoom_ActualSizeOnLargeImage ui-3 +TestSyncWindowToZoom_GridVisibleDoesNotResize ui-3 +TestSyncWindowToZoom_TinyImageZoomOutStaysAtFloor ui-3 +TestTappableArea_HoverInvokesOnHoverCallback ui-3 +TestTappableArea_HoverIsOptional ui-3 +TestTappableArea_TappedInvokesCallback ui-3 +TestToggleInfoOverlay_ContentAndPersistenceAcrossNavigation ui-3 +TestToggleInfoOverlay_RAWMarksPreview ui-3 +TestTogglePictureFrameMode_EntersAndExitsFullScreen ui-3 +TestToggleSort_CyclesThroughAllModesAndBackToName ui-3 +TestUpdateCheck_NewerReleaseStagesBinary ui-3 +TestUpdateCheck_SameVersionRecordsDayNoStage ui-3 +TestUpdateCheck_VerifierFailurePreservesLifecycle ui-3 +TestUpdateFailureBodySize_TracksTheCanvasBetweenItsBounds ui-3 +TestUpdateFailureMessage_AccessDeniedNamesFolderAccessAndPath ui-3 +TestUpdateFailureMessage_ControlledFolderAccessIsWindowsOnly ui-3 +TestUpdateFailureMessage_EachClassifiedReasonHasItsOwnWording ui-3 +TestUpdateFailureMessage_EmptyRecordStillSaysSomethingUseful ui-3 +TestUpdateFailureMessage_EveryReasonReassuresAndStaysWellFormed ui-3 +TestUpdateFailureMessage_SharingViolationSaysTheFileWasInUse ui-3 +TestUpdateFailureMessage_UnknownReasonFallsBackToGeneric ui-3 +TestUpdateFailureMessage_VirusBlockedNamesTheScannerWithoutThePath ui-3 +TestVectorFloorMatchesStartWindowSize ui-3 +TestVectorNeedsRender ui-3 +TestVectorRasterTarget ui-3 +TestViewerFileStateSlicesRemainEquivalentAcrossTransitions ui-3 +TestViewerModesApplyBeforeAndAfterLoadingFiles ui-3 +TestViewerReset_ReshowsRestoreLinkWhenSessionUnconsumed ui-3 +TestViewerShow_AllFilesBrokenFallsBackToEmptyState ui-3 +TestViewerShow_DecodeErrorKeepsHint ui-3 +TestWindowMenu_CloseFilesDisablesGridPictureFrameExifAndViewer ui-3 +TestWindowMenu_FreshViewerDisablesSurfacesExceptHelp ui-3 +TestWindowMenu_PictureFrameLeavesGridItemDisabled ui-3 +TestWindowMenu_PictureFrameToggleGreysPFEnablesViewerDisablesGrid ui-3 +TestWindowSizeTracker_RecordsResizes ui-3 +TestWireAddFavoritesShortcut_DoesNothingWhileTheDeleteCardIsUp ui-3 +TestWireAddFavoritesShortcut_OpensTheDialog ui-3 +TestWireClipboardShortcuts_CopiesImageAndPath ui-3 +TestWireDeleteShortcut_ShiftDeleteOpensConfirmation_PlainCutIgnored ui-3 +TestWireFavoriteShortcutsMapsDigitsToFavoriteSlots ui-3 +TestWireManageFavoritesShortcut_DoesNothingWhileTheDeleteCardIsUp ui-3 +TestWireManageFavoritesShortcut_OpensTheDialog ui-3 +TestZoomGeometryCallbackSkipsInactiveMode ui-3 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e218ab3..190ec01 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -18,8 +18,9 @@ permissions: contents: read jobs: - build-and-test: - runs-on: ubuntu-latest + validation: + name: Validation (attempt ${{ github.run_attempt }}) + runs-on: ubuntu-24.04 steps: - name: Check out code uses: actions/checkout@v7 @@ -41,6 +42,9 @@ jobs: sudo apt-get update sudo apt-get install -y gcc libgl1-mesa-dev xorg-dev libwayland-dev libxkbcommon-dev + - name: Validate UI shard manifest + run: make check-test-shards-direct + - name: Vet run: go vet ./... @@ -64,19 +68,72 @@ jobs: # the same reason. run: GOOS=windows GOARCH=amd64 go vet ./internal/... - - name: Test - # The runner's default locale is "C", which is not a well-formed - # BCP-47 tag, so Fyne logs a three-line parse fault before any test - # runs. Naming a real one keeps that noise out of the test output. - # - # -timeout 30m: go test defaults to 10m per package. internal/ui on - # ubuntu-latest already took ~9m39s before folder-sibling tests; the - # next commit then panicked at 10m0s while a later test was starting. + linux-race: + name: Linux race (${{ matrix.partition }}, attempt ${{ github.run_attempt }}) + runs-on: ubuntu-24.04 + strategy: + fail-fast: false + matrix: + partition: [non-ui, ui-1, ui-2, ui-3] + steps: + - name: Check out code + uses: actions/checkout@v7 + + - name: Set up Go + uses: actions/setup-go@v7 + with: + go-version-file: go.mod + cache-dependency-path: go.sum + + - name: Install Linux GUI build dependencies + run: | + sudo apt-get update + sudo apt-get install -y gcc libgl1-mesa-dev xorg-dev libwayland-dev libxkbcommon-dev + + - name: Run ${{ matrix.partition }} race contract (attempt ${{ github.run_attempt }}) + id: race + env: + TEST_CAPTURE: ${{ runner.temp }}/go-test-linux-race-${{ matrix.partition }}.json + run: | + case "${{ matrix.partition }}" in + non-ui) + make --no-print-directory test-race-non-ui-direct + ;; + ui-1|ui-2|ui-3) + make --no-print-directory test-race-ui-direct TEST_SHARD="${{ matrix.partition }}" + ;; + *) + echo "Unexpected Linux race partition: ${{ matrix.partition }}" >&2 + exit 2 + ;; + esac + + - name: Summarize ${{ matrix.partition }} race result (attempt ${{ github.run_attempt }}) + if: always() env: - LANG: en_US.UTF-8 - run: go test -timeout 30m -race ./... + PARTITION: ${{ matrix.partition }} + RACE_OUTCOME: ${{ steps.race.outcome }} + RUN_ATTEMPT: ${{ github.run_attempt }} + run: | + { + echo "## Linux race \`$PARTITION\` (attempt \`$RUN_ATTEMPT\`)" + echo + echo "- Workflow attempt: \`$RUN_ATTEMPT\`" + echo "- Test step outcome: \`$RACE_OUTCOME\`" + echo "- Raw-event artifact: \`linux-race-$PARTITION-$GITHUB_RUN_ID-attempt-$GITHUB_RUN_ATTEMPT\`" + } >> "$GITHUB_STEP_SUMMARY" + + - name: Upload ${{ matrix.partition }} raw events (attempt ${{ github.run_attempt }}) + if: always() + uses: actions/upload-artifact@v7 + with: + name: linux-race-${{ matrix.partition }}-${{ github.run_id }}-attempt-${{ github.run_attempt }} + path: ${{ runner.temp }}/go-test-linux-race-${{ matrix.partition }}.json + if-no-files-found: error + retention-days: 14 windows-test: + name: Windows tests (attempt ${{ github.run_attempt }}) runs-on: windows-latest steps: - name: Check out code @@ -93,8 +150,8 @@ jobs: # runner doesn't have by default. Not ./...: internal/ui isn't # excluded for lack of a display driver - it uses the same headless # fyne.io/fyne/v2/test driver as internal/ui/autoupdate below. It's - # excluded because it already costs ~9m39s on ubuntu-latest (see the - # Test step above) and could carry golden-master/font differences on - # Windows; these two packages are the file-replacement update logic, - # not the concurrent UI. + # excluded because it already costs ~9m39s in a monolithic Ubuntu run + # and could carry golden-master/font differences on Windows; these two + # packages are the file-replacement update logic, not the concurrent + # UI. run: go test ./internal/update/... ./internal/ui/autoupdate/... diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index af37960..6bf68e7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -15,6 +15,22 @@ Entry point only. `main.go` calls `openwith.Install` (first statement, see `ui.Run`. `main_darwin_test.go` asserts the graft landed — this is the only test binary that links the Cocoa driver. +### `scripts/testshards` + +Repository tooling for measured Linux race-test sharding. It parses complete +`go test -json` streams, emits deterministic package/top-level-test summaries, +plans a reviewable manifest from median durations with stable LPT tie-breaking, +checks that manifest against the build-selected Linux test inventory, derives +the fail-closed non-UI package partition, and emits exact shard filters. Its +capture path streams compact partition/package/test diagnostics while retaining +the raw events outside the repository; the generated assignment lives at +`.github/testshards/internal-ui.tsv`. + +| File | Responsibility | +|------|----------------| +| `main.go` | `summarize`, `plan`, `check`, `regex`, `partition`, and `capture` command paths: complete event-stream validation, deterministic assignment, live build-selected inventory and parallel-call checks, exact anchored filters, exact-package subtraction, concise diagnostics, and raw preservation. | +| `main_test.go` | Command-boundary fixtures for event streams and capture, deterministic planning, every manifest rejection, build-selected runnable forms, parallel-call refusal, exact filter generation, package partitioning, and Make contract expansion. | + ### `internal/ui` The application. Unexported `appState` is the file-set model (scan/drop @@ -405,6 +421,7 @@ see `AGENTS.md`. - "How is the last session saved/restored?" → `internal/session` + `session.go` `restoreSession`. - "How do in-app updates work?" → `internal/update` + `internal/ui/autoupdate` (serialized automatic/manual checks, staging, apply intent, What's-New cache, apply-failure cache) + `internal/ui/autoupdate.go` (`maybeStartUpdateCheck` / `CheckForUpdatesNow` / `PerformUpdate` / `maybeShowWhatsNew` / `maybeShowUpdateFailure`) + `settingswin` (manual dialogs) + `help/whatsnew.go` (the window). Automatic checks are off by default (`preferences.CheckForUpdates`) and stage silently. Apply remains OnStopped: normal shutdown installs without relaunch; explicit Perform update adds a post-apply relaunch. On Windows that relaunch starts the new executable with `PICFETCH_UPDATE_AWAIT_PID` set to the installing process's PID; `update.CleanupPredecessor` (`internal/update/await.go`), called from `main.go` before `app.NewWithID`, waits on that PID before preferences are touched, unsets the variable, and sweeps leftovers from pre-2026-08-30 updates. If `update.Apply` fails, `ClassifyApplyError` (`internal/update/applyerr.go`) records the reason via `autoupdate.SaveApplyFailure`, and `maybeShowUpdateFailure` explains it on the next launch with a button to the releases page — releases are unsigned, so Controlled Folder Access can still deny the write even to `picfetch.exe` itself. GitHub TUF bootstrap expiry: `tufroot.go`. - "How are GitHub release notes written?" → `todos.md` `## Done` + `scripts/releasenotes` + `make release` + `.github/workflows/release.yml` `body_path`. +- "How are Linux race-test shards measured and assigned?" → `scripts/testshards` + `.github/testshards/internal-ui.tsv` + the measured CI sharding plan. - "How is a WinGet publish gated after Release?" → `.github/workflows/winget.yml` + `scripts/wingettag` (vX.Y.Z allowlist; `workflow_run` must be `release.yml` on a published tag). - "How does a macOS Open With reach the viewer?" → `internal/openwith` (queue + Objective-C graft) + `main.go` `openwith.Install` + `internal/ui/openwith.go` + `run.go` `SetOnStarted`. - "How does the packaged macOS app declare file/folder associations (Open With)?" → `internal/imaging/loader.go` `SupportedExtensions` + `scripts/plistdoctypes` + `Makefile` `package-mac`. diff --git a/Makefile b/Makefile index 847efa6..970ee71 100644 --- a/Makefile +++ b/Makefile @@ -17,8 +17,14 @@ TEST_TIMEOUT := 30m TEST_IMAGE := ubuntu:24.04 TEST_CONTAINER_LABEL := io.github.frathe.picfetch.test=true TEST_RACE := +TEST_RACE_FLAGS := -race -count=1 -timeout $(TEST_TIMEOUT) +TEST_LOCALE := en_US.UTF-8 +TEST_SHARD_MANIFEST := .github/testshards/internal-ui.tsv +TEST_SHARD_PACKAGE := ./internal/ui +TEST_PARTITION := +TEST_CAPTURE ?= /tmp/picfetch-test-$(TEST_PARTITION).json -.PHONY: all build build-linux-all run fmt fmt-check vet test update-test-image enter-test-container test-native test-race verify golden tidy clean package-mac package-windows package-windows-debug package-linux package-linux-debug build-all install-tools install-linux-tools security security-govulncheck security-github bump-version release check-tuf-root sync-tuf-root sync-qodana-test-exclusions check-qodana-test-exclusions help +.PHONY: all build build-linux-all run fmt fmt-check vet test update-test-image enter-test-container test-native test-race test-race-direct test-race-non-ui-direct test-race-ui-direct verify golden tidy clean package-mac package-windows package-windows-debug package-linux package-linux-debug build-all install-tools install-linux-tools security security-govulncheck security-github bump-version release check-tuf-root sync-tuf-root sync-qodana-test-exclusions check-qodana-test-exclusions check-test-shards check-test-shards-direct help all: build @@ -116,6 +122,53 @@ check-qodana-test-exclusions: ## Fail if qodana.yaml does not exclude every *_te vet: ## Run go vet go vet ./... +check-test-shards: ## Validate the UI shard manifest against the live Linux/amd64 test inventory + docker run --rm --platform linux/amd64 \ + --label "$(TEST_CONTAINER_LABEL)" \ + -v "$(CURDIR):/work" -w /work \ + -v picfetch-go-build-linux-amd64:/root/.cache/go-build \ + -v picfetch-go-mod-linux-amd64:/root/go/pkg/mod \ + $(TEST_IMAGE) bash -c '\ + set -e; \ + apt-get update -qq; \ + apt-get install -y -qq make gcc libgl1-mesa-dev xorg-dev libwayland-dev libxkbcommon-dev golang-go ca-certificates >/dev/null; \ + make --no-print-directory check-test-shards-direct \ + ' + +# Internal entry point for a prepared Linux/amd64 runner. Use the public Docker +# target above for canonical validation from any host. +check-test-shards-direct: + go run ./scripts/testshards check -package "$(TEST_SHARD_PACKAGE)" -manifest "$(TEST_SHARD_MANIFEST)" + +# Internal entry points for a prepared Linux/amd64 runner. The public test-race +# target enters Docker once and runs this sequence there; hosted CI can call the +# partition targets directly without nesting Docker. +test-race-direct: + @$(MAKE) --no-print-directory check-test-shards-direct + @$(MAKE) --no-print-directory test-race-non-ui-direct + @$(MAKE) --no-print-directory test-race-ui-direct TEST_SHARD=ui-1 + @$(MAKE) --no-print-directory test-race-ui-direct TEST_SHARD=ui-2 + @$(MAKE) --no-print-directory test-race-ui-direct TEST_SHARD=ui-3 + +test-race-non-ui-direct: override TEST_PARTITION := non-ui +test-race-non-ui-direct: + bash -c '\ + set -eu -o pipefail; \ + packages="$$(go run ./scripts/testshards partition -package "$(TEST_SHARD_PACKAGE)")"; \ + LANG="$(TEST_LOCALE)" go test $(TEST_RACE_FLAGS) -json $$packages | \ + go run ./scripts/testshards capture -out "$(TEST_CAPTURE)" -partition "$(TEST_PARTITION)" \ + ' + +test-race-ui-direct: override TEST_PARTITION = $(TEST_SHARD) +test-race-ui-direct: + bash -c '\ + set -eu -o pipefail; \ + case "$(TEST_SHARD)" in ui-1|ui-2|ui-3) ;; *) echo "TEST_SHARD must be one of ui-1, ui-2, or ui-3" >&2; exit 2;; esac; \ + filter="$$(go run ./scripts/testshards regex -manifest "$(TEST_SHARD_MANIFEST)" -shard "$(TEST_SHARD)")"; \ + LANG="$(TEST_LOCALE)" go test $(TEST_RACE_FLAGS) -json -run "$$filter" $(TEST_SHARD_PACKAGE) | \ + go run ./scripts/testshards capture -out "$(TEST_CAPTURE)" -partition "$(TEST_PARTITION)" \ + ' + update-test-image: ## Pull the latest Linux/amd64 Ubuntu image used by Docker tests docker pull --platform linux/amd64 "$(TEST_IMAGE)" @@ -129,7 +182,7 @@ test: ## Run tests in Linux/amd64 Docker, matching CI and golden rendering (need $(TEST_IMAGE) bash -c '\ set -e; \ apt-get update -qq; \ - apt-get install -y -qq gcc libgl1-mesa-dev xorg-dev libwayland-dev libxkbcommon-dev golang-go ca-certificates locales procps htop >/dev/null; \ + apt-get install -y -qq make gcc libgl1-mesa-dev xorg-dev libwayland-dev libxkbcommon-dev golang-go ca-certificates locales procps htop >/dev/null; \ locale-gen en_US.UTF-8 >/dev/null; \ export LANG=en_US.UTF-8; \ status=0; \ @@ -161,8 +214,23 @@ enter-test-container: ## Open Bash in the running test container (htop/top avail test-native: ## Run tests directly on the current OS/architecture go test -timeout $(TEST_TIMEOUT) ./... -test-race: TEST_RACE := -race -test-race: test +test-race: ## Run the guarded race partitions sequentially in one Linux/amd64 Docker container + docker run --rm --platform linux/amd64 \ + --label "$(TEST_CONTAINER_LABEL)" \ + -v "$(CURDIR):/work" -w /work \ + -v picfetch-go-build-linux-amd64:/root/.cache/go-build \ + -v picfetch-go-mod-linux-amd64:/root/go/pkg/mod \ + -e HOST_UID=$$(id -u) -e HOST_GID=$$(id -g) \ + $(TEST_IMAGE) bash -c '\ + set -e; \ + apt-get update -qq; \ + apt-get install -y -qq make gcc libgl1-mesa-dev xorg-dev libwayland-dev libxkbcommon-dev golang-go ca-certificates locales procps htop >/dev/null; \ + locale-gen $(TEST_LOCALE) >/dev/null; \ + status=0; \ + make --no-print-directory test-race-direct || status=$$?; \ + if [ -d internal/ui/testdata/failed ]; then chown -R "$$HOST_UID:$$HOST_GID" internal/ui/testdata/failed; fi; \ + exit $$status \ + ' verify: fmt-check check-tuf-root check-qodana-test-exclusions ## Run the same checks CI does (format, TUF root, Qodana exclusions, vet, build, race tests) go vet ./... diff --git a/plans/2026-09-02-address-pr-13-review.md b/finished_refactorings/2026-09-02-address-pr-13-review.md similarity index 100% rename from plans/2026-09-02-address-pr-13-review.md rename to finished_refactorings/2026-09-02-address-pr-13-review.md diff --git a/.scratch/swipe-unlinked-pointer-routing/issues/01-route-swipe-input-by-reveal.md b/finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/issues/01-route-swipe-input-by-reveal.md similarity index 100% rename from .scratch/swipe-unlinked-pointer-routing/issues/01-route-swipe-input-by-reveal.md rename to finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/issues/01-route-swipe-input-by-reveal.md diff --git a/.scratch/swipe-unlinked-pointer-routing/issues/02-preserve-right-wheel-anchor.md b/finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/issues/02-preserve-right-wheel-anchor.md similarity index 100% rename from .scratch/swipe-unlinked-pointer-routing/issues/02-preserve-right-wheel-anchor.md rename to finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/issues/02-preserve-right-wheel-anchor.md diff --git a/.scratch/swipe-unlinked-pointer-routing/issues/03-document-review-and-verify.md b/finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/issues/03-document-review-and-verify.md similarity index 100% rename from .scratch/swipe-unlinked-pointer-routing/issues/03-document-review-and-verify.md rename to finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/issues/03-document-review-and-verify.md diff --git a/.scratch/swipe-unlinked-pointer-routing/spec.md b/finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/spec.md similarity index 100% rename from .scratch/swipe-unlinked-pointer-routing/spec.md rename to finished_refactorings/2026-09-02-swipe-unlinked-pointer-routing/spec.md diff --git a/plans/2026-09-02-windows-release-signing.md b/finished_refactorings/2026-09-02-windows-release-signing.md similarity index 100% rename from plans/2026-09-02-windows-release-signing.md rename to finished_refactorings/2026-09-02-windows-release-signing.md diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding.md new file mode 100644 index 0000000..fad910e --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding.md @@ -0,0 +1,994 @@ +# Measured CI test sharding + +Status: Complete. The controlled monolithic and sharded checkpoints each have +three accepted same-commit attempts; the final topology passes its coverage, +failure-gating, balance, elapsed-time, and runner-cost assessment. + +Route: Deep + +Prerequisite: satisfied. The first signed Windows release completed in the +[`v0.2.17` release run](https://github.com/frathe/picfetch/actions/runs/33668786016), +including the Windows signing gate. This work remains a separate pull request. + +## Deliverable + +Reduce the elapsed time of the reusable GitHub Actions CI gate by running its +independent work concurrently and by splitting the exact `internal/ui` package +across three isolated Linux race-test runners. Preserve every existing test, +package, race-detector invocation, timeout, locale, Windows check, and release +gate. Base shard assignment and the final assessment on measured data rather +than an assumed 8-12 minute result. + +The pull request has two mandatory CI checkpoints: + +1. an instrumented but still monolithic Linux race run, repeated three times at + one commit; and +2. the final sharded workflow, repeated three times at one later commit. + +The agent does not create either commit. At each checkpoint the user commits +and pushes the prepared tree, then reruns the same workflow attempt twice so +all three samples use the same SHA. + +## Scope + +- Instrument the current CI well enough to separate workflow/job setup, + validation/build work, non-UI packages, and exact `internal/ui` tests. +- Add a small standard-library Go command at `scripts/testshards` for duration + summaries, deterministic assignment, manifest validation, anchored shard + regexes, and concise JSON capture. +- Check in an exact Linux `top-level test name -> shard` manifest for three UI + shards. +- Make local `make verify` exercise the same four Linux race commands + sequentially in one Linux/amd64 Docker container. +- Split CI into one validation/build job, a four-entry Linux race matrix, and + the existing independent Windows test job. +- Record both elapsed-time improvement and the runner-capacity trade-off. + +## Non-goals + +- Removing `-race`, changing the 30-minute package timeout, skipping tests, + narrowing package coverage, making jobs optional, or using + `continue-on-error`. +- Broadly adding `t.Parallel()` to UI tests. The exact `internal/ui` package + intentionally depends on process isolation for Fyne and other global state. +- Adding a fourth UI shard automatically. A result above 12 minutes is a + measurement and review point, not permission to weaken the suite or consume + more runners. +- Refactoring application packages or changing product behavior. +- Changing the Windows test command or adding a Windows race toolchain. +- Changing release packaging/signing, `.github/workflows/release.yml`, branch + protection, or repository rulesets. +- Committing raw JSON artifacts or keeping local/CI timing files in the source + tree. +- Creating `CONTEXT.md` or an ADR. This is a reversible CI implementation and + introduces no new domain terminology or durable architectural policy. + +## Evidence before implementation + +### Existing CI observations + +The current `build-and-test` job serializes validation/build work before one +`go test -race ./...` command. The exact `internal/ui` package dominates that +command. These three recent current-code executions are useful coarse evidence, +but they are not the controlled baseline required by this plan because one is +a reusable release invocation and the attempts do not share one SHA. + +| Execution | Critical job | Before race step | Race step | Exact `internal/ui` | +|---|---:|---:|---:|---:| +| [Current `main`](https://github.com/frathe/picfetch/actions/runs/33673120188) | 22:00 | 2:23 | 19:36 | 16:19 | +| [Release commit on `main`](https://github.com/frathe/picfetch/actions/runs/33668780732) | 23:52 | 4:15 | 19:34 | 16:16 | +| [Reusable CI in the release](https://github.com/frathe/picfetch/actions/runs/33668786016) | 21:42 | 2:29 | 19:10 | 16:03 | +| Median | **22:00** | **2:29** | **19:34** | **16:16** | + +The durations are not additive: Go can run packages concurrently within the +single race command. Phase 1 must replace this coarse comparison with three +instrumented attempts and explicit package/test data. + +### Fresh local Linux/amd64 baseline + +Docker was restarted and the canonical Make target was run once at clean HEAD +`6af8c51f9bdfb0b36907be8b5c8241586ec3a602`: + +```sh +/usr/bin/time -p make --silent test-race \ + TEST_RACE='-race -count=1 -json' \ + > /private/tmp/picfetch-baseline.dKv4tT/go-test-linux-amd64.json +``` + +| Observation | Duration/result | +|---|---:| +| Docker wall time, including image setup and locale installation | 13:22.04 | +| First-to-last JSON event | 11:13.53 | +| Setup/compile difference | 2:08.51 | +| Exact `internal/ui` package | 11:12.513 | +| UI top-level terminal events | 567/567 passed | +| Sum of UI top-level elapsed values | 11:11.36 | +| Last non-UI package completion after the first timed event | 0:27.92 | + +The raw file has 11,401 lines, is 2,480,273 bytes, and has SHA-256 +`e728a20d192448490691b87cb11ed7ed21ed4fda64f06b5c21669d2222debe5c`. +It is temporary evidence, not a repository artifact. + +The five slowest UI tests in that run were: + +| Test | Elapsed | +|---|---:| +| `TestCompareCommandEntryPoints_MenuAndFeatureCallbacksAreIgnored` | 29.05s | +| `TestCopySelectionCancelsBeforeOtherCommands` | 26.19s | +| `TestCompareOpenRefusal_DropDialogShortcutAndOpenWithAreDiscarded` | 9.85s | +| `TestCopySelectionAvailability` | 8.33s | +| `TestCompareHelp_F1OpensManualWithoutLeavingComparison` | 7.03s | + +A deterministic longest-processing-time allocation of this single run would +produce theoretical test-only totals of 223.72s, 223.81s, and 223.83s, with +211, 177, and 179 top-level tests. Those figures are only a feasibility check: +the checked-in manifest must be generated from the three fresh CI baseline +attempts, and real shards also repeat compilation and `TestMain` setup. + +The large local/CI difference confirms that local data is suitable for test +weights and correctness checks, while CI data is authoritative for CI elapsed +time and acceptance. + +## Locked decisions + +| Area | Decision | +|---|---| +| Primary metric | Reusable CI gate wall time, from workflow creation until every required called-workflow job completes. | +| Secondary metrics | Queue-excluded execution window, each job/step duration, shard/package/test medians, and summed runner minutes. | +| Target | Aim for an 8-12 minute median. It is not a hard gate; test quality and complete coverage take priority. | +| Comparable samples | Three attempts at one monolithic checkpoint SHA and three attempts at one sharded checkpoint SHA. Each Linux race command uses `-count=1`; rerun request time is recorded for the queue-inclusive metric. | +| Linux runner | Pin all Linux jobs to `ubuntu-24.04`; do not use the moving `ubuntu-latest` label. | +| Job topology | One validation/build job, one four-entry `linux-race` matrix (`non-ui`, `ui-1`, `ui-2`, `ui-3`), and the existing Windows test job: six concurrent job slots in total. | +| Dependencies | The six jobs have no `needs` edges between them. Completion of the reusable workflow remains the release gate, so every job is mandatory. | +| Matrix failure policy | `fail-fast: false`, with the workflow's existing `cancel-in-progress: true`; no optional jobs and no hidden failures. | +| UI shard count | Exactly three initially. Do not add a fourth without a new measured decision. | +| Assignment | A checked-in exact mapping from each Linux-selected top-level `Test`, `Fuzz`, or `Example` in package `internal/ui` to exactly one of `ui-1..ui-3`. Subtests remain with their parent. | +| Initial balance | Median each top-level test's duration across the three baseline JSON files, then use deterministic longest-processing-time assignment. Sort equal durations by test name; break equal shard loads by shard number. | +| Guard | Fail on an unassigned inventory name, duplicate assignment, stale name, malformed row, unknown shard, empty shard, or newly introduced `t.Parallel()` call in the exact UI package. | +| Non-UI scope | `go list ./...` minus only the exact import path returned by `go list ./internal/ui`; keep the root package, scripts, and all `internal/ui/...` subpackages. | +| Linux safety flags | Every final Linux test invocation retains `LANG=en_US.UTF-8`, `-race`, `-count=1`, and `-timeout 30m`. | +| Windows scope | Preserve `go test ./internal/update/... ./internal/ui/autoupdate/...` as an independent `windows-latest` job. | +| Logs/artifacts | Stream concise failure/test summaries, upload raw `go test -json` for every Linux matrix entry with `if: always()`, `if-no-files-found: error`, and 14-day retention. Include matrix shard and run attempt in names. | +| Local parity | Public `make check-test-shards` is Linux/amd64-Docker canonical. `make verify` checks the manifest and runs `non-ui`, then `ui-1..ui-3`, sequentially in one such container with the same flags and package selections as CI. | +| Rebalancing | No timing-based CI guard. Rebalance within three shards only after three post-change attempts if slowest/fastest shard medians differ by more than 20%, or the gate misses the target because of shard imbalance. Any rebalance creates a new SHA and restarts the three-attempt comparison. | +| Runner cost | Report it. More than 2x the baseline runner-minute median triggers review, never automatic test weakening or rejection by itself. | +| Over-target result | If three safe, reasonably balanced shards still have a median above 12 minutes, stop before final handoff and present the measured critical path. Do not silently add capacity or weaken checks. | +| Release gate | Leave `release.yml` unchanged. A failed job in the called reusable workflow already fails its caller and blocks every release job that needs `test`. | + +## Honest limit + +After sharding, no required job runs all exact `internal/ui` top-level tests in +one process. This can change behavior if two tests accidentally depend on +process order or on state left by a test assigned to another shard. That loss +of whole-package process-order coverage is accepted because the package has no +current `t.Parallel()` calls, its harness drains and resets owned background +state, and the requested isolation boundary is separate hosted runners. + +The manifest guard prevents coverage gaps and overlap; it cannot prove that +tests are independent. The new `t.Parallel()` guard forces an explicit safety +review before this concurrency model can change. A future ordering-dependent +failure must be fixed as a test-isolation defect or motivate a separately +approved monolithic check; it must not be hidden by retries or optional jobs. + +## Planned file changes + +| File | Purpose | +|---|---| +| `.github/workflows/ci.yml` | First add monolithic measurement capture; later replace the serialized job with the validation job and four-entry race matrix while preserving Windows and reusable-workflow gating. | +| `.github/testshards/internal-ui.tsv` | Checked-in exact `test-nameui-N` assignment plus provenance comments for the baseline SHA/attempts. | +| `scripts/testshards/main.go` | Standard-library CLI for JSON capture/summaries, deterministic planning, inventory validation, parallel-call guard, and anchored regex output. | +| `scripts/testshards/main_test.go` | Parser, balancing, manifest, regex, inventory, capture, and negative guard tests. | +| `Makefile` | Shared direct race commands and one-container sequential local orchestration; `make verify` parity. | +| `qodana.yaml` | Exact exclusion for `scripts/testshards/main_test.go`. | +| `ARCHITECTURE.md` | Add the new repository tooling package and its responsibility. | +| `todos.md` | Move this item to Done only after measured acceptance. | +| This plan | Record both checkpoints, outcome, exceptions, and the final cost ledger; move it to `finished_refactorings/` only when complete. | + +`.github/workflows/release.yml` is intentionally absent from the change set. + +## Helper contract + +The CLI stays small, deterministic, and standard-library-only. Exact flag +spelling may be adjusted during the red-green cycle, but these behaviors are +the contract: + +```text +testshards summarize -json RUN1 -json RUN2 -json RUN3 +testshards plan -package ./internal/ui -shards 3 \ + -json RUN1 -json RUN2 -json RUN3 +testshards check -package ./internal/ui \ + -manifest .github/testshards/internal-ui.tsv +testshards regex -manifest .github/testshards/internal-ui.tsv \ + -shard ui-1 +testshards capture -out RAW.json +``` + +- `summarize` reports package durations and terminal top-level test results for + one file, and medians for multiple files. It rejects malformed/truncated JSON + and reports failed or missing terminal events rather than inventing weights. +- `plan` requires every current inventory item to have a terminal result in + every supplied baseline. It outputs all names exactly once, ordered by shard + and then name, using the locked median/LPT/tie-break rules. +- `check` uses `go list -json` and `go test -list` in the current build context. + The checked-in manifest is accepted only through the Linux/amd64 container + target or on a Linux CI runner, never from a Darwin inventory. It compares + exact sets, requires all three shards to be non-empty, and scans selected + exact-package test files for `.Parallel()` calls. +- `regex` parses and structurally validates the entire manifest before emitting + one Go-compatible, `regexp.QuoteMeta`-escaped expression of the form + `^(TestA|TestB)$`. It never emits a match-all or empty expression. +- `capture` reads newline-delimited `go test -json` from stdin, preserves it in + `RAW.json`, and prints compact package/test/failure lines. The shell pipeline + uses `pipefail`, so either a failed `go test` or failed capture step fails the + job. Raw artifacts remain available even on failure. + +`TestMain` is package harness code, not an independently runnable test, and is +therefore not a manifest entry. A top-level parent owns all names below its +slash-separated subtest path. + +## Acceptance criteria + +| ID | Criterion | Verification | +|---|---|---| +| AC1 | Three instrumented monolithic attempts exist at one SHA, each with readable raw JSON and timing metadata. | For each N in 1-3, run `gh run view --attempt N --json attempt,headSha,status,conclusion,createdAt,startedAt,updatedAt,jobs`; download and hash that attempt's artifacts before starting N+1. | +| AC2 | The checked-in manifest is deterministic and covers the exact Linux UI inventory once, with no stale entries and no empty shard. | `make check-test-shards` twice, plus `git diff --exit-code -- .github/testshards/internal-ui.tsv` after regenerating from the same inputs. | +| AC3 | Deliberately missing, duplicate, stale, malformed, unknown-shard, empty-shard, and `.Parallel()` fixtures are rejected with actionable names. | `go test ./scripts/testshards -run 'TestCheckRejects|TestParallelGuardRejects' -count=1 -v` | +| AC4 | Each generated expression is anchored, non-empty, and matches exactly its own manifest names. | `go test ./scripts/testshards -run 'TestRegex' -count=1 -v` | +| AC5 | The non-UI set is exactly all module packages except `github.com/frathe/picfetch/internal/ui`; UI subpackages, root, and scripts remain included. | `go test ./scripts/testshards -run 'TestPackagePartition' -count=1 -v`, then inspect the `non-ui` CI summary against `go list ./...`. | +| AC6 | Every Linux matrix entry uses the explicit locale, race detector, fresh execution, and 30-minute package timeout; all four pass. | Inspect the command lines in `gh run view --log`; each must contain `LANG=en_US.UTF-8`, `-race`, `-count=1`, and `-timeout 30m`. | +| AC7 | A test or capture failure keeps the exact shard/test visible and fails the job while still uploading raw JSON. | `go test ./scripts/testshards -run 'TestCapture' -count=1 -v`, plus one temporary negative workflow/helper verification before restoring the tree. | +| AC8 | Formatting/TUF/vet/build/Windows cross-build are independent from Linux race tests, and Windows tests remain independent and unchanged. | `gh run view --json jobs`; confirm six job executions overlap where runners permit and compare the Windows command with the base revision. | +| AC9 | A failure in any validation, matrix, or Windows job fails the reusable CI caller; no shard is optional. | Inspect the completed workflow graph and `.github/workflows/ci.yml`; there must be no `continue-on-error`, and matrix `fail-fast` must be `false`. | +| AC10 | Release gating is unchanged. | `git diff --exit-code -- .github/workflows/release.yml` | +| AC11 | Local verification runs the same package partition and shard expressions sequentially in one Linux/amd64 container. | `make verify` | +| AC12 | Three final attempts share one SHA and include gate, queue-excluded, validation/setup, shard, package/test, and runner-minute medians. No coverage or failure is lost. | For each N in 1-3, run `gh run view --attempt N --json attempt,headSha,status,conclusion,createdAt,startedAt,updatedAt,jobs`; download and hash that attempt's artifacts before starting N+1. | +| AC13 | Median gate time is preferably 8-12 minutes; any accepted exception identifies the measured critical path. A >20% shard imbalance is addressed before acceptance. | Run `testshards summarize` over all three final-attempt artifacts and calculate the workflow/job medians from the Actions API record. | +| AC14 | The concurrency trade-off is explicit. | Record pre/post median runner minutes and peak six-job topology in the Outcome and `todos.md` Done entry. | + +## Task graph + +```text +T1 measurement-only workflow + -> C1 user commit/push + three baseline attempts + -> T2 baseline analysis + -> T3 helper and manifest (test-first) + -> T4 shared Makefile commands + -> T5 final parallel workflow + -> T6 local and negative safety verification + -> C2 user commit/push + three sharded attempts + -> T7 measured comparison/rebalance decision + -> T8 documentation and final handoff +``` + +C1 and C2 are real external checkpoints. Work must not skip them or substitute +different SHAs for convenience. + +## Tasks + +### T1 — Add the measurement-only checkpoint + +- **Owner:** Inline implementation agent. +- **Files:** `.github/workflows/ci.yml` only. +- **Depends:** None. +- **Contract:** Keep the current job topology and commands, pin its one Linux + job to `ubuntu-24.04`, add `-count=1 -json` to the monolithic Linux race + command, preserve its exit status through a `pipefail` capture, upload the raw + stream with `if: always()` and 14-day retention, and expose enough step timing + to separate setup/validation from the race command. Include + `github.run_attempt` in the artifact name. Do not introduce shards in this + diff. +- **Test:** Review the workflow diff for unchanged validation, package scope, + test flags other than the additive `-count=1 -json`, Windows job, + concurrency, and `workflow_call` trigger. The explicit Linux runner pin is + the only other execution change. +- **Verify:** `git diff --check`; `make fmt-check`; GitHub validates and runs + the workflow at C1. +- **Budget:** One focused edit/review pass; no full local race rerun and no + delegated work. + +### C1 — Capture three fresh monolithic CI attempts + +- **Owner:** User for commit/push; GitHub Actions for execution; implementation + agent for read-only collection. +- **Files:** No further source edit until all attempts are available. +- **Depends:** T1. +- **Contract:** The user commits and pushes the measurement-only tree. Let the + first PR workflow finish, rerun that same run twice, and retain attempts 1-3. + [GitHub reruns retain the original SHA and ref](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs). + Immediately after each attempt, use `gh run view --attempt N` and download + its uniquely named artifact to an off-tree directory before requesting the + next rerun; do not depend on an old attempt's artifacts remaining + downloadable after a rerun. Record UTC just before each + `gh run rerun ` request. For attempt 1, use the Actions `createdAt`; + for attempts 2 and 3, use that recorded request time as the queue-inclusive + origin because the run's original `createdAt` is unchanged. + Record run ID, attempt, head SHA, runner image, Go version/cache outcome, + creation/start/completion timestamps, every step duration, total runner + minutes, package durations, top-level UI durations, failures, and artifact + names. All attempts must use the same SHA and `-count=1`; do not substitute + older `main` runs. +- **Test:** Each JSON stream parses to a terminal package result and complete UI + inventory. Any infrastructure-failed attempt is reported and repeated rather + than included silently. +- **Verify:** For N in 1-3, `gh run view --attempt N --json + attempt,headSha,status,conclusion,createdAt,startedAt,updatedAt,jobs`, followed + immediately by `gh run download -D ` and a + SHA-256 inventory. +- **Budget:** Exactly three valid attempts; reruns, not empty commits. + +### T2 — Freeze the measured baseline and proposed allocation + +- **Owner:** Inline implementation agent. +- **Files:** This plan's evidence/outcome sections; raw data stays outside the + repository. +- **Depends:** C1. +- **Contract:** Calculate per-attempt and median primary/secondary metrics. + Explain queue time separately. Produce the three-shard LPT proposal from + median top-level CI durations and verify it contains the Linux inventory + exactly once. Record any test whose outcomes/durations were not comparable. +- **Test:** Independently total manifest membership and compare package sets; + do not treat summed per-test elapsed values as workflow wall time. +- **Verify:** Re-run the summary/allocation command over the same inputs and + require byte-identical normalized output. +- **Budget:** One analysis pass and one arithmetic review; no workflow changes. + +### T3 — Build the helper and manifest test-first + +- **Owner:** Inline implementation agent. The parser, inventory, and workflow + contract are tightly coupled, so this is not a safe delegation seam. +- **Files:** `scripts/testshards/main.go`, + `scripts/testshards/main_test.go`, `.github/testshards/internal-ui.tsv`, + `qodana.yaml`, `ARCHITECTURE.md`. +- **Depends:** T2. +- **Contract:** Implement the helper contract above with pure internal + functions around a thin CLI. Write failing tests first for JSON truncation, + missing terminal results, deterministic median/LPT ties, every manifest + rejection mode, exact package partition, Go-compatible quoting/anchoring, + empty regex refusal, parallel-call detection, concise failures, and raw JSON + preservation. Generate the initial manifest only from the three accepted C1 + files. Add the exact new `_test.go` path to Qodana exclusions and document + the tooling package in the architecture map. +- **Execution note:** Run initial planning/validation in an explicit + Linux/amd64 container even though the reusable Make target is added in T4; + never generate the checked-in Linux manifest from the host's Darwin test + inventory. +- **Test:** `go test ./scripts/testshards -count=1`; each negative fixture must + first demonstrate the intended red result before implementation makes the + suite green. +- **Verify:** `go test ./scripts/testshards -count=1 -v`; + `make check-qodana-test-exclusions`; deterministic regenerate-and-diff over + the accepted C1 Linux inputs; `git diff --check`. The canonical live Linux + inventory check begins in T4 after its Docker target exists. +- **Budget:** One red-green-refactor cycle plus one review pass; no full suite. + +### T4 — Give Makefile and CI one race-test contract + +- **Owner:** Inline implementation agent. +- **Files:** `Makefile`. +- **Depends:** T3. +- **Contract:** Keep `make test` as the complete non-race Linux/amd64 Docker + suite. Make `make test-race` and `make verify` validate the manifest and run + the four final commands sequentially inside one Docker container so package + compilation/cache and apt setup are reused. Provide direct in-container + targets used by Actions for the guard, `non-ui`, and one selected UI shard. + Public `make check-test-shards` must enter Linux/amd64 Docker; the direct + target is explicitly internal/CI-only so a macOS inventory cannot be mistaken + for the canonical one. Derive the non-UI list from `go list ./...` minus only + `go list ./internal/ui`; reject an empty set. Centralize + `-race -count=1 -timeout 30m` and the explicit locale so local and CI + invocations cannot drift. +- **Test:** Exercise helper unit tests and dry-inspect all four expanded command + lines before the expensive run. Confirm the union is all packages and the + intersection between exact UI and non-UI is empty. +- **Verify:** `make check-test-shards`; focused direct command checks inside the + Linux container; the single final `make verify` is reserved for T6. +- **Budget:** One Makefile refactor pass; at most one focused Docker setup. + +### T5 — Split the reusable CI workflow + +- **Owner:** Inline implementation agent. +- **Files:** `.github/workflows/ci.yml`. +- **Depends:** T4. +- **Contract:** Replace `build-and-test` with: + + 1. `validation`, pinned to `ubuntu-24.04`, containing checkout/setup, + formatting, TUF-root validation, Linux GUI dependencies, manifest guard, + vet, normal build, both Windows cross-builds, and Windows vet; + 2. `linux-race`, pinned to `ubuntu-24.04`, with a four-value matrix and + `fail-fast: false`; every entry performs its own checkout/setup/dependency + installation and invokes the matching shared Make target; and + 3. the existing `windows-test`, still independent and command-identical. + + No job has `needs` on another. Retain `workflow_call`, read-only permissions, + the current cancellation group/behavior, and required failure semantics. + Each race entry captures concise output plus a raw JSON artifact on success + or failure. Artifact and visible job names include `non-ui` or the exact UI + shard, plus the workflow attempt where appropriate. Do not edit + `release.yml`. +- **Test:** Review all old steps against the new job map one by one. Search for + forbidden `continue-on-error`, unintended `needs`, moving Linux labels, or a + matrix-level fail-fast default. +- **Verify:** `git diff --check`; `git diff --exit-code -- + .github/workflows/release.yml`; actual GitHub validation at C2. +- **Budget:** One workflow edit and one independent contract review; no local + duplicate full-suite run. + +### T6 — Verify safety locally and with deliberate negative inputs + +- **Owner:** Inline implementation agent. +- **Files:** Only fixes revealed by verification. +- **Depends:** T5. +- **Contract:** Prove each guard fails for a temporary missing entry, duplicate + entry, stale entry, malformed row, unknown shard, empty shard, and UI + `.Parallel()` call, then restore the valid inputs. Prove capture reports an + exact failed test while retaining its raw event stream and nonzero pipeline + status. Run the complete sequential Linux race suite once through the final + Makefile path. Do not regenerate golden files. +- **Test:** Focused helper negative tests, followed by the full repository gate. +- **Verify:** `go test ./scripts/testshards -count=1 -v`; + `make check-test-shards`; `make verify`; `git diff --check`; + `git status --short` with no failed golden renders or timing artifacts. +- **Budget:** One full local `make verify`; one fix-and-rerun is allowed only if + the first run finds a real defect. + +### C2 — Capture three fresh sharded CI attempts + +- **Owner:** User for commit/push; GitHub Actions for execution; implementation + agent for read-only collection. +- **Files:** No source edit until the three attempts are classified. +- **Depends:** T6. +- **Contract:** The user commits and pushes the sharded tree. Run the initial + attempt and rerun the same workflow twice. Collect the same fields as C1, + plus each shard's test count, median/maximum test duration, compile/setup + overhead, and raw artifact. Confirm all six job executions are mandatory and + that all four Linux package/test sets passed. As at C1, download and hash each + attempt before triggering the next rerun, and record the rerun request time + used by the queue-inclusive metric. +- **Test:** All attempts share one SHA and produce a complete exact-set union. + Infrastructure failures are labeled and replaced, not averaged into product + performance. +- **Verify:** For N in 1-3, `gh run view --attempt N --json + attempt,headSha,status,conclusion,createdAt,startedAt,updatedAt,jobs`, followed + immediately by `gh run download -D ` and a + SHA-256 inventory. +- **Budget:** Exactly three valid attempts at the accepted sharded SHA. + +### T7 — Compare, rebalance only if justified, and apply the stop rule + +- **Owner:** Inline implementation agent. +- **Files:** Manifest and this plan only if measurement justifies a rebalance; + workflow/Makefile only for a demonstrated defect. +- **Depends:** C2. +- **Contract:** Compare medians for gate wall time, queue-excluded window, + critical job, validation/setup, race execution, UI shards, non-UI, and total + runner minutes. Confirm zero missing/duplicate package or test names. + + - If slowest/fastest UI shard median is at most 1.20, do not churn the + manifest merely to make totals prettier. + - If it exceeds 1.20 and that imbalance affects the critical path, recompute + within the same three shards from C2 data, rerun T6, create a new user + checkpoint, and collect three new attempts before comparison. + - Allow at most one measured rebalancing round in this pull request. + - If safe, balanced CI still has a median above 12 minutes, stop and present + the critical path to the user. Do not add a fourth shard or weaken tests. + - If total runner minutes exceed 2x baseline, flag and explain the source; + this is a review signal, not a reason to remove coverage. + +- **Test:** Recompute summaries from raw data; reconcile Actions job timing + with JSON package/test timing without adding overlapping durations. +- **Verify:** Deterministic helper summary plus a checked arithmetic table in + this plan. +- **Budget:** One comparison pass; at most one rebalance-and-three-run loop. + +### T8 — Close documentation and hand off + +- **Owner:** Inline implementation agent. +- **Files:** This plan, `todos.md`; move the plan to `finished_refactorings/` + only after every accepted criterion passes. +- **Depends:** T7 and any required user decision for an over-target result. +- **Contract:** Record run IDs, SHAs, attempt table, before/after medians, + critical path, runner minutes, shard membership counts, safety verification, + and any accepted target exception. Move the TODO to Done with the explicit + trade-off that lower elapsed time uses more concurrent capacity and may use + more total runner minutes. Report the honest whole-process limit. Leave no + raw timing artifacts in the repository. +- **Test:** Read the finished record against AC1-AC14 and inspect the final + source diff for unrelated changes. +- **Verify:** `make verify`; `git diff --check`; `git status --short`; final + successful GitHub run links. +- **Budget:** Documentation-only pass after CI acceptance; do not repeat the + full suite unless code changed after T6. + +## Delegation and verification budget + +Reconnaissance used three bounded, read-only scouts for current workflow, +measurement, and UI-test inventory facts. Implementation stays inline because +the helper, manifest, Makefile commands, and workflow form one safety contract +and touch shared files. No implementation subagents are budgeted. + +- Maximum implementation review loops: two (contract review, then final diff + review). +- Expensive local full suites: one at T6, plus at most one defect-driven rerun. +- CI executions: three valid C1 attempts and three valid C2 attempts, plus at + most three after one justified rebalancing checkpoint. +- Golden regeneration: zero. +- Workflow/release mutations outside the listed files: zero. + +## Outcome + +C1 and T2 are complete. The measurement-only workflow was committed as +`fcd11103969ac8d1b5a1a220270c0730cb5e1913` and exercised in three successful +attempts of [Actions run 33734286005](https://github.com/frathe/picfetch/actions/runs/33734286005) +for [draft PR 14](https://github.com/frathe/picfetch/pull/14). All attempts used +the same pull-request ref, Ubuntu 24.04, Go 1.27.1, a primary setup-cache hit, +and the unchanged package/test commands. There were no infrastructure or test +failures to discard. + +### Controlled monolithic baseline + +The primary gate origin is the workflow creation time for attempt 1 and the +recorded UTC rerun-request time for attempts 2 and 3. Completion is the later +of the two required workflow jobs. `Qodana for Go` is a separately injected, +zero-second check rather than a job declared by `ci.yml`; it is excluded from +the workflow execution and runner-minute totals. + +| Attempt | Origin (UTC) | Queue | Gate | Queue-excluded | Linux job | Windows job | Runner minutes | +|---:|---|---:|---:|---:|---:|---:|---:| +| 1 | 08:36:41 (created) | 0:02 | 23:12 | 23:10 | 23:10 | 1:13 | 24:23 | +| 2 | 09:01:13 (rerun request) | 0:07 | 22:17 | 22:10 | 22:10 | 1:16 | 23:26 | +| 3 | 09:24:20 (rerun request) | 0:05 | 22:07 | 22:02 | 22:02 | 1:20 | 23:22 | +| Median | - | **0:05** | **22:17** | **22:10** | **22:10** | **1:16** | **23:26** | + +The required Linux job's step timings were: + +| Step | Attempt 1 | Attempt 2 | Attempt 3 | Median | +|---|---:|---:|---:|---:| +| Set up job | 0:01 | 0:01 | 0:01 | 0:01 | +| Checkout | 0:01 | 0:03 | 0:03 | 0:03 | +| Set up Go | 0:12 | 0:17 | 0:16 | 0:16 | +| Formatting | 0:05 | 0:04 | 0:06 | 0:05 | +| TUF-root validation | 0:01 | 0:01 | 0:01 | 0:01 | +| Linux GUI dependencies | 0:13 | 0:24 | 0:21 | 0:21 | +| Vet | 0:10 | 0:10 | 0:11 | 0:10 | +| Normal build | 0:04 | 0:04 | 0:04 | 0:04 | +| Windows cross-builds | 1:10 | 1:15 | 1:13 | 1:13 | +| Windows vet | 0:09 | 0:10 | 0:10 | 0:10 | +| Linux race test | 21:01 | 19:35 | 19:26 | **19:35** | +| Raw artifact upload | 0:01 | 0:02 | 0:06 | 0:02 | + +From Linux-job start to the race step was 2:07, 2:30, and 2:27, for a +**2:27 median**. This is the serial validation/build work that the final +topology can overlap with testing. + +The machine-readable race streams contained the same 44 package terminal +events and the same 567 exact-`internal/ui` top-level tests in every attempt. +All 567 tests passed in every attempt with no top-level skips. The only package +skip was the consistent no-test-files `internal/ui/assets` package. + +| Attempt | Compile/start gap | JSON event span | Non-UI tail from first event | Exact `internal/ui` | Sum of UI top-level elapsed values | +|---:|---:|---:|---:|---:|---:| +| 1 | 3:15.022 | 17:45.933 | 1:00.635 | 17:41.764 | 17:40.420 | +| 2 | 3:17.373 | 16:18.196 | 1:02.854 | 16:14.021 | 16:12.450 | +| 3 | 3:10.968 | 16:15.323 | 1:01.564 | 16:11.149 | 16:09.690 | +| Median | **3:15.022** | **16:18.196** | **1:01.564** | **16:14.021** | **16:12.450** | + +The non-UI tail is measured from the first test event, after the shared compile +gap. Measured from race-step start, the last non-UI package completed at a +4:15.657 median. Package elapsed values overlap and are not summed into a +wall-clock figure. The slowest package medians were: + +| Package | Attempt 1 | Attempt 2 | Attempt 3 | Median | +|---|---:|---:|---:|---:| +| `internal/ui` | 1061.764s | 974.021s | 971.149s | **974.021s** | +| `internal/ui/compare` | 55.462s | 57.668s | 56.376s | 56.376s | +| `internal/imaging` | 36.054s | 37.578s | 36.704s | 36.704s | +| `internal/ui/help` | 14.419s | 14.876s | 15.397s | 14.876s | +| `internal/ui/favorites` | 9.447s | 10.644s | 9.867s | 9.867s | +| `internal/ui/exifwin` | 6.780s | 7.058s | 6.616s | 6.780s | +| `internal/ui/copyselection` | 4.382s | 5.201s | 4.977s | 4.977s | +| `internal/ui/settingswin` | 4.208s | 4.226s | 4.057s | 4.208s | +| `internal/update` | 3.815s | 3.856s | 3.876s | 3.856s | +| `internal/ui/grid` | 3.483s | 3.674s | 3.622s | 3.622s | + +Every other package had a median at or below 2.149s. The complete 44-package +table and all 567 per-attempt top-level test durations are retained in the +off-tree normalized record described below. The slowest UI-test medians were: + +| Test | Median | +|---|---:| +| `TestCompareCommandEntryPoints_MenuAndFeatureCallbacksAreIgnored` | 40.31s | +| `TestCopySelectionCancelsBeforeOtherCommands` | 35.99s | +| `TestCopySelectionAvailability` | 18.83s | +| `TestCompareOpenRefusal_DropDialogShortcutAndOpenWithAreDiscarded` | 13.19s | +| `TestCompareHelp_F1OpensManualWithoutLeavingComparison` | 10.05s | +| `TestCompareLinkToggle_RequiresExactPhysicalControlL` | 9.33s | +| `TestYieldingMenuCallbacksWrapsEveryField` | 8.82s | +| `TestBuildMainMenu_ManualOpenedObserverSyncsWindowHelp` | 8.43s | +| `TestAdvance_ShuffleOnNeverRepeatsCurrentIndex` | 7.46s | +| `TestCopySelectionKeyboard` | 6.02s | + +### Baseline artifacts and reproducibility + +Raw streams were downloaded and hashed outside the repository before each +subsequent rerun: + +| Attempt | Artifact | Lines | Bytes | SHA-256 | +|---:|---|---:|---:|---| +| 1 | `linux-race-baseline-33734286005-attempt-1` | 11,402 | 2,481,918 | `c31eea242ae6a6a63cd8a64e93a04416939a55a6b63083b1fdc798503803b07e` | +| 2 | `linux-race-baseline-33734286005-attempt-2` | 11,402 | 2,481,964 | `852ba59fe2cef71ae76d7cd17d144ff42c83e709d36f8f123038297e436beb61` | +| 3 | `linux-race-baseline-33734286005-attempt-3` | 11,402 | 2,482,006 | `5e1da150e8524848b5b682e16f57199875e095ecb234d59c3492ca7665f8671d` | + +The off-tree directory is +`/private/tmp/picfetch-ci-baseline.XeJlI7`. It contains the raw streams, +required-job logs, Actions metadata, analysis scripts, and two independently +generated copies of each normalized report. The 1,191-line test/package/LPT +report reproduced byte-for-byte at SHA-256 +`bb61c7b69d41e98b22ef15575a7614b13363b14d029a4cd9b96be6d236454803`; +the Actions timing report reproduced at +`39f50e66026a211121e7dc3fc15dd3b4b1e69a51542dc50434a9db27b4723522`. +The 18-file checksum inventory itself hashes to +`6aefc52fcdfcacff0ae5c8fb96ea767d4d9086fb209639f0be869c85af4d4b3d`. +No raw timing or log file is in the repository. + +The raw artifacts were protected at each rerun boundary as planned. Full +required-job log files were materialized and hashed after attempt 3 rather +than before attempts 2 and 3 were requested. GitHub's attempt-addressable logs +remained intact, and their timing metadata had already been inspected, so no +evidence was lost. This is a documented collection-order deviation from +ticket 01, not a substituted or incomparable sample. + +### Proposed three-shard allocation + +Deterministic longest-processing-time allocation using each test's median over +the three CI streams produced this exact-set proposal: + +| Shard | Tests | Sum of median weights | +|---|---:|---:| +| `ui-1` | 177 | 326.75s | +| `ui-2` | 178 | 327.07s | +| `ui-3` | 212 | 326.74s | + +The 567-name union is exact and contains no duplicate assignment or empty +shard. Repeated Linux/amd64 generation produced the checked-in manifest +byte-for-byte at SHA-256 +`6468a50ce0925ade39d253be7e7b0e728aa3d38fdc9f32a6b5547347155c5a18`. +These are theoretical test-only weights, not predicted job wall times; +each real runner will repeat checkout, toolchain/dependency setup, package +compilation, and package harness work. The measured critical path nevertheless +supports the planned boundary: exact `internal/ui` dominates the race command, +the non-UI set finishes early, and the serial validation/build phase can run +independently. Ticket 03 now supplies the live Linux inventory guard and exact +filter generation around this assignment. + +### Live assignment safety + +Ticket 03 is complete. The helper now checks the entire manifest against test +files and top-level `Test`, `Fuzz`, and `Example` names selected by the current +Go build, ignores `TestMain` and subtest paths, rejects `.Parallel()` calls in +selected exact-package test files, and emits escaped exact-match filters only +after full structural validation. A public `make check-test-shards` runs that +acceptance boundary in Linux/amd64 Docker; the direct target refuses Darwin or +a mismatched cross-build context as canonical. + +Two independent Docker checks each reported 567 runnable names across three +non-empty shards without modifying `.github/testshards/internal-ui.tsv`; its +SHA-256 remains +`6468a50ce0925ade39d253be7e7b0e728aa3d38fdc9f32a6b5547347155c5a18`. +Focused command tests cover every required rejection and all three generated +filters. + +### Shared local race contract + +Ticket 04 is complete. `make test` remains the complete unsharded non-race +Linux/amd64 suite. `make test-race` and `make verify` enter one Linux/amd64 +container, run the canonical manifest guard, then execute the exact non-UI +package complement and `ui-1` through `ui-3` sequentially. The prepared-runner +targets expose the same guard, non-UI, and selected-UI commands for the final CI +matrix without nesting Docker. Each race command visibly uses +`LANG=en_US.UTF-8`, `-race`, `-count=1`, `-timeout 30m`, JSON capture, and Bash +`pipefail`. + +The helper's `partition` path resolved the repository to 45 packages: 44 +non-UI packages plus exact `github.com/frathe/picfetch/internal/ui`. The +complement retains the root, every tooling package, and every UI feature +subpackage, and fails closed if the selected package is absent or leaves an +empty complement. Its `capture` path preserved an existing 2,481,918-byte CI +stream byte-for-byte while reducing live output to partitioned package/test +terminals and failure details. + +A deliberate failing test stream returned pipeline status 1 and named the +partition, package, and exact test; a deliberate capture-open failure returned +capture status 1. The final post-review `make verify` run checked all 567 UI +runnables, passed the non-UI complement, then passed `ui-1` in 229.604s, +`ui-2` in 228.490s, and `ui-3` in 211.661s. No golden was regenerated and no +raw event stream was left in the repository. The initial full run also passed; +the plan's one allowed defect-driven rerun covered a review fix that made the +direct command lines visible for later CI log inspection. + +### Parallel reusable CI topology + +Ticket 05 is complete. The reusable workflow now exposes one Ubuntu 24.04 +validation job, one fail-fast-disabled Ubuntu 24.04 race matrix with exact +`non-ui`, `ui-1`, `ui-2`, and `ui-3` entries, and the existing independent +native Windows job. None has a `needs` edge. Validation retains formatting, +TUF-root checking, Linux GUI dependencies, live manifest validation, vet, +normal build, both Windows cross-builds, and Windows vet. Each race entry owns +checkout, Go setup, Linux GUI dependencies, the matching direct Make contract, +an attempt-qualified summary, and an always-run raw artifact upload that fails +closed when the stream is absent and retains it for 14 days. + +A parsed workflow contract confirmed the exact topology, names, matrix, +failure policy, triggers, permissions, cancellation group, artifact settings, +and unchanged Windows command. A separate parsed comparison against the +pre-ticket `dd43c77` revision showed that every pre-existing validation/build +step and the native Windows job body survived unchanged; `release.yml` has no +source diff from `main`. +`make fmt-check`, `go vet ./...`, `go test ./scripts/testshards -count=1`, all +four direct-target dry runs, and `git diff --check` pass. The full local race +evidence remains ticket 04's final `make verify`, since this ticket changes +only Actions YAML. The C2 evidence below now confirms the hosted six-slot +execution, required results, and aggregate workflow success. + +### Sharded checkpoint validity + +The user committed and pushed the final topology as +`b91d56ce67b991b595484a50d9b65c52f1a09dba`. Three successful attempts of +[Actions run 33758913251](https://github.com/frathe/picfetch/actions/runs/33758913251) +on `feature/CI-test-performance-improvements` used that exact SHA and ref. The +initial workflow completed before attempt 2 was requested, and attempt 2's raw +artifacts, required-job logs, metadata, and checksums were secured before +attempt 3 was requested. No test or infrastructure failure required a +replacement attempt. + +The queue-inclusive origin is the workflow creation time for attempt 1 and the +UTC timestamp recorded immediately before each rerun request for attempts 2 +and 3. Completion is the last required job completion. The separately injected +neutral `Qodana for Go` check declares no workflow job or runner time and is +excluded exactly as it was from the baseline. + +| Attempt | Origin (UTC) | First required start | Required completion | Queue | Gate | Queue-excluded | Critical entry | Runner minutes | Result | +|---:|---|---|---|---:|---:|---:|---|---:|---| +| 1 | 13:04:11 (created) | 13:04:13 | 13:10:26 | 0:02 | 6:15 | 6:13 | `ui-3` | 24:50 | Pass | +| 2 | 13:13:19 (rerun request) | 13:13:35 | 13:21:11 | 0:16 | 7:52 | 7:36 | `ui-1` | 31:01 | Pass | +| 3 | 13:22:53 (rerun request) | 13:23:16 | 13:30:52 | 0:23 | 7:59 | 7:36 | `ui-3` | 30:43 | Pass | +| Median | - | - | - | **0:16** | **7:52** | **7:36** | UI race | **30:43** | **3/3 pass** | + +All six required job intervals overlapped in every attempt: validation, +Windows, `non-ui`, and all three UI shards. This observed peak of six proves +that the workflow has no scheduling dependency between them when hosted-runner +capacity is available. + +### Required jobs, validation, and setup + +| Required job | Attempt 1 | Attempt 2 | Attempt 3 | Median | +|---|---:|---:|---:|---:| +| Validation | 2:32 | 2:42 | 2:24 | **2:32** | +| Windows tests | 1:12 | 1:36 | 1:14 | **1:14** | +| Linux race `non-ui` | 4:22 | 5:03 | 5:34 | **5:03** | +| Linux race `ui-1` | 4:29 | 7:30 | 7:07 | **7:07** | +| Linux race `ui-2` | 6:03 | 7:02 | 6:48 | **6:48** | +| Linux race `ui-3` | 6:12 | 7:08 | 7:36 | **7:08** | + +The independent validation job retained every pre-sharding check. Its +step-level medians were: + +| Validation step | Median | +|---|---:| +| Job setup | 0:01 | +| Checkout | 0:02 | +| Go setup | 0:13 | +| Formatting | 0:07 | +| TUF-root validation | 0:01 | +| Linux GUI dependencies | 0:19 | +| UI manifest validation | 0:11 | +| Vet | 0:10 | +| Normal build | 0:03 | +| Windows cross-builds | 1:13 | +| Windows vet | 0:10 | + +The Windows test step itself had a 0:06 median. All three attempts ran the +unchanged `go test ./internal/update/... ./internal/ui/autoupdate/...` command +successfully. + +### Linux race partitions, coverage, and balance + +The runner-setup column spans job start through the race-contract step. The +compile/helper gap is inside that contract, from its start to the first Go JSON +event. `non-ui` has many concurrently executed packages, so its final column is +the first-to-last JSON span; each UI value is the exact package terminal. + +| Entry | Inventory per attempt | Job median | Runner setup | Race contract | Compile/helper gap | JSON/package execution | Result | +|---|---|---:|---:|---:|---:|---:|---| +| `non-ui` | 44 packages; 1,486 top-level tests | 5:03 | 1:21 | 3:33 | 2:50.331 | 0:42.891 span | Pass | +| `ui-1` | 177 tests | 7:07 | 0:38 | 6:24 | 0:58.913 | 5:21.919 | Pass | +| `ui-2` | 178 tests | 6:48 | 0:36 | 5:57 | 0:54.565 | 5:02.435 | Pass | +| `ui-3` | 212 tests | 7:08 | 0:48 | 6:15 | 0:57.727 | 5:10.880 | Pass | + +The UI top-level elapsed-sum medians were 5:20.640, 5:01.300, and 5:09.550. +The slowest/fastest exact-package median ratio is +`321.919 / 302.435 = 1.064`; the full-job ratio is +`428 / 408 = 1.049`. Both are safely below the 1.20 review threshold, so the +manifest receives no cosmetic rebalance. The apparently fast `ui-1` first +attempt is ordinary runner variance and does not override the three-run +median. + +Each attempt's `non-ui` package terminals equal the current 44-package module +complement after subtracting only exact `internal/ui`. The three UI result sets +equal the manifest's 567-name union, with exactly 177, 178, and 212 names and +no missing, stale, or repeated test. All 567 UI tests passed in every attempt. +The non-UI stream consistently had 1,485 passing top-level tests and one +expected zero-second skip, +`TestCheckRejectsDarwinDerivedInventoryAsCanonical`; the only skipped package +was the no-test-files `internal/ui/assets`. Thus the final union is all 45 +current module packages plus every exact UI top-level runnable. + +All 12 Linux job logs show the literal `LANG=en_US.UTF-8`, `-race`, +`-count=1`, and `-timeout 30m` command. Each job uploaded its required raw +stream after the test step. The workflow contains no `needs` or +`continue-on-error`, keeps matrix `fail-fast: false`, and the overall run was +successful only after all six required results succeeded. Earlier deliberate +helper and pipeline failures returned nonzero while retaining and naming the +raw failing test. `release.yml` remains unchanged and still makes every +release job depend on the reusable CI caller. + +### Final package and test durations + +The slowest non-UI package terminal medians were: + +| Package | Attempt 1 | Attempt 2 | Attempt 3 | Median | +|---|---:|---:|---:|---:| +| `internal/ui/compare` | 37.588s | 37.812s | 48.611s | **37.812s** | +| `internal/imaging` | 27.352s | 27.840s | 34.887s | 27.840s | +| `internal/ui/help` | 11.137s | 11.357s | 15.939s | 11.357s | +| `internal/ui/favorites` | 8.891s | 10.199s | 11.469s | 10.199s | +| `internal/ui/exifwin` | 5.907s | 5.249s | 7.335s | 5.907s | +| `internal/ui/copyselection` | 3.383s | 4.125s | 5.245s | 4.125s | +| `internal/ui/grid` | 3.170s | 3.985s | 3.928s | 3.928s | +| `internal/update` | 3.827s | 3.696s | 3.733s | 3.733s | +| `internal/ui/settingswin` | 3.472s | 3.402s | 4.669s | 3.472s | +| `scripts/testshards` | 2.820s | 2.859s | 3.314s | 2.859s | + +Every other non-UI package had a median at or below 2.006s. The slowest exact +UI top-level test medians were: + +| Test | Shard | Attempt 1 | Attempt 2 | Attempt 3 | Median | +|---|---|---:|---:|---:|---:| +| `TestCompareCommandEntryPoints_MenuAndFeatureCallbacksAreIgnored` | `ui-1` | 23.80s | 44.04s | 39.86s | **39.86s** | +| `TestCopySelectionCancelsBeforeOtherCommands` | `ui-2` | 30.69s | 35.91s | 33.48s | 33.48s | +| `TestCompareOpenRefusal_DropDialogShortcutAndOpenWithAreDiscarded` | `ui-3` | 11.63s | 13.04s | 14.60s | 13.04s | +| `TestCopySelectionAvailability` | `ui-3` | 10.00s | 11.31s | 12.57s | 11.31s | +| `TestCompareHelp_F1OpensManualWithoutLeavingComparison` | `ui-3` | 8.68s | 9.89s | 10.83s | 9.89s | +| `TestYieldingMenuCallbacksWrapsEveryField` | `ui-1` | 5.26s | 9.70s | 8.79s | 8.79s | +| `TestCompareLinkToggle_RequiresExactPhysicalControlL` | `ui-2` | 8.03s | 9.25s | 8.76s | 8.76s | +| `TestBuildMainMenu_ManualOpenedObserverSyncsWindowHelp` | `ui-3` | 7.09s | 8.12s | 8.88s | 8.12s | +| `TestAdvance_ShuffleOnNeverRepeatsCurrentIndex` | `ui-2` | 6.42s | 7.37s | 7.04s | 7.04s | +| `TestCopySelectionKeyboard` | `ui-1` | 3.60s | 6.61s | 6.08s | 6.08s | + +The complete 44-package, 1,486 non-UI top-level-test, and 567 UI-test tables +are retained in the reproducible off-tree summaries below; package elapsed +values overlap and are never summed into wall-clock figures. + +### Final assessment + +| Metric | Monolithic median | Sharded median | Change | +|---|---:|---:|---:| +| Queue-inclusive required gate | 22:17 | **7:52** | -14:25 (-64.7%) | +| Queue-excluded execution | 22:10 | **7:36** | -14:34 (-65.7%) | +| Initial queue | 0:05 | 0:16 | +0:11 | +| Validation/build preparation | 2:27 serial before race | 2:32 independent job | +0:05, now overlapped | +| Linux race command / slowest entry | 19:35 | **6:24** | -13:11 (-67.3%) | +| Exact UI package / slowest shard | 16:14.021 | **5:21.919** | -10:52.102 (-67.0%) | +| Required runner minutes | 23:26 | **30:43** | +7:17 (+31.1%) | +| Peak required job slots | 2 | **6** | +4 | + +The 7:52 gate slightly surpasses the 8-12 minute aim. That range was never a +correctness threshold: complete race coverage, native Windows tests, +validation, raw failure evidence, and release gating were retained first. The +critical path is now the UI race group; the per-attempt last entries were +`ui-3`, `ui-1`, and `ui-3`, and the slowest median UI job is `ui-3` at 7:08. + +The 1.064 package-work ratio does not permit a rebalance, so there is no second +checkpoint and no fourth shard. The result is below 12 minutes, so the +over-target stop rule does not trigger. Median required runner time is 1.31x +baseline and remains below the two-times threshold of 46:52; nevertheless, the +accepted trade-off is explicit: about 65% less elapsed feedback uses four more +concurrent runner slots and about 31% more summed runner time. No target or +cost exception was accepted. + +The honest whole-process limit remains: no required process executes all 567 +exact UI tests together. The manifest proves exact coverage and non-overlap, +but cannot prove the absence of dependencies on package-wide test order or +state left by a test assigned to another shard. Separate hosted runners, +existing harness cleanup, and the guard against unreviewed `.Parallel()` calls +are the accepted safety boundary. + +### Sharded artifacts and reproducibility + +Every raw stream was downloaded and hashed outside the repository before the +next rerun request: + +| Attempt | Entry | Lines | Bytes | SHA-256 | +|---:|---|---:|---:|---| +| 1 | `non-ui` | 8,653 | 1,870,485 | `8b299c13be26f4e602409c6e72a41c4b2306c38b1480900f1e4fbd143ea7545d` | +| 1 | `ui-1` | 868 | 192,319 | `a9c53dc5b6ebbee68b823707696c9804c5d5e89118b5261679cec15b0c61e61c` | +| 1 | `ui-2` | 905 | 199,951 | `5be8edd5f1d4fcbce34080c0cae7263a4e8b3dc53f2035ccf9aa5a082c030e4c` | +| 1 | `ui-3` | 1,177 | 264,482 | `13f0802971148a8f4aeedbe1d7e617c655352ce0de7a9d70e7dcd577601573ca` | +| 2 | `non-ui` | 8,653 | 1,870,592 | `7001bd134f748e60df6faa9af420f1fd384630d60c1c1ec8d3d75089856126f2` | +| 2 | `ui-1` | 868 | 192,289 | `74cb55a3752a0f9fd8330d801aaf70e625051822d896c32699cd8a4316858dae` | +| 2 | `ui-2` | 905 | 199,931 | `eb6e0e9187e48849f43d87ba098d77d41fe5341f87089b3873399dcf5a0d09bd` | +| 2 | `ui-3` | 1,177 | 264,450 | `188d53f18c32cac9202be9c0872a861964b87387e81237e189d14e3f59b9a39e` | +| 3 | `non-ui` | 8,653 | 1,870,675 | `54dbc33d0689d714172fcd0d8ae7c284e6fe00c101a6e66f8e21d0983de373d3` | +| 3 | `ui-1` | 868 | 192,335 | `2f45b71e98203dbcd8d70f21ba66db3b17027990bb9292a63f6014145af61e33` | +| 3 | `ui-2` | 905 | 199,946 | `2af5a53867f75cdae217431db576d97cf58e76ab05aaa07ac4b2dfb82938dc48` | +| 3 | `ui-3` | 1,177 | 264,459 | `616b18dcf62f6281173c80d78b6720079a1fe28e5d4a6d1c56e23915f41c6f5b` | + +The off-tree record is `/private/tmp/picfetch-ci-sharded.EK5bnR`. It contains +each attempt's API metadata, artifact listing, four raw streams, separate logs +for all six required jobs, per-file checksums, the recorded origins, and the +analysis scripts. GitHub's aggregate-log command encountered the injected +Qodana check's absent log on attempt 1, so all required logs were fetched by +job ID instead; the partial aggregate output is retained and hashed rather than +mistaken for a complete log. + +Two independent raw-event analyses reproduced byte-for-byte at SHA-256 +`d791bfecdd6bfc6aee10be8ced9bd702af178d7925364354bd89704b02aac159`; +the two Actions timing analyses reproduced at +`d80437d9c4667d043bd8752e5fb36a674b2bd356eca60ebd1296eccacc350265`. +Independent three-run helper summaries reproduced at +`75792ee459e8c2a0b38446a0ce9f73038e94894d0742ebe44d7fa5b622f00e20` +for `non-ui`, `7a4c5bd62efd4abff3e5afb6ad5df153bf20552d867c273bea9c15e0fbbf2f9a` +for `ui-1`, `43726218909254d4757b487f236babc04b898138f3f45d39ab5249bef28a6114` +for `ui-2`, and +`b1605d31f39d81c93dab680f8e6eef1a2d73fdd4ff1c70738335ef4fbe81da53` +for `ui-3`. The verified 65-file checksum inventory hashes to +`290ceefb16f4f3bd94bb00c83d79f5e580c894992665741dae7743390c6a6072`. +No raw CI, log, or local timing artifact is tracked by Git. + +### Final verification + +After the assessment and documentation-only edits, `make fmt-check`, +`go vet ./...`, `go build ./...`, and +`go test ./scripts/testshards -count=1` passed. The canonical +`make check-test-shards` Linux/amd64 run reported 567 runnables across three +shards. Static checks reconfirmed the unchanged Windows test command, no +`needs` or `continue-on-error` in `ci.yml`, exactly one `fail-fast: false`, and +no source diff in `release.yml` from the branch base. All 12 downloaded race +logs contained the required locale, race, fresh-count, and timeout flags, and +the off-tree checksum inventory verified in full. + +The expensive `make verify` race suite was not repeated after these +documentation-only edits, per T8's budget. The identical code and workflow +checkpoint had already passed T6's final local `make verify`, then all four +hosted race entries in each of the three C2 attempts. Final `git diff --check` +passed, the open TODO moved to Done, this plan moved to +`finished_refactorings`, and status showed only those intended documentation +changes and the plan move. + +### Measurement record + +| Checkpoint | SHA | Run ID | Attempts | Gate median | Queue-excluded median | Runner-minute median | +|---|---|---|---:|---:|---:|---:| +| Monolithic C1 | `fcd11103969ac8d1b5a1a220270c0730cb5e1913` | [33734286005](https://github.com/frathe/picfetch/actions/runs/33734286005) | 3/3 | 22:17 | 22:10 | 23:26 | +| Sharded C2 | `b91d56ce67b991b595484a50d9b65c52f1a09dba` | [33758913251](https://github.com/frathe/picfetch/actions/runs/33758913251) | 3/3 | **7:52** | **7:36** | **30:43** | + +### Shard record + +| Entry | Tests/packages | Median race contract | Median setup | Result | +|---|---:|---:|---:|---| +| `non-ui` | 44 packages / 1,486 tests | 3:33 | 1:21 | 3/3 pass | +| `ui-1` | 177 tests | 6:24 | 0:38 | 3/3 pass | +| `ui-2` | 178 tests | 5:57 | 0:36 | 3/3 pass | +| `ui-3` | 212 tests | 6:15 | 0:48 | 3/3 pass | + +## Cost ledger + +| Item | Planned | Actual | +|---|---:|---:| +| Read-only reconnaissance scouts | 3 | 3 | +| Implementation subagents | 0 | 0 | +| Local full race/verify runs after planning | 1, plus at most 1 defect rerun | 2; initial pass plus allowed log-visibility rerun | +| Fresh monolithic CI attempts | 3 | 3 | +| Fresh sharded CI attempts | 3 | 3 | +| Optional rebalance CI attempts | 0 or 3 | 0; 1.064 ratio did not trigger it | +| Peak required job slots | 6 | 6 configured and observed | +| Median runner minutes, before/after | report both | 23:26 / 30:43 (+31.1%) | diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/01-capture-controlled-monolithic-ci-baseline.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/01-capture-controlled-monolithic-ci-baseline.md new file mode 100644 index 0000000..1bca339 --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/01-capture-controlled-monolithic-ci-baseline.md @@ -0,0 +1,62 @@ +# 01: Capture a controlled monolithic CI baseline + +**What to build:** Give maintainers a trustworthy, reproducible picture of the +current required CI critical path before changing its topology. The existing +Linux race suite remains monolithic while producing machine-readable evidence +from three attempts at one commit, allowing setup, validation/build, non-UI, +and exact UI time to be distinguished without changing coverage. + +**Blocked by:** None (can start immediately). + +**Status:** resolved + +- [x] The monolithic Linux job is pinned to Ubuntu 24.04 and retains the + English UTF-8 locale, race detector, 30-minute package timeout, and full + package scope. +- [x] Test-result caching is disabled and machine-readable Go test events are + captured while the real test exit status remains authoritative. +- [x] Raw output is uploaded on success or failure under an attempt-specific + name, fails visibly when absent, and uses 14-day retention. +- [x] Formatting, trust-root validation, vetting, normal build, Windows + cross-build, Windows vet, native Windows tests, cancellation, and + reusable-workflow release gating retain their existing behavior. +- [x] The user commits and pushes the measurement-only checkpoint, and the + initial run is rerun twice only after each preceding attempt completes. +- [x] All three valid attempts have the same Actions SHA and ref. The initial + creation time and both rerun request times are recorded so queue-inclusive + durations remain meaningful. +- [x] Each attempt's raw artifact is downloaded and hashed outside the + repository before the next rerun begins. Required-job logs are also + downloaded and hashed; the documented comment below records that the + full log files were materialized after attempt 3. +- [x] Infrastructure-failed or incomplete attempts are identified and replaced + rather than included in the baseline median. +- [x] The baseline record includes per-attempt and median gate time, + queue-excluded execution, step timings, total runner minutes, package + durations, and exact UI top-level test durations. +- [x] No raw timing artifact is added to source control. + +## Comments + +- 2026-09-03: Prepared the measurement-only checkpoint. YAML parsing, + `git diff --check`, and `make fmt-check` pass. The workflow diff changes only + the explicit Linux runner pin, race-test measurement flags/capture, and raw + artifact upload. Waiting for the user-owned commit and push before collecting + the three same-SHA Actions attempts; repository policy leaves commits to the + user. +- 2026-09-03: Resolved from three successful attempts of + [run 33734286005](https://github.com/frathe/picfetch/actions/runs/33734286005) + at exact SHA `fcd11103969ac8d1b5a1a220270c0730cb5e1913`. Median required-gate + wall time is 22:17, queue-excluded execution is 22:10, the Linux race step is + 19:35, exact `internal/ui` is 16:14.021, and total required runner time is + 23:26. All attempts contain the same 44 package terminals and the same 567 + passing top-level UI tests with no failures or top-level skips. +- 2026-09-03: Raw attempt artifacts were downloaded and hashed before each + subsequent rerun. Full required-job logs were downloaded and hashed after the + third attempt; GitHub retained the attempt-addressable logs and their timing + metadata had already been inspected, so this collection-order deviation lost + no evidence. The complete off-tree record is under + `/private/tmp/picfetch-ci-baseline.XeJlI7`; its deterministic 1,191-line + package/test/allocation report hashes to + `bb61c7b69d41e98b22ef15575a7614b13363b14d029a4cd9b96be6d236454803`. + No raw data is tracked by Git. diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/02-generate-deterministic-shard-manifest.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/02-generate-deterministic-shard-manifest.md new file mode 100644 index 0000000..314b011 --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/02-generate-deterministic-shard-manifest.md @@ -0,0 +1,59 @@ +# 02: Generate the deterministic shard manifest + +**What to build:** Let maintainers turn the three accepted baseline event +streams into a reproducible, reviewable three-shard assignment. The same inputs +must always produce the same summary and assignment, and incomplete timing data +must fail rather than silently inventing weights. + +**Blocked by:** 01: Capture a controlled monolithic CI baseline. + +**Status:** resolved + +- [x] A standard-library Go command reads interleaved machine-readable test + streams and reports package durations plus terminal top-level test + outcomes. +- [x] Malformed or truncated input, duplicate terminal outcomes, missing + terminal outcomes, and tests absent from any required baseline fail with + actionable diagnostics. +- [x] Three-run summaries use the median duration for each top-level test and + never treat summed test durations as workflow wall time. +- [x] Planning assigns every observed Linux top-level UI test, fuzz target, and + example to exactly one of three shards while keeping subtests with their + parent. +- [x] Planning uses deterministic longest-processing-time balancing with + test-name ordering for equal durations and shard-number ordering for equal + loads. +- [x] Repeating summary and planning with identical inputs produces + byte-identical normalized output. +- [x] The checked-in manifest records baseline provenance and the complete + initial name-to-shard assignment generated from the three accepted CI + attempts, not from the one local feasibility run. +- [x] Command-level tests cover interleaved packages, pass/fail/skip terminals, + subtests, median calculation, incomplete baselines, and every balancing + tie-break. +- [x] The new tooling package is recorded in the architecture map and its test + file is included in the repository's exact Qodana test exclusions. +- [x] Focused tooling tests and the Qodana exclusion check pass without running + the full race suite. + +## Comments + +- 2026-09-03: Claimed after ticket 01 produced three accepted same-SHA CI + streams. The implementation seam is the `scripts/testshards` command's + observable files/stdout/stderr/exit status; live Linux inventory validation + and filter generation remain ticket 03. +- 2026-09-03: Resolved with standard-library `summarize` and `plan` command + paths plus command-boundary tests. The real three-run summary contains 44 + package rows and 2,027 top-level runnable rows and reproduced byte-for-byte + at SHA-256 + `933691863a4ed010c18beaea88d5b2b3420fc892b157e34d1968a19ca041fefd`. +- 2026-09-03: The checked-in manifest was generated from Actions run + `33734286005`, attempts 1-3, at baseline SHA + `fcd11103969ac8d1b5a1a220270c0730cb5e1913`. Host and repeated Linux/amd64 + Docker generation were byte-identical at SHA-256 + `6468a50ce0925ade39d253be7e7b0e728aa3d38fdc9f32a6b5547347155c5a18`. + Its 567 unique entries are split 177/178/212 with median-weight sums + 326.750s/327.070s/326.740s; no subtest or `TestMain` is assigned. +- 2026-09-03: `go test ./scripts/testshards -count=1 -v`, `make vet`, + `make fmt-check`, `make check-qodana-test-exclusions`, and + `git diff --check` pass. Per the ticket budget, no full race suite ran. diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/03-enforce-safe-complete-shard-assignment.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/03-enforce-safe-complete-shard-assignment.md new file mode 100644 index 0000000..fb1d146 --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/03-enforce-safe-complete-shard-assignment.md @@ -0,0 +1,54 @@ +# 03: Enforce safe and complete shard assignment + +**What to build:** Make an invalid UI shard assignment impossible to accept as +a successful test run. Maintainers receive a build-aware Linux guard and exact +test filters that fail closed whenever current test inventory and the reviewed +manifest diverge. + +**Blocked by:** 02: Generate the deterministic shard manifest. + +**Status:** resolved + +- [x] The guard obtains the selected test files and runnable top-level test, + fuzz, and example inventory from the current Linux build context. +- [x] The guard compares exact sets and rejects every unassigned current name, + duplicate assignment, stale name, malformed row, unknown shard, and empty + shard. +- [x] Every rejection names the offending test, row, or shard and exits + nonzero; the guard never repairs or broadly reshuffles the manifest by + itself. +- [x] A newly introduced parallel-test call in a Linux-selected test file of + the exact main UI package fails with a message requiring explicit safety + review. +- [x] The guard ignores `TestMain` as harness code and keeps every slash-named + subtest under its top-level parent. +- [x] Filter generation validates the complete manifest before producing one + shard's expression. +- [x] Every produced filter is valid for Go's regular-expression engine, + escaped, anchored at both ends, non-empty, and incapable of matching a + name assigned to another shard. +- [x] Deliberate fixtures demonstrate rejection of missing, duplicate, stale, + malformed, unknown-shard, empty-shard, and parallel-call cases. +- [x] Running the canonical live check under Linux/amd64 succeeds twice without + changing the checked-in manifest. +- [x] A Darwin-derived inventory cannot be mistaken for the canonical Linux + acceptance result. + +## Comments + +- 2026-09-03: Claimed. The established seam is the `scripts/testshards` + command boundary. Implementation will add fail-closed `check` and `regex` + paths plus a public Linux/amd64 Docker check; no manifest regeneration or + application behavior is in scope. +- 2026-09-03: Resolved. `check` derives build-selected test files with + `go list`, derives runnable top-level names with `go test -list`, compares + exact manifest membership, scans only those files for `.Parallel()` calls, + and refuses canonical success outside a matching Linux/amd64 executable and + Go build context. `regex` validates all rows and shards before emitting an + escaped, anchored, non-empty expression. +- 2026-09-03: `go test ./scripts/testshards -count=1`, `make vet`, + `make fmt-check`, `make check-qodana-test-exclusions`, and `git diff --check` + pass. Two independent `make check-test-shards` Docker runs each reported + 567 runnable names across three shards. The checked-in manifest remained + byte-identical at SHA-256 + `6468a50ce0925ade39d253be7e7b0e728aa3d38fdc9f32a6b5547347155c5a18`. diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/04-run-sharded-race-contract-locally.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/04-run-sharded-race-contract-locally.md new file mode 100644 index 0000000..0384b89 --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/04-run-sharded-race-contract-locally.md @@ -0,0 +1,60 @@ +# 04: Run the sharded race contract locally + +**What to build:** Give contributors one canonical local verification path +that exercises precisely the package partition and three UI shards intended +for hosted CI. It runs sequentially in one Linux/amd64 container for practical +local use while preserving the flags, inventory, diagnostics, and raw evidence +that parallel CI will use. + +**Blocked by:** 03: Enforce safe and complete shard assignment. + +**Status:** resolved + +- [x] The normal non-race Docker test command remains a complete unsharded + suite with its current behavior. +- [x] The race and full verification commands first run the canonical Linux + manifest guard, then execute the non-UI partition and all three UI shards + sequentially in one Linux/amd64 container. +- [x] The non-UI partition is the complete module package inventory minus only + the exact main UI package; root, tooling, and every UI feature subpackage + remain included. +- [x] An empty or inconsistent package partition fails before tests run. +- [x] Every race partition shares the English UTF-8 locale, race detector, + fresh execution count, and 30-minute package timeout. +- [x] Direct in-container commands are available for the guard, non-UI tests, + and one selected UI shard so hosted CI can call the same contract without + nesting Docker. +- [x] Test capture prints concise package/test/failure information while + preserving a readable raw event stream. +- [x] A deliberate failed stream proves that either a test failure or capture + failure keeps the overall pipeline failed and identifies the exact + package, shard, and test. +- [x] The public manifest-check command enters Linux/amd64 Docker; a local + macOS inventory is never treated as authoritative. +- [x] The full repository verification command passes once through this final + sequential race path, with no golden screenshots regenerated and no raw + timing files left in the repository. + +## Comments + +- 2026-09-03: Claimed. The established seams are the public and prepared-runner + Make targets plus the `scripts/testshards` command boundary. The normal + unsharded `make test` behavior remains fixed; application code, hosted-CI + topology, translations, and golden files are out of scope. +- 2026-09-03: Resolved. `test-race` now enters one Linux/amd64 container and + runs the live manifest guard, the module inventory minus only exact + `internal/ui`, then `ui-1`, `ui-2`, and `ui-3`. Prepared runners can call + `check-test-shards-direct`, `test-race-non-ui-direct`, or + `test-race-ui-direct TEST_SHARD=ui-N`; all race commands visibly share the + locale, race, fresh-count, timeout, JSON-capture, and pipefail contract. +- 2026-09-03: `go test ./scripts/testshards -count=1 -v`, + `make check-test-shards`, and the final `make verify` pass. The final race + rerun checked 567 UI runnables, passed the complete non-UI partition, then + passed `ui-1` in 229.604s, `ui-2` in 228.490s, and `ui-3` in 211.661s. The + repository inventory was 45 packages = 44 non-UI + exact UI. +- 2026-09-03: A deliberate failed test stream returned pipeline status 1 and + named `non-ui`, `scripts/testshards`, and + `TestDeliberateCaptureFailure`; a forced capture-open failure returned + capture status 1. Capture also reproduced an existing 2,481,918-byte raw CI + stream byte-for-byte. The temporary failure fixture was removed, and no raw + stream or failed golden render remains in the repository. diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/05-run-required-ci-work-concurrently.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/05-run-required-ci-work-concurrently.md new file mode 100644 index 0000000..a232019 --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/05-run-required-ci-work-concurrently.md @@ -0,0 +1,61 @@ +# 05: Run all required CI work concurrently + +**What to build:** Reduce avoidable CI waiting by starting all independent +required work together. Contributors see separately named validation, non-UI, +UI-shard, and Windows results, while release operators retain one mandatory +reusable-CI gate that fails when any entry fails. + +**Blocked by:** 04: Run the sharded race contract locally. + +**Status:** resolved + +- [x] One Ubuntu 24.04 validation job retains formatting, trust-root + validation, Linux GUI dependency setup, shard-manifest validation, + vetting, normal build, both Windows cross-builds, and Windows vetting. +- [x] One Ubuntu 24.04 Linux race matrix has exactly four entries: non-UI and + the three UI shards. +- [x] Every race entry independently checks out the code, sets up Go, installs + Linux GUI dependencies, and calls the shared local/CI test contract. +- [x] The validation job, all four race entries, and the existing Windows test + job have no dependency edges between them. +- [x] Matrix fail-fast is disabled, superseded-workflow cancellation remains + enabled, and no job or step is allowed to hide a failure as optional. +- [x] Every Linux race entry retains the explicit locale, race detector, fresh + execution count, 30-minute package timeout, and intended package/test + scope. +- [x] Visible job, step, summary, and artifact names identify `non-ui` or the + exact UI shard and distinguish repeated workflow attempts. +- [x] Every Linux entry uploads its raw event stream on success or failure, + treats a missing artifact as an error, and uses 14-day retention. +- [x] The native Windows test job remains independent and its test command is + identical to the base revision. +- [x] The reusable-workflow trigger, read-only permissions, concurrency group, + and release gate remain intact, with no release-workflow source change. +- [x] Static workflow review and the previously passing full local verification + show no lost validation step, package, test, or safety flag before the + user creates the sharded checkpoint commit. + +## Comments + +- 2026-09-03: Claimed. The existing Deep plan fixes the topology and shared + Make target contracts. This ticket changes only the reusable CI workflow and + ticket/plan bookkeeping; release packaging, the Windows test command, shard + assignment, Make targets, application code, and test scope remain fixed. +- 2026-09-03: Resolved. `ci.yml` now has one independent Ubuntu 24.04 + validation job, a fail-fast-disabled `non-ui`/`ui-1`/`ui-2`/`ui-3` Ubuntu + race matrix, and the unchanged independent native Windows test job. Each + race entry uses the prepared-runner Make contract and publishes an + attempt-qualified job summary and required 14-day raw artifact even after a + failed test step. +- 2026-09-03: The static YAML contract passed, including exact job topology, + retained triggers/permissions/concurrency, absence of `needs` and + `continue-on-error`, exact matrix values, failure policy, names, summary, + artifact behavior, and Windows command. A parsed before/after comparison + retained every prior validation/build step and the full Windows job body; + `.github/workflows/release.yml` has no diff from `main`. +- 2026-09-03: `make fmt-check`, `go vet ./...`, + `go test ./scripts/testshards -count=1`, all four direct-target dry runs, and + `git diff --check` pass. Ticket 04's final `make verify` remains the full + local race evidence because this ticket changes only Actions YAML. Actual + six-slot execution, reusable-caller failure propagation, and artifact + collection remain the user-owned sharded CI checkpoint in ticket 06. diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/06-measure-assess-close-optimization.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/06-measure-assess-close-optimization.md new file mode 100644 index 0000000..7f3c2a3 --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/issues/06-measure-assess-close-optimization.md @@ -0,0 +1,70 @@ +# 06: Measure, assess, and close the optimization + +**What to build:** Give maintainers an evidence-backed decision on the final +CI topology. Three same-commit sharded attempts must prove coverage, failure +gating, balance, elapsed time, and runner cost before the optimization is +closed or an over-target critical path is presented for explicit review. + +**Blocked by:** 05: Run all required CI work concurrently. + +**Status:** resolved + +- [x] The user commits and pushes the prepared sharded checkpoint, and the + initial workflow is rerun twice only after each preceding attempt + completes. +- [x] All three valid attempts share one Actions SHA and ref; creation/rerun + request times, logs, and artifacts are recorded separately. +- [x] Each attempt's raw artifacts are downloaded and hashed outside the + repository before the next rerun starts, and infrastructure-failed + attempts are identified and replaced. +- [x] Every attempt proves that validation, all four Linux race entries, and + native Windows tests are mandatory, independently scheduled where runner + capacity permits, and successful. +- [x] The union of final Linux results equals the complete module package and + exact UI top-level test inventories with no missing or repeated entry. +- [x] The comparison reports pre/post medians for queue-inclusive reusable-CI + gate time, queue-excluded execution, validation/setup, every race entry, + package/test durations, the critical path, and summed runner minutes. +- [x] The target assessment treats an 8-12 minute median as an aim rather than + a correctness gate and gives test quality priority. +- [x] A slowest/fastest UI-shard median ratio at or below 1.20 causes no + cosmetic manifest churn. +- [x] A ratio above 1.20 that affects the critical path permits at most one + measured rebalance within the same three shards, followed by local + verification and three new same-commit CI attempts. +- [x] A safe, reasonably balanced result above 12 minutes pauses closure and + presents the measured critical path for explicit maintainer disposition; + it does not automatically add a fourth shard or weaken testing. +- [x] A post-change runner-minute median above twice baseline is highlighted + and explained without removing coverage or hiding failures. +- [x] The final work record documents the whole-process ordering limitation, + run IDs, SHAs, attempt results, shard counts, accepted exception if any, + and the increased concurrent-capacity/runner-minute trade-off. +- [x] The open work item moves to Done and the implementation plan moves to the + completed-work record only after every accepted safety and measurement + criterion passes. +- [x] No raw CI or local timing artifact remains in source control, and final + formatting/status checks show only intended changes. + +## Comments + +- 2026-09-03: Resolved from three successful attempts of + [run 33758913251](https://github.com/frathe/picfetch/actions/runs/33758913251) + at exact SHA `b91d56ce67b991b595484a50d9b65c52f1a09dba` on + `feature/CI-test-performance-improvements`. Attempt 1 used the workflow + creation time; the attempt 2 and 3 origins were recorded immediately before + their rerun requests. Each preceding attempt's four raw streams and six + required-job logs were downloaded, parsed, and hashed off-tree before the + next request. No infrastructure attempt needed replacement. +- 2026-09-03: The median required gate fell from 22:17 to 7:52 and the + queue-excluded window from 22:10 to 7:36. The exact final inventory is 44 + non-UI packages plus 567 UI top-level tests split 177/178/212; every UI test + passed in every attempt with no missing or duplicate assignment. The UI + package medians have a 1.064 slowest/fastest ratio, so the 1.20 rebalance + condition did not trigger and the manifest was left unchanged. +- 2026-09-03: Median required runner time rose from 23:26 to 30:43 (+31.1%) + while peak required capacity rose from two jobs to six. This remains below + the 46:52 two-times review threshold. The completed implementation record + contains the full timing, coverage, artifact, failure-gating, critical-path, + and whole-process ordering assessment. No target exception or rebalance was + accepted. diff --git a/finished_refactorings/2026-09-03-measured-ci-test-sharding/spec.md b/finished_refactorings/2026-09-03-measured-ci-test-sharding/spec.md new file mode 100644 index 0000000..74718ee --- /dev/null +++ b/finished_refactorings/2026-09-03-measured-ci-test-sharding/spec.md @@ -0,0 +1,336 @@ +# Spec: measured CI test sharding + +Status: complete + +## Problem Statement + +PicFetch's required GitHub Actions CI gate takes roughly 21-24 minutes even +though much of its validation, build, Windows, non-UI test, and UI test work is +independent. The current Linux job serializes validation and build steps before +one race-enabled test command, and the exact `internal/ui` package dominates +that command. + +Maintainers and contributors need faster feedback without trading away test +quality. Every currently covered Linux and Windows package must remain covered, +the Linux race detector and 30-minute package timeout must remain enabled, and +failures must continue to block reusable CI and tag releases. The change must +be justified by repeatable CI measurements rather than by one unusually fast +or slow runner. + +## Solution + +Measure the existing monolithic Linux race suite three times at one commit, +then run independent validation/build, Linux race, and Windows test work at the +same time. Keep non-UI Linux packages in one race entry and distribute the +exact `internal/ui` top-level tests across three isolated hosted runners using +a deterministic, duration-balanced manifest. + +Add a guard that compares the checked-in assignment with the Linux-selected +test inventory and fails the required CI gate when a test is missing, repeated, +stale, malformed, or assigned to an invalid or empty shard. Use a small Go +helper to summarize machine-readable test output, build and validate the +assignment, emit safe exact-match filters, and retain concise failure output +alongside raw timing artifacts. + +Measure the sharded result three times at one later commit. Aim for an 8-12 +minute median reusable-CI gate, but treat that range as a goal rather than a +hard condition. Complete, reliable testing wins over elapsed time. Report both +the critical path and the increase in concurrent runner capacity and total +runner minutes. + +## User Stories + +1. As a contributor, I want required CI feedback sooner, so that I can respond + to defects without waiting through avoidable serialization. +2. As a maintainer, I want performance changes based on three comparable CI + attempts, so that runner variance is not mistaken for an improvement. +3. As a maintainer, I want all baseline attempts to use one commit, so that + source changes do not contaminate the comparison. +4. As a maintainer, I want all post-change attempts to use one later commit, so + that the final median is internally comparable. +5. As a maintainer, I want test-result caching disabled for measurements, so + that every attempt executes the test bodies being timed. +6. As a reviewer, I want workflow creation, job execution, setup, package, and + top-level test durations reported separately, so that I can identify the + real critical path. +7. As a reviewer, I want initial queue delay separated from execution time, so + that runner availability is not confused with code or workflow performance. +8. As a project owner, I want validation and build work to start independently + from race tests, so that neither group waits for work it does not consume. +9. As a project owner, I want formatting, trust-root validation, vetting, + normal builds, Windows cross-builds, and Windows vetting preserved, so that + faster CI does not weaken non-test validation. +10. As a Windows user, I want the existing native Windows package tests to + remain independent and unchanged, so that release confidence on Windows is + not traded for Linux speed. +11. As a release operator, I want every validation and test entry to remain a + mandatory part of reusable CI, so that a tag release cannot proceed after + any failure. +12. As a maintainer, I want the Linux race detector retained for every Linux + package, so that concurrency regressions remain visible. +13. As a maintainer, I want the 30-minute package timeout retained in every + Linux race invocation, so that long UI packages do not fall back to Go's + shorter default timeout. +14. As a maintainer, I want the explicit English locale retained, so that Fyne + does not emit irrelevant locale parsing failures. +15. As a reviewer, I want Linux jobs pinned to one named runner image, so that + moving aliases do not undermine baseline/post-change comparability. +16. As a test author, I want all non-UI module packages included automatically, + so that root, tooling, and UI feature subpackages are not accidentally + omitted. +17. As a test author, I want only the exact main UI package removed from the + non-UI set, so that similarly prefixed UI subpackages still run with the + race detector. +18. As a test author, I want each top-level UI test assigned exactly once, so + that sharding neither loses coverage nor repeats work. +19. As a test author, I want subtests to remain with their top-level parent, so + that existing setup, cleanup, and process-state assumptions remain intact. +20. As a test author, I want a newly added top-level test to fail the guard + until it receives an explicit assignment, so that coverage cannot silently + decay. +21. As a test author, I want future tests, fuzz targets, and examples included + in the Linux inventory, so that supported top-level test forms cannot evade + the guard. +22. As a maintainer, I want duplicate and stale manifest names rejected, so + that the assignment remains an exact description of the current suite. +23. As a maintainer, I want malformed rows, unknown shard names, and empty + shards rejected, so that invalid configuration cannot degrade into a + successful no-op test command. +24. As a maintainer, I want every shard filter anchored and non-empty, so that + similarly named tests or accidental match-all expressions cannot alter + coverage. +25. As a reviewer, I want shard generation deterministic, so that identical + timing inputs produce an identical reviewable manifest. +26. As a reviewer, I want initial balancing based on median CI durations, so + that one outlier does not dominate assignment. +27. As a reviewer, I want deterministic tie-breaking during balancing, so that + manifest changes do not depend on map or filesystem iteration order. +28. As a maintainer, I want UI shards isolated on separate hosted runners, so + that Fyne process-global state is not made concurrent inside one process. +29. As a maintainer, I want new `t.Parallel()` use in the exact UI test package + to force an explicit safety review, so that in-process concurrency is not + introduced casually. +30. As a contributor, I want a failed job name to identify `non-ui` or the + exact UI shard, so that the failing area is immediately visible. +31. As a contributor, I want failure output to identify the exact package and + test while remaining concise, so that diagnosis does not require scanning + an entire raw event stream. +32. As a CI investigator, I want raw machine-readable test events retained on + both success and failure, so that timings and failures can be audited after + the run. +33. As a CI investigator, I want artifact names to include the matrix entry and + workflow attempt, so that repeated same-commit measurements cannot be + confused. +34. As a project owner, I want all matrix entries to finish after one entry + fails, so that one run reveals every failing shard rather than only the + first one. +35. As a project owner, I want a newer superseding workflow to cancel the older + one, so that obsolete work does not consume runners unnecessarily. +36. As a local contributor, I want one canonical verification command to run + the same race partitions sequentially in Linux/amd64 Docker, so that local + verification matches CI without needing six hosted runners. +37. As a macOS contributor, I want manifest validation to use a Linux build + inventory, so that Darwin-only tests are not mistaken for Linux coverage + gaps. +38. As a maintainer, I want the three UI shards rebalanced only after measured + post-change evidence, so that routine test additions do not create noisy + global reshuffles. +39. As a maintainer, I want a greater-than-20-percent shard imbalance reviewed, + so that one poorly balanced shard does not become the avoidable critical + path. +40. As a project owner, I want a result above 12 minutes explained rather than + hidden, so that performance goals never override test quality. +41. As a project owner, I want a fourth shard to require a separate measured + decision, so that concurrent runner consumption does not grow + automatically. +42. As a project owner, I want pre/post total runner minutes reported, so that + lower elapsed time is considered alongside capacity and resource cost. +43. As a project owner, I want a more-than-twofold runner-minute increase + highlighted for review, so that unusually expensive parallelization is + visible without turning cost into a reason to weaken tests. +44. As a release operator, I want the established reusable-workflow release + gate left intact, so that this optimization cannot bypass signed-release + safety. +45. As a maintainer, I want raw measurement data kept outside source control, + so that the repository contains decisions and summaries rather than + transient CI artifacts. +46. As a reviewer, I want the concurrency trade-off and whole-process test + limitation documented, so that acceptance is based on an honest safety + model. + +## Implementation Decisions + +- Use two explicit implementation checkpoints. The first changes only + measurement behavior and pins the Linux runner; the second introduces the + final parallel topology and shard execution. +- The user creates and pushes each checkpoint commit. Each checkpoint is run + once and rerun twice so its three samples retain the same Actions SHA and + ref. +- For rerun attempts, record the rerun request time before submission and + download/hash each attempt's artifact before starting the next attempt. + This keeps the queue-inclusive origin and per-attempt evidence unambiguous. +- Pin all Linux work to Ubuntu 24.04. +- Use one validation/build job, one four-entry Linux race matrix, and the + existing independent Windows test job. This produces six required job slots: + validation, non-UI race tests, three UI race shards, and Windows tests. +- Do not place dependency edges between those jobs. The reusable workflow's + aggregate result remains the dependency used by tag releases. +- Keep workflow-level cancellation of superseded work. Disable matrix + fail-fast so every shard reports its result. Do not permit optional failures. +- Keep the exact current Windows test command and its existing lack of the race + detector; Windows scope does not expand or contract in this work. +- Every Linux race invocation uses the English UTF-8 locale, the race detector, + a fresh test execution count, and the 30-minute package timeout. +- Determine the non-UI package set from the module's complete package list by + subtracting only the exact `internal/ui` import path. Refuse an empty or + inconsistent partition. +- Use exactly three UI shards initially. Each top-level Linux-selected test, + fuzz target, or example in the exact UI package belongs to exactly one shard; + subtests remain attached to their parent. +- Store the assignment as a checked-in, reviewable name-to-shard manifest with + baseline provenance. New tests require an explicit manifest update rather + than an automatic broad reshuffle. +- Build the initial assignment from the median duration of each top-level test + across the three controlled baseline attempts. +- Balance with deterministic longest-processing-time assignment: descending + median duration, test-name ordering for equal durations, and shard-number + ordering for equal shard loads. +- Add a standard-library Go helper with command-level behaviors for JSON + capture, duration summaries, deterministic planning, live Linux inventory + validation, safe shard-filter generation, and readable failure summaries. +- Validate the whole manifest before emitting a shard filter. Filters are + anchored, escaped for Go's regular-expression engine, and never empty or + match-all. +- The live guard rejects missing, duplicate, stale, malformed, unknown-shard, + and empty-shard configurations. It also rejects newly introduced parallel + calls in Linux-selected tests of the exact UI package pending explicit + review. +- Run the canonical manifest check under Linux/amd64. A direct checker may run + on CI Linux, but a public local command must not accept a Darwin-derived + inventory as canonical. +- Keep the normal non-race Docker suite intact. Change the race and full + verification paths to run the non-UI partition and three UI shards + sequentially in one Linux/amd64 container, reusing setup and build caches. +- In CI, every race entry performs its own checkout, Go setup, and Linux GUI + dependency setup so it is independently runnable. +- Stream compact package/test/failure information while persisting the raw Go + JSON stream. Shell pipeline semantics must preserve a failure from either the + test command or the capture helper. +- Upload one raw artifact per Linux matrix entry even after test failure. Treat + a missing artifact as an error and retain routine artifacts for 14 days. +- Measure the reusable-CI gate from attempt submission/creation to the last + required completion. Also report the execution window from first required + job start to last completion, individual job/step medians, shard/package/test + medians, and summed runner minutes. +- Do not add a timing-based CI pass/fail threshold. Timing determines review + and rebalancing, not correctness. +- Rebalance within the same three shards only when three post-change attempts + show a slowest/fastest median ratio above 1.20 that affects the critical path, + or when imbalance explains a missed target. Permit at most one measured + rebalancing round in this pull request. +- If a safe and reasonably balanced three-shard result remains above 12 + minutes, stop before final acceptance and present the measured critical path. + Do not add a fourth shard, skip checks, retry failures, or weaken coverage + automatically. +- Treat a total-runner-minute median above twice the monolithic baseline as a + mandatory review note, not as an automatic rejection or permission to remove + tests. +- Update the repository architecture map for the new tooling package and close + the open work item only after the final three-run comparison is accepted. +- Do not create a domain ADR: the sharding topology is reversible operational + configuration and does not change PicFetch's product domain model. + +## Testing Decisions + +- The highest useful seams are the test-sharding command, the canonical + Make/Docker verification command, and completed GitHub Actions runs. No + application-level seam or UI interaction is introduced for this work. +- Test the helper through CLI-equivalent inputs and observable outputs: raw Go + JSON streams, manifest text, current inventory, emitted filters, summaries, + diagnostics, and exit status. Avoid assertions on incidental parser data + structures or algorithm implementation details. +- Use fixture event streams to cover interleaved packages, top-level tests and + subtests, pass/fail/skip terminals, malformed and truncated JSON, missing + terminal events, and repeated same-test observations across three runs. +- Prove deterministic planning by supplying equal durations and equal shard + loads, rerunning with identical inputs, and requiring byte-identical + normalized output. +- Prove the manifest safety boundary with deliberately invalid fixtures for an + unassigned current test, duplicate assignment, stale name, malformed row, + unknown shard, empty shard, and newly added parallel call. Each diagnostic + names the offending test, row, or shard. +- Prove that each emitted filter is anchored, non-empty, valid under Go's + regular-expression implementation, matches every assigned name, and matches + no name owned by another shard. +- Prove package coverage by comparing the observed non-UI set plus the exact UI + package with the full module package inventory. Root, tooling, and all UI + feature subpackages must appear in the union. +- Prove capture behavior with a deliberately failed test stream: concise output + names the package/test, the raw stream remains readable, and the overall + pipeline remains failed. +- Existing Make targets are the prior art for canonical Linux/amd64 Docker + execution. The final full local acceptance seam remains `make verify`. +- The existing UI harness and its explicit cleanup/drain behavior are prior art + for process-state containment. No broad UI-test parallelism is added. +- The first workflow checkpoint must produce three valid monolithic attempts. + An infrastructure failure is labeled and replaced rather than included in a + performance median. +- The final checkpoint must produce three valid sharded attempts at one SHA. + Every required job must pass and all raw artifacts must parse to a complete, + non-overlapping package/test union. +- Inspect actual job logs to confirm the locale, race, count, and timeout flags; + do not infer them solely from generated configuration. +- Inspect the completed workflow graph to confirm all six entries are required, + independent where runners permit, and visible after a shard failure. +- Compare the Windows command against the base revision and verify the release + workflow has no source diff. +- Run the complete local race/verification suite once after the final topology + is assembled, with at most one defect-driven rerun. Do not regenerate golden + screenshots. +- Record the final gate median, queue-excluded median, critical entry, shard + balance, package/test coverage, and runner-minute median in the completed + work record. + +## Out of Scope + +- Removing, skipping, retrying, or making any existing validation or test + optional. +- Weakening Linux race coverage, the package timeout, locale handling, or + package inventory. +- Broad use of in-process test parallelism. +- Automatically increasing the UI shard count beyond three. +- Changing product code, UI behavior, translations, golden screenshots, or + application package boundaries. +- Expanding or shrinking Windows test coverage or installing a Windows race + toolchain. +- Changing release packaging, signing, publication, or reusable-workflow + dependencies. +- Changing repository rulesets, branch protection, or required-status-check + administration. +- Committing raw CI JSON or local timing artifacts. +- Creating new domain terminology, `CONTEXT` entries, or an ADR. + +## Further Notes + +- The prerequisite first signed Windows release is complete; this work remains + a separate pull request. +- Recent coarse CI evidence has a 22:00 median critical job, 2:29 median work + before the race step, 19:34 median race step, and 16:16 median exact UI + package. Controlled checkpoint data supersedes these observations. +- A fresh Linux/amd64 Docker run at clean commit + `6af8c51f9bdfb0b36907be8b5c8241586ec3a602` completed in 13:22.04 wall time. + Its JSON execution span was 11:13.53; the exact UI package took 11:12.513 and + all 567 Linux top-level UI tests passed. +- A three-way deterministic allocation of that one local run produced + theoretical test-only totals of 223.72s, 223.81s, and 223.83s. This proves + feasibility only; the initial checked-in manifest uses the three fresh CI + baselines. +- Lower elapsed time consumes up to six concurrent hosted runners and may use + more total runner minutes than the monolithic job. +- Honest limit: after sharding, no required job executes every exact UI test in + one process. The manifest proves coverage and non-overlap but cannot prove + the absence of test-order dependencies. Separate runners, the existing UI + harness cleanup, and the prohibition on unreviewed parallel calls are the + accepted safety boundary. +- The companion Deep implementation plan contains the phased task graph, + acceptance commands, measurement tables, stop rule, and cost ledger. diff --git a/plans/2026-09-03-microsoft-store-msix.md b/plans/2026-09-03-microsoft-store-msix.md new file mode 100644 index 0000000..ad39297 --- /dev/null +++ b/plans/2026-09-03-microsoft-store-msix.md @@ -0,0 +1,248 @@ +# Microsoft Store MSIX + +## Frame + +Deliver a Microsoft Store-ready x64/ARM64 MSIX bundle whose manifest uses the +Partner Center identity for PicFetch, registers every supported image type, +and whose Store build defers application updates to Microsoft Store. + +Route: **Deep**. This adds a Windows shipping format, build-channel-specific +runtime behavior, release automation, and Store identity metadata. + +## Problem + +PicFetch currently publishes signed portable Windows executables inside ZIP +archives. Partner Center has an MSIX product reservation, but the repository +has no MSIX manifest, package assets, bundle builder, or Store distribution +channel. The existing in-app updater replaces its running executable, which is +not a valid update mechanism for an MSIX installed under WindowsApps. + +## Decisions — do not relitigate + +| Decision | Choice | +|---|---| +| Partner Center product name | `PicFetch` | +| Package identity name | `OpenSourceDeveloperFloria.PicFetch` | +| Package publisher | `CN=D9654E56-586C-4C1E-ABC8-71CCDC33B78F` | +| Publisher display name | `Open Source Developer Florian Rathe` | +| Store ID | `9P0DM0KTH01K` (documentation only; not a manifest field) | +| Package format | One `.msixbundle` containing x64 and ARM64 packages | +| Store package version | `1.0..0`; for Build 440, `1.0.440.0` | +| Runtime model | Packaged classic desktop app, medium integrity, full trust | +| Minimum Windows | Windows 10 build 19041, required by the chosen uap10 manifest vocabulary | +| File associations | One alternate image-viewer association containing exactly `imaging.SupportedExtensions()` | +| Build selection | `microsoftstore` Go build tag; ordinary builds remain self-updating | +| Store Updates tab | Show version plus “Updates are managed by Microsoft Store.”; no update controls | +| Release behavior | Existing portable archives and Certum signing stay unchanged; add a separately built Store artifact | +| Store submission | Manual first upload; Partner Center API automation is deferred | + +## Test seams confirmed by the user + +1. `make package-windows-store` is the packaging command and produces one + x64/ARM64 `.msixbundle` on a Windows host with MakeAppx. +2. `scripts/msix` is the deterministic package-layout seam: tests inspect its + manifest, identity, package version, assets, architectures, and file types. +3. `settingswin.New(..., storeManagedUpdates)` is the presentation seam: Store + mode shows the managed-update message and exposes no self-update controls. +4. `viewer` update entry points are the behavior seam: Store mode performs no + automatic/manual check, requests no apply/relaunch, and skips staged apply + on shutdown; portable mode retains the existing tests and behavior. + +## Acceptance criteria + +### AC1 — Store update behavior is inert + +Store mode restores no self-update preference, starts no automatic or manual +worker, never applies a staged portable update during shutdown, and never quits +to self-update. Portable behavior is unchanged. + +Verify: + +```sh +go test ./internal/ui/... -run 'TestStoreManagedUpdates|TestManualUpdateFlow|TestCheckForUpdates' +``` + +### AC2 — Settings explains Store-managed updates + +The Store Updates tab contains the version label and localized managed-update +message, but not the automatic checkbox or manual button. Portable mode keeps +all three existing objects. + +Verify: + +```sh +go test ./internal/ui/settingswin/... -run 'Test(StoreManagedUpdates|UpdatesTab|UpdateNow)' +go test . -run 'TestTranslations' +``` + +### AC3 — MSIX layout is derived and exact + +The layout generator emits a valid XML manifest containing the exact Partner +Center identity, `PicFetch` display name, x64 or ARM64 architecture, version +`1.0..0`, packaged-classic/full-trust declarations, all supported image +extensions, and every referenced PNG asset plus `picfetch.exe`. + +Verify: + +```sh +go test ./scripts/msix/... +``` + +### AC4 — The public packaging command builds both architectures + +`make package-windows-store` builds Store-tagged x64 and ARM64 executables, +creates both MSIX packages with MakeAppx semantic validation enabled, bundles +them, and writes `bin/PicFetch-Store.msixbundle`. It fails clearly when run +without Windows SDK tooling. + +Verify on Windows: + +```powershell +make package-windows-store +Test-Path bin/PicFetch-Store.msixbundle +``` + +Verify statically on this host: + +```sh +go test ./scripts/msix/... && make -n package-windows-store +``` + +### AC5 — Release and CI retain existing channels and add the Store artifact + +The release workflow still publishes the existing signed ZIPs and also uploads +the Store bundle. CI compiles the Store-tagged UI path and tests the layout +generator on Windows. + +Verify: + +```sh +go test ./scripts/msix/... && rg -n 'package-windows-store|PicFetch-Store.msixbundle|microsoftstore' Makefile .github/workflows +``` + +### AC6 — Repository records match the new package map and workflow + +`ARCHITECTURE.md`, `README.md`, and `todos.md` describe the Store channel, +package generator, release artifact, manual first submission, and Windows-only +validation limit. + +Verify: + +```sh +rg -n 'Microsoft Store|MSIX|scripts/msix|microsoftstore' ARCHITECTURE.md README.md todos.md +``` + +## Non-goals + +- Operating Partner Center, uploading the first package, or submitting it for + certification. +- Automating Partner Center submissions before the first manual certification. +- Replacing the portable ZIP/WinGet release channel or its updater. +- Migrating settings between an existing portable copy and the packaged copy. +- Adding Microsoft commerce, WNS, Xbox, or telemetry integrations. +- Producing an EXE/MSI installer. + +## Honest limit + +This host cannot install or launch an MSIX, run MakeAppx, or run the Windows App +Certification Kit. The generator, manifests, build graph, Go behavior, and +cross-compilation can be verified here; actual package creation, installation, +file-association activation, upgrade/uninstall behavior, and WACK results must +be verified by the Windows workflow and then on a clean Windows machine before +upload. + +## File map + +| Area | Files | +|---|---| +| Store build selection | create `internal/ui/distribution_default.go`, `internal/ui/distribution_microsoftstore.go`; modify `startup.go`, `viewer.go` | +| Update behavior | modify `internal/ui/autoupdate.go`, `run.go`, `features.go`, `autoupdate_test.go` | +| Settings presentation | modify `internal/ui/settingswin/settingswin.go`, `settingswin_test.go`, both translation bundles | +| Package generator | create `scripts/msix/main.go`, `scripts/msix/package.go`, `scripts/msix/package_test.go` | +| Packaging/release | modify `Makefile`, `.github/workflows/ci.yml`, `.github/workflows/release.yml` | +| Project records | modify `ARCHITECTURE.md`, `README.md`, `todos.md`, `qodana.yaml` | + +## Task graph + +```text +T1 Store update policy -----> T2 Settings presentation + | | + +------------+---------------+ + v +T3 MSIX layout generator ---> T4 Makefile/workflows ---> T5 docs/final gate +``` + +### Task 1 — Store update policy + +Owner: T0 inline +Files: `internal/ui/distribution_*.go`, `startup.go`, `viewer.go`, `features.go`, +`run.go`, `autoupdate.go`, `autoupdate_test.go` +Depends: none +Contract: `startupState.storeManagedUpdates` is immutable after construction; +the zero value preserves portable behavior. +Test: Store-mode viewer update entry points are inert and shutdown skips apply. +Verify: AC1 command +Budget: 0 spawns · ≤ 2 review rounds · full suite: no + +### Task 2 — Store Settings presentation + +Owner: T0 inline +Files: `internal/ui/settingswin/settingswin.go`, `settingswin_test.go`, +`translations/en.json`, `translations/de.json` +Depends: T1 field contract +Contract: `settingswin.New(app, host, storeManagedUpdates)`; portable layout +unchanged. +Test: Store Updates tab object identity and host non-interaction. +Verify: AC2 commands +Budget: 0 spawns · ≤ 2 review rounds · full suite: no + +### Task 3 — MSIX layout generator + +Owner: T0 inline +Files: `scripts/msix/*`, `qodana.yaml` +Depends: none +Contract: CLI `msix --arch amd64|arm64 --exe PATH --out DIR`; output is an +unpacked MSIX layout for MakeAppx. +Test: black-box `run` plus parsed XML and decoded asset dimensions. +Verify: AC3 command +Budget: 0 spawns · ≤ 3 review rounds · full suite: no + +### Task 4 — Packaging and workflow integration + +Owner: T0 inline +Files: `Makefile`, `.github/workflows/ci.yml`, `.github/workflows/release.yml` +Depends: T1, T3 +Contract: `make package-windows-store` -> `bin/PicFetch-Store.msixbundle`. +Test: generator tests plus static dry-run/workflow assertions. +Verify: AC4 and AC5 commands +Budget: 0 spawns · ≤ 2 review rounds · full suite: no + +### Task 5 — Records and final gate + +Owner: T0 inline +Files: `ARCHITECTURE.md`, `README.md`, `todos.md`, this plan +Depends: T1–T4 +Contract: no new package/file moves remain undocumented; first upload remains +manual and Windows runtime verification remains explicit. +Test: documentation grep, all AC commands, negative guard checks, final gate. +Verify: AC6 plus `make verify` +Budget: 0 spawns · ≤ 2 review rounds · full suite: yes, once + +## Delegation routing + +All tasks remain T0 inline. T1 and T2 fail G5 because the lead holds the active +architecture and packaging context. T3 and T4 also fail G5 and fall under the +workflow’s “platform-specific behavior that cannot be tested here” prohibition. +T5 is review/final-gate work and is never delegated. Rule S applies only to +generated PNG resizing and formatting; those deterministic operations run as +commands rather than model work. + +## Cost ledger + +| Task | Spawns (budget/actual) | Review rounds | Full suite | Notes | +|---|---:|---:|---|---| +| T1 | 0 / 0 | 0 | no | | +| T2 | 0 / 0 | 0 | no | | +| T3 | 0 / 0 | 0 | no | | +| T4 | 0 / 0 | 0 | no | | +| T5/gate | — | — | yes | | diff --git a/qodana.yaml b/qodana.yaml index 10972c3..e7ddf9c 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -241,5 +241,6 @@ exclude: - "scripts/releasenotes/main_test.go" - "scripts/releasenotes/notes_test.go" - "scripts/synctuf/main_test.go" + - "scripts/testshards/main_test.go" - "scripts/wingettag/tag_test.go" - "scripts/wingettag/workflow_test.go" diff --git a/scripts/testshards/main.go b/scripts/testshards/main.go new file mode 100644 index 0000000..288c09d --- /dev/null +++ b/scripts/testshards/main.go @@ -0,0 +1,1096 @@ +// Command testshards summarizes and captures Go test events, plans deterministic +// shards, validates live assignments, partitions packages, and emits exact shard +// filters. +package main + +import ( + "bufio" + "bytes" + "encoding/json" + "errors" + "flag" + "fmt" + "go/ast" + "go/parser" + "go/token" + "io" + "math" + "os" + "os/exec" + "path" + "path/filepath" + "regexp" + "runtime" + "slices" + "sort" + "strconv" + "strings" + "time" +) + +type pathsFlag []string + +func (p *pathsFlag) String() string { + return strings.Join(*p, ",") +} + +func (p *pathsFlag) Set(value string) error { + if value == "" { + return errors.New("path must not be empty") + } + *p = append(*p, value) + return nil +} + +type goEvent struct { + Action string `json:"Action"` + Package string `json:"Package"` + Test string `json:"Test"` + Output string `json:"Output"` + Elapsed *float64 `json:"Elapsed"` +} + +type terminal struct { + action string + elapsed time.Duration +} + +type testKey struct { + packageName string + name string +} + +type streamSummary struct { + packages map[string]terminal + tests map[testKey]terminal +} + +func main() { + if err := run(os.Args[1:], os.Stdout, os.Stderr); err != nil { + _, _ = fmt.Fprintf(os.Stderr, "testshards: %v\n", err) + os.Exit(1) + } +} + +func run(args []string, stdout, stderr io.Writer) error { + return runWithInput(args, os.Stdin, stdout, stderr) +} + +func runWithInput(args []string, stdin io.Reader, stdout, stderr io.Writer) error { + if len(args) == 0 { + return errors.New("usage: testshards ...") + } + + switch args[0] { + case "summarize": + return runSummarize(args[1:], stdout, stderr) + case "plan": + return runPlan(args[1:], stdout, stderr) + case "check": + return runCheck(args[1:], stdout, stderr) + case "regex": + return runRegex(args[1:], stdout, stderr) + case "capture": + return runCapture(args[1:], stdin, stdout, stderr) + case "partition": + return runPartition(args[1:], stdout, stderr) + default: + return fmt.Errorf("unknown command %q", args[0]) + } +} + +func runPartition(args []string, stdout, stderr io.Writer) error { + fs := flag.NewFlagSet("partition", flag.ContinueOnError) + fs.SetOutput(stderr) + packageArg := fs.String("package", "", "exact package to remove from the module inventory") + if err := fs.Parse(args); err != nil { + return err + } + if fs.NArg() != 0 || *packageArg == "" { + return errors.New("usage: testshards partition -package PACKAGE") + } + + packages, err := loadPackagePartition(*packageArg) + if err != nil { + return err + } + for _, packageName := range packages { + if _, err := fmt.Fprintln(stdout, packageName); err != nil { + return err + } + } + return nil +} + +func loadPackagePartition(packageArg string) ([]string, error) { + if strings.HasPrefix(packageArg, "-") || strings.Contains(packageArg, "...") { + return nil, fmt.Errorf("package %s must name one exact package", packageArg) + } + selectedOutput, err := goCommandOutput("list", "-f", "{{.ImportPath}}", "--", packageArg) + if err != nil { + return nil, err + } + selected, err := packageNames(selectedOutput, "selected package") + if err != nil { + return nil, err + } + if len(selected) != 1 { + return nil, fmt.Errorf("package %s resolved to %d packages; want exactly one", packageArg, len(selected)) + } + + moduleOutput, err := goCommandOutput("list", "-f", "{{.ImportPath}}", "--", "./...") + if err != nil { + return nil, err + } + modulePackages, err := packageNames(moduleOutput, "module inventory") + if err != nil { + return nil, err + } + return partitionPackageNames(modulePackages, selected[0]) +} + +func packageNames(output []byte, source string) ([]string, error) { + lines := strings.Split(strings.TrimSpace(string(output)), "\n") + if len(lines) == 1 && lines[0] == "" { + return nil, fmt.Errorf("%s is empty", source) + } + for index := range lines { + if lines[index] == "" || strings.TrimSpace(lines[index]) != lines[index] || strings.ContainsAny(lines[index], "\t\r ") { + return nil, fmt.Errorf("%s contains malformed package name %q", source, lines[index]) + } + } + return lines, nil +} + +func partitionPackageNames(modulePackages []string, selected string) ([]string, error) { + seen := make(map[string]struct{}, len(modulePackages)) + partition := make([]string, 0, len(modulePackages)) + selectedFound := false + for _, packageName := range modulePackages { + if _, exists := seen[packageName]; exists { + return nil, fmt.Errorf("module inventory contains duplicate package %s", packageName) + } + seen[packageName] = struct{}{} + if packageName == selected { + selectedFound = true + continue + } + partition = append(partition, packageName) + } + if !selectedFound { + return nil, fmt.Errorf("selected package %s is absent from the module inventory", selected) + } + if len(partition) == 0 { + return nil, errors.New("non-UI package partition is empty") + } + sort.Strings(partition) + return partition, nil +} + +func runCapture(args []string, stdin io.Reader, stdout, stderr io.Writer) error { + fs := flag.NewFlagSet("capture", flag.ContinueOnError) + fs.SetOutput(stderr) + outPath := fs.String("out", "", "path that receives the raw go test -json stream") + partition := fs.String("partition", "", "partition label included in concise output") + if err := fs.Parse(args); err != nil { + return err + } + if fs.NArg() != 0 || *outPath == "" || *partition == "" { + return errors.New("usage: testshards capture -out PATH -partition NAME") + } + if strings.TrimSpace(*partition) != *partition || strings.ContainsAny(*partition, "\t\r\n") { + return fmt.Errorf("partition %q must be a non-empty single field", *partition) + } + + raw, err := os.Create(*outPath) + if err != nil { + return fmt.Errorf("create capture %s: %w", *outPath, err) + } + captureErr := captureStream(stdin, raw, stdout, *partition) + if err := raw.Close(); err != nil { + captureErr = errors.Join(captureErr, fmt.Errorf("close capture %s: %w", *outPath, err)) + } + return captureErr +} + +func captureStream(input io.Reader, raw io.Writer, concise io.Writer, partition string) error { + reader := bufio.NewReader(input) + pendingOutput := make(map[testKey][]string) + eventCount := 0 + for { + line, readErr := reader.ReadBytes('\n') + if len(line) != 0 { + if _, err := raw.Write(line); err != nil { + return fmt.Errorf("write raw event stream: %w", err) + } + eventCount++ + var event goEvent + if err := json.Unmarshal(bytes.TrimSpace(line), &event); err != nil { + return finishCapture(reader, raw, fmt.Errorf("parse captured event at line %d: %w", eventCount, err)) + } + if event.Action == "output" && event.Output != "" { + key := testKey{packageName: event.Package, name: event.Test} + pendingOutput[key] = append(pendingOutput[key], captureOutputLines(event.Output)...) + } + if isTerminal(event.Action) { + key := testKey{packageName: event.Package, name: event.Test} + if event.Action == "fail" { + for _, output := range pendingOutput[key] { + if err := writeCaptureFailure(concise, partition, key, output); err != nil { + return finishCapture(reader, raw, err) + } + } + } + delete(pendingOutput, key) + if event.Test == "" { + if err := writeCaptureTerminal(concise, "package", partition, event.Package, "", event.Action, event.Elapsed); err != nil { + return finishCapture(reader, raw, err) + } + } else if event.Action == "fail" || isTopLevelRunnable(event.Test) { + if err := writeCaptureTerminal(concise, "test", partition, event.Package, event.Test, event.Action, event.Elapsed); err != nil { + return finishCapture(reader, raw, err) + } + } + } + } + if readErr != nil { + if errors.Is(readErr, io.EOF) { + break + } + return fmt.Errorf("read event stream: %w", readErr) + } + } + if eventCount == 0 { + return errors.New("captured stream contains no Go test events") + } + return nil +} + +func finishCapture(input io.Reader, raw io.Writer, captureErr error) error { + if _, err := io.Copy(raw, input); err != nil { + return errors.Join(captureErr, fmt.Errorf("preserve remaining raw event stream: %w", err)) + } + return captureErr +} + +func captureOutputLines(output string) []string { + output = strings.TrimRight(output, "\r\n") + if output == "" { + return nil + } + lines := strings.Split(output, "\n") + for index := range lines { + lines[index] = strings.TrimSuffix(lines[index], "\r") + } + return lines +} + +func writeCaptureFailure(out io.Writer, partition string, key testKey, output string) error { + if key.name == "" { + _, err := fmt.Fprintf(out, "failure\tpartition=%s\tpackage=%s\t%s\n", partition, key.packageName, output) + return err + } + _, err := fmt.Fprintf(out, "failure\tpartition=%s\tpackage=%s\ttest=%s\t%s\n", partition, key.packageName, key.name, output) + return err +} + +func writeCaptureTerminal(out io.Writer, kind, partition, packageName, testName, action string, elapsed *float64) error { + elapsedText := "unknown" + if elapsed != nil { + elapsedText = strconv.FormatFloat(*elapsed, 'f', 3, 64) + } + if testName == "" { + _, err := fmt.Fprintf(out, "%s\tpartition=%s\tpackage=%s\taction=%s\telapsed=%s\n", kind, partition, packageName, action, elapsedText) + return err + } + _, err := fmt.Fprintf(out, "%s\tpartition=%s\tpackage=%s\tname=%s\taction=%s\telapsed=%s\n", kind, partition, packageName, testName, action, elapsedText) + return err +} + +func runSummarize(args []string, stdout, stderr io.Writer) error { + fs := flag.NewFlagSet("summarize", flag.ContinueOnError) + fs.SetOutput(stderr) + var paths pathsFlag + fs.Var(&paths, "json", "path to go test -json output (repeatable)") + if err := fs.Parse(args); err != nil { + return err + } + if fs.NArg() != 0 || len(paths) == 0 { + return errors.New("usage: testshards summarize -json PATH [-json PATH ...]") + } + + summaries, err := loadComparableStreams(paths) + if err != nil { + return err + } + return writeSummary(stdout, summaries) +} + +func runPlan(args []string, stdout, stderr io.Writer) error { + fs := flag.NewFlagSet("plan", flag.ContinueOnError) + fs.SetOutput(stderr) + var paths pathsFlag + fs.Var(&paths, "json", "path to go test -json output (must be repeated three times)") + packageName := fs.String("package", "", "exact package import path represented by the manifest") + shardCount := fs.Int("shards", 0, "number of shards") + baselineSHA := fs.String("baseline-sha", "", "source commit shared by the baseline attempts") + baselineRun := fs.String("baseline-run", "", "GitHub Actions baseline run ID") + if err := fs.Parse(args); err != nil { + return err + } + if fs.NArg() != 0 || len(paths) != 3 || *packageName == "" || *shardCount < 1 || *baselineSHA == "" || *baselineRun == "" { + return errors.New("usage: testshards plan -package PACKAGE -shards N -baseline-sha SHA -baseline-run RUN -json RUN1 -json RUN2 -json RUN3") + } + + summaries, err := loadComparableStreams(paths) + if err != nil { + return err + } + return writePlan(stdout, summaries, *packageName, *shardCount, *baselineSHA, *baselineRun) +} + +func runRegex(args []string, stdout, stderr io.Writer) error { + fs := flag.NewFlagSet("regex", flag.ContinueOnError) + fs.SetOutput(stderr) + manifestPath := fs.String("manifest", "", "path to the shard manifest") + shardName := fs.String("shard", "", "shard whose exact-match filter to emit") + if err := fs.Parse(args); err != nil { + return err + } + if fs.NArg() != 0 || *manifestPath == "" || *shardName == "" { + return errors.New("usage: testshards regex -manifest PATH -shard ui-N") + } + + manifest, err := readShardManifest(*manifestPath) + if err != nil { + return err + } + names, exists := manifest.namesByShard[*shardName] + if !exists { + return fmt.Errorf("manifest %s has no shard %s", *manifestPath, *shardName) + } + + sort.Strings(names) + quoted := make([]string, len(names)) + for index, name := range names { + quoted[index] = regexp.QuoteMeta(name) + } + filter := "^(" + strings.Join(quoted, "|") + ")$" + if filter == "^()$" { + return fmt.Errorf("manifest shard %s is empty", *shardName) + } + if _, err := regexp.Compile(filter); err != nil { + return fmt.Errorf("compile filter for shard %s: %w", *shardName, err) + } + _, err = fmt.Fprintln(stdout, filter) + return err +} + +func runCheck(args []string, stdout, stderr io.Writer) error { + fs := flag.NewFlagSet("check", flag.ContinueOnError) + fs.SetOutput(stderr) + packageArg := fs.String("package", "", "exact package whose runnable inventory to validate") + manifestPath := fs.String("manifest", "", "path to the shard manifest") + if err := fs.Parse(args); err != nil { + return err + } + if fs.NArg() != 0 || *packageArg == "" || *manifestPath == "" { + return errors.New("usage: testshards check -package PACKAGE -manifest PATH") + } + + manifest, err := readShardManifest(*manifestPath) + if err != nil { + return err + } + buildContext, err := currentGoBuildContext() + if err != nil { + return err + } + if buildContext.GOOS != runtime.GOOS || buildContext.GOARCH != runtime.GOARCH { + return fmt.Errorf("Go build context %s/%s does not match running testshards executable %s/%s", buildContext.GOOS, buildContext.GOARCH, runtime.GOOS, runtime.GOARCH) + } + inventory, err := loadRunnableInventory(*packageArg) + if err != nil { + return err + } + if manifest.packageName != inventory.packageName { + return fmt.Errorf("manifest package %s does not match selected package %s", manifest.packageName, inventory.packageName) + } + if err := validateExactAssignment(manifest, inventory.names); err != nil { + return err + } + if err := rejectParallelCalls(inventory); err != nil { + return err + } + if runtime.GOOS != "linux" || runtime.GOARCH != "amd64" { + return fmt.Errorf("canonical shard validation requires linux/amd64; running testshards executable is %s/%s (use make check-test-shards)", runtime.GOOS, runtime.GOARCH) + } + + _, err = fmt.Fprintf(stdout, "checked\tpackage=%s\trunnables=%d\tshards=%d\n", inventory.packageName, len(inventory.names), len(manifest.namesByShard)) + return err +} + +type goBuildContext struct { + GOOS string + GOARCH string +} + +func currentGoBuildContext() (goBuildContext, error) { + output, err := goCommandOutput("env", "-json", "GOOS", "GOARCH") + if err != nil { + return goBuildContext{}, err + } + var context goBuildContext + if err := json.Unmarshal(output, &context); err != nil { + return goBuildContext{}, fmt.Errorf("parse go env build context: %w", err) + } + if context.GOOS == "" || context.GOARCH == "" { + return goBuildContext{}, errors.New("go env returned an empty GOOS or GOARCH") + } + return context, nil +} + +type listedPackage struct { + Dir string + ImportPath string + TestGoFiles []string + XTestGoFiles []string +} + +type runnableInventory struct { + packageName string + dir string + testFiles []string + names map[string]struct{} +} + +func loadRunnableInventory(packageArg string) (runnableInventory, error) { + if strings.HasPrefix(packageArg, "-") || strings.Contains(packageArg, "...") { + return runnableInventory{}, fmt.Errorf("package %s must name one exact package", packageArg) + } + output, err := goCommandOutput("list", "-json", "--", packageArg) + if err != nil { + return runnableInventory{}, err + } + decoder := json.NewDecoder(bytes.NewReader(output)) + var listed listedPackage + if err := decoder.Decode(&listed); err != nil { + return runnableInventory{}, fmt.Errorf("parse go list result for %s: %w", packageArg, err) + } + var extra listedPackage + if err := decoder.Decode(&extra); !errors.Is(err, io.EOF) { + if err == nil { + return runnableInventory{}, fmt.Errorf("package %s resolved to more than one package", packageArg) + } + return runnableInventory{}, fmt.Errorf("parse trailing go list result for %s: %w", packageArg, err) + } + if listed.Dir == "" || listed.ImportPath == "" { + return runnableInventory{}, fmt.Errorf("go list returned incomplete package metadata for %s", packageArg) + } + testFiles := append([]string(nil), listed.TestGoFiles...) + testFiles = append(testFiles, listed.XTestGoFiles...) + sort.Strings(testFiles) + if len(testFiles) == 0 { + return runnableInventory{}, fmt.Errorf("package %s has no build-selected test files", listed.ImportPath) + } + + output, err = goCommandOutput("test", "-run", "^$", "-list", "^(Test|Fuzz|Example)", packageArg) + if err != nil { + return runnableInventory{}, err + } + names := make(map[string]struct{}) + scanner := bufio.NewScanner(bytes.NewReader(output)) + for scanner.Scan() { + name := strings.TrimSpace(scanner.Text()) + if name == "TestMain" || !isTopLevelRunnable(name) { + continue + } + if _, exists := names[name]; exists { + return runnableInventory{}, fmt.Errorf("go test -list returned duplicate runnable %s", name) + } + names[name] = struct{}{} + } + if err := scanner.Err(); err != nil { + return runnableInventory{}, fmt.Errorf("read go test -list output for %s: %w", listed.ImportPath, err) + } + if len(names) == 0 { + return runnableInventory{}, fmt.Errorf("package %s has no top-level tests, fuzz targets, or examples", listed.ImportPath) + } + return runnableInventory{ + packageName: listed.ImportPath, + dir: listed.Dir, + testFiles: testFiles, + names: names, + }, nil +} + +func validateExactAssignment(manifest shardManifest, inventory map[string]struct{}) error { + names := make([]string, 0, len(inventory)) + for name := range inventory { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + if _, assigned := manifest.assignment[name]; !assigned { + return fmt.Errorf("current runnable %s is unassigned in the shard manifest", name) + } + } + + names = names[:0] + for name := range manifest.assignment { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + if _, current := inventory[name]; !current { + return fmt.Errorf("manifest test %s is stale: absent from the current runnable inventory", name) + } + } + return nil +} + +func rejectParallelCalls(inventory runnableInventory) error { + files := append([]string(nil), inventory.testFiles...) + sort.Strings(files) + for _, name := range files { + if filepath.Base(name) != name { + return fmt.Errorf("go list returned invalid selected test file %q", name) + } + fileSet := token.NewFileSet() + filePath := filepath.Join(inventory.dir, name) + syntax, err := parser.ParseFile(fileSet, filePath, nil, 0) + if err != nil { + return fmt.Errorf("parse selected test file %s: %w", filePath, err) + } + for _, declaration := range syntax.Decls { + function, ok := declaration.(*ast.FuncDecl) + if !ok || function.Body == nil { + continue + } + var parallel *ast.CallExpr + ast.Inspect(function.Body, func(node ast.Node) bool { + if parallel != nil { + return false + } + call, ok := node.(*ast.CallExpr) + if !ok { + return true + } + selector, ok := call.Fun.(*ast.SelectorExpr) + if ok && selector.Sel.Name == "Parallel" { + parallel = call + return false + } + return true + }) + if parallel != nil { + position := fileSet.Position(parallel.Pos()) + return fmt.Errorf("selected test file %s:%d calls .Parallel() in %s; explicit safety review is required before exact UI package tests run in parallel", name, position.Line, function.Name.Name) + } + } + } + return nil +} + +func goCommandOutput(args ...string) ([]byte, error) { + command := exec.Command("go", args...) + var stderr bytes.Buffer + command.Stderr = &stderr + output, err := command.Output() + if err == nil { + return output, nil + } + detail := strings.TrimSpace(stderr.String()) + if detail == "" { + return nil, fmt.Errorf("go %s: %w", strings.Join(args, " "), err) + } + return nil, fmt.Errorf("go %s: %w: %s", strings.Join(args, " "), err, detail) +} + +func loadComparableStreams(paths []string) ([]streamSummary, error) { + summaries := make([]streamSummary, 0, len(paths)) + for _, singlePath := range paths { + summary, err := readEventStream(singlePath) + if err != nil { + return nil, err + } + summaries = append(summaries, summary) + } + if err := validateComparable(summaries); err != nil { + return nil, err + } + return summaries, nil +} + +func validateComparable(summaries []streamSummary) error { + packageRuns := make(map[string]int) + testRuns := make(map[testKey]int) + for runIndex, summary := range summaries { + for name := range summary.packages { + if _, exists := packageRuns[name]; !exists { + packageRuns[name] = runIndex + } + } + for name := range summary.tests { + if _, exists := testRuns[name]; !exists { + testRuns[name] = runIndex + } + } + } + + packageNames := make([]string, 0, len(packageRuns)) + for name := range packageRuns { + packageNames = append(packageNames, name) + } + sort.Strings(packageNames) + for _, name := range packageNames { + for runIndex, summary := range summaries { + if _, exists := summary.packages[name]; !exists { + return fmt.Errorf("run %d: missing package %s observed in run %d", runIndex+1, name, packageRuns[name]+1) + } + } + } + + testNames := make([]testKey, 0, len(testRuns)) + for name := range testRuns { + testNames = append(testNames, name) + } + sortTestKeys(testNames) + for _, name := range testNames { + for runIndex, summary := range summaries { + if _, exists := summary.tests[name]; !exists { + return fmt.Errorf("run %d: missing top-level test %s/%s observed in run %d", runIndex+1, name.packageName, name.name, testRuns[name]+1) + } + } + } + return nil +} + +func readEventStream(path string) (streamSummary, error) { + file, err := os.Open(path) + if err != nil { + return streamSummary{}, fmt.Errorf("read %s: %w", path, err) + } + defer func() { _ = file.Close() }() + + summary := streamSummary{ + packages: make(map[string]terminal), + tests: make(map[testKey]terminal), + } + packageStarts := make(map[string]struct{}) + testRuns := make(map[testKey]struct{}) + testTerminals := make(map[testKey]struct{}) + decoder := json.NewDecoder(file) + eventCount := 0 + for { + var event goEvent + if err := decoder.Decode(&event); err != nil { + if errors.Is(err, io.EOF) { + break + } + return streamSummary{}, fmt.Errorf("parse %s at byte %d: %w", path, decoder.InputOffset(), err) + } + eventCount++ + if event.Action == "start" && event.Test == "" { + packageStarts[event.Package] = struct{}{} + continue + } + if event.Action == "run" && event.Test != "" { + testRuns[testKey{packageName: event.Package, name: event.Test}] = struct{}{} + continue + } + if !isTerminal(event.Action) { + continue + } + if event.Elapsed == nil { + if event.Test == "" { + return streamSummary{}, fmt.Errorf("%s: terminal %s for package %s is missing Elapsed", path, event.Action, event.Package) + } + return streamSummary{}, fmt.Errorf("%s: terminal %s for %s/%s is missing Elapsed", path, event.Action, event.Package, event.Test) + } + elapsed, err := elapsedDuration(*event.Elapsed) + if err != nil { + return streamSummary{}, fmt.Errorf("%s: invalid Elapsed for %s/%s: %w", path, event.Package, event.Test, err) + } + result := terminal{action: event.Action, elapsed: elapsed} + if event.Test == "" { + if _, exists := packageStarts[event.Package]; !exists { + return streamSummary{}, fmt.Errorf("%s: terminal outcome for package %s has no start event", path, event.Package) + } + if _, exists := summary.packages[event.Package]; exists { + return streamSummary{}, fmt.Errorf("%s: duplicate terminal outcome for package %s", path, event.Package) + } + summary.packages[event.Package] = result + continue + } + key := testKey{packageName: event.Package, name: event.Test} + if _, exists := testRuns[key]; !exists { + return streamSummary{}, fmt.Errorf("%s: terminal outcome for %s/%s has no run event", path, event.Package, event.Test) + } + if _, exists := testTerminals[key]; exists { + return streamSummary{}, fmt.Errorf("%s: duplicate terminal outcome for %s/%s", path, event.Package, event.Test) + } + testTerminals[key] = struct{}{} + if isTopLevelRunnable(event.Test) { + summary.tests[key] = result + } + } + if eventCount == 0 { + return streamSummary{}, fmt.Errorf("%s: contains no Go test events", path) + } + if len(packageStarts) == 0 { + return streamSummary{}, fmt.Errorf("%s: contains no package start events", path) + } + + packageNames := make([]string, 0, len(packageStarts)) + for name := range packageStarts { + packageNames = append(packageNames, name) + } + sort.Strings(packageNames) + for _, name := range packageNames { + if _, exists := summary.packages[name]; !exists { + return streamSummary{}, fmt.Errorf("%s: missing terminal outcome for package %s", path, name) + } + } + + testNames := make([]testKey, 0, len(testRuns)) + for name := range testRuns { + testNames = append(testNames, name) + } + sortTestKeys(testNames) + for _, name := range testNames { + if _, exists := testTerminals[name]; !exists { + return streamSummary{}, fmt.Errorf("%s: missing terminal outcome for %s/%s", path, name.packageName, name.name) + } + } + return summary, nil +} + +func isTerminal(action string) bool { + return action == "pass" || action == "fail" || action == "skip" +} + +func isTopLevelRunnable(name string) bool { + if strings.Contains(name, "/") { + return false + } + return strings.HasPrefix(name, "Test") || strings.HasPrefix(name, "Fuzz") || strings.HasPrefix(name, "Example") +} + +func writeSummary(out io.Writer, summaries []streamSummary) error { + if _, err := fmt.Fprintf(out, "summary\truns=%d\n", len(summaries)); err != nil { + return err + } + + packageNames := make([]string, 0, len(summaries[0].packages)) + for name := range summaries[0].packages { + packageNames = append(packageNames, name) + } + sort.Strings(packageNames) + for _, name := range packageNames { + values := make([]terminal, len(summaries)) + for index, summary := range summaries { + values[index] = summary.packages[name] + } + if err := writeResultRow(out, []string{"package", name}, values); err != nil { + return err + } + } + + testNames := make([]testKey, 0, len(summaries[0].tests)) + for name := range summaries[0].tests { + testNames = append(testNames, name) + } + sortTestKeys(testNames) + for _, name := range testNames { + values := make([]terminal, len(summaries)) + for index, summary := range summaries { + values[index] = summary.tests[name] + } + if err := writeResultRow(out, []string{"test", name.packageName, name.name}, values); err != nil { + return err + } + } + return nil +} + +type weightedTest struct { + name string + weight time.Duration +} + +type shardPlan struct { + name string + load time.Duration + tests []string +} + +const ( + manifestMagic = "# PicFetch test shard manifest v1" + expectedShardCount = 3 + manifestPackageLine = "# package: " + manifestShardsLine = "# shards: " +) + +type shardManifest struct { + packageName string + assignment map[string]string + assignmentRow map[string]int + namesByShard map[string][]string +} + +func readShardManifest(manifestPath string) (shardManifest, error) { + file, err := os.Open(manifestPath) + if err != nil { + return shardManifest{}, fmt.Errorf("read manifest %s: %w", manifestPath, err) + } + defer func() { _ = file.Close() }() + + manifest := shardManifest{ + assignment: make(map[string]string), + assignmentRow: make(map[string]int), + namesByShard: make(map[string][]string), + } + packageRow := 0 + shardsRow := 0 + shardCount := 0 + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 64*1024), 1024*1024) + row := 0 + for scanner.Scan() { + row++ + line := scanner.Text() + if row == 1 { + if line != manifestMagic { + return shardManifest{}, fmt.Errorf("manifest %s row 1 is malformed: want %q", manifestPath, manifestMagic) + } + continue + } + if line == "" { + continue + } + if strings.HasPrefix(line, "#") { + switch { + case strings.HasPrefix(line, manifestPackageLine): + if packageRow != 0 { + return shardManifest{}, fmt.Errorf("manifest %s row %d duplicates package header from row %d", manifestPath, row, packageRow) + } + manifest.packageName = strings.TrimPrefix(line, manifestPackageLine) + if manifest.packageName == "" || strings.TrimSpace(manifest.packageName) != manifest.packageName { + return shardManifest{}, fmt.Errorf("manifest %s row %d has malformed package header", manifestPath, row) + } + packageRow = row + case strings.HasPrefix(line, manifestShardsLine): + if shardsRow != 0 { + return shardManifest{}, fmt.Errorf("manifest %s row %d duplicates shard-count header from row %d", manifestPath, row, shardsRow) + } + value := strings.TrimPrefix(line, manifestShardsLine) + shardCount, err = strconv.Atoi(value) + if err != nil || shardCount != expectedShardCount { + return shardManifest{}, fmt.Errorf("manifest %s row %d has malformed shard count %q: want %d", manifestPath, row, value, expectedShardCount) + } + for index := 1; index <= shardCount; index++ { + manifest.namesByShard[fmt.Sprintf("ui-%d", index)] = nil + } + shardsRow = row + } + continue + } + + fields := strings.Split(line, "\t") + if len(fields) != 2 || fields[0] == "" || fields[1] == "" || strings.TrimSpace(fields[0]) != fields[0] || strings.TrimSpace(fields[1]) != fields[1] { + return shardManifest{}, fmt.Errorf("manifest %s row %d is malformed: want test-nameshard", manifestPath, row) + } + if shardsRow == 0 { + return shardManifest{}, fmt.Errorf("manifest %s row %d is malformed: assignment precedes shard-count header", manifestPath, row) + } + name, shardName := fields[0], fields[1] + if name == "TestMain" || !isTopLevelRunnable(name) { + return shardManifest{}, fmt.Errorf("manifest %s row %d has malformed top-level runnable name %q", manifestPath, row, name) + } + if firstRow, exists := manifest.assignmentRow[name]; exists { + return shardManifest{}, fmt.Errorf("manifest %s row %d duplicates test %s assigned at row %d", manifestPath, row, name, firstRow) + } + if _, exists := manifest.namesByShard[shardName]; !exists { + return shardManifest{}, fmt.Errorf("manifest %s row %d assigns test %s to unknown shard %s", manifestPath, row, name, shardName) + } + manifest.assignment[name] = shardName + manifest.assignmentRow[name] = row + manifest.namesByShard[shardName] = append(manifest.namesByShard[shardName], name) + } + if err := scanner.Err(); err != nil { + return shardManifest{}, fmt.Errorf("read manifest %s: %w", manifestPath, err) + } + if row == 0 { + return shardManifest{}, fmt.Errorf("manifest %s is empty", manifestPath) + } + if packageRow == 0 { + return shardManifest{}, fmt.Errorf("manifest %s is missing package header", manifestPath) + } + if shardsRow == 0 { + return shardManifest{}, fmt.Errorf("manifest %s is missing shard-count header", manifestPath) + } + for index := 1; index <= shardCount; index++ { + shardName := fmt.Sprintf("ui-%d", index) + if len(manifest.namesByShard[shardName]) == 0 { + return shardManifest{}, fmt.Errorf("manifest shard %s is empty", shardName) + } + } + return manifest, nil +} + +func writePlan(out io.Writer, summaries []streamSummary, packageName string, shardCount int, baselineSHA, baselineRun string) error { + resolvedPackage, err := resolvePackage(packageName, summaries[0]) + if err != nil { + return err + } + packageName = resolvedPackage + + weighted := make([]weightedTest, 0) + for key := range summaries[0].tests { + if key.packageName != packageName { + continue + } + durations := make([]time.Duration, len(summaries)) + for index, summary := range summaries { + result := summary.tests[key] + if result.action != "pass" { + return fmt.Errorf("run %d: %s/%s ended with %s; planning requires pass", index+1, key.packageName, key.name, result.action) + } + durations[index] = result.elapsed + } + weighted = append(weighted, weightedTest{name: key.name, weight: medianDuration(durations)}) + } + if len(weighted) == 0 { + return fmt.Errorf("package %s has no terminal top-level tests, fuzz targets, or examples", packageName) + } + sort.Slice(weighted, func(i, j int) bool { + if weighted[i].weight != weighted[j].weight { + return weighted[i].weight > weighted[j].weight + } + return weighted[i].name < weighted[j].name + }) + + shards := make([]shardPlan, shardCount) + for index := range shards { + shards[index].name = fmt.Sprintf("ui-%d", index+1) + } + for _, test := range weighted { + target := 0 + for index := 1; index < len(shards); index++ { + if shards[index].load < shards[target].load { + target = index + } + } + shards[target].tests = append(shards[target].tests, test.name) + shards[target].load += test.weight + } + + header := []string{ + "# PicFetch test shard manifest v1", + "# package: " + packageName, + fmt.Sprintf("# shards: %d", shardCount), + "# baseline-sha: " + baselineSHA, + "# baseline-run: " + baselineRun, + "# baseline-attempts: 1,2,3", + } + for _, line := range header { + if _, err := fmt.Fprintln(out, line); err != nil { + return err + } + } + for index := range shards { + sort.Strings(shards[index].tests) + if _, err := fmt.Fprintf(out, "# %s: %d entries, %.3fs median-weight sum\n", shards[index].name, len(shards[index].tests), durationSeconds(shards[index].load)); err != nil { + return err + } + } + if _, err := fmt.Fprintln(out, "# name\tshard"); err != nil { + return err + } + for _, shard := range shards { + for _, name := range shard.tests { + if _, err := fmt.Fprintf(out, "%s\t%s\n", name, shard.name); err != nil { + return err + } + } + } + return nil +} + +func resolvePackage(requested string, summary streamSummary) (string, error) { + candidates := make(map[string]struct{}) + for key := range summary.tests { + if key.packageName == requested { + return requested, nil + } + candidates[key.packageName] = struct{}{} + } + if !strings.HasPrefix(requested, "./") { + return "", fmt.Errorf("package %s is absent from the event inventory", requested) + } + + relative := strings.TrimPrefix(path.Clean(requested), "./") + if relative == "." || relative == "" || strings.Contains(relative, "...") { + return "", fmt.Errorf("package %s must name one exact package", requested) + } + matches := make([]string, 0) + for candidate := range candidates { + if candidate == relative || strings.HasSuffix(candidate, "/"+relative) { + matches = append(matches, candidate) + } + } + sort.Strings(matches) + if len(matches) == 0 { + return "", fmt.Errorf("package %s is absent from the event inventory", requested) + } + if len(matches) > 1 { + return "", fmt.Errorf("package %s is ambiguous in the event inventory: %s", requested, strings.Join(matches, ", ")) + } + return matches[0], nil +} + +func sortTestKeys(names []testKey) { + sort.Slice(names, func(i, j int) bool { + if names[i].packageName != names[j].packageName { + return names[i].packageName < names[j].packageName + } + return names[i].name < names[j].name + }) +} + +func writeResultRow(out io.Writer, prefix []string, values []terminal) error { + fields := append([]string(nil), prefix...) + durations := make([]time.Duration, len(values)) + for index, value := range values { + fields = append(fields, fmt.Sprintf("run-%d=%s:%.3f", index+1, value.action, durationSeconds(value.elapsed))) + durations[index] = value.elapsed + } + fields = append(fields, fmt.Sprintf("median=%.3f", durationSeconds(medianDuration(durations)))) + _, err := fmt.Fprintln(out, strings.Join(fields, "\t")) + return err +} + +func elapsedDuration(seconds float64) (time.Duration, error) { + if math.IsNaN(seconds) || math.IsInf(seconds, 0) || seconds < 0 { + return 0, fmt.Errorf("must be a finite non-negative number, got %g", seconds) + } + if seconds > float64(math.MaxInt64/int64(time.Second)) { + return 0, fmt.Errorf("duration %g seconds is too large", seconds) + } + return time.Duration(math.Round(seconds * float64(time.Second))), nil +} + +func durationSeconds(duration time.Duration) float64 { + return float64(duration) / float64(time.Second) +} + +func medianDuration(values []time.Duration) time.Duration { + sorted := append([]time.Duration(nil), values...) + slices.Sort(sorted) + middle := len(sorted) / 2 + if len(sorted)%2 == 1 { + return sorted[middle] + } + return sorted[middle-1] + (sorted[middle]-sorted[middle-1])/2 +} diff --git a/scripts/testshards/main_test.go b/scripts/testshards/main_test.go new file mode 100644 index 0000000..2cf447c --- /dev/null +++ b/scripts/testshards/main_test.go @@ -0,0 +1,977 @@ +package main + +import ( + "bytes" + "fmt" + "os" + "os/exec" + "path/filepath" + "regexp" + "runtime" + "strings" + "testing" +) + +func TestSummarize_ReportsInterleavedPackagesAndTopLevelTests(t *testing.T) { + path := writeEventStream(t, ` +{"Action":"start","Package":"example/b"} +{"Action":"start","Package":"example/a"} +{"Action":"run","Package":"example/a","Test":"TestParent"} +{"Action":"run","Package":"example/a","Test":"TestParent/child"} +{"Action":"pass","Package":"example/a","Test":"TestParent/child","Elapsed":0.5} +{"Action":"pass","Package":"example/a","Test":"TestParent","Elapsed":1.5} +{"Action":"skip","Package":"example/b","Elapsed":0} +{"Action":"pass","Package":"example/a","Elapsed":2} +`) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", path}, &stdout, &stderr) + if err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + + want := strings.TrimSpace(` +summary runs=1 +package example/a run-1=pass:2.000 median=2.000 +package example/b run-1=skip:0.000 median=0.000 +test example/a TestParent run-1=pass:1.500 median=1.500 +`) + "\n" + if stdout.String() != want { + t.Fatalf("stdout:\n%s\nwant:\n%s", stdout.String(), want) + } +} + +func TestSummarize_RejectsDuplicateTerminalOutcome(t *testing.T) { + path := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestRepeated"} +{"Action":"pass","Package":"example/ui","Test":"TestRepeated","Elapsed":1} +{"Action":"fail","Package":"example/ui","Test":"TestRepeated","Elapsed":2} +{"Action":"fail","Package":"example/ui","Elapsed":3} +`) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", path}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "duplicate terminal outcome for example/ui/TestRepeated") { + t.Fatalf("got %v, want duplicate TestRepeated diagnostic", err) + } +} + +func TestSummarize_RejectsMissingTerminalOutcome(t *testing.T) { + tests := []struct { + name string + events string + wantText string + }{ + { + name: "test", + events: ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestNeverFinished"} +{"Action":"fail","Package":"example/ui","Elapsed":1} +`, + wantText: "missing terminal outcome for example/ui/TestNeverFinished", + }, + { + name: "package", + events: ` +{"Action":"start","Package":"example/unfinished"} +`, + wantText: "missing terminal outcome for package example/unfinished", + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + path := writeEventStream(t, test.events) + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", path}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), test.wantText) { + t.Fatalf("got %v, want %q", err, test.wantText) + } + }) + } +} + +func TestSummarize_RejectsTopLevelTestAbsentFromRequiredRun(t *testing.T) { + first := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestPresent"} +{"Action":"pass","Package":"example/ui","Test":"TestPresent","Elapsed":1} +{"Action":"run","Package":"example/ui","Test":"TestMissing"} +{"Action":"pass","Package":"example/ui","Test":"TestMissing","Elapsed":2} +{"Action":"pass","Package":"example/ui","Elapsed":3} +`) + second := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestPresent"} +{"Action":"pass","Package":"example/ui","Test":"TestPresent","Elapsed":1.5} +{"Action":"pass","Package":"example/ui","Elapsed":2} +`) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", first, "-json", second}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "run 2") || !strings.Contains(err.Error(), "missing top-level test example/ui/TestMissing") { + t.Fatalf("got %v, want run 2 missing TestMissing diagnostic", err) + } +} + +func TestSummarize_RejectsMalformedTruncatedAndEmptyInput(t *testing.T) { + tests := []struct { + name string + contents string + wantText string + }{ + {name: "malformed", contents: `{not-json}`, wantText: "parse"}, + {name: "truncated", contents: `{"Action":"start"`, wantText: "unexpected EOF"}, + {name: "empty", contents: "", wantText: "contains no Go test events"}, + {name: "not test events", contents: `{"Time":"2026-09-03T00:00:00Z"}`, wantText: "contains no package start events"}, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + path := writeEventStream(t, test.contents) + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", path}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), test.wantText) { + t.Fatalf("got %v, want diagnostic containing %q", err, test.wantText) + } + }) + } +} + +func TestSummarize_RejectsTerminalWithoutBeginning(t *testing.T) { + tests := []struct { + name string + events string + wantText string + }{ + { + name: "package", + events: `{"Action":"pass","Package":"example/orphan","Elapsed":1}`, + wantText: "terminal outcome for package example/orphan has no start event", + }, + { + name: "test", + events: ` +{"Action":"start","Package":"example/ui"} +{"Action":"pass","Package":"example/ui","Test":"TestOrphan","Elapsed":1} +{"Action":"pass","Package":"example/ui","Elapsed":2} +`, + wantText: "terminal outcome for example/ui/TestOrphan has no run event", + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + path := writeEventStream(t, test.events) + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", path}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), test.wantText) { + t.Fatalf("got %v, want %q", err, test.wantText) + } + }) + } +} + +func TestSummarize_RejectsTerminalWithoutElapsed(t *testing.T) { + path := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestNoDuration"} +{"Action":"pass","Package":"example/ui","Test":"TestNoDuration"} +{"Action":"fail","Package":"example/ui","Elapsed":1} +`) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"summarize", "-json", path}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "terminal pass for example/ui/TestNoDuration is missing Elapsed") { + t.Fatalf("got %v, want missing TestNoDuration elapsed diagnostic", err) + } +} + +func TestSummarize_UsesMedianAndPreservesTerminalOutcomes(t *testing.T) { + paths := []string{ + writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestOutcome"} +{"Action":"pass","Package":"example/ui","Test":"TestOutcome","Elapsed":9} +{"Action":"pass","Package":"example/ui","Elapsed":10} +`), + writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestOutcome"} +{"Action":"skip","Package":"example/ui","Test":"TestOutcome","Elapsed":1} +{"Action":"skip","Package":"example/ui","Elapsed":2} +`), + writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestOutcome"} +{"Action":"fail","Package":"example/ui","Test":"TestOutcome","Elapsed":5} +{"Action":"fail","Package":"example/ui","Elapsed":6} +`), + } + args := []string{"summarize"} + for _, path := range paths { + args = append(args, "-json", path) + } + + var first bytes.Buffer + var stderr bytes.Buffer + if err := run(args, &first, &stderr); err != nil { + t.Fatalf("first run: %v\nstderr: %s", err, stderr.String()) + } + var second bytes.Buffer + stderr.Reset() + if err := run(args, &second, &stderr); err != nil { + t.Fatalf("second run: %v\nstderr: %s", err, stderr.String()) + } + if first.String() != second.String() { + t.Fatalf("identical inputs produced different output:\nfirst:\n%s\nsecond:\n%s", first.String(), second.String()) + } + + want := strings.TrimSpace(` +summary runs=3 +package example/ui run-1=pass:10.000 run-2=skip:2.000 run-3=fail:6.000 median=6.000 +test example/ui TestOutcome run-1=pass:9.000 run-2=skip:1.000 run-3=fail:5.000 median=5.000 +`) + "\n" + if first.String() != want { + t.Fatalf("stdout:\n%s\nwant:\n%s", first.String(), want) + } +} + +func TestPlan_UsesMedianLPTAndDeterministicTieBreaks(t *testing.T) { + paths := []string{ + writePlanningStream(t, map[string]float64{ + "ExampleA": 30, "FuzzB": 3, "TestC": 3, + "ExampleD": 10, "FuzzE": 1, "TestF": 1, + }), + writePlanningStream(t, map[string]float64{ + "ExampleA": 3, "FuzzB": 30, "TestC": 3, + "ExampleD": 1, "FuzzE": 10, "TestF": 1, + }), + writePlanningStream(t, map[string]float64{ + "ExampleA": 3, "FuzzB": 3, "TestC": 30, + "ExampleD": 1, "FuzzE": 1, "TestF": 10, + }), + } + args := []string{ + "plan", + "-package", "example/ui", + "-shards", "3", + "-baseline-sha", "abc123", + "-baseline-run", "42", + } + for _, path := range paths { + args = append(args, "-json", path) + } + + var first bytes.Buffer + var stderr bytes.Buffer + if err := run(args, &first, &stderr); err != nil { + t.Fatalf("first run: %v\nstderr: %s", err, stderr.String()) + } + var second bytes.Buffer + stderr.Reset() + if err := run(args, &second, &stderr); err != nil { + t.Fatalf("second run: %v\nstderr: %s", err, stderr.String()) + } + if first.String() != second.String() { + t.Fatalf("identical inputs produced different plans:\nfirst:\n%s\nsecond:\n%s", first.String(), second.String()) + } + + want := strings.TrimSpace(` +# PicFetch test shard manifest v1 +# package: example/ui +# shards: 3 +# baseline-sha: abc123 +# baseline-run: 42 +# baseline-attempts: 1,2,3 +# ui-1: 2 entries, 4.000s median-weight sum +# ui-2: 2 entries, 4.000s median-weight sum +# ui-3: 2 entries, 4.000s median-weight sum +# name shard +ExampleA ui-1 +ExampleD ui-1 +FuzzB ui-2 +FuzzE ui-2 +TestC ui-3 +TestF ui-3 +`) + "\n" + if first.String() != want { + t.Fatalf("stdout:\n%s\nwant:\n%s", first.String(), want) + } +} + +func TestPlan_ResolvesRelativePackageFromEventInventory(t *testing.T) { + path := writeEventStream(t, ` +{"Action":"start","Package":"example.com/project/internal/ui"} +{"Action":"run","Package":"example.com/project/internal/ui","Test":"TestOnly"} +{"Action":"pass","Package":"example.com/project/internal/ui","Test":"TestOnly","Elapsed":1} +{"Action":"pass","Package":"example.com/project/internal/ui","Elapsed":2} +`) + args := []string{ + "plan", + "-package", "./internal/ui", + "-shards", "1", + "-baseline-sha", "abc123", + "-baseline-run", "42", + "-json", path, + "-json", path, + "-json", path, + } + + var stdout bytes.Buffer + var stderr bytes.Buffer + if err := run(args, &stdout, &stderr); err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + if !strings.Contains(stdout.String(), "# package: example.com/project/internal/ui\n") { + t.Fatalf("manifest did not contain the resolved import path:\n%s", stdout.String()) + } + if !strings.Contains(stdout.String(), "TestOnly\tui-1\n") { + t.Fatalf("manifest did not assign TestOnly:\n%s", stdout.String()) + } +} + +func TestPlan_RejectsNonPassingBaselineOutcome(t *testing.T) { + passing := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestNeedsPass"} +{"Action":"pass","Package":"example/ui","Test":"TestNeedsPass","Elapsed":1} +{"Action":"pass","Package":"example/ui","Elapsed":2} +`) + skipped := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestNeedsPass"} +{"Action":"skip","Package":"example/ui","Test":"TestNeedsPass","Elapsed":0} +{"Action":"pass","Package":"example/ui","Elapsed":1} +`) + args := []string{ + "plan", + "-package", "example/ui", + "-shards", "1", + "-baseline-sha", "abc123", + "-baseline-run", "42", + "-json", passing, + "-json", skipped, + "-json", passing, + } + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run(args, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "run 2") || !strings.Contains(err.Error(), "TestNeedsPass ended with skip") { + t.Fatalf("got %v, want run 2 skipped TestNeedsPass diagnostic", err) + } +} + +func TestPlan_EqualDecimalLoadsPreferLowerShard(t *testing.T) { + path := writeEventStream(t, ` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestA"} +{"Action":"pass","Package":"example/ui","Test":"TestA","Elapsed":0.8} +{"Action":"run","Package":"example/ui","Test":"TestB"} +{"Action":"pass","Package":"example/ui","Test":"TestB","Elapsed":0.7} +{"Action":"run","Package":"example/ui","Test":"TestC"} +{"Action":"pass","Package":"example/ui","Test":"TestC","Elapsed":0.1} +{"Action":"run","Package":"example/ui","Test":"TestD"} +{"Action":"pass","Package":"example/ui","Test":"TestD","Elapsed":0.05} +{"Action":"pass","Package":"example/ui","Elapsed":2} +`) + args := []string{ + "plan", + "-package", "example/ui", + "-shards", "2", + "-baseline-sha", "abc123", + "-baseline-run", "42", + "-json", path, + "-json", path, + "-json", path, + } + + var stdout bytes.Buffer + var stderr bytes.Buffer + if err := run(args, &stdout, &stderr); err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + for _, row := range []string{"TestA\tui-1\n", "TestD\tui-1\n", "TestB\tui-2\n", "TestC\tui-2\n"} { + if !strings.Contains(stdout.String(), row) { + t.Fatalf("manifest missing equal-load tie assignment %q:\n%s", strings.TrimSpace(row), stdout.String()) + } + } +} + +func TestRegex_ProducesExactGoCompatibleFilter(t *testing.T) { + manifest := writeManifest(t, ` +TestA+B ui-1 +TestDollar$ ui-1 +FuzzDot. ui-2 +ExampleBrackets[1] ui-3 +`) + tests := []struct { + shard string + want string + matches []string + nonMatches []string + }{ + { + shard: "ui-1", + want: `^(TestA\+B|TestDollar\$)$`, + matches: []string{"TestA+B", "TestDollar$"}, + nonMatches: []string{"FuzzDot.", "ExampleBrackets[1]", "TestA+B/child", "prefixTestA+B"}, + }, + { + shard: "ui-2", + want: `^(FuzzDot\.)$`, + matches: []string{"FuzzDot."}, + nonMatches: []string{"TestA+B", "FuzzDotX"}, + }, + { + shard: "ui-3", + want: `^(ExampleBrackets\[1\])$`, + matches: []string{"ExampleBrackets[1]"}, + nonMatches: []string{"TestDollar$", "ExampleBrackets1"}, + }, + } + + for _, test := range tests { + t.Run(test.shard, func(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"regex", "-manifest", manifest, "-shard", test.shard}, &stdout, &stderr) + if err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + + got := strings.TrimSpace(stdout.String()) + if got != test.want { + t.Fatalf("filter = %q, want %q", got, test.want) + } + compiled, err := regexp.Compile(got) + if err != nil { + t.Fatalf("compile filter %q: %v", got, err) + } + for _, name := range test.matches { + if !compiled.MatchString(name) { + t.Errorf("filter %q does not match assigned name %q", got, name) + } + } + for _, name := range test.nonMatches { + if compiled.MatchString(name) { + t.Errorf("filter %q unexpectedly matches %q", got, name) + } + } + }) + } +} + +func TestRegex_ValidatesWholeManifestBeforeSelectingShard(t *testing.T) { + manifest := writeManifest(t, ` +TestOwned ui-1 +TestOther ui-2 +this row is malformed +TestThird ui-3 +`) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"regex", "-manifest", manifest, "-shard", "ui-1"}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "row 9") || !strings.Contains(err.Error(), "malformed") { + t.Fatalf("got %v, want malformed row 9 diagnostic", err) + } + if stdout.Len() != 0 { + t.Fatalf("invalid manifest emitted filter %q", stdout.String()) + } +} + +func TestCheck_UsesBuildSelectedTopLevelInventory(t *testing.T) { + prepareCheckFixture(t, false) + manifest := writeManifest(t, validCheckRows) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"check", "-package", "./ui", "-manifest", manifest}, &stdout, &stderr) + if runtime.GOOS == "linux" && runtime.GOARCH == "amd64" { + if err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + want := "checked\tpackage=example.com/project/ui\trunnables=5\tshards=3\n" + if stdout.String() != want { + t.Fatalf("stdout = %q, want %q", stdout.String(), want) + } + return + } + + if err == nil || !strings.Contains(err.Error(), "canonical shard validation requires linux/amd64") { + t.Fatalf("got %v, want non-linux/amd64 canonical-validation refusal", err) + } + if strings.Contains(err.Error(), "TestMain") || strings.Contains(err.Error(), "TestParent/child") || strings.Contains(err.Error(), "TestIgnored") { + t.Fatalf("unselected harness or subtest escaped into inventory: %v", err) + } +} + +func TestCheckRejectsInvalidAssignments(t *testing.T) { + tests := []struct { + name string + rows string + wantText []string + }{ + { + name: "missing", + rows: ` +TestAlpha ui-1 +TestPlatform ui-2 +FuzzBeta ui-3 +ExampleGamma ui-3 +`, + wantText: []string{"TestParent", "unassigned"}, + }, + { + name: "duplicate", + rows: validCheckRows + "\nTestAlpha\tui-2", + wantText: []string{"TestAlpha", "row", "duplicates"}, + }, + { + name: "stale", + rows: validCheckRows + "\nTestStale\tui-1", + wantText: []string{"TestStale", "stale"}, + }, + { + name: "malformed", + rows: validCheckRows + "\nTestBroken ui-1", + wantText: []string{"row", "malformed"}, + }, + { + name: "unknown shard", + rows: ` +TestAlpha ui-4 +TestParent ui-2 +TestPlatform ui-2 +FuzzBeta ui-3 +ExampleGamma ui-3 +`, + wantText: []string{"TestAlpha", "ui-4", "unknown shard"}, + }, + { + name: "empty shard", + rows: ` +TestAlpha ui-1 +TestParent ui-2 +TestPlatform ui-2 +FuzzBeta ui-2 +ExampleGamma ui-2 +`, + wantText: []string{"ui-3", "empty"}, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + prepareCheckFixture(t, false) + manifest := writeManifest(t, test.rows) + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"check", "-package", "./ui", "-manifest", manifest}, &stdout, &stderr) + if err == nil { + t.Fatal("invalid assignment was accepted") + } + for _, want := range test.wantText { + if !strings.Contains(err.Error(), want) { + t.Fatalf("got %v, want diagnostic containing %q", err, want) + } + } + if stdout.Len() != 0 { + t.Fatalf("invalid assignment emitted success output %q", stdout.String()) + } + }) + } +} + +func TestCheckRejectsDarwinDerivedInventoryAsCanonical(t *testing.T) { + if runtime.GOOS != "darwin" { + t.Skip("Darwin-specific canonical-context guard") + } + prepareCheckFixture(t, false) + manifest := writeManifest(t, validCheckRows) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"check", "-package", "./ui", "-manifest", manifest}, &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "canonical shard validation requires linux/amd64") || !strings.Contains(err.Error(), "darwin/") { + t.Fatalf("got %v, want explicit Darwin canonical-validation refusal", err) + } +} + +func TestParallelGuardRejectsSelectedParallelCall(t *testing.T) { + prepareCheckFixture(t, true) + manifest := writeManifest(t, validCheckRows) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := run([]string{"check", "-package", "./ui", "-manifest", manifest}, &stdout, &stderr) + if err == nil { + t.Fatal("selected parallel test call was accepted") + } + for _, want := range []string{"TestAlpha", "common_test.go", ".Parallel()", "explicit safety review"} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("got %v, want diagnostic containing %q", err, want) + } + } +} + +func TestCapture_PreservesRawFailedStreamAndReportsPartitionPackageAndTest(t *testing.T) { + stream := strings.TrimLeft(` +{"Action":"start","Package":"example/ui"} +{"Action":"run","Package":"example/ui","Test":"TestBroken"} +{"Action":"output","Package":"example/ui","Test":"TestBroken","Output":"main_test.go:12: deliberate failure\n"} +{"Action":"fail","Package":"example/ui","Test":"TestBroken","Elapsed":0.25} +{"Action":"fail","Package":"example/ui","Elapsed":0.5} +`, "\n") + rawPath := filepath.Join(t.TempDir(), "raw.json") + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := runWithInput( + []string{"capture", "-out", rawPath, "-partition", "ui-2"}, + strings.NewReader(stream), + &stdout, + &stderr, + ) + if err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + + raw, err := os.ReadFile(rawPath) + if err != nil { + t.Fatal(err) + } + if string(raw) != stream { + t.Fatalf("raw stream:\n%s\nwant:\n%s", raw, stream) + } + want := strings.TrimSpace(` +failure partition=ui-2 package=example/ui test=TestBroken main_test.go:12: deliberate failure +test partition=ui-2 package=example/ui name=TestBroken action=fail elapsed=0.250 +package partition=ui-2 package=example/ui action=fail elapsed=0.500 +`) + "\n" + if stdout.String() != want { + t.Fatalf("stdout:\n%s\nwant:\n%s", stdout.String(), want) + } +} + +func TestCapture_RejectsCaptureFailure(t *testing.T) { + var stdout bytes.Buffer + var stderr bytes.Buffer + err := runWithInput( + []string{"capture", "-out", t.TempDir(), "-partition", "non-ui"}, + strings.NewReader("{\"Action\":\"start\",\"Package\":\"example/pkg\"}\n"), + &stdout, + &stderr, + ) + if err == nil || !strings.Contains(err.Error(), "create capture") { + t.Fatalf("got %v, want capture creation failure", err) + } + if stdout.Len() != 0 { + t.Fatalf("failed capture emitted output %q", stdout.String()) + } +} + +func TestPackagePartition_ExcludesOnlyExactMainUIPackage(t *testing.T) { + root := preparePartitionFixture(t, true) + t.Chdir(root) + + var stdout bytes.Buffer + var stderr bytes.Buffer + err := runWithInput([]string{"partition", "-package", "./internal/ui"}, strings.NewReader(""), &stdout, &stderr) + if err != nil { + t.Fatalf("run: %v\nstderr: %s", err, stderr.String()) + } + want := strings.TrimSpace(` +example.com/project +example.com/project/internal/ui/grid +example.com/project/scripts/tool +`) + "\n" + if stdout.String() != want { + t.Fatalf("stdout:\n%s\nwant:\n%s", stdout.String(), want) + } +} + +func TestPackagePartition_RejectsEmptyOrInconsistentPartition(t *testing.T) { + t.Run("empty", func(t *testing.T) { + root := preparePartitionFixture(t, false) + t.Chdir(root) + var stdout bytes.Buffer + var stderr bytes.Buffer + err := runWithInput([]string{"partition", "-package", "./internal/ui"}, strings.NewReader(""), &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "non-UI package partition is empty") { + t.Fatalf("got %v, want empty partition diagnostic", err) + } + }) + + t.Run("selected package absent from module inventory", func(t *testing.T) { + root := preparePartitionFixture(t, true) + t.Chdir(root) + var stdout bytes.Buffer + var stderr bytes.Buffer + err := runWithInput([]string{"partition", "-package", "fmt"}, strings.NewReader(""), &stdout, &stderr) + if err == nil || !strings.Contains(err.Error(), "selected package fmt is absent from the module inventory") { + t.Fatalf("got %v, want inconsistent partition diagnostic", err) + } + }) +} + +func TestMakeTestRemainsCompleteAndUnsharded(t *testing.T) { + output := makeDryRun(t, "test") + for _, want := range []string{"docker run --rm --platform linux/amd64", "go test -timeout 30m", "./..."} { + if !strings.Contains(output, want) { + t.Fatalf("make test output is missing %q:\n%s", want, output) + } + } + for _, forbidden := range []string{"testshards capture", "testshards partition", "-count=1", "-json"} { + if strings.Contains(output, forbidden) { + t.Fatalf("make test unexpectedly contains %q:\n%s", forbidden, output) + } + } +} + +func TestMakeRaceRunsCanonicalSequentialContractInOneContainer(t *testing.T) { + public := makeDryRun(t, "test-race") + if count := strings.Count(public, "docker run --rm --platform linux/amd64"); count != 1 { + t.Fatalf("make test-race starts %d Linux/amd64 containers, want 1:\n%s", count, public) + } + for _, want := range []string{"locale-gen en_US.UTF-8", "make --no-print-directory test-race-direct"} { + if !strings.Contains(public, want) { + t.Fatalf("make test-race output is missing %q:\n%s", want, public) + } + } + + direct := makeDryRun(t, "test-race-direct") + ordered := []string{ + "testshards check", + "testshards partition", + "-shard \"ui-1\"", + "-shard \"ui-2\"", + "-shard \"ui-3\"", + } + position := -1 + for _, want := range ordered { + next := strings.Index(direct[position+1:], want) + if next < 0 { + t.Fatalf("sequential race contract is missing %q after byte %d:\n%s", want, position, direct) + } + position += next + 1 + } +} + +func TestMakeDirectRacePartitionsShareFlagsLocaleAndCapture(t *testing.T) { + tests := []struct { + name string + args []string + want []string + }{ + { + name: "non-ui", + args: []string{"test-race-non-ui-direct"}, + want: []string{ + "testshards partition -package \"./internal/ui\"", + "-partition \"non-ui\"", + }, + }, + { + name: "selected UI shard", + args: []string{"test-race-ui-direct", "TEST_SHARD=ui-2"}, + want: []string{ + "testshards regex -manifest \".github/testshards/internal-ui.tsv\" -shard \"ui-2\"", + "-run \"$filter\" ./internal/ui", + "-partition \"ui-2\"", + }, + }, + } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + output := makeDryRun(t, test.args...) + shared := []string{ + "set -eu -o pipefail", + "LANG=\"en_US.UTF-8\" go test -race -count=1 -timeout 30m -json", + "testshards capture -out", + } + for _, want := range append(shared, test.want...) { + if !strings.Contains(output, want) { + t.Fatalf("direct %s output is missing %q:\n%s", test.name, want, output) + } + } + }) + } +} + +func TestMakeDirectUIRaceRejectsInvalidShardAndPrintsTheContract(t *testing.T) { + command := exec.Command( + "make", + "--no-print-directory", + "test-race-ui-direct", + "TEST_SHARD=ui-4", + ) + command.Dir = filepath.Join("..", "..") + output, err := command.CombinedOutput() + if err == nil { + t.Fatal("invalid UI shard was accepted") + } + for _, want := range []string{ + "TEST_SHARD must be one of ui-1, ui-2, or ui-3", + "LANG=\"en_US.UTF-8\" go test -race -count=1 -timeout 30m -json", + } { + if !strings.Contains(string(output), want) { + t.Fatalf("invalid-shard output is missing %q:\n%s", want, output) + } + } +} + +func writeEventStream(t *testing.T, contents string) string { + t.Helper() + path := filepath.Join(t.TempDir(), "events.json") + if err := os.WriteFile(path, []byte(strings.TrimSpace(contents)+"\n"), 0o644); err != nil { + t.Fatal(err) + } + return path +} + +func writeManifest(t *testing.T, rows string) string { + t.Helper() + path := filepath.Join(t.TempDir(), "manifest.tsv") + contents := `# PicFetch test shard manifest v1 +# package: example.com/project/ui +# shards: 3 +# baseline-sha: abc123 +# baseline-run: 42 +# name shard +` + strings.TrimSpace(rows) + "\n" + if err := os.WriteFile(path, []byte(contents), 0o644); err != nil { + t.Fatal(err) + } + return path +} + +const validCheckRows = ` +TestAlpha ui-1 +TestParent ui-2 +TestPlatform ui-2 +FuzzBeta ui-3 +ExampleGamma ui-3 +` + +func prepareCheckFixture(t *testing.T, parallel bool) { + t.Helper() + root := t.TempDir() + writeTestFile(t, filepath.Join(root, "go.mod"), "module example.com/project\n\ngo 1.27.1\n") + writeTestFile(t, filepath.Join(root, "ui", "ui.go"), "package ui\n\nfunc Gamma() {}\n") + + parallelCall := "" + if parallel { + parallelCall = "\tt.Parallel()\n" + } + commonTests := fmt.Sprintf(`package ui + +import ( + "fmt" + "os" + "testing" +) + +func TestAlpha(t *testing.T) { +%s} + +func TestParent(t *testing.T) { + t.Run("child", func(t *testing.T) {}) +} + +func FuzzBeta(f *testing.F) { + f.Add("seed") + f.Fuzz(func(t *testing.T, input string) {}) +} + +func ExampleGamma() { + fmt.Println("gamma") + // Output: gamma +} + +func TestMain(m *testing.M) { + os.Exit(m.Run()) +} +`, parallelCall) + writeTestFile(t, filepath.Join(root, "ui", "common_test.go"), commonTests) + platformTests := fmt.Sprintf("//go:build %s\n\npackage ui\n\nimport \"testing\"\n\nfunc TestPlatform(t *testing.T) {}\n", runtime.GOOS) + writeTestFile(t, filepath.Join(root, "ui", "platform_"+runtime.GOOS+"_test.go"), platformTests) + writeTestFile(t, filepath.Join(root, "ui", "ignored_test.go"), `//go:build picfetch_unselected_fixture + +package ui + +import "testing" + +func TestIgnored(t *testing.T) { + t.Parallel() +} +`) + t.Setenv("GOOS", runtime.GOOS) + t.Setenv("GOARCH", runtime.GOARCH) + t.Setenv("GOWORK", "off") + t.Chdir(root) +} + +func writeTestFile(t *testing.T, path, contents string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(contents), 0o644); err != nil { + t.Fatal(err) + } +} + +func preparePartitionFixture(t *testing.T, includeNonUI bool) string { + t.Helper() + root := t.TempDir() + writeTestFile(t, filepath.Join(root, "go.mod"), "module example.com/project\n\ngo 1.27.1\n") + writeTestFile(t, filepath.Join(root, "internal", "ui", "ui.go"), "package ui\n") + if includeNonUI { + writeTestFile(t, filepath.Join(root, "root.go"), "package project\n") + writeTestFile(t, filepath.Join(root, "internal", "ui", "grid", "grid.go"), "package grid\n") + writeTestFile(t, filepath.Join(root, "scripts", "tool", "main.go"), "package main\n") + } + t.Setenv("GOWORK", "off") + return root +} + +func makeDryRun(t *testing.T, args ...string) string { + t.Helper() + commandArgs := append([]string{"--no-print-directory", "-n"}, args...) + command := exec.Command("make", commandArgs...) + command.Dir = filepath.Join("..", "..") + output, err := command.CombinedOutput() + if err != nil { + t.Fatalf("make %s: %v\n%s", strings.Join(args, " "), err, output) + } + return string(output) +} + +func writePlanningStream(t *testing.T, durations map[string]float64) string { + t.Helper() + var stream strings.Builder + stream.WriteString("{\"Action\":\"start\",\"Package\":\"example/ui\"}\n") + order := []string{"TestF", "ExampleA", "TestC", "FuzzE", "ExampleD", "FuzzB"} + for _, name := range order { + _, _ = fmt.Fprintf(&stream, "{\"Action\":\"run\",\"Package\":\"example/ui\",\"Test\":%q}\n", name) + if name == "TestC" { + stream.WriteString("{\"Action\":\"run\",\"Package\":\"example/ui\",\"Test\":\"TestC/child\"}\n") + stream.WriteString("{\"Action\":\"pass\",\"Package\":\"example/ui\",\"Test\":\"TestC/child\",\"Elapsed\":0.25}\n") + } + _, _ = fmt.Fprintf(&stream, "{\"Action\":\"pass\",\"Package\":\"example/ui\",\"Test\":%q,\"Elapsed\":%g}\n", name, durations[name]) + } + stream.WriteString("{\"Action\":\"pass\",\"Package\":\"example/ui\",\"Elapsed\":40}\n") + return writeEventStream(t, stream.String()) +} diff --git a/todos.md b/todos.md index a427352..e046989 100644 --- a/todos.md +++ b/todos.md @@ -10,48 +10,23 @@ #### Internal +- Reduced the measured median reusable CI gate from 22:17 to 7:52 while + retaining Linux race coverage, native Windows tests, validation, and release + failure gating. The final topology can use six concurrent required runners + instead of two and raises median runner time from 23:26 to 30:43 (+31.1%). + Its exact manifest proves UI coverage and non-overlap, but no one process now + exercises the whole UI suite's ordering; see the + [completed measurement record](finished_refactorings/2026-09-03-measured-ci-test-sharding.md). + ## TODO -### Reduce CI wall-clock time through measured test sharding - -The full GitHub Actions CI currently takes roughly 21 minutes. Reduce its -wall-clock time without removing the race detector, weakening the test scope, -or hiding failures behind optional jobs. Treat this as a separate pull request -after the first signed Windows release. - -1. **Measure the current critical path.** - - Capture package and top-level test durations from a representative Linux - `-race` run, preferably as machine-readable `go test -json` output. - - Record at least three comparable `main` runs so runner variance is not - mistaken for an improvement. - - Confirm how much time is spent in setup, validation/build steps, the - non-UI packages, and `internal/ui` before choosing shard boundaries. -2. **Start independent work in parallel.** - - Split formatting, TUF-root validation, vetting, normal builds, and Windows - cross-builds from the Linux race tests so neither group waits for the - other after checkout and toolchain setup. - - Keep the existing Windows test job independent. - - Preserve every job as a required dependency of the reusable CI workflow, - so tag releases cannot proceed after any shard or validation job fails. -3. **Shard the long `internal/ui` package safely.** - - Run the remaining packages with `-race` in their own job. - - Divide `internal/ui` top-level tests into approximately three - duration-balanced jobs running on separate GitHub-hosted runners. Do not - broadly add `t.Parallel()`: Fyne and UI tests can share process-global - state, whereas separate runners provide isolation. - - Make shard assignment deterministic and add a guard that fails when a - top-level test is unassigned or assigned more than once. New tests must be - covered automatically or force an explicit shard update. -4. **Preserve and verify the safety properties.** - - Retain `-race`, the 30-minute package timeout, the explicit locale, and - the current package coverage on Linux and Windows. - - Ensure cancellation, logs, and failure names still identify the exact - failed shard and test. - - Compare at least three post-change runs with the recorded baseline. Aim - for an 8-12 minute median wall-clock time; accept a different result only - if the measured critical path explains it and no coverage was lost. - - Document the trade-off: lower elapsed time uses more concurrent runner - capacity and may consume more total runner minutes. +### Publish PicFetch in Microsoft Store + +Build the Partner Center-reserved PicFetch product as one x64/ARM64 MSIX +bundle, make the Store build defer updates to Microsoft Store, retain the +portable GitHub/WinGet channel, validate on Windows with WACK, prepare the +English/German listing and privacy policy, then submit it for certification. +Implementation plan: `plans/2026-09-03-microsoft-store-msix.md`. ### Functional test coverage