diff --git a/.claude/docs/synchronizer-model.md b/.claude/docs/synchronizer-model.md index d3c5ff67011..8f64feff15b 100644 --- a/.claude/docs/synchronizer-model.md +++ b/.claude/docs/synchronizer-model.md @@ -12,6 +12,8 @@ Existing synchronizer implementations (grep `extends BaseSynchronizer` / `extend JS/TS user code is **not** synchronized — it is loaded on demand by `engine-javascript` (`JavascriptEndpoint` at `/services/js/...`, `/public/js/...`) via `DirigibleJavascriptCodeRunner` backed by Graalium/GraalVM polyglot. The `api-*` Java modules under `components/api/` register the JS-callable APIs (`@dirigible/db`, `@dirigible/http`, etc.) into the GraalJS context — pre-built TS/JS bundles for those APIs live in `components/api/api-modules-javascript/src/main/resources/META-INF/dirigible/modules/`. -**A pass only runs when something says the registry changed — and the file-system watcher is shallow.** `SynchronizationProcessor.processSynchronizers()` returns immediately unless `SynchronizationWatcher.isModified()`, and that watcher registers **`/registry/public` itself, non-recursively**: a write several folders deep produces no event on Linux at all. A publish is fine — `SynchronizationWatcherPublisherHandler.afterPublish` forces a pass — but anything else that writes the registry must do the same, and must additionally bracket the write with `RegistryMutationTracker` so a pass looking into a half-applied copy does not reap artefacts whose sources are still arriving. `RecursiveFolderWatcher` (`DIRIGIBLE_REGISTRY_EXTERNAL_FOLDER`) did neither, so a boot that lost the race against the copy installed a partial client-Java generation that was never rebuilt — controllers answering 404 for the life of the process with all their sources on disk (#7192). `RegistryMutationTracker`'s javadoc states the rule: bracket the write, do not enumerate callers. +**A pass only runs when something says the registry changed, and a deep write says it through `LocalRegistryWatcher`.** `SynchronizationProcessor.processSynchronizers()` returns immediately unless `SynchronizationWatcher.isModified()`, and that watcher registers **`/registry/public` itself, non-recursively**: a write several folders deep produces no event on Linux at all. A publish was fine — `SynchronizationWatcherPublisherHandler.afterPublish` forces a pass — but anything else that wrote the registry was invisible for the life of the process, which is how a boot that lost the race against the external-folder copy installed a partial client-Java generation that was never rebuilt: controllers answering 404 with all their sources on disk (#7192, fixed at that writer by #7299). `LocalRegistryWatcher` (`core-registry`) now closes the gap generically — it watches `/registry/public` **recursively**, registers folders as they appear, and marks the registry modified on every create/modify/delete, so whatever writes the registry, a pass follows (#7303). Its `DIRIGIBLE_REGISTRY_LOCAL_IGNORED_FOLDERS` top-level folders are neither watched nor reported. + +This does **not** retire `RegistryMutationTracker`: a pass is now scheduled while a multi-file write is still arriving, and only the bracket tells that pass to defer its cleanup instead of reaping artefacts whose sources have not landed yet. So a component that writes the registry outside the publisher pipeline still **brackets the write with `RegistryMutationTracker`** — it just no longer has to remember to announce it. `RegistryMutationTracker`'s javadoc states the rule: bracket the write, do not enumerate callers. **A failure of a collaborator records `FAILED` and is retried; `FATAL` is not a synchronizer's answer to it** (#7248). A `.listener` whose subscription the embedded broker refused while it was still taking its store lease, a `.job` the scheduler could not schedule yet, a `.camel` route missing a bean, a `.schema` naming a `.datasource` published on a later pass - each of these is transient, and each of the four synchronizers used to promote the second failure to `FATAL`, after which `SynchronizationProcessor.parseDefinitions` strips the artefact from every later pass until the file's bytes change (a topic subscription lost to a boot race stayed lost, with every message discarded). The shape now, in all four: a failed start registers `FAILED` **with its cause** (never `CREATED` while nothing runs) and returns `false`, which hands the artefact to the in-pass cross-retry loop (`DIRIGIBLE_SYNCHRONIZER_CROSS_RETRY_COUNT` x `_INTERVAL_MILLIS`, so a permanently failing artefact costs that loop every pass - the same cost a `FAILED` view carries since #6942); the `START` phase retries it and heals it to `CREATED` on success. **A pass runs its phases only when it carries a `NEW`/`MODIFIED` artefact** (`isSynchronizationNeeded` answers "the registry changed", and the test framework waits on that answer for a quiet period - do not widen it), so the processor additionally gives every `FAILED` artefact ONE `START` attempt per pass on an idle instance every `DIRIGIBLE_SYNCHRONIZER_FAILED_RETRY_INTERVAL_SECONDS` (30) via a private gate (`isFailedRetryDue` / `retryFailed`) - no cross-retry loop, and a repeat of the same error logs at DEBUG rather than stack-tracing at ERROR on every attempt. Two multitenant details: `MultitenantBaseSynchronizer` completes one shared artefact once per tenant and resets only the lifecycle between tenants, so a retry gate must include `FAILED` and not rely on `running` alone (the first tenant to subscribe flips it and would skip the rest), and the managers' idempotence per tenant (`ListenersManager.LISTENERS.containsKey`, `JobsManager.scheduleJob`'s `checkExists`) is what makes the repeated attempt safe. `ListenersManager.startListener` throws for a missing handler rather than returning normally - a normal return read as CREATED and running with nothing subscribed. Artefacts already persisted as `FATAL` by an earlier version stay stripped until republished; `case BROKEN:` in `parseDefinitions` is the same philosophy one level down (a definition that failed to parse is re-parsed every pass). diff --git a/CLAUDE_FEATURES.md b/CLAUDE_FEATURES.md index 597784ed1f4..b5f09b8f2ad 100644 --- a/CLAUDE_FEATURES.md +++ b/CLAUDE_FEATURES.md @@ -434,7 +434,6 @@ Interfaces that are either explicit SPIs (intended for extension), or load-beari | `UserAccessVerifier` | Plug-in role / access verification. | | `CustomSecurityConfigurator` | Add Spring Security configuration without forking `BasicSecurityConfig`. | | `SynchronizationWalkerCallback` | Callback used by the initializer / registry walker. | -| `LocalRegistryWatcherHandler` | Hook into local filesystem changes under the registry. | | `DataSourceLifecycleListener` | React to datasource registration / lifecycle. | #### CMS (`components/engine/engine-cms/`) diff --git a/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcher.java b/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcher.java index 4d558f7b73b..e3b243f510e 100644 --- a/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcher.java +++ b/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcher.java @@ -29,7 +29,6 @@ import java.util.Arrays; import java.util.Collections; import java.util.HashMap; -import java.util.List; import java.util.Map; import java.util.Set; import java.util.concurrent.ExecutorService; @@ -38,17 +37,42 @@ import java.util.stream.Collectors; import org.eclipse.dirigible.commons.config.DirigibleConfig; +import org.eclipse.dirigible.components.base.synchronizer.SynchronizationWatcher; import org.eclipse.dirigible.repository.api.IRepository; import org.eclipse.dirigible.repository.api.IRepositoryStructure; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.DisposableBean; -import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Scope; import org.springframework.stereotype.Component; /** - * The Class LocalRegistryWatcher. + * Watches {@code /registry/public} recursively and marks the registry modified whenever + * something in it changes, so that the next synchronization pass actually runs. + * + *

+ * {@link SynchronizationWatcher} - the thing {@code SynchronizationProcessor} asks before it does + * anything at all - registers the registry root and nothing below it, so a file written several + * folders deep produces no event and no pass is ever scheduled for it. A publish is covered + * ({@code SynchronizationWatcherPublisherHandler} forces a pass) and so is the external-folder copy + * ({@link RecursiveFolderWatcher} brackets and forces its own writes), but a writer that does + * neither used to be invisible for the life of the process - the shape of #7192, where a partial + * client-Java generation stayed installed with all its sources on disk. This watcher closes that + * gap generically: whatever writes the registry, the write is seen and a pass follows. + * + *

+ * It does not make + * {@link org.eclipse.dirigible.components.base.registry.RegistryMutationTracker} optional. A + * pass is now scheduled while a multi-file write is still arriving, and only the bracket tells that + * pass to defer its cleanup instead of reaping artefacts whose sources have not landed yet. A + * component that writes the registry outside the publisher pipeline still brackets the write - what + * it no longer has to do is remember to announce it. + * + *

+ * Marking is deliberately all this does. {@link SynchronizationWatcher#force()} sets a flag; which + * pass runs, when, and over what is the processor's decision, so a copy of a thousand files costs a + * thousand flag writes and one pass rather than a pass per file. The folders named by + * {@code DIRIGIBLE_REGISTRY_LOCAL_IGNORED_FOLDERS} (top level only) are neither watched nor marked. */ @Component @Scope("singleton") @@ -106,19 +130,18 @@ public class LocalRegistryWatcher implements DisposableBean { /** The repository. */ private final IRepository repository; - /** The handlers. */ - private final List handlers; + /** Told that the registry changed, so a synchronization pass is scheduled. */ + private final SynchronizationWatcher synchronizationWatcher; /** * Instantiates a new local registry watcher. * * @param repository the repository - * @param handlers the handlers + * @param synchronizationWatcher the synchronization watcher */ - @Autowired - public LocalRegistryWatcher(IRepository repository, List handlers) { + public LocalRegistryWatcher(IRepository repository, SynchronizationWatcher synchronizationWatcher) { this.repository = repository; - this.handlers = handlers; + this.synchronizationWatcher = synchronizationWatcher; } /** @@ -149,15 +172,12 @@ public synchronized void initialize() { this.watchService = FileSystems.getDefault() .newWatchService(); - // Initial sync before start watching - initialSync(); - // Register watchers recursively registerAll(sourceDir); // Start actual watching this.startWatching(); - } catch (IOException | InterruptedException e) { + } catch (IOException e) { logger.error("Error during initializing the Local Registry Watcher", e); } }); @@ -197,35 +217,6 @@ private static String sanitizeFolderName(String folderName) { .replace("\n", ""); } - /** - * Perform initial sync of all files and folders. - * - * @throws IOException Signals that an I/O exception has occurred. - */ - private void initialSync() throws IOException { - logger.info("Performing initial sync..."); - Files.walkFileTree(sourceDir, new SimpleFileVisitor<>() { - @Override - public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException { - if (isIgnored(dir)) { - logger.debug("Skipping ignored directory: {}", dir); - return FileVisitResult.SKIP_SUBTREE; - } - directoryRegistered(dir); - return FileVisitResult.CONTINUE; - } - - @Override - public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException { - if (!isIgnored(file)) { - fileRegistered(file); - } - return FileVisitResult.CONTINUE; - } - }); - logger.info("Initial sync complete."); - } - /** * Checks if is ignored. * @@ -289,10 +280,9 @@ private void register(Path dir) throws IOException { * Start watching. * * @throws IOException Signals that an I/O exception has occurred. - * @throws InterruptedException the interrupted exception */ - public void startWatching() throws IOException, InterruptedException { - logger.info("Recursively watching: " + sourceDir); + private void startWatching() throws IOException { + logger.info("Recursively watching: {}", sourceDir); watching = true; watchThread = Thread.currentThread(); @@ -352,43 +342,33 @@ private void watchLoop() throws IOException { for (WatchEvent event : key.pollEvents()) { WatchEvent.Kind kind = event.kind(); - if (kind == OVERFLOW) + if (kind == OVERFLOW) { + // Events were dropped, so what changed is unknown - which is exactly when a pass + // is most needed. Reconciling the whole registry is what a pass does anyway. + registryChanged(dir, "overflow"); continue; + } Path name = (Path) event.context(); Path sourcePath = dir.resolve(name); - if (kind == ENTRY_CREATE) { - if (Files.isDirectory(sourcePath)) { - // Register new directory + if (kind == ENTRY_CREATE && Files.isDirectory(sourcePath)) { + // A folder and everything already inside it. Register FIRST, report second: a + // file written into it before the registration produces no event of its own, and + // is only covered because the pass this report schedules walks the subtree after + // that file has landed. Reporting first would leave exactly that window open. + try { registerAll(sourcePath); - // Also sync its contents - try { - Files.walkFileTree(sourcePath, new SimpleFileVisitor<>() { - @Override - public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException { - fileCreated(file); - return FileVisitResult.CONTINUE; - } - - @Override - public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException { - directoryCreated(dir); - return FileVisitResult.CONTINUE; - } - }); - } catch (IOException e) { - logger.error("Failed to sync new folder: " + sourcePath, e); - } - } else { - fileCreated(sourcePath); + } catch (IOException e) { + logger.error("Failed to watch the new registry folder: " + sourcePath, e); } - } else if (kind == ENTRY_MODIFY) { - if (!Files.isDirectory(sourcePath)) { - fileModified(sourcePath); - } - } else if (kind == ENTRY_DELETE) { - fileDeleted(sourcePath); + registryChanged(sourcePath, "created"); + } else if (kind == ENTRY_MODIFY && Files.isDirectory(sourcePath)) { + // A directory's own timestamp moves whenever a child is added or removed, and that + // child's event is reported in its own right - reporting this one too is noise. + logger.debug("Ignoring the modification of the directory: {}", sourcePath); + } else { + registryChanged(sourcePath, kind == ENTRY_CREATE ? "created" : kind == ENTRY_DELETE ? "deleted" : "modified"); } } @@ -396,118 +376,31 @@ public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) th if (!valid) { keyToPathMap.remove(key); if (keyToPathMap.isEmpty()) { - break; + // The registry root itself is gone, so there is nothing left to register against. + // Say so: from here on a deep write schedules no pass until the watcher is + // re-initialized, which is the very failure this watcher exists to prevent. + logger.warn("Nothing left to watch under [{}] - the Local Registry Watcher is stopping." + + " Registry changes will no longer schedule a synchronization pass.", sourceDir); + return; } } } } /** - * Directory registered. - * - * @param path the path - */ - private void directoryRegistered(Path path) { - if (!Files.isDirectory(path) || isIgnored(path)) { - return; - } - for (LocalRegistryWatcherHandler handler : handlers) { - try { - handler.directoryRegistered(path); - } catch (Exception e) { - logger.error("Failed to handle registration of a directory: " + path, e); - } - } - } - - /** - * File registered. - * - * @param path the path - */ - private void fileRegistered(Path path) { - if (Files.isDirectory(path) || isIgnored(path)) { - return; - } - for (LocalRegistryWatcherHandler handler : handlers) { - try { - handler.fileRegistered(path); - } catch (Exception e) { - logger.error("Failed to handle registration of a file: " + path, e); - } - } - } - - /** - * Directory created. - * - * @param path the path - */ - private void directoryCreated(Path path) { - if (!Files.isDirectory(path) || isIgnored(path)) { - return; - } - for (LocalRegistryWatcherHandler handler : handlers) { - try { - handler.directoryCreated(path); - } catch (Exception e) { - logger.error("Failed to handle creation of a directory: " + path, e); - } - } - } - - /** - * File created. - * - * @param path the path - */ - private void fileCreated(Path path) { - if (Files.isDirectory(path) || isIgnored(path)) { - return; - } - for (LocalRegistryWatcherHandler handler : handlers) { - try { - handler.fileCreated(path); - } catch (Exception e) { - logger.error("Failed to handle creation of a file: " + path, e); - } - } - } - - /** - * File modified. + * Reports a change under the registry, which marks the registry modified so the next + * synchronization pass runs. Ignored folders are not reported. * - * @param path the path + * @param path the path that changed + * @param change what happened to it, for the log */ - private void fileModified(Path path) { - if (Files.isDirectory(path) || isIgnored(path)) { + private void registryChanged(Path path, String change) { + if (isIgnored(path)) { + logger.debug("Ignoring the {} entry: {}", change, path); return; } - for (LocalRegistryWatcherHandler handler : handlers) { - try { - handler.fileModified(path); - } catch (Exception e) { - logger.error("Failed to handle modification of a file: " + path, e); - } - } - } - - /** - * File deleted. - * - * @param path the path - */ - private void fileDeleted(Path path) { - if (Files.isDirectory(path) || isIgnored(path)) { - return; - } - for (LocalRegistryWatcherHandler handler : handlers) { - try { - handler.fileDeleted(path); - } catch (Exception e) { - logger.error("Failed to handle deletion of a file: " + path, e); - } - } + logger.debug("Registry entry {}: [{}] - scheduling a synchronization pass", change, path); + synchronizationWatcher.force(); } /** diff --git a/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherHandler.java b/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherHandler.java deleted file mode 100644 index afbada1d4f2..00000000000 --- a/components/core/core-registry/src/main/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherHandler.java +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright (c) 2010-2026 Eclipse Dirigible contributors - * - * All rights reserved. This program and the accompanying materials are made available under the - * terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v20.html - * - * SPDX-FileCopyrightText: Eclipse Dirigible contributors SPDX-License-Identifier: EPL-2.0 - */ -package org.eclipse.dirigible.components.registry.watcher; - -import java.nio.file.Path; - -/** - * The Interface LocalRegistryWatcherHandler. - */ -public interface LocalRegistryWatcherHandler { - - /** - * Directory registered. - * - * @param path the path - */ - public void directoryRegistered(Path path); - - /** - * Directory created. - * - * @param path the path - */ - public void directoryCreated(Path path); - - /** - * File registered. - * - * @param path the path - */ - public void fileRegistered(Path path); - - /** - * File created. - * - * @param path the path - */ - public void fileCreated(Path path); - - /** - * File modified. - * - * @param path the path - */ - public void fileModified(Path path); - - /** - * File deleted. - * - * @param path the path - */ - public void fileDeleted(Path path); - -} diff --git a/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherShutdownTest.java b/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherShutdownTest.java index 05a3f72865d..43ccfb08e91 100644 --- a/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherShutdownTest.java +++ b/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherShutdownTest.java @@ -25,11 +25,11 @@ import java.nio.file.WatchKey; import java.nio.file.WatchService; import java.time.Duration; -import java.util.List; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicBoolean; +import org.eclipse.dirigible.components.base.synchronizer.SynchronizationWatcher; import org.eclipse.dirigible.repository.api.IRepository; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; @@ -78,7 +78,7 @@ void destroyReturnsWhileTheWatchLoopIsBlockedInTake(@TempDir Path root) throws E IRepository repository = mock(IRepository.class); when(repository.getInternalResourcePath(anyString())).thenReturn(registryPublic.toString()); - LocalRegistryWatcher watcher = new LocalRegistryWatcher(repository, List.of()); + LocalRegistryWatcher watcher = new LocalRegistryWatcher(repository, mock(SynchronizationWatcher.class)); watcher.initialize(); awaitWatching(watcher); diff --git a/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherTest.java b/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherTest.java new file mode 100644 index 00000000000..73489790784 --- /dev/null +++ b/components/core/core-registry/src/test/java/org/eclipse/dirigible/components/registry/watcher/LocalRegistryWatcherTest.java @@ -0,0 +1,163 @@ +/* + * Copyright (c) 2010-2026 Eclipse Dirigible contributors + * + * All rights reserved. This program and the accompanying materials are made available under the + * terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v20.html + * + * SPDX-FileCopyrightText: Eclipse Dirigible contributors SPDX-License-Identifier: EPL-2.0 + */ +package org.eclipse.dirigible.components.registry.watcher; + +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.Mockito.clearInvocations; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.timeout; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; +import static org.mockito.Mockito.when; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; + +import org.eclipse.dirigible.commons.config.Configuration; +import org.eclipse.dirigible.components.base.synchronizer.SynchronizationWatcher; +import org.eclipse.dirigible.repository.api.IRepository; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import org.mockito.verification.VerificationMode; + +/** + * The registry watcher that everything depends on registers the registry root and nothing below it, + * so a write several folders deep schedules no synchronization pass at all - the shape of #7192. + * This watcher is the one that sees deep writes, and its whole job is to report them. + */ +class LocalRegistryWatcherTest { + + /** A change picked up by the watch service: up to ~10s where the JDK falls back to polling. */ + private static final long WATCH_TIMEOUT_MILLIS = 60_000; + + + /** How long to wait for the watch loop to be up before changing anything under it. */ + private static final long START_TIMEOUT_MILLIS = 30_000; + + private final SynchronizationWatcher synchronizationWatcher = mock(SynchronizationWatcher.class); + + private LocalRegistryWatcher watcher; + + @AfterEach + void stopWatching() { + Configuration.remove("DIRIGIBLE_REGISTRY_LOCAL_IGNORED_FOLDERS"); + if (watcher != null) { + // the watch service holds handles on the temp folders - Windows refuses to delete those + watcher.destroy(); + } + } + + @Test + void aFileWrittenDeepInTheRegistrySchedulesASynchronizationPass(@TempDir Path root) throws IOException { + Path registry = startWatching(root, "project/deep"); + + Files.writeString(registry.resolve("project") + .resolve("deep") + .resolve("artefact.txt"), + "content"); + + verify(synchronizationWatcher, reported()).force(); + } + + /** + * A folder is registered as it appears, or the project a publish or a copy drops in would be the + * one thing the watcher never watches. + */ + @Test + void aFolderCreatedAfterStartupIsWatchedTooAndItsContentSchedulesAPass(@TempDir Path root) throws IOException { + Path registry = startWatching(root, "existing"); + + Path fresh = Files.createDirectories(registry.resolve("fresh-project") + .resolve("deep")); + verify(synchronizationWatcher, reported()).force(); + clearInvocations(synchronizationWatcher); + + Files.writeString(fresh.resolve("artefact.txt"), "content"); + verify(synchronizationWatcher, reported()).force(); + } + + /** A deletion leaves runtime state behind just as a creation leaves it missing. */ + @Test + void aDeletedArtefactSchedulesASynchronizationPass(@TempDir Path root) throws IOException { + Path registry = startWatching(root, "project"); + Path artefact = registry.resolve("project") + .resolve("artefact.txt"); + Files.writeString(artefact, "content"); + verify(synchronizationWatcher, reported()).force(); + clearInvocations(synchronizationWatcher); + + Files.delete(artefact); + + verify(synchronizationWatcher, reported()).force(); + } + + /** An ignored top-level folder is neither watched nor reported - that is what the key is for. */ + @Test + void aChangeInAnIgnoredFolderSchedulesNothing(@TempDir Path root) throws IOException, InterruptedException { + Configuration.set("DIRIGIBLE_REGISTRY_LOCAL_IGNORED_FOLDERS", "ignored"); + Path registry = startWatching(root, "ignored/deep"); + + Files.writeString(registry.resolve("ignored") + .resolve("deep") + .resolve("artefact.txt"), + "content"); + // nothing to wait for, so give the watch service the time it would have needed to report it + Thread.sleep(5_000); + + verifyNoInteractions(synchronizationWatcher); + } + + private Path startWatching(Path root, String existingFolder) throws IOException { + Path registry = root.resolve("registry") + .resolve("public"); + Files.createDirectories(registry.resolve(existingFolder)); + + IRepository repository = mock(IRepository.class); + when(repository.getInternalResourcePath(anyString())).thenReturn(registry.toString()); + + watcher = new LocalRegistryWatcher(repository, synchronizationWatcher); + watcher.initialize(); + awaitWatching(); + return registry; + } + + /** + * The verification every assertion here uses: the write was reported at least once. How many + * events one write produces is the platform's business - inotify reports a created file as + * ENTRY_CREATE and again as ENTRY_MODIFY, while the polling watch service macOS falls back to + * reports it once - and marking the registry modified is idempotent, so the count carries nothing + * worth pinning down. The steps of a sequence are separated by clearing the recorded calls instead. + * + * @return the verification mode + */ + private static VerificationMode reported() { + return timeout(WATCH_TIMEOUT_MILLIS).atLeastOnce(); + } + + private void awaitWatching() { + long deadline = System.currentTimeMillis() + START_TIMEOUT_MILLIS; + while (!watcher.isWatching()) { + if (System.currentTimeMillis() > deadline) { + throw new AssertionError("The watcher did not start watching within " + START_TIMEOUT_MILLIS + " ms"); + } + try { + Thread.sleep(50); + } catch (InterruptedException e) { + Thread.currentThread() + .interrupt(); + throw new AssertionError("Interrupted while waiting for the watcher to start", e); + } + } + assertTrue(watcher.isWatching()); + } +} diff --git a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/RegistryDeepWriteSyncIT.java b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/RegistryDeepWriteSyncIT.java new file mode 100644 index 00000000000..a41c65896b2 --- /dev/null +++ b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/RegistryDeepWriteSyncIT.java @@ -0,0 +1,135 @@ +/* + * Copyright (c) 2010-2026 Eclipse Dirigible contributors + * + * All rights reserved. This program and the accompanying materials are made available under the + * terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v20.html + * + * SPDX-FileCopyrightText: Eclipse Dirigible contributors SPDX-License-Identifier: EPL-2.0 + */ +package org.eclipse.dirigible.integration.tests.api; + +import static io.restassured.RestAssured.given; +import static org.hamcrest.Matchers.containsString; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.concurrent.TimeUnit; + +import org.apache.commons.io.FileUtils; +import org.awaitility.Awaitility; +import org.eclipse.dirigible.components.base.synchronizer.SynchronizationWatcher; +import org.eclipse.dirigible.components.initializers.synchronizer.SynchronizationProcessor; +import org.eclipse.dirigible.repository.api.IRepository; +import org.eclipse.dirigible.repository.api.IRepositoryStructure; +import org.eclipse.dirigible.tests.base.IntegrationTest; +import org.eclipse.dirigible.tests.framework.restassured.RestAssuredExecutor; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.annotation.DirtiesContext; + +/** + * A write several folders deep into the registry reaches the runtime on its own - whoever made it + * (issue #7303). + * + *

+ * {@code SynchronizationWatcher}, which decides whether a synchronization pass runs at all, + * registers the registry root and nothing below it, so only a publish (which forces a pass) and the + * external-folder copy (which forces its own, #7299) were ever seen. {@code LocalRegistryWatcher} + * watches the registry recursively and marks it modified, which is what makes any other writer - a + * git clone straight into the registry, a tool copying a project in - self-healing too. + * + *

+ * Nothing here publishes, forces a pass, or mounts an external folder: the platform has to notice + * by itself, so the assertion waits for the scheduled pass instead. On master this test fails by + * timing out on a 404 with the source sitting on disk. + */ +@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS) +class RegistryDeepWriteSyncIT extends IntegrationTest { + + private static final String PROJECT = "registry-deep-write-it"; + + /** + * The folder the source is written into, INSIDE the project. A file added directly under the + * project folder moves the mtime of a direct child of the registry root, which the root watcher + * notices on the polling watch service macOS falls back to - the test would then pass for the wrong + * reason. + */ + private static final String FOLDER = "rdwit"; + + private static final String ENDPOINT = "/services/java/" + PROJECT + "/" + FOLDER + "/DeepWrite"; + + /** The synchronization job fires on its own schedule (10s by default), then javac runs. */ + private static final long AWAIT_SECONDS = 180; + + @Autowired + private RestAssuredExecutor restAssuredExecutor; + + @Autowired + private IRepository repository; + + @Autowired + private SynchronizationWatcher synchronizationWatcher; + + @Autowired + private SynchronizationProcessor synchronizationProcessor; + + @AfterEach + void removeTheProject() throws IOException { + FileUtils.deleteDirectory(projectFolder().toFile()); + } + + @Test + void a_source_written_deep_into_the_registry_is_compiled_and_served() throws IOException { + Files.createDirectories(sourceFolder()); + // Settle first, or the assertion proves nothing: the project folder is a direct child of the + // registry root, which the root watcher DOES see, and the pass that schedules would pick the + // source up on its own. Once no change is pending and no pass is running, nothing else can + // schedule one - so the write below is the only possible cause of the next pass. + awaitAnIdlePlatform(); + + Files.writeString(sourceFolder().resolve("DeepWrite.java"), handlerSource()); + + restAssuredExecutor.execute(() -> given().when() + .get(ENDPOINT) + .then() + .statusCode(200) + .body(containsString("hello from a deep registry write")), + AWAIT_SECONDS); + } + + private void awaitAnIdlePlatform() { + Awaitility.await() + .atMost(2, TimeUnit.MINUTES) + .pollInterval(1, TimeUnit.SECONDS) + .until(() -> !synchronizationWatcher.isModified() && !synchronizationProcessor.isSynchronizationRunning()); + } + + private Path sourceFolder() { + return projectFolder().resolve(FOLDER); + } + + private Path projectFolder() { + return Path.of(repository.getInternalResourcePath(IRepositoryStructure.PATH_REGISTRY_PUBLIC)) + .resolve(PROJECT); + } + + private static String handlerSource() { + return """ + package %s; + import jakarta.servlet.http.HttpServletRequest; + import jakarta.servlet.http.HttpServletResponse; + import org.eclipse.dirigible.engine.java.handler.JavaHandler; + public class DeepWrite implements JavaHandler { + @Override + public void handle(HttpServletRequest request, HttpServletResponse response) throws Exception { + response.setContentType("application/json"); + response.getWriter().write("{\\"message\\": \\"hello from a deep registry write\\"}"); + } + } + """.formatted(FOLDER); + } + +}