feat: static themes and /api/v2 surface for headless UI#3136
Open
marevol wants to merge 117 commits into
Open
Conversation
Serves the resolved static theme's HTML entry and asset files via a single LastaFlute action. Dispatching between index and asset is driven by request attributes (REQ_ATTR_MODE, REQ_ATTR_ASSET_PATH) that StaticThemeFilter will populate before forwarding to /theme-view, sidestepping wildcard route limitations. The resolveAsset() helper rejects null, absolute, and "..-bearing paths, then re-verifies the normalized result stays under the theme base directory to prevent path traversal.
Implements Batch 2.C of the static-theme Plan 2 (tasks 2.7-2.9). Each handler delegates to the existing v1 helper (SuggestHelper, LabelTypeHelper, PopularWordHelper) and maps the result into a snake_case Map<String,Object> wrapped by V2EnvelopeWriter. - /suggest-words reads q/num/fn/lang (and label/virtual-host tags) and exposes SuggestItem.getTags() as the "types" field per plan. - /labels mirrors v1's SearchRequestType.JSON usage; the enum has no JSON_API variant despite the plan's wording, so JSON is reused. - /popular-words honors fessConfig.isWebApiPopularWord() and emits an invalid_request envelope when the feature is gated off. - All three reject non-GET methods with invalid_request (400). Extends the in-test StubRequest to carry a parameter map and a custom method so handlers can be exercised end-to-end without Mockito.
Extracts /api/v2/search into SearchHandler (with V2JsonRequestParams) under api/v2/handlers/ rather than growing SearchApiV2Manager toward v1's 1,431-line shape. The manager holds a single SearchHandler instance (stateless, so a shared singleton avoids per-request allocation) and dispatches via the existing switch. V2JsonRequestParams mirrors v1's JsonRequestParams inner class verbatim (parameter names q, start, num, fields.*, as.*, lang, sort, sdh, ex_q, offset) so wire compatibility is preserved. Payload keys match the v1 search JSON shape — q, query_id, exec_time, query_time, page_size, page_number, record_count (long), record_count_relation, page_count, highlight_params, next_page, prev_page, start_record_number, end_record_number, page_numbers, partial, search_query, requested_time, related_query, related_contents, data, facet_field, facet_query — and document fields are filtered through QueryFieldConfig#isApiResponseField. InvalidQueryException / ResultOffsetExceededException surface as invalid_request 400; everything else becomes internal_error 500. Includes three handler tests (empty-index data, POST method gate, top-level envelope shape) plus one manager-level dispatch test. The existing small handlers (/health, /suggest-words, /labels, /popular-words) are left in the manager to keep churn focused on the large endpoints.
Adds ScrollSearchHandler implementing GET /api/v2/documents/all as an NDJSON
stream. Mirrors v1's processScrollSearchRequest but each line is a
{"data":<doc>} envelope written via Jackson (no hand-rolled StringBuilder),
filtered through QueryFieldConfig.isApiResponseField. The endpoint is gated
by api.search.scroll and rejects non-GET methods through V2EnvelopeWriter.
The handler is not yet wired into SearchApiV2Manager; that comes with the
favorite endpoint in the next commit so both new routes ship behind the
same path-matching change.
Adds FavoriteGetHandler implementing read-only favorite lookup. Resolves the doc id to its URL via SearchHelper.getDocumentByDocId, then asks FavoriteLogService.getUrlList whether the current user code has favorited it. The favorite_count field on the document is forwarded as the response count. docId is validated against [A-Za-z0-9_-]+ to keep obviously malformed input from reaching the search backend. The feature is gated by isUserFavorite(); when disabled, the handler emits invalid_request (400). When the user code is blank (anonymous), favorite is reported as false but count still reflects the stored value. POST/DELETE handling is deferred to Plan 3 (requires auth + CSRF). The handler is not yet wired into SearchApiV2Manager; that comes in the next commit alongside the scroll endpoint.
Wires the new ScrollSearchHandler and FavoriteGetHandler into
SearchApiV2Manager. Adds path-matching before the existing static switch:
"/documents/all" dispatches to the scroll handler; "/documents/{id}/favorite"
extracts the doc id from the path and dispatches to the favorite handler.
Order matters — /documents/all is checked first so "all" isn't interpreted
as a doc id by the suffix-match pattern below it.
Manager-level dispatch tests confirm both routes reach the right handler
shape (NDJSON stream or v2 envelope) and that a malformed doc id surfaces
through the manager as invalid_request.
fess_api.xml already registers searchApiV2Manager (commit 47c724d from
Batch 2.B); no XML change needed here.
The new tests use fully-qualified Assertions.* calls to avoid ambiguity with UnitFessTestCase's inherited JUnit 4-style overloads, so the static imports were never used.
- PasswordChangeHandler: require and verify current_password via FessLoginAssist.findLoginUser before allowing password change to block session-riding account takeover. - LoginHandler: rotate JSESSIONID via changeSessionId() after auth to defeat session fixation; reorder rate-limit so USER slot is consumed only on credential failure, not on backend errors. - LoginRateLimiter: add Scope.CHAT and peek() for non-consuming bucket check. - ChatHandler/ChatStreamHandler: apply per-user CHAT rate limit; refactor SSE writer lifecycle to avoid reopening a closed PrintWriter and double-emitting error events. - SearchApiV2Manager: short-circuit the broad catch when the response is already committed (SSE/NDJSON) and emit a sanitized "internal error" message instead of leaking e.getMessage().
- StaticThemeInstaller: chain rollback IOException via addSuppressed on the original move failure so post-mortem diagnostics see both exceptions; reject ZIP entries whose path segments start with "." to block hidden-file (.env, .htaccess, .git) shipment; log WARN on success-path attic cleanup failure so operators notice accretion. - ThemeViewAction: always set X-Content-Type-Options: nosniff and Referrer-Policy: same-origin on theme asset responses to defeat MIME sniffing attacks on SVG/HTML assets. - ThemeRegistry: narrow lookupDefaultThemeName's catch from Throwable to Exception so Error/OutOfMemoryError propagate. - StaticThemeFilter: promote the DI-not-ready fallback log to a one-shot WARN guarded by AtomicBoolean for first-occurrence observability without log spam.
Extract hasZipExtension(String) and switch toLowerCase to toLowerCase(Locale.ROOT). On a Turkish-locale JVM, the default toLowerCase otherwise converts "ZIP" to "zıp" (dotless i), causing legitimate theme uploads to be rejected.
Split the lastaToActionFilter mapping so REQUEST stays on /* while FORWARD is scoped to /theme-view, the only URL that actually relies on it (StaticThemeFilter forwards to /theme-view for ThemeViewAction to handle). Prevents future RequestDispatcher.forward() calls to extensionless URLs from being unintentionally re-routed through the action mapper.
Add the 25 labels.theme_* / labels.menu_theme keys and the 11 theme-related errors/success messages introduced for the static theme admin UI to the 15 locale bundles that were previously missing them (en, de, es, fr, hi, id, it, ko, nl, pl, pt_BR, ru, tr, zh_CN, zh_TW). English values are committed verbatim as machine-translation pending stubs, matching the established repo precedent (entraid_* keys, AI Mode rename). Also add LabelMessageThemeParityTest to fail-fast in future PRs if a locale falls behind.
…on tests - PasswordChangeHandler: on successful change, rotate the session CSRF token via SessionCsrfTokenManager and surface the freshly issued token as `csrf_token` in the response payload. Invalidates any previously exfiltrated token so SPA clients are forced to refresh after the change (M-03). - PasswordChangeHandlerTest: add minimal HttpSession stub + StubRequest#withSession fixture, restore the rotate/issue assertion, drop JUnit 5 static imports shadowed by UnitFessTestCase inherited helpers. - FavoritePostHandlerTest: add regression coverage for M-13 (non-string query_id no longer yields 500) and for the sanitized INTERNAL_ERROR message branch.
Tightens the /api/v2 surface following review of PR #3136: - Add V2ErrorCode.METHOD_NOT_ALLOWED (405) and CONFLICT (409); switch all 14 handlers from 400 invalid_request to 405 method_not_allowed for wrong-method rejections. - V2EnvelopeWriter: set UTF-8 character encoding before getWriter(), guard writeError with isCommitted(), map status by httpStatus >= 500, add writeInternalError(res, cause, logger, tag) so the wire never carries upstream e.getMessage() (CWE-209). - Apply the helper across SearchApiV2Manager, Search/Scroll/Cache/ Favorite/Click/UiConfig/Chat handlers and add WARN+throwable logs for catches that previously logged only at DEBUG or message-only. - ScrollSearchHandler: track wroteAnyLine so a mid-stream failure no longer appends a JSON error envelope to NDJSON output. - LoginRateLimiter.sweep(): re-check inside entries.compute to close the read-then-remove race against allow(); widen multiplies to long. - LoginHandler: add WARN log on unexpected RuntimeException, collapse "(ip)"/"(user)" suffixes into a single uniform "too many login attempts" message, one-shot WARN on resolveClientIp failure, reject return_to paths containing backslash or null byte. - PasswordChangeHandler: declare logger, add WARN on every catch, short-circuit blank new_password before the mismatch check, keep csrf_token in the response even if rotation throws. - ChatStreamHandler: SSE Content-Type now carries charset=utf-8, scheduled heartbeat comment frames every 15 seconds, writer.checkError() after each emit drops the loop on client disconnect, throwable kept in WARN logs. - ChatRequestBody: WARN once when the label/ex_q allowlist load fails (previously dropped silently, making filters look broken). - V2JsonBody / CsrfRequirement: lowercase/uppercase via Locale.ROOT to dodge the Turkish dotless-i pitfall. Tests: extend SearchApiV2ManagerCsrfTest with 4 new CSRF gates (/click, /favorite, /chat, /chat/stream); add concurrency tests to LoginRateLimiter; add V2EnvelopeWriter charset/commit-guard tests and V2ErrorCode 405/409 tests; update all existing method-gate tests to expect 405; add PasswordChangeHandler blank-newPw and weak-password tests; add no-message-leak assertions for /health, suggest, labels, popular. Suite at 5803 passes plus 42 new assertions, 0 regressions.
Addresses theme-infrastructure findings from the PR #3136 review pass: - StaticThemeInstaller: tighten extraction guards. Switch from "any segment starting with '.'" to an explicit denylist (.git, .svn, .hg, __MACOSX, .DS_Store) so legitimate .well-known directories survive; reject ".." per-segment instead of the over-broad contains(".."); reject backslashes and null bytes; add Instant+UUID suffix to attic names so two installs within the same millisecond no longer collide; on each install and delete, sweep .attic-* older than the configured retention and .staging-* orphans older than one hour (the previously documented but never-enforced retention). Promote error mapping by introducing InstallException.Code (INVALID_NAME, ACTIVE_DEFAULT, JSP_TYPE, NOT_FOUND, EXTRACT_FAILED, MOVE_FAILED, MANIFEST_INVALID, SIZE_LIMIT, ENTRY_LIMIT, RATIO_LIMIT, OTHER) so callers no longer need to string-match the message. - ThemeManifest: parse with SafeConstructor + LoaderOptions (codepoint and alias caps) instead of the default Yaml() constructor; reject non-mapping roots with a typed exception; harden the entry field validator against backslash/null/Windows drive letters; cap description/homepage/author at 4 KB; coerce supportedLocales scalar to a singleton list. - ThemeRegistry: lower-case virtualHostKey via Locale.ROOT so host resolution matches theme names regardless of header casing; bake the resolved default theme name into the snapshot so reads no longer dip into FessConfig system properties on each request. - ThemeViewAction: emit a strict CSP for serveIndex (default-src 'self', frame-ancestors 'none', etc.) and a defense-in-depth CSP for SVG (default-src 'none'); add ETag / Last-Modified / Content- Length and honour If-None-Match for 304 short-circuits; add .mjs, .json, and .webmanifest MIME mappings with charset; lowercase the filename suffix via Locale.ROOT so .CSS / .JS no longer slip to octet-stream; switch resolveAsset's traversal check from contains("..") to a per-segment exact match. - StaticThemeFilter: add /favicon.ico, /robots.txt, /sitemap.xml, /manifest.webmanifest, /.well-known to the pass-through list so static themes never shadow these top-level paths; drop the dead startsWith(p + "?") branch (servlet URI never carries the query); honour the manifest spaFallback flag so themes can opt out of SPA routing; DEBUG-log the resolveHostKey catch instead of swallowing. - SessionCsrfTokenManager.rotate(): now invalidates the existing token AND issues a fresh one in a single call, matching the name callers reasonably expect. - AdminThemeAction: dispatch error display by InstallException.Code instead of brittle msg.contains("active") / msg.contains("JSP") string-matching. Tests cover: SafeConstructor non-mapping root, entry backslash / null / drive-letter rejection, supportedLocales scalar coercion, description length cap; virtualHost case-insensitive resolution and snapshot stability; denylisted dotfile / dotdir rejection with new "Denied" diagnostic; .mjs MIME, CSP-on-SVG, ETag-on-asset, content- type case-insensitive; pass-through for favicon/robots/well-known; SessionCsrfTokenManager rotate-then-verify and contention safety.
Aligns the bundled bootstrap reference theme with the hardened server contract and addresses frontend findings from the PR #3136 review: - api.js: add sseStream(path, body, onEvent, onError) that POSTs with the X-Fess-CSRF-Token header and reads the response via a ReadableStream + TextDecoder, splitting on blank-line SSE frames and dispatching parsed event/data tuples. Returns the AbortController so callers can cancel. The native EventSource path (api.sse) is kept but marked @deprecated because EventSource cannot carry a CSRF header and /chat/stream is POST-only. Wrap fetch network failures in a typed NetworkError so callers can distinguish offline / DNS from server errors. Thread an options parameter through api.get() so request-level AbortControllers can be wired. - chat.js: switch to api.sseStream, abort the previous stream before submitting a new question, route NetworkError to t("error.network") and other failures to t("error.server"). - search.js: AbortController per runSearch() (previous in-flight call aborted before the next), module-level `attached` guard so the fess:auth:login handler no longer re-registers form/suggest/click listeners; export refresh() for post-login data reload. - app.js: call search.refresh() on login instead of search.attach(). - auth.js: rotateCsrf accepts the parsed logout envelope and picks up a fresh token from it when the server supplies one, falling back to GET /ui/config for older builds. - i18n.js: t() uses replaceAll so a key whose value happens to repeat is fully substituted; init() also updates document.title from messages["page.title"]; document applyDom for dynamically inserted subtrees. - styles.css: scope the global *:focus-visible rule to #page-root so the outline rule cannot leak into modals or admin contexts. - index.html: add a Content-Security-Policy meta (script-src 'self', frame-ancestors 'none', etc.) plus referrer policy; insert a visually-hidden <h1> so screen readers and SEO see a proper heading-1 before the <h2> result titles. - messages.{en,ja}.json: add error.network, page.title, page.heading. - README: document the streaming-chat fetch+CSRF pattern. No XSS regressions: dynamic content still rendered exclusively via createElement + textContent + setAttribute; the only innerHTML usage remains the empty-string clear pattern. All six modules pass `node --check`.
Replace the English machine-translation pending stubs for the theme
admin UI keys (24 labels.menu_theme/labels.theme_* and 11
errors.*theme*/success.*theme* messages) with proper translations
across 14 non-en/ja locales: de, es, fr, hi, id, it, ko, nl, pl,
pt_BR, ru, tr, zh_CN, zh_TW.
Common loanwords identical to English ("OK", "Manifest", "Type",
"Name", "Version", "Theme", "Default") are kept as-is where natural
in the target locale. French and Italian apostrophes follow the
existing single-quote convention used by the surrounding errors.*
entries in those files.
LabelMessageThemeParityTest still passes (key parity preserved).
Add <session-config><cookie-config> block setting HttpOnly=true, Secure=true, and tracking-mode=COOKIE. Production deployments require HTTPS; SameSite must be configured at the Tomcat CookieProcessor level (META-INF/context.xml). Documents the deployment expectation inline.
- V2ErrorCode: add UNSUPPORTED_MEDIA_TYPE (415), PAYLOAD_TOO_LARGE (413), SERVICE_UNAVAILABLE (503), NOT_ACCEPTABLE (406); fix CONFLICT Javadoc copy-paste error (MJ-16, MJ-17). - V2EnvelopeWriter.writeSuccess: reject payload keys that would clobber the reserved status/version envelope fields (MJ-15). - V2JsonBody: enforce UTF-8 charset, strip leading BOM, require Content-Type header, and apply StreamReadConstraints (maxNestingDepth=32, maxNumberLength =1000, maxStringLength=1MiB) via JsonFactory.builder (MJ-24). - V2JsonRequestParams.getPageSize: throw InvalidPageSizeException for num<=0; clamp num>max and expose isPageSizeClamped() so callers can surface the warning (MJ-25). - SearchApiV2Manager: prefix-boundary check in matches() prevents claim of /api/v2foo; handleHealth returns HTTP 503 SERVICE_UNAVAILABLE on red cluster status; subPath strips trailing slash (MJ-19, MJ-26). - CsrfRequirement: add class-level policy doc; new CsrfRequirementCompleteCoverageTest enumerates every dispatched endpoint to pin the require/exempt table against drift (MJ-31). Tests: +12 V2ErrorCode cases (new codes), +reserved-key clobber, charset/ BOM/depth rejection, num<=0 + clamp flag, prefix-boundary, red→503, trailing-slash, complete-coverage CSRF audit.
- All 405 responses set the RFC 7231-required Allow header (GET or POST per handler) before writing the v2 error envelope (MJ-18). - CacheHandler: honor isLoginRequired() — anonymous request to a login- required deployment now returns 401 AUTH_REQUIRED, matching v1 CacheAction behavior (MJ-20). - ScrollSearchHandler: emit a final NDJSON error record when an exception fires after partial output so clients can distinguish a truncated stream from a complete one; hoist ObjectMapper to static final (MJ-22, m-11). - FavoritePostHandler: drop URLDecoder.decode on query_id — the field arrives as plain JSON text, not a URL-encoded form value; the v1 double-decode was a latent bug (MJ-23). - ClickHandler: reorder so session/auth check fires before body parse, so anonymous callers short-circuit with logged:false instead of erroring on malformed JSON; tolerate missing UserInfoHelper in unit DI graphs (m-15). - UiConfigHandler: emit csrf_required boolean so SPA clients can branch on it; empty csrf_token when CSRF is disabled (m-14). - SearchHandler/ScrollSearchHandler: document the intentional omission of the v1 Referer allowlist (CSRF replaces it for state-changing endpoints; idempotent GETs do not need it). Release-note item (MJ-21). Tests updated to assert Allow header on every 405, login-required parity, NDJSON error trailer, no URL-decode of query_id, anonymous-short-circuit ordering, and csrf_required field truthiness.
- ChatStreamHandler: move every gate check (method, isRagChatEnabled, body parse, blank message, rate-limit) BEFORE setSseHeaders() and getWriter(). Pre-stream failures now return the correct HTTP status (405/400/429) with application/json instead of the prior HTTP 200 + text/event-stream that silently masked errors and let cross-origin clients hold worker threads via EventSource. Adds Allow: POST on 405 (CR-1, MJ-18). - LogoutHandler: remove the dead-code csrf.rotate(session) call (the next line invalidates the session anyway); guard session.invalidate() with try/catch (IllegalStateException) so an already-invalidated session does not leak as an internal_error envelope. Adds Allow: POST on 405 (CR-2, MJ-18). Tests: HEAD/OPTIONS reject with 405 application/json; chat-disabled, oversized body/message, rate-limit each return their correct HTTP status; LogoutHandler exercises session-invalidate and IllegalStateException- swallowed branches.
…ation - LoginRateLimiter: cap entries (default 100,000, configurable via theme.api.login.rate.limit.max.entries) with FIFO eviction; schedule sweep() every 5 minutes via TimeoutManager; null/empty key now denies (defense in depth); Math.max guard in lockOut so a shorter lockoutSeconds cannot shrink an active lockout; clear(Scope, String) lets LoginHandler reset buckets on successful auth (MJ-4, MJ-5, MJ-6). - LoginHandler: detect already-authenticated session before re-auth — same user returns early with the existing user shape + current CSRF token, different user is logged at INFO with prevUserId/newUserId; harden return_to with a strict regex + control-character scan (MJ-8, MJ-27); invoke limiter.clear on both USER and IP buckets after success (MJ-5); delegate user payload to shared UserPayloads.toJson. - UserPayloads (new): canonical user JSON shape with null-safe array fields so MeHandler and LoginHandler emit byte-identical roles/groups/permissions arrays — never JSON null (MJ-28). - MeHandler: route through UserPayloads.toJson (MJ-28). - PasswordChangeHandler: add re_login_required:true to the success payload so SPAs can prompt re-login; documents that other sessions are NOT invalidated by design (MJ-7). - ChatRequestBody: getWarnings() exposes per-field rejected-value lists for extra_queries / fields.label allowlist violations; ChatHandler / ChatStreamHandler surfacing is a follow-up (MJ-29). - SessionCsrfTokenManager: DEBUG-gated logging on issue/rotate/verify records token length and session id only (never values) (m-9). - FessConfig + fess_config.properties: new theme.api.login.rate.limit.max.entries (default 100000). - Handler Javadoc clarifies that error.message is developer-facing English; clients use error.code for i18n (MJ-30 — release note). Tests added for: memory cap, sweep timer, clear-on-success, peek empty-key deny, lockOut Math.max, return_to CRLF / protocol-relative rejection, SSO same/different user paths, MeHandler null-array shape, ChatRequestBody warnings unmodifiable map, password mismatch for authenticated user.
- StaticThemeInstaller.deleteRecursivelyQuiet: wrap Files.walk in try-with-resources to close the underlying DirectoryStream — previously leaked a file descriptor on every cleanup invocation (MJ-11). - StaticThemeInstaller @PostConstruct: wire theme.upload.max.* properties through to the runtime fields so operator config actually takes effect; retain hardcoded defaults as unit-test fallback (MJ-12); also invoke cleanupOldAtticDirs on startup to sweep JVM-crash orphans (m-9). - ThemeViewAction & ThemeRegistry & StaticThemeInstaller cleanup: use LinkOption.NOFOLLOW_LINKS and BasicFileAttributes symlink check so a symlinked entry (operator-side restore, future installer changes) cannot serve files outside the theme tree or get registered as a theme (MJ-1). - ThemeViewAction.resolveAsset: reject dotfile names and theme.yml / README.md / CHANGELOG.md / LICENSE* so manifest / build metadata cannot be served to anonymous users (MJ-2). - ThemeViewAction: remove duplicate requestManagerRef field; promote FessBaseAction.requestManager to protected for inheritance access; SVG charset & wasm MIME type; Vary: Accept-Encoding on cacheable responses; notFound() uses StandardCharsets.UTF_8 (MJ-13, m-5, m-7, m-8). - StaticThemeFilter: defensive instanceof guard so non-HTTP requests do not ClassCastException (MJ-10). - ThemeManifest: SnakeYAML nestingDepthLimit=20; uniform checkFieldLength on displayName/license/minFessVersion/entry/supportedLocales items (m-2, m-10); reference shared NAME_PATTERN from installer (m-4). - ThemeViewAction.serveIndex: documented theoretical race between Files.size and stream open (MJ-14). Tests: 17 new methods across installer / view / registry / filter / manifest covering FD-close, config-wired-through, symlink rejection, dotfile rejection, cast guard, field-length checks, attic startup sweep.
…dal id - AdminThemeAction.reload catch now uses addErrorsFailedToReloadTheme (new) instead of the misleading addErrorsFailedToChangeDefaultTheme (MJ-33). - AdminThemeAction.setdefault now branches on empty name and uses the new addSuccessClearDefaultTheme instead of an English "(none)" literal (m-26). - admin_theme.jsp: surface <la:errors property="_global"/> and ="name" near the top so validation failures from details/upload/delete/setdefault flows are visible after redirect-back (MJ-32). Modal id and data-target switched from f:h(t.name) to s.index for defense-in-depth (MJ-34). - admin_theme_upload.jsp: JS size guard reads fessConfig.themeUploadMaxSizeAsInteger so it stays in sync with the server cap (was hardcoded 10 MB while server default is 50 MB); accept attribute lists ZIP MIME variants (m-25, m-27). - Form classes: document the intentional crudMode omission (m-7). - FessMessages: addErrorsFailedToReloadTheme and addSuccessClearDefaultTheme generated stubs; matching constants. - fess_message*.properties (base + 16 locales): new errors.failed_to_reload_theme and success.clear_default_theme keys with proper translations (no English stubs) — verified parity across all 17 locale files; LabelMessageThemeParityTest regex auto-covers them.
…hten
- search.js: safeHref() applies an http/https/ftp/ftps scheme allowlist
before setAttribute("href", value) so a crawled javascript: URL cannot
execute as stored XSS on click (CR-3); add null guards around
#search-input.value (MJ-40).
- search.js + index.html: wire ARIA combobox attributes — role="combobox",
aria-expanded, aria-controls, aria-autocomplete, aria-haspopup on the
input; per-item aria-selected + aria-activedescendant; ArrowUp wraps to
the last item (MJ-35, MJ-36).
- index.html: remove 'unsafe-inline' from style-src; Bootstrap 5 CSS is
loaded same-origin and the bundled theme has no inline styles, so the
unsafe directive only weakened defense (MJ-37).
- api.js: flush TextDecoder after the SSE read loop exits so the trailing
multi-byte UTF-8 sequence is not silently dropped or replaced with
U+FFFD when the final chunk splits mid-character (MJ-38).
- chat.js: abort any active SSE stream on pagehide so a back-button or
SPA navigation does not leak the network connection / dispatch events
into a destroyed DOM (MJ-39); replace verbatim server error rendering
with an allowlisted data.code → i18n-key mapping so backend stack
fragments cannot surface in the chat bubble (MJ-41).
- messages.en.json & messages.ja.json: 5 new error.* keys for the chat
allowlist; sets remain in lockstep (BundledBootstrapThemeStructuralTest
asserts parity).
- theme.yml: add type: static and thumbnail: thumbnail.png for forward-
compat (m-33).
- snake_case across request body, response, and SSE event data;
ChatSource: rank/doc_id/url_link/go_url; SSE: session_id/
html_content/original_query/new_query/max_attempts/sleep_ms/
elapsed_ms/timeout_ms/hit_count/error_code
- DELETE /api/v2/chat/sessions/{id} replaces the POST /chat clear
flag; CsrfRequirement gates the new path
- fields: { label } nested structure; ex_q -> extra_queries
- envelope.version removed (path /api/v2/ already conveys version)
- ChatApiHelper extracted for shared parsing/config util
- OpenAPI split into v1/ + v2/ directories; v1 marked deprecated
- webapp/js/chat.js migrated to v2 (POST + fetch SSE + DELETE clear);
bootstrap theme chat.js snake_case alignment
- v1 ChatApiManager/SearchApiManager @deprecated(since=15.7.0,
forRemoval=true); behavior unchanged
- tests: +ChatApiHelperTest, +ChatSessionClearHandlerTest, +SSE event
snake_case, +CSRF coverage; existing tests updated for new wire
format
…plit The file is superseded by src/main/config/openapi/v1/openapi-user.yaml and src/main/config/openapi/v2/openapi-user.yaml introduced in the preceding commit.
…orms
Adds class/method/field/enum-constant javadoc and explicit default
constructors across the v2 handler set, the theme subsystem
(Theme, ThemeManifest, ThemeRegistry, ThemeType, StaticThemeInstaller,
ThemeManifestException), the admin/theme form objects, and
SessionCsrfTokenManager. Also fixes a stale {@link} target in
SearchHandler to point at the relocated CsrfRequirement class.
Documentation-only; no behavior changes.
…-review) Comprehensive fix wave addressing PR #3136 review findings across the /api/v2 surface, theme installer hardening, and supporting i18n. API v2 hardening - LoginHandler: drop same-user fast path; strict instanceof string coercion - LoginRateLimiter: idle-only eviction (never evict locked-out entries) - PasswordChangeHandler: BCrypt change now invalidates session + calls FessLoginAssist.logout(); response signals re_login_required and omits csrf_token (token would belong to a destroyed session) - FavoritePostHandler: idempotent 200 with already_existed:true marker - ChatStreamHandler: SSE keep-alive moved from per-request executor to a shared 2-thread daemon pool (lifecycle via @PostConstruct/@PreDestroy) - ClickHandler: epoch->UTC LocalDateTime (was JVM-default zone) - CacheHandler / PasswordChangeHandler: split exception handling so ComponentNotFound surfaces 500 instead of being masked - SearchApiV2Manager: pre-switch NOT_FOUND for unknown /documents/{id}/X branches; /health red cluster -> SERVICE_UNAVAILABLE envelope - V2EnvelopeWriter: buffer reset on uncommitted error before write - V2JsonRequestParams: explicit *Initialized flags replace -1 sentinels Theme installer hardening - StaticThemeInstaller: incremental zip-bomb cumulative ratio check (defaults: ratio_max=50, threshold=65536 bytes), attic retention guard, ThemeManifestException Code enum for stable i18n codes - StaticThemeFilter / ThemeRegistry: warn-once on missing themes dir - AdminThemeAction: server-side upload size check; install-exception to i18n key mapping via Code enum Security / observability - SessionCsrfTokenManager: log session id as SHA-256 prefix hash (8 hex) - web.xml: staticThemeFilter ordered after httpHeaderSecurity / rateLimit / loadControl / cors Config / i18n - New: api.v2.chat.stream.keepalive.interval.ms (default 15000) - New: theme.upload.zip.ratio.max, theme.upload.zip.ratio.check.threshold.bytes - 21 locales: theme manifest error keys, zip-bomb-ratio key Tests - Added targeted unit tests across theme, /api/v2 handlers, helpers - UnitFessTestCase: JUnit 5 style assertEquals(int,int,String) overload - FessConfig.SimpleImpl: getRateLimitTrustedProxiesAsSet/WhitelistIpsAsSet /BlockedIpsAsSet + toStringSet helper (M-2 reverse-proxy XFF support)
…atch
The source-introspection assertion in
searchApiV2Manager_handlerThrowsAfterCommit_doesNotWriteEnvelope picked
up the IOException catch's log statement (added by the prior
IOException/Exception split) because it ran indexOf on the shared log
prefix "/api/v2 handler failed for". The IOException catch re-throws on
commit (semantically distinct from the broad catch's early-return), so
its `isCommitted()` precedes the log — leaving writeError before
isCommitted in the 2000-char window that follows the log anchor.
Switch the anchor to `} catch (final Exception e) {` so the window is
scoped to the broad Exception catch where the ordering invariant
(isCommitted -> return -> writeError) actually applies.
Silences the `no comment` javadoc warning on the private final Code field.
Cookie / session config — align with existing tomcat_config.properties mechanism so the default http://localhost:8080 OSS experience works: - Remove <secure>true</secure> from web.xml (broke HTTP localhost) - Delete META-INF/context.xml (was dead code; FessBoot overrides it) - Add sameSiteCookies = lax to tomcat_config.properties so the existing FessBoot.FessBootPropsTranslator path applies SameSite=Lax by default - HTTPS deployments add Secure at reverse proxy / Tomcat connector Response header consistency: - SearchApiV2Manager.writeHeaders(): apply api.json.response.headers config (Referrer-Policy etc.) to v2 envelopes — matching v1 - Extract SSE headers to new SseResponseHelper, used by both v1 ChatApiManager and v2 ChatStreamHandler - V2EnvelopeWriter error paths: set Cache-Control: no-store after resetBuffer (overwrites SSE prelude no-cache) - ThemeViewAction: drop redundant X-Content-Type-Options: nosniff (Tomcat HttpHeaderSecurityFilter provides it) Plus round-1 Critical/Major fixes: - CsrfRequirement: secure-by-default (unknown paths require token) - LoginHandler: account-switch audit log moved after credential check - StaticThemeInstaller: NOFOLLOW_LINKS + symlink rejection - V2EnvelopeWriter.writeSuccess: symmetric isCommitted/resetBuffer - ChatHandler/ChatStreamHandler/ClickHandler/FavoritePostHandler/ PasswordChangeHandler: split V2JsonBody multi-catch into 413/415/400 - ComponentUtil: getThemeRegistry / getStaticThemeInstaller accessors - AdminThemeAction: null-safe fileName + InstallException.Code mapping - ChatStreamHandler: client-disconnect detection (PrintWriter.checkError) - OpenAPI v2: drop stale same-user fast-path, add 429 to /auth/password, add already_existed to FavoritePostResponse - Bootstrap theme JS: script-order guard, SSE 1MB buffer cap, NetworkError handling, aria-live cleanup Tests: 25+ new test cases. Full suite: 6056 tests / 0 failures / 0 errors / 5 skipped.
…nused clamping flag
Post-merge spec audit found drift between openapi-user.yaml and the v2
handlers. Aligning the spec (not wire behavior) and removing one
unused code path:
- responses: add 413/415 to /auth/password, /click, /chat,
/chat/stream, /documents/{docId}/favorite POST (handlers already
emit these via V2JsonBody.PayloadTooLargeException /
UnsupportedMediaTypeException)
- responses: add 400/500 to /chat/sessions/{session_id} DELETE
(ChatSessionClearHandler emits both)
- /auth/login: note this endpoint deliberately folds 413/415 into 400
unlike the others
- info: document that CSRF verification runs before authentication, so
anonymous state-changing requests receive 403 rather than 401
- LoginRequest.return_to: note the protocol-relative `//` and ASCII
control-character rejection enforced by LoginHandler
- V2JsonRequestParams: remove the page_size_clamped field, setter
call, getter, AtomicBoolean import, and three Javadoc references
— no handler ever consumed the flag
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch introduces a static theme system and a new
/api/v2surface designed for headless single-page-app frontends, alongside a bundled reference theme implemented in Bootstrap 5 + vanilla ES2022 modules.themes.directory.path. Atomic install from ZIP with security guards (path traversal, max entries, max extracted size, compression ratio). Atomicized delete via an attic directory. Theme served by a dedicated filter + action./api/v2: new endpoint family with a unified envelope (status,schema,data/error). Covers search, suggest, labels, popular-words, NDJSON scroll, document cache, favorite GET/POST, click logging, authentication (login/logout/me/password), CSRF tokens, login rate limiting, UI config, and a chat pair (/chatPOST and/chat/streamSSE)./admin/theme/LastaFlute action with list / details / upload / delete / set-default / reload, JSPs, EN+JA i18n labels and messages, sidebar entry.index.htmlSPA shell, CSS overrides, EN/JA i18n bundles, thumbnail, README. Packaged into the WAR underthemes/bootstrap/.yuicompressor-maven-pluginexcludes theme JS so ES module syntax survives the build.api/v2/and the newtheme//helper/SessionCsrfTokenManager/filter/StaticThemeFilter/app/web/theme//app/web/admin/theme/namespaces.The work was decomposed across six implementation plans (core / v2-search / v2-actions / v2-chat / admin-ui / bootstrap-theme), each executed via the subagent-driven-development flow with two-stage spec + code review per task.
What changed
theme/Theme.java,ThemeType.java,ThemeManifest.java,ThemeManifestException.java,ThemeRegistry.java,StaticThemeInstaller.java,filter/StaticThemeFilter.java,helper/SessionCsrfTokenManager.java,app/web/theme/ThemeViewAction.java/api/v2frameworkapi/v2/SearchApiV2Manager.java,V2EnvelopeWriter.java,V2ErrorCode.javaBaseApiManager/api/v2handlersapi/v2/handlers/includingSearchHandler,ScrollSearchHandler,FavoriteGetHandler,FavoritePostHandler,CacheHandler,ClickHandler,LoginHandler,LogoutHandler,MeHandler,PasswordChangeHandler,UiConfigHandler,ChatHandler,ChatStreamHandler, helpers (V2JsonBody,V2JsonRequestParams,ChatRequestBody,CsrfRequirement,LoginRateLimiter)SearchApiV2Managerthinapp/web/admin/theme/AdminThemeAction.java+ fourFormclasses, three JSPs underwebapp/WEB-INF/view/admin/theme/, sidebar entry, EN+JA i18n labels & messagesStaticThemeInstaller.delete()with default-guardwebapp/themes/bootstrap/—theme.yml,index.html,assets/{api,i18n,auth,search,chat,app}.js,assets/styles.css,i18n/messages.{en,ja}.json,thumbnail.png,README.mdel(tag, opts)(createElement+textContent+setAttribute); onlyinnerHTML = ""(empty clear) usedweb.xml(StaticThemeFilter + FORWARD dispatcher on lastaToActionFilter),app.xml(4 DI components),fess_api.xml(manager),pom.xml(yuicompressor exclude + war webResources), 15 new keys infess_config.propertiesSecurity
api/v2/.ComponentUtil.getPasswordHashHelper().encode()(BCrypt), notFessLoginAssist.encryptPassword, per the project policy./auth/logout,/auth/password,/click,/documents/{id}/favorite,/chat,/chat/stream); explicitly exempt for/auth/login. Rule table inCsrfRequirement.java.innerHTML = '<...>'(dynamic-string) assignment anywhere; all dynamic data rendered viadocument.createElement+textContent+setAttribute.git diff origin/master -- src/main/java/org/codelibs/fess/api/json/is empty.Test plan
org.codelibs.fess.api.v2.**Test): 117 tests, 0 failures, 1 disabled-by-designorg.codelibs.fess.theme.**Test+ filter + admin action + view action + helper): greenmvn package -DskipTests); 407 MB; 16themes/bootstrap/*entriesnode --checkfor all 6 filesinnerHTMLassignmentsThemeRegistryunder 8 reader threads × 200 iterations + 2 concurrent reloads — no exceptionsMigration / compatibility notes
theme.directory.pathand upload artifacts.web.xmladds<dispatcher>FORWARD</dispatcher>to the existinglastaToActionFiltermapping so theme-forwarded requests still flow through LastaFlute. This is the only edit to a pre-existing servlet wiring element and is exercised by the full test suite.pom.xmlexcludes theme JS from yuicompressor, so themes can ship native ES2022 module syntax.@Disabledtests inAdminThemeActionTestmark cases that need the LastaFlute container — left as a follow-up; current coverage exercises form regex and exception-mapping paths.Implementation plans (workspace docs, not in repo)
The implementation was driven by six plan documents committed to the parent fess-workspace; they are not part of this PR:
2026-05-21-fess-static-theme-core.md2026-05-21-fess-static-theme-v2-search.md2026-05-21-fess-static-theme-v2-actions.md2026-05-21-fess-static-theme-v2-chat.md2026-05-21-fess-static-theme-admin-ui.md2026-05-21-fess-static-theme-bootstrap-theme.md