Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 30 additions & 25 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,27 +27,33 @@ jobs:
# The -Pbenchmark profile DECLARES TWELVE k6 goals: the eight cross-gateway matrix aspects, the
# two retained non-matrix health benchmarks (healthLiveCheck, gatewayHealth), and the two
# API-Sheriff-only passthrough-relay executions (mapped relay throughput and the empty-mode
# no-regression run). TEN of them execute: uploadLarge and websocketEcho each carry a <skip>
# bound to a property that defaults to true in benchmarks/pom.xml, because each is blocked by an
# open gateway defect (see the "Deliberately skipped" table in the coverage step below). Maven
# therefore never runs them, so they consume no wall time and cannot abort the suite. Both are
# still ordered last in the profile as defense-in-depth, so that flipping a skip property to
# false cannot mask a goal that would otherwise produce a result.
# no-regression run). ELEVEN of them execute: only uploadLarge still carries a <skip> bound to a
# property that defaults to true in benchmarks/pom.xml, because it remains blocked by an open
# gateway defect (see the "Deliberately skipped" table in the coverage step below). Maven
# therefore never runs it, so it consumes no wall time and cannot abort the suite. It stays
# ordered last in the profile as defense-in-depth, so that flipping its skip property to false
# cannot mask a goal that would otherwise produce a result.
#
# Budget, from per-goal timings measured locally 2026-07-28 rather than estimated:
# * goal execution ~12 min -- sized on the TEN goals that actually execute, which is now the
# * goal execution ~13 min -- sized on the ELEVEN goals that actually execute, which is the
# real basis: a skipped goal is not run at all, so it costs nothing but the
# Maven bookkeeping for a skipped execution.
# 10 goals x (60s k6 window + ~10s compose/k6 start and summary write).
# Re-enabling a skipped goal costs another ~70s, and upload-50MB more than
# that: it is transfer-bound at reduced concurrency, so it does not finish
# in the same wall time as a request-rate run. Re-derive this term when a
# skip property is flipped to false.
# * stack startup ~90 s -- since #118 the lane boots FOUR native gateway instances
# (api-sheriff, api-sheriff-mtls, api-sheriff-cookie, api-sheriff-cookie-2)
# alongside Keycloak, go-httpbin, nginx-static, passthrough-backend,
# grpc-echo, toxiproxy, asset-origin and prometheus, because
# start-integration-container.sh runs a bare `up -d`.
# 11 goals x (60s k6 window + ~10s compose/k6 start and summary write).
# The websocketEcho term is the ~70s a re-enabled request-rate goal costs,
# now that its blocking permit leak is fixed. Re-enabling upload-50MB would
# cost more than that: it is transfer-bound at reduced concurrency, so it
# does not finish in the same wall time as a request-rate run. Re-derive
# this term when a skip property is flipped to false.
# * stack startup ~100 s -- the lane boots FIVE native gateway instances (api-sheriff,
# api-sheriff-mtls, api-sheriff-cookie, api-sheriff-cookie-2 and
# api-sheriff-ws-admission, the last added for the WebSocket relay-permit
# exhaustion regression) alongside Keycloak, go-httpbin, nginx-static,
# passthrough-backend, grpc-echo, toxiproxy, asset-origin and prometheus,
# because start-integration-container.sh runs a bare `up -d`. Each added
# gateway instance costs roughly another 10 s of the readiness wait — they
# share one native image, so the term grows with instance count, not with
# image builds. Re-derive this term whenever an api-sheriff* service is
# added to or removed from integration-tests/docker-compose.yml.
# * native compile the dominant and most variable term on a cold cache.
#
# 75 minutes holds with headroom for the cold native compile; it is kept unchanged because the
Expand Down Expand Up @@ -113,10 +119,10 @@ jobs:
# twelve goals: the eight cross-gateway matrix aspects (unauth, bearer, http2,
# graphql, upload-1MB, upload-50MB, ws, grpc), the two retained non-matrix health
# benchmarks (healthLiveCheck, gatewayHealth), and the two API-Sheriff-only
# passthrough-relay executions (mapped and empty). Ten of the twelve execute and are
# expected to produce a CI result; upload-50MB (uploadLarge) and ws (websocketEcho) are
# skipped by property against open gateway defects, so Maven does not run them and they
# cannot abort the suite. The on-demand APISIX comparison lane is deliberately NOT run
# passthrough-relay executions (mapped and empty). Eleven of the twelve execute and are
# expected to produce a CI result; only upload-50MB (uploadLarge) is still skipped by
# property against an open gateway defect, so Maven does not run it and it cannot abort
# the suite. The on-demand APISIX comparison lane is deliberately NOT run
# here. Maven still fails fast on the goals that do run, so the coverage step below
# remains what proves which goals actually produced a result.
echo "Running k6 integration benchmarks with native Quarkus..."
Expand All @@ -128,8 +134,8 @@ jobs:
ls -la benchmarks/target/benchmark-results/

# Maven fails fast, so a goal that errors aborts every goal after it and a goal that is never
# reached leaves no trace in the job output — a suite that ran 3 of the 10 executing goals
# renders exactly like one that ran all 10. This step diffs the goals that MUST produce a
# reached leaves no trace in the job output — a suite that ran 3 of the 11 executing goals
# renders exactly like one that ran all 11. This step diffs the goals that MUST produce a
# summary document against the documents k6 actually wrote, so a partial suite is visibly
# partial and a goal that silently vanished fails the job. When Maven failed, it also states
# in words whether the suite was truncated by that failure. Skipped goals are named with their
Expand All @@ -145,7 +151,7 @@ jobs:

# Goals expected to write a summary. A goal settled as deliberately skipped is NOT listed
# here — it belongs in the skip table below, which is what keeps the two states distinct.
expected="healthLiveCheck gatewayHealth proxiedStatic passthroughRelay passthroughRelayEmpty bearerProxied http2 graphql uploadSmall grpcUnary"
expected="healthLiveCheck gatewayHealth proxiedStatic passthroughRelay passthroughRelayEmpty bearerProxied http2 graphql uploadSmall grpcUnary websocketEcho"
Comment thread
coderabbitai[bot] marked this conversation as resolved.

missing=0
missing_names=""
Expand Down Expand Up @@ -173,7 +179,6 @@ jobs:
echo "| Benchmark | Skipped by | Reason |"
echo "| --- | --- | --- |"
echo "| \`uploadLarge\` | \`skip.benchmark.upload.large\` (defaults to \`true\`) | The gateway rejects the 50MB body with 413: the Quarkus HTTP body limit is not derived from the upload anchor's 64 MiB \`max_body_bytes\`, so the declared cap is unreachable. Needs a gateway-side fix; re-enable with \`-Dskip.benchmark.upload.large=false\`. |"
echo "| \`websocketEcho\` | \`skip.benchmark.websocket.echo\` (defaults to \`true\`) | Each WebSocket upgrade leaks an edge admission permit; once exhausted the gateway rejects all traffic with 503 until restarted, which would also poison every goal after it. Needs a gateway-side fix; re-enable with \`-Dskip.benchmark.websocket.echo=false\`. |"
echo "| \`sessionMediated\` | not wired | Wired to no Maven goal on purpose — BFF session mediation belongs to PLAN-07A. |"
} >> "$GITHUB_STEP_SUMMARY"

Expand Down
4 changes: 3 additions & 1 deletion .plan/project-architecture/benchmarks/enriched.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
{
"best_practices": [],
"insights": [],
"insights": [
"The benchmark coverage 'expected=' list in .github/workflows/benchmark.yml is an independent oracle, not a mirror of the run-k6-* executions in benchmarks/pom.xml. Review-bot suggestions to derive it from the pom are declined: deriving it would make a goal deleted from the pom vanish from both sides and the coverage step go green, losing the silent-deletion detection that is the step's main reason to exist. Skip state is also property-driven and CLI-overridable, so a naive derivation would demand a summary for a deliberately skipped goal. Maintain the list by hand when a goal is added or un-skipped."
],
Comment thread
coderabbitai[bot] marked this conversation as resolved.
"internal_dependencies": [],
"key_dependencies": [
"de.cuioss:benchmarking-common",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
/*
* Copyright © 2026 CUI-OpenSource-Software (info@cuioss.de)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package de.cuioss.sheriff.gateway.config.model;

import java.util.Objects;
import java.util.Optional;

import lombok.Builder;

/**
* The operator-facing {@code edge_hardening} block: the gateway's admission budget.
* <p>
* <strong>Two caps, not one.</strong> {@code admission_cap} bounds how many requests may be in flight
* across the whole edge at once. {@code websocket_relay_cap} is a <em>sub-budget acquired in addition
* to</em> that general permit, never instead of it, and exists because a WebSocket relay holds its
* admission permit for the connection's entire lifetime rather than for one request/response
* round-trip. Without the sub-budget a modest number of long-lived relays would consume the general
* pool and starve ordinary HTTP traffic; with it, WebSocket pressure is bounded independently and
* HTTP keeps the remaining headroom. A {@code websocket_relay_cap} larger than {@code admission_cap}
* is meaningless — the sub-budget can never bind — and is refused at boot by the config validator.
* <p>
* Both members are optional and are independently omissible. An omitted block resolves to the
* {@link #defaults() documented defaults}, so an operator who never writes the block keeps today's
* behaviour. An omitted {@code websocket_relay_cap} resolves to a quarter of the <em>effective</em>
* {@code admission_cap} rather than to a fixed constant, so lowering {@code admission_cap} alone
* keeps the pair consistent — the relay sub-budget follows the pool it draws from instead of
* overshooting it and self-rejecting at boot. Values are validated at boot (both {@code >= 1});
* this record only carries them.
*
* @param admissionCap the maximum number of concurrently in-flight requests the edge admits,
* empty when the operator did not declare one
* @param websocketRelayCap the maximum number of concurrently established WebSocket relays, empty
* when the operator did not declare one
* @author API Sheriff Team
* @since 1.0
*/
@Builder
public record EdgeHardeningConfig(Optional<Integer> admissionCap, Optional<Integer> websocketRelayCap) {

/** The admission cap applied when the operator declares none — the gateway's historical bound. */
public static final int DEFAULT_ADMISSION_CAP = 2048;

/**
* The WebSocket-relay sub-budget carried by {@link #defaults()}: a quarter of
* {@link #DEFAULT_ADMISSION_CAP}. A relay holds its permit for the connection's lifetime, so the
* default deliberately leaves three quarters of the general pool for ordinary request/response
* traffic while still admitting far more concurrent relays than a typical deployment sustains.
* When only {@code admission_cap} is declared, {@link #effectiveWebsocketRelayCap()} derives the
* same quarter proportion from <em>that</em> cap rather than reading this constant, so the ratio
* — not the literal value — is what the default preserves.
*/
public static final int DEFAULT_WEBSOCKET_RELAY_CAP = 512;

/** Canonical constructor normalizing absent optionals to {@link Optional#empty()}. */
public EdgeHardeningConfig {
admissionCap = Objects.requireNonNullElse(admissionCap, Optional.empty());
websocketRelayCap = Objects.requireNonNullElse(websocketRelayCap, Optional.empty());
}

/**
* @return the block an omitted {@code edge_hardening} resolves to — both caps at their documented
* defaults, preserving the behaviour of a gateway that never declares the block
*/
public static EdgeHardeningConfig defaults() {
return new EdgeHardeningConfig(Optional.of(DEFAULT_ADMISSION_CAP),
Optional.of(DEFAULT_WEBSOCKET_RELAY_CAP));
}

/**
* @return the declared admission cap, or {@link #DEFAULT_ADMISSION_CAP} when the member is absent
*/
public int effectiveAdmissionCap() {
return admissionCap.orElse(DEFAULT_ADMISSION_CAP);
}

/**
* The implicit relay sub-budget is derived from the pool it draws from rather than fixed, so a
* partially declared block can never resolve to a relay cap above its own admission cap: with
* {@code admission_cap: 64} and no {@code websocket_relay_cap}, this yields {@code 16} instead of
* a self-rejecting {@code 512}. At the shipped {@link #DEFAULT_ADMISSION_CAP} the quarter is
* exactly {@link #DEFAULT_WEBSOCKET_RELAY_CAP}. An explicitly declared cap is returned untouched
* — an explicit relay cap above the admission cap remains a boot-time error.
*
* @return the declared WebSocket-relay sub-budget, or — when the member is absent — a quarter of
* the {@linkplain #effectiveAdmissionCap() effective admission cap}, never below
* {@code 1}
*/
public int effectiveWebsocketRelayCap() {
return websocketRelayCap.orElseGet(() -> Math.max(1, effectiveAdmissionCap() / 4));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,17 @@
* @param forwarded the forwarded-header trust policy, empty when omitted
* @param tokenValidation the offline bearer-validation settings, empty when omitted
* @param oidc the confidential-client settings, empty when omitted
* @param edgeHardening the admission-budget settings ({@code admission_cap} and the WebSocket
* relay sub-budget), empty when omitted — the documented defaults then apply
* @author API Sheriff Team
* @since 1.0
*/
@Builder
public record GatewayConfig(int version, Optional<Metadata> metadata, Optional<TlsConfig> tls,
Optional<SecurityHeadersConfig> securityHeaders, Optional<SecurityDefaultsConfig> securityDefaults,
List<HttpMethod> allowedMethods, Map<String, AnchorConfig> anchors, Optional<UpstreamDefaultsConfig> upstreamDefaults,
Optional<ForwardedConfig> forwarded, Optional<TokenValidationConfig> tokenValidation, Optional<OidcConfig> oidc) {
Optional<ForwardedConfig> forwarded, Optional<TokenValidationConfig> tokenValidation, Optional<OidcConfig> oidc,
Optional<EdgeHardeningConfig> edgeHardening) {

/**
* Canonical constructor defensively copying {@code allowedMethods} and
Expand All @@ -73,5 +76,6 @@ public record GatewayConfig(int version, Optional<Metadata> metadata, Optional<T
forwarded = Objects.requireNonNullElse(forwarded, Optional.empty());
tokenValidation = Objects.requireNonNullElse(tokenValidation, Optional.empty());
oidc = Objects.requireNonNullElse(oidc, Optional.empty());
edgeHardening = Objects.requireNonNullElse(edgeHardening, Optional.empty());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ public final class ConfigValidator {
private static final String ENDPOINT_ANCHOR_POINTER = "/endpoint/anchor";
private static final String ENDPOINT_ROUTES_POINTER = "/endpoint/routes";
private static final String FORWARDED_TRUSTED_POINTER = "/forwarded/trusted_proxies";
private static final String EDGE_HARDENING_ADMISSION_POINTER = "/edge_hardening/admission_cap";
private static final String EDGE_HARDENING_WEBSOCKET_POINTER = "/edge_hardening/websocket_relay_cap";
private static final int IPV4_BITS = 32;
private static final int IPV6_BITS = 128;
private static final int BROAD_PREFIX_IPV4 = 8;
Expand Down Expand Up @@ -141,7 +143,8 @@ public final class ConfigValidator {
(gateway, endpoints, topology, errors) -> validateLoginPath(gateway, errors),
(gateway, endpoints, topology, errors) -> validatePassthroughHostCollision(gateway, endpoints, errors),
(gateway, endpoints, topology, errors) -> validatePassthroughAliasResolvable(gateway, topology, errors),
(gateway, endpoints, topology, errors) -> validateWebSocketConfig(gateway, endpoints, errors));
(gateway, endpoints, topology, errors) -> validateWebSocketConfig(gateway, endpoints, errors),
(gateway, endpoints, topology, errors) -> validateEdgeHardening(gateway, errors));

private final List<ValidationRule> rules;

Expand Down Expand Up @@ -185,6 +188,34 @@ private static void validateVersion(GatewayConfig gateway, List<ConfigError> err
}
}

/**
* Bounds the operator-facing admission budget. Both caps must admit at least one request — a cap
* of zero or below would refuse every request (or every relay) and is always a misconfiguration
* rather than a deliberate posture. The WebSocket sub-budget must additionally stay within the
* general pool it draws from: a {@code websocket_relay_cap} above {@code admission_cap} can never
* bind, so it would silently disable the sub-cap the operator meant to impose.
*/
private static void validateEdgeHardening(GatewayConfig gateway, List<ConfigError> errors) {
gateway.edgeHardening().ifPresent(hardening -> {
hardening.admissionCap()
.filter(cap -> cap < 1)
.ifPresent(cap -> errors.add(new ConfigError(GATEWAY_FILE, EDGE_HARDENING_ADMISSION_POINTER,
"admission_cap must be at least 1 (was %d)".formatted(cap))));
hardening.websocketRelayCap()
.filter(cap -> cap < 1)
.ifPresent(cap -> errors.add(new ConfigError(GATEWAY_FILE, EDGE_HARDENING_WEBSOCKET_POINTER,
"websocket_relay_cap must be at least 1 (was %d)".formatted(cap))));
int admission = hardening.effectiveAdmissionCap();
int relay = hardening.effectiveWebsocketRelayCap();
if (admission >= 1 && relay >= 1 && relay > admission) {
errors.add(new ConfigError(GATEWAY_FILE, EDGE_HARDENING_WEBSOCKET_POINTER,
"websocket_relay_cap %d must not exceed admission_cap %d — the relay budget is a "
.formatted(relay, admission)
+ "sub-budget of the admission pool and could never bind"));
}
});
}

private static void validateEndpointIdUniqueness(List<EndpointConfig> endpoints, List<ConfigError> errors) {
Set<String> seen = new HashSet<>();
for (EndpointConfig endpoint : endpoints) {
Expand Down
Loading
Loading