diff --git a/docs/admins/operations/monitoring-basics.md b/docs/admins/operations/monitoring-basics.md index d6345bd..97a5623 100644 --- a/docs/admins/operations/monitoring-basics.md +++ b/docs/admins/operations/monitoring-basics.md @@ -20,7 +20,7 @@ If a new environment namespace is added, update the `targetNamespaces` and `sess ### Shared build cache metrics -The `theia-shared-cache` (Gradle build cache) exposes Prometheus metrics at `/metrics` via a Redis Exporter sidecar. Enable the ServiceMonitor for it in the chart values: +The `theia-shared-cache` (Gradle and Bazel build cache) exposes Prometheus metrics at `/metrics` via a Redis Exporter sidecar. Enable the ServiceMonitor for it in the chart values: ```yaml metrics: @@ -75,9 +75,13 @@ Monitor: `kube_resourcequota` for `persistentvolumeclaims` and `requests.storage ### Build cache hit rate -A low cache hit rate for the Gradle shared cache degrades CI build times but does not affect user sessions directly. +A low cache hit rate for the shared cache degrades CI build times but does not affect user sessions directly. Gradle and Bazel are tracked with separate counters. -Monitor: `gradle_cache_cache_hits` / (`gradle_cache_cache_hits` + `gradle_cache_cache_misses`). +Monitor (Gradle): `gradle_cache_cache_hits` / (`gradle_cache_cache_hits` + `gradle_cache_cache_misses`). + +Monitor (Bazel): `bazel_cache_cache_hits` / (`bazel_cache_cache_hits` + `bazel_cache_cache_misses`). + +Also watch `bazel_cache_hash_mismatches` — a non-zero rate indicates Bazel CAS uploads failing content-hash verification and should be investigated. **Informational threshold:** < 50% over a 24-hour window warrants investigation. diff --git a/docs/admins/platform/provisioning.md b/docs/admins/platform/provisioning.md index 490ee55..f8a6bbc 100644 --- a/docs/admins/platform/provisioning.md +++ b/docs/admins/platform/provisioning.md @@ -29,14 +29,13 @@ Before a first deployment to a new cluster, confirm the following are in place: - Helm 3 is installed on the runner (handled automatically in CI) - You have cluster-admin permissions - **`cert-manager`** is installed on the cluster — manages TLS certificates via the `cert-manager.io` CRDs -- **`trust-manager`** is installed on the cluster — distributes the internal CA trust bundle to target namespaces via the `trust.cert-manager.io` CRDs (required by `theia-internal-tls`) - A Keycloak instance is running and you have admin access to the relevant realm - DNS entries for the environment domain are configured and propagating - GitHub environment secrets are set (see [GitHub environment secrets](#github-environment-secrets)) -### Installing cert-manager and trust-manager +### Installing cert-manager -These are cluster-level prerequisites installed once. If they are already present, skip this. +This is a cluster-level prerequisite installed once. If it is already present, skip this. ```bash # cert-manager (check https://cert-manager.io for the latest version) @@ -44,13 +43,9 @@ helm upgrade --install cert-manager jetstack/cert-manager \ --namespace cert-manager --create-namespace \ --set crds.enabled=true \ --set config.enableGatewayAPI=true - -# trust-manager (part of the cert-manager project) -helm upgrade --install trust-manager jetstack/trust-manager \ - --namespace cert-manager ``` -Verify both are running: +Verify it is running: ```bash kubectl get pods -n cert-manager ``` @@ -64,7 +59,6 @@ EduIDE is deployed through a set of layered Helm charts. Cluster-scoped charts a | `theia-cloud-crds` | Cluster | Custom resource definitions for workspaces, sessions, app definitions | | `theia-cloud-base` | Cluster | Cluster-wide shared resources | | `theia-shared-gateway` | Cluster | Shared Envoy Gateway API entry point | -| `theia-internal-tls` | Cluster | Internal CA and trust bundle (requires trust-manager) | | `theia-cloud-combined` | Environment | Main application: service, operator, landing page, OAuth2 proxy | | `theia-appdefinitions` | Environment | IDE session type definitions | | `theia-certificates` | Environment | TLS certificate management for the environment domain | @@ -72,7 +66,7 @@ EduIDE is deployed through a set of layered Helm charts. Cluster-scoped charts a ## Step 1: Install cluster-scoped charts -These are installed once per cluster. The pipeline handles this via the `deploy_shared_gateway` input flag and the internal TLS step. If setting up a brand new cluster manually: +These are installed once per cluster. The pipeline handles this via the `deploy_shared_gateway` input flag. If setting up a brand new cluster manually: ```bash # CRDs first — operator and service depend on these @@ -85,10 +79,6 @@ helm upgrade --install theia-cloud-base ./charts/theia-cloud-base helm upgrade --install theia-shared-gateway ./charts/theia-shared-gateway \ -n gateway-system --create-namespace \ -f - -# Internal TLS CA and trust bundle (namespace: cert-manager) -helm upgrade --install theia-internal-tls ./charts/theia-internal-tls \ - -n cert-manager ``` ## Step 2: Prepare Keycloak diff --git a/docs/developer/projects/eduide-shared-cache.md b/docs/developer/projects/eduide-shared-cache.md index 0d60dc6..8243589 100644 --- a/docs/developer/projects/eduide-shared-cache.md +++ b/docs/developer/projects/eduide-shared-cache.md @@ -7,13 +7,15 @@ sidebar_label: EduIDE Shared Cache ## Overview -The **EduIDE Shared Cache** repository provides a high-performance, Kubernetes-native HTTP build cache server specifically designed for Gradle builds. It implements the Gradle HTTP Build Cache API, allowing development teams and CI/CD pipelines to share build artifacts, which significantly reduces build times and resource consumption across the EduIDE ecosystem. +The **EduIDE Shared Cache** repository provides a high-performance, Kubernetes-native HTTP build cache server for both **Gradle** and **Bazel** builds. It implements the Gradle HTTP Build Cache API and the Bazel HTTP remote cache protocol (action cache + content-addressable store), allowing student sessions and CI/CD pipelines to share build artifacts through a single backend. This significantly reduces build times and resource consumption across the EduIDE ecosystem. ## Key Features -- **Gradle Compatibility**: Fully implements the Gradle HTTP Build Cache protocol for seamless integration. +- **Gradle and Bazel Compatibility**: Implements both the Gradle HTTP Build Cache protocol and the Bazel HTTP remote cache (AC + CAS) through one service. - **In-Memory Storage**: Uses Redis for fast cache lookups and storage of build artifacts. -- **Role-Based Authentication**: HTTP Basic Authentication with separate reader and writer roles for fine-grained access control. +- **Role-Based Authentication**: HTTP Basic Authentication with separate reader (read-only) and writer (read/write) roles for fine-grained access control. +- **Artifact Validation**: Optional static analysis of uploaded Gradle artifacts, inspecting Java bytecode for forbidden API usage (network, process execution, reflection, filesystem) before an entry is admitted to the cache. +- **CAS Integrity**: Optional content-hash verification for Bazel CAS uploads. - **Kubernetes-Native**: Designed with containerization in mind, including production-ready Helm charts for easy deployment. - **Observability**: Built-in Prometheus metrics, pre-built Grafana dashboard, Redis exporter sidecar, and structured JSON logging. - **Dependency Proxy**: Optional Reposilite integration for caching Maven/Gradle dependencies. @@ -38,13 +40,17 @@ The **EduIDE Shared Cache** repository provides a high-performance, Kubernetes-n | `/ping` | GET | No | Health check (liveness probe) | | `/health` | GET | No | Storage connectivity check (readiness probe) | | `/metrics` | GET | No | Prometheus metrics | -| `/cache/:key` | GET | reader/writer | Retrieve cache entry | -| `/cache/:key` | HEAD | reader/writer | Check if cache entry exists | -| `/cache/:key` | PUT | writer only | Store cache entry | +| `/gradle/:key` | GET | reader/writer | Retrieve a Gradle cache entry | +| `/gradle/:key` | HEAD | reader/writer | Check if a Gradle cache entry exists | +| `/gradle/:key` | PUT | writer only | Store a Gradle cache entry (subject to artifact validation) | +| `/bazel/ac/:hash` | GET | reader/writer | Retrieve a Bazel action-cache entry | +| `/bazel/ac/:hash` | PUT | writer only | Store a Bazel action-cache entry | +| `/bazel/cas/:hash` | GET | reader/writer | Retrieve a Bazel CAS blob | +| `/bazel/cas/:hash` | PUT | writer only | Store a Bazel CAS blob (optional hash verification) | ## Integration -In the EduIDE ecosystem, **EduIDE Shared Cache** serves as a shared Gradle build cache for students working in cloud IDE sessions. When a student triggers a Gradle build, previously computed build outputs are served from the shared cache, significantly reducing build times. Since many students work on the same exercise templates, cache hit rates are high and redundant compilation work is avoided across sessions. +In the EduIDE ecosystem, **EduIDE Shared Cache** serves as a shared build cache for students working in cloud IDE sessions. When a student triggers a Gradle or Bazel build, previously computed build outputs are served from the shared cache, significantly reducing build times. Since many students work on the same exercise templates, cache hit rates are high and redundant compilation work is avoided across sessions. ## Repository diff --git a/docs/developer/projects/projects-overview.md b/docs/developer/projects/projects-overview.md index 9089e7f..b1165d7 100644 --- a/docs/developer/projects/projects-overview.md +++ b/docs/developer/projects/projects-overview.md @@ -11,7 +11,7 @@ The developer section includes the project catalog that previously lived in the - [EduIDE Deployment](/developer/projects/eduide-deployment): Infrastructure-as-code and deployment workflows for EduIDE environments. - [Theia Scale Tests](/developer/projects/theia-scale-tests): End-to-end and scalability validation for classroom-scale IDE usage. -- [EduIDE Shared Cache](/developer/projects/eduide-shared-cache): Shared Gradle build cache service for faster builds. +- [EduIDE Shared Cache](/developer/projects/eduide-shared-cache): Shared Gradle and Bazel build cache service for faster builds. - [EduIDE LSP Extension](/developer/projects/eduide-lsp-extension): Remote language-server connector for Theia and VS Code. - [EduIDE Data Bridge](/developer/projects/eduide-data-bridge): Runtime data and credential injection bridge for cloud IDE sessions. - [EduIDE](/developer/projects/eduide): Custom Eclipse Theia Docker images and packaged IDE environments.