Report Core process metrics and node host history

CONTRIBUTING.md

@@ -218,6 +218,19 @@ response; never record request/response bodies or infer this count from every
 service or storage system is required. Keep the frontend response shape aligned
 with the paired console contract.
 
+Core process CPU, RSS and cgroup limits are sampled by the existing 30-second
+Core metrics loop; Go heap and goroutine reads retain their meaning. Process
+series use the same bounded ring and complete buckets, with null first CPU
+intervals and restart gaps. Do not substitute host usage for process usage.
+
+Administrator node detail adds host observations and history as documented in
+[node-host-history.md](contracts/agents-api/node-host-history.md). Keep the node
+list unchanged. Reuse authenticated heartbeat ownership, the Runtime sampling
+sweep and PostgreSQL retention cleanup; node observations have their own table
+because they do not belong to a Project, Session or Environment. History is
+best-effort telemetry, never scheduling truth. No read-triggered sampling or
+offline backfill is allowed.
+
 ## Architecture boundaries
 
 The following execution rules are retained from the source contributor guide.

contracts/agents-api/core-metrics.md

@@ -26,10 +26,11 @@ the current partial bucket is excluded from range aggregates and series.
 | 7d | 7200 seconds | 84 |
 
 Current gauges and range aggregates are intentionally different: current worker,
-connection pool and process values are read when requested; queue gauges, database
-size and deployment maintenance are sampled every 30 seconds with bounded I/O.
+connection pool, Go heap and goroutine values are read when requested; process
+CPU, RSS and limits, queue gauges, database size and deployment maintenance are
+sampled every 30 seconds with bounded I/O.
 Samples older than 60 seconds are not reported as current. Queue, running and
-pool series report the highest **observed** value in each bucket, not a claim
+pool and process series report the highest **observed** value in each bucket, not a claim
 that all intermediate peaks were captured. Missing observations and the process's
 partial first bucket stay null. Successful periodic ping samples produce linear
 interpolated p50/p95; there is no request-triggered ping.
@@ -79,6 +80,24 @@ present.
   one value does not turn it into zero.
 - `process.memory_bytes` is Go `runtime.MemStats.Alloc` (allocated heap bytes),
   not RSS or container memory. `goroutines` is `runtime.NumGoroutine()`.
+- `process.cpu_cores` is the increase in this process's user plus system CPU
+  time divided by elapsed sampling time. Linux uses `getrusage(RUSAGE_SELF)`;
+  subprocess and whole-host CPU are excluded. The first interval is null.
+  Missing/invalid counters, counter resets, nonpositive elapsed time and gaps
+  longer than 60 seconds reset the baseline; they never manufacture a zero.
+- `process.rss_bytes` is Linux `/proc/self/status` `VmRSS`, converted from KiB
+  to bytes. `cpu_limit_cores` is the process's cgroup v2 `cpu.max` quota/period,
+  or `GOMAXPROCS` when its quota is `max` or the actual cgroup root has no quota interface. `memory_limit_bytes` is that cgroup's
+  finite `memory.max`; `max` is null. Membership and mount information resolve
+  the process's cgroup, including nested and subtree mounts. These are that
+  cgroup's configured limits, not whole-host metrics or ancestor-limit discovery.
+  Unreadable or malformed values remain null. Non-Linux builds report null CPU,
+  RSS and memory limit, with `GOMAXPROCS` as CPU capacity.
+- `process.series` always contains the range's complete buckets with `start`,
+  `cpu_cores` and `rss_bytes`. Each measurement uses its own observed maximum;
+  missing samples and the partial first bucket stay null. These samples share
+  the existing bounded ring and restart gaps. Unsupported process measurements
+  do not by themselves change execution or database health.
 
 ## Background jobs
 

contracts/agents-api/node-host-history.md

@@ -0,0 +1,69 @@
+# Administrator node host observations
+
+`GET /core/v1/sandbox/nodes/{node_id}?range=1h|6h|24h` requires deployment
+administrator authentication. Core Web forwards it through its authenticated
+server; Project, enrollment and node credentials do not authorize this read.
+The omitted range defaults to `1h`. Invalid, repeated or unknown query parameters
+return the existing invalid-input response; missing or removed nodes return 404.
+
+The response contains the existing node fields plus these two objects. The node
+list response is unchanged. This endpoint does not alter enrollment, administrator
+capacity approval, scheduling or any public Agents API contract.
+
+```json
+{
+  "host": {
+    "effective_cpu_cores": 4,
+    "cpu_utilization": 0.35,
+    "total_memory_bytes": 17179869184,
+    "available_memory_bytes": 8589934592,
+    "available_disk_bytes": 107374182400,
+    "observed_at": "2026-09-25T09:00:00Z"
+  },
+  "history": {
+    "resolution_seconds": 60,
+    "points": [{
+      "start": "2026-09-25T08:59:00Z",
+      "cpu_utilization_max": 0.4,
+      "memory_used_bytes_max": 8589934592,
+      "available_disk_bytes_min": 107374182400
+    }]
+  }
+}
+```
+
+Every unavailable measurement is `null`, including unobserved timestamps. `host`
+is the last received observation; an offline node retains its original timestamp
+and last-known measurements. Clients must use `online` and `observed_at` when
+presenting freshness. Reading the endpoint never samples or writes history.
+
+On Linux, CPU utilization is the increase in aggregate busy `/proc/stat` ticks
+divided by total ticks between heartbeats, across the whole visible host. Idle
+and I/O-wait ticks are not busy; guest counters are not counted twice. The first
+observation and a reset or unavailable baseline have null utilization. This is
+not node-process CPU or the sum of sandbox utilization. Total memory and available
+memory come from `MemTotal` and `MemAvailable`. Effective CPU capacity accounts for
+observable process affinity and cgroup limits; it is null if those limits cannot
+be established, rather than an invented scheduling limit. Available disk refers
+to the node's existing state filesystem, not a sandbox quota. The node should run
+on the machine being monitored; namespace visibility limits what it can observe.
+
+The existing Runtime sampling sweep copies fresh, authenticated node heartbeats
+into `node_host_history_samples` in the same PostgreSQL database. It uses the
+Runtime history cadence (30 seconds by default) and seven-day retention cleanup.
+A node observation is keyed by node and its original timestamp, so repeated sweeps
+do not manufacture additional observations. Disconnected, stale, future-dated or
+removed observations are not copied. The sampler is best effort with a bounded
+query timeout; failures do not change execution or admission authority.
+
+Ranges use complete UTC buckets: `1h` has 60-second buckets, `6h` has 300-second
+buckets and `24h` has 900-second buckets. Each response includes every bucket in
+the range. CPU and memory are maxima of recorded observations; used memory is
+`total_memory_bytes - available_memory_bytes` from the same observation. Disk is
+the minimum recorded available space. Missing measurements and offline buckets
+stay null, independently for each metric. The active partial bucket is excluded.
+No interpolation, offline backfill or whole-host process inventory is performed.
+
+History survives Core and node restarts until normal retention expiry. Core
+process history remains a separate in-memory series and resets when Core restarts.
+Neither series contains credentials, model data, paths or user content.

contracts/agents-api/sandbox-manager.openapi.yaml

@@ -304,12 +304,39 @@ definitions:
     type: object
   coremetrics.Process:
     properties:
+      cpu_cores:
+        type: number
+        x-nullable: true
+      cpu_limit_cores:
+        type: number
+        x-nullable: true
       goroutines:
         type: integer
         x-nullable: true
       memory_bytes:
         type: integer
         x-nullable: true
+      memory_limit_bytes:
+        type: integer
+        x-nullable: true
+      rss_bytes:
+        type: integer
+        x-nullable: true
+      series:
+        items:
+          $ref: '#/definitions/coremetrics.ProcessBucket'
+        type: array
+    type: object
+  coremetrics.ProcessBucket:
+    properties:
+      cpu_cores:
+        type: number
+        x-nullable: true
+      rss_bytes:
+        type: integer
+        x-nullable: true
+      start:
+        type: string
     type: object
   coremetrics.Range:
     properties:
@@ -516,6 +543,29 @@ definitions:
       state:
         type: string
     type: object
+  store.NodeHostHistory:
+    properties:
+      points:
+        items:
+          $ref: '#/definitions/store.NodeHostHistoryPoint'
+        type: array
+      resolution_seconds:
+        type: integer
+    type: object
+  store.NodeHostHistoryPoint:
+    properties:
+      available_disk_bytes_min:
+        type: integer
+        x-nullable: true
+      cpu_utilization_max:
+        type: number
+        x-nullable: true
+      memory_used_bytes_max:
+        type: integer
+        x-nullable: true
+      start:
+        type: string
+    type: object
   store.Project:
     properties:
       active_key_count:
@@ -681,6 +731,51 @@ definitions:
       specification_digest:
         type: string
     type: object
+  store.RuntimeNodeDetail:
+    properties:
+      active:
+        type: integer
+      available_disk_bytes:
+        type: integer
+      available_memory_bytes:
+        type: integer
+      cleanup_pending:
+        type: integer
+      cpu_count:
+        type: integer
+      created_at:
+        type: string
+      diagnostic:
+        type: string
+      history:
+        $ref: '#/definitions/store.NodeHostHistory'
+      host:
+        $ref: '#/definitions/store.RuntimeNodeHost'
+      id:
+        type: string
+      last_seen_at:
+        type: string
+      max_active:
+        type: integer
+      max_retained:
+        type: integer
+      name:
+        type: string
+      online:
+        type: boolean
+      provider:
+        type: string
+      provider_ready:
+        type: boolean
+      reserved:
+        type: integer
+      retained:
+        type: integer
+      running:
+        type: integer
+      snapshots:
+        type: integer
+    type: object
   store.RuntimeNodeEnrollment:
     properties:
       backend_fingerprint:
@@ -698,6 +793,27 @@ definitions:
       specification_digest:
         type: string
     type: object
+  store.RuntimeNodeHost:
+    properties:
+      available_disk_bytes:
+        type: integer
+        x-nullable: true
+      available_memory_bytes:
+        type: integer
+        x-nullable: true
+      cpu_utilization:
+        type: number
+        x-nullable: true
+      effective_cpu_cores:
+        type: number
+        x-nullable: true
+      observed_at:
+        type: string
+        x-nullable: true
+      total_memory_bytes:
+        type: integer
+        x-nullable: true
+    type: object
   store.RuntimeNodeIdentity:
     properties:
       deployment_generation:
@@ -5859,6 +5975,54 @@ paths:
       summary: Remove a sandbox node with no retained resources
       tags:
         - Sandbox Manager
+    get:
+      description: Deployment administrator only. Complete UTC buckets. Missing host measurements and offline history are null; reads never sample or backfill.
+      parameters:
+        - description: Sandbox node UUID
+          in: path
+          name: node_id
+          required: true
+          type: string
+        - description: Time range (default 1h)
+          enum:
+            - 1h
+            - 6h
+            - 24h
+          in: query
+          name: range
+          type: string
+      produces:
+        - application/json
+      responses:
+        "200":
+          description: OK
+          schema:
+            $ref: '#/definitions/store.RuntimeNodeDetail'
+        "400":
+          description: Bad Request
+          schema:
+            $ref: '#/definitions/v1.ErrorResponse'
+        "401":
+          description: Unauthorized
+          schema:
+            $ref: '#/definitions/v1.ErrorResponse'
+        "404":
+          description: Not Found
+          schema:
+            $ref: '#/definitions/v1.ErrorResponse'
+        "500":
+          description: Internal Server Error
+          schema:
+            $ref: '#/definitions/v1.ErrorResponse'
+        "503":
+          description: Service Unavailable
+          schema:
+            $ref: '#/definitions/v1.ErrorResponse'
+      security:
+        - DeploymentAdminAuth: []
+      summary: Retrieve sandbox node and host history
+      tags:
+        - Sandbox Manager
     patch:
       consumes:
         - application/json

docs/api/README.md

@@ -42,3 +42,6 @@ Generated schemas do not establish complete compatibility or real execution
 support. The [coverage record](../../contracts/agents-api/README.md) identifies
 qualified workflows, native differences and unresolved behavior. Update the
 relevant contract and this index when adding or moving an API surface.
+
+Node host observations and retained charts are deployment-administrator reads;
+see [node host history](../../contracts/agents-api/node-host-history.md).

docs/api/web-management.md

@@ -124,3 +124,10 @@ Agent execution access. See the [exact measurement contract](../../contracts/age
 for complete buckets, null values, units and process-local retention. Frontend
 implementation is maintained separately; this backend change does not modify
 Agent metrics or the public Agent API.
+
+## Node host history
+
+Deployment administrators can read a node and its host history through
+`GET /core/v1/sandbox/nodes/{node_id}?range=1h|6h|24h`. See the
+[node host history contract](../../contracts/agents-api/node-host-history.md)
+for nullable observations, freshness and aggregation. The node list is unchanged.