diff --git a/README-zh.md b/README-zh.md index f68313ba..03a0e530 100644 --- a/README-zh.md +++ b/README-zh.md @@ -3,7 +3,7 @@

GhostScope

⚡ 下一代 eBPF 用户态运行时追踪器

- Printf 调试的进化 — 无需停止或修改应用即可实时追踪。 + Printf 调试的进化 — 无需 stop-the-world 检查即可实时追踪。

@@ -18,7 +18,9 @@ ## 概述 -GhostScope 是一个 **面向源码语义的用户态追踪器**。有 DWARF 调试信息时，它可以在不停住目标进程的前提下，按函数、源码行或指令粒度设点，打印真正重要的信息，并输出源码感知的调用栈。 +GhostScope 是一个 **面向源码语义的用户态追踪器**。有 DWARF 调试信息时，它可以在不由调试器控制目标暂停的前提下，按函数、源码行或指令粒度设点，打印真正重要的信息，并输出源码感知的调用栈。 + +支持范围以及可信输出所依赖的不变量，统一定义在[设计保证与可信性模型](docs/zh/design-contract.md)中。 > *"The most effective debugging tool is still careful thought, coupled with judiciously placed print statements."* — Brian Kernighan @@ -27,7 +29,7 @@ GhostScope 是一个 **面向源码语义的用户态追踪器**。有 DWARF 调 - 你在排查线上正在运行的服务，不能接受 GDB 式 stop-the-world 带来的巨大性能扰动，同时又更希望使用基于 eBPF 的工作流，获得比传统内核模块式探测更好的安全边界和更低开销。 - 你需要在源码行级探针里读取真实的局部变量、参数、全局变量和复杂数据，而不只是函数入口参数。 - 你想在同一个 live probe 里既打印关键状态，又输出基于 DWARF unwind 的调用栈 (backtrace)。 -- 你需要解释一次请求、函数、源码行或代码路径是如何被执行到的，同时又不能停止目标进程。 +- 你需要解释一次请求、函数、源码行或代码路径是如何被执行到的，同时又不能做 stop-the-world 检查。 - 你想把“这里要是能加一条 printf 就好了”快速变成一个可重复运行的 CLI 追踪脚本。 ### 什么时候不适合用 GhostScope @@ -147,7 +149,7 @@ GhostScope 把编译后的二进制重新变成“可观测系统”。在 TUI 想象一下，你面对的是一片广袤的二进制数据荒野 —— 内存地址、寄存器值、栈帧数据 —— 没有上下文，它们只是毫无意义的数字。**DWARF 调试信息就是我们的地图**：它告诉我们栈地址 `RSP-0x18` 存储着局部变量 `count`，堆地址 `0x5621a8c0` 处的结构体是 `user` 对象，其偏移 `+0x20` 处是字符串指针 `user.name`；它追踪每个变量在程序执行过程中的位置变化 —— 参数 `x` 现在在寄存器 `RDI` 中，之后会被移到栈上 `RSP-0x10` 的位置。 -有了这张地图，GhostScope 利用 **eBPF 和 uprobe** 技术从运行中程序的指令点安全地提取二进制数据。这种组合威力强大：DWARF 揭示进程虚拟地址空间中数据的含义，eBPF 安全地获取我们需要的数据。结果呢？你可以在不停止、不修改程序的情况下打印变量值（局部变量或全局变量）、函数参数、复杂数据结构，以及基于 DWARF unwind 的调用栈。 +有了这张地图，GhostScope 利用 **eBPF 和 uprobe** 技术从运行中程序的指令点安全地提取二进制数据。DWARF 揭示进程虚拟地址空间中数据的含义，eBPF 获取请求的数据，从而在不由调试器控制暂停、也不主动修改应用状态的前提下，输出源码感知的变量值和调用栈。每次 uprobe 命中仍会同步执行 trap 和 eBPF 路径，因此追踪并非对时序完全透明。 ## ✨ 核心特性 @@ -157,9 +159,9 @@ GhostScope 把编译后的二进制重新变成“可观测系统”。在 TUI Performance
- 低开销 + 按需开销
- 仅需一次上下文切换 + eBPF 执行 + 每次命中执行 uprobe + 有界 eBPF 工作 Real-time @@ -173,7 +175,7 @@ GhostScope 把编译后的二进制重新变成“可观测系统”。在 TUI
DWARF 感知
- 完整调试信息支持 + 基于 PC 的源码语义 Rust @@ -186,15 +188,15 @@ GhostScope 把编译后的二进制重新变成“可观测系统”。在 TUI -## ⚠️ 实验性工具声明 +## ⚠️ 可信性与开发状态 -> **GhostScope 目前处于早期开发阶段**，仍在积极迭代中。虽然我们努力确保数据准确性，但在某些情况下可能出现错误或不完整的追踪信息，更多的应该是探测功能不支持。 +> **GhostScope 目前处于早期开发阶段**，仍在积极迭代中。在支持范围内，已知的恢复或归因失败应当显式呈现，不能被展示成看似合理的值。 > -> **建议**：将 GhostScope 采集的数据作为问题排查的**辅助参考**，而非唯一依据。在做出关键决策前，请结合其他调试工具交叉验证。 +> 实现缺陷仍可能违反这份契约。建议将 GhostScope 采集的数据作为问题排查的**辅助参考**；在做出关键决策前，请结合其他调试工具交叉验证。 > -> 我们正在持续改进稳定性和准确性，期待未来版本能够移除此声明。 +> 规范性的保证、前提和失败语义见[设计保证与可信性模型](docs/zh/design-contract.md)。 -当前的硬性限制和软性限制，请参阅 [使用限制](docs/zh/limitations.md)。 +不支持和会显式降级的场景见[使用限制](docs/zh/limitations.md)。 ## 📚 文档 @@ -245,6 +247,9 @@ GhostScope 把编译后的二进制重新变成“可观测系统”。在 TUI - [**架构概览**](docs/zh/architecture.md) 系统设计和内部原理 +- [**设计保证与可信性模型**](docs/zh/design-contract.md) + 支持范围、不变量和失败语义 + - [**开发指南**](docs/zh/development.md) 构建和扩展 GhostScope diff --git a/README.md b/README.md index 61af7439..e496179b 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@

GhostScope

⚡ Next-generation eBPF userspace runtime tracer

- Printf debugging evolved — Real-time tracing without stopping your application. + Printf debugging evolved — Real-time tracing without stop-the-world inspection.

@@ -22,7 +22,9 @@ ## Overview -GhostScope is a **source-aware userspace tracer** for live Linux processes. With DWARF debug info, it lets you attach at function, source-line, or instruction granularity, print the values that matter, and emit source-aware call stacks without stopping the target. +GhostScope is a **source-aware userspace tracer** for live Linux processes. With DWARF debug info, it lets you attach at function, source-line, or instruction granularity, print the values that matter, and emit source-aware call stacks without debugger-controlled suspension of the target. + +The supported operating envelope and the invariants behind trustworthy output are defined in [Design Guarantees and Trust Model](docs/design-contract.md). > *"The most effective debugging tool is still careful thought, coupled with judiciously placed print statements."* — Brian Kernighan @@ -31,7 +33,7 @@ GhostScope is a **source-aware userspace tracer** for live Linux processes. With - You are diagnosing a production service that must stay online: GDB-style stop-the-world debugging would cause too much disruption, and you prefer an eBPF-based workflow with stronger safety boundaries and lower overhead than traditional kernel-module instrumentation. - You need source-line probes that can read real locals, parameters, globals, and complex data, not just function entry arguments. - You want DWARF-unwound stack backtraces from the same live probe that prints the relevant state. -- You need to explain how execution reached a specific source line, function, request, or code path without stopping the process. +- You need to explain how execution reached a specific source line, function, request, or code path without stop-the-world inspection. - You want a low-friction path from "I wish I had one more printf here" to a repeatable CLI trace script. ### When Not To Use GhostScope @@ -152,7 +154,7 @@ The demo below follows exactly that path on a DWARF-enabled nginx worker: locate Imagine navigating a vast, uncharted forest of binary data — memory addresses, register values, stack frames — all meaningless numbers without context. **DWARF debug information is our map**: it tells us that stack address `RSP-0x18` stores local variable `count`, heap address `0x5621a8c0` is a `user` object with string pointer `user.name` at offset `+0x20`; it tracks where each variable lives throughout program execution — parameter `x` is in register `RDI` now but will move to stack offset `RSP-0x10` later. -With this map in hand, GhostScope leverages **eBPF and uprobe** technology to safely extract binary data from instruction points in your running program. The combination is powerful: DWARF reveals the meaning of bytes in the process's virtual address space, while eBPF safely retrieves exactly what we need. The result? You can print variable values (local or global), function arguments, complex data structures, and DWARF-unwound call stacks without stopping or modifying the program. +With this map in hand, GhostScope leverages **eBPF and uprobe** technology to safely extract binary data from instruction points in your running program. The combination is powerful: DWARF reveals the meaning of bytes in the process's virtual address space, while eBPF retrieves the requested data. The result is source-aware variable values and call stacks without debugger-controlled suspension or intentional application-state changes. Each uprobe hit still adds synchronous trap and eBPF work, so tracing is not timing-transparent. ## ✨ Highlights @@ -162,9 +164,9 @@ With this map in hand, GhostScope leverages **eBPF and uprobe** technology to sa Performance
- Low Overhead + Targeted Overhead
- One context switch + eBPF execution + Per-hit uprobe + bounded eBPF work Real-time @@ -178,7 +180,7 @@ With this map in hand, GhostScope leverages **eBPF and uprobe** technology to sa
DWARF-Aware
- Full debug info support + PC-specific source semantics Rust @@ -191,15 +193,15 @@ With this map in hand, GhostScope leverages **eBPF and uprobe** technology to sa -## ⚠️ Experimental Tool Disclaimer +## ⚠️ Reliability and Development Status -> **GhostScope is currently in early development** and under active iteration. While we strive for data accuracy, trace information may be incorrect or incomplete in certain scenarios, primarily due to unsupported features. +> **GhostScope is currently in early development** and under active iteration. Within the supported envelope, a known inability to recover or attribute data should be reported explicitly rather than rendered as a plausible value. > -> **Recommendation**: Use GhostScope's collected data as an **auxiliary reference** for troubleshooting, not as the sole source of truth. Cross-validate with other debugging tools before making critical decisions. +> Implementation defects may still violate that contract. Use GhostScope's collected data as an **auxiliary reference** for troubleshooting, and cross-validate with other debugging tools before making critical decisions. > -> We are continuously improving stability and accuracy, and look forward to removing this disclaimer in future versions. +> The normative guarantees, assumptions, and failure semantics are documented in [Design Guarantees and Trust Model](docs/design-contract.md). -See [Limitations](docs/limitations.md) for the current list of hard and soft constraints. +See [Limitations](docs/limitations.md) for unsupported and explicitly degraded scenarios. ## 📚 Documentation @@ -250,6 +252,9 @@ See [Limitations](docs/limitations.md) for the current list of hard and soft con - [**Architecture Overview**](docs/architecture.md) System design and internals +- [**Design Guarantees and Trust Model**](docs/design-contract.md) + Supported operating envelope, invariants, and failure semantics + - [**Development Guide**](docs/development.md) Build and extend GhostScope diff --git a/docs/architecture.md b/docs/architecture.md index d4419add..c47a4abc 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,6 +2,43 @@ A deep dive into the design and implementation of GhostScope's eBPF-based runtime tracing system. +The normative guarantees and failure semantics are defined in +[Design Guarantees and Trust Model](design-contract.md). This document explains +the mechanisms that enforce that contract. + +## Trustworthy Observation Path + +GhostScope does not obtain source semantics from any single component. A +rendered value is the end of an evidence chain: + +```text +requested target scope + | + v +runtime PID / namespace / module mapping + | + v +module-relative probe PC + matching DWARF + | + v +PC-specific scope, type, and location plan + | + v +bounded eBPF read + attributed event + | + v +validated protocol record or explicit failure state +``` + +| Stage | Invariants | Enforcement and failure behavior | +|---|---|---| +| Target selection | `IDENT-1` | `-p`/`-t` mode semantics, PID filters, and namespace-aware discovery define the allowed runtime scope; unavailable scope fails setup. | +| Module and debug information | `IDENT-1`, `SEM-1` | Runtime mappings, module cookies, load offsets, and available debug-file identity checks bind semantics to a module; unverifiable or loose matches are surfaced as weakened evidence. | +| Semantic planning | `SEM-1`, `FAIL-1` | The DWARF engine resolves scope, type, location, and relocation at the probe PC; unsupported plans produce diagnostics instead of guessed values. | +| eBPF execution | `SAFE-1`, `COST-1` | The compiler emits bounded observational programs, and the kernel verifier rejects unsafe programs. | +| Event transport | `LOSS-1` | RingBuf or PerfEventArray carries events; output-helper failures increment per-trace loss counters. | +| Protocol and rendering | `IDENT-1`, `FAIL-1` | Trace/PID/TID metadata and structured unavailable, expression-error, and backtrace states remain visible to consumers. | + ## System Overview ``` @@ -58,7 +95,7 @@ GhostScope uses Cargo workspace for modular design: | **ghostscope** | Main binary and runtime coordinator - orchestrates all components via async event loop | | **ghostscope-compiler** | Script compilation pipeline - transforms user scripts into verified eBPF bytecode via LLVM | | **ghostscope-dwarf** | PC-context DWARF semantic engine - resolves source locations, visible variables, type layouts, address mappings, and compiler read plans | -| **ghostscope-loader** | eBPF program lifecycle manager - handles uprobe attachment and ring buffer management via Aya | +| **ghostscope-loader** | eBPF program lifecycle manager - handles uprobe attachment and RingBuf/PerfEventArray event transport via Aya | | **ghostscope-ui** | Terminal user interface - implements interactive TUI with TEA (The Elm Architecture) pattern | | **ghostscope-protocol** | Communication protocol - defines message format for eBPF-userspace data exchange | | **ghostscope-platform** | Platform abstraction - encapsulates architecture-specific code (calling conventions, ABIs) | @@ -203,9 +240,12 @@ The diagram below is from [Crafting Interpreters](https://craftinginterpreters.c ### 7. eBPF to Userspace Communication -**Core mechanism**: Ring buffer (per-CPU circular buffer in kernel memory). +**Core mechanism**: A kernel event transport selected at startup. GhostScope +prefers RingBuf on supported kernels and falls back to PerfEventArray. RingBuf +is one shared multi-producer buffer across CPUs; PerfEventArray uses independent +per-CPU buffers. -#### Ring Buffer Architecture +#### Event Transport Architecture ``` ┌───────────────────────────────────────────────────────┐ @@ -218,11 +258,13 @@ The diagram below is from [Crafting Interpreters](https://craftinginterpreters.c │ └─────┬──────┘ ↓ │ │ │ Serialize to protocol format │ │ │ ↓ │ -│ │ bpf_ringbuf_output() │ +│ │ bpf_ringbuf_output() / │ +│ │ bpf_perf_event_output() │ │ │ ↓ │ │ └────────►┌─────────────────────┐ │ -│ │ Ring Buffer │ │ -│ │ (per-CPU, 256KB) │ │ +│ │ RingBuf (shared) │ │ +│ │ or Perf buffers │ │ +│ │ (per CPU) │ │ │ │ │ │ │ │ [Event1][Event2]...│ │ │ └──────────┬──────────┘ │ @@ -234,7 +276,7 @@ The diagram below is from [Crafting Interpreters](https://craftinginterpreters.c │ │ │ │ ┌──────────────────────────▼──────────┐ │ │ │ Trace Manager │ │ -│ │ (polls ring buffer) │ │ +│ │ (polls selected transport) │ │ │ └──────────────────────┬───────────────┘ │ │ │ │ │ Read events (non-blocking) │ @@ -259,16 +301,17 @@ The diagram below is from [Crafting Interpreters](https://craftinginterpreters.c - Uprobe fires when target instruction executes - eBPF program collects data (registers, stack, memory via DWARF locations) - Serializes data according to protocol format - - Calls `bpf_ringbuf_output()` to write to ring buffer + - Calls the selected RingBuf or PerfEventArray output helper + - Increments the trace's loss counter if the output helper fails 2. **Event Polling** (User Space): - - Trace manager polls ring buffer (via Aya framework) + - Trace manager polls the selected transport (via Aya framework) - Non-blocking: Returns immediately if no events - - Memory-mapped: Zero-copy access to kernel buffer + - RingBuf uses a shared memory-mapped buffer; PerfEventArray drains per-CPU buffers 3. **Event Parsing**: - Streaming parser handles variable-length messages - - State machine tracks partial reads across chunks + - Parser applies transport-specific framing and tracks partial RingBuf reads - Reconstructs complete events 4. **Event Delivery**: @@ -276,6 +319,11 @@ The diagram below is from [Crafting Interpreters](https://craftinginterpreters.c - Coordinator forwards to UI via channel - UI updates display in real-time +5. **Loss Reporting**: + - Runtime periodically reads each trace's eBPF output-failure counter + - CLI and TUI report interval and cumulative loss totals + - A nonzero counter marks the observation interval as incomplete + #### Protocol Format GhostScope uses an **instruction-based protocol** for flexible trace event representation: diff --git a/docs/comparison.md b/docs/comparison.md index 3d671301..008d2f35 100644 --- a/docs/comparison.md +++ b/docs/comparison.md @@ -17,18 +17,18 @@ GhostScope is designed for a narrow but high-value job: **source-aware userspace | Feature | GhostScope | GDB | |---|---|---| | Type | Tracer, closer to production printf debugging | Interactive debugger | -| Execution model | Trace a live process without stopping it | Stop, inspect, and control execution | +| Execution model | Trace without debugger-controlled suspension; each hit still runs a uprobe and eBPF program synchronously | Stop, inspect, and control execution | | Runtime overhead | Usually low when used selectively | High once breakpoints are involved | -| Process interruption | Never | Yes | +| Process interruption | No stop-the-world inspection; the hit thread still pays synchronous probe work | Yes, as part of the debugging model | | Production use | Designed for production-friendly observation | Better suited to development and postmortem work | -| Timing preservation | Yes | No, breakpoints and stepping change timing | -| Concurrency debugging | Strong when you need to preserve real timing | Often harder because stop-the-world behavior distorts interleavings | +| Timing impact | Usually much smaller than interactive breakpoints, but not timing-transparent | Breakpoints and stepping deliberately change timing | +| Concurrency debugging | Preserves normal execution better than stop-the-world inspection, while probes can still perturb interleavings | Stop-the-world behavior can strongly distort interleavings | | Interactive control | TUI and scripting, but not execution control | Full execution control: breakpoints, stepping, continue, mutation | | Variable access | Reads values at chosen trace points using DWARF + eBPF | Reads state after pausing execution | | Best at | Production-style runtime observation | Interactive debugging and state mutation | | When to choose it | You need observability | You need control | -Use GhostScope when preserving process timing matters. Use GDB when you need to stop the world and reason step by step. +Use GhostScope when avoiding stop-the-world inspection matters and you can budget for per-hit probe overhead. Use GDB when you need to control execution and reason step by step. Neither tool preserves the original timing exactly. ## GhostScope vs GDB Performance Snapshot diff --git a/docs/design-contract.md b/docs/design-contract.md new file mode 100644 index 00000000..3a73dd60 --- /dev/null +++ b/docs/design-contract.md @@ -0,0 +1,213 @@ +# Design Guarantees and Trust Model + +GhostScope turns live machine state into source-aware observations. This +document defines the conditions under which those observations are intended to +be trustworthy and the invariants that the implementation must preserve. + +This is a normative design contract. The [architecture](architecture.md) +describes how the current implementation enforces it, while +[limitations](limitations.md) describe behavior outside the supported envelope +or behavior that degrades explicitly. Future guarantees belong in the +[roadmap](roadmap.md), not in this document. + +A violation of an invariant inside the supported envelope is a correctness bug, +not a limitation. Early-development status means such bugs may still exist; it +does not permit known unsupported paths to return plausible but unproven data. + +## Supported Operating Envelope + +- Official release artifacts and correctness testing target Linux x86_64. + Source code compiling on another architecture does not make that architecture + supported. +- Source-aware values require DWARF that corresponds to the module being + traced. Embedded DWARF is read from that module. For separate debug files, + available `.gnu_debuglink` CRC and Build ID evidence is checked strictly by + default. +- C has the strongest end-to-end coverage. C++ and Rust currently use a more + limited, DWARF-layout-oriented model. Language coverage is documented in + [limitations](limitations.md). +- The kernel must provide the required eBPF, uprobe, and event transport + facilities, and GhostScope must have permission to use them. + +`--allow-loose-debug-match` explicitly opts out of strict separate-debug-file +matching. When enabled, GhostScope warns about the mismatch, but the user has +weakened the evidence behind source-level interpretation. An explicit debug +file for which neither CRC nor a comparable Build ID is available also cannot +be proven identical to the module; the warning makes that a user-supplied trust +decision outside the strict evidence envelope. + +## Trust Boundaries + +GhostScope relies on the Linux kernel, the eBPF verifier and helpers, the target +module and its compiler-produced DWARF, and process metadata exposed through +`/proc`. It validates and narrows those inputs where it can, but it cannot prove +that a compiler emitted semantically correct DWARF. + +An event is a point-in-time observation at a uprobe hit, not a process-wide +atomic snapshot. Registers and the current thread's frame correspond to that +hit. Memory shared with other threads can change while a multi-field event is +being collected. + +Read-only also does not mean zero-impact. Linux uprobes install trap points, and +the target thread synchronously pays the uprobe and eBPF work on every hit. +GhostScope avoids debugger-controlled suspension and application-state writes, +but it cannot guarantee unchanged timing. + +## Invariants + +### SCOPE-1: The supported platform is explicit + +**Guarantee.** Official builds and correctness claims are scoped to Linux +x86_64. + +**Enforcement.** Release artifacts are built and named for x86_64, and +architecture-specific register, ABI, TLS, and unwind behavior is tested in that +environment. + +**Failure boundary.** Other architectures must not be presented as supported +until their platform mappings and end-to-end behavior are implemented and +tested. Accidental compilation is not evidence of support. + +### SAFE-1: Observation does not control the target + +**Guarantee.** GhostScope exposes no operation that intentionally changes +application-visible variables, memory, or control flow, and it does not suspend +the process for interactive inspection, stepping, or continuation. + +**Enforcement.** The script language is observational, generated eBPF programs +use target-read, internal-map, and event-output operations, and the kernel +verifier constrains loaded programs. + +**Failure boundary.** Uprobe traps and eBPF execution still perturb scheduling +and latency. A feature that writes target memory or changes control flow would +change this contract and must not be introduced as an ordinary tracing feature. + +### IDENT-1: Every observation matches the requested target scope + +**Guarantee.** An event must be attributable to the trace and runtime target +scope that produced it. GhostScope must not silently use another PID, module, or +address-space interpretation when the requested one is unavailable. + +The startup modes define that scope: + +- `-p ` selects a process view and its loaded modules. +- `-t ` selects a target-module, multi-process view. +- `-t -p ` resolves trace targets in the selected module and limits + runtime events to the selected process. + +**Enforcement.** Trace IDs, PID/TID metadata, PID filters, PID-namespace-aware +process discovery, module cookies, and runtime module offsets carry identity +through attach, collection, and rendering. + +**Failure boundary.** Missing process visibility, an unavailable module +mapping, or an ambiguous target must fail setup or make the dependent semantic +operation explicitly unavailable. It must not fall back to an unrelated scope. + +### SEM-1: Source semantics require a complete evidence chain + +**Guarantee.** GhostScope renders a source-level value only when it can connect +the target module and probe PC to the applicable DWARF scope, type, location, +and runtime read plan. + +The evidence chain is: + +```text +target scope -> runtime module -> module-relative PC -> DWARF scope/type/location + -> typed read plan -> runtime read result -> rendered observation +``` + +**Enforcement.** Strict debug-file matching, module-aware PC contexts, +ASLR/PIE relocation, lexical and inline scope resolution, typed lowering, and +runtime read status preserve the meaning of each step. + +**Failure boundary.** If any step cannot be established, GhostScope must report +an unsupported location, `OptimizedOut`, a read error, `ExprError`, or another +explicit unavailable status. It must not substitute a same-named variable from +another scope, guess an address, or render an unverified value as valid. + +### FAIL-1: Known uncertainty is explicit + +**Guarantee.** A known inability to compile, attach, read, unwind, parse, or +attribute an observation must remain visible to the user or automation. + +**Enforcement.** Setup and lowering failures are errors; runtime expression and +memory failures are structured protocol states; backtraces report complete, +truncated, and stopped states with reasons. + +**Failure boundary.** Best-effort behavior may return partial results only when +the missing portion is marked. Logging a warning while emitting an apparently +complete but unsupported value does not satisfy this invariant. + +### LOSS-1: Detectable event loss is not silent + +**Guarantee.** A successful trace does not imply a lossless event stream. When +the eBPF output helper rejects an event, GhostScope counts the failure per trace +and reports interval and cumulative loss to CLI and TUI users. + +**Enforcement.** Generated programs update loss counters when RingBuf or +PerfEventArray output fails. The runtime periodically reads and reports those +counters. + +**Failure boundary.** These counters cover kernel output-helper failures. They +cannot identify which events were lost and do not prove the absence of losses +outside that measurement point, such as events before attachment or after +shutdown. Any nonzero report means the corresponding observation interval is +incomplete. + +### COST-1: Per-event work is bounded, aggregate impact is workload-dependent + +**Guarantee.** Generated eBPF execution, trace-event size, memory reads, and +backtrace depth are subject to verifier-compatible and configurable bounds. +Reaching a semantic or resource bound must reject the trace or produce an +explicit truncation/unavailable state. + +**Enforcement.** The compiler applies event-size and read bounds, the verifier +checks eBPF control flow, transports have configured capacity, and DWARF +unwinding uses depth and tail-call budgets. + +**Failure boundary.** GhostScope does not promise a fixed total overhead. A +bounded program attached to a hot path can still materially slow the target. +Probe frequency and payload cost must be included in the operator's budget. + +## Result States + +The invariants imply four distinct outcomes: + +| Outcome | Meaning | +|---|---| +| Complete | The requested operation completed within the supported envelope. | +| Explicitly partial | Some requested data is unavailable or truncated, and the event carries that status. | +| Rejected | Setup, compilation, verification, or attachment could not establish the required conditions. | +| Stream incomplete | One or more detectable events were lost; delivered events may still be valid, but the interval is not complete. | + +These states must not be collapsed into a generic success path. + +## Verification Expectations + +Tests are evidence for this contract, not the definition of the contract. The +main evidence currently lives in these areas: + +| Invariant | Primary evidence | +|---|---| +| `SCOPE-1` | x86_64 release workflow and platform-specific unit tests | +| `SAFE-1` | Script/compiler operation surface, eBPF helper usage, verifier-backed load tests | +| `IDENT-1` | PID-specific execution tests and container-topology tests | +| `SEM-1` | PC-context, scalar, global, optimized-code, and cross-module fixtures with exact-value oracles | +| `FAIL-1` | Compile-error, `OptimizedOut`, `ExprError`, read-failure, and backtrace-status tests | +| `LOSS-1` | eBPF output-failure counters and CLI/TUI reporting paths; pressure coverage is required when this path changes | +| `COST-1` | Compiler bound checks, verifier-backed loading, configured-depth and deep-backtrace tests | + +When a change touches target selection, process identity, module/PC mapping, +DWARF lowering, generated helpers, event protocol, transport, or rendering, its +validation should identify the affected invariant and provide a positive oracle +and a relevant failure-path oracle. + +## Documentation Discipline + +- Put current, enforced guarantees in this document. +- Put supported-but-degraded cases and unsupported conditions in + [limitations](limitations.md). +- Put intended future guarantees in the [roadmap](roadmap.md). +- Keep user-facing claims in the README and comparison document no stronger + than this contract. +- Update the English and Chinese versions together when the contract changes. diff --git a/docs/development.md b/docs/development.md index 3a7386c6..37d3ffe3 100644 --- a/docs/development.md +++ b/docs/development.md @@ -143,6 +143,33 @@ After rebuilding, a regular workspace build will pick up the new objects automat ## Testing +### Invariant-Oriented Validation + +The [Design Guarantees and Trust Model](design-contract.md) defines the behavior +that validation must protect. Tests should prove an observable contract, not +only that a command exits successfully. + +| Invariant | Minimum evidence when affected | +|---|---| +| `SCOPE-1` | Build/release target remains Linux x86_64, plus platform-specific unit coverage | +| `SAFE-1` | Generated helper/operation review and a verifier-backed load test for new eBPF behavior | +| `IDENT-1` | Positive target attribution and a wrong-PID/module/namespace negative case | +| `SEM-1` | A fixture with an exact source-level value oracle at a known PC, including relevant optimization/module variants | +| `FAIL-1` | A negative fixture that asserts the structured error, unavailable marker, or stop reason | +| `LOSS-1` | Counter propagation and CLI/TUI rendering, plus pressure coverage when transport behavior changes | +| `COST-1` | Boundary/rejection behavior and a verifier-backed load for changed event-size, read, or unwind budgets | + +Relevant runtime evidence is organized in `e2e-tests/tests/`: PID filtering in +`script_execution.rs`, namespace behavior in `container_topology_execution.rs`, +PC-context semantics in `member_pointer_compilation.rs`, optimized values in +`optimized_inline_call_value_execution.rs`, structured expression failures in +`globals_execution.rs`, and unwind status/depth in `backtrace_execution.rs`. + +For a behavior change, identify the affected invariant in the review or +handoff, run the corresponding positive oracle, and run at least one relevant +failure-path oracle. The test command is evidence only when the asserted output +matches the invariant. + ### Workspace Tests ```bash diff --git a/docs/limitations.md b/docs/limitations.md index a1efd5f5..49ee73cb 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -1,18 +1,23 @@ # Limitations +This document describes conditions outside the supported operating envelope or +cases that degrade explicitly. The guarantees that remain mandatory inside the +supported envelope are defined in the +[Design Guarantees and Trust Model](design-contract.md). + ## Hard Limitations ### 1. Privilege Requirements The underlying eBPF mechanism requires elevated privileges to access kernel tracing infrastructure. Root access or CAP_BPF/CAP_SYS_ADMIN capabilities are needed. ### 2. Do Not Use with GDB Simultaneously -Both uprobe and GDB modify target process instructions (inserting breakpoints). Using them together may lead to conflicts and unpredictable behavior. +Both uprobes and GDB install breakpoint-based instrumentation in the target process. Using them together may lead to conflicts and unpredictable behavior. ### 3. Read-Only Access, Cannot Modify Program Behavior -Although eBPF technically supports modifying process behavior to some extent, GhostScope is designed as a read-only tool and cannot modify program state, variable values, or control flow. This ensures safety in production environments. +GhostScope's script and compiler surface is observational: it does not provide operations that intentionally modify application-visible state, variable values, or control flow. This does not mean the target is untouched. Uprobes install trap points, and each hit synchronously adds uprobe and eBPF execution to the target thread. -### 4. Platform Support -Currently only supports **Linux** operating system due to its core dependency on **eBPF** and **uprobe**. +### 4. Platform and Architecture Support +Official builds and correctness testing currently support **Linux x86_64 (AMD64)** only because GhostScope depends on Linux **eBPF** and **uprobes** and currently implements x86_64 register, ABI, TLS, and unwind behavior. Other architectures are outside the support contract even if the source happens to compile on them. ## Soft Limitations @@ -21,10 +26,7 @@ Primary focus is on **C language**, which currently has the best end-to-end supp For interpreted languages (Lua, Python, Ruby, etc.), only the interpreter itself can be traced (since interpreters are typically implemented in compiled languages). Tracing script code is technically feasible but requires substantial development time. JIT language support is an even more distant goal. -### 2. Architecture Support -Currently only supports **x86_64 (AMD64)** architecture. Other platforms (such as ARM64) are technically feasible but require time for adaptation and testing. - -### 3. User-Memory Reads via `bpf_probe_read_user` +### 2. User-Memory Reads via `bpf_probe_read_user` In traditional non-sleepable probe paths, helpers such as `bpf_probe_read_user` cannot resolve user-space page faults, so reads from a target virtual address may still fail if the page is not resident or otherwise faults on access. This is no longer an absolute eBPF limitation. Linux now supports sleepable uprobes (`uprobe.s` / `uretprobe.s`), and sleepable programs can use helpers such as `bpf_copy_from_user_task()` for fault-capable user-memory reads. GhostScope currently emits regular `uprobe` programs, so this remains a practical limitation today, but it is better described as a soft implementation limitation rather than a fundamental design limit of eBPF. @@ -34,26 +36,30 @@ This is no longer an absolute eBPF limitation. Linux now supports sleepable upro - https://docs.kernel.org/bpf/libbpf/program_types.html - https://man7.org/linux/man-pages/man7/bpf-helpers.7.html -### 4. Performance Impact -Based on uprobe implementation, each probe trigger incurs a context switch overhead plus eBPF program execution time. If probes are set on hot paths, they may significantly impact the monitored process performance. Use with caution. +### 3. Performance Impact +Each uprobe hit synchronously runs trap handling and the generated eBPF program. Even though per-event work is bounded, probes on hot paths may significantly affect the monitored process. GhostScope does not guarantee unchanged timing or a fixed total overhead; use targeted probes and include hit rate in the performance budget. + +### 4. Event Loss (Backpressure) +GhostScope uses RingBuf when available and PerfEventArray as a fallback. Either transport can reject output when event generation exceeds available capacity. Generated programs increment a per-trace output-failure counter, and the CLI and TUI periodically report the interval and cumulative loss totals. -### 5. Event Loss (Backpressure) -Uses ring buffer to pass events between eBPF programs and userspace. If event generation rate exceeds userspace consumption capacity, the kernel will drop events, leading to trace data loss. GhostScope's error reporting is not yet comprehensive (will learn from bpftrace's approach in the future); avoid setting too many probes on high-frequency paths. +This provides loss visibility, not a lossless guarantee. The counters cover failures at the eBPF output helper; they cannot identify which events were lost or prove that no loss occurred outside that measurement point. Treat any nonzero loss report as an incomplete observation interval and avoid placing large or numerous probes on high-frequency paths. -### 6. DWARF Support Coverage +### 5. DWARF Support Coverage Primarily tested and validated with DWARF 5 format. Theoretically supports DWARF 2-5, but other versions may have compatibility issues. Some DWARF expression instructions are not yet supported for conversion to eBPF (purely due to implementation not being completed yet) and will provide clear error messages when encountered. GhostScope recognizes `DW_OP_form_tls_address`, but the runtime TLS address resolver currently handles only x86_64 executable static TLS. GhostScope resolves the current thread's TLS base at probe time, so a trace running on different pthreads reads each thread's own TLS instance for that supported executable case. The same DWARF operation is also used for dynamic/shared-library TLS; those cases require DTV/module TLS lookup and are not modeled yet, so GhostScope rejects shared-object TLS instead of guessing an address. -### 7. Stack Backtrace Coverage +GhostScope strictly checks any available `.gnu_debuglink` CRC and comparable Build ID for separate debug files by default. Enabling `--allow-loose-debug-match` permits a mismatch with a warning and weakens the source-semantic evidence guarantee. An explicitly supplied debug file with neither an available CRC nor a comparable Build ID is also accepted with a warning, but its identity is then a user-supplied trust assumption rather than a verified match. + +### 6. Stack Backtrace Coverage `bt` uses DWARF CFI only. GhostScope does not fall back to kernel stack helpers or frame-pointer walking, and it reports an explicit stop status when CFI is unavailable, not supported by the compact eBPF fast path, or a user-stack memory read fails. Cross-module frames can be symbolized from their raw IPs when the process module map is available, and runtime module refresh can append compact DWARF rows for newly mapped modules up to `backtrace_unwind_rows_max_entries`. A trace event that fires before the map-change refresh reaches userspace can still stop at the newly loaded module until a later event observes the appended rows. Deep DWARF unwinding is split through an eBPF tail-call step program so the default `backtrace_depth = 128` avoids LLVM branch-distance and verifier-size limits; `status=truncated` means the configured depth or the tail-call unwind budget was reached before a natural stop. Runtime mode also affects `bt` coverage. `-p ` is a process-level view, so GhostScope loads the modules already mapped in that PID's `/proc//maps`; cross-module unwinding and symbolization are usually best in this mode. Standalone `-t ` is a target-file, multi-process trace view that primarily guarantees probes and variables in the target module. Once the call stack leaves that module, backtrace quality is best-effort and depends on runtime module mappings, maintained `proc_module_offsets`, and compact DWARF CFI being available for the other modules. If you need both target-module scoping and a fuller single-process backtrace, prefer `-t -p `. -### 8. Highly Optimized Code Support +### 7. Highly Optimized Code Support Compiler optimizations (-O2, -O3) can cause variables to be optimized away or generate complex DWARF expressions. GhostScope will attempt to parse them, including inline function support, but some variables may be inaccessible (shown as OptimizedOut) because the compiler optimized them away. -### 9. Dynamically Loaded Libraries (dlopen) +### 8. Dynamically Loaded Libraries (dlopen) GhostScope scans `/proc/PID/maps` at startup, and runtime map-change monitoring now refreshes module mappings for `-p ` and standalone `-t ` runs while sysmon is enabled. For `bt`/`backtrace`, this can add compact DWARF CFI rows and module offsets for libraries loaded later through `dlopen`, subject to `backtrace_unwind_rows_max_entries` and the map-change race noted above. `-p ` has one startup edge case: trace setup snapshots the PID's @@ -68,7 +74,7 @@ library target avoids the race. This runtime refresh does not automatically create new trace probes or make print/global-variable targets available for a library that was unknown when the script was compiled and attached. Those targets still depend on the target module and debug information being available during trace setup. -### 10. Global Variables in `-t` Mode +### 9. Global Variables in `-t` Mode - **Executable targets**: When `-t` points to an executable (`-t /path/to/app`), GhostScope treats that binary as the primary module and globals are supported by default. - **Shared-library targets (existing processes)**: If GhostScope starts after the library has already been mapped (e.g., tracing a running process that loaded `libfoo.so` earlier), globals work without extra steps. @@ -77,7 +83,11 @@ This runtime refresh does not automatically create new trace probes or make prin In `-t` mode, globals depend on `proc_module_offsets`, which tracks runtime address offsets by `(pid, module)`. Standalone `-t` maintains target-module offsets for multiple PIDs, so it is suitable for observing globals in that module; it is not a complete process view, and `bt` after leaving the target module is not equivalent to `-p`. Use `-p ` when you need the full module context for one process. Use `-t -p ` when you need target-module trace resolution while keeping that process context. -### 11. `-p ` Mode inside Containers or WSL +### 10. Container / WSL Limitations for `-p ` Mode - See [Container Environments](container.md) for the full explanation of container / WSL scenarios, PID namespace terminology, the scenario matrix, and current implementation limits. - See [PID namespaces manual](https://www.man7.org/linux/man-pages/man7/pid_namespaces.7.html), [WSL issue #12408](https://github.com/microsoft/WSL/issues/12408), and [WSL issue #12115](https://github.com/microsoft/WSL/issues/12115) for background. + +### 11. Observation Consistency + +A trace event is not a process-wide atomic snapshot. Register values and the current thread's frame correspond to the uprobe hit, but other threads can modify shared memory while GhostScope reads multiple fields. Values collected in one event can therefore represent a short observation interval rather than one globally synchronized instant. diff --git a/docs/scripting.md b/docs/scripting.md index 7533e6c2..c4ded7d9 100644 --- a/docs/scripting.md +++ b/docs/scripting.md @@ -728,7 +728,7 @@ backtrace: truncated, 2 frames (max 2) `status=complete` means DWARF unwinding reached a natural stop before the configured depth cap. `status=truncated` means GhostScope hit the configured depth cap or the eBPF tail-call unwind budget before a natural stop. Other statuses explain where unwinding stopped, for example no unwind rows for the current PC, unsupported CFI, unavailable module offsets, a failed user-memory read, or an invalid next frame. When available, `stopped:` includes a stable reason label and numeric code. -`bt` can cross module boundaries when process module mappings and compact DWARF CFI are available for those modules. In `-p` and standalone `-t` runs with sysmon enabled, runtime module refresh can add backtrace metadata for libraries loaded later through `dlopen`; it does not create new trace probes for targets that were unknown during setup. See [Limitations](limitations.md#7-stack-backtrace-coverage) for the exact coverage rules. +`bt` can cross module boundaries when process module mappings and compact DWARF CFI are available for those modules. In `-p` and standalone `-t` runs with sysmon enabled, runtime module refresh can add backtrace metadata for libraries loaded later through `dlopen`; it does not create new trace probes for targets that were unknown during setup. See [Limitations](limitations.md#6-stack-backtrace-coverage) for the exact coverage rules. ## Examples diff --git a/docs/zh/architecture.md b/docs/zh/architecture.md index 8169e469..09f1cbd9 100644 --- a/docs/zh/architecture.md +++ b/docs/zh/architecture.md @@ -2,6 +2,40 @@ 深入探讨基于 eBPF 的运行时追踪系统 GhostScope 的设计与实现。 +规范性的保证与失败语义定义在[设计保证与可信性模型](design-contract.md)中。本文档说明当前实现通过哪些机制执行这份契约。 + +## 可信观测链 + +GhostScope 的源码语义并不来自某一个独立组件。最终展示的值位于一条完整证据链的末端： + +```text +请求的目标范围 + | + v +运行时 PID / namespace / 模块映射 + | + v +模块内 probe PC + 匹配的 DWARF + | + v +基于 PC 的作用域、类型和位置计划 + | + v +有界 eBPF 读取 + 带归因信息的事件 + | + v +通过验证的协议记录或显式失败状态 +``` + +| 阶段 | 不变量 | 保证机制与失败行为 | +|---|---|---| +| 目标选择 | `IDENT-1` | `-p`/`-t` 模式语义、PID 过滤和感知 namespace 的进程发现共同定义允许运行的范围；范围不可用时 setup 失败。 | +| 模块与调试信息 | `IDENT-1`、`SEM-1` | 运行时映射、模块 Cookie、加载偏移和可用的调试文件身份检查把语义绑定到模块；无法验证或 loose match 的情况会被标记为证据减弱。 | +| 语义规划 | `SEM-1`、`FAIL-1` | DWARF 引擎在 probe PC 上解析作用域、类型、位置和重定位；不支持的计划产生诊断，而不是猜测值。 | +| eBPF 执行 | `SAFE-1`、`COST-1` | 编译器生成有界的观测程序，内核 verifier 拒绝不安全程序。 | +| 事件传输 | `LOSS-1` | RingBuf 或 PerfEventArray 传输事件；输出 helper 失败会增加按 trace 维护的丢失计数。 | +| 协议与展示 | `IDENT-1`、`FAIL-1` | Trace/PID/TID 元数据以及结构化的不可用、表达式错误和 backtrace 状态会保留给消费方。 | + ## 系统概览 ``` @@ -58,7 +92,7 @@ GhostScope 使用 Cargo workspace 进行模块化设计： | **ghostscope** | 主程序和运行时协调器 - 通过异步事件循环协调所有组件 | | **ghostscope-compiler** | 脚本编译流水线 - 通过 LLVM 将用户脚本转换为经过验证的 eBPF 字节码 | | **ghostscope-dwarf** | PC 上下文 DWARF 语义引擎 - 解析源码位置、可见变量、类型布局、地址映射和编译器读取计划 | -| **ghostscope-loader** | eBPF 程序生命周期管理器 - 通过 Aya 处理 uprobe 附加和 ring buffer 管理 | +| **ghostscope-loader** | eBPF 程序生命周期管理器 - 通过 Aya 处理 uprobe 附加和 RingBuf/PerfEventArray 事件传输 | | **ghostscope-ui** | 终端用户界面 - 实现基于 TEA (The Elm Architecture) 模式的交互式 TUI | | **ghostscope-protocol** | 通信协议 - 定义 eBPF 与用户态数据交换的消息格式 | | **ghostscope-platform** | 平台抽象层 - 封装架构特定代码（调用约定、ABI） | @@ -202,9 +236,9 @@ TODO: 但是依然很慢，需要继续研究 GDB 是怎么提升解析 DWARF ### 7. eBPF 到用户态通信 -**核心机制**：Ring buffer（内核内存中的每 CPU 循环缓冲区）。 +**核心机制**：启动时选择内核事件传输。GhostScope 在内核支持时优先使用 RingBuf，否则回退到 PerfEventArray。RingBuf 是跨 CPU 共享的多生产者缓冲区；PerfEventArray 使用相互独立的 per-CPU 缓冲区。 -#### Ring Buffer 架构 +#### 事件传输架构 ``` ┌───────────────────────────────────────────────────────┐ @@ -217,11 +251,13 @@ TODO: 但是依然很慢，需要继续研究 GDB 是怎么提升解析 DWARF │ └─────┬──────┘ ↓ │ │ │ 序列化为协议格式 │ │ │ ↓ │ -│ │ bpf_ringbuf_output() │ +│ │ bpf_ringbuf_output() / │ +│ │ bpf_perf_event_output() │ │ │ ↓ │ │ └────────►┌─────────────────────┐ │ -│ │ Ring Buffer │ │ -│ │ (每CPU，256KB) │ │ +│ │ RingBuf（共享） │ │ +│ │ 或 Perf buffers │ │ +│ │ （per-CPU） │ │ │ │ │ │ │ │ [事件1][事件2]... │ │ │ └──────────┬──────────┘ │ @@ -233,7 +269,7 @@ TODO: 但是依然很慢，需要继续研究 GDB 是怎么提升解析 DWARF │ │ │ │ ┌──────────────────────────▼──────────┐ │ │ │ Trace Manager │ │ -│ │ (轮询 ring buffer) │ │ +│ │ (轮询当前传输) │ │ │ └──────────────────────┬───────────────┘ │ │ │ │ │ 读取事件（非阻塞） │ @@ -258,16 +294,17 @@ TODO: 但是依然很慢，需要继续研究 GDB 是怎么提升解析 DWARF - 目标指令执行时 Uprobe 触发 - eBPF 程序收集数据（通过 DWARF 位置读取寄存器、栈、内存） - 根据协议格式序列化数据 - - 调用 `bpf_ringbuf_output()` 写入 ring buffer + - 调用当前选择的 RingBuf 或 PerfEventArray 输出 helper + - 输出 helper 失败时增加该 trace 的丢失计数 2. **事件轮询**（用户空间）： - - Trace 管理器轮询 ring buffer（通过 Aya 框架） + - Trace 管理器通过 Aya 轮询当前选择的传输 - 非阻塞：如果没有事件立即返回 - - 内存映射：零拷贝访问内核缓冲区 + - RingBuf 使用共享的内存映射缓冲区；PerfEventArray 分别消费 per-CPU 缓冲区 3. **事件解析**： - 流式解析器处理可变长度消息 - - 状态机跟踪跨 chunk 的部分读取 + - 解析器按传输类型处理 framing，并跟踪 RingBuf 跨 chunk 的部分读取 - 重建完整事件 4. **事件投递**： @@ -275,6 +312,11 @@ TODO: 但是依然很慢，需要继续研究 GDB 是怎么提升解析 DWARF - 协调器通过通道转发到 UI - UI 实时更新显示 +5. **丢失报告**： + - 运行时定期读取每个 trace 的 eBPF 输出失败计数 + - CLI 和 TUI 报告区间增量与累计丢失量 + - 非零计数表示对应观测区间不完整 + #### 协议格式 GhostScope 使用**基于指令的协议**实现灵活的追踪事件表示： diff --git a/docs/zh/comparison.md b/docs/zh/comparison.md index f3b3bf2c..0b389b14 100644 --- a/docs/zh/comparison.md +++ b/docs/zh/comparison.md @@ -17,22 +17,22 @@ GhostScope 的目标很明确：**针对带有 DWARF 调试信息的活跃进程 | 特性 | GhostScope | GDB | |---|---|---| | 类型 | 追踪器，更接近生产环境下的 printf 调试 | 交互式调试器 | -| 执行模型 | 不停进程，直接做运行时追踪 | 停住进程，再检查和控制执行 | +| 执行模型 | 不由调试器控制暂停；每次命中仍会同步执行 uprobe 和 eBPF 程序 | 停住进程，再检查和控制执行 | | 运行时开销 | 选择性使用时通常较低 | 一旦依赖断点，开销通常较高 | -| 进程中断 | 从不 | 会 | +| 进程中断 | 不做 stop-the-world 检查；命中线程仍会同步承担探针工作 | 会，这是调试模型的一部分 | | 生产环境使用 | 面向生产友好的在线观测 | 更适合开发环境和事后调试 | -| 时序保持 | 是 | 否，断点和单步都会改变时序 | -| 并发调试 | 当你需要保留真实时序时更有优势 | stop-the-world 行为更容易扰动并发交错 | +| 时序影响 | 通常远小于交互式断点，但并非对时序透明 | 断点和单步会主动改变时序 | +| 并发调试 | 比 stop-the-world 检查更接近正常执行，但探针仍可能扰动并发交错 | stop-the-world 行为会明显扰动并发交错 | | 交互控制 | 有 TUI 和脚本，但不控制执行流程 | 完整执行控制：断点、单步、继续、改状态 | | 变量读取方式 | 在设定的追踪点上，借助 DWARF + eBPF 读值 | 在暂停后的时刻读取调试状态 | | 最擅长 | 生产风格的在线观测 | 交互式调试和状态修改 | | 适合的需求 | 你需要观测 | 你需要控制 | -如果时序不能被破坏，优先用 GhostScope。如果你需要一步一步推进执行过程，优先用 GDB。 +如果必须避免 stop-the-world，并且能够为每次命中的探针开销做预算，优先用 GhostScope。如果需要控制执行过程，优先用 GDB。两者都不能保证原始时序完全不变。 ## GhostScope vs GDB 实测性能快照 -现在仓库里已经有一套可重复执行的单线程 benchmark，用来回答一个很窄但很实际的问题：**在热点函数每次命中时，读取同一个局部变量到底要付出多大代价？** harness 在 [`../scripts/compare/compare_hot_function_bench.py`](../scripts/compare/compare_hot_function_bench.py)，目标程序在 [`../scripts/compare/compare_hot_function_target.c`](../scripts/compare/compare_hot_function_target.c)，runner service 的入口在 [`../e2e-tests/tests/manual_gdb_ghostscope_benchmark.rs`](../e2e-tests/tests/manual_gdb_ghostscope_benchmark.rs)。 +现在仓库里已经有一套可重复执行的单线程 benchmark，用来回答一个很窄但很实际的问题：**在热点函数每次命中时，读取同一个局部变量到底要付出多大代价？** harness 在 [`../../scripts/compare/compare_hot_function_bench.py`](../../scripts/compare/compare_hot_function_bench.py)，目标程序在 [`../../scripts/compare/compare_hot_function_target.c`](../../scripts/compare/compare_hot_function_target.c)，runner service 的入口在 [`../../e2e-tests/tests/manual_gdb_ghostscope_benchmark.rs`](../../e2e-tests/tests/manual_gdb_ghostscope_benchmark.rs)。 ### 方法 diff --git a/docs/zh/design-contract.md b/docs/zh/design-contract.md new file mode 100644 index 00000000..d354122c --- /dev/null +++ b/docs/zh/design-contract.md @@ -0,0 +1,132 @@ +# 设计保证与可信性模型 + +GhostScope 把运行中程序的机器状态转换为源码感知的观测结果。本文档定义这些结果在什么条件下应当可信，以及实现必须持续维护哪些不变量。 + +这是一份规范性的设计契约。[架构文档](architecture.md)说明当前实现如何保证这些不变量，[使用限制](limitations.md)说明支持范围之外或会显式降级的行为。未来才准备提供的保证应写入[未来规划](roadmap.md)，而不是本文档。 + +在支持范围内违反不变量属于正确性缺陷，而不是“使用限制”。项目处于早期开发阶段意味着仍可能存在缺陷，但不能因此让已知的不支持路径返回看似合理、实际没有证据的数据。 + +## 支持范围 + +- 官方发布产物和正确性测试面向 Linux x86_64。源码可以在其他架构上编译，不代表该架构受到支持。 +- 源码级值依赖与目标模块对应的 DWARF。内嵌 DWARF 直接来自目标模块；独立调试文件默认严格检查可用的 `.gnu_debuglink` CRC 和 Build ID 证据。 +- C 语言的端到端覆盖最完整。C++ 和 Rust 目前主要采用能力更有限的“按 DWARF 布局访问”模型。具体语言覆盖见[使用限制](limitations.md)。 +- 内核必须提供所需的 eBPF、uprobe 和事件传输能力，GhostScope 也必须具备使用这些设施的权限。 + +`--allow-loose-debug-match` 会显式退出独立调试文件的严格匹配。启用后 GhostScope 会对不匹配发出警告，但用户同时也削弱了源码语义解释所依赖的证据。如果一个显式指定的调试文件既没有可用 CRC，也没有可比较的 Build ID，系统同样无法证明它与目标模块一致；相应 warning 表示这成为用户提供的信任假设，不属于严格证据范围。 + +## 信任边界 + +GhostScope 依赖 Linux 内核、eBPF verifier 和 helper、目标模块及编译器生成的 DWARF，以及 `/proc` 暴露的进程信息。系统会尽可能验证并缩小这些输入的范围，但无法证明编译器生成的 DWARF 在语义上一定正确。 + +一个事件表示 uprobe 命中时的一次瞬时观测，而不是整个进程的原子快照。寄存器和当前线程栈帧对应命中时刻；在一个事件依次采集多个字段期间，其他线程仍可能修改共享内存。 + +只读也不等于零影响。Linux uprobe 会安装 trap 点，目标线程每次命中时会同步承担 uprobe 和 eBPF 的执行开销。GhostScope 不会像调试器一样控制目标暂停，也不会主动写入应用状态，但不能保证运行时序完全不变。 + +## 不变量 + +### SCOPE-1：支持平台必须明确 + +**保证。** 官方构建和正确性声明的范围是 Linux x86_64。 + +**保证机制。** 发布产物按 x86_64 构建和命名，架构相关的寄存器、ABI、TLS 与 unwind 行为也在该环境中验证。 + +**失败边界。** 在完成平台映射和端到端测试之前，其他架构不能被描述为已支持。源码偶然能够编译不构成支持证据。 + +### SAFE-1：观测不能控制目标程序 + +**保证。** GhostScope 不提供主动修改应用可见变量、内存或控制流的操作，也不会为了交互式检查、单步或继续执行而暂停进程。 + +**保证机制。** 脚本语言只提供观测能力，生成的 eBPF 程序使用目标读取、内部 map 和事件输出操作，内核 verifier 对加载程序施加约束。 + +**失败边界。** Uprobe trap 和 eBPF 执行仍然会扰动调度和延迟。任何写入目标内存或改变控制流的能力都会改变这份契约，不能作为普通追踪功能引入。 + +### IDENT-1：每个观测结果必须符合请求的目标范围 + +**保证。** 每个事件必须能够归属于产生它的 trace 和运行时目标范围。当请求的 PID、模块或地址空间解释不可用时，GhostScope 不能静默改用另一个范围。 + +启动模式定义了目标范围： + +- `-p ` 选择该进程及其已加载模块形成的进程视图。 +- `-t ` 选择以目标模块为中心的多进程视图。 +- `-t -p ` 在指定模块内解析 trace 目标，并把运行时事件限定到指定进程。 + +**保证机制。** Trace ID、PID/TID 元数据、PID 过滤、感知 PID namespace 的进程发现、模块 Cookie 和运行时模块偏移，把身份信息贯穿附加、采集和展示过程。 + +**失败边界。** 无法观察进程、缺少模块映射或目标存在歧义时，必须让 setup 失败，或者明确标记依赖该信息的语义操作不可用，不能回退到无关范围。 + +### SEM-1：源码语义必须具有完整证据链 + +**保证。** 只有在目标模块和 probe PC 能够连接到适用的 DWARF 作用域、类型、位置以及运行时读取计划时，GhostScope 才会展示源码级值。 + +证据链为： + +```text +目标范围 -> 运行时模块 -> 模块内 PC -> DWARF 作用域/类型/位置 + -> 类型化读取计划 -> 运行时读取结果 -> 展示结果 +``` + +**保证机制。** 严格调试文件匹配、模块感知的 PC 上下文、ASLR/PIE 重定位、词法与 inline 作用域解析、类型化 lowering 和运行时读取状态，共同保持每一步的含义。 + +**失败边界。** 任意一步无法成立时，GhostScope 必须报告不支持的位置、`OptimizedOut`、读取错误、`ExprError` 或其他明确的不可用状态。不能改用其他作用域里的同名变量，不能猜测地址，也不能把未验证的值展示为有效值。 + +### FAIL-1：已知的不确定性必须显式呈现 + +**保证。** 已知的编译、附加、读取、unwind、解析或归因失败，必须对用户或自动化调用方保持可见。 + +**保证机制。** Setup 和 lowering 失败以错误返回；运行时表达式与内存读取失败使用结构化协议状态；调用栈使用带原因的 complete、truncated 和 stopped 状态。 + +**失败边界。** Best-effort 行为只有在缺失部分被明确标记时才能返回部分结果。仅记录 warning，同时输出一个看似完整但并不受支持的值，不满足这个不变量。 + +### LOSS-1：可检测的事件丢失不能静默发生 + +**保证。** Trace 成功启动不代表事件流无损。当 eBPF 输出 helper 拒绝事件时，GhostScope 会按 trace 统计失败次数，并向 CLI 和 TUI 用户报告区间增量与累计丢失量。 + +**保证机制。** RingBuf 或 PerfEventArray 输出失败时，生成的程序会更新丢失计数；运行时定期读取并报告这些计数。 + +**失败边界。** 这些计数覆盖内核输出 helper 的失败。它们不能说明具体丢失了哪些事件，也不能证明该计量点之外没有丢失，例如附加之前或关闭之后的事件。任何非零报告都表示对应观测区间不完整。 + +### COST-1：单事件工作量必须有界，总体影响取决于负载 + +**保证。** 生成的 eBPF 执行、trace 事件大小、内存读取和 backtrace 深度，都受到 verifier 兼容限制或可配置上限约束。达到语义或资源边界时，必须拒绝 trace，或者产生明确的截断/不可用状态。 + +**保证机制。** 编译器限制事件大小和读取范围，verifier 检查 eBPF 控制流，传输层具有配置容量，DWARF unwind 使用深度和 tail-call 预算。 + +**失败边界。** GhostScope 不承诺固定的总体开销。即使单次执行有界，把探针放在热点路径上仍可能显著拖慢目标。操作方必须把命中频率和每次事件负载纳入开销预算。 + +## 结果状态 + +这些不变量导出四种不同结果： + +| 结果 | 含义 | +|---|---| +| 完整 | 请求的操作在支持范围内完整执行。 | +| 显式部分结果 | 部分请求数据不可用或被截断，事件携带相应状态。 | +| 拒绝 | Setup、编译、验证或附加过程无法建立所需条件。 | +| 事件流不完整 | 检测到一个或多个事件丢失；已收到的事件仍可能有效，但对应区间并不完整。 | + +这些状态不能被合并成一个笼统的成功路径。 + +## 验证要求 + +测试是这份契约的证据，而不是契约本身的定义。当前主要证据分布如下： + +| 不变量 | 主要证据 | +|---|---| +| `SCOPE-1` | x86_64 release workflow 和平台相关单元测试 | +| `SAFE-1` | 脚本/编译器操作面、eBPF helper 使用范围、经过 verifier 的加载测试 | +| `IDENT-1` | PID 专属执行测试和容器拓扑测试 | +| `SEM-1` | PC 上下文、标量、全局变量、优化代码和跨模块 fixture 的精确值 oracle | +| `FAIL-1` | 编译错误、`OptimizedOut`、`ExprError`、读取失败和 backtrace 状态测试 | +| `LOSS-1` | eBPF 输出失败计数与 CLI/TUI 报告链路；修改这条路径时必须补充压力覆盖 | +| `COST-1` | 编译器边界检查、经过 verifier 的加载、配置深度和深栈 backtrace 测试 | + +修改目标选择、进程身份、模块/PC 映射、DWARF lowering、生成的 helper、事件协议、传输层或展示逻辑时，验证说明应指出受影响的不变量，并提供一个正向 oracle 和一个相关失败路径 oracle。 + +## 文档维护规则 + +- 当前已经执行的保证写在本文档中。 +- 支持但会降级的场景和不支持的条件写入[使用限制](limitations.md)。 +- 未来准备实现的保证写入[未来规划](roadmap.md)。 +- README 和工具对比中的用户承诺不能强于这份契约。 +- 契约变更时必须同步更新中英文版本。 diff --git a/docs/zh/development.md b/docs/zh/development.md index e96b224e..90650dee 100644 --- a/docs/zh/development.md +++ b/docs/zh/development.md @@ -144,6 +144,24 @@ docker build -t ghostscope-builder:ubuntu20.04 -f docker/base-build/Dockerfile . ## 测试 +### 面向不变量的验证 + +[设计保证与可信性模型](design-contract.md)定义验证必须保护的行为。测试需要证明可观测契约，而不能只证明命令成功退出。 + +| 不变量 | 受影响时的最低证据要求 | +|---|---| +| `SCOPE-1` | 构建/发布目标仍是 Linux x86_64，并具有平台相关单元测试 | +| `SAFE-1` | 审查新增的 helper/操作，并为新的 eBPF 行为提供经过 verifier 的加载测试 | +| `IDENT-1` | 正向目标归因，以及错误 PID/模块/namespace 的负向场景 | +| `SEM-1` | 在已知 PC 上使用具有精确源码级值 oracle 的 fixture，并覆盖相关优化或模块变体 | +| `FAIL-1` | 使用负向 fixture 断言结构化错误、不可用标记或停止原因 | +| `LOSS-1` | 验证计数传递和 CLI/TUI 展示；传输行为变化时补充压力覆盖 | +| `COST-1` | 验证边界/拒绝行为，并为事件大小、读取或 unwind 预算变化提供经过 verifier 的加载测试 | + +主要运行时证据位于 `e2e-tests/tests/`：`script_execution.rs` 覆盖 PID 过滤，`container_topology_execution.rs` 覆盖 namespace 行为，`member_pointer_compilation.rs` 覆盖 PC 上下文语义，`optimized_inline_call_value_execution.rs` 覆盖优化代码中的值，`globals_execution.rs` 覆盖结构化表达式失败，`backtrace_execution.rs` 覆盖 unwind 状态与深度。 + +行为变化需要在 review 或交接说明中指出受影响的不变量，运行对应的正向 oracle，并至少运行一个相关失败路径 oracle。只有断言结果与不变量一致时，测试命令才构成有效证据。 + ### Workspace 测试 ```bash diff --git a/docs/zh/limitations.md b/docs/zh/limitations.md index 7b4cf1d8..019cb992 100644 --- a/docs/zh/limitations.md +++ b/docs/zh/limitations.md @@ -1,18 +1,20 @@ # GhostScope 的限制 +本文档描述支持范围之外的条件，以及会显式降级的场景。在支持范围内仍然必须成立的保证，统一定义在[设计保证与可信性模型](design-contract.md)中。 + ## 硬性限制 ### 1. 权限需求 eBPF 底层机制要求提升的权限以访问内核追踪基础设施，需要 root 权限或 CAP_BPF/CAP_SYS_ADMIN 能力。 ### 2. 不要与 GDB 同时使用 -uprobe 和 GDB 都会修改目标进程的指令（插入断点），同时使用可能导致冲突和不可预测的行为。 +uprobe 和 GDB 都会在目标进程中安装基于断点的探测机制，同时使用可能导致冲突和不可预测的行为。 ### 3. 只读访问，不可修改程序行为 -虽然 eBPF 在技术上支持一定程度上修改进程行为，但是 GhostScope 设计为只读工具，无法修改程序状态、变量值或控制流。这确保了在生产环境中的安全性。 +GhostScope 的脚本和编译器能力面只用于观测，不提供主动修改应用可见状态、变量值或控制流的操作。但这不代表目标完全不受影响：uprobe 会安装 trap 点，每次命中都会让目标线程同步承担 uprobe 和 eBPF 执行。 -### 4. 平台支持 -目前仅支持 **Linux** 操作系统，因为核心技术依赖 **eBPF** 和 **uprobe**。 +### 4. 平台与架构支持 +官方构建和正确性测试目前只支持 **Linux x86_64 (AMD64)**。GhostScope 依赖 Linux **eBPF** 和 **uprobe**，并且当前实现的是 x86_64 寄存器、ABI、TLS 和 unwind 行为。即使源码偶然能在其他架构上编译，这些架构仍不在支持契约内。 ## 软性限制 @@ -21,10 +23,7 @@ uprobe 和 GDB 都会修改目标进程的指令（插入断点），同时使 对于解释型语言（Lua、Python、Ruby 等），目前只能追踪解释器本身（因为解释器通常用编译型语言实现）。追踪脚本代码在技术上可行，但需要大量开发时间，JIT 语言支持计划就更远了。 -### 2. 架构支持 -目前仅支持 **x86_64 (AMD64)** 架构。其他平台（如 ARM64）技术上可行，但需要投入时间适配和测试。 - -### 3. 通过 `bpf_probe_read_user` 读取用户内存 +### 2. 通过 `bpf_probe_read_user` 读取用户内存 在传统的非 sleepable probe 路径里，`bpf_probe_read_user` 这类 helper 仍然不能处理用户态缺页，因此当目标虚拟地址对应的页面尚未驻留，或访问时会触发 fault，读取就可能失败。 但这已经不是 eBPF 的绝对硬限制。Linux 已支持 sleepable uprobe（`uprobe.s` / `uretprobe.s`），而 sleepable 程序可以使用 `bpf_copy_from_user_task()` 之类的 helper 执行可睡眠的用户态内存读取。GhostScope 当前仍生成普通 `uprobe` 程序，所以现阶段它在实践中仍然是个限制；但更准确地说，这属于实现层面的软限制，而不是 eBPF 的根本设计上限。 @@ -34,26 +33,30 @@ uprobe 和 GDB 都会修改目标进程的指令（插入断点），同时使 - https://docs.kernel.org/bpf/libbpf/program_types.html - https://man7.org/linux/man-pages/man7/bpf-helpers.7.html -### 4. 性能影响 -基于 uprobe 的实现，每次探针触发会产生一次上下文切换开销以及 eBPF 程序执行时间。如果探针设置在热点路径上，可能对被监测进程造成明显的性能影响，使用时需要谨慎。 +### 3. 性能影响 +每次 uprobe 命中都会同步执行 trap 处理和生成的 eBPF 程序。即使单事件工作量有界，把探针放在热点路径上仍可能显著影响目标进程。GhostScope 不保证原始时序完全不变，也不承诺固定的总体开销；应当有针对性地设置探针，并把命中频率纳入性能预算。 + +### 4. 事件丢失（背压） +GhostScope 优先使用 RingBuf，并在需要时回退到 PerfEventArray。事件产生速度超过可用容量时，两种传输都可能拒绝输出。生成的程序会按 trace 增加输出失败计数，CLI 和 TUI 会定期报告区间增量和累计丢失量。 -### 5. 事件丢失（背压） -使用 ring buffer 在 eBPF 程序和用户态之间传递事件。如果事件产生速率过快，超过用户态消费能力，内核会丢弃事件，导致追踪数据缺失。目前 GhostScope 的错误提示还不够完善（后面会学习 bpftrace 是怎么处理这种的），建议避免在高频路径上设置过多探针。 +这保证的是丢失可见，而不是事件流无损。计数只覆盖 eBPF 输出 helper 的失败，不能说明具体丢失了哪些事件，也不能证明该计量点之外没有丢失。任何非零丢失报告都表示对应观测区间不完整；同时应避免在高频路径上放置数量过多或负载过大的探针。 -### 6. DWARF 支持范围 +### 5. DWARF 支持范围 主要测试和验证了 DWARF 5 格式。理论上支持 DWARF 2-5，但其他版本可能存在兼容性问题。部分 DWARF 表达式指令暂不支持转换为 eBPF（纯实现还没有支持的原因），遇到时会给出明确的错误提示。 GhostScope 能识别 `DW_OP_form_tls_address`，但运行时 TLS 地址解析目前只支持 x86_64 可执行文件里的 static TLS。GhostScope 会在探针触发时解析当前线程的 TLS 基址，因此在这个已支持的可执行文件场景下，同一个 trace 在不同 pthread 上会读取各自线程的 TLS 实例。同一个 DWARF 操作也会用于 dynamic/shared-library TLS；这类场景需要 DTV/module TLS lookup，目前尚未建模，所以 GhostScope 会拒绝共享对象里的 TLS，而不是猜测地址。 -### 7. 栈回溯覆盖范围 +GhostScope 默认严格检查独立调试文件所有可用的 `.gnu_debuglink` CRC 和可比较 Build ID。启用 `--allow-loose-debug-match` 后，可以在 warning 提示下使用不匹配的调试文件，但这会削弱源码语义所依赖的证据保证。显式指定的调试文件如果既没有可用 CRC，也没有可比较的 Build ID，同样会在 warning 后继续使用，但此时文件身份属于用户提供的信任假设，而不是经过验证的匹配。 + +### 6. 栈回溯覆盖范围 `bt` 只使用 DWARF CFI。GhostScope 不会回退到内核 stack helper 或 frame pointer walking；当 CFI 不可用、无法转换为 compact eBPF fast path，或读取用户栈内存失败时，会输出明确的停止状态。只要进程模块映射可用，跨模块栈帧可以通过 raw IP 做正确符号化；运行时模块刷新也可以为新映射模块追加 compact DWARF rows，直到达到 `backtrace_unwind_rows_max_entries`。如果某个 trace 事件早于 map-change 刷新抵达用户态，它仍可能先停在新加载模块，后续事件才能看到已追加的 rows。深栈 DWARF unwind 已经通过 eBPF tail-call step program 分段执行，因此默认 `backtrace_depth = 128` 不会触发 LLVM 分支距离和 verifier 程序大小限制；`status=truncated` 表示达到配置深度或 tail-call unwind 预算后仍未自然停止。 运行模式也会影响 `bt` 的覆盖面。`-p ` 是进程级视图，GhostScope 会从该 PID 的 `/proc//maps` 加载已映射模块，因此跨模块 unwind 和符号化通常最完整。独立 `-t ` 是目标文件级、多进程 trace 视图，主要保证目标模块内的探针和变量；调用栈跨出目标模块后属于 best-effort，依赖运行时模块映射、`proc_module_offsets` 维护结果以及相关模块是否有可用 compact DWARF CFI。若既需要限定 trace 目标模块，又需要更完整的单进程 backtrace，优先使用 `-t -p `。 -### 8. 高度优化代码的支持 +### 7. 高度优化代码的支持 编译器优化（-O2、-O3）会导致变量被优化掉或生成复杂的 DWARF 表达式。GhostScope 会尽力解析，包括内联函数的支持，但部分变量可能无法访问（显示为 OptimizedOut），这是因为编译器优化掉了。 -### 9. 动态加载库（dlopen） +### 8. 动态加载库（dlopen） GhostScope 启动时会扫描进程的 `/proc/PID/maps` 获取已加载的动态库信息；现在在 `-p ` 和启用 sysmon 的独立 `-t ` 下，也会通过运行时 map-change 监控刷新模块映射。对 `bt`/`backtrace` 来说，这可以为后续通过 `dlopen` 加载的库追加 compact DWARF CFI rows 和模块偏移，但仍受 `backtrace_unwind_rows_max_entries` 以及上文提到的 map-change 竞态限制。 `-p ` 有一个启动边界：trace setup 会先快照该 PID 当前的 @@ -66,7 +69,7 @@ setup 可能在运行时 map-change 监控有机会刷新之前失败。这主 这条运行时刷新链路不会自动新增 trace probe，也不会让脚本编译/附加时尚未知的库立即支持 print 或全局变量目标。这些目标仍依赖 trace setup 阶段能够看到对应目标模块和调试信息。 -### 10. -t 模式下的全局变量支持 +### 9. -t 模式下的全局变量支持 - **可执行目标**：当 `-t` 指向可执行文件（`-t /path/to/app`）时，会以该二进制作为主模块，默认支持全局变量。 - **共享库目标（已有进程）**：若 GhostScope 启动时，目标库已经被现有进程加载，例如追踪一个已运行但早先加载好 `libfoo.so` 的进程，能够直接解析全局变量。 @@ -77,7 +80,11 @@ setup 可能在运行时 map-change 监控有机会刷新之前失败。这主 `-t` 下的全局变量依赖 `proc_module_offsets`，也就是按 `(pid, module)` 维护的运行时地址偏移。独立 `-t` 会面向多个 PID 维护目标模块偏移，因此适合观察目标模块里的全局变量；但它不是完整进程视图，`bt` 跨出目标模块后的 unwind/符号化能力不等同于 `-p`。需要单个进程的完整模块上下文时，使用 `-p `；需要把 trace 目标限定到某个模块并保留该进程上下文时，使用 `-t -p `。 -### 11. 容器 / WSL 场景下 `-p ` 模式的软限制 +### 10. 容器 / WSL 场景下 `-p ` 模式的软限制 - 容器 / WSL 场景、PID namespace 术语、场景矩阵，以及当前实现限制的完整说明，见 [容器环境](container.md)。 - 参考 [PID namespaces 手册](https://www.man7.org/linux/man-pages/man7/pid_namespaces.7.html)、[WSL issue #12408](https://github.com/microsoft/WSL/issues/12408) 和 [WSL issue #12115](https://github.com/microsoft/WSL/issues/12115)。 + +### 11. 观测一致性 + +一个 trace 事件不是整个进程的原子快照。寄存器值和当前线程栈帧对应 uprobe 命中时刻，但 GhostScope 依次读取多个字段期间，其他线程仍可能修改共享内存。因此，同一个事件中的多个值可能来自一个很短的观测区间，而不是经过全局同步的同一瞬间。 diff --git a/docs/zh/scripting.md b/docs/zh/scripting.md index 06a4114c..f4cf15c8 100644 --- a/docs/zh/scripting.md +++ b/docs/zh/scripting.md @@ -757,7 +757,7 @@ backtrace: truncated, 2 frames (max 2) `status=complete` 表示 DWARF unwind 在达到配置的深度上限前自然结束。`status=truncated` 表示 GhostScope 达到了配置的深度上限，或先达到了 eBPF tail-call unwind 预算。其他状态会说明 unwind 停止的原因，例如当前 PC 没有可用的 unwind row、CFI 不支持、模块偏移不可用、读取用户栈内存失败，或下一帧地址/CFA 不合法。可用时，`stopped:` 会附带稳定的原因标签和数字 code。 -只要进程模块映射和对应模块的 compact DWARF CFI 可用，`bt` 可以跨模块 unwind。在 `-p` 和启用 sysmon 的独立 `-t` 下，运行时模块刷新可以为后续通过 `dlopen` 加载的库补充 backtrace 元数据；它不会为 setup 阶段尚未知的目标自动新增 trace probe。具体覆盖边界见[使用限制](limitations.md#7-栈回溯覆盖范围)。 +只要进程模块映射和对应模块的 compact DWARF CFI 可用，`bt` 可以跨模块 unwind。在 `-p` 和启用 sysmon 的独立 `-t` 下，运行时模块刷新可以为后续通过 `dlopen` 加载的库补充 backtrace 元数据；它不会为 setup 阶段尚未知的目标自动新增 trace probe。具体覆盖边界见[使用限制](limitations.md#6-栈回溯覆盖范围)。 ## 示例 @@ -905,7 +905,7 @@ trace foo { - 所有变量都是动态类型 - 字符串字面量必须使用双引号 - 大多数语句需要分号 -- 追踪模式匹配支持文件模糊匹配（参见[命令参考](command-reference.md)） +- 追踪模式匹配支持文件模糊匹配（参见[命令参考](input-commands.md)） ## 运行时表达式失败（ExprError） diff --git a/e2e-tests/README.md b/e2e-tests/README.md index 410d670c..ca14540a 100644 --- a/e2e-tests/README.md +++ b/e2e-tests/README.md @@ -3,6 +3,12 @@ This crate contains the workspace-level end-to-end test suite for the `ghostscope` CLI. +The suite provides runtime evidence for the project's +[Design Guarantees and Trust Model](../docs/design-contract.md). E2E tests +should assert a domain oracle, such as an exact source-level value, target PID, +explicit failure state, loss report, or backtrace stop reason. Process exit +status alone is not sufficient evidence for an invariant. + Routine workspace commands exclude this crate via workspace `default-members`, so standard `cargo test` stays focused on unit and integration coverage for the main workspace crates.