From f3b348358dfd6ed12bc3c2198dda91b9c49fe0c4 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 24 Aug 2026 07:57:41 +0100 Subject: [PATCH] refactor: migrate repository documentation from Markdown to AsciiDoc --- .claude/PROJECT.adoc | 34 + .claude/PROJECT.md | 37 - ABI-FFI-README.md => ABI-FFI-README.adoc | 244 +++--- ARCHITECTURE.adoc | 48 ++ ARCHITECTURE.md | 47 -- CHANGELOG.adoc | 225 +++++ CHANGELOG.md | 202 ----- CODE_OF_CONDUCT.adoc | 134 +++ CODE_OF_CONDUCT.md | 137 ---- CONTRIBUTING.adoc | 109 +++ CONTRIBUTING.md | 120 --- GOVERNANCE.adoc | 178 +--- GOVERNANCE.md | 60 -- MAINTAINERS.adoc | 173 +++- MAINTAINERS.md | 154 ---- PROOF-NEEDS.adoc | 103 +++ PROOF-NEEDS.md | 75 -- PROOF-STATUS.adoc | 135 +++ PROOF-STATUS.md | 90 -- README.adoc | 634 +++----------- README.md | 131 --- RUST-STATUS.adoc | 152 ++++ RUST-STATUS.md | 152 ---- SAFETY-FEATURES.adoc | 335 ++++++++ SAFETY-FEATURES.md | 288 ------- SECURITY.adoc | 226 +++++ SECURITY.md | 219 ----- TOOLING.adoc | 92 +++ TOOLING.md | 89 -- TPCF.adoc | 351 ++++++++ TPCF.md | 363 -------- conformance/README.adoc | 83 ++ conformance/README.md | 86 -- ...-COMPARISON.md => BETLANG-COMPARISON.adoc} | 10 +- docs/api-reference.adoc | 725 ++++++++++++++++ docs/api-reference.md | 672 --------------- docs/architecture.adoc | 8 + docs/architecture.md | 12 - docs/experimental-ideas.adoc | 63 ++ docs/experimental-ideas.md | 41 - docs/hyperpolymath-julia-integration.adoc | 597 ++++++++++++++ docs/hyperpolymath-julia-integration.md | 560 ------------- docs/julia-backend-design.adoc | 602 ++++++++++++++ docs/julia-backend-design.md | 542 ------------ docs/proof-debt.adoc | 221 +++-- docs/proof-debt.md | 147 ---- docs/semantics.adoc | 394 +++++++++ docs/semantics.md | 343 -------- docs/tech-debt-2026-05-26.adoc | 84 ++ docs/tech-debt-2026-05-26.md | 70 -- docs/{tutorial.md => tutorial.adoc} | 395 +++++---- docs/v3-optimization-workflow.adoc | 38 + docs/v3-optimization-workflow.md | 30 - editors/neovim/README.adoc | 123 +++ editors/neovim/README.md | 118 --- editors/vscode/README.adoc | 43 + editors/vscode/README.md | 37 - julia-backend/README.adoc | 268 ++++++ julia-backend/README.md | 244 ------ proofs/README.adoc | 256 ++++++ proofs/README.md | 180 ---- .../complexity/computational-complexity.adoc | 772 ++++++++++++++++++ proofs/complexity/computational-complexity.md | 656 --------------- proofs/formal-semantics.adoc | 607 ++++++++++++++ proofs/formal-semantics.md | 392 --------- proofs/mathematical-foundations.adoc | 522 ++++++++++++ proofs/mathematical-foundations.md | 376 --------- proofs/papers/betlang-whitepaper.adoc | 461 +++++++++++ proofs/papers/betlang-whitepaper.md | 378 --------- .../betlang-whitepaper.md.invariants.adoc | 26 + .../betlang-whitepaper.md.invariants.md | 32 - proofs/papers/ppl-comparison.adoc | 598 ++++++++++++++ proofs/papers/ppl-comparison.md | 501 ------------ proofs/theorems/category-theory.adoc | 596 ++++++++++++++ proofs/theorems/category-theory.md | 483 ----------- proofs/theorems/convergence-statistics.adoc | 660 +++++++++++++++ proofs/theorems/convergence-statistics.md | 424 ---------- proofs/theorems/information-theory.adoc | 641 +++++++++++++++ proofs/theorems/information-theory.md | 409 ---------- proofs/theorems/mcmc-inference.adoc | 612 ++++++++++++++ proofs/theorems/mcmc-inference.md | 426 ---------- proofs/theorems/measure-theory.adoc | 612 ++++++++++++++ proofs/theorems/measure-theory.md | 436 ---------- proofs/theorems/soundness-completeness.adoc | 550 +++++++++++++ proofs/theorems/soundness-completeness.md | 442 ---------- proofs/theorems/termination-analysis.adoc | 610 ++++++++++++++ proofs/theorems/termination-analysis.md | 541 ------------ proofs/theorems/type-theory.adoc | 599 ++++++++++++++ proofs/theorems/type-theory.md | 500 ------------ ...rification.md => formal-verification.adoc} | 581 +++++++------ site/index.adoc | 26 + site/index.md | 25 - spec/system-specs.adoc | 168 ++++ spec/system-specs.md | 147 ---- 94 files changed, 14372 insertions(+), 12766 deletions(-) create mode 100644 .claude/PROJECT.adoc delete mode 100644 .claude/PROJECT.md rename ABI-FFI-README.md => ABI-FFI-README.adoc (74%) create mode 100644 ARCHITECTURE.adoc delete mode 100644 ARCHITECTURE.md create mode 100644 CHANGELOG.adoc delete mode 100644 CHANGELOG.md create mode 100644 CODE_OF_CONDUCT.adoc delete mode 100644 CODE_OF_CONDUCT.md create mode 100644 CONTRIBUTING.adoc delete mode 100644 CONTRIBUTING.md delete mode 100644 GOVERNANCE.md delete mode 100644 MAINTAINERS.md create mode 100644 PROOF-NEEDS.adoc delete mode 100644 PROOF-NEEDS.md create mode 100644 PROOF-STATUS.adoc delete mode 100644 PROOF-STATUS.md delete mode 100644 README.md create mode 100644 RUST-STATUS.adoc delete mode 100644 RUST-STATUS.md create mode 100644 SAFETY-FEATURES.adoc delete mode 100644 SAFETY-FEATURES.md create mode 100644 SECURITY.adoc delete mode 100644 SECURITY.md create mode 100644 TOOLING.adoc delete mode 100644 TOOLING.md create mode 100644 TPCF.adoc delete mode 100644 TPCF.md create mode 100644 conformance/README.adoc delete mode 100644 conformance/README.md rename docs/{BETLANG-COMPARISON.md => BETLANG-COMPARISON.adoc} (96%) create mode 100644 docs/api-reference.adoc delete mode 100644 docs/api-reference.md create mode 100644 docs/architecture.adoc delete mode 100644 docs/architecture.md create mode 100644 docs/experimental-ideas.adoc delete mode 100644 docs/experimental-ideas.md create mode 100644 docs/hyperpolymath-julia-integration.adoc delete mode 100644 docs/hyperpolymath-julia-integration.md create mode 100644 docs/julia-backend-design.adoc delete mode 100644 docs/julia-backend-design.md delete mode 100644 docs/proof-debt.md create mode 100644 docs/semantics.adoc delete mode 100644 docs/semantics.md create mode 100644 docs/tech-debt-2026-05-26.adoc delete mode 100644 docs/tech-debt-2026-05-26.md rename docs/{tutorial.md => tutorial.adoc} (60%) create mode 100644 docs/v3-optimization-workflow.adoc delete mode 100644 docs/v3-optimization-workflow.md create mode 100644 editors/neovim/README.adoc delete mode 100644 editors/neovim/README.md create mode 100644 editors/vscode/README.adoc delete mode 100644 editors/vscode/README.md create mode 100644 julia-backend/README.adoc delete mode 100644 julia-backend/README.md create mode 100644 proofs/README.adoc delete mode 100644 proofs/README.md create mode 100644 proofs/complexity/computational-complexity.adoc delete mode 100644 proofs/complexity/computational-complexity.md create mode 100644 proofs/formal-semantics.adoc delete mode 100644 proofs/formal-semantics.md create mode 100644 proofs/mathematical-foundations.adoc delete mode 100644 proofs/mathematical-foundations.md create mode 100644 proofs/papers/betlang-whitepaper.adoc delete mode 100644 proofs/papers/betlang-whitepaper.md create mode 100644 proofs/papers/betlang-whitepaper.md.invariants.adoc delete mode 100644 proofs/papers/betlang-whitepaper.md.invariants.md create mode 100644 proofs/papers/ppl-comparison.adoc delete mode 100644 proofs/papers/ppl-comparison.md create mode 100644 proofs/theorems/category-theory.adoc delete mode 100644 proofs/theorems/category-theory.md create mode 100644 proofs/theorems/convergence-statistics.adoc delete mode 100644 proofs/theorems/convergence-statistics.md create mode 100644 proofs/theorems/information-theory.adoc delete mode 100644 proofs/theorems/information-theory.md create mode 100644 proofs/theorems/mcmc-inference.adoc delete mode 100644 proofs/theorems/mcmc-inference.md create mode 100644 proofs/theorems/measure-theory.adoc delete mode 100644 proofs/theorems/measure-theory.md create mode 100644 proofs/theorems/soundness-completeness.adoc delete mode 100644 proofs/theorems/soundness-completeness.md create mode 100644 proofs/theorems/termination-analysis.adoc delete mode 100644 proofs/theorems/termination-analysis.md create mode 100644 proofs/theorems/type-theory.adoc delete mode 100644 proofs/theorems/type-theory.md rename proofs/verification/{formal-verification.md => formal-verification.adoc} (65%) create mode 100644 site/index.adoc delete mode 100644 site/index.md create mode 100644 spec/system-specs.adoc delete mode 100644 spec/system-specs.md diff --git a/.claude/PROJECT.adoc b/.claude/PROJECT.adoc new file mode 100644 index 0000000..2fc87eb --- /dev/null +++ b/.claude/PROJECT.adoc @@ -0,0 +1,34 @@ +== BetLang - Claude Code Instructions + +This repository contains the BetLang programming language compiler and +tooling. + +=== Project Structure + +.... +betlang/ +├── .claude/ # AI assistant instructions +├── .git/ # Version control +├── .gitignore # Git ignore rules +├── .editorconfig # Editor configuration +└── ... # Compiler files +.... + +=== Build Commands + +Refer to project-specific documentation. + +=== Coding Conventions + +* Follow hyperpolymath standards +* All code must have SPDX license headers +* Use approved languages only (see CLAUDE.md) +* Document all non-obvious decisions + +=== Security + +* No hardcoded secrets +* All secrets through environment variables or secret management +* SHA-pinned dependencies where applicable +* HTTPS only, no HTTP URLs +* No MD5/SHA1 for security purposes diff --git a/.claude/PROJECT.md b/.claude/PROJECT.md deleted file mode 100644 index c883570..0000000 --- a/.claude/PROJECT.md +++ /dev/null @@ -1,37 +0,0 @@ - -# BetLang - Claude Code Instructions - -This repository contains the BetLang programming language compiler and tooling. - -## Project Structure - -``` -betlang/ -├── .claude/ # AI assistant instructions -├── .git/ # Version control -├── .gitignore # Git ignore rules -├── .editorconfig # Editor configuration -└── ... # Compiler files -``` - -## Build Commands - -Refer to project-specific documentation. - -## Coding Conventions - -- Follow hyperpolymath standards -- All code must have SPDX license headers -- Use approved languages only (see CLAUDE.md) -- Document all non-obvious decisions - -## Security - -- No hardcoded secrets -- All secrets through environment variables or secret management -- SHA-pinned dependencies where applicable -- HTTPS only, no HTTP URLs -- No MD5/SHA1 for security purposes diff --git a/ABI-FFI-README.md b/ABI-FFI-README.adoc similarity index 74% rename from ABI-FFI-README.md rename to ABI-FFI-README.adoc index 8010527..04be86a 100644 --- a/ABI-FFI-README.md +++ b/ABI-FFI-README.adoc @@ -1,23 +1,22 @@ - -{{~ Aditionally delete this line and fill out the template below ~}} +\{\{~ Aditionally delete this line and fill out the template below ~}} -# {{PROJECT}} ABI/FFI Documentation +== \{\{PROJECT}} ABI/FFI Documentation -## Overview +=== Overview -This library follows the **Hyperpolymath RSR Standard** for ABI and FFI design: +This library follows the *Hyperpolymath RSR Standard* for ABI and FFI +design: -- **ABI (Application Binary Interface)** defined in **Idris2** with formal proofs -- **FFI (Foreign Function Interface)** implemented in **Zig** for C compatibility -- **Generated C headers** bridge Idris2 ABI to Zig FFI -- **Any language** can call through standard C ABI +* *ABI (Application Binary Interface)* defined in *Idris2* with formal +proofs +* *FFI (Foreign Function Interface)* implemented in *Zig* for C +compatibility +* *Generated C headers* bridge Idris2 ABI to Zig FFI +* *Any language* can call through standard C ABI -## Architecture +=== Architecture -``` +.... ┌─────────────────────────────────────────────┐ │ ABI Definitions (Idris2) │ │ src/abi/ │ @@ -49,11 +48,11 @@ This library follows the **Hyperpolymath RSR Standard** for ABI and FFI design: │ Any Language via C ABI │ │ - Rust, AffineScript, Julia, Python, etc. │ └─────────────────────────────────────────────┘ -``` +.... -## Directory Structure +=== Directory Structure -``` +.... {{project}}/ ├── src/ │ ├── abi/ # ABI definitions (Idris2) @@ -81,15 +80,17 @@ This library follows the **Hyperpolymath RSR Standard** for ABI and FFI design: ├── rust/ ├── affinescript/ └── julia/ -``` +.... -## Why Idris2 for ABI? +=== Why Idris2 for ABI? -### 1. **Formal Verification** +==== 1. *Formal Verification* -Idris2's dependent types allow proving properties about the ABI at compile-time: +Idris2’s dependent types allow proving properties about the ABI at +compile-time: -```idris +[source,idris] +---- -- Prove struct size is correct public export exampleStructSize : HasSize ExampleStruct 16 @@ -101,13 +102,14 @@ fieldAligned : Divides 8 (offsetOf ExampleStruct.field) -- Prove ABI is platform-compatible public export abiCompatible : Compatible (ABI 1) (ABI 2) -``` +---- -### 2. **Type Safety** +==== 2. *Type Safety* Encode invariants that C/Zig cannot express: -```idris +[source,idris] +---- -- Non-null pointer guaranteed at type level data Handle : Type where MkHandle : (ptr : Bits64) -> {auto 0 nonNull : So (ptr /= 0)} -> Handle @@ -115,13 +117,14 @@ data Handle : Type where -- Array with length proof data Buffer : (n : Nat) -> Type where MkBuffer : Vect n Byte -> Buffer n -``` +---- -### 3. **Platform Abstraction** +==== 3. *Platform Abstraction* Platform-specific types with compile-time selection: -```idris +[source,idris] +---- CInt : Platform -> Type CInt Linux = Bits32 CInt Windows = Bits32 @@ -129,13 +132,14 @@ CInt Windows = Bits32 CSize : Platform -> Type CSize Linux = Bits64 CSize Windows = Bits64 -``` +---- -### 4. **Safe Evolution** +==== 4. *Safe Evolution* Prove that new ABI versions are backward-compatible: -```idris +[source,idris] +---- -- Compiler enforces compatibility abiUpgrade : ABI 1 -> ABI 2 abiUpgrade old = MkABI2 { @@ -144,71 +148,78 @@ abiUpgrade old = MkABI2 { -- Can add new fields new_features = defaults } -``` +---- -## Why Zig for FFI? +=== Why Zig for FFI? -### 1. **C ABI Compatibility** +==== 1. *C ABI Compatibility* Zig exports C-compatible functions naturally: -```zig +[source,zig] +---- export fn library_function(param: i32) i32 { return param * 2; } -``` +---- -### 2. **Memory Safety** +==== 2. *Memory Safety* Compile-time safety without runtime overhead: -```zig +[source,zig] +---- // Null check enforced at compile time const handle = init() orelse return error.InitFailed; defer free(handle); -``` +---- -### 3. **Cross-Compilation** +==== 3. *Cross-Compilation* Built-in cross-compilation to any platform: -```bash +[source,bash] +---- zig build -Dtarget=x86_64-linux zig build -Dtarget=aarch64-macos zig build -Dtarget=x86_64-windows -``` +---- -### 4. **Zero Dependencies** +==== 4. *Zero Dependencies* No runtime, no libc required (unless explicitly needed): -```zig +[source,zig] +---- // Minimal binary size pub const lib = @import("std"); // Only includes what you use -``` +---- -## Building +=== Building -### Build FFI Library +==== Build FFI Library -```bash +[source,bash] +---- cd ffi/zig zig build # Build debug zig build -Doptimize=ReleaseFast # Build optimized zig build test # Run tests -``` +---- -### Generate C Header from Idris2 ABI +==== Generate C Header from Idris2 ABI -```bash +[source,bash] +---- cd src/abi idris2 --cg c-header Types.idr -o ../../generated/abi/{{project}}.h -``` +---- -### Cross-Compile +==== Cross-Compile -```bash +[source,bash] +---- cd ffi/zig # Linux x86_64 @@ -219,13 +230,14 @@ zig build -Dtarget=aarch64-macos # Windows x86_64 zig build -Dtarget=x86_64-windows -``` +---- -## Usage +=== Usage -### From C +==== From C -```c +[source,c] +---- #include "{{project}}.h" int main() { @@ -241,16 +253,19 @@ int main() { {{project}}_free(handle); return 0; } -``` +---- Compile with: -```bash + +[source,bash] +---- gcc -o example example.c -l{{project}} -L./zig-out/lib -``` +---- -### From Idris2 +==== From Idris2 -```idris +[source,idris] +---- import {{PROJECT}}.ABI.Foreign main : IO () @@ -263,11 +278,12 @@ main = do free handle putStrLn "Success" -``` +---- -### From Rust +==== From Rust -```rust +[source,rust] +---- #[link(name = "{{project}}")] extern "C" { fn {{project}}_init() -> *mut std::ffi::c_void; @@ -286,11 +302,12 @@ fn main() { {{project}}_free(handle); } } -``` +---- -### From Julia +==== From Julia -```julia +[source,julia] +---- const lib{{project}} = "lib{{project}}" function init() @@ -316,27 +333,30 @@ try finally cleanup(handle) end -``` +---- -## Testing +=== Testing -### Unit Tests (Zig) +==== Unit Tests (Zig) -```bash +[source,bash] +---- cd ffi/zig zig build test -``` +---- -### Integration Tests +==== Integration Tests -```bash +[source,bash] +---- cd ffi/zig zig build test-integration -``` +---- -### ABI Verification (Idris2) +==== ABI Verification (Idris2) -```idris +[source,idris] +---- -- Compile-time verification %runElab verifyABI @@ -346,44 +366,44 @@ main = do verifyLayoutsCorrect verifyAlignmentsCorrect putStrLn "ABI verification passed" -``` +---- -## Contributing +=== Contributing When modifying the ABI/FFI: -1. **Update ABI first** (`src/abi/*.idr`) - - Modify type definitions - - Update proofs - - Ensure backward compatibility - -2. **Generate C header** - ```bash - idris2 --cg c-header src/abi/Types.idr -o generated/abi/{{project}}.h - ``` - -3. **Update FFI implementation** (`ffi/zig/src/main.zig`) - - Implement new functions - - Match ABI types exactly - -4. **Add tests** - - Unit tests in Zig - - Integration tests - - ABI verification tests - -5. **Update documentation** - - Function signatures - - Usage examples - - Migration guide (if breaking changes) - -## License +[arabic] +. *Update ABI first* (`+src/abi/*.idr+`) +* Modify type definitions +* Update proofs +* Ensure backward compatibility +. *Generate C header* ++ +[source,bash] +---- +idris2 --cg c-header src/abi/Types.idr -o generated/abi/{{project}}.h +---- +. *Update FFI implementation* (`+ffi/zig/src/main.zig+`) +* Implement new functions +* Match ABI types exactly +. *Add tests* +* Unit tests in Zig +* Integration tests +* ABI verification tests +. *Update documentation* +* Function signatures +* Usage examples +* Migration guide (if breaking changes) + +=== License MPL-2.0 -## See Also +=== See Also -- [Idris2 Documentation](https://idris2.readthedocs.io) -- [Zig Documentation](https://ziglang.org/documentation/master/) -- [Rhodium Standard Repositories](https://github.com/hyperpolymath/rhodium-standard-repositories) -- [FFI Migration Guide](../ffi-migration-guide.md) -- [ABI Migration Guide](../abi-migration-guide.md) +* https://idris2.readthedocs.io[Idris2 Documentation] +* https://ziglang.org/documentation/master/[Zig Documentation] +* https://github.com/hyperpolymath/rhodium-standard-repositories[Rhodium +Standard Repositories] +* link:../ffi-migration-guide.md[FFI Migration Guide] +* link:../abi-migration-guide.md[ABI Migration Guide] diff --git a/ARCHITECTURE.adoc b/ARCHITECTURE.adoc new file mode 100644 index 0000000..1c0a7a6 --- /dev/null +++ b/ARCHITECTURE.adoc @@ -0,0 +1,48 @@ +== Architecture + +=== Overview + +This repository follows a modular, maintainable architecture designed +for clarity, scalability, and long-term sustainability. + +=== Directory Structure + +.... +. +├── src/ # Source code +├── tests/ # Test suites +├── docs/ # Documentation +├── scripts/ # Utility scripts +├── config/ # Configuration files +├── LICENSE # License file +├── LICENSES/ # Full license texts +└── README.adoc # Project documentation +.... + +=== Design Principles + +* *Separation of Concerns*: Each module has a single responsibility +* *Testability*: Code is written to be easily testable +* *Documentation*: All public APIs are documented +* *Configuration*: Environment-specific settings are externalized + +=== Dependencies + +* External dependencies are minimized and clearly declared +* Version pinning is used for reproducibility + +=== Security Considerations + +* Sensitive data is never committed to the repository +* Secrets are managed through environment variables or secure vaults +* Regular dependency audits are performed + +=== Maintainability + +* Code follows consistent style guidelines +* Pull requests require review and CI checks +* Issues and discussions are tracked transparently + +''''' + +_Last updated: 2026-07-18_ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md deleted file mode 100644 index 607e3d8..0000000 --- a/ARCHITECTURE.md +++ /dev/null @@ -1,47 +0,0 @@ -# Architecture - -## Overview - -This repository follows a modular, maintainable architecture designed for clarity, scalability, and long-term sustainability. - -## Directory Structure - -``` -. -├── src/ # Source code -├── tests/ # Test suites -├── docs/ # Documentation -├── scripts/ # Utility scripts -├── config/ # Configuration files -├── LICENSE # License file -├── LICENSES/ # Full license texts -└── README.adoc # Project documentation -``` - -## Design Principles - -- **Separation of Concerns**: Each module has a single responsibility -- **Testability**: Code is written to be easily testable -- **Documentation**: All public APIs are documented -- **Configuration**: Environment-specific settings are externalized - -## Dependencies - -- External dependencies are minimized and clearly declared -- Version pinning is used for reproducibility - -## Security Considerations - -- Sensitive data is never committed to the repository -- Secrets are managed through environment variables or secure vaults -- Regular dependency audits are performed - -## Maintainability - -- Code follows consistent style guidelines -- Pull requests require review and CI checks -- Issues and discussions are tracked transparently - ---- - -*Last updated: 2026-07-18* diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc new file mode 100644 index 0000000..4829c4f --- /dev/null +++ b/CHANGELOG.adoc @@ -0,0 +1,225 @@ +== Changelog + +All notable changes to betlang will be documented in this file. + +The format is based on https://keepachangelog.com/en/1.0.0/[Keep a +Changelog], and this project adheres to +https://semver.org/spec/v2.0.0.html[Semantic Versioning]. + +=== https://github.com/yourusername/betlang/compare/v0.1.0...HEAD[Unreleased] + +==== Changed + +* `+proofs/BetLang.lean+`: discharged `+substTop_preserves_typing+` +axiom with a fully mechanised proof (`+Ctx.insertAt+` + lookup lemmas, +weakening, shift cancellation/commutation, combined +substitute-and-shift-down lemma). No `+axiom+` declarations remain in +the file; the only axiom dependencies are Lean 4 core (`+propext+`, +`+Classical.choice+`, `+Quot.sound+`). Closes #23. + +==== Added + +* RSR (Rhodium Standard Repository) compliance improvements +* `+.gitignore+` for Racket projects +* `+CONTRIBUTING.md+` with comprehensive contribution guidelines +* `+SECURITY.md+` with security policy and vulnerability reporting +* `+CODE_OF_CONDUCT.md+` (Contributor Covenant 2.1) +* `+MAINTAINERS.md+` with project maintainer information +* `+.github/workflows/test.yml+` for automated CI/CD testing +* `+info.rkt+` for Racket package metadata +* `+.well-known/+` directory with RFC 9116 compliant files +* `+TPCF.md+` for Tri-Perimeter Contribution Framework + +=== https://github.com/yourusername/betlang/releases/tag/v0.1.0[0.1.0] - 2025-11-22 + +==== Added + +===== Core Language (`+core/betlang.rkt+`) + +* `+bet+` - Basic ternary bet primitive +* `+bet/weighted+` - Weighted probability distributions +* `+bet/conditional+` - Conditional betting +* `+bet/lazy+` - Lazy evaluation for bets +* `+bet-chain+` - Chain bets together +* `+bet-compose+` - Compose functions into bets +* `+bet-map+`, `+bet-fold+`, `+bet-filter+` - List operations with bets +* `+bet-parallel+` - Run multiple parallel trials +* `+bet-repeat+`, `+bet-until+` - Iteration primitives +* `+bet-with-seed+` - Deterministic randomness +* `+bet-probability+`, `+bet-entropy+`, `+bet-expect+` - Statistical +utilities +* `+all-bets+`, `+make-bet-generator+` - Utility functions + +===== Libraries + +====== Statistics (`+lib/statistics.rkt+`) + +* Descriptive statistics: `+mean+`, `+median+`, `+mode+`, `+variance+`, +`+stddev+` +* Correlation: `+covariance+`, `+correlation+` +* Distributions: `+percentile+`, `+z-score+`, `+normalize+`, +`+histogram+` +* Statistical tests: `+chi-square-test+`, `+kolmogorov-smirnov+` +* Resampling: `+bootstrap+`, `+jackknife+` +* Time series: `+moving-average+`, `+exponential-smoothing+` +* Monte Carlo: `+run-simulation+`, `+monte-carlo-pi+` + +====== Distributions (`+lib/distributions.rkt+`) + +* Discrete: `+uniform+`, `+bernoulli+`, `+binomial+`, `+geometric+`, +`+poisson+` +* Continuous: `+normal+`, `+exponential+`, `+gamma+`, `+beta+`, +`+chi-square+` +* Advanced: `+student-t+`, `+f-distribution+`, `+weibull+`, `+pareto+`, +`+cauchy+`, `+laplace+` +* Multivariate: `+multinomial+`, `+dirichlet+`, `+categorical+`, +`+zipf+` +* Stochastic processes: `+random-walk+`, `+brownian-motion+`, +`+levy-flight+` + +====== Bayesian Inference (`+lib/bayesian.rkt+`) + +* Bayes’ theorem utilities +* Conjugate priors: `+conjugate-beta-binomial+`, `+conjugate-normal+` +* MCMC: `+metropolis-hastings+`, `+gibbs-sampler+` +* Sampling: `+importance-sampling+`, `+rejection-sampling+` +* ABC: `+abc-algorithm+` +* Inference: `+bayesian-update+`, `+credible-interval+`, +`+bayes-factor+` +* Posterior predictive: `+posterior-predictive+`, `+empirical-bayes+` + +====== Optimization (`+lib/optimization.rkt+`) + +* `+simulated-annealing+` - Simulated annealing algorithm +* `+genetic-algorithm+` - Genetic algorithm with ternary operations +* `+particle-swarm+` - Particle swarm optimization +* `+hill-climbing+` - Hill climbing with ternary moves +* `+random-search+` - Random search baseline +* `+evolutionary-strategy+` - Evolution strategies +* `+cross-entropy-method+` - Cross-entropy optimization +* `+ternary-search+` - Ternary search for unimodal functions + +====== Advanced Sampling (`+lib/sampling.rkt+`) + +* `+stratified-sampling+` - Stratified sampling +* `+latin-hypercube-sampling+` - LHS for high dimensions +* `+sobol-sequence+`, `+halton-sequence+` - Quasi-random sequences +* `+importance-resampling+` - Importance sampling with resampling +* `+sequential-monte-carlo+` - Particle filters +* `+slice-sampling+` - Slice sampling for MCMC +* `+hamiltonian-monte-carlo+` - HMC (simplified) +* `+ternary-sampling+` - betlang-specific sampling +* Variance reduction: `+antithetic-variates+`, `+control-variates+` + +====== Markov Chains (`+lib/markov.rkt+`) + +* `+make-markov-chain+`, `+markov-step+`, `+markov-simulate+` +* `+markov-stationary+` - Stationary distribution estimation +* `+estimate-transitions+` - Learn transition matrix from data +* `+markov-classify+` - Sequence classification +* `+hmm-viterbi+` - Hidden Markov Model Viterbi algorithm +* `+generate-text-markov+` - Text generation +* `+ternary-markov+` - Ternary-specific Markov chains + +====== Combinators (`+lib/combinators.rkt+`) + +* Monadic: `+bet-pure+`, `+bet-bind+`, `+bet-join+`, `+bet-lift+`, +`+bet-ap+` +* Logical: `+bet-or+`, `+bet-and+`, `+bet-xor+`, `+bet-not+` +* Error handling: `+bet-try+`, `+bet-guard+`, `+bet-retry+`, +`+bet-fallback+` +* Conditional: `+bet-when+`, `+bet-unless+`, `+bet-cond+`, `+bet-case+` +* Performance: `+bet-memoize+`, `+bet-cache+`, `+bet-throttle+`, +`+bet-debounce+` +* Composition: `+bet-pipeline+`, `+bet-fork-join+`, `+bet-race+` + +====== Ternary Logic (`+lib/ternary.rkt+`) + +* `+ternary-and+`, `+ternary-or+`, `+ternary-not+`, `+ternary-xor+` +* `+ternary-implies+`, `+ternary-equiv+` +* `+ternary-majority+`, `+ternary-consensus+` +* `+ternary-min+`, `+ternary-max+`, `+ternary-median+` +* `+ternary-fold+`, `+ternary-map+`, `+ternary-filter+`, +`+ternary-reduce+` +* `+make-ternary-table+` - Truth table generation +* Constants: `+TRUE+`, `+FALSE+`, `+UNKNOWN+` + +===== Examples + +* `+examples/basic-tutorial.rkt+` - Step-by-step tutorial with 10 +sections +* `+examples/monte-carlo.rkt+` - 7 Monte Carlo simulations +* `+examples/game-theory.rkt+` - 6 game theory applications +* `+examples/finance.rkt+` - 10 financial modeling examples +* `+examples/probabilistic-structures.rkt+` - 7 probabilistic data +structures + +===== Tools + +* `+tools/analyzer.rkt+` - Bet analysis and visualization +** `+analyze-bet+`, `+compare-bets+`, `+convergence-analysis+` +** `+text-histogram+`, `+probability-report+`, `+entropy-analysis+` +* `+benchmarks/performance.rkt+` - 15 performance benchmarks + +===== REPL (`+repl/shell.rkt+`) + +* Interactive shell with help system +* Commands: `+:help+`, `+:stats+`, `+:reset-stats+`, `+:history+`, +`+:examples+`, `+:quit+` +* Session statistics tracking +* Command history +* Error handling and pretty-printing +* Persistent logging + +===== Documentation + +* `+README.md+` - Comprehensive project overview (285 lines) +* `+docs/tutorial.md+` - Complete tutorial (500+ lines) +* `+docs/semantics.md+` - Formal language semantics (340+ lines) +* `+docs/api-reference.md+` - API documentation (600+ lines) +* `+docs/architecture.md+` - Project structure +* `+CLAUDE.md+` - AI assistant context + +===== Tests + +* `+tests/basics.rkt+` - 25 comprehensive test cases +** Basic functionality tests +** Probability distribution tests +** Statistical function tests +** Edge case coverage + +=== https://github.com/yourusername/betlang/releases/tag/v0.0.1[0.0.1] - 2025-07-28 + +==== Added + +* Initial prototype +* Basic `+bet+` primitive +* Simple REPL +* Minimal documentation + +''''' + +=== Version History + +* *0.1.0* (2025-11-22): Major expansion - Full-featured probabilistic +programming language +* *0.0.1* (2025-07-28): Initial prototype + +=== Migration Guide + +==== From 0.0.1 to 0.1.0 + +*Breaking Changes:* None (0.0.1 was minimal prototype) + +*New Features:* - All features listed above are new in 0.1.0 - Core +`+bet+` primitive remains unchanged - Fully backward compatible + +*Recommended Updates:* 1. Explore new libraries (`+lib/*.rkt+`) 2. +Review examples for usage patterns 3. Read tutorial for comprehensive +guide 4. Check API reference for function signatures + +''''' + +*Note:* betlang follows Semantic Versioning: - *MAJOR*: Incompatible API +changes - *MINOR*: Backward-compatible functionality - *PATCH*: +Backward-compatible bug fixes diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index af2c613..0000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,202 +0,0 @@ - -# Changelog - -All notable changes to betlang will be documented in this file. - -The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), -and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - -## [Unreleased] - -### Changed -- `proofs/BetLang.lean`: discharged `substTop_preserves_typing` axiom with a fully mechanised proof (`Ctx.insertAt` + lookup lemmas, weakening, shift cancellation/commutation, combined substitute-and-shift-down lemma). No `axiom` declarations remain in the file; the only axiom dependencies are Lean 4 core (`propext`, `Classical.choice`, `Quot.sound`). Closes #23. - -### Added -- RSR (Rhodium Standard Repository) compliance improvements -- `.gitignore` for Racket projects -- `CONTRIBUTING.md` with comprehensive contribution guidelines -- `SECURITY.md` with security policy and vulnerability reporting -- `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1) -- `MAINTAINERS.md` with project maintainer information -- `.github/workflows/test.yml` for automated CI/CD testing -- `info.rkt` for Racket package metadata -- `.well-known/` directory with RFC 9116 compliant files -- `TPCF.md` for Tri-Perimeter Contribution Framework - -## [0.1.0] - 2025-11-22 - -### Added - -#### Core Language (`core/betlang.rkt`) -- `bet` - Basic ternary bet primitive -- `bet/weighted` - Weighted probability distributions -- `bet/conditional` - Conditional betting -- `bet/lazy` - Lazy evaluation for bets -- `bet-chain` - Chain bets together -- `bet-compose` - Compose functions into bets -- `bet-map`, `bet-fold`, `bet-filter` - List operations with bets -- `bet-parallel` - Run multiple parallel trials -- `bet-repeat`, `bet-until` - Iteration primitives -- `bet-with-seed` - Deterministic randomness -- `bet-probability`, `bet-entropy`, `bet-expect` - Statistical utilities -- `all-bets`, `make-bet-generator` - Utility functions - -#### Libraries - -##### Statistics (`lib/statistics.rkt`) -- Descriptive statistics: `mean`, `median`, `mode`, `variance`, `stddev` -- Correlation: `covariance`, `correlation` -- Distributions: `percentile`, `z-score`, `normalize`, `histogram` -- Statistical tests: `chi-square-test`, `kolmogorov-smirnov` -- Resampling: `bootstrap`, `jackknife` -- Time series: `moving-average`, `exponential-smoothing` -- Monte Carlo: `run-simulation`, `monte-carlo-pi` - -##### Distributions (`lib/distributions.rkt`) -- Discrete: `uniform`, `bernoulli`, `binomial`, `geometric`, `poisson` -- Continuous: `normal`, `exponential`, `gamma`, `beta`, `chi-square` -- Advanced: `student-t`, `f-distribution`, `weibull`, `pareto`, `cauchy`, `laplace` -- Multivariate: `multinomial`, `dirichlet`, `categorical`, `zipf` -- Stochastic processes: `random-walk`, `brownian-motion`, `levy-flight` - -##### Bayesian Inference (`lib/bayesian.rkt`) -- Bayes' theorem utilities -- Conjugate priors: `conjugate-beta-binomial`, `conjugate-normal` -- MCMC: `metropolis-hastings`, `gibbs-sampler` -- Sampling: `importance-sampling`, `rejection-sampling` -- ABC: `abc-algorithm` -- Inference: `bayesian-update`, `credible-interval`, `bayes-factor` -- Posterior predictive: `posterior-predictive`, `empirical-bayes` - -##### Optimization (`lib/optimization.rkt`) -- `simulated-annealing` - Simulated annealing algorithm -- `genetic-algorithm` - Genetic algorithm with ternary operations -- `particle-swarm` - Particle swarm optimization -- `hill-climbing` - Hill climbing with ternary moves -- `random-search` - Random search baseline -- `evolutionary-strategy` - Evolution strategies -- `cross-entropy-method` - Cross-entropy optimization -- `ternary-search` - Ternary search for unimodal functions - -##### Advanced Sampling (`lib/sampling.rkt`) -- `stratified-sampling` - Stratified sampling -- `latin-hypercube-sampling` - LHS for high dimensions -- `sobol-sequence`, `halton-sequence` - Quasi-random sequences -- `importance-resampling` - Importance sampling with resampling -- `sequential-monte-carlo` - Particle filters -- `slice-sampling` - Slice sampling for MCMC -- `hamiltonian-monte-carlo` - HMC (simplified) -- `ternary-sampling` - betlang-specific sampling -- Variance reduction: `antithetic-variates`, `control-variates` - -##### Markov Chains (`lib/markov.rkt`) -- `make-markov-chain`, `markov-step`, `markov-simulate` -- `markov-stationary` - Stationary distribution estimation -- `estimate-transitions` - Learn transition matrix from data -- `markov-classify` - Sequence classification -- `hmm-viterbi` - Hidden Markov Model Viterbi algorithm -- `generate-text-markov` - Text generation -- `ternary-markov` - Ternary-specific Markov chains - -##### Combinators (`lib/combinators.rkt`) -- Monadic: `bet-pure`, `bet-bind`, `bet-join`, `bet-lift`, `bet-ap` -- Logical: `bet-or`, `bet-and`, `bet-xor`, `bet-not` -- Error handling: `bet-try`, `bet-guard`, `bet-retry`, `bet-fallback` -- Conditional: `bet-when`, `bet-unless`, `bet-cond`, `bet-case` -- Performance: `bet-memoize`, `bet-cache`, `bet-throttle`, `bet-debounce` -- Composition: `bet-pipeline`, `bet-fork-join`, `bet-race` - -##### Ternary Logic (`lib/ternary.rkt`) -- `ternary-and`, `ternary-or`, `ternary-not`, `ternary-xor` -- `ternary-implies`, `ternary-equiv` -- `ternary-majority`, `ternary-consensus` -- `ternary-min`, `ternary-max`, `ternary-median` -- `ternary-fold`, `ternary-map`, `ternary-filter`, `ternary-reduce` -- `make-ternary-table` - Truth table generation -- Constants: `TRUE`, `FALSE`, `UNKNOWN` - -#### Examples - -- `examples/basic-tutorial.rkt` - Step-by-step tutorial with 10 sections -- `examples/monte-carlo.rkt` - 7 Monte Carlo simulations -- `examples/game-theory.rkt` - 6 game theory applications -- `examples/finance.rkt` - 10 financial modeling examples -- `examples/probabilistic-structures.rkt` - 7 probabilistic data structures - -#### Tools - -- `tools/analyzer.rkt` - Bet analysis and visualization - - `analyze-bet`, `compare-bets`, `convergence-analysis` - - `text-histogram`, `probability-report`, `entropy-analysis` -- `benchmarks/performance.rkt` - 15 performance benchmarks - -#### REPL (`repl/shell.rkt`) -- Interactive shell with help system -- Commands: `:help`, `:stats`, `:reset-stats`, `:history`, `:examples`, `:quit` -- Session statistics tracking -- Command history -- Error handling and pretty-printing -- Persistent logging - -#### Documentation - -- `README.md` - Comprehensive project overview (285 lines) -- `docs/tutorial.md` - Complete tutorial (500+ lines) -- `docs/semantics.md` - Formal language semantics (340+ lines) -- `docs/api-reference.md` - API documentation (600+ lines) -- `docs/architecture.md` - Project structure -- `CLAUDE.md` - AI assistant context - -#### Tests - -- `tests/basics.rkt` - 25 comprehensive test cases - - Basic functionality tests - - Probability distribution tests - - Statistical function tests - - Edge case coverage - -## [0.0.1] - 2025-07-28 - -### Added -- Initial prototype -- Basic `bet` primitive -- Simple REPL -- Minimal documentation - ---- - -## Version History - -- **0.1.0** (2025-11-22): Major expansion - Full-featured probabilistic programming language -- **0.0.1** (2025-07-28): Initial prototype - -## Migration Guide - -### From 0.0.1 to 0.1.0 - -**Breaking Changes:** None (0.0.1 was minimal prototype) - -**New Features:** -- All features listed above are new in 0.1.0 -- Core `bet` primitive remains unchanged -- Fully backward compatible - -**Recommended Updates:** -1. Explore new libraries (`lib/*.rkt`) -2. Review examples for usage patterns -3. Read tutorial for comprehensive guide -4. Check API reference for function signatures - ---- - -**Note:** betlang follows Semantic Versioning: -- **MAJOR**: Incompatible API changes -- **MINOR**: Backward-compatible functionality -- **PATCH**: Backward-compatible bug fixes - -[unreleased]: https://github.com/yourusername/betlang/compare/v0.1.0...HEAD -[0.1.0]: https://github.com/yourusername/betlang/releases/tag/v0.1.0 -[0.0.1]: https://github.com/yourusername/betlang/releases/tag/v0.0.1 diff --git a/CODE_OF_CONDUCT.adoc b/CODE_OF_CONDUCT.adoc new file mode 100644 index 0000000..6e98df8 --- /dev/null +++ b/CODE_OF_CONDUCT.adoc @@ -0,0 +1,134 @@ +== Contributor Covenant Code of Conduct + +=== Our Pledge + +We as members, contributors, and leaders pledge to make participation in +our community a harassment-free experience for everyone, regardless of +age, body size, visible or invisible disability, ethnicity, sex +characteristics, gender identity and expression, level of experience, +education, socio-economic status, nationality, personal appearance, +race, caste, color, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, +welcoming, diverse, inclusive, and healthy community. + +=== Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our +mistakes, and learning from the experience +* Focusing on what is best not just for us as individuals, but for the +overall community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or +advances of any kind +* Trolling, insulting or derogatory comments, and personal or political +attacks +* Public or private harassment +* Publishing others’ private information, such as a physical or email +address, without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a +professional setting + +=== Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our +standards of acceptable behavior and will take appropriate and fair +corrective action in response to any behavior that they deem +inappropriate, threatening, offensive, or harmful. + +Community leaders have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other +contributions that are not aligned to this Code of Conduct, and will +communicate reasons for moderation decisions when appropriate. + +=== Scope + +This Code of Conduct applies within all community spaces, and also +applies when an individual is officially representing the community in +public spaces. Examples of representing our community include using an +official e-mail address, posting via an official social media account, +or acting as an appointed representative at an online or offline event. + +=== Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may +be reported to the community leaders responsible for enforcement at +[CONDUCT_EMAIL - TO BE CONFIGURED]. + +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security +of the reporter of any incident. + +=== Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in +determining the consequences for any action they deem in violation of +this Code of Conduct: + +==== 1. Correction + +*Community Impact*: Use of inappropriate language or other behavior +deemed unprofessional or unwelcome in the community. + +*Consequence*: A private, written warning from community leaders, +providing clarity around the nature of the violation and an explanation +of why the behavior was inappropriate. A public apology may be +requested. + +==== 2. Warning + +*Community Impact*: A violation through a single incident or series of +actions. + +*Consequence*: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, for a specified period of +time. This includes avoiding interactions in community spaces as well as +external channels like social media. Violating these terms may lead to a +temporary or permanent ban. + +==== 3. Temporary Ban + +*Community Impact*: A serious violation of community standards, +including sustained inappropriate behavior. + +*Consequence*: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No +public or private interaction with the people involved, including +unsolicited interaction with those enforcing the Code of Conduct, is +allowed during this period. Violating these terms may lead to a +permanent ban. + +==== 4. Permanent Ban + +*Community Impact*: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of +individuals. + +*Consequence*: A permanent ban from any sort of public interaction +within the community. + +=== Attribution + +This Code of Conduct is adapted from the +https://www.contributor-covenant.org[Contributor Covenant], version 2.1, +available at +https://www.contributor-covenant.org/version/2/1/code_of_conduct.html. + +Community Impact Guidelines were inspired by +https://github.com/mozilla/diversity[Mozilla’s code of conduct +enforcement ladder]. + +For answers to common questions about this code of conduct, see the FAQ +at https://www.contributor-covenant.org/faq. Translations are available +at https://www.contributor-covenant.org/translations. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md deleted file mode 100644 index d1db27f..0000000 --- a/CODE_OF_CONDUCT.md +++ /dev/null @@ -1,137 +0,0 @@ - -# Contributor Covenant Code of Conduct - -## Our Pledge - -We as members, contributors, and leaders pledge to make participation in our -community a harassment-free experience for everyone, regardless of age, body -size, visible or invisible disability, ethnicity, sex characteristics, gender -identity and expression, level of experience, education, socio-economic status, -nationality, personal appearance, race, caste, color, religion, or sexual -identity and orientation. - -We pledge to act and interact in ways that contribute to an open, welcoming, -diverse, inclusive, and healthy community. - -## Our Standards - -Examples of behavior that contributes to a positive environment for our -community include: - -* Demonstrating empathy and kindness toward other people -* Being respectful of differing opinions, viewpoints, and experiences -* Giving and gracefully accepting constructive feedback -* Accepting responsibility and apologizing to those affected by our mistakes, - and learning from the experience -* Focusing on what is best not just for us as individuals, but for the overall - community - -Examples of unacceptable behavior include: - -* The use of sexualized language or imagery, and sexual attention or advances of - any kind -* Trolling, insulting or derogatory comments, and personal or political attacks -* Public or private harassment -* Publishing others' private information, such as a physical or email address, - without their explicit permission -* Other conduct which could reasonably be considered inappropriate in a - professional setting - -## Enforcement Responsibilities - -Community leaders are responsible for clarifying and enforcing our standards of -acceptable behavior and will take appropriate and fair corrective action in -response to any behavior that they deem inappropriate, threatening, offensive, -or harmful. - -Community leaders have the right and responsibility to remove, edit, or reject -comments, commits, code, wiki edits, issues, and other contributions that are -not aligned to this Code of Conduct, and will communicate reasons for moderation -decisions when appropriate. - -## Scope - -This Code of Conduct applies within all community spaces, and also applies when -an individual is officially representing the community in public spaces. -Examples of representing our community include using an official e-mail address, -posting via an official social media account, or acting as an appointed -representative at an online or offline event. - -## Enforcement - -Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported to the community leaders responsible for enforcement at -[CONDUCT_EMAIL - TO BE CONFIGURED]. - -All complaints will be reviewed and investigated promptly and fairly. - -All community leaders are obligated to respect the privacy and security of the -reporter of any incident. - -## Enforcement Guidelines - -Community leaders will follow these Community Impact Guidelines in determining -the consequences for any action they deem in violation of this Code of Conduct: - -### 1. Correction - -**Community Impact**: Use of inappropriate language or other behavior deemed -unprofessional or unwelcome in the community. - -**Consequence**: A private, written warning from community leaders, providing -clarity around the nature of the violation and an explanation of why the -behavior was inappropriate. A public apology may be requested. - -### 2. Warning - -**Community Impact**: A violation through a single incident or series of -actions. - -**Consequence**: A warning with consequences for continued behavior. No -interaction with the people involved, including unsolicited interaction with -those enforcing the Code of Conduct, for a specified period of time. This -includes avoiding interactions in community spaces as well as external channels -like social media. Violating these terms may lead to a temporary or permanent -ban. - -### 3. Temporary Ban - -**Community Impact**: A serious violation of community standards, including -sustained inappropriate behavior. - -**Consequence**: A temporary ban from any sort of interaction or public -communication with the community for a specified period of time. No public or -private interaction with the people involved, including unsolicited interaction -with those enforcing the Code of Conduct, is allowed during this period. -Violating these terms may lead to a permanent ban. - -### 4. Permanent Ban - -**Community Impact**: Demonstrating a pattern of violation of community -standards, including sustained inappropriate behavior, harassment of an -individual, or aggression toward or disparagement of classes of individuals. - -**Consequence**: A permanent ban from any sort of public interaction within the -community. - -## Attribution - -This Code of Conduct is adapted from the [Contributor Covenant][homepage], -version 2.1, available at -[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]. - -Community Impact Guidelines were inspired by -[Mozilla's code of conduct enforcement ladder][Mozilla CoC]. - -For answers to common questions about this code of conduct, see the FAQ at -[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at -[https://www.contributor-covenant.org/translations][translations]. - -[homepage]: https://www.contributor-covenant.org -[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html -[Mozilla CoC]: https://github.com/mozilla/diversity -[FAQ]: https://www.contributor-covenant.org/faq -[translations]: https://www.contributor-covenant.org/translations diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc new file mode 100644 index 0000000..93741d8 --- /dev/null +++ b/CONTRIBUTING.adoc @@ -0,0 +1,109 @@ +== Clone the repository + +git clone https://github.com/hyperpolymath/nextgen-languages.git cd +nextgen-languages + +== Using Guix (recommended for reproducibility) + +guix develop + +== Or using toolbox/distrobox + +toolbox create nextgen-languages-dev toolbox enter nextgen-languages-dev +# Install dependencies manually + +== Verify setup + +just check # or: cargo check / mix compile / etc. just test # Run test +suite + +.... + +### Repository Structure +.... + +nextgen-languages/ ├── src/ # Source code (Perimeter 1-2) ├── lib/ # +Library code (Perimeter 1-2) ├── extensions/ # Extensions (Perimeter 2) +├── plugins/ # Plugins (Perimeter 2) ├── tools/ # Tooling (Perimeter 2) +├── docs/ # Documentation (Perimeter 3) │ ├── architecture/ # ADRs, +specs (Perimeter 2) │ └── proposals/ # RFCs (Perimeter 3) ├── examples/ +# Examples (Perimeter 3) ├── spec/ # Spec tests (Perimeter 3) ├── tests/ +# Test suite (Perimeter 2-3) ├── .well-known/ # Protocol files +(Perimeter 1-3) ├── .github/ # GitHub config (Perimeter 1) │ ├── +ISSUE_TEMPLATE/ │ └── workflows/ ├── CHANGELOG.md ├── CODE_OF_CONDUCT.md +├── CONTRIBUTING.md # This file ├── GOVERNANCE.md ├── LICENSE ├── +MAINTAINERS.md ├── README.adoc ├── SECURITY.md ├── flake.guix # Guix +flake (Perimeter 1) └── Justfile # Task runner (Perimeter 1) + +.... + +--- + +## How to Contribute + +### Reporting Bugs + +**Before reporting**: +1. Search existing issues +2. Check if it's already fixed in `main` +3. Determine which perimeter the bug affects + +**When reporting**: + +Use the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md) and include: + +- Clear, descriptive title +- Environment details (OS, versions, toolchain) +- Steps to reproduce +- Expected vs actual behaviour +- Logs, screenshots, or minimal reproduction + +### Suggesting Features + +**Before suggesting**: +1. Check the [roadmap](ROADMAP.md) if available +2. Search existing issues and discussions +3. Consider which perimeter the feature belongs to + +**When suggesting**: + +Use the [feature request template](.github/ISSUE_TEMPLATE/feature_request.md) and include: + +- Problem statement (what pain point does this solve?) +- Proposed solution +- Alternatives considered +- Which perimeter this affects + +### Your First Contribution + +Look for issues labelled: + +- [`good first issue`](https://github.com/hyperpolymath/nextgen-languages/labels/good%20first%20issue) — Simple Perimeter 3 tasks +- [`help wanted`](https://github.com/hyperpolymath/nextgen-languages/labels/help%20wanted) — Community help needed +- [`documentation`](https://github.com/hyperpolymath/nextgen-languages/labels/documentation) — Docs improvements +- [`perimeter-3`](https://github.com/hyperpolymath/nextgen-languages/labels/perimeter-3) — Community sandbox scope + +--- + +## Development Workflow + +### Branch Naming +.... + +docs/short-description # Documentation (P3) test/what-added # Test +additions (P3) feat/short-description # New features (P2) +fix/issue-number-description # Bug fixes (P2) refactor/what-changed # +Code improvements (P2) security/what-fixed # Security fixes (P1-2) + +.... + +### Commit Messages + +We follow [Conventional Commits](https://www.conventionalcommits.org/): +.... + +(): + +{empty}[optional body] + +{empty}[optional footer] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index df122a5..0000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,120 +0,0 @@ - -# Clone the repository -git clone https://github.com/hyperpolymath/nextgen-languages.git -cd nextgen-languages - -# Using Guix (recommended for reproducibility) -guix develop - -# Or using toolbox/distrobox -toolbox create nextgen-languages-dev -toolbox enter nextgen-languages-dev -# Install dependencies manually - -# Verify setup -just check # or: cargo check / mix compile / etc. -just test # Run test suite -``` - -### Repository Structure -``` -nextgen-languages/ -├── src/ # Source code (Perimeter 1-2) -├── lib/ # Library code (Perimeter 1-2) -├── extensions/ # Extensions (Perimeter 2) -├── plugins/ # Plugins (Perimeter 2) -├── tools/ # Tooling (Perimeter 2) -├── docs/ # Documentation (Perimeter 3) -│ ├── architecture/ # ADRs, specs (Perimeter 2) -│ └── proposals/ # RFCs (Perimeter 3) -├── examples/ # Examples (Perimeter 3) -├── spec/ # Spec tests (Perimeter 3) -├── tests/ # Test suite (Perimeter 2-3) -├── .well-known/ # Protocol files (Perimeter 1-3) -├── .github/ # GitHub config (Perimeter 1) -│ ├── ISSUE_TEMPLATE/ -│ └── workflows/ -├── CHANGELOG.md -├── CODE_OF_CONDUCT.md -├── CONTRIBUTING.md # This file -├── GOVERNANCE.md -├── LICENSE -├── MAINTAINERS.md -├── README.adoc -├── SECURITY.md -├── flake.guix # Guix flake (Perimeter 1) -└── Justfile # Task runner (Perimeter 1) -``` - ---- - -## How to Contribute - -### Reporting Bugs - -**Before reporting**: -1. Search existing issues -2. Check if it's already fixed in `main` -3. Determine which perimeter the bug affects - -**When reporting**: - -Use the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md) and include: - -- Clear, descriptive title -- Environment details (OS, versions, toolchain) -- Steps to reproduce -- Expected vs actual behaviour -- Logs, screenshots, or minimal reproduction - -### Suggesting Features - -**Before suggesting**: -1. Check the [roadmap](ROADMAP.md) if available -2. Search existing issues and discussions -3. Consider which perimeter the feature belongs to - -**When suggesting**: - -Use the [feature request template](.github/ISSUE_TEMPLATE/feature_request.md) and include: - -- Problem statement (what pain point does this solve?) -- Proposed solution -- Alternatives considered -- Which perimeter this affects - -### Your First Contribution - -Look for issues labelled: - -- [`good first issue`](https://github.com/hyperpolymath/nextgen-languages/labels/good%20first%20issue) — Simple Perimeter 3 tasks -- [`help wanted`](https://github.com/hyperpolymath/nextgen-languages/labels/help%20wanted) — Community help needed -- [`documentation`](https://github.com/hyperpolymath/nextgen-languages/labels/documentation) — Docs improvements -- [`perimeter-3`](https://github.com/hyperpolymath/nextgen-languages/labels/perimeter-3) — Community sandbox scope - ---- - -## Development Workflow - -### Branch Naming -``` -docs/short-description # Documentation (P3) -test/what-added # Test additions (P3) -feat/short-description # New features (P2) -fix/issue-number-description # Bug fixes (P2) -refactor/what-changed # Code improvements (P2) -security/what-fixed # Security fixes (P1-2) -``` - -### Commit Messages - -We follow [Conventional Commits](https://www.conventionalcommits.org/): -``` -(): - -[optional body] - -[optional footer] diff --git a/GOVERNANCE.adoc b/GOVERNANCE.adoc index e41020d..9b836fb 100644 --- a/GOVERNANCE.adoc +++ b/GOVERNANCE.adoc @@ -1,162 +1,60 @@ -// SPDX-License-Identifier: CC-BY-SA-4.0 -// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell -= Governance Model -:toc: preamble +== Governance -This document describes the governance model for this repository. +=== Overview -== Overview +This project is governed by the following principles and structures to +ensure transparent, inclusive, and effective decision-making. -This repository follows a **Sole Maintainer Governance Model**: +=== Roles and Responsibilities -* Single maintainer (@hyperpolymath) has full authority over the project -* All contributions are welcome and reviewed by the maintainer -* Decisions are made transparently through GitHub issues and discussions -* The project adheres to the hyperpolymath estate policies where applicable +==== Maintainers -== Core Principles +Maintainers are responsible for: - Reviewing and merging pull requests - +Managing releases and versioning - Ensuring code quality and standards - +Triaging issues and bug reports - Community engagement and support -[cols="1,2"] -|=== -| Principle | Description +==== Contributors -| **Benevolent Dictatorship** | Maintainer has final decision authority but seeks community input +Contributors are expected to: - Follow the code of conduct - Submit +well-documented pull requests - Write tests for new functionality - +Maintain existing tests - Update documentation as needed -| **Meritocracy** | Contributions are judged on technical merit, not contributor identity +=== Decision Making -| **Transparency** | All significant decisions are documented publicly +==== Minor Changes -| **Consensus-Seeking** | Maintainer prefers consensus but will decide when necessary +* Can be made by any maintainer +* Include bug fixes, documentation updates, dependency updates -| **Open Contribution** | Anyone can contribute via fork and pull request +==== Major Changes -|=== +* Require discussion in issues or pull requests +* Include new features, architectural changes, API changes +* Need approval from at least 2 maintainers -== Roles and Permissions +==== Breaking Changes -[cols="1,2,2"] -|=== -| Role | Permissions | Assignment +* Require RFC (Request for Comments) process +* Need approval from majority of maintainers +* Must include migration guide -| **Maintainer** | Write access, merge rights, admin | @hyperpolymath -| **Contributors** | Read access, fork, submit PRs | All GitHub users -| **Users** | Use the software, report issues | All GitHub users +=== Code of Conduct -|=== +All participants are expected to follow our Code of Conduct. Violations +can be reported to the maintainers. -== Decision Making Framework +=== Communication -=== Routine Decisions +* *Issues*: For bug reports and feature requests +* *Discussions*: For questions and general discussion +* *Pull Requests*: For code contributions -* Bug fixes -* Documentation improvements -* Minor feature additions -* Dependency updates +=== Licensing -**Process**: Maintainer reviews and merges PRs that meet quality standards. +All contributions are made under the terms of the repository’s LICENSE +file. By submitting a pull request, you agree to license your +contributions accordingly. -=== Significant Changes +''''' -* New major features -* API changes -* Architecture modifications -* Breaking changes - -**Process**: -. Open issue describing the change -. Discuss with community (minimum 72 hours) -. Maintainer makes final decision -. Document rationale in issue/PR - -=== Structural Decisions - -* Repository purpose/renaming -* License changes -* Ownership transfer -* Deprecation/archival - -**Process**: -. Extended discussion (minimum 1 week) -. Maintainer makes final decision -. Document in CHANGELOG and governance docs - -== Contribution Lifecycle - -[cols="1,2"] -|=== -| Stage | Process - -| **Ideation** | Open issue, discuss feasibility - -| **Development** | Fork, implement, test thoroughly - -| **Review** | Submit PR, maintainer reviews within 7 days - -| **Merge** | Maintainer merges or requests changes - -| **Release** | Maintainer publishes according to project conventions - -|=== - -== Conflict Resolution - -In case of disagreements: - -. Discuss in the relevant GitHub issue or PR -. Provide technical justification for positions -. Maintainer mediates and makes final decision -. Decision is documented and can be revisited later - -== Project Policies - -This repository adheres to hyperpolymath estate-wide policies: - -* **License**: MPL-2.0 for code, CC-BY-SA-4.0 for prose (per standards/LICENCE-POLICY.adoc) -* **Code of Conduct**: Follows hyperpolymath CODE_OF_CONDUCT.md -* **Security**: Follows hyperpolymath SECURITY.md -* **Contributing**: Follows hyperpolymath CONTRIBUTING.adoc conventions - -== Repository-Specific Conventions - -[cols="1,2"] -|=== -| Convention | Description - -| **Signing** | All commits must be signed (SSH or GPG) - -| **SPDX Headers** | All source files must have SPDX license identifiers - -| **Contractiles** | Mustfile, Trustfile, Intendfile, Adjustfile in root - -| **Machine Readable** | META.a2ml in .machine_readable/6a2/ - -| **CI/CD** | GitHub Actions workflows in .github/workflows/ - -|=== - -== Governance Evolution - -As the project grows, this governance model may evolve: - -* **Adding Co-Maintainers**: When contribution volume warrants it -* **Forming a Team**: For complex multi-maintainer projects -* **Adopting TPCF**: For large, multi-repository projects (see rhodium-standard-repositories) - -Changes to this document require the same process as Significant Changes above. - -== See Also - -* link:MAINTAINERS.adoc[Maintainers] -* link:CODE_OF_CONDUCT.md[Code of Conduct] -* link:CONTRIBUTING.adoc[Contributing Guide] -* link:https://github.com/hyperpolymath/standards/blob/main/LICENCE-POLICY.adoc[Estate License Policy] -* link:https://github.com/hyperpolymath/standards[rhodium-standard-repositories (TPCF)] - -== Changelog - -[cols="1,1,1"] -|=== -| Date | Change | By - -| 2026-06-07 | Initial governance model established | @hyperpolymath -|=== +_Last updated: 2026-07-18_ diff --git a/GOVERNANCE.md b/GOVERNANCE.md deleted file mode 100644 index e27364c..0000000 --- a/GOVERNANCE.md +++ /dev/null @@ -1,60 +0,0 @@ -# Governance - -## Overview - -This project is governed by the following principles and structures to ensure transparent, inclusive, and effective decision-making. - -## Roles and Responsibilities - -### Maintainers - -Maintainers are responsible for: -- Reviewing and merging pull requests -- Managing releases and versioning -- Ensuring code quality and standards -- Triaging issues and bug reports -- Community engagement and support - -### Contributors - -Contributors are expected to: -- Follow the code of conduct -- Submit well-documented pull requests -- Write tests for new functionality -- Maintain existing tests -- Update documentation as needed - -## Decision Making - -### Minor Changes -- Can be made by any maintainer -- Include bug fixes, documentation updates, dependency updates - -### Major Changes -- Require discussion in issues or pull requests -- Include new features, architectural changes, API changes -- Need approval from at least 2 maintainers - -### Breaking Changes -- Require RFC (Request for Comments) process -- Need approval from majority of maintainers -- Must include migration guide - -## Code of Conduct - -All participants are expected to follow our Code of Conduct. Violations can be reported to the maintainers. - -## Communication - -- **Issues**: For bug reports and feature requests -- **Discussions**: For questions and general discussion -- **Pull Requests**: For code contributions - -## Licensing - -All contributions are made under the terms of the repository's LICENSE file. -By submitting a pull request, you agree to license your contributions accordingly. - ---- - -*Last updated: 2026-07-18* diff --git a/MAINTAINERS.adoc b/MAINTAINERS.adoc index daf8a78..d6e633d 100644 --- a/MAINTAINERS.adoc +++ b/MAINTAINERS.adoc @@ -1,65 +1,150 @@ -// SPDX-License-Identifier: CC-BY-SA-4.0 -// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell -= Maintainers -:toc: preamble +== Maintainers -== Current Maintainers +This document lists the maintainers of the betlang project. -[cols="2,3,2",options="header"] -|=== -| Name | Role | Contact +=== Current Maintainers -| Jonathan D.A. Jewell | Sole Maintainer | https://github.com/hyperpolymath[@hyperpolymath] -|=== +==== Lead Maintainer -== Maintainer Responsibilities +*[TO BE CONFIGURED]* - GitHub: https://github.com/username[@username] - +Email: maintainer@betlang.org (TO BE CONFIGURED) - Role: Project lead, +architecture decisions, releases - Timezone: UTC - Active since: +2025-07-28 -As the sole maintainer, all responsibilities apply to @hyperpolymath: +=== Maintainer Responsibilities -* Reviewing and merging pull requests -* Triaging issues and feature requests -* Ensuring code quality and security standards -* Managing releases and versioning -* Upholding the project's Code of Conduct -* Maintaining documentation and examples -* Responding to security vulnerabilities +Maintainers are responsible for: -== Contribution Process +[arabic] +. *Code Review* +* Review and merge pull requests +* Ensure code quality and adherence to ternary philosophy +* Verify test coverage and documentation +. *Issue Triage* +* Label and prioritize issues +* Respond to bug reports +* Guide feature discussions +. *Release Management* +* Create releases and tags +* Update CHANGELOG.md +* Announce releases +. *Community Management* +* Enforce Code of Conduct +* Welcome new contributors +* Foster inclusive community +. *Documentation* +* Keep documentation up-to-date +* Review documentation PRs +* Write tutorials and guides +. *Security* +* Respond to security reports +* Coordinate security fixes +* Publish security advisories -This is a sole-maintainer project. All contributions are welcome via: +=== Becoming a Maintainer -1. **Issues**: Report bugs, request features, ask questions -2. **Pull Requests**: Submit improvements for review -3. **Discussions**: Engage in community discussions +We welcome new maintainers! To become a maintainer: -All contributions will be reviewed by the maintainer. +[arabic] +. *Consistent Contributions* +* Regular, high-quality code contributions +* Active participation in issues and PRs +* Demonstrated understanding of betlang’s philosophy +. *Community Involvement* +* Help other contributors +* Participate in discussions +* Review pull requests +. *Nomination* +* Current maintainers nominate candidates +* Consensus among existing maintainers +* Public announcement of new maintainer -== Decision Making +==== Nomination Process -* Routine decisions (bug fixes, minor improvements): Made by maintainer -* Significant changes: Discussed in issues before implementation -* Breaking changes: Announced in advance with migration path +[arabic] +. Current maintainer proposes candidate +. Discussion among maintainers (1 week) +. Consensus decision (or majority vote if needed) +. Invite sent to candidate +. If accepted, add to this file and grant repository access -== Becoming a Maintainer +=== Maintainer Levels -This project currently has a single maintainer. If you're interested in becoming a co-maintainer: +==== Core Maintainer -1. Demonstrate consistent, high-quality contributions -2. Show understanding of project goals and standards -3. Participate constructively in discussions -4. Express interest to the current maintainer +* Full repository access +* Can merge PRs +* Can create releases +* Can modify repository settings -Co-maintainers may be added at the discretion of the current maintainer. +==== Module Maintainer -== Contact +* Ownership of specific modules (e.g., `+lib/statistics.rkt+`) +* Review PRs for owned modules +* Expertise in specific domain -For questions about project governance: +==== Documentation Maintainer -* Open a GitHub issue in this repository -* Contact: https://github.com/hyperpolymath +* Focus on documentation quality +* Review documentation PRs +* Write tutorials and guides -== See Also +=== Emeritus Maintainers -* link:GOVERNANCE.adoc[Governance Model] -* link:CODE_OF_CONDUCT.md[Code of Conduct] -* link:CONTRIBUTING.adoc[Contributing Guide] +Maintainers who have stepped down but contributed significantly: + +(None yet - project is new!) + +=== Contact + +*General inquiries:* Open an +https://github.com/hyperpolymath/betlang/issues[issue] + +*Security reports:* See SECURITY.md + +*Code of Conduct violations:* conduct@betlang.org (TO BE CONFIGURED) + +*Maintainer nominations:* Contact current maintainers privately + +=== Decision-Making Process + +For major decisions: + +[arabic] +. *Proposal:* Open an issue with `+[RFC]+` tag +. *Discussion:* Allow 2 weeks for community input +. *Decision:* Maintainers make final call +. *Documentation:* Update relevant docs + +For minor decisions: + +* PR review process +* Consensus among reviewers +* Fast-track for bug fixes + +=== Conflict Resolution + +[arabic] +. Discussion among maintainers +. Attempt consensus +. If consensus fails, majority vote +. Lead maintainer has tie-breaking vote + +=== Inactive Maintainers + +Maintainers inactive for 6+ months will be contacted: + +* Check if they want to continue +* Offer emeritus status if stepping down +* Remove write access if unresponsive + +=== Attribution + +This maintainers structure is inspired by: - +https://www.rust-lang.org/governance[Rust Project Governance] - +https://github.com/nodejs/node/blob/main/GOVERNANCE.md[Node.js Project +Governance] - Open source best practices + +''''' + +*Last Updated:* 2025-11-22 diff --git a/MAINTAINERS.md b/MAINTAINERS.md deleted file mode 100644 index 992a59c..0000000 --- a/MAINTAINERS.md +++ /dev/null @@ -1,154 +0,0 @@ - -# Maintainers - -This document lists the maintainers of the betlang project. - -## Current Maintainers - -### Lead Maintainer - -**[TO BE CONFIGURED]** -- GitHub: [@username](https://github.com/username) -- Email: maintainer@betlang.org (TO BE CONFIGURED) -- Role: Project lead, architecture decisions, releases -- Timezone: UTC -- Active since: 2025-07-28 - -## Maintainer Responsibilities - -Maintainers are responsible for: - -1. **Code Review** - - Review and merge pull requests - - Ensure code quality and adherence to ternary philosophy - - Verify test coverage and documentation - -2. **Issue Triage** - - Label and prioritize issues - - Respond to bug reports - - Guide feature discussions - -3. **Release Management** - - Create releases and tags - - Update CHANGELOG.md - - Announce releases - -4. **Community Management** - - Enforce Code of Conduct - - Welcome new contributors - - Foster inclusive community - -5. **Documentation** - - Keep documentation up-to-date - - Review documentation PRs - - Write tutorials and guides - -6. **Security** - - Respond to security reports - - Coordinate security fixes - - Publish security advisories - -## Becoming a Maintainer - -We welcome new maintainers! To become a maintainer: - -1. **Consistent Contributions** - - Regular, high-quality code contributions - - Active participation in issues and PRs - - Demonstrated understanding of betlang's philosophy - -2. **Community Involvement** - - Help other contributors - - Participate in discussions - - Review pull requests - -3. **Nomination** - - Current maintainers nominate candidates - - Consensus among existing maintainers - - Public announcement of new maintainer - -### Nomination Process - -1. Current maintainer proposes candidate -2. Discussion among maintainers (1 week) -3. Consensus decision (or majority vote if needed) -4. Invite sent to candidate -5. If accepted, add to this file and grant repository access - -## Maintainer Levels - -### Core Maintainer -- Full repository access -- Can merge PRs -- Can create releases -- Can modify repository settings - -### Module Maintainer -- Ownership of specific modules (e.g., `lib/statistics.rkt`) -- Review PRs for owned modules -- Expertise in specific domain - -### Documentation Maintainer -- Focus on documentation quality -- Review documentation PRs -- Write tutorials and guides - -## Emeritus Maintainers - -Maintainers who have stepped down but contributed significantly: - -(None yet - project is new!) - -## Contact - -**General inquiries:** Open an [issue](https://github.com/hyperpolymath/betlang/issues) - -**Security reports:** See [SECURITY.md](SECURITY.md) - -**Code of Conduct violations:** [conduct@betlang.org](mailto:conduct@betlang.org) (TO BE CONFIGURED) - -**Maintainer nominations:** Contact current maintainers privately - -## Decision-Making Process - -For major decisions: - -1. **Proposal:** Open an issue with `[RFC]` tag -2. **Discussion:** Allow 2 weeks for community input -3. **Decision:** Maintainers make final call -4. **Documentation:** Update relevant docs - -For minor decisions: - -- PR review process -- Consensus among reviewers -- Fast-track for bug fixes - -## Conflict Resolution - -1. Discussion among maintainers -2. Attempt consensus -3. If consensus fails, majority vote -4. Lead maintainer has tie-breaking vote - -## Inactive Maintainers - -Maintainers inactive for 6+ months will be contacted: - -- Check if they want to continue -- Offer emeritus status if stepping down -- Remove write access if unresponsive - -## Attribution - -This maintainers structure is inspired by: -- [Rust Project Governance](https://www.rust-lang.org/governance) -- [Node.js Project Governance](https://github.com/nodejs/node/blob/main/GOVERNANCE.md) -- Open source best practices - ---- - -**Last Updated:** 2025-11-22 diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc new file mode 100644 index 0000000..a6ce798 --- /dev/null +++ b/PROOF-NEEDS.adoc @@ -0,0 +1,103 @@ +== PROOF-NEEDS.md — BetLang + +This file defines _what_ must be proven. Completion is tracked in +`+PROOF-STATUS.md+`; the phased plan is in +`+docs/AFFINESCRIPT-ALIGNMENT.adoc+`; the trusted-base ledger is in +`+docs/proof-debt.adoc+`. + +=== Obligation Categories + +[width="100%",cols="15%,23%,37%,25%",options="header",] +|=== +|Code |Category |Default prover |Rationale +|TP |Typing / metatheory |Lean4 |Core calculus soundness +(progress/preservation, monad laws) + +|SEM |Semantics |Lean4 |Operational ↔ denotational adequacy; continuous +measure semantics + +|STAT |Statistics |Lean4 |Limit theorems, entropy bounds, MC convergence + +|ABI |ABI / FFI |Idris2 |Rust/Julia FFI boundary safety (mandatory, +mirrors AffineScript) + +|CONC |Concurrency |TLA+ |Parallel bet-execution model +|=== + +=== Required Proofs + +[width="100%",cols="9%,23%,19%,15%,19%,15%",options="header",] +|=== +|ID |Obligation |Category |Prover |Priority |Status +|TP-1 |Progress (well-typed ⇒ value or steps) |TP |Lean4 |P1 |✅ done + +|TP-2 |Preservation (typing preserved under step) |TP |Lean4 |P1 |✅ +done + +|TP-3 |Distribution monad laws (×3) |TP |Lean4 |P1 |✅ done + +|TP-4 |Discharge `+substTop_preserves_typing+` axiom |TP |Lean4 |P1 |✅ +done + +|TP-5 |Echo intro/elim typing rules + metatheory (Progress/Preservation +re-established) |TP |Lean4 |P2 |✅ done + +|TP-5b |Richer echo surface ops +(echo_map/echo_duplicate/echo_to_residue/sample_echo) + comonad laws |TP +|Lean4 |P3 |remaining + +|SEM-1 |Continuous measure-theoretic denotation |SEM |Lean4 |P2 +|remaining + +|STAT-1 |Maximum entropy of uniform ternary = log₂3 |STAT |Lean4 |P2 +|remaining + +|STAT-2 |SLLN for bet sample means |STAT |Lean4 |P2 |remaining + +|ABI-1 |FFI non-null pointer safety |ABI |Idris2 |P1 |remaining + +|ABI-2 |FFI memory-layout correctness |ABI |Idris2 |P1 |remaining + +|ABI-3 |Platform type-size proofs |ABI |Idris2 |P1 |remaining + +|ABI-4 |Foreign return-type proofs |ABI |Idris2 |P1 |remaining + +|ABI-5 |C-ABI compliance |ABI |Idris2 |P1 |remaining + +|CONC-1 |Parallel bet-execution model |CONC |TLA+ |P3 |remaining +|=== + +=== Banned Patterns + +No `+sorry+` / `+admit+` (Lean), `+Admitted+` (Coq), `+postulate+` / +`+believe_me+` / `+assert_total+` (Idris2/Agda), `+unsafeCoerce+`. A +single *classified* `+axiom+` is permitted under the standards#203 +trusted-base-reduction policy and must be registered in +`+docs/proof-debt.adoc+`. Enforced by `+tools/proof-scan.sh+`. + +=== How to Add a Proof + +[arabic] +. Choose the prover (see categories above). +. Place the file in the correct home (Lean: `+proofs/+`; others: +`+proofs//+`, exposed via `+verification/proofs/+`). +. Lean: rely on `+lake build+`; Idris2: `+%default total+`. +. Run `+just proof-check-all+`. +. Update `+PROOF-STATUS.md+` and, for any new escape hatch, +`+docs/proof-debt.adoc+`. + +''''' + +=== Historical Note — Template ABI Cleanup (2026-03-29) + +Template ABI removed — was creating a false impression of formal +verification. The removed files (`+Types.idr+`, `+Layout.idr+`, +`+Foreign.idr+`) contained only RSR template scaffolding with unresolved +`+{{PROJECT}}+`/`+{{AUTHOR}}+` placeholders and no domain-specific +proofs. + +When this project needs formal ABI verification (obligations *ABI-1..5* +above, Phase 3), create domain-specific Idris2 proofs following the +pattern in repos like `+typed-wasm+`, `+proven+`, `+echidna+`, or +`+boj-server+`, and in the AffineScript estate +(`+affinescript-vite/verification/proofs/idris2/ABI/+`). diff --git a/PROOF-NEEDS.md b/PROOF-NEEDS.md deleted file mode 100644 index 10d9e30..0000000 --- a/PROOF-NEEDS.md +++ /dev/null @@ -1,75 +0,0 @@ - - - - - - -# PROOF-NEEDS.md — BetLang - -This file defines *what* must be proven. Completion is tracked in -`PROOF-STATUS.md`; the phased plan is in -`docs/AFFINESCRIPT-ALIGNMENT.adoc`; the trusted-base ledger is in -`docs/proof-debt.adoc`. - -## Obligation Categories - -| Code | Category | Default prover | Rationale | -|------|----------|----------------|-----------| -| TP | Typing / metatheory | Lean4 | Core calculus soundness (progress/preservation, monad laws) | -| SEM | Semantics | Lean4 | Operational ↔ denotational adequacy; continuous measure semantics | -| STAT | Statistics | Lean4 | Limit theorems, entropy bounds, MC convergence | -| ABI | ABI / FFI | Idris2 | Rust/Julia FFI boundary safety (mandatory, mirrors AffineScript) | -| CONC | Concurrency | TLA+ | Parallel bet-execution model | - -## Required Proofs - -| ID | Obligation | Category | Prover | Priority | Status | -|----|------------|----------|--------|----------|--------| -| TP-1 | Progress (well-typed ⇒ value or steps) | TP | Lean4 | P1 | ✅ done | -| TP-2 | Preservation (typing preserved under step) | TP | Lean4 | P1 | ✅ done | -| TP-3 | Distribution monad laws (×3) | TP | Lean4 | P1 | ✅ done | -| TP-4 | Discharge `substTop_preserves_typing` axiom | TP | Lean4 | P1 | ✅ done | -| TP-5 | Echo intro/elim typing rules + metatheory (Progress/Preservation re-established) | TP | Lean4 | P2 | ✅ done | -| TP-5b | Richer echo surface ops (echo_map/echo_duplicate/echo_to_residue/sample_echo) + comonad laws | TP | Lean4 | P3 | remaining | -| SEM-1 | Continuous measure-theoretic denotation | SEM | Lean4 | P2 | remaining | -| STAT-1 | Maximum entropy of uniform ternary = log₂3 | STAT | Lean4 | P2 | remaining | -| STAT-2 | SLLN for bet sample means | STAT | Lean4 | P2 | remaining | -| ABI-1 | FFI non-null pointer safety | ABI | Idris2 | P1 | remaining | -| ABI-2 | FFI memory-layout correctness | ABI | Idris2 | P1 | remaining | -| ABI-3 | Platform type-size proofs | ABI | Idris2 | P1 | remaining | -| ABI-4 | Foreign return-type proofs | ABI | Idris2 | P1 | remaining | -| ABI-5 | C-ABI compliance | ABI | Idris2 | P1 | remaining | -| CONC-1 | Parallel bet-execution model | CONC | TLA+ | P3 | remaining | - -## Banned Patterns - -No `sorry` / `admit` (Lean), `Admitted` (Coq), `postulate` / -`believe_me` / `assert_total` (Idris2/Agda), `unsafeCoerce`. A single -**classified** `axiom` is permitted under the standards#203 -trusted-base-reduction policy and must be registered in -`docs/proof-debt.adoc`. Enforced by `tools/proof-scan.sh`. - -## How to Add a Proof - -1. Choose the prover (see categories above). -2. Place the file in the correct home (Lean: `proofs/`; others: - `proofs//`, exposed via `verification/proofs/`). -3. Lean: rely on `lake build`; Idris2: `%default total`. -4. Run `just proof-check-all`. -5. Update `PROOF-STATUS.md` and, for any new escape hatch, - `docs/proof-debt.adoc`. - ---- - -## Historical Note — Template ABI Cleanup (2026-03-29) - -Template ABI removed — was creating a false impression of formal -verification. The removed files (`Types.idr`, `Layout.idr`, -`Foreign.idr`) contained only RSR template scaffolding with unresolved -`{{PROJECT}}`/`{{AUTHOR}}` placeholders and no domain-specific proofs. - -When this project needs formal ABI verification (obligations **ABI-1..5** -above, Phase 3), create domain-specific Idris2 proofs following the -pattern in repos like `typed-wasm`, `proven`, `echidna`, or -`boj-server`, and in the AffineScript estate -(`affinescript-vite/verification/proofs/idris2/ABI/`). diff --git a/PROOF-STATUS.adoc b/PROOF-STATUS.adoc new file mode 100644 index 0000000..2bda4fe --- /dev/null +++ b/PROOF-STATUS.adoc @@ -0,0 +1,135 @@ +== Proof Status — BetLang + +See `+docs/AFFINESCRIPT-ALIGNMENT.adoc+` for the phased plan and +`+docs/proof-debt.adoc+` for the trusted-base reduction ledger. + +=== Summary + +[cols=",,,,,",options="header",] +|=== +|Category |Total |Done |In Progress |Blocked |Remaining +|Typing / metatheory (TP) |6 |5 |0 |0 |1 +|Semantics (SEM) |1 |0 |0 |0 |1 +|Statistics (STAT) |2 |0 |0 |0 |2 +|ABI / FFI (ABI) |5 |0 |0 |0 |5 +|Concurrency (CONC) |1 |0 |0 |0 |1 +|*Total* |*15* |*5* |*0* |*0* |*10* +|=== + +*Overall*: 33% proven (5 / 15). Lean core metatheory mechanised and (as +of Phase 1) machine-checked in CI. + +=== Proofs Done + +[width="100%",cols="12%,18%,21%,15%,34%",options="header",] +|=== +|ID |Proof |Prover |File |Verified By +|TP-1 |Progress — well-typed closed term is a value or steps |Lean4 +|`+proofs/BetLang.lean+` |`+lake build+` (CI: `+proofs.yml+`) + +|TP-2 |Preservation — typing preserved under reduction |Lean4 +|`+proofs/BetLang.lean+` |`+lake build+` (CI: `+proofs.yml+`) + +|TP-3 |Distribution monad laws (left id, right id, assoc) |Lean4 +|`+proofs/BetLang.lean+` |`+lake build+` (CI: `+proofs.yml+`) + +|TP-4 |Discharge `+substTop_preserves_typing+` (de Bruijn subst lemma) — +now a proved theorem, axiom-free |Lean4 |`+proofs/BetLang.lean+` +|`+lake build+` (CI: `+proofs.yml+`) + +|TP-5 |Echo intro/elim metatheory — `+echoIntro+`/`+echoElim+` +(echo_output) typing + β-rule + congruences + `+canonical_echo+`; +Progress/Preservation re-established over the extended calculus |Lean4 +|`+proofs/BetLang.lean+` |`+lake build+` (CI: `+proofs.yml+`) +|=== + +____ +Note: TP-2 is axiom-free. `+substTop_preserves_typing+` +(`+proofs/BetLang.lean:918+`) is a fully proved `+theorem+` — the former +classified axiom (TP-4) has been discharged (see +`+docs/proof-debt.adoc+` §(a)). No `+axiom+` declarations remain. +____ + +=== Proofs In Progress + +[cols=",,,",options="header",] +|=== +|ID |Proof |Prover |Notes +|— |— |— |— +|=== + +=== Proofs Blocked + +[cols=",,,",options="header",] +|=== +|ID |Proof |Blocked By |Notes +|— |— |— |— +|=== + +=== Proofs Remaining + +[width="100%",cols="11%,15%,21%,17%,15%,21%",options="header",] +|=== +|ID |Proof |Category |Prover |Phase |Priority +|TP-5b |Richer echo surface ops in Lean — +`+echo_map+`/`+echo_duplicate+`/`+echo_to_residue+`/`+sample_echo+` +(compose from `+echoIntro+`/`+echoElim+`) + the comonad laws from +`+EchoGradedComonad.agda+` |TP |Lean4 |2 |P3 + +|SEM-1 |Continuous measure-theoretic denotational semantics |SEM |Lean4 +|2 |P2 + +|STAT-1 |Maximum entropy of uniform ternary = log₂3 bits |STAT |Lean4 |2 +|P2 + +|STAT-2 |SLLN for bet sample means (a.s. convergence to expectation) +|STAT |Lean4 |2 |P2 + +|ABI-1 |FFI non-null pointer safety |ABI |Idris2 |3 |P1 + +|ABI-2 |FFI memory-layout correctness |ABI |Idris2 |3 |P1 + +|ABI-3 |Platform type-size proofs |ABI |Idris2 |3 |P1 + +|ABI-4 |Foreign return-type proofs |ABI |Idris2 |3 |P1 + +|ABI-5 |C-ABI compliance |ABI |Idris2 |3 |P1 + +|CONC-1 |Parallel bet-execution model |CONC |TLA+ |3 |P3 +|=== + +=== Verification Commands + +[source,bash] +---- +just proof-check-all # banned-pattern scan + lake build +just proof-check-lean4 # machine-check the Lean 4 formalisation +just proof-scan # banned-pattern gate only +---- + +=== Banned Patterns + +`+sorry+` / `+admit+` (Lean), `+Admitted+` (Coq), `+postulate+` / +`+believe_me+` / `+assert_total+` (Idris2/Agda), `+unsafeCoerce+`. +Enforced by `+tools/proof-scan.sh+` in CI +(`+.github/workflows/proofs.yml+`). No `+axiom+` declarations remain; +any future classified `+axiom+` would be permitted by policy +(standards#203) and registered in `+docs/proof-debt.adoc+`. + +=== Changelog + +[width="100%",cols="32%,42%,26%",options="header",] +|=== +|Date |Change |By +|2026-06-02 |Phase 1: Lean proofs made CI-machine-checked; status table +created. |alignment branch + +|2026-06-03 |Echo operations typed in `+bet-check+` +(`+echo+`/`+echo_output+`/`+echo_to_residue+`/`+sample_echo+`); +registered TP-5 for the Lean metatheory mirror. |echo-types pass + +|2026-06-15 |TP-5 discharged: `+echoIntro+`/`+echoElim+` modelled in +Lean (typing + β-rule + congruences + `+canonical_echo+`); +Progress/Preservation re-established; zero `+sorry+`. Richer surface ops +split to TP-5b. |TP-5 pass +|=== diff --git a/PROOF-STATUS.md b/PROOF-STATUS.md deleted file mode 100644 index e1c0732..0000000 --- a/PROOF-STATUS.md +++ /dev/null @@ -1,90 +0,0 @@ - - - - - - -# Proof Status — BetLang - -See `docs/AFFINESCRIPT-ALIGNMENT.adoc` for the phased plan and -`docs/proof-debt.adoc` for the trusted-base reduction ledger. - -## Summary - -| Category | Total | Done | In Progress | Blocked | Remaining | -|----------|-------|------|-------------|---------|-----------| -| Typing / metatheory (TP) | 6 | 5 | 0 | 0 | 1 | -| Semantics (SEM) | 1 | 0 | 0 | 0 | 1 | -| Statistics (STAT) | 2 | 0 | 0 | 0 | 2 | -| ABI / FFI (ABI) | 5 | 0 | 0 | 0 | 5 | -| Concurrency (CONC) | 1 | 0 | 0 | 0 | 1 | -| **Total** | **15** | **5** | **0** | **0** | **10** | - -**Overall**: 33% proven (5 / 15). Lean core metatheory mechanised and -(as of Phase 1) machine-checked in CI. - -## Proofs Done - -| ID | Proof | Prover | File | Verified By | -|----|-------|--------|------|-------------| -| TP-1 | Progress — well-typed closed term is a value or steps | Lean4 | `proofs/BetLang.lean` | `lake build` (CI: `proofs.yml`) | -| TP-2 | Preservation — typing preserved under reduction | Lean4 | `proofs/BetLang.lean` | `lake build` (CI: `proofs.yml`) | -| TP-3 | Distribution monad laws (left id, right id, assoc) | Lean4 | `proofs/BetLang.lean` | `lake build` (CI: `proofs.yml`) | -| TP-4 | Discharge `substTop_preserves_typing` (de Bruijn subst lemma) — now a proved theorem, axiom-free | Lean4 | `proofs/BetLang.lean` | `lake build` (CI: `proofs.yml`) | -| TP-5 | Echo intro/elim metatheory — `echoIntro`/`echoElim` (echo_output) typing + β-rule + congruences + `canonical_echo`; Progress/Preservation re-established over the extended calculus | Lean4 | `proofs/BetLang.lean` | `lake build` (CI: `proofs.yml`) | - -> Note: TP-2 is axiom-free. `substTop_preserves_typing` -> (`proofs/BetLang.lean:918`) is a fully proved `theorem` — the former -> classified axiom (TP-4) has been discharged (see `docs/proof-debt.adoc` -> §(a)). No `axiom` declarations remain. - -## Proofs In Progress - -| ID | Proof | Prover | Notes | -|----|-------|--------|-------| -| — | — | — | — | - -## Proofs Blocked - -| ID | Proof | Blocked By | Notes | -|----|-------|------------|-------| -| — | — | — | — | - -## Proofs Remaining - -| ID | Proof | Category | Prover | Phase | Priority | -|----|-------|----------|--------|-------|----------| -| TP-5b | Richer echo surface ops in Lean — `echo_map`/`echo_duplicate`/`echo_to_residue`/`sample_echo` (compose from `echoIntro`/`echoElim`) + the comonad laws from `EchoGradedComonad.agda` | TP | Lean4 | 2 | P3 | -| SEM-1 | Continuous measure-theoretic denotational semantics | SEM | Lean4 | 2 | P2 | -| STAT-1 | Maximum entropy of uniform ternary = log₂3 bits | STAT | Lean4 | 2 | P2 | -| STAT-2 | SLLN for bet sample means (a.s. convergence to expectation) | STAT | Lean4 | 2 | P2 | -| ABI-1 | FFI non-null pointer safety | ABI | Idris2 | 3 | P1 | -| ABI-2 | FFI memory-layout correctness | ABI | Idris2 | 3 | P1 | -| ABI-3 | Platform type-size proofs | ABI | Idris2 | 3 | P1 | -| ABI-4 | Foreign return-type proofs | ABI | Idris2 | 3 | P1 | -| ABI-5 | C-ABI compliance | ABI | Idris2 | 3 | P1 | -| CONC-1 | Parallel bet-execution model | CONC | TLA+ | 3 | P3 | - -## Verification Commands - -```bash -just proof-check-all # banned-pattern scan + lake build -just proof-check-lean4 # machine-check the Lean 4 formalisation -just proof-scan # banned-pattern gate only -``` - -## Banned Patterns - -`sorry` / `admit` (Lean), `Admitted` (Coq), `postulate` / `believe_me` / -`assert_total` (Idris2/Agda), `unsafeCoerce`. Enforced by -`tools/proof-scan.sh` in CI (`.github/workflows/proofs.yml`). No `axiom` -declarations remain; any future classified `axiom` would be permitted by -policy (standards#203) and registered in `docs/proof-debt.adoc`. - -## Changelog - -| Date | Change | By | -|------|--------|-----| -| 2026-06-02 | Phase 1: Lean proofs made CI-machine-checked; status table created. | alignment branch | -| 2026-06-03 | Echo operations typed in `bet-check` (`echo`/`echo_output`/`echo_to_residue`/`sample_echo`); registered TP-5 for the Lean metatheory mirror. | echo-types pass | -| 2026-06-15 | TP-5 discharged: `echoIntro`/`echoElim` modelled in Lean (typing + β-rule + congruences + `canonical_echo`); Progress/Preservation re-established; zero `sorry`. Richer surface ops split to TP-5b. | TP-5 pass | diff --git a/README.adoc b/README.adoc index f3e23f4..46bac3f 100644 --- a/README.adoc +++ b/README.adoc @@ -1,595 +1,153 @@ -// SPDX-License-Identifier: CC-BY-SA-4.0 -// Copyright (c) Jonathan D.A. Jewell -# BetLang +https://github.com/sponsors/hyperpolymath[image:https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github[Sponsor]] +link:LICENSE[image:https://img.shields.io/badge/License-MPL--2.0-blue.svg[License: +MPL-2.0]] -**A Symbolic Probabilistic Metalanguage / Probabilistic CAS** +== BetLang -BetLang is a domain-specific language for *symbolic probabilistic computation*. It is not a betting language—it is a compositional system for reasoning under uncertainty, built around a minimal ternary core and an extensible tower of uncertainty-aware number systems. +*A Symbolic Probabilistic Metalanguage / Probabilistic CAS* -At its heart is a single idea: +BetLang is a minimal ternary DSL hosted in Racket for _symbolic +probabilistic computation_. Its core primitive is a three-way stochastic +choice, supported by a Lean 4–mechanised type system and a Rust compiler +front-end. -> Computation is structured choice under uncertainty. +____ +Computation is structured choice under uncertainty. +____ ---- +For the full documentation see *README.adoc* and *EXPLAINME.adoc*. -== Core Concept -image:https://img.shields.io/badge/OpenSSF-Best_Practices-green?logo=openssourcesecurity[OpenSSF Best Practices,link="https://www.bestpractices.dev/en/projects/new?repo_url=https://github.com/hyperpolymath/betlang"] +''''' +=== Core Primitive -The fundamental primitive is the ternary form: - -``` -(bet A B C) -``` - -This represents a probabilistic, lazy choice between three branches. - -Unlike conventional probabilistic languages: - -* Evaluation is **lazy** (only the selected branch is computed) -* Choice is **first-class and compositional** -* The system is **symbolic-first**, not purely numeric - -This makes BetLang closer to a **probabilistic computer algebra system (CAS)** than a simulation DSL. - ---- - -== What BetLang *Is* - -* A **Symbolic Probabilistic Metalanguage (SPML)** -* A **Probabilistic CAS** for uncertainty-aware computation -* A **ternary computation model** with lazy semantics -* A **hosted language in Racket** with formalizable semantics -* A system with a **rich uncertainty-aware number tower (14 systems)** - -== What BetLang *Is Not* - -* Not a DeFi or gambling language -* Not just a Monte Carlo scripting tool -* Not limited to numeric probability (supports epistemic uncertainty, intervals, belief functions, etc.) - ---- - -== v1.0 Architecture - -BetLang is a **multi-layer system** with clearly separated responsibilities: - -=== Racket — Authoritative Frontend / Specification - -* `#lang betlang` defines the language -* `syntax-parse` + nanopass for IR and transformations -* Lazy ternary semantics are **canonical and non-negotiable** -* Optional type/refinement layer via Typed Racket / Turnstile+ - -This is the **source of truth** for semantics. - -=== Julia — Compute Kernel - -* High-performance execution backend -* Primary path for numerical and statistical workloads -* Integration with: - - * `Distributions.jl` - * `StatsBase.jl` - * `Random.jl` -* Planned: - - * `AbstractAlgebra.jl` - * `IntervalArithmetic.jl` - * Differentiable inference via `Zygote` / `Enzyme` - -Over time, the full **uncertainty number tower migrates here**. - -=== Lean 4 — Proof / Verification Layer - -* Machine-checks **Progress**, **Preservation**, and distribution **monad laws** -* **Zero `sorry`, zero `axiom`** — `substTop_preserves_typing` is fully proved - (formerly a classified axiom, since discharged; see `docs/proof-debt.adoc`) -* `lakefile.lean` + `lean-toolchain` make proofs a first-class Lake project -* `.github/workflows/proofs.yml` verifies every PR via `lake build` - -This layer ensures BetLang can become **formally trustworthy** — and already does, -for the mechanised fragment. - -=== Rust — Compiler Tooling Layer - -* `compiler/bet-core/` — core type definitions (incl. `Echo T`, `EchoR T`) -* `compiler/bet-check/` — Hindley-Milner type checker + unifier (27 tests pass) -* `compiler/bet-parse/` — LALRPOP-based surface parser -* `compiler/bet-wasm/` — WASM backend (builds; scaffolding — emits default-return stubs pending AST lowering) - -Rust is the compiler tooling layer. Core semantics remain in Racket. - ---- - -== Core Features - -=== Ternary Computation - -* `(bet A B C)` — primitive stochastic choice (uniform) -* `(bet/weighted '(A w1) '(B w2) '(C w3))` — non-uniform probabilities -* `(bet/conditional pred A B C)` — predicate-driven selection -* `(bet/lazy thunk-a thunk-b thunk-c)` — only selected thunk is invoked -* `(bet-with-seed seed thunk)` — deterministic seed for reproducibility - -=== Lazy Semantics - -* Only the chosen branch is evaluated -* Enables symbolic and infinite structures -* Prevents unnecessary computation - -=== Compositionality - -* Bets compose like algebraic objects -* Supports chaining, mapping, folding, and higher-order composition - -=== Echo Types — Structured Loss - -BetLang's type system includes **structured-loss type formers** from -link:https://github.com/hyperpolymath/echo-types[`hyperpolymath/echo-types`] (Agda). - -[cols="1,3"] -|=== -| Type | Meaning - -| `T` -| An ordinary value - -| `Echo T` -| A `T`-value carrying a **proof-relevant residue** of retained loss. - _Distinct from `T`_: `unify(Echo T, T)` fails by design. - -| `EchoR T` -| The strict, non-recoverable residue of `Echo T`. Introduced by - `echo_to_residue`; further operations deferred. -|=== - -`Echo T` and `EchoR T` erase to `T` at runtime (ghost types) until operations -demand a payload. The canonical introduction site is probabilistic support retention: -`sample : Dist T → T` discards which branch fired; -`sample_echo : Dist T → Echo T` retains that residue. The operations now -type-check in `bet-check` (type-level; the runtime residue payload remains -deferred): introduction `echo` (`'a → Echo 'a`); the **functor + comonad -surface** `echo_map` (`('a→'b) → Echo 'a → Echo 'b`), `echo_output` -(`Echo 'a → 'a`, the counit), `echo_duplicate` (`Echo 'a → Echo (Echo 'a)`); -the residue lowering `echo_to_residue` (`Echo 'a → EchoR 'a`); and the -probabilistic bridge `sample_echo` (`Dist 'a → Echo 'a`). These mirror the -graded comonad proved in `hyperpolymath/echo-types`. - -See link:docs/echo-types.adoc[`docs/echo-types.adoc`] for the full design rationale. - ---- - -== The Real Moat: Uncertainty-Aware Number Systems - -BetLang includes **14 distinct number systems** for representing uncertainty: - -* Gaussian distributions -* Interval / affine arithmetic -* Fuzzy numbers -* Bayesian numbers -* Risk-based numbers (VaR / CVaR) -* Surreal and hyperreal systems -* p-adic probability systems -* Imprecise probabilities -* Dempster–Shafer belief functions - -These are not addons—they are the **type system of the language**. - -See: `docs/number-tower.md` - ---- - -== Semantics - -BetLang distinguishes between: - -* **Ternary logic** (Kleene-style truth values) -* **Ternary probabilistic belief** (distributional uncertainty) - -This distinction is critical and formalized in: - -* `docs/ternary-semantics.md` - ---- - -== Interactive Playgrounds - -BetLang provides two playground environments for exploration and experimentation: - -=== Web-Based Quantum Playground (`ui/`) - -A **quantum-inspired interactive web environment** for exploring Betlang's probabilistic model. - -* **Location**: `ui/` directory -* **Features**: - - * Code editor with syntax hints - * 8 quantum-analog examples (superposition, entanglement, measurement) - * Probability distribution histograms (SVG) - * Ternary bet tree visualization - * Configurable sample counts (1-100,000) - * Dark/light theme switching - * History navigation -* **Technology**: AffineScript + Vite + Deno -* **URL**: https://betlang.org/playground (when deployed) - -See: link:ui/README.adoc[ui/README.adoc] for complete documentation. - -=== Experimental Playground (`playground/`) - -A **sandbox workspace** for FFI development, configuration, and experimental features. - -* **Location**: `playground/` directory -* **Features**: - - * FFI bindings (Zig, WASM planned) - * Project configuration (Nickel, Must, Just) - * Experimental code snippets - * RSR compliance checking - * Architecture documentation -* **Technology**: Deno, Zig, Nickel, Just - -See: link:playground/README.adoc[playground/README.adoc] for complete documentation. - -=== Shareable URLs - -Both playgrounds support **URL-encoded code sharing**: - -[source] +[source,scheme] ---- -# Share a specific snippet -https://betlang.org/playground#code= - -# Share with configuration -https://betlang.org/playground?theme=dark&samples=5000#code=... - -# Load specific example -https://betlang.org/playground?example=superposition ----- - -=== QR Code Generation - -Generate QR codes for physical sharing: - -[source,bash] +(bet A B C) ;; uniform ternary choice +(bet/weighted '(A 7) '(B 2) '(C 1)) ;; non-uniform +(bet/lazy thunk-a thunk-b thunk-c) ;; only selected thunk runs +(bet-with-seed 42 (lambda () ...)) ;; reproducible ---- -# Using qrencode CLI -qrencode -t ANSIUTF8 "https://betlang.org/playground#code=$(echo 'bet {1,2,3}' | base64)" -# Using Python -python -c "import qrcode; img = qrcode.make('https://betlang.org/playground#code=...'); img.save('betlang.png')" ----- +The selected branch is the only branch evaluated (lazy semantics). -=== Social Media Integration +''''' -Share via popular platforms: +=== Architecture -[cols="2,3"] +[width="100%",cols="34%,28%,38%",options="header",] |=== -| Twitter/X | `https://twitter.com/intent/tweet?text=Betlang%20Playground%20https://betlang.org/playground` -| Mastodon | `https://mastodon.social/share?text=Exploring%20Betlang%20https://betlang.org/playground` -| LinkedIn | `https://www.linkedin.com/sharing/share-offsite/?url=https://betlang.org/playground` -| Reddit | `https://www.reddit.com/submit?url=https://betlang.org/playground&title=Betlang%20Playground` -| Bluesky | `https://bsky.app/intent/compose?text=Check%20out%20Betlang%20https://betlang.org/playground` -| Discord | `https://discord.com/app?url=https://betlang.org/playground` -| Matrix | `https://matrix.to/#/#betlang:matrix.org?web-instance[element.io]=https://betlang.org/playground` -|=== - ---- - -== Tooling Roadmap - -Planned unified CLI: - -``` -bet check # static + semantic validation -bet fmt # canonical formatting -bet lsp # editor integration -bet run --julia # execute via Julia backend -bet trace # execution tracing -bet hash # reproducibility / identity -``` - -See: `docs/toolchain-roadmap.md` +|Layer |Role |Status +|*Racket* |Language and canonical semantics |✅ Active ---- +|*Lean 4* |Mechanised proofs (Progress + Preservation + monad laws) |✅ +Machine-checked -== Ecosystem Integration +|*Rust* |Type checker + compiler (`+bet-check+`, `+bet-core+`) |✅ +Active -=== K9 Contractiles - -K9 (Kennel-Yard-Hunt) **self-validating contractiles** provide configuration and deployment automation: - -[cols="1,2,2"] +|*Julia* |High-performance compute backend |🟡 Development |=== -| Level | Trust | Use Case - -| Kennel | High | Pure data, schemas, no execution -| Yard | Medium | Validated configuration with Nickel -| Hunt | Low | Full execution, requires signature -|=== - -* **Location**: `.machine_readable/svc/k9/` -* **Purpose**: Configuration management, deployment validation, security auditing -=== ADR (Architecture Decision Records) +''''' -All architectural decisions are documented as ADRs: +=== Echo Types -* **ADR-001**: K9 service component organization -* **ADR-002**: RSR compliance enforcement -* **ADR-003**: Machine-readable metadata first +BetLang’s type system includes structured-loss formers from +https://github.com/hyperpolymath/echo-types[`+hyperpolymath/echo-types+`] +(Agda source of truth): -* **Location**: `.machine_readable/svc/` - -=== Machine-Readable Metadata - -The project maintains extensive machine-readable metadata: - -[source] ----- -.machine_readable/ -├── 6a2/ -│ ├── AGENTIC.a2ml # AI agent constraints and permissions -│ ├── ECOSYSTEM.a2ml # Project ecosystem positioning -│ ├── META.a2ml # Project metadata and versioning -│ ├── NEUROSYM.a2ml # Neural-symbolic integration -│ ├── PLAYBOOK.a2ml # Operational procedures -│ └── STATE.a2ml # Current project state -└── svc/ - └── k9/ - └── README.adoc # K9 contractiles documentation ----- - -Each `.a2ml` file provides structured configuration for: - -* AI agent interactions and constraints -* Project positioning within the broader ecosystem -* Metadata versioning and updates -* Neural-symbolic integration patterns -* Operational playbooks and incident response -* Current state tracking - -=== AI Agent Integration (0-AI-MANIFEST.a2ml) - -The project includes comprehensive AI agent guidance: - -[source] ----- -0-AI-MANIFEST.a2ml # Universal AI Agent Gateway -├── Identity and project metadata -├── Critical invariants (banned languages, file locations) -├── Canonical file locations -├── Taxonomy index -└── Agent constraints ----- - -Key constraints for AI agents: - -* SCM files ONLY in `.machine_readable/` (root copies are symlinks) -* NEVER delete spec files: grammar, SPEC.core.scm -* NEVER use banned languages: TypeScript, Node.js, npm, Go, Python -* All GitHub Actions must be SHA-pinned -* All source files must have SPDX headers - ---- - -== Example - -``` -(bet 'win 'draw 'lose) -``` - -``` -(bet/weighted '(common 7) '(uncommon 2) '(rare 1)) -``` - -``` -(bet (expensive-computation) - (cheap-approximation) - (fallback)) -``` - -Only one branch is evaluated. - ---- - -== Use Cases - -* Probabilistic programming -* Bayesian inference -* Uncertainty quantification -* Symbolic statistics -* Decision theory -* Risk modeling -* Scientific computing -* Research in probabilistic semantics -* **Education** - Teaching probability and uncertainty concepts -* **Prototyping** - Quick experimentation with probabilistic models - ---- - -== Quantum-Analog Concepts - -For those familiar with quantum computing, BetLang provides analogous concepts: - -[cols="1,2,2"] +[width="100%",cols="40%,60%",options="header",] |=== -| Quantum Concept | BetLang Analog | Description - -| Qubit | Ternary Bet | `bet { |0>, |1>, |unknown> }` - three possible states -| Superposition | Weighted Bet | `bet { a @ w1, b @ w2, c @ w3 }` - probability amplitudes -| Entanglement | Nested Bets | `bet { x, y, bet { a, b } }` - correlated outcomes -| Measurement | Switch/Case | `switch result { | a => ..., | b => ... }` - collapse to value -| Interference | Probability Weights | Weighted combinations affect outcome distribution -| Unitary Evolution | Function Composition | Functions preserve probabilistic structure +|Type |Meaning +|`+Echo T+` |`+T+`-value with proof-relevant retained-loss residue. +*Distinct from `+T+`.* + +|`+EchoR T+` |Strict non-recoverable residue. Reserved; operations +deferred. |=== -This mapping makes BetLang accessible to quantum programmers while maintaining its own semantic rigor. +`+unify(Echo T, T)+` fails by design. Both types are ghost-erased at +runtime until operations demand a payload. See docs/echo-types.adoc. ---- +''''' -== Philosophy +=== Proofs -BetLang is built on three principles: +`+proofs/BetLang.lean+` machine-checks Progress, Preservation, and monad +laws with *0 `+sorry+`*. -1. **Ternary over binary** — real-world decisions are rarely yes/no -2. **Uncertainty as structure** — not noise, but a first-class object -3. **Symbolic first** — computation should remain inspectable and composable +* `+lakefile.lean+` + `+lean-toolchain+` → buildable Lake project +* `+.github/workflows/proofs.yml+` → CI-checked on every PR +(`+lake build+` + banned-pattern scan) +* Axiom-free: `+substTop_preserves_typing+` is fully proved — no +`+axiom+`/`+sorry+` (see `+docs/proof-debt.adoc+`) ---- +''''' -== Status +=== Status -[cols="2,1,1"] +[width="100%",cols="44%,30%,26%",options="header",] |=== -| Component | Status | Notes - -| Racket frontend | ✅ Authoritative | Canonical semantics -| Julia backend | 🟡 Active development | Primary compute kernel -| Lean 4 proofs | ✅ Machine-checked | Progress + Preservation + monad laws -| Rust type-checker | ✅ Active | bet-check / bet-core (incl. Echo T + operations) -| VS Code extension | 🟡 In progress | AffineScript source (replaces AffineScript) -| Web Playground (ui/) | ✅ Available | Quantum-inspired features -| FFI (Zig) | 🟡 Experimental | playground/ffi/zig/ -| WASM backend | 🟡 Builds (scaffolding) | E0308 fixed; emits default-return stubs pending AST lowering +|Component |Status |Notes +|Racket frontend |✅ Authoritative |Canonical semantics +|Lean 4 proofs |✅ Machine-checked |Progress + Preservation + monad laws +|Rust type-checker |✅ Active |bet-check / bet-core (incl. Echo T) +|Julia backend |🟡 Development |Core language features +|VS Code extension |🟡 In progress |AffineScript source +|WASM backend |⏸️ Paused |Pre-existing build issue |=== ---- - -== Project Structure +''''' -[source] ----- -betlang/ -├── README.adoc # This file -├── LICENSE # MPL-2.0 -├── 0-AI-MANIFEST.a2ml # AI agent guidance -├── .machine_readable/ # Machine-readable metadata -│ ├── 6a2/ # Project state and config -│ └── svc/ # Service components -│ └── k9/ # K9 contractiles -├── compiler/ # Rust compiler components -│ ├── bet-wasm/ # WASM backend -│ └── ... -├── core/ # Racket core language -├── julia-backend/ # Julia compute kernel -├── ui/ # Web playground (quantum-inspired) -│ ├── public/ # Static assets -│ │ ├── index.html -│ │ └── styles.css -│ ├── src/ # AffineScript source -│ │ └── App.res -│ ├── deno.json -│ ├── affinescript.json -│ ├── vite.config.js -│ └── Justfile -├── playground/ # Experimental sandbox -│ ├── ARCHITECTURE.adoc -│ ├── README.adoc # Playground documentation -│ ├── Justfile -│ ├── config.ncl -│ ├── mustfile.toml -│ ├── deno.json -│ ├── affinescript.json -│ ├── ffi/ # FFI bindings -│ │ └── zig/ -│ └── examples/ -└── docs/ # Documentation ----- - ---- - -== Getting Started - -=== Prerequisites - -* Racket (for language frontend) -* Julia (for compute backend, optional) -* Deno (for playground/web, optional) -* Rust (for compiler, optional) - -=== Quick Install +=== Quick Start [source,bash] ---- -# Clone the repository -git clone https://github.com/hyperpolymath/betlang.git -cd betlang +# Core Racket DSL +racket tests/basics.rkt -# For Racket development -raco pkg install betlang +# Proofs +lake build # requires Lean 4 (see lean-toolchain) -# For web playground -git submodule update --init --recursive -cd ui -just dev +# Rust type-checker +cargo test -p bet-check -# For experimental playground -cd playground -just check +# All via just +just proof-check-all ---- -=== Running Examples - -[source,bash] ----- -# Run conformance tests -racket -f conformance/smoke.bet - -# Run Julia backend examples -cd julia-backend/examples -julia coin-flip-game.bet - -# Start web playground -cd ui -just dev ----- - ---- - -== Contributing - -See link:CONTRIBUTING.md[CONTRIBUTING.md] for development guidelines. - -Key principles: +''''' -* All files must have SPDX license headers (`MPL-2.0`) -* Machine-readable metadata in `.machine_readable/6a2/` must be kept in sync -* Changes must pass RSR compliance checks (`just validate`) -* Follow ADR patterns for architectural decisions (see `META.a2ml`) -* No TypeScript outside `playground/` — use AffineScript -* No `sorry` / `admit` in Lean proofs — `axiom` is permitted only for classified obligations -* `Echo T` must remain distinct from `T` — do not add implicit coercion -* New agents: read `EXPLAINME.adoc` first, then `.machine_readable/6a2/AGENTIC.a2ml` +=== Language Policy ---- +* *No TypeScript* outside `+playground/+` (approved sandbox exemption, +see `+.claude/CLAUDE.md+`) +* *No Python, Go, Java, Kotlin, Swift* — see full policy in +`+.claude/CLAUDE.md+` +* *AffineScript* replaces TypeScript/AffineScript for editor tooling +* *Deno* replaces Node/npm +* All files must have SPDX `+MPL-2.0+` headers -== License +''''' -BetLang uses **MPL-2.0 (Palimpsest Public License)**. +=== Contributing -See `LICENSE` for the precise definition and terms. +See CONTRIBUTING.md. For a guided tour, read EXPLAINME.adoc first. ---- +''''' -== Closing +=== License -BetLang is not about betting. +BetLang is licensed under *MPL-2.0*. SPDX identifier: `+MPL-2.0+`. -It is about **making uncertainty programmable**. +See LICENSE and PALIMPSEST.adoc. ---- +''''' -== Links +=== Links -* link:https://github.com/hyperpolymath/betlang[GitHub Repository] -* link:https://betlang.org[Project Homepage] -* link:EXPLAINME.adoc[EXPLAINME — guided tour for contributors and agents] -* link:ui/README.adoc[Quantum Playground Documentation] -* link:playground/README.adoc[Experimental Playground Documentation] -* link:docs/echo-types.adoc[Echo Types Design Doc] -* link:docs/AFFINESCRIPT-ALIGNMENT.adoc[AffineScript Alignment Plan] -* link:.machine_readable/6a2/ECOSYSTEM.a2ml[Ecosystem Position] -* link:.machine_readable/6a2/AGENTIC.a2ml[AI Agent Manifest] +* EXPLAINME.adoc — guided tour for contributors and agents +* docs/echo-types.adoc — Echo Types design +* docs/AFFINESCRIPT-ALIGNMENT.adoc — alignment plan +* .machine_readable/6a2/ECOSYSTEM.a2ml — ecosystem position diff --git a/README.md b/README.md deleted file mode 100644 index 9b56634..0000000 --- a/README.md +++ /dev/null @@ -1,131 +0,0 @@ - - - - -[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github)](https://github.com/sponsors/hyperpolymath) -[![License: MPL-2.0](https://img.shields.io/badge/License-MPL--2.0-blue.svg)](LICENSE) - -# BetLang - -**A Symbolic Probabilistic Metalanguage / Probabilistic CAS** - -BetLang is a minimal ternary DSL hosted in Racket for *symbolic probabilistic computation*. -Its core primitive is a three-way stochastic choice, supported by a Lean 4–mechanised type -system and a Rust compiler front-end. - -> Computation is structured choice under uncertainty. - -For the full documentation see **[README.adoc](README.adoc)** and **[EXPLAINME.adoc](EXPLAINME.adoc)**. - ---- - -## Core Primitive - -```scheme -(bet A B C) ;; uniform ternary choice -(bet/weighted '(A 7) '(B 2) '(C 1)) ;; non-uniform -(bet/lazy thunk-a thunk-b thunk-c) ;; only selected thunk runs -(bet-with-seed 42 (lambda () ...)) ;; reproducible -``` - -The selected branch is the only branch evaluated (lazy semantics). - ---- - -## Architecture - -| Layer | Role | Status | -|-------|------|--------| -| **Racket** | Language and canonical semantics | ✅ Active | -| **Lean 4** | Mechanised proofs (Progress + Preservation + monad laws) | ✅ Machine-checked | -| **Rust** | Type checker + compiler (`bet-check`, `bet-core`) | ✅ Active | -| **Julia** | High-performance compute backend | 🟡 Development | - ---- - -## Echo Types - -BetLang's type system includes structured-loss formers from -[`hyperpolymath/echo-types`](https://github.com/hyperpolymath/echo-types) (Agda source of truth): - -| Type | Meaning | -|------|---------| -| `Echo T` | `T`-value with proof-relevant retained-loss residue. **Distinct from `T`.** | -| `EchoR T` | Strict non-recoverable residue. Reserved; operations deferred. | - -`unify(Echo T, T)` fails by design. Both types are ghost-erased at runtime until operations -demand a payload. See [docs/echo-types.adoc](docs/echo-types.adoc). - ---- - -## Proofs - -`proofs/BetLang.lean` machine-checks Progress, Preservation, and monad laws with **0 `sorry`**. - -- `lakefile.lean` + `lean-toolchain` → buildable Lake project -- `.github/workflows/proofs.yml` → CI-checked on every PR (`lake build` + banned-pattern scan) -- Axiom-free: `substTop_preserves_typing` is fully proved — no `axiom`/`sorry` (see `docs/proof-debt.adoc`) - ---- - -## Status - -| Component | Status | Notes | -|-----------|--------|-------| -| Racket frontend | ✅ Authoritative | Canonical semantics | -| Lean 4 proofs | ✅ Machine-checked | Progress + Preservation + monad laws | -| Rust type-checker | ✅ Active | bet-check / bet-core (incl. Echo T) | -| Julia backend | 🟡 Development | Core language features | -| VS Code extension | 🟡 In progress | AffineScript source | -| WASM backend | ⏸️ Paused | Pre-existing build issue | - ---- - -## Quick Start - -```bash -# Core Racket DSL -racket tests/basics.rkt - -# Proofs -lake build # requires Lean 4 (see lean-toolchain) - -# Rust type-checker -cargo test -p bet-check - -# All via just -just proof-check-all -``` - ---- - -## Language Policy - -- **No TypeScript** outside `playground/` (approved sandbox exemption, see `.claude/CLAUDE.md`) -- **No Python, Go, Java, Kotlin, Swift** — see full policy in `.claude/CLAUDE.md` -- **AffineScript** replaces TypeScript/AffineScript for editor tooling -- **Deno** replaces Node/npm -- All files must have SPDX `MPL-2.0` headers - ---- - -## Contributing - -See [CONTRIBUTING.md](CONTRIBUTING.md). For a guided tour, read [EXPLAINME.adoc](EXPLAINME.adoc) first. - ---- - -## License - -BetLang is licensed under **MPL-2.0**. SPDX identifier: `MPL-2.0`. - -See [LICENSE](LICENSE) and [PALIMPSEST.adoc](PALIMPSEST.adoc). - ---- - -## Links - -- [EXPLAINME.adoc](EXPLAINME.adoc) — guided tour for contributors and agents -- [docs/echo-types.adoc](docs/echo-types.adoc) — Echo Types design -- [docs/AFFINESCRIPT-ALIGNMENT.adoc](docs/AFFINESCRIPT-ALIGNMENT.adoc) — alignment plan -- [.machine_readable/6a2/ECOSYSTEM.a2ml](.machine_readable/6a2/ECOSYSTEM.a2ml) — ecosystem position diff --git a/RUST-STATUS.adoc b/RUST-STATUS.adoc new file mode 100644 index 0000000..c1143c0 --- /dev/null +++ b/RUST-STATUS.adoc @@ -0,0 +1,152 @@ +== Rust Compiler Status + +*Last Updated:* 2026-02-07 *Status:* Blocked - LALRPOP Parser Conflicts + +=== Overview + +The Rust compiler is *optional, non-authoritative tooling* for betlang. +The *authoritative implementation is Racket* (see `+core/betlang.rkt+` +and `+lib/*.rkt+`), which is *100% complete and production-ready*. + +=== Current Blockers + +==== LALRPOP Parser Generation Failures + +The parser generator fails with *3 shift/reduce conflicts* in +`+compiler/bet-parse/src/grammar.lalrpop+`: + +[arabic] +. *Type parsing conflict (line 157):* ++ +.... +TypeAtom: Type = { + "(" ")" => t, +} +.... ++ +Parser cannot decide when to reduce `+Type+` vs shift `+)+` token. +. *Pattern list parsing conflict (line 410):* ++ +.... +PatternAtom: Pattern = { + "[" >> "]" => Pattern::List(elems, None), +} +.... ++ +The `+Comma+` macro creates ambiguity in list pattern parsing. +. *Let expression conflict (line 378):* ++ +.... +DoStatement vs ExprAtom (let-in) +.... ++ +Ambiguity between do-block let statements and let-in expressions. + +==== Root Cause + +These are *LR(1) grammar conflicts* caused by: - Recursive productions +with parentheses - The `+Comma+` and `+Sep+` helper macros not +being LR(1) compatible - Overlapping syntax between do-blocks and let-in +expressions + +==== Partial Fixes Completed (2026-02-07) + +The following compilation issues were resolved: + +✅ *Serde serialization:* - Enabled `+im+` crate serde feature for +HashMap/Vector - Implemented custom Arc serde helpers - Manual PartialEq +implementation for Value enum + +✅ *Type system:* - Added lifetime specifiers to reference-returning +functions - Fixed return type mismatches + +✅ *API differences:* - Fixed `+im::HashSet+` API usage (is_superset → +is_subset) - Fixed `+im::HashSet+` intersection method + +==== Blocked Components + +Because the parser fails to generate, the entire Rust build is blocked: + +* ❌ Parser (generation fails) +* ❌ Type checker (stubbed, cannot test) +* ❌ Interpreter (stubbed, cannot test) +* ❌ Code generator (stubbed, cannot test) +* ❌ CLI tool (depends on parser) +* ❌ LSP server (depends on parser) + +=== Solutions + +==== Option 1: Use Racket (Recommended) + +*The Racket implementation is complete and production-ready.* + +[source,bash] +---- +# Run betlang programs +racket examples/safety-features.rkt + +# Interactive REPL +racket repl/shell.rkt + +# Run tests +racket tests/basics.rkt +---- + +All 4 safety features and all 14 number systems work perfectly. + +==== Option 2: Fix LALRPOP Grammar (Future Work) + +Estimated effort: *4-8 hours* for experienced LALRPOP developer + +Required changes: 1. Replace `+Sep+` and `+Comma+` macros with +LALRPOP built-ins (`++`, `++`) 2. Refactor Type/Pattern +productions to eliminate parenthesis ambiguity 3. Add precedence +declarations for let-in vs do-let 4. Possibly introduce intermediate +non-terminals to break LR(1) cycles + +==== Option 3: Alternative Parser (Major Rewrite) + +Switch to: - *nom* (parser combinator library) - *pest* (PEG parser +generator) - *Hand-written recursive descent parser* + +Estimated effort: *8-16 hours* + +=== Current Rust Completion Status + +[cols=",,,",options="header",] +|=== +|Component |Status |Completion |Notes +|Lexer |✅ Working |100% |Logos-based, all tokens +|Parser |❌ Blocked |0% |LALRPOP conflicts +|AST |✅ Complete |100% |All node types defined +|Type Checker |⚠️ Stubbed |10% |Framework exists, logic needed +|Interpreter |⚠️ Stubbed |10% |Framework exists, logic needed +|Code Generator |⚠️ Stubbed |5% |Placeholder only +|Runtime Library |✅ Working |90% |Some compilation issues fixed +|CLI |❌ Blocked |0% |Depends on parser +|LSP |❌ Blocked |0% |Depends on parser +|=== + +*Overall Rust Completion:* ~30% + +=== Recommendation + +*Use the Racket implementation.* It is: - ✅ Complete and tested - ✅ +Authoritative per project design - ✅ Production-ready - ✅ Fully +documented - ✅ Includes REPL, examples, tests, benchmarks + +The Rust compiler is *optional future work* and not required for using +betlang. + +=== Contributing + +If you’d like to fix the Rust compiler: + +[arabic] +. Familiarity with LALRPOP parser generators required +. See `+compiler/bet-parse/src/grammar.lalrpop+` +. Focus on resolving the 3 shift/reduce conflicts +. Test with `+cargo build+` in repository root +. Submit a pull request + +For questions: j.d.a.jewell@open.ac.uk diff --git a/RUST-STATUS.md b/RUST-STATUS.md deleted file mode 100644 index 028b976..0000000 --- a/RUST-STATUS.md +++ /dev/null @@ -1,152 +0,0 @@ - -# Rust Compiler Status - -**Last Updated:** 2026-02-07 -**Status:** Blocked - LALRPOP Parser Conflicts - -## Overview - -The Rust compiler is **optional, non-authoritative tooling** for betlang. The **authoritative implementation is Racket** (see `core/betlang.rkt` and `lib/*.rkt`), which is **100% complete and production-ready**. - -## Current Blockers - -### LALRPOP Parser Generation Failures - -The parser generator fails with **3 shift/reduce conflicts** in `compiler/bet-parse/src/grammar.lalrpop`: - -1. **Type parsing conflict (line 157):** - ``` - TypeAtom: Type = { - "(" ")" => t, - } - ``` - Parser cannot decide when to reduce `Type` vs shift `)` token. - -2. **Pattern list parsing conflict (line 410):** - ``` - PatternAtom: Pattern = { - "[" >> "]" => Pattern::List(elems, None), - } - ``` - The `Comma` macro creates ambiguity in list pattern parsing. - -3. **Let expression conflict (line 378):** - ``` - DoStatement vs ExprAtom (let-in) - ``` - Ambiguity between do-block let statements and let-in expressions. - -### Root Cause - -These are **LR(1) grammar conflicts** caused by: -- Recursive productions with parentheses -- The `Comma` and `Sep` helper macros not being LR(1) compatible -- Overlapping syntax between do-blocks and let-in expressions - -### Partial Fixes Completed (2026-02-07) - -The following compilation issues were resolved: - -✅ **Serde serialization:** -- Enabled `im` crate serde feature for HashMap/Vector -- Implemented custom Arc serde helpers -- Manual PartialEq implementation for Value enum - -✅ **Type system:** -- Added lifetime specifiers to reference-returning functions -- Fixed return type mismatches - -✅ **API differences:** -- Fixed `im::HashSet` API usage (is_superset → is_subset) -- Fixed `im::HashSet` intersection method - -### Blocked Components - -Because the parser fails to generate, the entire Rust build is blocked: - -- ❌ Parser (generation fails) -- ❌ Type checker (stubbed, cannot test) -- ❌ Interpreter (stubbed, cannot test) -- ❌ Code generator (stubbed, cannot test) -- ❌ CLI tool (depends on parser) -- ❌ LSP server (depends on parser) - -## Solutions - -### Option 1: Use Racket (Recommended) - -**The Racket implementation is complete and production-ready.** - -```bash -# Run betlang programs -racket examples/safety-features.rkt - -# Interactive REPL -racket repl/shell.rkt - -# Run tests -racket tests/basics.rkt -``` - -All 4 safety features and all 14 number systems work perfectly. - -### Option 2: Fix LALRPOP Grammar (Future Work) - -Estimated effort: **4-8 hours** for experienced LALRPOP developer - -Required changes: -1. Replace `Sep` and `Comma` macros with LALRPOP built-ins (``, ``) -2. Refactor Type/Pattern productions to eliminate parenthesis ambiguity -3. Add precedence declarations for let-in vs do-let -4. Possibly introduce intermediate non-terminals to break LR(1) cycles - -### Option 3: Alternative Parser (Major Rewrite) - -Switch to: -- **nom** (parser combinator library) -- **pest** (PEG parser generator) -- **Hand-written recursive descent parser** - -Estimated effort: **8-16 hours** - -## Current Rust Completion Status - -| Component | Status | Completion | Notes | -|-----------|--------|------------|-------| -| Lexer | ✅ Working | 100% | Logos-based, all tokens | -| Parser | ❌ Blocked | 0% | LALRPOP conflicts | -| AST | ✅ Complete | 100% | All node types defined | -| Type Checker | ⚠️ Stubbed | 10% | Framework exists, logic needed | -| Interpreter | ⚠️ Stubbed | 10% | Framework exists, logic needed | -| Code Generator | ⚠️ Stubbed | 5% | Placeholder only | -| Runtime Library | ✅ Working | 90% | Some compilation issues fixed | -| CLI | ❌ Blocked | 0% | Depends on parser | -| LSP | ❌ Blocked | 0% | Depends on parser | - -**Overall Rust Completion:** ~30% - -## Recommendation - -**Use the Racket implementation.** It is: -- ✅ Complete and tested -- ✅ Authoritative per project design -- ✅ Production-ready -- ✅ Fully documented -- ✅ Includes REPL, examples, tests, benchmarks - -The Rust compiler is **optional future work** and not required for using betlang. - -## Contributing - -If you'd like to fix the Rust compiler: - -1. Familiarity with LALRPOP parser generators required -2. See `compiler/bet-parse/src/grammar.lalrpop` -3. Focus on resolving the 3 shift/reduce conflicts -4. Test with `cargo build` in repository root -5. Submit a pull request - -For questions: j.d.a.jewell@open.ac.uk diff --git a/SAFETY-FEATURES.adoc b/SAFETY-FEATURES.adoc new file mode 100644 index 0000000..c8c89aa --- /dev/null +++ b/SAFETY-FEATURES.adoc @@ -0,0 +1,335 @@ +== BetLang Safety Features + +*Status:* ✅ *IMPLEMENTED* (2026-02-01) + +BetLang is the first probabilistic programming language with +comprehensive gambling harm reduction and mathematical safety +guarantees. + +=== Four Safety Pillars + +==== 1. Dutch Book Prevention ✅ + +*Module:* `+lib/dutch-book.rkt+` + +Prevents arbitrage opportunities by validating that probabilities sum to +1.0 within tolerance. + +*Features:* - Compile-time probability validation - Dutch book detection +from betting odds - Probability normalization - Bookmaker margin +calculation - Validated bet execution (`+bet/validated+`) + +*Example:* + +[source,racket] +---- +(require "lib/dutch-book.rkt") + +;; Detect Dutch book +(define odds '(2.5 3.2 3.0)) ;; Bookmaker odds +(detect-dutch-book odds) +;; => "Dutch book detected! Margin: 0.0458 (probabilities sum to 1.0458)" + +;; Normalize to fair probabilities +(normalize-probabilities '(0.4 0.3125 0.3333)) +;; => (0.3825 0.2988 0.3187) + +;; Validated bet (ensures Dutch book safety) +(bet/validated '(10 0.3) '(20 0.4) '(30 0.3)) +---- + +==== 2. Risk-of-Ruin Protection ✅ + +*Module:* `+lib/risk-of-ruin.rkt+` + +Prevents catastrophic loss through Kelly criterion and Monte Carlo +simulation. + +*Features:* - Kelly criterion for optimal bet sizing - Risk-of-ruin +probability calculation - Stake safety validation - Wealth trajectory +simulation - Maximum drawdown analysis - Value-at-Risk (VaR) integration + +*Example:* + +[source,racket] +---- +(require "lib/risk-of-ruin.rkt") + +(define bankroll 10000) +(define win-prob 0.55) +(define odds 2.0) + +;; Calculate optimal stake (1/4 Kelly) +(optimal-stake bankroll win-prob odds 0.25) +;; => $687.50 + +;; Check if stake is safe +(safe-stake? 500 bankroll win-prob odds) +;; => #t + +;; Calculate ruin probability +(ruin-probability bankroll 500 win-prob odds 0) +;; => 0.08 (8% chance of ruin) +---- + +==== 3. Cool-Off Mechanism ✅ + +*Module:* `+lib/cool-off.rkt+` + +Time-locked betting prevention to reduce compulsive gambling. + +*Features:* - Mandatory delays between bets - Violation tracking and +alerting - Session statistics (bets per minute, duration) - Adaptive +cool-off (increases with rapid betting) - Self-exclusion support + +*Example:* + +[source,racket] +---- +(require "lib/cool-off.rkt") + +;; Create tracker with 5-second cool-off +(define tracker (make-cool-off-tracker 5 #t)) + +;; First bet succeeds +(bet/cool-off tracker (lambda (x) x) 100) + +;; Second bet blocked (cool-off active) +(bet/cool-off tracker (lambda (x) x) 100) +;; => Error: Cool-off period active. Please wait 5 seconds. + +;; Session stats +(session-stats tracker) +;; => (hash 'total-bets 1 'bets-per-minute 12 'violations 1) +---- + +==== 4. Uncertainty-Aware Number Systems ✅ + +*Module:* `+lib/number-systems.rkt+` + +14 number types for representing and computing with uncertainty. + +*Support matrix (all 14 systems implemented):* + +[width="100%",cols="20%,20%,20%,20%,20%",options="header",] +|=== +|System |Best For |Accessibility |Mathematical Rigor |Julia Integration +|DistnumberNormal |Everyday probabilistic arithmetic |High |Medium |Easy +(FFI) + +|DistnumberBeta |Bounded probabilities |High |High |Easy (FFI) + +|AffineNumber |Correlated uncertainties |Medium |High |Moderate (FFI) + +|FuzzyTriangular |Interpretability |High |Medium |Easy (FFI) + +|BayesianNumber |Learning from data |Medium |High |Moderate (FFI) + +|RiskNumber |Financial/gambling risk |High |Medium |Custom (FFI) + +|SurrealFuzzy |Infinitesimal tolerance |Medium |High |Possible (FFI) + +|PAdicProbability |Hierarchical models |Low |Very High |Hard (FFI) + +|LotteryNumber |Gambling/decision theory |High |Medium |Easy (FFI) + +|Hyperreal |Non-standard analysis |Low |Very High |Hard (FFI) + +|SurrealAdvanced |Full surreal arithmetic |Low |Very High |Hard (FFI) + +|PAdicAdvanced |Complete p-adic system |Low |Very High |Hard (FFI) + +|ImpreciseProbability |Interval-valued bounds |Medium |High |Moderate +(FFI) + +|DempsterShafer |Belief functions |Medium |High |Moderate (FFI) +|=== + +*Implemented (all 14 types):* + +===== DistnumberNormal - Gaussian Distributions + +[source,racket] +---- +(define height (make-distnumber-normal 170 10)) ;; 170cm ± 10cm +(distnumber-sample height) ;; => 178.5 +---- + +===== AffineNumber - Interval Arithmetic + +[source,racket] +---- +(define temp (make-affine-number 18 22)) ;; [18, 22]°C +(affine-contains? temp 20) ;; => #t +---- + +===== FuzzyTriangular - Fuzzy Logic + +[source,racket] +---- +(define warm (make-fuzzy-triangular 15 25 35)) +(fuzzy-membership warm 20) ;; => 0.5 +---- + +===== BayesianNumber - Bayesian Inference + +[source,racket] +---- +(define prior (make-bayesian-number 0.01)) +(bayesian-update prior 0.9 0.05) ;; Update with evidence +---- + +===== RiskNumber - VaR/CVaR + +[source,racket] +---- +(define risk (make-risk-number '(-10 -5 0 5 10) 0.95)) +(value-at-risk risk) ;; => -10 (95% VaR) +(conditional-var risk) ;; => -7.5 (Expected shortfall) +---- + +===== SurrealFuzzy - Infinitesimal tolerance + +[source,racket] +---- +(define sf (make-surreal-fuzzy 0 5 10 0.25)) +(surreal-fuzzy-membership sf 0.1) ;; => small positive membership +---- + +===== p-Adic Probability - Hierarchical digits + +[source,racket] +---- +(define pp (make-padic-probability 5 '(2 0 1))) +(padic-probability->real pp) ;; => 0.408 +---- + +===== LotteryNumber - Weighted outcomes + +[source,racket] +---- +(define ln (make-lottery-number '(0 10 20) '(1 1 2))) +(lottery-number-expected-value ln) ;; => 12.5 +---- + +===== DistnumberBeta - Beta Distributions + +[source,racket] +---- +(define prob (make-distnumber-beta 8 2)) ;; Beta(8, 2) +(distnumber-beta-mean prob) ;; => 0.8 +(distnumber-beta-mode prob) ;; => 0.875 +---- + +===== Hyperreal - Non-standard Analysis + +[source,racket] +---- +(define h (make-hyperreal 5 0.01)) ;; 5 + 0.01ε +(hyperreal-add h (make-hyperreal 3 0.02)) ;; => 8 + 0.03ε +---- + +===== SurrealAdvanced - Full Surreal Numbers + +[source,racket] +---- +(define s (make-surreal-advanced '(0 1) '(3 4))) ;; {0,1|3,4} = 2 +(surreal-add s (make-surreal-advanced '(1) '(2))) +---- + +===== PAdicAdvanced - Complete p-Adic Numbers + +[source,racket] +---- +(define p (make-padic-advanced 5 3 '(2 1 4))) ;; 5-adic number +(padic-add p (make-padic-advanced 5 3 '(3 0 1))) +---- + +===== ImpreciseProbability - Interval Bounds + +[source,racket] +---- +(define ip (make-imprecise-probability 0.3 0.7)) ;; [0.3, 0.7] +(imprecise-midpoint ip) ;; => 0.5 +(imprecise-width ip) ;; => 0.4 +---- + +===== DempsterShafer - Belief Functions + +[source,racket] +---- +(define ds (make-dempster-shafer '((a) (b) (a b)) '(0.3 0.4 0.3))) +(belief ds '(a)) ;; => 0.3 +(plausibility ds '(a)) ;; => 0.6 +---- + +=== Safety Guarantees + +==== Compile-Time Guarantees + +* ✅ No Dutch books (probabilities validated at parse time) +* ✅ Type-safe uncertainty quantification +* ⏳ Formal proofs via Idris2 ABI (planned) + +==== Runtime Guarantees + +* ✅ Risk-of-ruin protection (Monte Carlo validation) +* ✅ Cool-off enforcement (non-bypassable time locks) +* ✅ Stake limits (Kelly criterion compliance) + +==== Responsible Gambling + +* ✅ Cool-off mechanism (reduces rapid betting) +* ✅ Session statistics tracking +* ✅ Violation alerting +* ✅ Self-exclusion support +* ⏳ Integration with GamCare/GamStop (planned) + +=== Testing + +All safety features have comprehensive test suites: + +[source,bash] +---- +# Run individual module tests +racket lib/dutch-book.rkt +racket lib/risk-of-ruin.rkt +racket lib/cool-off.rkt +racket lib/number-systems.rkt + +# Run comprehensive safety demo +racket examples/safety-features.rkt +---- + +=== Academic Validation + +BetLang’s safety features are based on: + +[arabic] +. *Dutch Book Theorem* - Frank Ramsey (1926), Bruno de Finetti (1937) +. *Kelly Criterion* - John Larry Kelly Jr. (1956) +. *Risk-of-Ruin* - Gambler’s Ruin Problem, classic probability theory +. *Responsible Gambling* - GamCare, National Council on Problem Gambling + +=== Publication Targets + +* *PLDI 2027* - Programming Language Design & Implementation +* *POPL 2027* - Principles of Programming Languages +* *ICFP 2027* - International Conference on Functional Programming +* *q-fin journals* - Quantitative finance audience + +=== License + +MPL-2.0 (MPL-2.0) + +=== Authors + +Jonathan D.A. Jewell j.d.a.jewell@open.ac.uk + +=== References + +* Ramsey, F.P. (1926). "`Truth and Probability`" +* de Finetti, B. (1937). "`La Prévision: ses lois logiques, ses sources +subjectives`" +* Kelly, J.L. (1956). "`A New Interpretation of Information Rate`" +* Thorp, E.O. (1966). "`Beat the Dealer`" (Kelly criterion application) diff --git a/SAFETY-FEATURES.md b/SAFETY-FEATURES.md deleted file mode 100644 index 5543606..0000000 --- a/SAFETY-FEATURES.md +++ /dev/null @@ -1,288 +0,0 @@ - -# BetLang Safety Features - -**Status:** ✅ **IMPLEMENTED** (2026-02-01) - -BetLang is the first probabilistic programming language with comprehensive gambling harm reduction and mathematical safety guarantees. - -## Four Safety Pillars - -### 1. Dutch Book Prevention ✅ - -**Module:** `lib/dutch-book.rkt` - -Prevents arbitrage opportunities by validating that probabilities sum to 1.0 within tolerance. - -**Features:** -- Compile-time probability validation -- Dutch book detection from betting odds -- Probability normalization -- Bookmaker margin calculation -- Validated bet execution (`bet/validated`) - -**Example:** -```racket -(require "lib/dutch-book.rkt") - -;; Detect Dutch book -(define odds '(2.5 3.2 3.0)) ;; Bookmaker odds -(detect-dutch-book odds) -;; => "Dutch book detected! Margin: 0.0458 (probabilities sum to 1.0458)" - -;; Normalize to fair probabilities -(normalize-probabilities '(0.4 0.3125 0.3333)) -;; => (0.3825 0.2988 0.3187) - -;; Validated bet (ensures Dutch book safety) -(bet/validated '(10 0.3) '(20 0.4) '(30 0.3)) -``` - -### 2. Risk-of-Ruin Protection ✅ - -**Module:** `lib/risk-of-ruin.rkt` - -Prevents catastrophic loss through Kelly criterion and Monte Carlo simulation. - -**Features:** -- Kelly criterion for optimal bet sizing -- Risk-of-ruin probability calculation -- Stake safety validation -- Wealth trajectory simulation -- Maximum drawdown analysis -- Value-at-Risk (VaR) integration - -**Example:** -```racket -(require "lib/risk-of-ruin.rkt") - -(define bankroll 10000) -(define win-prob 0.55) -(define odds 2.0) - -;; Calculate optimal stake (1/4 Kelly) -(optimal-stake bankroll win-prob odds 0.25) -;; => $687.50 - -;; Check if stake is safe -(safe-stake? 500 bankroll win-prob odds) -;; => #t - -;; Calculate ruin probability -(ruin-probability bankroll 500 win-prob odds 0) -;; => 0.08 (8% chance of ruin) -``` - -### 3. Cool-Off Mechanism ✅ - -**Module:** `lib/cool-off.rkt` - -Time-locked betting prevention to reduce compulsive gambling. - -**Features:** -- Mandatory delays between bets -- Violation tracking and alerting -- Session statistics (bets per minute, duration) -- Adaptive cool-off (increases with rapid betting) -- Self-exclusion support - -**Example:** -```racket -(require "lib/cool-off.rkt") - -;; Create tracker with 5-second cool-off -(define tracker (make-cool-off-tracker 5 #t)) - -;; First bet succeeds -(bet/cool-off tracker (lambda (x) x) 100) - -;; Second bet blocked (cool-off active) -(bet/cool-off tracker (lambda (x) x) 100) -;; => Error: Cool-off period active. Please wait 5 seconds. - -;; Session stats -(session-stats tracker) -;; => (hash 'total-bets 1 'bets-per-minute 12 'violations 1) -``` - -### 4. Uncertainty-Aware Number Systems ✅ - -**Module:** `lib/number-systems.rkt` - -14 number types for representing and computing with uncertainty. - -**Support matrix (all 14 systems implemented):** - -| System | Best For | Accessibility | Mathematical Rigor | Julia Integration | -| --- | --- | --- | --- | --- | -| DistnumberNormal | Everyday probabilistic arithmetic | High | Medium | Easy (FFI) | -| DistnumberBeta | Bounded probabilities | High | High | Easy (FFI) | -| AffineNumber | Correlated uncertainties | Medium | High | Moderate (FFI) | -| FuzzyTriangular | Interpretability | High | Medium | Easy (FFI) | -| BayesianNumber | Learning from data | Medium | High | Moderate (FFI) | -| RiskNumber | Financial/gambling risk | High | Medium | Custom (FFI) | -| SurrealFuzzy | Infinitesimal tolerance | Medium | High | Possible (FFI) | -| PAdicProbability | Hierarchical models | Low | Very High | Hard (FFI) | -| LotteryNumber | Gambling/decision theory | High | Medium | Easy (FFI) | -| Hyperreal | Non-standard analysis | Low | Very High | Hard (FFI) | -| SurrealAdvanced | Full surreal arithmetic | Low | Very High | Hard (FFI) | -| PAdicAdvanced | Complete p-adic system | Low | Very High | Hard (FFI) | -| ImpreciseProbability | Interval-valued bounds | Medium | High | Moderate (FFI) | -| DempsterShafer | Belief functions | Medium | High | Moderate (FFI) | - -**Implemented (all 14 types):** - -#### DistnumberNormal - Gaussian Distributions -```racket -(define height (make-distnumber-normal 170 10)) ;; 170cm ± 10cm -(distnumber-sample height) ;; => 178.5 -``` - -#### AffineNumber - Interval Arithmetic -```racket -(define temp (make-affine-number 18 22)) ;; [18, 22]°C -(affine-contains? temp 20) ;; => #t -``` - -#### FuzzyTriangular - Fuzzy Logic -```racket -(define warm (make-fuzzy-triangular 15 25 35)) -(fuzzy-membership warm 20) ;; => 0.5 -``` - -#### BayesianNumber - Bayesian Inference -```racket -(define prior (make-bayesian-number 0.01)) -(bayesian-update prior 0.9 0.05) ;; Update with evidence -``` - -#### RiskNumber - VaR/CVaR -```racket -(define risk (make-risk-number '(-10 -5 0 5 10) 0.95)) -(value-at-risk risk) ;; => -10 (95% VaR) -(conditional-var risk) ;; => -7.5 (Expected shortfall) -``` - -#### SurrealFuzzy - Infinitesimal tolerance -```racket -(define sf (make-surreal-fuzzy 0 5 10 0.25)) -(surreal-fuzzy-membership sf 0.1) ;; => small positive membership -``` - -#### p-Adic Probability - Hierarchical digits -```racket -(define pp (make-padic-probability 5 '(2 0 1))) -(padic-probability->real pp) ;; => 0.408 -``` - -#### LotteryNumber - Weighted outcomes -```racket -(define ln (make-lottery-number '(0 10 20) '(1 1 2))) -(lottery-number-expected-value ln) ;; => 12.5 -``` - -#### DistnumberBeta - Beta Distributions -```racket -(define prob (make-distnumber-beta 8 2)) ;; Beta(8, 2) -(distnumber-beta-mean prob) ;; => 0.8 -(distnumber-beta-mode prob) ;; => 0.875 -``` - -#### Hyperreal - Non-standard Analysis -```racket -(define h (make-hyperreal 5 0.01)) ;; 5 + 0.01ε -(hyperreal-add h (make-hyperreal 3 0.02)) ;; => 8 + 0.03ε -``` - -#### SurrealAdvanced - Full Surreal Numbers -```racket -(define s (make-surreal-advanced '(0 1) '(3 4))) ;; {0,1|3,4} = 2 -(surreal-add s (make-surreal-advanced '(1) '(2))) -``` - -#### PAdicAdvanced - Complete p-Adic Numbers -```racket -(define p (make-padic-advanced 5 3 '(2 1 4))) ;; 5-adic number -(padic-add p (make-padic-advanced 5 3 '(3 0 1))) -``` - -#### ImpreciseProbability - Interval Bounds -```racket -(define ip (make-imprecise-probability 0.3 0.7)) ;; [0.3, 0.7] -(imprecise-midpoint ip) ;; => 0.5 -(imprecise-width ip) ;; => 0.4 -``` - -#### DempsterShafer - Belief Functions -```racket -(define ds (make-dempster-shafer '((a) (b) (a b)) '(0.3 0.4 0.3))) -(belief ds '(a)) ;; => 0.3 -(plausibility ds '(a)) ;; => 0.6 -``` - -## Safety Guarantees - -### Compile-Time Guarantees -- ✅ No Dutch books (probabilities validated at parse time) -- ✅ Type-safe uncertainty quantification -- ⏳ Formal proofs via Idris2 ABI (planned) - -### Runtime Guarantees -- ✅ Risk-of-ruin protection (Monte Carlo validation) -- ✅ Cool-off enforcement (non-bypassable time locks) -- ✅ Stake limits (Kelly criterion compliance) - -### Responsible Gambling -- ✅ Cool-off mechanism (reduces rapid betting) -- ✅ Session statistics tracking -- ✅ Violation alerting -- ✅ Self-exclusion support -- ⏳ Integration with GamCare/GamStop (planned) - -## Testing - -All safety features have comprehensive test suites: - -```bash -# Run individual module tests -racket lib/dutch-book.rkt -racket lib/risk-of-ruin.rkt -racket lib/cool-off.rkt -racket lib/number-systems.rkt - -# Run comprehensive safety demo -racket examples/safety-features.rkt -``` - -## Academic Validation - -BetLang's safety features are based on: - -1. **Dutch Book Theorem** - Frank Ramsey (1926), Bruno de Finetti (1937) -2. **Kelly Criterion** - John Larry Kelly Jr. (1956) -3. **Risk-of-Ruin** - Gambler's Ruin Problem, classic probability theory -4. **Responsible Gambling** - GamCare, National Council on Problem Gambling - -## Publication Targets - -- **PLDI 2027** - Programming Language Design & Implementation -- **POPL 2027** - Principles of Programming Languages -- **ICFP 2027** - International Conference on Functional Programming -- **q-fin journals** - Quantitative finance audience - -## License - -MPL-2.0 (MPL-2.0) - -## Authors - -Jonathan D.A. Jewell - -## References - -- Ramsey, F.P. (1926). "Truth and Probability" -- de Finetti, B. (1937). "La Prévision: ses lois logiques, ses sources subjectives" -- Kelly, J.L. (1956). "A New Interpretation of Information Rate" -- Thorp, E.O. (1966). "Beat the Dealer" (Kelly criterion application) diff --git a/SECURITY.adoc b/SECURITY.adoc new file mode 100644 index 0000000..6cfeaf8 --- /dev/null +++ b/SECURITY.adoc @@ -0,0 +1,226 @@ +== Security Policy + +=== Supported Versions + +We release patches for security vulnerabilities in the following +versions: + +[cols=",",options="header",] +|=== +|Version |Supported +|main |:white_check_mark: +|< 1.0 |:x: +|=== + +*Note:* betlang is currently in active development. We recommend using +the latest commit from the `+main+` branch. + +=== Security Considerations for Probabilistic Code + +betlang is a probabilistic programming DSL for modeling uncertainty. +While the language itself has a minimal attack surface, users should be +aware of these security considerations: + +==== 1. Randomness Quality + +* *Issue:* Racket’s built-in `+random+` function uses a pseudorandom +number generator (PRNG) +* *Risk:* PRNGs are *not cryptographically secure* +* *Mitigation:* +** Do NOT use betlang for cryptographic purposes +** Do NOT use betlang for security-critical random number generation +** For cryptographic randomness, use a dedicated crypto library + +==== 2. Denial of Service via Computation + +* *Issue:* Probabilistic simulations can be computationally expensive +* *Risk:* Malicious or poorly written code could cause excessive +CPU/memory usage +* *Mitigation:* +** Be cautious when running untrusted bet code +** Monitor resource usage for large simulations +** Set reasonable limits on iteration counts (e.g., `+bet-parallel+` n +parameter) + +==== 3. Floating Point Precision + +* *Issue:* Statistical calculations involve floating-point arithmetic +* *Risk:* Precision errors, rounding issues, potential for numerical +instability +* *Mitigation:* +** Validate input ranges for statistical functions +** Be aware of floating-point limitations in critical applications +** Use appropriate precision for your domain + +==== 4. No Network or Filesystem Access + +* *Good News:* betlang is *offline-first* and has no network +dependencies +* *Risk Level:* LOW - No remote code execution, no data exfiltration, no +network attacks + +==== 5. Memory Safety + +* *Good News:* Racket has automatic garbage collection +* *Risk Level:* LOW - No buffer overflows, use-after-free, or memory +corruption + +=== Reporting a Vulnerability + +We take security seriously. If you discover a security vulnerability in +betlang, please report it responsibly. + +==== How to Report + +*For security vulnerabilities, please DO NOT open a public issue.* + +Instead: + +[arabic] +. *Email:* Send details to [SECURITY_EMAIL - TO BE CONFIGURED] +* Subject: "`[SECURITY] betlang vulnerability report`" +* Include: description, steps to reproduce, potential impact +. *GitHub Security Advisories (preferred):* +* Go to the Security tab in the repository +* Click "`Report a vulnerability`" +* Fill out the advisory form + +==== What to Include + +* Description of the vulnerability +* Steps to reproduce +* Potential impact/severity +* Affected versions +* Suggested fix (if you have one) +* Your contact information (optional, for follow-up) + +==== Response Timeline + +* *24 hours:* Acknowledgment of your report +* *7 days:* Initial assessment and severity classification +* *30 days:* Fix developed and tested (for confirmed vulnerabilities) +* *60 days:* Public disclosure (after fix is released) + +==== Responsible Disclosure + +We follow these principles: + +* We will acknowledge your report within 24 hours +* We will work with you to understand the issue +* We will keep you informed of our progress +* We will credit you in the security advisory (unless you prefer +anonymity) +* We will publicly disclose the issue after a fix is available + +==== Hall of Fame + +We recognize security researchers who responsibly disclose +vulnerabilities: + +* (No reports yet - you could be first!) + +=== Security Best Practices for Users + +==== When Using betlang in Production: + +[arabic] +. *Pin Your Version:* +* Use a specific commit or tag, not `+main+` branch +* Test thoroughly before upgrading +. *Validate Inputs:* +* Sanitize user-provided parameters to bet functions +* Set reasonable bounds on iteration counts +* Validate statistical assumptions +. *Monitor Resource Usage:* +* Track CPU and memory consumption +* Set timeouts for long-running simulations +* Implement circuit breakers for computational limits +. *Review Dependencies:* +* betlang has minimal dependencies (only Racket) +* Keep Racket runtime updated +* Monitor Racket security advisories +. *Code Review:* +* Review statistical correctness of probabilistic models +* Verify randomness sources are appropriate for your use case +* Audit third-party code before integration + +=== Scope + +==== In Scope + +* Security vulnerabilities in betlang core code +* Security issues in standard library functions +* Documentation that could lead to insecure usage +* Build/deployment security issues + +==== Out of Scope + +* Issues in user-written code using betlang +* Statistical correctness (report as bugs, not security issues) +* Performance issues (report as bugs, not security issues) +* Racket language vulnerabilities (report to Racket project) + +=== Security Tooling + +==== Static Analysis + +Currently, betlang does not use automated security scanning tools. +Contributions welcome for: + +* Racket-specific static analysis integration +* Dependency vulnerability scanning +* Code quality tools + +==== Dependencies + +betlang has minimal dependencies: + +* *Racket:* Core language runtime (updated separately) +* *rackunit:* Testing framework (part of Racket distribution) + +We do not use third-party libraries, minimizing supply chain risk. + +=== Known Limitations + +[arabic] +. *No Cryptographic Use:* +* betlang’s randomness is NOT suitable for cryptography +* Use a dedicated cryptographic library instead +. *No Input Validation:* +* betlang does not sanitize user inputs by default +* Users must validate inputs in their applications +. *No Sandboxing:* +* betlang code runs with full Racket VM permissions +* Do not execute untrusted code +. *No Resource Limits:* +* No built-in limits on computation time or memory +* Users must implement their own resource controls + +=== Security Roadmap + +Future security enhancements under consideration: + +* [ ] Cryptographically secure random number generator option +* [ ] Resource limit APIs (`+bet-with-limits+`, timeouts) +* [ ] Input validation helpers +* [ ] Security audit of statistical functions +* [ ] Formal verification of core primitives +* [ ] Supply chain security (SBOM, provenance) + +=== Compliance + +betlang does not currently comply with specific security standards +(e.g., FIPS, Common Criteria). If you need compliance for a specific +standard, please open an issue to discuss. + +=== Contact + +For non-security issues: - Open an issue: link:../../issues[GitHub +Issues] - Contribute: CONTRIBUTING.md + +For security issues: - Email: [SECURITY_EMAIL - TO BE CONFIGURED] - +GitHub Security Advisories: link:../../security[Repository Security Tab] + +''''' + +*Last Updated:* 2025-11-22 *Version:* 1.0 diff --git a/SECURITY.md b/SECURITY.md deleted file mode 100644 index 31a4215..0000000 --- a/SECURITY.md +++ /dev/null @@ -1,219 +0,0 @@ - -# Security Policy - -## Supported Versions - -We release patches for security vulnerabilities in the following versions: - -| Version | Supported | -| ------- | ------------------ | -| main | :white_check_mark: | -| < 1.0 | :x: | - -**Note:** betlang is currently in active development. We recommend using the latest commit from the `main` branch. - -## Security Considerations for Probabilistic Code - -betlang is a probabilistic programming DSL for modeling uncertainty. While the language itself has a minimal attack surface, users should be aware of these security considerations: - -### 1. Randomness Quality - -- **Issue:** Racket's built-in `random` function uses a pseudorandom number generator (PRNG) -- **Risk:** PRNGs are **not cryptographically secure** -- **Mitigation:** - - Do NOT use betlang for cryptographic purposes - - Do NOT use betlang for security-critical random number generation - - For cryptographic randomness, use a dedicated crypto library - -### 2. Denial of Service via Computation - -- **Issue:** Probabilistic simulations can be computationally expensive -- **Risk:** Malicious or poorly written code could cause excessive CPU/memory usage -- **Mitigation:** - - Be cautious when running untrusted bet code - - Monitor resource usage for large simulations - - Set reasonable limits on iteration counts (e.g., `bet-parallel` n parameter) - -### 3. Floating Point Precision - -- **Issue:** Statistical calculations involve floating-point arithmetic -- **Risk:** Precision errors, rounding issues, potential for numerical instability -- **Mitigation:** - - Validate input ranges for statistical functions - - Be aware of floating-point limitations in critical applications - - Use appropriate precision for your domain - -### 4. No Network or Filesystem Access - -- **Good News:** betlang is **offline-first** and has no network dependencies -- **Risk Level:** LOW - No remote code execution, no data exfiltration, no network attacks - -### 5. Memory Safety - -- **Good News:** Racket has automatic garbage collection -- **Risk Level:** LOW - No buffer overflows, use-after-free, or memory corruption - -## Reporting a Vulnerability - -We take security seriously. If you discover a security vulnerability in betlang, please report it responsibly. - -### How to Report - -**For security vulnerabilities, please DO NOT open a public issue.** - -Instead: - -1. **Email:** Send details to [SECURITY_EMAIL - TO BE CONFIGURED] - - Subject: "[SECURITY] betlang vulnerability report" - - Include: description, steps to reproduce, potential impact - -2. **GitHub Security Advisories (preferred):** - - Go to the Security tab in the repository - - Click "Report a vulnerability" - - Fill out the advisory form - -### What to Include - -- Description of the vulnerability -- Steps to reproduce -- Potential impact/severity -- Affected versions -- Suggested fix (if you have one) -- Your contact information (optional, for follow-up) - -### Response Timeline - -- **24 hours:** Acknowledgment of your report -- **7 days:** Initial assessment and severity classification -- **30 days:** Fix developed and tested (for confirmed vulnerabilities) -- **60 days:** Public disclosure (after fix is released) - -### Responsible Disclosure - -We follow these principles: - -- We will acknowledge your report within 24 hours -- We will work with you to understand the issue -- We will keep you informed of our progress -- We will credit you in the security advisory (unless you prefer anonymity) -- We will publicly disclose the issue after a fix is available - -### Hall of Fame - -We recognize security researchers who responsibly disclose vulnerabilities: - -- (No reports yet - you could be first!) - -## Security Best Practices for Users - -### When Using betlang in Production: - -1. **Pin Your Version:** - - Use a specific commit or tag, not `main` branch - - Test thoroughly before upgrading - -2. **Validate Inputs:** - - Sanitize user-provided parameters to bet functions - - Set reasonable bounds on iteration counts - - Validate statistical assumptions - -3. **Monitor Resource Usage:** - - Track CPU and memory consumption - - Set timeouts for long-running simulations - - Implement circuit breakers for computational limits - -4. **Review Dependencies:** - - betlang has minimal dependencies (only Racket) - - Keep Racket runtime updated - - Monitor Racket security advisories - -5. **Code Review:** - - Review statistical correctness of probabilistic models - - Verify randomness sources are appropriate for your use case - - Audit third-party code before integration - -## Scope - -### In Scope - -- Security vulnerabilities in betlang core code -- Security issues in standard library functions -- Documentation that could lead to insecure usage -- Build/deployment security issues - -### Out of Scope - -- Issues in user-written code using betlang -- Statistical correctness (report as bugs, not security issues) -- Performance issues (report as bugs, not security issues) -- Racket language vulnerabilities (report to Racket project) - -## Security Tooling - -### Static Analysis - -Currently, betlang does not use automated security scanning tools. Contributions welcome for: - -- Racket-specific static analysis integration -- Dependency vulnerability scanning -- Code quality tools - -### Dependencies - -betlang has minimal dependencies: - -- **Racket:** Core language runtime (updated separately) -- **rackunit:** Testing framework (part of Racket distribution) - -We do not use third-party libraries, minimizing supply chain risk. - -## Known Limitations - -1. **No Cryptographic Use:** - - betlang's randomness is NOT suitable for cryptography - - Use a dedicated cryptographic library instead - -2. **No Input Validation:** - - betlang does not sanitize user inputs by default - - Users must validate inputs in their applications - -3. **No Sandboxing:** - - betlang code runs with full Racket VM permissions - - Do not execute untrusted code - -4. **No Resource Limits:** - - No built-in limits on computation time or memory - - Users must implement their own resource controls - -## Security Roadmap - -Future security enhancements under consideration: - -- [ ] Cryptographically secure random number generator option -- [ ] Resource limit APIs (`bet-with-limits`, timeouts) -- [ ] Input validation helpers -- [ ] Security audit of statistical functions -- [ ] Formal verification of core primitives -- [ ] Supply chain security (SBOM, provenance) - -## Compliance - -betlang does not currently comply with specific security standards (e.g., FIPS, Common Criteria). If you need compliance for a specific standard, please open an issue to discuss. - -## Contact - -For non-security issues: -- Open an issue: [GitHub Issues](../../issues) -- Contribute: [CONTRIBUTING.md](CONTRIBUTING.md) - -For security issues: -- Email: [SECURITY_EMAIL - TO BE CONFIGURED] -- GitHub Security Advisories: [Repository Security Tab](../../security) - ---- - -**Last Updated:** 2025-11-22 -**Version:** 1.0 diff --git a/TOOLING.adoc b/TOOLING.adoc new file mode 100644 index 0000000..32e54d7 --- /dev/null +++ b/TOOLING.adoc @@ -0,0 +1,92 @@ +== Betlang Optional Tooling + +This document identifies optional tooling components that are NOT part +of the authoritative betlang implementation. + +=== Authoritative Implementation + +The *Racket implementation* in `+core/betlang.rkt+` is the sole +authoritative reference implementation. See `+SPEC.core.scm+` for the +formal semantics. + +*Authoritative components:* - `+core/betlang.rkt+` - Core DSL primitives +- `+lib/*.rkt+` - Standard library modules - `+repl/shell.rkt+` - +Interactive REPL - `+tests/basics.rkt+` - Test suite - `+conformance/+` +- Conformance test corpus - `+SPEC.core.scm+` - Formal specification + +=== Optional Tooling (Non-Authoritative) + +The following components are *optional tooling* that enhances the +developer experience but *does not define or affect betlang semantics*: + +==== Rust/Cargo Components + +[cols=",,",options="header",] +|=== +|Directory |Purpose |Status +|`+compiler/bet-syntax/+` |AST definitions |Optional tooling +|`+compiler/bet-parse/+` |Parser |Optional tooling +|`+compiler/bet-core/+` |Core types |Optional tooling +|`+compiler/bet-check/+` |Type checking |Optional tooling +|`+compiler/bet-eval/+` |Evaluator |Optional tooling +|`+compiler/bet-codegen/+` |Code generation |Optional tooling +|`+runtime/bet-rt/+` |Runtime |Optional tooling +|`+runtime/bet-rand/+` |Random number generation |Optional tooling +|`+runtime/bet-viz/+` |Visualization |Optional tooling +|`+tools/bet-cli/+` |CLI tool |Optional tooling +|=== + +*Note:* These Rust components may be developed as alternative +implementations or performance-optimized tooling, but they are NOT +authoritative. Any discrepancy between Rust behavior and Racket behavior +means the Racket implementation is correct. + +==== Language Bindings + +[cols=",,",options="header",] +|=== +|Directory |Purpose |Status +|`+bindings/chapel/+` |Chapel language bindings |Optional tooling +|`+bindings/julia/+` |Julia language bindings |Optional tooling +|=== + +*Note:* These bindings MUST conform to SPEC.core.scm semantics. They are +convenience wrappers, not authoritative implementations. + +==== Container Definitions + +[width="100%",cols="27%,39%,34%",options="header",] +|=== +|File |Purpose |Status +|`+containers/Containerfile+` |Production container |Optional tooling + +|`+containers/Containerfile.dev+` |Development container |Optional +tooling +|=== + +=== Semantic Authority Hierarchy + +[arabic] +. *SPEC.core.scm* - Formal specification (normative) +. *core/betlang.rkt* - Reference implementation (authoritative) +. *conformance/* - Conformance tests (verification) +. *Everything else* - Optional tooling (non-authoritative) + +=== Guidelines for Optional Tooling + +When developing optional tooling: + +[arabic] +. *MUST NOT* modify betlang semantics +. *MUST* pass all conformance tests when applicable +. *MUST* defer to Racket implementation on ambiguities +. *SHOULD* document any intentional behavioral differences +. *SHOULD* clearly mark as "`optional`" in documentation + +=== Forbidden in f0 (First Phase) + +Per the anchor document: - Replacing Racket as the authoritative +implementation - Adding Julia or other backends as primary +implementations - Unseeded randomness in tests + +These restrictions may be relaxed in future phases (f1+). diff --git a/TOOLING.md b/TOOLING.md deleted file mode 100644 index 47d3d4e..0000000 --- a/TOOLING.md +++ /dev/null @@ -1,89 +0,0 @@ - -# Betlang Optional Tooling - -This document identifies optional tooling components that are NOT part of the -authoritative betlang implementation. - -## Authoritative Implementation - -The **Racket implementation** in `core/betlang.rkt` is the sole authoritative -reference implementation. See `SPEC.core.scm` for the formal semantics. - -**Authoritative components:** -- `core/betlang.rkt` - Core DSL primitives -- `lib/*.rkt` - Standard library modules -- `repl/shell.rkt` - Interactive REPL -- `tests/basics.rkt` - Test suite -- `conformance/` - Conformance test corpus -- `SPEC.core.scm` - Formal specification - -## Optional Tooling (Non-Authoritative) - -The following components are **optional tooling** that enhances the developer -experience but **does not define or affect betlang semantics**: - -### Rust/Cargo Components - -| Directory | Purpose | Status | -|-----------|---------|--------| -| `compiler/bet-syntax/` | AST definitions | Optional tooling | -| `compiler/bet-parse/` | Parser | Optional tooling | -| `compiler/bet-core/` | Core types | Optional tooling | -| `compiler/bet-check/` | Type checking | Optional tooling | -| `compiler/bet-eval/` | Evaluator | Optional tooling | -| `compiler/bet-codegen/` | Code generation | Optional tooling | -| `runtime/bet-rt/` | Runtime | Optional tooling | -| `runtime/bet-rand/` | Random number generation | Optional tooling | -| `runtime/bet-viz/` | Visualization | Optional tooling | -| `tools/bet-cli/` | CLI tool | Optional tooling | - -**Note:** These Rust components may be developed as alternative implementations -or performance-optimized tooling, but they are NOT authoritative. Any -discrepancy between Rust behavior and Racket behavior means the Racket -implementation is correct. - -### Language Bindings - -| Directory | Purpose | Status | -|-----------|---------|--------| -| `bindings/chapel/` | Chapel language bindings | Optional tooling | -| `bindings/julia/` | Julia language bindings | Optional tooling | - -**Note:** These bindings MUST conform to SPEC.core.scm semantics. They are -convenience wrappers, not authoritative implementations. - -### Container Definitions - -| File | Purpose | Status | -|------|---------|--------| -| `containers/Containerfile` | Production container | Optional tooling | -| `containers/Containerfile.dev` | Development container | Optional tooling | - -## Semantic Authority Hierarchy - -1. **SPEC.core.scm** - Formal specification (normative) -2. **core/betlang.rkt** - Reference implementation (authoritative) -3. **conformance/** - Conformance tests (verification) -4. **Everything else** - Optional tooling (non-authoritative) - -## Guidelines for Optional Tooling - -When developing optional tooling: - -1. **MUST NOT** modify betlang semantics -2. **MUST** pass all conformance tests when applicable -3. **MUST** defer to Racket implementation on ambiguities -4. **SHOULD** document any intentional behavioral differences -5. **SHOULD** clearly mark as "optional" in documentation - -## Forbidden in f0 (First Phase) - -Per the anchor document: -- Replacing Racket as the authoritative implementation -- Adding Julia or other backends as primary implementations -- Unseeded randomness in tests - -These restrictions may be relaxed in future phases (f1+). diff --git a/TPCF.adoc b/TPCF.adoc new file mode 100644 index 0000000..038340b --- /dev/null +++ b/TPCF.adoc @@ -0,0 +1,351 @@ +== Tri-Perimeter Contribution Framework (TPCF) + +=== Introduction + +betlang adopts the *Tri-Perimeter Contribution Framework (TPCF)* to +define clear boundaries for different types of contributions. This +framework ensures that: + +* *Core stability* is maintained +* *Community innovation* is encouraged +* *Contribution paths* are clear and welcoming + +=== The Three Perimeters + +==== Perimeter 1: Inner Sanctum (Core Language) + +*Purpose:* Preserve language semantics and ternary philosophy + +*Scope:* - `+core/betlang.rkt+` - Core bet primitives - Ternary +abstraction (A-B-C structure) - Fundamental language semantics + +*Contribution Policy:* - ✅ *Bug fixes*: Always welcome - ⚠️ *New +primitives*: Requires RFC and maintainer approval - ❌ *Breaking +changes*: Extremely rare, require community consensus - ❌ *Non-ternary +primitives*: Not acceptable + +*Rationale:* The core `+bet+` primitive defines betlang’s identity. +Changes here affect all users and must be carefully considered. + +*Examples:* + +✅ *Acceptable:* + +[source,racket] +---- +;; Bug fix: Fix edge case in bet/weighted +(define (bet/weighted . weighted-choices) + (unless (= (length weighted-choices) 3) ;; Add validation + (error 'bet/weighted "expected exactly 3 weighted choices")) + ...) +---- + +❌ *Not Acceptable:* + +[source,racket] +---- +;; Adding binary or quaternary primitives +(define (binary-bet a b) ...) ;; Breaks ternary philosophy +(define (quad-bet a b c d) ...) ;; Not ternary +---- + +*Contribution Process:* 1. Open an issue with `+[RFC: Core]+` tag 2. +Community discussion (minimum 2 weeks) 3. Maintainer review and decision +4. If approved: PR with comprehensive tests 5. Documentation update +required + +*Approval Requirement:* Unanimous maintainer approval + +''''' + +==== Perimeter 2: Middle Ring (Standard Library) + +*Purpose:* Provide high-quality, well-tested utilities + +*Scope:* - `+lib/*.rkt+` - All library modules - `+tools/*.rkt+` - +Analysis and utility tools - `+repl/shell.rkt+` - REPL implementation + +*Contribution Policy:* - ✅ *New functions*: Welcome with tests and docs +- ✅ *Optimizations*: Welcome with benchmarks - ✅ *Bug fixes*: Always +welcome - ⚠️ *Breaking API changes*: Require deprecation period - ✅ +*New libraries*: Welcome with justification + +*Rationale:* Libraries extend betlang’s capabilities without changing +core semantics. We encourage innovation here while maintaining quality +standards. + +*Examples:* + +✅ *Acceptable:* + +[source,racket] +---- +;; New distribution in lib/distributions.rkt +(define (generalized-pareto alpha beta) + "Generalized Pareto distribution" + ...) + +;; New statistical test +(define (anderson-darling-test samples) + "Anderson-Darling normality test" + ...) + +;; Performance optimization +(define (fast-mean samples) + "Optimized mean calculation" + (/ (apply + samples) (length samples))) +---- + +⚠️ *Requires Discussion:* + +[source,racket] +---- +;; Changing existing API (needs deprecation) +(define (mean samples #:weighted weights) ;; Adding parameter + ...) +---- + +*Contribution Process:* 1. Open an issue or skip to PR for small +additions 2. Submit PR with: - Implementation - Tests (>70% coverage +preferred) - Documentation (docstrings + API reference update) - Example +usage 3. Maintainer review (1-2 weeks) 4. Address feedback 5. Merge + +*Approval Requirement:* One maintainer approval + +*Quality Standards:* - All exported functions must have docstrings - +Tests required for non-trivial functions - No breaking changes without +deprecation - Follow Racket style guide + +''''' + +==== Perimeter 3: Outer Circle (Examples & Community) + +*Purpose:* Share knowledge, explore applications, teach concepts + +*Scope:* - `+examples/*.rkt+` - Example programs - `+benchmarks/*.rkt+` +- Performance tests - `+docs/*.md+` - Documentation - `+homepage/+` - +Website assets - Community contributions (blog posts, papers, talks) + +*Contribution Policy:* - ✅ *New examples*: Always welcome - ✅ +*Documentation improvements*: Highly encouraged - ✅ *Benchmarks*: +Welcome - ✅ *Tutorials*: Very welcome - ✅ *Translations*: Welcome - ✅ +*Bug fixes*: Always welcome + +*Rationale:* The outer circle is where creativity flourishes. We want +this to be an open, welcoming space for all skill levels. + +*Examples:* + +✅ *Highly Encouraged:* + +[source,racket] +---- +;; New domain-specific example +;; examples/epidemiology.rkt +(require "../core/betlang.rkt") +(require "../lib/distributions.rkt") + +;; SIR model simulation with betlang +(define (sir-model population infected ...) + ...) +---- + +[source,markdown] +---- + +# Tutorial: Using betlang for A/B Testing + +Learn how to model A/B tests with probabilistic programming... +---- + +*Contribution Process:* 1. Submit PR directly (no issue needed) 2. Brief +review for quality and accuracy 3. Quick merge (usually <3 days) + +*Approval Requirement:* Any contributor or maintainer can merge + +*Quality Standards:* - Examples should run without errors - Code should +be readable and well-commented - Documentation should be clear and +accurate - No offensive or inappropriate content + +''''' + +=== Perimeter Matrix + +[cols=",,,",options="header",] +|=== +|Aspect |Inner (Core) |Middle (Library) |Outer (Examples) +|*Scope* |Core primitives |Standard library |Examples, docs +|*Stability* |Very High |High |Medium +|*Innovation* |Low |Medium |High +|*Approval* |Unanimous |1 maintainer |Any contributor +|*RFC Required* |Yes |Sometimes |No +|*Tests Required* |Comprehensive |Yes |Recommended +|*Docs Required* |Yes |Yes |Helpful +|*Breaking Changes* |Rare |Deprecated |Allowed +|*Contribution Speed* |Slow |Medium |Fast +|=== + +=== Special Cases + +==== Cross-Perimeter Changes + +Some contributions affect multiple perimeters: + +*Example:* Adding a new bet primitive + library functions - Core +primitive → Inner perimeter process - Helper functions → Middle +perimeter process - Examples → Outer perimeter process + +*Process:* 1. Follow Inner perimeter process for core change 2. Once +core is approved, add library functions (Middle process) 3. Add examples +freely (Outer process) + +==== Security Fixes + +Security vulnerabilities bypass normal processes: - *All perimeters:* +Immediate fix, coordinate with SECURITY.md - *Disclosure:* Follow +responsible disclosure timeline - *Process:* Fast-track review and merge + +==== Documentation-Only Changes + +Pure documentation changes (typos, clarifications): - *All perimeters:* +Fast-track approval - *Process:* Direct PR, quick merge + +=== Contribution Levels + +==== Beginner-Friendly (Outer Circle) + +* Fix typos in documentation +* Add code comments +* Create new examples +* Improve tutorials +* Translate documentation + +==== Intermediate (Middle Ring) + +* Add new library functions +* Optimize existing code +* Expand test coverage +* Create new tools +* Write comprehensive examples + +==== Advanced (Inner Sanctum) + +* Propose new core primitives +* Refactor core architecture +* Design new language features +* Formal verification +* Performance optimization of core + +=== Migration Between Perimeters + +Code can graduate from outer to middle to inner: + +[arabic] +. *Outer → Middle:* +* Example becomes useful enough for library inclusion +* Extract, test, document, move to `+lib/+` +. *Middle → Inner:* +* Rarely needed +* Only if functionality is truly fundamental +* Requires community consensus + +=== Contribution Rights + +==== All Contributors Have: + +* Right to propose ideas in any perimeter +* Right to fair review of contributions +* Right to appeal decisions +* Right to fork under CC0 license + +==== Maintainers Commit To: + +* Timely review of contributions +* Clear feedback on changes needed +* Transparent decision-making +* Respecting contributor time + +=== Frequently Asked Questions + +*Q: I want to add a new probability distribution. Which perimeter?* *A:* +Middle ring (`+lib/distributions.rkt+`). Submit PR with tests and docs. + +*Q: Can I add a binary bet primitive?* *A:* No. Core is ternary-only. +However, you can build binary on top: + +[source,racket] +---- +(define (binary-bet a b) (bet a b a)) ;; In user code, not core +---- + +*Q: I found a typo in the tutorial.* *A:* Outer circle. Fix it and +submit a PR directly! + +*Q: I want to change how `+bet+` works fundamentally.* *A:* Inner +sanctum. Open an [RFC] issue first. Expect long discussion. + +*Q: Can I add a new example file?* *A:* Outer circle. Yes! Add it to +`+examples/+` and submit a PR. + +*Q: I want to add type annotations.* *A:* Middle ring for libraries, +Inner for core. Open an issue to discuss approach first. + +=== Versioning and Compatibility + +==== Inner Perimeter (Core) + +* Changes trigger *MAJOR* version bump +* Backward compatibility is critical +* Breaking changes require 6+ months deprecation + +==== Middle Perimeter (Library) + +* New functions trigger *MINOR* version bump +* Breaking changes trigger *MAJOR* version bump +* Deprecation period: 3 months minimum + +==== Outer Perimeter (Examples) + +* Changes don’t affect version number +* Can change freely +* Keep aligned with current API + +=== Governance + +*Decision Authority:* - *Inner:* All maintainers (consensus) - *Middle:* +Any maintainer (1 approval needed) - *Outer:* Any contributor +(self-merge after 24 hours) + +*Appeals Process:* 1. Contributor can appeal to all maintainers 2. +Discussion in public issue 3. Final decision by lead maintainer if no +consensus + +=== Evolution of This Framework + +This TPCF declaration is itself subject to the framework: + +* *Framework changes:* Middle perimeter process +* *Open to discussion:* Community input welcome +* *Versioned:* Track changes in CHANGELOG.md + +=== Contact + +Questions about TPCF: - Open an issue: [Questions about contribution +perimeters] - Email: maintainers@betlang.org - See: CONTRIBUTING.md for +detailed guidelines + +''''' + +*TPCF Version:* 1.0 *Last Updated:* 2025-11-22 *Applies to:* betlang +v0.1.0+ + +''''' + +=== Summary + +*Inner Perimeter:* Core language - stable, carefully guarded *Middle +Perimeter:* Standard library - quality-focused, innovation-friendly +*Outer Perimeter:* Examples & docs - open, welcoming, fast iteration + +All contributions valued. Different perimeters = different processes. +Choose your contribution level, and we’ll guide you through! 🎲 diff --git a/TPCF.md b/TPCF.md deleted file mode 100644 index fd9febd..0000000 --- a/TPCF.md +++ /dev/null @@ -1,363 +0,0 @@ - -# Tri-Perimeter Contribution Framework (TPCF) - -## Introduction - -betlang adopts the **Tri-Perimeter Contribution Framework (TPCF)** to define clear boundaries for different types of contributions. This framework ensures that: - -- **Core stability** is maintained -- **Community innovation** is encouraged -- **Contribution paths** are clear and welcoming - -## The Three Perimeters - -### Perimeter 1: Inner Sanctum (Core Language) - -**Purpose:** Preserve language semantics and ternary philosophy - -**Scope:** -- `core/betlang.rkt` - Core bet primitives -- Ternary abstraction (A-B-C structure) -- Fundamental language semantics - -**Contribution Policy:** -- ✅ **Bug fixes**: Always welcome -- ⚠️ **New primitives**: Requires RFC and maintainer approval -- ❌ **Breaking changes**: Extremely rare, require community consensus -- ❌ **Non-ternary primitives**: Not acceptable - -**Rationale:** -The core `bet` primitive defines betlang's identity. Changes here affect all users and must be carefully considered. - -**Examples:** - -✅ **Acceptable:** -```racket -;; Bug fix: Fix edge case in bet/weighted -(define (bet/weighted . weighted-choices) - (unless (= (length weighted-choices) 3) ;; Add validation - (error 'bet/weighted "expected exactly 3 weighted choices")) - ...) -``` - -❌ **Not Acceptable:** -```racket -;; Adding binary or quaternary primitives -(define (binary-bet a b) ...) ;; Breaks ternary philosophy -(define (quad-bet a b c d) ...) ;; Not ternary -``` - -**Contribution Process:** -1. Open an issue with `[RFC: Core]` tag -2. Community discussion (minimum 2 weeks) -3. Maintainer review and decision -4. If approved: PR with comprehensive tests -5. Documentation update required - -**Approval Requirement:** Unanimous maintainer approval - ---- - -### Perimeter 2: Middle Ring (Standard Library) - -**Purpose:** Provide high-quality, well-tested utilities - -**Scope:** -- `lib/*.rkt` - All library modules -- `tools/*.rkt` - Analysis and utility tools -- `repl/shell.rkt` - REPL implementation - -**Contribution Policy:** -- ✅ **New functions**: Welcome with tests and docs -- ✅ **Optimizations**: Welcome with benchmarks -- ✅ **Bug fixes**: Always welcome -- ⚠️ **Breaking API changes**: Require deprecation period -- ✅ **New libraries**: Welcome with justification - -**Rationale:** -Libraries extend betlang's capabilities without changing core semantics. We encourage innovation here while maintaining quality standards. - -**Examples:** - -✅ **Acceptable:** -```racket -;; New distribution in lib/distributions.rkt -(define (generalized-pareto alpha beta) - "Generalized Pareto distribution" - ...) - -;; New statistical test -(define (anderson-darling-test samples) - "Anderson-Darling normality test" - ...) - -;; Performance optimization -(define (fast-mean samples) - "Optimized mean calculation" - (/ (apply + samples) (length samples))) -``` - -⚠️ **Requires Discussion:** -```racket -;; Changing existing API (needs deprecation) -(define (mean samples #:weighted weights) ;; Adding parameter - ...) -``` - -**Contribution Process:** -1. Open an issue or skip to PR for small additions -2. Submit PR with: - - Implementation - - Tests (>70% coverage preferred) - - Documentation (docstrings + API reference update) - - Example usage -3. Maintainer review (1-2 weeks) -4. Address feedback -5. Merge - -**Approval Requirement:** One maintainer approval - -**Quality Standards:** -- All exported functions must have docstrings -- Tests required for non-trivial functions -- No breaking changes without deprecation -- Follow Racket style guide - ---- - -### Perimeter 3: Outer Circle (Examples & Community) - -**Purpose:** Share knowledge, explore applications, teach concepts - -**Scope:** -- `examples/*.rkt` - Example programs -- `benchmarks/*.rkt` - Performance tests -- `docs/*.md` - Documentation -- `homepage/` - Website assets -- Community contributions (blog posts, papers, talks) - -**Contribution Policy:** -- ✅ **New examples**: Always welcome -- ✅ **Documentation improvements**: Highly encouraged -- ✅ **Benchmarks**: Welcome -- ✅ **Tutorials**: Very welcome -- ✅ **Translations**: Welcome -- ✅ **Bug fixes**: Always welcome - -**Rationale:** -The outer circle is where creativity flourishes. We want this to be an open, welcoming space for all skill levels. - -**Examples:** - -✅ **Highly Encouraged:** -```racket -;; New domain-specific example -;; examples/epidemiology.rkt -(require "../core/betlang.rkt") -(require "../lib/distributions.rkt") - -;; SIR model simulation with betlang -(define (sir-model population infected ...) - ...) -``` - -```markdown - -# Tutorial: Using betlang for A/B Testing - -Learn how to model A/B tests with probabilistic programming... -``` - -**Contribution Process:** -1. Submit PR directly (no issue needed) -2. Brief review for quality and accuracy -3. Quick merge (usually <3 days) - -**Approval Requirement:** Any contributor or maintainer can merge - -**Quality Standards:** -- Examples should run without errors -- Code should be readable and well-commented -- Documentation should be clear and accurate -- No offensive or inappropriate content - ---- - -## Perimeter Matrix - -| Aspect | Inner (Core) | Middle (Library) | Outer (Examples) | -|--------|-------------|------------------|------------------| -| **Scope** | Core primitives | Standard library | Examples, docs | -| **Stability** | Very High | High | Medium | -| **Innovation** | Low | Medium | High | -| **Approval** | Unanimous | 1 maintainer | Any contributor | -| **RFC Required** | Yes | Sometimes | No | -| **Tests Required** | Comprehensive | Yes | Recommended | -| **Docs Required** | Yes | Yes | Helpful | -| **Breaking Changes** | Rare | Deprecated | Allowed | -| **Contribution Speed** | Slow | Medium | Fast | - -## Special Cases - -### Cross-Perimeter Changes - -Some contributions affect multiple perimeters: - -**Example:** Adding a new bet primitive + library functions -- Core primitive → Inner perimeter process -- Helper functions → Middle perimeter process -- Examples → Outer perimeter process - -**Process:** -1. Follow Inner perimeter process for core change -2. Once core is approved, add library functions (Middle process) -3. Add examples freely (Outer process) - -### Security Fixes - -Security vulnerabilities bypass normal processes: -- **All perimeters:** Immediate fix, coordinate with SECURITY.md -- **Disclosure:** Follow responsible disclosure timeline -- **Process:** Fast-track review and merge - -### Documentation-Only Changes - -Pure documentation changes (typos, clarifications): -- **All perimeters:** Fast-track approval -- **Process:** Direct PR, quick merge - -## Contribution Levels - -### Beginner-Friendly (Outer Circle) -- Fix typos in documentation -- Add code comments -- Create new examples -- Improve tutorials -- Translate documentation - -### Intermediate (Middle Ring) -- Add new library functions -- Optimize existing code -- Expand test coverage -- Create new tools -- Write comprehensive examples - -### Advanced (Inner Sanctum) -- Propose new core primitives -- Refactor core architecture -- Design new language features -- Formal verification -- Performance optimization of core - -## Migration Between Perimeters - -Code can graduate from outer to middle to inner: - -1. **Outer → Middle:** - - Example becomes useful enough for library inclusion - - Extract, test, document, move to `lib/` - -2. **Middle → Inner:** - - Rarely needed - - Only if functionality is truly fundamental - - Requires community consensus - -## Contribution Rights - -### All Contributors Have: -- Right to propose ideas in any perimeter -- Right to fair review of contributions -- Right to appeal decisions -- Right to fork under CC0 license - -### Maintainers Commit To: -- Timely review of contributions -- Clear feedback on changes needed -- Transparent decision-making -- Respecting contributor time - -## Frequently Asked Questions - -**Q: I want to add a new probability distribution. Which perimeter?** -**A:** Middle ring (`lib/distributions.rkt`). Submit PR with tests and docs. - -**Q: Can I add a binary bet primitive?** -**A:** No. Core is ternary-only. However, you can build binary on top: -```racket -(define (binary-bet a b) (bet a b a)) ;; In user code, not core -``` - -**Q: I found a typo in the tutorial.** -**A:** Outer circle. Fix it and submit a PR directly! - -**Q: I want to change how `bet` works fundamentally.** -**A:** Inner sanctum. Open an [RFC] issue first. Expect long discussion. - -**Q: Can I add a new example file?** -**A:** Outer circle. Yes! Add it to `examples/` and submit a PR. - -**Q: I want to add type annotations.** -**A:** Middle ring for libraries, Inner for core. Open an issue to discuss approach first. - -## Versioning and Compatibility - -### Inner Perimeter (Core) -- Changes trigger **MAJOR** version bump -- Backward compatibility is critical -- Breaking changes require 6+ months deprecation - -### Middle Perimeter (Library) -- New functions trigger **MINOR** version bump -- Breaking changes trigger **MAJOR** version bump -- Deprecation period: 3 months minimum - -### Outer Perimeter (Examples) -- Changes don't affect version number -- Can change freely -- Keep aligned with current API - -## Governance - -**Decision Authority:** -- **Inner:** All maintainers (consensus) -- **Middle:** Any maintainer (1 approval needed) -- **Outer:** Any contributor (self-merge after 24 hours) - -**Appeals Process:** -1. Contributor can appeal to all maintainers -2. Discussion in public issue -3. Final decision by lead maintainer if no consensus - -## Evolution of This Framework - -This TPCF declaration is itself subject to the framework: - -- **Framework changes:** Middle perimeter process -- **Open to discussion:** Community input welcome -- **Versioned:** Track changes in CHANGELOG.md - -## Contact - -Questions about TPCF: -- Open an issue: [Questions about contribution perimeters] -- Email: maintainers@betlang.org -- See: CONTRIBUTING.md for detailed guidelines - ---- - -**TPCF Version:** 1.0 -**Last Updated:** 2025-11-22 -**Applies to:** betlang v0.1.0+ - ---- - -## Summary - -**Inner Perimeter:** Core language - stable, carefully guarded -**Middle Perimeter:** Standard library - quality-focused, innovation-friendly -**Outer Perimeter:** Examples & docs - open, welcoming, fast iteration - -All contributions valued. Different perimeters = different processes. Choose your contribution level, and we'll guide you through! 🎲 diff --git a/conformance/README.adoc b/conformance/README.adoc new file mode 100644 index 0000000..403ee9e --- /dev/null +++ b/conformance/README.adoc @@ -0,0 +1,83 @@ +== Betlang Conformance Test Corpus + +This directory contains the canonical conformance test suite for +betlang. Any implementation claiming conformance to betlang semantics +MUST pass all tests in this corpus. + +=== Test Files + +==== smoke.bet + +Basic smoke test for REPL verification. + +[source,bash] +---- +racket repl/shell.rkt < conformance/smoke.bet +---- + +*Success criteria:* - Program runs without errors - Output is produced +for each expression - REPL exits cleanly on `+:quit+` + +==== deterministic.rkt + +Tests for deterministic behavior that must be identical across +implementations. + +[source,bash] +---- +racket conformance/deterministic.rkt +---- + +*Covers:* - Idempotent bets: `+(bet X X X) = X+` - Conditional +true-branch: `+(bet/conditional #t A B C) = A+` - `+all-bets+` utility - +Seeded bet reproducibility - Lazy evaluation (single thunk execution) - +Bet chain and repeat operations - Entropy calculation edge cases + +==== stochastic-seeded.rkt + +Statistical tests using seeded randomness for reproducibility. + +[source,bash] +---- +racket conformance/stochastic-seeded.rkt +---- + +*Covers:* - Uniform distribution verification - Weighted distribution +verification - Probability estimation accuracy - Expected value +calculation - Composed function distribution - Entropy convergence to +theoretical values - Full reproducibility across runs + +=== Running All Tests + +[source,bash] +---- +# Smoke test +racket repl/shell.rkt < conformance/smoke.bet + +# Deterministic tests +racket conformance/deterministic.rkt + +# Stochastic tests +racket conformance/stochastic-seeded.rkt +---- + +=== Conformance Requirements + +Per SPEC.core.scm, implementations MUST: + +[arabic] +. *Seedability*: All stochastic tests use `+bet-with-seed+` for +reproducibility +. *Determinism*: Identical seeds produce identical results +. *Distribution accuracy*: Statistical properties match formal semantics +. *Error handling*: Invalid inputs produce deterministic diagnostics + +=== Adding New Tests + +When adding conformance tests: + +[arabic] +. Use `+bet-with-seed+` for ALL stochastic operations +. Document expected output/behavior +. Include in appropriate file (deterministic vs stochastic) +. Verify test passes on reference implementation (Racket) diff --git a/conformance/README.md b/conformance/README.md deleted file mode 100644 index ae4227b..0000000 --- a/conformance/README.md +++ /dev/null @@ -1,86 +0,0 @@ - -# Betlang Conformance Test Corpus - -This directory contains the canonical conformance test suite for betlang. -Any implementation claiming conformance to betlang semantics MUST pass all -tests in this corpus. - -## Test Files - -### smoke.bet -Basic smoke test for REPL verification. - -```bash -racket repl/shell.rkt < conformance/smoke.bet -``` - -**Success criteria:** -- Program runs without errors -- Output is produced for each expression -- REPL exits cleanly on `:quit` - -### deterministic.rkt -Tests for deterministic behavior that must be identical across implementations. - -```bash -racket conformance/deterministic.rkt -``` - -**Covers:** -- Idempotent bets: `(bet X X X) = X` -- Conditional true-branch: `(bet/conditional #t A B C) = A` -- `all-bets` utility -- Seeded bet reproducibility -- Lazy evaluation (single thunk execution) -- Bet chain and repeat operations -- Entropy calculation edge cases - -### stochastic-seeded.rkt -Statistical tests using seeded randomness for reproducibility. - -```bash -racket conformance/stochastic-seeded.rkt -``` - -**Covers:** -- Uniform distribution verification -- Weighted distribution verification -- Probability estimation accuracy -- Expected value calculation -- Composed function distribution -- Entropy convergence to theoretical values -- Full reproducibility across runs - -## Running All Tests - -```bash -# Smoke test -racket repl/shell.rkt < conformance/smoke.bet - -# Deterministic tests -racket conformance/deterministic.rkt - -# Stochastic tests -racket conformance/stochastic-seeded.rkt -``` - -## Conformance Requirements - -Per SPEC.core.scm, implementations MUST: - -1. **Seedability**: All stochastic tests use `bet-with-seed` for reproducibility -2. **Determinism**: Identical seeds produce identical results -3. **Distribution accuracy**: Statistical properties match formal semantics -4. **Error handling**: Invalid inputs produce deterministic diagnostics - -## Adding New Tests - -When adding conformance tests: - -1. Use `bet-with-seed` for ALL stochastic operations -2. Document expected output/behavior -3. Include in appropriate file (deterministic vs stochastic) -4. Verify test passes on reference implementation (Racket) diff --git a/docs/BETLANG-COMPARISON.md b/docs/BETLANG-COMPARISON.adoc similarity index 96% rename from docs/BETLANG-COMPARISON.md rename to docs/BETLANG-COMPARISON.adoc index e17e48c..d93c0c3 100644 --- a/docs/BETLANG-COMPARISON.md +++ b/docs/BETLANG-COMPARISON.adoc @@ -1,10 +1,6 @@ - -# StatistEase + BetLang Capability Comparison +== StatistEase + BetLang Capability Comparison -``` +.... ┌─────────────────────────────┬──────────────────────┬──────────────────────┬──────────────┐ │ Capability │ BetLang (Racket) │ StatistEase (Julia) │ Winner │ ├─────────────────────────────┼──────────────────────┼──────────────────────┼──────────────┤ @@ -38,4 +34,4 @@ Copyright (c) Jonathan D.A. Jewell COMBINED POWER: StatistEase uses BetLang for Bayesian/sampling/optimization/uncertainty. BetLang uses StatistEase for hypothesis testing/formal verification/cross-verification. -``` +.... diff --git a/docs/api-reference.adoc b/docs/api-reference.adoc new file mode 100644 index 0000000..1bdad2a --- /dev/null +++ b/docs/api-reference.adoc @@ -0,0 +1,725 @@ +== betlang API Reference + +Complete API documentation for all betlang functions and libraries. + +=== Core Module (`+core/betlang.rkt+`) + +==== Basic Operations + +===== `+(bet A B C)+` + +Randomly selects one of three values with equal probability (1/3 each). + +*Parameters:* - `+A+`, `+B+`, `+C+`: Any Racket values + +*Returns:* One of the three input values + +*Example:* + +[source,racket] +---- +(bet 1 2 3) ; Returns 1, 2, or 3 +---- + +''''' + +===== `+(bet/weighted '(A weight-a) '(B weight-b) '(C weight-c))+` + +Selects one of three weighted choices where probabilities are +proportional to weights. + +*Parameters:* - Three lists, each containing a value and its weight + +*Returns:* One of the three values, weighted by probability + +*Example:* + +[source,racket] +---- +(bet/weighted '(rare 1) '(uncommon 3) '(common 6)) +; 10% rare, 30% uncommon, 60% common +---- + +''''' + +===== `+(bet/conditional predicate A B C)+` + +Returns A if predicate is true, otherwise performs bet between B and C. + +*Parameters:* - `+predicate+`: Boolean expression - `+A+`, `+B+`, `+C+`: +Any Racket values + +*Returns:* A value based on the condition + +*Example:* + +[source,racket] +---- +(bet/conditional (> x 10) 'large 'medium 'small) +---- + +''''' + +===== `+(bet/lazy thunk-a thunk-b thunk-c)+` + +Lazy version that only evaluates the selected branch. + +*Parameters:* - Three zero-argument functions (thunks) + +*Returns:* Result of calling the selected thunk + +*Example:* + +[source,racket] +---- +(bet/lazy + (lambda () (expensive-a)) + (lambda () (expensive-b)) + (lambda () (expensive-c))) +---- + +''''' + +==== Composition Operations + +===== `+(bet-chain n f init)+` + +Chains n bets together, threading results through function f. + +*Parameters:* - `+n+`: Number of iterations - `+f+`: Function to apply +at each step - `+init+`: Initial value + +*Returns:* Final value after n applications + +*Example:* + +[source,racket] +---- +(bet-chain 10 (lambda (x) (+ x 1)) 0) +; Returns 10 +---- + +''''' + +===== `+(bet-compose f g h)+` + +Creates a function that randomly selects one of three functions to +apply. + +*Parameters:* - `+f+`, `+g+`, `+h+`: Functions to compose + +*Returns:* A function that applies one of f, g, or h + +*Example:* + +[source,racket] +---- +(define process (bet-compose add1 identity sub1)) +(process 10) ; Returns 9, 10, or 11 +---- + +''''' + +===== `+(bet-map f lst)+` + +Maps a function over a list with probabilistic selection. + +*Parameters:* - `+f+`: Function to apply - `+lst+`: List of values + +*Returns:* Transformed list + +''''' + +===== `+(bet-fold f init lst)+` + +Fold operation with probabilistic choices. + +*Parameters:* - `+f+`: Binary function - `+init+`: Initial accumulator - +`+lst+`: List to fold over + +*Returns:* Accumulated value + +''''' + +===== `+(bet-filter pred lst)+` + +Filters list with probabilistic predicate evaluation. + +*Parameters:* - `+pred+`: Predicate function - `+lst+`: List to filter + +*Returns:* Filtered list + +''''' + +==== Parallel Operations + +===== `+(bet-parallel n A B C)+` + +Runs n independent trials and returns all results. + +*Parameters:* - `+n+`: Number of trials - `+A+`, `+B+`, `+C+`: Values to +bet on + +*Returns:* List of n results + +*Example:* + +[source,racket] +---- +(bet-parallel 100 'heads 'tails 'edge) +; Returns list of 100 outcomes +---- + +''''' + +===== `+(bet-repeat n thunk)+` + +Repeats a bet n times and collects results. + +*Parameters:* - `+n+`: Number of repetitions - `+thunk+`: Zero-argument +function to call + +*Returns:* List of n results + +''''' + +===== `+(bet-sequence bet1 bet2 bet3 ...)+` + +Executes bets in sequence. + +*Parameters:* - Variable number of bet specifications + +*Returns:* List of results + +''''' + +==== Control Flow + +===== `+(bet-until predicate thunk)+` + +Repeats bet until predicate is satisfied. + +*Parameters:* - `+predicate+`: Function testing result - `+thunk+`: +Zero-argument function generating results + +*Returns:* First result satisfying predicate + +*Example:* + +[source,racket] +---- +(bet-until + (lambda (x) (equal? x 'target)) + (lambda () (bet 'target 'miss 'miss))) +---- + +''''' + +===== `+(bet-with-seed seed thunk)+` + +Executes bet with specific random seed. + +*Parameters:* - `+seed+`: Integer seed for random number generator - +`+thunk+`: Zero-argument function to execute + +*Returns:* Result of thunk with deterministic randomness + +*Example:* + +[source,racket] +---- +(bet-with-seed 42 (lambda () (bet 1 2 3))) +; Always returns same value +---- + +''''' + +==== Utility Functions + +===== `+(all-bets A B C)+` + +Returns all three possible outcomes as a list. + +*Returns:* List containing all outcomes + +''''' + +===== `+(make-bet-generator A B C)+` + +Creates a generator function for repeated betting. + +*Returns:* Zero-argument function that generates bet results + +''''' + +==== Statistical Functions + +===== `+(bet-probability n predicate A B C)+` + +Estimates probability that predicate holds over n trials. + +*Parameters:* - `+n+`: Number of trials - `+predicate+`: Function +testing outcomes - `+A+`, `+B+`, `+C+`: Bet values + +*Returns:* Probability estimate (0.0 to 1.0) + +*Example:* + +[source,racket] +---- +(bet-probability 10000 + (lambda (x) (equal? x 'A)) + 'A 'B 'C) +; ≈ 0.333 +---- + +''''' + +===== `+(bet-entropy samples)+` + +Calculates Shannon entropy of samples in bits. + +*Parameters:* - `+samples+`: List of sample values + +*Returns:* Entropy value + +''''' + +===== `+(bet-expect n f A B C)+` + +Calculates expected value of function f over n trials. + +*Parameters:* - `+n+`: Number of trials - `+f+`: Function to apply to +outcomes - `+A+`, `+B+`, `+C+`: Bet values + +*Returns:* Expected value + +''''' + +=== Statistics Module (`+lib/statistics.rkt+`) + +==== Descriptive Statistics + +===== `+(mean samples)+` + +Arithmetic mean of samples. + +''''' + +===== `+(median samples)+` + +Median value. + +''''' + +===== `+(mode samples)+` + +Most frequent value(s). + +''''' + +===== `+(variance samples)+` + +Sample variance. + +''''' + +===== `+(stddev samples)+` + +Standard deviation. + +''''' + +===== `+(percentile samples p)+` + +p-th percentile (p between 0 and 1). + +''''' + +==== Correlation and Covariance + +===== `+(covariance samples1 samples2)+` + +Covariance between two samples. + +''''' + +===== `+(correlation samples1 samples2)+` + +Pearson correlation coefficient. + +''''' + +==== Statistical Tests + +===== `+(chi-square-test observed expected)+` + +Chi-square goodness of fit test. + +*Parameters:* - `+observed+`: List of observed frequencies - +`+expected+`: List of expected frequencies + +*Returns:* Chi-square statistic + +''''' + +===== `+(kolmogorov-smirnov samples1 samples2)+` + +KS test statistic for comparing distributions. + +''''' + +==== Resampling Methods + +===== `+(bootstrap samples n statistic)+` + +Bootstrap resampling. + +*Parameters:* - `+samples+`: Original sample - `+n+`: Number of +bootstrap samples - `+statistic+`: Function to compute on each resample + +*Returns:* List of n statistic values + +''''' + +===== `+(jackknife samples statistic)+` + +Jackknife resampling. + +''''' + +==== Time Series + +===== `+(moving-average samples window)+` + +Moving average with window size. + +''''' + +===== `+(exponential-smoothing samples alpha)+` + +Exponential smoothing with parameter alpha. + +''''' + +==== Simulation + +===== `+(run-simulation n experiment)+` + +Runs experiment n times. + +''''' + +===== `+(monte-carlo-pi n)+` + +Estimates π using Monte Carlo method. + +''''' + +=== Distributions Module (`+lib/distributions.rkt+`) + +==== Discrete Distributions + +===== `+(uniform a b)+` + +Discrete uniform distribution from a to b. + +''''' + +===== `+(bernoulli p)+` + +Bernoulli trial with success probability p. + +''''' + +===== `+(binomial n p)+` + +Binomial distribution: n trials, probability p. + +''''' + +===== `+(geometric p)+` + +Geometric distribution: trials until first success. + +''''' + +===== `+(poisson lambda)+` + +Poisson distribution with rate lambda. + +''''' + +===== `+(categorical probs)+` + +Categorical distribution given probability list. + +''''' + +==== Continuous Distributions + +===== `+(normal mu sigma)+` + +Normal (Gaussian) distribution. + +*Parameters:* - `+mu+`: Mean - `+sigma+`: Standard deviation + +''''' + +===== `+(exponential lambda)+` + +Exponential distribution with rate lambda. + +''''' + +===== `+(gamma shape scale)+` + +Gamma distribution. + +''''' + +===== `+(beta alpha beta)+` + +Beta distribution. + +''''' + +===== `+(student-t df)+` + +Student’s t-distribution with df degrees of freedom. + +''''' + +===== `+(chi-square k)+` + +Chi-square distribution with k degrees of freedom. + +''''' + +==== Stochastic Processes + +===== `+(random-walk n)+` + +Random walk of n steps. + +*Returns:* List of positions + +''''' + +===== `+(brownian-motion n dt)+` + +Brownian motion simulation. + +*Parameters:* - `+n+`: Number of steps - `+dt+`: Time increment + +''''' + +===== `+(levy-flight n alpha)+` + +Lévy flight with stability parameter alpha. + +''''' + +=== Combinators Module (`+lib/combinators.rkt+`) + +==== Monadic Operations + +===== `+(bet-pure x)+` + +Returns deterministic bet always returning x. + +''''' + +===== `+(bet-bind m f)+` + +Monadic bind operation. + +''''' + +==== Logical Combinators + +===== `+(bet-or a b c)+`, `+(bet-and a b c)+`, `+(bet-xor a b c)+` + +Logical operations on ternary values. + +''''' + +==== Error Handling + +===== `+(bet-try thunk handler)+` + +Try-catch for bets. + +''''' + +===== `+(bet-fallback thunk1 thunk2 ...)+` + +Try thunks in order until one succeeds. + +''''' + +===== `+(bet-retry n thunk)+` + +Retry up to n times on failure. + +''''' + +==== Performance + +===== `+(bet-memoize thunk)+` + +Memoize bet result (cache permanently). + +''''' + +===== `+(bet-cache ttl thunk)+` + +Cache with time-to-live in milliseconds. + +''''' + +===== `+(bet-throttle interval thunk)+` + +Throttle execution (minimum interval between calls). + +''''' + +=== Markov Chains Module (`+lib/markov.rkt+`) + +==== `+(make-markov-chain states transitions initial)+` + +Creates a Markov chain. + +*Parameters:* - `+states+`: List of possible states - `+transitions+`: +Hash of transition probabilities - `+initial+`: Initial state + +''''' + +==== `+(markov-step chain current-state)+` + +Takes one step in the chain. + +''''' + +==== `+(markov-simulate chain n)+` + +Simulates chain for n steps. + +''''' + +==== `+(markov-stationary chain n-simulations)+` + +Estimates stationary distribution. + +''''' + +==== `+(estimate-transitions data)+` + +Learns transition matrix from observed data. + +''''' + +==== `+(ternary-markov a b c)+` + +Creates 3-state Markov chain with given transition weights. + +''''' + +=== Analysis Tools (`+tools/analyzer.rkt+`) + +==== `+(analyze-bet bet-fn n)+` + +Comprehensive analysis of a bet function. + +*Parameters:* - `+bet-fn+`: Zero-argument function to analyze - `+n+`: +Number of samples + +*Displays:* Frequency distribution, statistics, entropy, histogram + +''''' + +==== `+(compare-bets bet-fns names n)+` + +Compare multiple betting strategies. + +''''' + +==== `+(convergence-analysis bet-fn target-outcome trials)+` + +Analyze convergence to theoretical probability. + +''''' + +==== `+(text-histogram freq max-width)+` + +Display text-based histogram. + +''''' + +=== REPL Commands + +* `+:help+` - Show help message +* `+:stats+` - Display session statistics +* `+:reset-stats+` - Reset statistics +* `+:history+` - Show command history +* `+:examples+` - Show example usage +* `+:quit+` - Exit REPL + +''''' + +=== Type Signatures (Informal) + +.... +bet : α → β → γ → (α | β | γ) +bet/weighted : (α × ℝ) → (β × ℝ) → (γ × ℝ) → (α | β | γ) +bet-parallel : ℕ → α → β → γ → List[α | β | γ] +bet-probability : ℕ → (α → 𝔹) → α → β → γ → ℝ ∈ [0,1] +mean : List[ℝ] → ℝ +normal : ℝ → ℝ → ℝ +markov-simulate : MarkovChain → ℕ → List[State] +.... + +''''' + +=== Constants and Special Values + +* `+pi+` - Mathematical constant π (from Racket) +* Random seed can be any integer for `+bet-with-seed+` + +''''' + +=== Error Conditions + +All functions raise `+exn:fail?+` exceptions on invalid input: - +`+bet/weighted+` requires exactly 3 weighted choices - Statistical +functions require non-empty lists - Probabilities must be in range [0, +1] - Markov chains require valid state transitions + +''''' + +=== Performance Notes + +* `+bet-parallel+` is O(n) in number of trials +* `+bet-chain+` is O(n) in chain length +* Distribution sampling varies (normal: O(1), gamma: varies) +* Markov simulation is O(steps × states) + +''''' + +=== Thread Safety + +Bet operations are *not thread-safe* by default. Use Racket’s +synchronization primitives (`+semaphore+`, `+channel+`, etc.) for +concurrent access. + +''''' + +=== Version + +API Version: 2.0 Last Updated: 2025 + +''''' + +For more information, see: - link:tutorial.md[Tutorial] - Learn by +example - link:semantics.md[Semantics] - Formal specifications - +link:../examples/[Examples] - Code examples diff --git a/docs/api-reference.md b/docs/api-reference.md deleted file mode 100644 index 66765c5..0000000 --- a/docs/api-reference.md +++ /dev/null @@ -1,672 +0,0 @@ - -# betlang API Reference - -Complete API documentation for all betlang functions and libraries. - -## Core Module (`core/betlang.rkt`) - -### Basic Operations - -#### `(bet A B C)` -Randomly selects one of three values with equal probability (1/3 each). - -**Parameters:** -- `A`, `B`, `C`: Any Racket values - -**Returns:** One of the three input values - -**Example:** -```racket -(bet 1 2 3) ; Returns 1, 2, or 3 -``` - ---- - -#### `(bet/weighted '(A weight-a) '(B weight-b) '(C weight-c))` -Selects one of three weighted choices where probabilities are proportional to weights. - -**Parameters:** -- Three lists, each containing a value and its weight - -**Returns:** One of the three values, weighted by probability - -**Example:** -```racket -(bet/weighted '(rare 1) '(uncommon 3) '(common 6)) -; 10% rare, 30% uncommon, 60% common -``` - ---- - -#### `(bet/conditional predicate A B C)` -Returns A if predicate is true, otherwise performs bet between B and C. - -**Parameters:** -- `predicate`: Boolean expression -- `A`, `B`, `C`: Any Racket values - -**Returns:** A value based on the condition - -**Example:** -```racket -(bet/conditional (> x 10) 'large 'medium 'small) -``` - ---- - -#### `(bet/lazy thunk-a thunk-b thunk-c)` -Lazy version that only evaluates the selected branch. - -**Parameters:** -- Three zero-argument functions (thunks) - -**Returns:** Result of calling the selected thunk - -**Example:** -```racket -(bet/lazy - (lambda () (expensive-a)) - (lambda () (expensive-b)) - (lambda () (expensive-c))) -``` - ---- - -### Composition Operations - -#### `(bet-chain n f init)` -Chains n bets together, threading results through function f. - -**Parameters:** -- `n`: Number of iterations -- `f`: Function to apply at each step -- `init`: Initial value - -**Returns:** Final value after n applications - -**Example:** -```racket -(bet-chain 10 (lambda (x) (+ x 1)) 0) -; Returns 10 -``` - ---- - -#### `(bet-compose f g h)` -Creates a function that randomly selects one of three functions to apply. - -**Parameters:** -- `f`, `g`, `h`: Functions to compose - -**Returns:** A function that applies one of f, g, or h - -**Example:** -```racket -(define process (bet-compose add1 identity sub1)) -(process 10) ; Returns 9, 10, or 11 -``` - ---- - -#### `(bet-map f lst)` -Maps a function over a list with probabilistic selection. - -**Parameters:** -- `f`: Function to apply -- `lst`: List of values - -**Returns:** Transformed list - ---- - -#### `(bet-fold f init lst)` -Fold operation with probabilistic choices. - -**Parameters:** -- `f`: Binary function -- `init`: Initial accumulator -- `lst`: List to fold over - -**Returns:** Accumulated value - ---- - -#### `(bet-filter pred lst)` -Filters list with probabilistic predicate evaluation. - -**Parameters:** -- `pred`: Predicate function -- `lst`: List to filter - -**Returns:** Filtered list - ---- - -### Parallel Operations - -#### `(bet-parallel n A B C)` -Runs n independent trials and returns all results. - -**Parameters:** -- `n`: Number of trials -- `A`, `B`, `C`: Values to bet on - -**Returns:** List of n results - -**Example:** -```racket -(bet-parallel 100 'heads 'tails 'edge) -; Returns list of 100 outcomes -``` - ---- - -#### `(bet-repeat n thunk)` -Repeats a bet n times and collects results. - -**Parameters:** -- `n`: Number of repetitions -- `thunk`: Zero-argument function to call - -**Returns:** List of n results - ---- - -#### `(bet-sequence bet1 bet2 bet3 ...)` -Executes bets in sequence. - -**Parameters:** -- Variable number of bet specifications - -**Returns:** List of results - ---- - -### Control Flow - -#### `(bet-until predicate thunk)` -Repeats bet until predicate is satisfied. - -**Parameters:** -- `predicate`: Function testing result -- `thunk`: Zero-argument function generating results - -**Returns:** First result satisfying predicate - -**Example:** -```racket -(bet-until - (lambda (x) (equal? x 'target)) - (lambda () (bet 'target 'miss 'miss))) -``` - ---- - -#### `(bet-with-seed seed thunk)` -Executes bet with specific random seed. - -**Parameters:** -- `seed`: Integer seed for random number generator -- `thunk`: Zero-argument function to execute - -**Returns:** Result of thunk with deterministic randomness - -**Example:** -```racket -(bet-with-seed 42 (lambda () (bet 1 2 3))) -; Always returns same value -``` - ---- - -### Utility Functions - -#### `(all-bets A B C)` -Returns all three possible outcomes as a list. - -**Returns:** List containing all outcomes - ---- - -#### `(make-bet-generator A B C)` -Creates a generator function for repeated betting. - -**Returns:** Zero-argument function that generates bet results - ---- - -### Statistical Functions - -#### `(bet-probability n predicate A B C)` -Estimates probability that predicate holds over n trials. - -**Parameters:** -- `n`: Number of trials -- `predicate`: Function testing outcomes -- `A`, `B`, `C`: Bet values - -**Returns:** Probability estimate (0.0 to 1.0) - -**Example:** -```racket -(bet-probability 10000 - (lambda (x) (equal? x 'A)) - 'A 'B 'C) -; ≈ 0.333 -``` - ---- - -#### `(bet-entropy samples)` -Calculates Shannon entropy of samples in bits. - -**Parameters:** -- `samples`: List of sample values - -**Returns:** Entropy value - ---- - -#### `(bet-expect n f A B C)` -Calculates expected value of function f over n trials. - -**Parameters:** -- `n`: Number of trials -- `f`: Function to apply to outcomes -- `A`, `B`, `C`: Bet values - -**Returns:** Expected value - ---- - -## Statistics Module (`lib/statistics.rkt`) - -### Descriptive Statistics - -#### `(mean samples)` -Arithmetic mean of samples. - ---- - -#### `(median samples)` -Median value. - ---- - -#### `(mode samples)` -Most frequent value(s). - ---- - -#### `(variance samples)` -Sample variance. - ---- - -#### `(stddev samples)` -Standard deviation. - ---- - -#### `(percentile samples p)` -p-th percentile (p between 0 and 1). - ---- - -### Correlation and Covariance - -#### `(covariance samples1 samples2)` -Covariance between two samples. - ---- - -#### `(correlation samples1 samples2)` -Pearson correlation coefficient. - ---- - -### Statistical Tests - -#### `(chi-square-test observed expected)` -Chi-square goodness of fit test. - -**Parameters:** -- `observed`: List of observed frequencies -- `expected`: List of expected frequencies - -**Returns:** Chi-square statistic - ---- - -#### `(kolmogorov-smirnov samples1 samples2)` -KS test statistic for comparing distributions. - ---- - -### Resampling Methods - -#### `(bootstrap samples n statistic)` -Bootstrap resampling. - -**Parameters:** -- `samples`: Original sample -- `n`: Number of bootstrap samples -- `statistic`: Function to compute on each resample - -**Returns:** List of n statistic values - ---- - -#### `(jackknife samples statistic)` -Jackknife resampling. - ---- - -### Time Series - -#### `(moving-average samples window)` -Moving average with window size. - ---- - -#### `(exponential-smoothing samples alpha)` -Exponential smoothing with parameter alpha. - ---- - -### Simulation - -#### `(run-simulation n experiment)` -Runs experiment n times. - ---- - -#### `(monte-carlo-pi n)` -Estimates π using Monte Carlo method. - ---- - -## Distributions Module (`lib/distributions.rkt`) - -### Discrete Distributions - -#### `(uniform a b)` -Discrete uniform distribution from a to b. - ---- - -#### `(bernoulli p)` -Bernoulli trial with success probability p. - ---- - -#### `(binomial n p)` -Binomial distribution: n trials, probability p. - ---- - -#### `(geometric p)` -Geometric distribution: trials until first success. - ---- - -#### `(poisson lambda)` -Poisson distribution with rate lambda. - ---- - -#### `(categorical probs)` -Categorical distribution given probability list. - ---- - -### Continuous Distributions - -#### `(normal mu sigma)` -Normal (Gaussian) distribution. - -**Parameters:** -- `mu`: Mean -- `sigma`: Standard deviation - ---- - -#### `(exponential lambda)` -Exponential distribution with rate lambda. - ---- - -#### `(gamma shape scale)` -Gamma distribution. - ---- - -#### `(beta alpha beta)` -Beta distribution. - ---- - -#### `(student-t df)` -Student's t-distribution with df degrees of freedom. - ---- - -#### `(chi-square k)` -Chi-square distribution with k degrees of freedom. - ---- - -### Stochastic Processes - -#### `(random-walk n)` -Random walk of n steps. - -**Returns:** List of positions - ---- - -#### `(brownian-motion n dt)` -Brownian motion simulation. - -**Parameters:** -- `n`: Number of steps -- `dt`: Time increment - ---- - -#### `(levy-flight n alpha)` -Lévy flight with stability parameter alpha. - ---- - -## Combinators Module (`lib/combinators.rkt`) - -### Monadic Operations - -#### `(bet-pure x)` -Returns deterministic bet always returning x. - ---- - -#### `(bet-bind m f)` -Monadic bind operation. - ---- - -### Logical Combinators - -#### `(bet-or a b c)`, `(bet-and a b c)`, `(bet-xor a b c)` -Logical operations on ternary values. - ---- - -### Error Handling - -#### `(bet-try thunk handler)` -Try-catch for bets. - ---- - -#### `(bet-fallback thunk1 thunk2 ...)` -Try thunks in order until one succeeds. - ---- - -#### `(bet-retry n thunk)` -Retry up to n times on failure. - ---- - -### Performance - -#### `(bet-memoize thunk)` -Memoize bet result (cache permanently). - ---- - -#### `(bet-cache ttl thunk)` -Cache with time-to-live in milliseconds. - ---- - -#### `(bet-throttle interval thunk)` -Throttle execution (minimum interval between calls). - ---- - -## Markov Chains Module (`lib/markov.rkt`) - -#### `(make-markov-chain states transitions initial)` -Creates a Markov chain. - -**Parameters:** -- `states`: List of possible states -- `transitions`: Hash of transition probabilities -- `initial`: Initial state - ---- - -#### `(markov-step chain current-state)` -Takes one step in the chain. - ---- - -#### `(markov-simulate chain n)` -Simulates chain for n steps. - ---- - -#### `(markov-stationary chain n-simulations)` -Estimates stationary distribution. - ---- - -#### `(estimate-transitions data)` -Learns transition matrix from observed data. - ---- - -#### `(ternary-markov a b c)` -Creates 3-state Markov chain with given transition weights. - ---- - -## Analysis Tools (`tools/analyzer.rkt`) - -#### `(analyze-bet bet-fn n)` -Comprehensive analysis of a bet function. - -**Parameters:** -- `bet-fn`: Zero-argument function to analyze -- `n`: Number of samples - -**Displays:** Frequency distribution, statistics, entropy, histogram - ---- - -#### `(compare-bets bet-fns names n)` -Compare multiple betting strategies. - ---- - -#### `(convergence-analysis bet-fn target-outcome trials)` -Analyze convergence to theoretical probability. - ---- - -#### `(text-histogram freq max-width)` -Display text-based histogram. - ---- - -## REPL Commands - -- `:help` - Show help message -- `:stats` - Display session statistics -- `:reset-stats` - Reset statistics -- `:history` - Show command history -- `:examples` - Show example usage -- `:quit` - Exit REPL - ---- - -## Type Signatures (Informal) - -``` -bet : α → β → γ → (α | β | γ) -bet/weighted : (α × ℝ) → (β × ℝ) → (γ × ℝ) → (α | β | γ) -bet-parallel : ℕ → α → β → γ → List[α | β | γ] -bet-probability : ℕ → (α → 𝔹) → α → β → γ → ℝ ∈ [0,1] -mean : List[ℝ] → ℝ -normal : ℝ → ℝ → ℝ -markov-simulate : MarkovChain → ℕ → List[State] -``` - ---- - -## Constants and Special Values - -- `pi` - Mathematical constant π (from Racket) -- Random seed can be any integer for `bet-with-seed` - ---- - -## Error Conditions - -All functions raise `exn:fail?` exceptions on invalid input: -- `bet/weighted` requires exactly 3 weighted choices -- Statistical functions require non-empty lists -- Probabilities must be in range [0, 1] -- Markov chains require valid state transitions - ---- - -## Performance Notes - -- `bet-parallel` is O(n) in number of trials -- `bet-chain` is O(n) in chain length -- Distribution sampling varies (normal: O(1), gamma: varies) -- Markov simulation is O(steps × states) - ---- - -## Thread Safety - -Bet operations are **not thread-safe** by default. Use Racket's synchronization primitives (`semaphore`, `channel`, etc.) for concurrent access. - ---- - -## Version - -API Version: 2.0 -Last Updated: 2025 - ---- - -For more information, see: -- [Tutorial](tutorial.md) - Learn by example -- [Semantics](semantics.md) - Formal specifications -- [Examples](../examples/) - Code examples diff --git a/docs/architecture.adoc b/docs/architecture.adoc new file mode 100644 index 0000000..2988ed7 --- /dev/null +++ b/docs/architecture.adoc @@ -0,0 +1,8 @@ +== Project Architecture + +* core/betlang.rkt — DSL primitives + +* lib/ternary.rkt — Conditional abstraction + +* repl/shell.rkt — Interactive REPL + logging + +* tests/ — Probabilistic test suite + +* docs/ — Theory & semantics + +* homepage/ — Public web interface diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 1a893b8..0000000 --- a/docs/architecture.md +++ /dev/null @@ -1,12 +0,0 @@ - -# Project Architecture - -- core/betlang.rkt — DSL primitives -- lib/ternary.rkt — Conditional abstraction -- repl/shell.rkt — Interactive REPL + logging -- tests/ — Probabilistic test suite -- docs/ — Theory & semantics -- homepage/ — Public web interface diff --git a/docs/experimental-ideas.adoc b/docs/experimental-ideas.adoc new file mode 100644 index 0000000..86c9991 --- /dev/null +++ b/docs/experimental-ideas.adoc @@ -0,0 +1,63 @@ +== Betlang experimental ideas + +=== Purpose + +This note collects the "`next question`" ideas we discussed so we can +experiment safely without letting scope creep derail the stable heart of +Betlang. Keep the core grammar/number systems working, then prototype +here. + +=== Current focus + +* `+compiler/bet-parse+` now accepts `+end+`-terminated blocks for +`+bet+`, `+let … in+`, `+if … then … else+`, `+match+`, `+do+`, +`+parallel+`. + +* The brace/`+{…}+` syntax remains supported for backward compatibility, +so nothing breaks downstream. + +* Julia bindings already surface the same features via the C FFI. +* Tests should be run (`+cargo test+`), but the rust toolchain needs a +default (see below). + +=== Experiment ideas backlog + +==== 1. Aspect-oriented grammar injections + +* Investigate how `+julia-the-viper+` handles `+adder+`/`+havard+` block +injection. +* Define hooks or directives so Betlang can optionally parse new blocks +that desugar to current AST. +* Include instructions for turning these hooks on/off for experiments +before landing them in docs. + +==== 2. Quantum / multi-log layers + +* Prototype as libraries (e.g., `+lib/quantum-preview.rkt+`) so they can +be used today without touching grammar. +* If the pattern matures, define a keyword extension with optional +grammar rules (ad-hoc `+quantum … end+`) that maps to standard +semantics. +* Keep proofs/test coverage separated from the core to minimize safety +risk. + +==== 3. Base/radix/representation experiments + +* Continue using real arithmetic but wrap p-adic probability base +parameterization to explore digit expansion ideas. +* Document conversion helpers in `+lib/number-systems.rkt+` or new +helper module rather than altering core numeric types. + +==== 4. Safety / social controls + +* Use modules (like `+lib/cool-off+`) to surface new +responsible-gambling layers, and extend docs with checklists for trial +runs. +* Keep verified guarantees (Dutch book, risk-of-ruin) central when +experimenting with new primitives. + +=== Notes + +* `+cargo test+` currently fails because the Rust toolchain is not +configured (`+rustup default stable+` needed). The compiler tests still +need to be run once the default toolchain is installed. +* Anything that touches grammar should remain optional until the +library-level experimentation proves consistent. diff --git a/docs/experimental-ideas.md b/docs/experimental-ideas.md deleted file mode 100644 index a3d5ade..0000000 --- a/docs/experimental-ideas.md +++ /dev/null @@ -1,41 +0,0 @@ - -# Betlang experimental ideas - -## Purpose - -This note collects the "next question" ideas we discussed so we can experiment safely without letting scope creep derail the stable heart of Betlang. Keep the core grammar/number systems working, then prototype here. - -## Current focus - -- `compiler/bet-parse` now accepts `end`-terminated blocks for `bet`, `let … in`, `if … then … else`, `match`, `do`, `parallel`. -- The brace/`{…}` syntax remains supported for backward compatibility, so nothing breaks downstream. -- Julia bindings already surface the same features via the C FFI. -- Tests should be run (`cargo test`), but the rust toolchain needs a default (see below). - -## Experiment ideas backlog - -### 1. Aspect-oriented grammar injections -- Investigate how `julia-the-viper` handles `adder`/`havard` block injection. -- Define hooks or directives so Betlang can optionally parse new blocks that desugar to current AST. -- Include instructions for turning these hooks on/off for experiments before landing them in docs. - -### 2. Quantum / multi-log layers -- Prototype as libraries (e.g., `lib/quantum-preview.rkt`) so they can be used today without touching grammar. -- If the pattern matures, define a keyword extension with optional grammar rules (ad-hoc `quantum … end`) that maps to standard semantics. -- Keep proofs/test coverage separated from the core to minimize safety risk. - -### 3. Base/radix/representation experiments -- Continue using real arithmetic but wrap p-adic probability base parameterization to explore digit expansion ideas. -- Document conversion helpers in `lib/number-systems.rkt` or new helper module rather than altering core numeric types. - -### 4. Safety / social controls -- Use modules (like `lib/cool-off`) to surface new responsible-gambling layers, and extend docs with checklists for trial runs. -- Keep verified guarantees (Dutch book, risk-of-ruin) central when experimenting with new primitives. - -## Notes - -- `cargo test` currently fails because the Rust toolchain is not configured (`rustup default stable` needed). The compiler tests still need to be run once the default toolchain is installed. -- Anything that touches grammar should remain optional until the library-level experimentation proves consistent. diff --git a/docs/hyperpolymath-julia-integration.adoc b/docs/hyperpolymath-julia-integration.adoc new file mode 100644 index 0000000..6e5be1e --- /dev/null +++ b/docs/hyperpolymath-julia-integration.adoc @@ -0,0 +1,597 @@ +== BetLang Integration with Hyperpolymath Julia Packages + +*Status:* Planning Phase (2026-02-07) *Purpose:* Connect betlang to +existing hyperpolymath Julia ecosystem + +=== IMPORTANT: Priority Clarification + +*This document covers Tier 1 (Extensions) - these are additions to the +core Julia ecosystem, not replacements.* + +==== Priority Order (Most → Least Critical): + +*Tier 0 (Foundation) - MOST IMPORTANT:* - Distributions.jl, +StatsBase.jl, Turing.jl, Optim.jl, etc. - See +`+julia-backend-design.md+` for comprehensive Tier 0 coverage - *These +must be implemented first* - they’re mature, essential packages + +*Tier 1 (This Document) - Valuable Extensions:* - BowtieRisk.jl, +ZeroProb.jl, Causals.jl, domain packages - Built *on top of* Tier 0 +foundation - Add domain-specific capabilities + +*Tier 2+ - Specialized:* - As needed for specific use cases + +=== Overview + +Once BetLang has solid integration with standard Julia packages (Tier +0), it can _also_ leverage your existing hyperpolymath packages to +create a comprehensive probabilistic programming + risk analysis + +causal inference ecosystem. + +=== Your Julia Packages (Discovered) + +==== Tier 1: Direct BetLang Integration (High Synergy) + +===== 1. *BowtieRisk.jl* - Bowtie Risk Modeling ⭐⭐⭐ + +*Purpose:* Hazard analysis with threats, barriers, and consequences +*Current deps:* Distributions.jl, JSON3 + +*Integration with BetLang:* + +*Synergy Points:* - *Risk-of-Ruin Protection* → Bowtie consequence +analysis - *Monte Carlo Simulation* → Bowtie barrier effectiveness - +*Probability Validation* → Dutch book prevention for threat +probabilities + +*Example Integration:* + +[source,julia] +---- +using BetLang, BowtieRisk + +# Define a gambling bowtie model +model = BowtieModel( + hazard = "Financial Ruin", + threats = [ + Threat("High Stakes", probability=0.3), + Threat("Rapid Betting", probability=0.4), + Threat("Chasing Losses", probability=0.5) + ], + barriers = [ + # BetLang safety features as barriers! + Barrier("Dutch Book Prevention", effectiveness=0.95), + Barrier("Risk-of-Ruin Check", effectiveness=0.90), + Barrier("Cool-Off Mechanism", effectiveness=0.85), + Barrier("Kelly Criterion", effectiveness=0.92) + ], + consequences = [ + Consequence("Bankruptcy", severity=10.0, probability=0.8) + ] +) + +# Run BetLang-powered Monte Carlo simulation +results = simulate_bowtie(model, n=100_000) +# Uses BetLang's bet primitives for sampling + +# Validate no Dutch books in threat probabilities +threats_probs = [t.probability for t in model.threats] +validate_dutch_book(threats_probs) # BetLang function +---- + +*New Features to Add to BowtieRisk:* - `+BetLangBarrier+` type - +barriers implemented as betlang functions - +`+gambling_harm_reduction_template()+` - pre-built model for gambling +risks - `+monte_carlo_with_betlang()+` - use betlang’s uncertainty +numbers + +===== 2. *ZeroProb.jl* - Zero-Probability Event Handling ⭐⭐⭐ + +*Purpose:* Handle continuous probability spaces with measure-zero events +*Current deps:* Distributions.jl, StatsBase, Plots, Makie + +*Integration with BetLang:* + +*Synergy Points:* - *Continuous Distributions* → betlang’s +DistnumberNormal, DistnumberBeta - *Theoretical Rigor* → Formal +foundations for betlang’s probability theory - *Edge Cases* → Handling +degenerate cases in number systems + +*Example Integration:* + +[source,julia] +---- +using BetLang, ZeroProb + +# BetLang distribution as continuous space +height_dist = DistnumberNormal(170, 10) + +# Query exact point (has zero probability but can occur) +point = 175.5 +density = density_ratio(height_dist, point) # ZeroProb relevance measure + +# ε-neighborhood probability (BetLang + ZeroProb) +prob_near_175 = epsilon_neighborhood_prob(height_dist, 175.5, ε=0.1) +# P(175.4 < height < 175.6) ≈ 0.008 + +# Hausdorff measure for zero-probability sets +fractal_support = hausdorff_measure( + betlang_distribution_support(height_dist), + dimension=1.5 +) +---- + +*New Features to Add to ZeroProb:* - `+DistnumberAdapter+` - wrap +betlang number systems - `+zero_prob_bet()+` - betting on measure-zero +events - `+continuous_game_theory()+` - combine with betlang game theory +examples + +===== 3. *Causals.jl* - Causal Inference ⭐⭐ + +*Purpose:* Dempster-Shafer, DAGs, do-calculus, counterfactuals *Current +deps:* (need to check) + +*Integration with BetLang:* + +*Synergy Points:* - *Dempster-Shafer* → betlang already has +DempsterShafer number system! - *Counterfactuals* → "`What if I had bet +differently?`" analysis - *Causal DAGs* → Model causal relationships in +probabilistic systems - *Do-Calculus* → Interventions in gambling +scenarios + +*Example Integration:* + +[source,julia] +---- +using BetLang, Causals + +# Model causal structure of gambling outcomes +dag = CausalDAG() +add_nodes!(dag, [:stake, :skill, :luck, :outcome, :bankroll]) +add_edges!(dag, [ + :stake => :outcome, + :skill => :outcome, + :luck => :outcome, + :outcome => :bankroll +]) + +# Counterfactual: "What if I had bet less?" +actual_outcome = simulate_bet(stake=1000, skill=0.6, luck=0.4) +counterfactual = intervene(dag, :stake => 500) +cf_outcome = simulate_bet(counterfactual) + +# Compare using betlang's number systems +diff = imprecise_probability( + lower = min(actual_outcome, cf_outcome), + upper = max(actual_outcome, cf_outcome) +) +---- + +*New Features to Add to Causals:* - `+gambling_dag_template()+` - +standard causal models for gambling - `+betlang_counterfactual()+` - +counterfactual reasoning with bets - `+causal_number_systems()+` - +propagate uncertainty through DAGs + +==== Tier 2: Domain-Specific Extensions + +===== 4. *HackenbushGames.jl* - Hackenbush Game Theory ⭐ + +*Purpose:* Combinatorial game theory (surreal numbers!) *Current deps:* +None + +*Integration with BetLang:* + +*Synergy Points:* - *Surreal Numbers* → betlang has SurrealAdvanced +number system! - *Game Theory* → betlang has +`+examples/game-theory.rkt+` - *Ternary Games* → Hackenbush with +3-player variants + +*Example Integration:* + +[source,julia] +---- +using BetLang, HackenbushGames + +# Hackenbush position as surreal number +position = analyze_hackenbush(blue_edges, red_edges) +surreal_value = to_surreal(position) # e.g., {0,1|2,3} + +# Convert to BetLang SurrealAdvanced +betlang_surreal = SurrealAdvanced( + left_set = surreal_value.left, + right_set = surreal_value.right +) + +# Use in betlang game theory +game = ternary_game( + payoffs = betlang_surreal, + players = 3 +) +nash_equilibrium(game) +---- + +*New Features to Add to HackenbushGames:* - `+ternary_hackenbush()+` - +3-player variant (blue/red/green) - `+surreal_to_betlang()+` - convert +to BetLang number system - `+probabilistic_hackenbush()+` - edges fail +with probability + +===== 5. *Cliometrics.jl* - Quantitative Economic History ⭐ + +*Purpose:* Growth analysis, convergence, institutional quality *Current +deps:* (recently implemented) + +*Integration with BetLang:* + +*Synergy Points:* - *Uncertainty in Historical Data* → betlang number +systems - *Monte Carlo Historical Simulation* → counterfactual histories +- *Economic Risk Analysis* → combine with BowtieRisk + +*Example Integration:* + +[source,julia] +---- +using BetLang, Cliometrics + +# Historical GDP with uncertainty +gdp_1900 = DistnumberNormal(1000, 100) # BetLang uncertainty +gdp_2000 = DistnumberNormal(50000, 5000) + +# Growth rate with error propagation +growth_rate = calculate_growth_rates( + [gdp_1900, gdp_2000], + years = [1900, 2000] +) +# Result: DistnumberNormal(3.9%, 0.2%) - uncertainty propagated! + +# Counterfactual: "What if industrial revolution started earlier?" +counterfactual_trajectory = bet( + scenario_a = early_industrialization_path(), + scenario_b = actual_path(), + scenario_c = late_industrialization_path() +) +---- + +===== 6. *Cliodynamics.jl* - Mathematical Historical Dynamics ⭐ + +*Purpose:* Turchin’s models, elite overproduction, secular cycles +*Current deps:* (recently implemented) + +*Integration with BetLang:* + +*Synergy Points:* - *Stochastic Historical Models* → betlang Monte Carlo +- *Uncertainty in Parameters* → betlang number systems - *Probabilistic +Forecasting* → crisis prediction with confidence + +*Example Integration:* + +[source,julia] +---- +using BetLang, Cliodynamics + +# Model parameters with uncertainty +elite_size = DistnumberBeta(8, 2) # High elite concentration +state_capacity = DistnumberNormal(0.7, 0.1) # Weakening state + +# Run probabilistic cliodynamic simulation +trajectories = bet_repeat(10_000) do + simulate_secular_cycle( + elite_size = sample(elite_size), + state_capacity = sample(state_capacity), + years = 100 + ) +end + +# Crisis probability distribution +crisis_probs = [trajectory.crisis_probability for trajectory in trajectories] +mean_crisis_prob = mean(crisis_probs) +ci_95 = percentile(crisis_probs, [2.5, 97.5]) +---- + +===== 7. *Cladistics.jl* - Phylogenetic Analysis + +*Purpose:* UPGMA, neighbor-joining, maximum parsimony, bootstrap +*Current deps:* (recently implemented) + +*Integration with BetLang:* + +*Synergy Points:* - *Bootstrap Support* → Monte Carlo uncertainty +quantification - *Uncertain Branch Lengths* → betlang number systems - +*Probabilistic Trees* → distribution over tree topologies + +*Example Integration:* + +[source,julia] +---- +using BetLang, Cladistics + +# Distance matrix with measurement uncertainty +distances = [ + DistnumberNormal(0, 0) DistnumberNormal(3, 0.5) DistnumberNormal(5, 0.7); + DistnumberNormal(3, 0.5) DistnumberNormal(0, 0) DistnumberNormal(4, 0.6); + DistnumberNormal(5, 0.7) DistnumberNormal(4, 0.6) DistnumberNormal(0, 0) +] + +# Build tree with uncertainty propagation +trees = bet_repeat(1000) do + D = sample.(distances) # Sample from each distance uncertainty + upgma(D) +end + +# Consensus tree with confidence intervals +consensus = consensus_tree(trees, threshold=0.7) +---- + +==== Tier 3: Theoretical Foundations + +===== 8. *KnotTheory.jl* - Knot Theory + +*Purpose:* Knot invariants, Jones polynomial, etc. *Integration:* +Possibly via surreal numbers? (Hackenbush-style) + +===== 9. *Axiology.jl* - Value Theory + +*Purpose:* Formal value systems *Integration:* Decision theory under +uncertainty (utility + betlang) + +===== 10. *SMTLib.jl* - SMT Solver Interface + +*Purpose:* Satisfiability Modulo Theories *Integration:* Verify betlang +safety properties formally + +===== 11. *ProvenCrypto.jl* - Proven Cryptography + +*Purpose:* Formally verified crypto *Integration:* Secure random number +generation for betlang + +=== Proposed Architecture + +==== Unified Ecosystem: BetLang + Hyperpolymath.jl + +.... + ┌──────────────┐ + │ BetLang │ + │ (Racket) │ + └──────┬───────┘ + │ Compiler + ▼ + ┌──────────────────────────┐ + │ BetLang.jl (Julia Core) │ + └──────┬───────────────────┘ + │ + ┌──────────┼──────────┬──────────┐ + ▼ ▼ ▼ ▼ + ┌─────────┐ ┌──────────┐ ┌────────┐ ┌────────┐ + │BowtieRisk│ │ ZeroProb │ │Causals │ │Clio- │ + │ .jl │ │ .jl │ │ .jl │ │metrics │ + └─────────┘ └──────────┘ └────────┘ └────────┘ + │ │ │ │ + └────────────┴───────────┴───────────┘ + │ + ┌───────▼────────┐ + │ Hyperpolymath │ + │ Julia Stack │ + └────────────────┘ +.... + +=== Implementation Phases + +==== Phase 1: Core BetLang.jl (v0.8.0) + +* Compile betlang → Julia +* Integrate Distributions.jl, StatsBase.jl +* Basic number systems + +==== Phase 2: Risk Integration (v0.9.0) + +* *BowtieRisk.jl* integration +** BetLang safety features as barriers +** Monte Carlo with betlang uncertainty +** Gambling harm reduction templates +* *ZeroProb.jl* integration +** Handle continuous betlang distributions +** Measure-zero event reasoning +** Theoretical rigor for edge cases + +==== Phase 3: Causal + Domain Extensions (v1.0.0) + +* *Causals.jl* integration +** Dempster-Shafer ↔ betlang number system +** Counterfactual gambling analysis +** Causal DAGs for probabilistic models +* *Domain packages* (Cliometrics, Cliodynamics, Cladistics) +** Use betlang for uncertainty quantification +** Monte Carlo historical simulations +** Probabilistic phylogenies +* *Game theory* (HackenbushGames) +** Surreal number interop +** Ternary game variants +** Combinatorial game theory + probability + +==== Phase 4: Theoretical Foundations (v1.1.0+) + +* *SMTLib.jl* - Verify safety properties +* *ProvenCrypto.jl* - Secure RNG +* *KnotTheory.jl* - Advanced surreal arithmetic + +=== Concrete Use Cases + +==== Use Case 1: Gambling Risk Management System + +*Components:* BetLang + BowtieRisk + Causals + +[source,julia] +---- +using BetLang, BowtieRisk, Causals + +# 1. Causal model of gambling addiction +addiction_dag = gambling_addiction_dag() # From Causals.jl + +# 2. Bowtie risk model +risk_model = gambling_bowtie( + barriers = betlang_safety_barriers() # Dutch book, risk-of-ruin, cool-off +) + +# 3. Run counterfactual analysis +actual = simulate_gambling_session( + bets_per_hour = 60, + stake_size = 100 +) + +counterfactual_with_cooloff = intervene( + addiction_dag, + :cool_off_seconds => 30 # BetLang cool-off +) + +# 4. Compare outcomes +risk_reduction = bowtie_probability_reduction( + risk_model, + actual vs counterfactual_with_cooloff +) +# => "Cool-off reduces ruin risk by 73%" +---- + +==== Use Case 2: Historical Counterfactual with Uncertainty + +*Components:* BetLang + Cliometrics + Cliodynamics + +[source,julia] +---- +using BetLang, Cliometrics, Cliodynamics + +# Historical GDP with measurement uncertainty (BetLang) +gdp_1850_britain = DistnumberNormal(2100, 200) # Million pounds +gdp_1850_france = DistnumberNormal(1900, 250) + +# Growth model with uncertainty propagation +growth_trajectory = bet( + industrial_revolution_earlier = cliometrics_model(start=1750), + actual_history = cliometrics_model(start=1800), + industrial_revolution_later = cliometrics_model(start=1850) +) + +# Cliodynamic crisis prediction with uncertainty +crisis_prob = cliodynamic_crisis_probability( + elite_overproduction = DistnumberBeta(7, 3), # High uncertainty + state_capacity = DistnumberNormal(0.6, 0.15) +) +# => ImpreciseProbability([0.35, 0.65]) - wide uncertainty band +---- + +==== Use Case 3: Phylogenetic Uncertainty Quantification + +*Components:* BetLang + Cladistics + ZeroProb + +[source,julia] +---- +using BetLang, Cladistics, ZeroProb + +# Genetic distances with measurement error +distances = measure_genetic_distances_with_uncertainty() # BetLang DistnumberNormal + +# Build distribution over tree topologies +tree_distribution = bet_repeat(10_000) do + D = sample.(distances) + neighbor_joining(D) +end + +# Zero-probability analysis: "What if this exact tree?" +exact_tree = consensus_tree(tree_distribution) +relevance = zero_prob_relevance(tree_distribution, exact_tree) # ZeroProb measure +---- + +=== Package Interdependencies + +.... +BetLang.jl (core) +├── Distributions.jl (required) +├── StatsBase.jl (required) +├── BowtieRisk.jl (optional, recommended) +│ └── Uses BetLang uncertainty types +├── ZeroProb.jl (optional, theoretical) +│ └── Extends BetLang continuous distributions +├── Causals.jl (optional) +│ └── DempsterShafer ↔ BetLang.DempsterShafer +├── Cliometrics.jl (optional) +│ └── Uses BetLang number systems for historical data +├── Cliodynamics.jl (optional) +│ └── Uses BetLang Monte Carlo for stochastic models +├── Cladistics.jl (optional) +│ └── Uses BetLang for bootstrap/uncertainty +└── HackenbushGames.jl (optional) + └── SurrealAdvanced ↔ Hackenbush positions +.... + +=== Next Steps + +==== Immediate (v0.8 Planning) + +[arabic] +. *Survey your packages* - Read full docs for each +. *Identify API surface* - What functions should BetLang call? +. *Design adapters* - BetLang types ↔ your package types +. *Prototype* - Small proof-of-concept integrations + +==== Questions to Resolve + +[arabic] +. *BowtieRisk:* Should BetLang safety features be first-class barrier +types? +. *ZeroProb:* Should this be bundled with BetLang.jl or separate? +. *Causals:* Should Dempster-Shafer be unified between both packages? +. *Domain packages:* Generic uncertainty interface vs custom +integration? +. *Package naming:* `+BetLang.jl+` or `+BetLangCore.jl+` + +`+BetLangRisk.jl+` etc? + +==== Benefits of Integration + +*For BetLang:* - Instant domain-specific applications (risk, history, +phylogenetics) - Theoretical rigor (ZeroProb, formal verification) - +Real-world use cases for all 14 number systems + +*For Your Packages:* - Uncertainty quantification via BetLang number +systems - Monte Carlo simulation capabilities - Probabilistic extensions +to deterministic models - Cross-package interoperability + +*For Ecosystem:* - *Unified hyperpolymath Julia stack* - Probabilistic + +Causal + Risk + Domain expertise - Competitive with Turing.jl but more +specialized - Unique ternary philosophy + safety features + +=== Repository Organization + +==== Option A: Monorepo + +.... +hyperpolymath-julia/ +├── BetLang.jl/ +├── BowtieRisk.jl/ +├── ZeroProb.jl/ +├── Causals.jl/ +└── ... +.... + +==== Option B: Separate Repos with Registry + +.... +HyperpolymathJulia Registry +├── BetLang.jl → github.com/hyperpolymath/BetLang.jl +├── BowtieRisk.jl → github.com/hyperpolymath/BowtieRisk.jl +└── ... (current structure) +.... + +*Recommendation:* Keep separate repos, create `+HyperpolymathJulia+` +metapackage + +=== Conclusion + +Your existing Julia ecosystem is *highly synergistic* with BetLang! +Priority integrations: + +[arabic] +. *BowtieRisk.jl* (⭐⭐⭐) - Perfect fit for safety features +. *ZeroProb.jl* (⭐⭐⭐) - Theoretical foundations +. *Causals.jl* (⭐⭐) - Dempster-Shafer unification + +With these integrations, BetLang becomes the *first probabilistic +programming language with integrated risk analysis and causal +inference*. diff --git a/docs/hyperpolymath-julia-integration.md b/docs/hyperpolymath-julia-integration.md deleted file mode 100644 index 0e95f7a..0000000 --- a/docs/hyperpolymath-julia-integration.md +++ /dev/null @@ -1,560 +0,0 @@ - -# BetLang Integration with Hyperpolymath Julia Packages - -**Status:** Planning Phase (2026-02-07) -**Purpose:** Connect betlang to existing hyperpolymath Julia ecosystem - -## IMPORTANT: Priority Clarification - -**This document covers Tier 1 (Extensions) - these are additions to the core Julia ecosystem, not replacements.** - -### Priority Order (Most → Least Critical): - -**Tier 0 (Foundation) - MOST IMPORTANT:** -- Distributions.jl, StatsBase.jl, Turing.jl, Optim.jl, etc. -- See `julia-backend-design.md` for comprehensive Tier 0 coverage -- **These must be implemented first** - they're mature, essential packages - -**Tier 1 (This Document) - Valuable Extensions:** -- BowtieRisk.jl, ZeroProb.jl, Causals.jl, domain packages -- Built **on top of** Tier 0 foundation -- Add domain-specific capabilities - -**Tier 2+ - Specialized:** -- As needed for specific use cases - -## Overview - -Once BetLang has solid integration with standard Julia packages (Tier 0), it can *also* leverage your existing hyperpolymath packages to create a comprehensive probabilistic programming + risk analysis + causal inference ecosystem. - -## Your Julia Packages (Discovered) - -### Tier 1: Direct BetLang Integration (High Synergy) - -#### 1. **BowtieRisk.jl** - Bowtie Risk Modeling ⭐⭐⭐ -**Purpose:** Hazard analysis with threats, barriers, and consequences -**Current deps:** Distributions.jl, JSON3 - -**Integration with BetLang:** - -**Synergy Points:** -- **Risk-of-Ruin Protection** → Bowtie consequence analysis -- **Monte Carlo Simulation** → Bowtie barrier effectiveness -- **Probability Validation** → Dutch book prevention for threat probabilities - -**Example Integration:** -```julia -using BetLang, BowtieRisk - -# Define a gambling bowtie model -model = BowtieModel( - hazard = "Financial Ruin", - threats = [ - Threat("High Stakes", probability=0.3), - Threat("Rapid Betting", probability=0.4), - Threat("Chasing Losses", probability=0.5) - ], - barriers = [ - # BetLang safety features as barriers! - Barrier("Dutch Book Prevention", effectiveness=0.95), - Barrier("Risk-of-Ruin Check", effectiveness=0.90), - Barrier("Cool-Off Mechanism", effectiveness=0.85), - Barrier("Kelly Criterion", effectiveness=0.92) - ], - consequences = [ - Consequence("Bankruptcy", severity=10.0, probability=0.8) - ] -) - -# Run BetLang-powered Monte Carlo simulation -results = simulate_bowtie(model, n=100_000) -# Uses BetLang's bet primitives for sampling - -# Validate no Dutch books in threat probabilities -threats_probs = [t.probability for t in model.threats] -validate_dutch_book(threats_probs) # BetLang function -``` - -**New Features to Add to BowtieRisk:** -- `BetLangBarrier` type - barriers implemented as betlang functions -- `gambling_harm_reduction_template()` - pre-built model for gambling risks -- `monte_carlo_with_betlang()` - use betlang's uncertainty numbers - -#### 2. **ZeroProb.jl** - Zero-Probability Event Handling ⭐⭐⭐ -**Purpose:** Handle continuous probability spaces with measure-zero events -**Current deps:** Distributions.jl, StatsBase, Plots, Makie - -**Integration with BetLang:** - -**Synergy Points:** -- **Continuous Distributions** → betlang's DistnumberNormal, DistnumberBeta -- **Theoretical Rigor** → Formal foundations for betlang's probability theory -- **Edge Cases** → Handling degenerate cases in number systems - -**Example Integration:** -```julia -using BetLang, ZeroProb - -# BetLang distribution as continuous space -height_dist = DistnumberNormal(170, 10) - -# Query exact point (has zero probability but can occur) -point = 175.5 -density = density_ratio(height_dist, point) # ZeroProb relevance measure - -# ε-neighborhood probability (BetLang + ZeroProb) -prob_near_175 = epsilon_neighborhood_prob(height_dist, 175.5, ε=0.1) -# P(175.4 < height < 175.6) ≈ 0.008 - -# Hausdorff measure for zero-probability sets -fractal_support = hausdorff_measure( - betlang_distribution_support(height_dist), - dimension=1.5 -) -``` - -**New Features to Add to ZeroProb:** -- `DistnumberAdapter` - wrap betlang number systems -- `zero_prob_bet()` - betting on measure-zero events -- `continuous_game_theory()` - combine with betlang game theory examples - -#### 3. **Causals.jl** - Causal Inference ⭐⭐ -**Purpose:** Dempster-Shafer, DAGs, do-calculus, counterfactuals -**Current deps:** (need to check) - -**Integration with BetLang:** - -**Synergy Points:** -- **Dempster-Shafer** → betlang already has DempsterShafer number system! -- **Counterfactuals** → "What if I had bet differently?" analysis -- **Causal DAGs** → Model causal relationships in probabilistic systems -- **Do-Calculus** → Interventions in gambling scenarios - -**Example Integration:** -```julia -using BetLang, Causals - -# Model causal structure of gambling outcomes -dag = CausalDAG() -add_nodes!(dag, [:stake, :skill, :luck, :outcome, :bankroll]) -add_edges!(dag, [ - :stake => :outcome, - :skill => :outcome, - :luck => :outcome, - :outcome => :bankroll -]) - -# Counterfactual: "What if I had bet less?" -actual_outcome = simulate_bet(stake=1000, skill=0.6, luck=0.4) -counterfactual = intervene(dag, :stake => 500) -cf_outcome = simulate_bet(counterfactual) - -# Compare using betlang's number systems -diff = imprecise_probability( - lower = min(actual_outcome, cf_outcome), - upper = max(actual_outcome, cf_outcome) -) -``` - -**New Features to Add to Causals:** -- `gambling_dag_template()` - standard causal models for gambling -- `betlang_counterfactual()` - counterfactual reasoning with bets -- `causal_number_systems()` - propagate uncertainty through DAGs - -### Tier 2: Domain-Specific Extensions - -#### 4. **HackenbushGames.jl** - Hackenbush Game Theory ⭐ -**Purpose:** Combinatorial game theory (surreal numbers!) -**Current deps:** None - -**Integration with BetLang:** - -**Synergy Points:** -- **Surreal Numbers** → betlang has SurrealAdvanced number system! -- **Game Theory** → betlang has `examples/game-theory.rkt` -- **Ternary Games** → Hackenbush with 3-player variants - -**Example Integration:** -```julia -using BetLang, HackenbushGames - -# Hackenbush position as surreal number -position = analyze_hackenbush(blue_edges, red_edges) -surreal_value = to_surreal(position) # e.g., {0,1|2,3} - -# Convert to BetLang SurrealAdvanced -betlang_surreal = SurrealAdvanced( - left_set = surreal_value.left, - right_set = surreal_value.right -) - -# Use in betlang game theory -game = ternary_game( - payoffs = betlang_surreal, - players = 3 -) -nash_equilibrium(game) -``` - -**New Features to Add to HackenbushGames:** -- `ternary_hackenbush()` - 3-player variant (blue/red/green) -- `surreal_to_betlang()` - convert to BetLang number system -- `probabilistic_hackenbush()` - edges fail with probability - -#### 5. **Cliometrics.jl** - Quantitative Economic History ⭐ -**Purpose:** Growth analysis, convergence, institutional quality -**Current deps:** (recently implemented) - -**Integration with BetLang:** - -**Synergy Points:** -- **Uncertainty in Historical Data** → betlang number systems -- **Monte Carlo Historical Simulation** → counterfactual histories -- **Economic Risk Analysis** → combine with BowtieRisk - -**Example Integration:** -```julia -using BetLang, Cliometrics - -# Historical GDP with uncertainty -gdp_1900 = DistnumberNormal(1000, 100) # BetLang uncertainty -gdp_2000 = DistnumberNormal(50000, 5000) - -# Growth rate with error propagation -growth_rate = calculate_growth_rates( - [gdp_1900, gdp_2000], - years = [1900, 2000] -) -# Result: DistnumberNormal(3.9%, 0.2%) - uncertainty propagated! - -# Counterfactual: "What if industrial revolution started earlier?" -counterfactual_trajectory = bet( - scenario_a = early_industrialization_path(), - scenario_b = actual_path(), - scenario_c = late_industrialization_path() -) -``` - -#### 6. **Cliodynamics.jl** - Mathematical Historical Dynamics ⭐ -**Purpose:** Turchin's models, elite overproduction, secular cycles -**Current deps:** (recently implemented) - -**Integration with BetLang:** - -**Synergy Points:** -- **Stochastic Historical Models** → betlang Monte Carlo -- **Uncertainty in Parameters** → betlang number systems -- **Probabilistic Forecasting** → crisis prediction with confidence - -**Example Integration:** -```julia -using BetLang, Cliodynamics - -# Model parameters with uncertainty -elite_size = DistnumberBeta(8, 2) # High elite concentration -state_capacity = DistnumberNormal(0.7, 0.1) # Weakening state - -# Run probabilistic cliodynamic simulation -trajectories = bet_repeat(10_000) do - simulate_secular_cycle( - elite_size = sample(elite_size), - state_capacity = sample(state_capacity), - years = 100 - ) -end - -# Crisis probability distribution -crisis_probs = [trajectory.crisis_probability for trajectory in trajectories] -mean_crisis_prob = mean(crisis_probs) -ci_95 = percentile(crisis_probs, [2.5, 97.5]) -``` - -#### 7. **Cladistics.jl** - Phylogenetic Analysis -**Purpose:** UPGMA, neighbor-joining, maximum parsimony, bootstrap -**Current deps:** (recently implemented) - -**Integration with BetLang:** - -**Synergy Points:** -- **Bootstrap Support** → Monte Carlo uncertainty quantification -- **Uncertain Branch Lengths** → betlang number systems -- **Probabilistic Trees** → distribution over tree topologies - -**Example Integration:** -```julia -using BetLang, Cladistics - -# Distance matrix with measurement uncertainty -distances = [ - DistnumberNormal(0, 0) DistnumberNormal(3, 0.5) DistnumberNormal(5, 0.7); - DistnumberNormal(3, 0.5) DistnumberNormal(0, 0) DistnumberNormal(4, 0.6); - DistnumberNormal(5, 0.7) DistnumberNormal(4, 0.6) DistnumberNormal(0, 0) -] - -# Build tree with uncertainty propagation -trees = bet_repeat(1000) do - D = sample.(distances) # Sample from each distance uncertainty - upgma(D) -end - -# Consensus tree with confidence intervals -consensus = consensus_tree(trees, threshold=0.7) -``` - -### Tier 3: Theoretical Foundations - -#### 8. **KnotTheory.jl** - Knot Theory -**Purpose:** Knot invariants, Jones polynomial, etc. -**Integration:** Possibly via surreal numbers? (Hackenbush-style) - -#### 9. **Axiology.jl** - Value Theory -**Purpose:** Formal value systems -**Integration:** Decision theory under uncertainty (utility + betlang) - -#### 10. **SMTLib.jl** - SMT Solver Interface -**Purpose:** Satisfiability Modulo Theories -**Integration:** Verify betlang safety properties formally - -#### 11. **ProvenCrypto.jl** - Proven Cryptography -**Purpose:** Formally verified crypto -**Integration:** Secure random number generation for betlang - -## Proposed Architecture - -### Unified Ecosystem: BetLang + Hyperpolymath.jl - -``` - ┌──────────────┐ - │ BetLang │ - │ (Racket) │ - └──────┬───────┘ - │ Compiler - ▼ - ┌──────────────────────────┐ - │ BetLang.jl (Julia Core) │ - └──────┬───────────────────┘ - │ - ┌──────────┼──────────┬──────────┐ - ▼ ▼ ▼ ▼ - ┌─────────┐ ┌──────────┐ ┌────────┐ ┌────────┐ - │BowtieRisk│ │ ZeroProb │ │Causals │ │Clio- │ - │ .jl │ │ .jl │ │ .jl │ │metrics │ - └─────────┘ └──────────┘ └────────┘ └────────┘ - │ │ │ │ - └────────────┴───────────┴───────────┘ - │ - ┌───────▼────────┐ - │ Hyperpolymath │ - │ Julia Stack │ - └────────────────┘ -``` - -## Implementation Phases - -### Phase 1: Core BetLang.jl (v0.8.0) -- Compile betlang → Julia -- Integrate Distributions.jl, StatsBase.jl -- Basic number systems - -### Phase 2: Risk Integration (v0.9.0) -- **BowtieRisk.jl** integration - - BetLang safety features as barriers - - Monte Carlo with betlang uncertainty - - Gambling harm reduction templates - -- **ZeroProb.jl** integration - - Handle continuous betlang distributions - - Measure-zero event reasoning - - Theoretical rigor for edge cases - -### Phase 3: Causal + Domain Extensions (v1.0.0) -- **Causals.jl** integration - - Dempster-Shafer ↔ betlang number system - - Counterfactual gambling analysis - - Causal DAGs for probabilistic models - -- **Domain packages** (Cliometrics, Cliodynamics, Cladistics) - - Use betlang for uncertainty quantification - - Monte Carlo historical simulations - - Probabilistic phylogenies - -- **Game theory** (HackenbushGames) - - Surreal number interop - - Ternary game variants - - Combinatorial game theory + probability - -### Phase 4: Theoretical Foundations (v1.1.0+) -- **SMTLib.jl** - Verify safety properties -- **ProvenCrypto.jl** - Secure RNG -- **KnotTheory.jl** - Advanced surreal arithmetic - -## Concrete Use Cases - -### Use Case 1: Gambling Risk Management System -**Components:** BetLang + BowtieRisk + Causals - -```julia -using BetLang, BowtieRisk, Causals - -# 1. Causal model of gambling addiction -addiction_dag = gambling_addiction_dag() # From Causals.jl - -# 2. Bowtie risk model -risk_model = gambling_bowtie( - barriers = betlang_safety_barriers() # Dutch book, risk-of-ruin, cool-off -) - -# 3. Run counterfactual analysis -actual = simulate_gambling_session( - bets_per_hour = 60, - stake_size = 100 -) - -counterfactual_with_cooloff = intervene( - addiction_dag, - :cool_off_seconds => 30 # BetLang cool-off -) - -# 4. Compare outcomes -risk_reduction = bowtie_probability_reduction( - risk_model, - actual vs counterfactual_with_cooloff -) -# => "Cool-off reduces ruin risk by 73%" -``` - -### Use Case 2: Historical Counterfactual with Uncertainty -**Components:** BetLang + Cliometrics + Cliodynamics - -```julia -using BetLang, Cliometrics, Cliodynamics - -# Historical GDP with measurement uncertainty (BetLang) -gdp_1850_britain = DistnumberNormal(2100, 200) # Million pounds -gdp_1850_france = DistnumberNormal(1900, 250) - -# Growth model with uncertainty propagation -growth_trajectory = bet( - industrial_revolution_earlier = cliometrics_model(start=1750), - actual_history = cliometrics_model(start=1800), - industrial_revolution_later = cliometrics_model(start=1850) -) - -# Cliodynamic crisis prediction with uncertainty -crisis_prob = cliodynamic_crisis_probability( - elite_overproduction = DistnumberBeta(7, 3), # High uncertainty - state_capacity = DistnumberNormal(0.6, 0.15) -) -# => ImpreciseProbability([0.35, 0.65]) - wide uncertainty band -``` - -### Use Case 3: Phylogenetic Uncertainty Quantification -**Components:** BetLang + Cladistics + ZeroProb - -```julia -using BetLang, Cladistics, ZeroProb - -# Genetic distances with measurement error -distances = measure_genetic_distances_with_uncertainty() # BetLang DistnumberNormal - -# Build distribution over tree topologies -tree_distribution = bet_repeat(10_000) do - D = sample.(distances) - neighbor_joining(D) -end - -# Zero-probability analysis: "What if this exact tree?" -exact_tree = consensus_tree(tree_distribution) -relevance = zero_prob_relevance(tree_distribution, exact_tree) # ZeroProb measure -``` - -## Package Interdependencies - -``` -BetLang.jl (core) -├── Distributions.jl (required) -├── StatsBase.jl (required) -├── BowtieRisk.jl (optional, recommended) -│ └── Uses BetLang uncertainty types -├── ZeroProb.jl (optional, theoretical) -│ └── Extends BetLang continuous distributions -├── Causals.jl (optional) -│ └── DempsterShafer ↔ BetLang.DempsterShafer -├── Cliometrics.jl (optional) -│ └── Uses BetLang number systems for historical data -├── Cliodynamics.jl (optional) -│ └── Uses BetLang Monte Carlo for stochastic models -├── Cladistics.jl (optional) -│ └── Uses BetLang for bootstrap/uncertainty -└── HackenbushGames.jl (optional) - └── SurrealAdvanced ↔ Hackenbush positions -``` - -## Next Steps - -### Immediate (v0.8 Planning) -1. **Survey your packages** - Read full docs for each -2. **Identify API surface** - What functions should BetLang call? -3. **Design adapters** - BetLang types ↔ your package types -4. **Prototype** - Small proof-of-concept integrations - -### Questions to Resolve -1. **BowtieRisk:** Should BetLang safety features be first-class barrier types? -2. **ZeroProb:** Should this be bundled with BetLang.jl or separate? -3. **Causals:** Should Dempster-Shafer be unified between both packages? -4. **Domain packages:** Generic uncertainty interface vs custom integration? -5. **Package naming:** `BetLang.jl` or `BetLangCore.jl` + `BetLangRisk.jl` etc? - -### Benefits of Integration - -**For BetLang:** -- Instant domain-specific applications (risk, history, phylogenetics) -- Theoretical rigor (ZeroProb, formal verification) -- Real-world use cases for all 14 number systems - -**For Your Packages:** -- Uncertainty quantification via BetLang number systems -- Monte Carlo simulation capabilities -- Probabilistic extensions to deterministic models -- Cross-package interoperability - -**For Ecosystem:** -- **Unified hyperpolymath Julia stack** -- Probabilistic + Causal + Risk + Domain expertise -- Competitive with Turing.jl but more specialized -- Unique ternary philosophy + safety features - -## Repository Organization - -### Option A: Monorepo -``` -hyperpolymath-julia/ -├── BetLang.jl/ -├── BowtieRisk.jl/ -├── ZeroProb.jl/ -├── Causals.jl/ -└── ... -``` - -### Option B: Separate Repos with Registry -``` -HyperpolymathJulia Registry -├── BetLang.jl → github.com/hyperpolymath/BetLang.jl -├── BowtieRisk.jl → github.com/hyperpolymath/BowtieRisk.jl -└── ... (current structure) -``` - -**Recommendation:** Keep separate repos, create `HyperpolymathJulia` metapackage - -## Conclusion - -Your existing Julia ecosystem is **highly synergistic** with BetLang! Priority integrations: - -1. **BowtieRisk.jl** (⭐⭐⭐) - Perfect fit for safety features -2. **ZeroProb.jl** (⭐⭐⭐) - Theoretical foundations -3. **Causals.jl** (⭐⭐) - Dempster-Shafer unification - -With these integrations, BetLang becomes the **first probabilistic programming language with integrated risk analysis and causal inference**. diff --git a/docs/julia-backend-design.adoc b/docs/julia-backend-design.adoc new file mode 100644 index 0000000..b5be733 --- /dev/null +++ b/docs/julia-backend-design.adoc @@ -0,0 +1,602 @@ +== BetLang Julia Backend Design (v0.8) + +*Status:* Planning Phase (2026-02-07) *Target:* v0.8.0 milestone + +=== Priority Summary + +*Integration priority (most → least important):* + +[arabic] +. *Tier 0 (Foundation):* Standard Julia ecosystem +* Distributions.jl, StatsBase.jl, Turing.jl ← *MOST CRITICAL* +* These are mature, widely-used, essential for any probabilistic +programming +. *Tier 1 (Extensions):* Hyperpolymath packages +* BowtieRisk.jl, ZeroProb.jl, Causals.jl, domain packages +* Valuable additions that extend BetLang’s capabilities +. *Tier 2+:* Specialized packages +* As needed for specific use cases + +*This document covers both, but Tier 0 packages are the foundation that +must be implemented first.* + +=== Architecture Overview + +BetLang → Julia compiler that translates Racket betlang to +high-performance Julia code, leveraging Julia’s scientific computing +ecosystem. + +.... +┌─────────────────┐ +│ BetLang (Racket)│ +│ Source Code │ +└────────┬────────┘ + │ Parse/Analyze + ▼ +┌─────────────────┐ +│ AST/IR │ +│ Representation │ +└────────┬────────┘ + │ Code Generation + ▼ +┌─────────────────┐ +│ Julia Code │ +│ (Generated) │ +└────────┬────────┘ + │ Runtime + ▼ +┌─────────────────┐ +│ Julia Packages │ +│ (Distributions,│ +│ StatsBase, │ +│ etc.) │ +└─────────────────┘ +.... + +=== Priority Structure + +*Tier 0 (Foundation):* Standard Julia ecosystem - MOST IMPORTANT *Tier 1 +(Extensions):* Hyperpolymath packages - valuable additions *Tier 2+:* +Specialized use cases + +''''' + +=== Tier 0: Core Julia Ecosystem (FOUNDATION - MOST CRITICAL) + +==== Must-Have Integrations (Standard Julia Packages) + +===== 1. *Distributions.jl* - Probability Distributions + +*Why:* Authoritative Julia package for probability distributions *Maps +to:* All betlang distribution operations + +*Integration points:* - `+bet+` → Sample from +`+Categorical([1/3, 1/3, 1/3])+` - `+bet/weighted+` → Sample from +`+Categorical(normalize(weights))+` - `+lib/distributions.rkt+` → Direct +Distributions.jl types - Number systems (DistnumberNormal, +DistnumberBeta) → `+Normal()+`, `+Beta()+` + +*Distributions.jl types to use:* + +[source,julia] +---- +# Discrete +Categorical, Bernoulli, Binomial, Geometric, Poisson, Multinomial + +# Continuous +Normal, Exponential, Gamma, Beta, Uniform, ChiSquare, TDist, +Weibull, Pareto, Cauchy, Laplace, Dirichlet + +# Functions +pdf(), cdf(), quantile(), mean(), var(), entropy(), fit() +---- + +===== 2. *StatsBase.jl* - Statistical Utilities + +*Why:* Comprehensive statistical functions *Maps to:* +`+lib/statistics.rkt+` (180+ functions) + +*Integration points:* - Descriptive stats: `+mean()+`, `+median()+`, +`+mode()+`, `+var()+`, `+std()+` - Correlation: `+cor()+`, `+cov()+`, +`+corspearman()+` - Sampling: `+sample()+`, `+wsample()+` (weighted +sampling) - Histograms: `+fit(Histogram, data)+` - Percentiles: +`+percentile()+`, `+quantile()+` + +===== 3. *Random.jl* (Stdlib) - RNG + +*Why:* Deterministic seeding for reproducibility *Maps to:* +`+bet-with-seed+`, all randomness + +*Integration points:* + +[source,julia] +---- +using Random +rng = MersenneTwister(seed) +rand(rng, Categorical([0.3, 0.4, 0.3])) +---- + +==== Tier 2: Number Systems Support + +===== 4. *IntervalArithmetic.jl* - Interval Arithmetic + +*Why:* Native interval types for AffineNumber *Maps to:* +`+AffineNumber+` in `+lib/number-systems.rkt+` + +[source,julia] +---- +using IntervalArithmetic +x = 18..22 # [18, 22]°C +y = 20..25 +z = x + y # Automatic interval arithmetic +---- + +===== 5. *Measurements.jl* - Uncertainty Propagation + +*Why:* Automatic uncertainty tracking (like DistNumber) *Maps to:* +`+DistnumberNormal+` operations + +[source,julia] +---- +using Measurements +height = 170 ± 10 # 170cm ± 10cm +weight = 75 ± 5 +bmi = weight / (height/100)^2 # Automatic error propagation +---- + +===== 6. *MonteCarloMeasurements.jl* - Particle-Based Uncertainty + +*Why:* Alternative to analytic uncertainty propagation *Maps to:* +Advanced number system operations + +[source,julia] +---- +using MonteCarloMeasurements +x = 5.0 ± 1.0 # Creates particle distribution +y = 3.0 ± 0.5 +z = x * y # Monte Carlo propagation +---- + +==== Tier 3: Bayesian/MCMC Integration + +===== 7. *Turing.jl* - Probabilistic Programming + +*Why:* Sister language! Can interoperate or compete *Maps to:* +`+lib/bayesian.rkt+` advanced features + +*Comparison:* + +[source,julia] +---- +# Turing.jl model +@model coin_flip(y) = begin + p ~ Beta(1, 1) + y .~ Bernoulli(p) +end + +# BetLang equivalent (compiled to Julia using Turing backend) +(define model + (bayesian-model + (prior 'p (beta-dist 1 1)) + (likelihood 'y (bernoulli-dist p)))) +---- + +*Decision:* Turing.jl could be: - *Option A:* Backend for betlang +Bayesian inference (use Turing’s MCMC) - *Option B:* Alternative target +(compile betlang → Turing.jl models) - *Option C:* Competition (betlang +implements own MCMC, benchmarks against Turing) + +*Recommendation:* Option A - Use Turing.jl as backend for MCMC samplers + +===== 8. *AdvancedMH.jl* - MCMC Samplers + +*Why:* Metropolis-Hastings, Gibbs, HMC implementations *Maps to:* +`+lib/bayesian.rkt+` MCMC functions + +===== 9. *Gen.jl* - Alternative Probabilistic Programming + +*Why:* MIT’s probabilistic programming framework *Maps to:* Potential +alternative backend + +==== Tier 4: Financial/Risk Applications + +===== 10. *TimeSeries.jl* - Time Series Analysis + +*Why:* Financial modeling with temporal data *Maps to:* +`+examples/finance.rkt+` time series functions + +===== 11. *Optim.jl* - Optimization + +*Why:* Portfolio optimization, parameter fitting *Maps to:* +`+lib/optimization.rkt+` + +[source,julia] +---- +using Optim +# Kelly criterion optimization +f(x) = -expected_log_wealth(x, probabilities, payoffs) +result = optimize(f, 0.0, 1.0) # Optimal bet fraction +---- + +===== 12. *JuMP.jl* - Mathematical Optimization + +*Why:* Constrained optimization (risk limits, portfolio constraints) +*Maps to:* Advanced portfolio optimization + +[source,julia] +---- +using JuMP, HiGHS +model = Model(HiGHS.Optimizer) +@variable(model, 0 <= stake <= max_stake) +@objective(model, Max, expected_value(stake)) +@constraint(model, risk_of_ruin(stake) <= 0.05) +optimize!(model) +---- + +==== Tier 5: Advanced Mathematics + +===== 13. *SpecialFunctions.jl* - Special Functions + +*Why:* Gamma, Beta, Bessel functions for distributions *Maps to:* +Distribution internals + +===== 14. *LinearAlgebra.jl* (Stdlib) - Matrix Operations + +*Why:* Multivariate distributions, covariance matrices *Maps to:* +Multivariate statistics + +===== 15. *Symbolics.jl* - Symbolic Math + +*Why:* Symbolic uncertainty propagation *Maps to:* Advanced number +systems (SurrealAdvanced, PAdicAdvanced) + +=== Compilation Strategy + +==== Phase 1: Direct Translation (Simple) + +*Input (BetLang):* + +[source,racket] +---- +(bet 'heads 'tails 'edge) +---- + +*Output (Julia):* + +[source,julia] +---- +using Distributions, Random +sample(Categorical([1/3, 1/3, 1/3]), 1)[1] # Returns :heads, :tails, or :edge +---- + +==== Phase 2: Function Translation + +*Input (BetLang):* + +[source,racket] +---- +(define (coin-flip-game trials) + (bet-parallel trials 'heads 'tails 'edge)) +---- + +*Output (Julia):* + +[source,julia] +---- +function coin_flip_game(trials::Int) + dist = Categorical([1/3, 1/3, 1/3]) + [sample(dist) for _ in 1:trials] +end +---- + +==== Phase 3: Number System Mapping + +*Input (BetLang):* + +[source,racket] +---- +(define height (make-distnumber-normal 170 10)) +(define weight (make-distnumber-normal 75 5)) +(define bmi (distnumber-div weight (distnumber-mul height height))) +---- + +*Output (Julia):* + +[source,julia] +---- +using Measurements +height = 170.0 ± 10.0 +weight = 75.0 ± 5.0 +bmi = weight / height^2 +---- + +==== Phase 4: Safety Features Integration + +*Dutch Book Prevention:* + +[source,julia] +---- +function validate_probabilities(probs::Vector{Float64}; tol=1e-6) + total = sum(probs) + if abs(total - 1.0) > tol + throw(DutchBookError("Probabilities sum to $total, not 1.0")) + end + return probs +end +---- + +*Risk-of-Ruin Protection:* + +[source,julia] +---- +using Distributions +function kelly_fraction(p::Float64, b::Float64, kelly_fraction::Float64=0.25) + edge = p * b - (1 - p) + full_kelly = edge / b + return full_kelly * kelly_fraction # Fractional Kelly +end +---- + +*Cool-Off Mechanism:* + +[source,julia] +---- +mutable struct CoolOffTracker + last_bet_time::Float64 + cool_off_seconds::Float64 + violations::Int +end + +function bet_with_cooloff!(tracker::CoolOffTracker, bet_fn::Function) + now = time() + elapsed = now - tracker.last_bet_time + if elapsed < tracker.cool_off_seconds + tracker.violations += 1 + error("Cool-off active. Wait $(tracker.cool_off_seconds - elapsed)s") + end + tracker.last_bet_time = now + return bet_fn() +end +---- + +=== Performance Benefits + +==== Why Julia Backend? + +[arabic] +. *Speed:* 10-100x faster than Racket for numerical code +. *Ecosystem:* Mature scientific computing libraries +. *Integration:* Easy FFI with C/Fortran/Python +. *JIT:* Compiled to native code, not interpreted +. *Parallelism:* Native multi-threading, GPU support +. *Type Stability:* Optional static typing for performance + +==== Benchmarks (Expected) + +[cols=",,,",options="header",] +|=== +|Operation |Racket |Julia |Speedup +|Monte Carlo (1M samples) |15s |0.2s |75x +|Matrix operations |8s |0.1s |80x +|MCMC sampling (10K) |120s |2s |60x +|Distribution sampling |5s |0.05s |100x +|=== + +=== Implementation Roadmap + +==== Step 1: Minimal Viable Backend (2 weeks) + +* [ ] Parse betlang AST from Racket +* [ ] Generate Julia code for basic `+bet+` operations +* [ ] Map to Distributions.jl Categorical +* [ ] Write generated code to `+.jl+` file +* [ ] Execute via `+julia generated.jl+` + +==== Step 2: Core Language (4 weeks) + +* [ ] Translate all bet primitives (bet, bet/weighted, bet/conditional) +* [ ] Function definitions and calls +* [ ] Variable bindings (let, define) +* [ ] Control flow (if, cond) +* [ ] List operations (map, filter, fold) + +==== Step 3: Standard Library (6 weeks) + +* [ ] Map `+lib/statistics.rkt+` → StatsBase.jl +* [ ] Map `+lib/distributions.rkt+` → Distributions.jl +* [ ] Map `+lib/bayesian.rkt+` → Turing.jl/AdvancedMH.jl +* [ ] Map `+lib/optimization.rkt+` → Optim.jl + +==== Step 4: Number Systems (4 weeks) + +* [ ] DistnumberNormal → Measurements.jl +* [ ] AffineNumber → IntervalArithmetic.jl +* [ ] BayesianNumber → Custom type +* [ ] RiskNumber → Custom type with Distributions.jl +* [ ] Other 10 systems → Custom types + +==== Step 5: Safety Features (2 weeks) + +* [ ] Dutch book validation (Julia-side checks) +* [ ] Risk-of-ruin protection (Kelly criterion) +* [ ] Cool-off mechanism (timer struct) + +==== Step 6: Performance Optimization (4 weeks) + +* [ ] Type stability analysis +* [ ] Precompilation of common operations +* [ ] GPU acceleration for Monte Carlo +* [ ] Multi-threading for parallel bets + +*Total Estimated Time:* 22 weeks (~5 months) + +=== Build System Integration + +==== Justfile Recipes + +[source,just] +---- +# Build Julia backend +build-julia: + racket compiler/betlang-to-julia.rkt examples/basic.bet --output build/basic.jl + julia build/basic.jl + +# Run tests with Julia backend +test-julia: + racket tests/julia-backend-tests.rkt + julia --project tests/run_tests.jl + +# Benchmark Julia vs Racket +benchmark: + racket benchmarks/performance.rkt --backend racket + racket compiler/betlang-to-julia.rkt benchmarks/performance.bet + julia benchmarks/performance.jl + julia benchmarks/compare.jl +---- + +=== Package Structure + +.... +betlang-julia/ +├── Project.toml # Julia package metadata +├── src/ +│ ├── BetLang.jl # Main module +│ ├── core.jl # Core bet primitives +│ ├── distributions.jl # Distribution mappings +│ ├── statistics.jl # Statistical functions +│ ├── bayesian.jl # Bayesian inference +│ ├── optimization.jl # Optimization algorithms +│ ├── number_systems.jl # 14 number systems +│ └── safety.jl # Safety features +├── test/ +│ └── runtests.jl +└── examples/ + ├── basic.jl + ├── finance.jl + └── monte_carlo.jl +.... + +==== Project.toml + +[source,toml] +---- +name = "BetLang" +uuid = "..." +authors = ["Jonathan D.A. Jewell "] +version = "0.8.0" + +[deps] +Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f" +StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" +Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" +IntervalArithmetic = "d1acc4aa-44c8-5952-acd4-ba5d80a2a253" +Measurements = "eff96d63-e80a-5855-80a2-b1b0885c5ab7" +Turing = "fce5fe82-541a-59a6-adf8-730c64b5f9a0" +Optim = "429524aa-4258-5aef-a3af-852621145aeb" + +[compat] +julia = "1.9" +Distributions = "0.25" +StatsBase = "0.34" +---- + +=== Integration with Existing Ecosystem + +==== Aggregate Library Bridge + +*Goal:* BetLang number systems → Aggregate library (via Julia FFI) + +[source,julia] +---- +# BetLang.jl provides C ABI for aggregate-library +function distnumber_to_c_array(dn::DistnumberNormal) + samples = [rand(Normal(dn.mean, dn.std)) for _ in 1:1000] + return pointer(samples), length(samples) +end +---- + +==== R Integration (Future) + +Via RCall.jl: + +[source,julia] +---- +using RCall +R""" +library(betlang) # R package wrapping BetLang.jl +result <- bet(c('heads', 'tails', 'edge')) +""" +---- + +==== Python Integration (Future) + +Via PyCall.jl or PythonCall.jl: + +[source,python] +---- +import juliacall +betlang = juliacall.newmodule("BetLang") +result = betlang.bet(['heads', 'tails', 'edge']) +---- + +=== Alternatives Considered + +==== 1. Compile to C/Rust instead? + +*Pros:* Maximum performance *Cons:* No ecosystem, must reimplement +everything *Verdict:* Julia is better - ecosystem + performance + +==== 2. Use Turing.jl directly instead of betlang? + +*Pros:* Mature, well-tested *Cons:* Doesn’t have ternary philosophy, +safety features, or number systems *Verdict:* BetLang offers unique +value, can use Turing as backend + +==== 3. Keep Racket-only? + +*Pros:* Simplest, already works *Cons:* Too slow for large-scale +simulations (millions of samples) *Verdict:* Julia backend is valuable +for performance-critical users + +=== Success Metrics + +==== v0.8.0 Definition of Done + +* [ ] Compile basic betlang programs to Julia +* [ ] All `+examples/*.rkt+` files compile and run +* [ ] Performance: 50x+ speedup on Monte Carlo benchmarks +* [ ] Integration: Distributions.jl, StatsBase.jl working +* [ ] Tests: 100% of Racket test suite passes on Julia backend +* [ ] Documentation: Tutorial for using Julia backend + +==== Performance Goals + +* Monte Carlo (1M samples): < 1 second +* MCMC sampling (10K iterations): < 5 seconds +* Portfolio optimization: < 100ms +* Distribution operations: < 1μs per sample + +=== Questions for User + +[arabic] +. *Priority order:* Which tier of packages should we focus on first? +. *Turing.jl relationship:* Backend, target, or competitor? +. *Number systems:* Implement all 14 in Julia, or just core 5? +. *Timeline:* Is 5 months acceptable, or should we compress? +. *Use cases:* What specific use cases need Julia performance? + +=== References + +* Distributions.jl: https://juliastats.org/Distributions.jl/stable/ +* StatsBase.jl: https://juliastats.org/StatsBase.jl/stable/ +* Turing.jl: https://turinglang.org/stable/ +* Measurements.jl: +https://juliaphysics.github.io/Measurements.jl/stable/ +* IntervalArithmetic.jl: +https://juliaintervals.github.io/IntervalArithmetic.jl/stable/ + +''''' + +*Next Steps:* 1. Get feedback on architecture 2. Prototype minimal +compiler (Step 1) 3. Benchmark proof-of-concept 4. Iterate based on +performance data diff --git a/docs/julia-backend-design.md b/docs/julia-backend-design.md deleted file mode 100644 index d6426fb..0000000 --- a/docs/julia-backend-design.md +++ /dev/null @@ -1,542 +0,0 @@ - -# BetLang Julia Backend Design (v0.8) - -**Status:** Planning Phase (2026-02-07) -**Target:** v0.8.0 milestone - -## Priority Summary - -**Integration priority (most → least important):** - -1. **Tier 0 (Foundation):** Standard Julia ecosystem - - Distributions.jl, StatsBase.jl, Turing.jl ← **MOST CRITICAL** - - These are mature, widely-used, essential for any probabilistic programming - -2. **Tier 1 (Extensions):** Hyperpolymath packages - - BowtieRisk.jl, ZeroProb.jl, Causals.jl, domain packages - - Valuable additions that extend BetLang's capabilities - -3. **Tier 2+:** Specialized packages - - As needed for specific use cases - -**This document covers both, but Tier 0 packages are the foundation that must be implemented first.** - -## Architecture Overview - -BetLang → Julia compiler that translates Racket betlang to high-performance Julia code, leveraging Julia's scientific computing ecosystem. - -``` -┌─────────────────┐ -│ BetLang (Racket)│ -│ Source Code │ -└────────┬────────┘ - │ Parse/Analyze - ▼ -┌─────────────────┐ -│ AST/IR │ -│ Representation │ -└────────┬────────┘ - │ Code Generation - ▼ -┌─────────────────┐ -│ Julia Code │ -│ (Generated) │ -└────────┬────────┘ - │ Runtime - ▼ -┌─────────────────┐ -│ Julia Packages │ -│ (Distributions,│ -│ StatsBase, │ -│ etc.) │ -└─────────────────┘ -``` - -## Priority Structure - -**Tier 0 (Foundation):** Standard Julia ecosystem - MOST IMPORTANT -**Tier 1 (Extensions):** Hyperpolymath packages - valuable additions -**Tier 2+:** Specialized use cases - ---- - -## Tier 0: Core Julia Ecosystem (FOUNDATION - MOST CRITICAL) - -### Must-Have Integrations (Standard Julia Packages) - -#### 1. **Distributions.jl** - Probability Distributions -**Why:** Authoritative Julia package for probability distributions -**Maps to:** All betlang distribution operations - -**Integration points:** -- `bet` → Sample from `Categorical([1/3, 1/3, 1/3])` -- `bet/weighted` → Sample from `Categorical(normalize(weights))` -- `lib/distributions.rkt` → Direct Distributions.jl types -- Number systems (DistnumberNormal, DistnumberBeta) → `Normal()`, `Beta()` - -**Distributions.jl types to use:** -```julia -# Discrete -Categorical, Bernoulli, Binomial, Geometric, Poisson, Multinomial - -# Continuous -Normal, Exponential, Gamma, Beta, Uniform, ChiSquare, TDist, -Weibull, Pareto, Cauchy, Laplace, Dirichlet - -# Functions -pdf(), cdf(), quantile(), mean(), var(), entropy(), fit() -``` - -#### 2. **StatsBase.jl** - Statistical Utilities -**Why:** Comprehensive statistical functions -**Maps to:** `lib/statistics.rkt` (180+ functions) - -**Integration points:** -- Descriptive stats: `mean()`, `median()`, `mode()`, `var()`, `std()` -- Correlation: `cor()`, `cov()`, `corspearman()` -- Sampling: `sample()`, `wsample()` (weighted sampling) -- Histograms: `fit(Histogram, data)` -- Percentiles: `percentile()`, `quantile()` - -#### 3. **Random.jl** (Stdlib) - RNG -**Why:** Deterministic seeding for reproducibility -**Maps to:** `bet-with-seed`, all randomness - -**Integration points:** -```julia -using Random -rng = MersenneTwister(seed) -rand(rng, Categorical([0.3, 0.4, 0.3])) -``` - -### Tier 2: Number Systems Support - -#### 4. **IntervalArithmetic.jl** - Interval Arithmetic -**Why:** Native interval types for AffineNumber -**Maps to:** `AffineNumber` in `lib/number-systems.rkt` - -```julia -using IntervalArithmetic -x = 18..22 # [18, 22]°C -y = 20..25 -z = x + y # Automatic interval arithmetic -``` - -#### 5. **Measurements.jl** - Uncertainty Propagation -**Why:** Automatic uncertainty tracking (like DistNumber) -**Maps to:** `DistnumberNormal` operations - -```julia -using Measurements -height = 170 ± 10 # 170cm ± 10cm -weight = 75 ± 5 -bmi = weight / (height/100)^2 # Automatic error propagation -``` - -#### 6. **MonteCarloMeasurements.jl** - Particle-Based Uncertainty -**Why:** Alternative to analytic uncertainty propagation -**Maps to:** Advanced number system operations - -```julia -using MonteCarloMeasurements -x = 5.0 ± 1.0 # Creates particle distribution -y = 3.0 ± 0.5 -z = x * y # Monte Carlo propagation -``` - -### Tier 3: Bayesian/MCMC Integration - -#### 7. **Turing.jl** - Probabilistic Programming -**Why:** Sister language! Can interoperate or compete -**Maps to:** `lib/bayesian.rkt` advanced features - -**Comparison:** -```julia -# Turing.jl model -@model coin_flip(y) = begin - p ~ Beta(1, 1) - y .~ Bernoulli(p) -end - -# BetLang equivalent (compiled to Julia using Turing backend) -(define model - (bayesian-model - (prior 'p (beta-dist 1 1)) - (likelihood 'y (bernoulli-dist p)))) -``` - -**Decision:** Turing.jl could be: -- **Option A:** Backend for betlang Bayesian inference (use Turing's MCMC) -- **Option B:** Alternative target (compile betlang → Turing.jl models) -- **Option C:** Competition (betlang implements own MCMC, benchmarks against Turing) - -**Recommendation:** Option A - Use Turing.jl as backend for MCMC samplers - -#### 8. **AdvancedMH.jl** - MCMC Samplers -**Why:** Metropolis-Hastings, Gibbs, HMC implementations -**Maps to:** `lib/bayesian.rkt` MCMC functions - -#### 9. **Gen.jl** - Alternative Probabilistic Programming -**Why:** MIT's probabilistic programming framework -**Maps to:** Potential alternative backend - -### Tier 4: Financial/Risk Applications - -#### 10. **TimeSeries.jl** - Time Series Analysis -**Why:** Financial modeling with temporal data -**Maps to:** `examples/finance.rkt` time series functions - -#### 11. **Optim.jl** - Optimization -**Why:** Portfolio optimization, parameter fitting -**Maps to:** `lib/optimization.rkt` - -```julia -using Optim -# Kelly criterion optimization -f(x) = -expected_log_wealth(x, probabilities, payoffs) -result = optimize(f, 0.0, 1.0) # Optimal bet fraction -``` - -#### 12. **JuMP.jl** - Mathematical Optimization -**Why:** Constrained optimization (risk limits, portfolio constraints) -**Maps to:** Advanced portfolio optimization - -```julia -using JuMP, HiGHS -model = Model(HiGHS.Optimizer) -@variable(model, 0 <= stake <= max_stake) -@objective(model, Max, expected_value(stake)) -@constraint(model, risk_of_ruin(stake) <= 0.05) -optimize!(model) -``` - -### Tier 5: Advanced Mathematics - -#### 13. **SpecialFunctions.jl** - Special Functions -**Why:** Gamma, Beta, Bessel functions for distributions -**Maps to:** Distribution internals - -#### 14. **LinearAlgebra.jl** (Stdlib) - Matrix Operations -**Why:** Multivariate distributions, covariance matrices -**Maps to:** Multivariate statistics - -#### 15. **Symbolics.jl** - Symbolic Math -**Why:** Symbolic uncertainty propagation -**Maps to:** Advanced number systems (SurrealAdvanced, PAdicAdvanced) - -## Compilation Strategy - -### Phase 1: Direct Translation (Simple) - -**Input (BetLang):** -```racket -(bet 'heads 'tails 'edge) -``` - -**Output (Julia):** -```julia -using Distributions, Random -sample(Categorical([1/3, 1/3, 1/3]), 1)[1] # Returns :heads, :tails, or :edge -``` - -### Phase 2: Function Translation - -**Input (BetLang):** -```racket -(define (coin-flip-game trials) - (bet-parallel trials 'heads 'tails 'edge)) -``` - -**Output (Julia):** -```julia -function coin_flip_game(trials::Int) - dist = Categorical([1/3, 1/3, 1/3]) - [sample(dist) for _ in 1:trials] -end -``` - -### Phase 3: Number System Mapping - -**Input (BetLang):** -```racket -(define height (make-distnumber-normal 170 10)) -(define weight (make-distnumber-normal 75 5)) -(define bmi (distnumber-div weight (distnumber-mul height height))) -``` - -**Output (Julia):** -```julia -using Measurements -height = 170.0 ± 10.0 -weight = 75.0 ± 5.0 -bmi = weight / height^2 -``` - -### Phase 4: Safety Features Integration - -**Dutch Book Prevention:** -```julia -function validate_probabilities(probs::Vector{Float64}; tol=1e-6) - total = sum(probs) - if abs(total - 1.0) > tol - throw(DutchBookError("Probabilities sum to $total, not 1.0")) - end - return probs -end -``` - -**Risk-of-Ruin Protection:** -```julia -using Distributions -function kelly_fraction(p::Float64, b::Float64, kelly_fraction::Float64=0.25) - edge = p * b - (1 - p) - full_kelly = edge / b - return full_kelly * kelly_fraction # Fractional Kelly -end -``` - -**Cool-Off Mechanism:** -```julia -mutable struct CoolOffTracker - last_bet_time::Float64 - cool_off_seconds::Float64 - violations::Int -end - -function bet_with_cooloff!(tracker::CoolOffTracker, bet_fn::Function) - now = time() - elapsed = now - tracker.last_bet_time - if elapsed < tracker.cool_off_seconds - tracker.violations += 1 - error("Cool-off active. Wait $(tracker.cool_off_seconds - elapsed)s") - end - tracker.last_bet_time = now - return bet_fn() -end -``` - -## Performance Benefits - -### Why Julia Backend? - -1. **Speed:** 10-100x faster than Racket for numerical code -2. **Ecosystem:** Mature scientific computing libraries -3. **Integration:** Easy FFI with C/Fortran/Python -4. **JIT:** Compiled to native code, not interpreted -5. **Parallelism:** Native multi-threading, GPU support -6. **Type Stability:** Optional static typing for performance - -### Benchmarks (Expected) - -| Operation | Racket | Julia | Speedup | -|-----------|--------|-------|---------| -| Monte Carlo (1M samples) | 15s | 0.2s | 75x | -| Matrix operations | 8s | 0.1s | 80x | -| MCMC sampling (10K) | 120s | 2s | 60x | -| Distribution sampling | 5s | 0.05s | 100x | - -## Implementation Roadmap - -### Step 1: Minimal Viable Backend (2 weeks) -- [ ] Parse betlang AST from Racket -- [ ] Generate Julia code for basic `bet` operations -- [ ] Map to Distributions.jl Categorical -- [ ] Write generated code to `.jl` file -- [ ] Execute via `julia generated.jl` - -### Step 2: Core Language (4 weeks) -- [ ] Translate all bet primitives (bet, bet/weighted, bet/conditional) -- [ ] Function definitions and calls -- [ ] Variable bindings (let, define) -- [ ] Control flow (if, cond) -- [ ] List operations (map, filter, fold) - -### Step 3: Standard Library (6 weeks) -- [ ] Map `lib/statistics.rkt` → StatsBase.jl -- [ ] Map `lib/distributions.rkt` → Distributions.jl -- [ ] Map `lib/bayesian.rkt` → Turing.jl/AdvancedMH.jl -- [ ] Map `lib/optimization.rkt` → Optim.jl - -### Step 4: Number Systems (4 weeks) -- [ ] DistnumberNormal → Measurements.jl -- [ ] AffineNumber → IntervalArithmetic.jl -- [ ] BayesianNumber → Custom type -- [ ] RiskNumber → Custom type with Distributions.jl -- [ ] Other 10 systems → Custom types - -### Step 5: Safety Features (2 weeks) -- [ ] Dutch book validation (Julia-side checks) -- [ ] Risk-of-ruin protection (Kelly criterion) -- [ ] Cool-off mechanism (timer struct) - -### Step 6: Performance Optimization (4 weeks) -- [ ] Type stability analysis -- [ ] Precompilation of common operations -- [ ] GPU acceleration for Monte Carlo -- [ ] Multi-threading for parallel bets - -**Total Estimated Time:** 22 weeks (~5 months) - -## Build System Integration - -### Justfile Recipes - -```just -# Build Julia backend -build-julia: - racket compiler/betlang-to-julia.rkt examples/basic.bet --output build/basic.jl - julia build/basic.jl - -# Run tests with Julia backend -test-julia: - racket tests/julia-backend-tests.rkt - julia --project tests/run_tests.jl - -# Benchmark Julia vs Racket -benchmark: - racket benchmarks/performance.rkt --backend racket - racket compiler/betlang-to-julia.rkt benchmarks/performance.bet - julia benchmarks/performance.jl - julia benchmarks/compare.jl -``` - -## Package Structure - -``` -betlang-julia/ -├── Project.toml # Julia package metadata -├── src/ -│ ├── BetLang.jl # Main module -│ ├── core.jl # Core bet primitives -│ ├── distributions.jl # Distribution mappings -│ ├── statistics.jl # Statistical functions -│ ├── bayesian.jl # Bayesian inference -│ ├── optimization.jl # Optimization algorithms -│ ├── number_systems.jl # 14 number systems -│ └── safety.jl # Safety features -├── test/ -│ └── runtests.jl -└── examples/ - ├── basic.jl - ├── finance.jl - └── monte_carlo.jl -``` - -### Project.toml - -```toml -name = "BetLang" -uuid = "..." -authors = ["Jonathan D.A. Jewell "] -version = "0.8.0" - -[deps] -Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f" -StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" -Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" -IntervalArithmetic = "d1acc4aa-44c8-5952-acd4-ba5d80a2a253" -Measurements = "eff96d63-e80a-5855-80a2-b1b0885c5ab7" -Turing = "fce5fe82-541a-59a6-adf8-730c64b5f9a0" -Optim = "429524aa-4258-5aef-a3af-852621145aeb" - -[compat] -julia = "1.9" -Distributions = "0.25" -StatsBase = "0.34" -``` - -## Integration with Existing Ecosystem - -### Aggregate Library Bridge - -**Goal:** BetLang number systems → Aggregate library (via Julia FFI) - -```julia -# BetLang.jl provides C ABI for aggregate-library -function distnumber_to_c_array(dn::DistnumberNormal) - samples = [rand(Normal(dn.mean, dn.std)) for _ in 1:1000] - return pointer(samples), length(samples) -end -``` - -### R Integration (Future) - -Via RCall.jl: -```julia -using RCall -R""" -library(betlang) # R package wrapping BetLang.jl -result <- bet(c('heads', 'tails', 'edge')) -""" -``` - -### Python Integration (Future) - -Via PyCall.jl or PythonCall.jl: -```python -import juliacall -betlang = juliacall.newmodule("BetLang") -result = betlang.bet(['heads', 'tails', 'edge']) -``` - -## Alternatives Considered - -### 1. Compile to C/Rust instead? -**Pros:** Maximum performance -**Cons:** No ecosystem, must reimplement everything -**Verdict:** Julia is better - ecosystem + performance - -### 2. Use Turing.jl directly instead of betlang? -**Pros:** Mature, well-tested -**Cons:** Doesn't have ternary philosophy, safety features, or number systems -**Verdict:** BetLang offers unique value, can use Turing as backend - -### 3. Keep Racket-only? -**Pros:** Simplest, already works -**Cons:** Too slow for large-scale simulations (millions of samples) -**Verdict:** Julia backend is valuable for performance-critical users - -## Success Metrics - -### v0.8.0 Definition of Done - -- [ ] Compile basic betlang programs to Julia -- [ ] All `examples/*.rkt` files compile and run -- [ ] Performance: 50x+ speedup on Monte Carlo benchmarks -- [ ] Integration: Distributions.jl, StatsBase.jl working -- [ ] Tests: 100% of Racket test suite passes on Julia backend -- [ ] Documentation: Tutorial for using Julia backend - -### Performance Goals - -- Monte Carlo (1M samples): < 1 second -- MCMC sampling (10K iterations): < 5 seconds -- Portfolio optimization: < 100ms -- Distribution operations: < 1μs per sample - -## Questions for User - -1. **Priority order:** Which tier of packages should we focus on first? -2. **Turing.jl relationship:** Backend, target, or competitor? -3. **Number systems:** Implement all 14 in Julia, or just core 5? -4. **Timeline:** Is 5 months acceptable, or should we compress? -5. **Use cases:** What specific use cases need Julia performance? - -## References - -- Distributions.jl: https://juliastats.org/Distributions.jl/stable/ -- StatsBase.jl: https://juliastats.org/StatsBase.jl/stable/ -- Turing.jl: https://turinglang.org/stable/ -- Measurements.jl: https://juliaphysics.github.io/Measurements.jl/stable/ -- IntervalArithmetic.jl: https://juliaintervals.github.io/IntervalArithmetic.jl/stable/ - ---- - -**Next Steps:** -1. Get feedback on architecture -2. Prototype minimal compiler (Step 1) -3. Benchmark proof-of-concept -4. Iterate based on performance data diff --git a/docs/proof-debt.adoc b/docs/proof-debt.adoc index 4378324..f084858 100644 --- a/docs/proof-debt.adoc +++ b/docs/proof-debt.adoc @@ -1,69 +1,152 @@ -// SPDX-License-Identifier: CC-BY-SA-4.0 -// Copyright (c) Jonathan D.A. Jewell -// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) -= Proof debt -:policy: https://github.com/hyperpolymath/standards/blob/main/docs/TRUSTED-BASE-REDUCTION-POLICY.adoc -:tapl: TAPL (Pierce 2002) Chapter 6 -:discharge-pr: https://github.com/hyperpolymath/betlang/pull/27 - -Per link:{policy}[standards#203] (trusted-base reduction policy), every -soundness-relevant escape hatch in this repo is classified below as one -of: (a) DISCHARGED, (b) BUDGETED, (c) NECESSARY AXIOM, or (d) DEBT. - -Supersedes the thin index in -https://github.com/hyperpolymath/betlang/pull/37[betlang#37] (which -declared "3 escape hatches" — the actual count is now 0; the single -classified axiom has since been discharged to a theorem, see §(a)). - -== Marker count - -0 soundness-relevant escape hatches in this repo. - -The former `axiom substTop_preserves_typing` has been discharged to a -proved theorem (`proofs/BetLang.lean:918`); see §(a). Zero `axiom`, zero -`sorry` declarations, zero `noncomputable` in soundness positions. - -== (a) Discharged - -=== `proofs/BetLang.lean:918` — `theorem substTop_preserves_typing` (formerly an axiom) - -[source,lean] ----- -theorem substTop_preserves_typing : - ∀ (Γ : Ctx) (S T : Ty) (body v : Expr), - HasType (S :: Γ) body T → HasType Γ v S → HasType Γ (substTop v body) T ----- - -* **Kind**: classical substitution lemma for typed lambda terms under - de Bruijn substitution — the canonical preservation prerequisite ({tapl}). -* **Status**: DISCHARGED. Formerly classified (c) NECESSARY AXIOM; now a - fully proved `theorem`, with no `axiom` declaration remaining anywhere in - `proofs/BetLang.lean`. Machine-checked by `lake build` (CI: `proofs.yml`). -* **Used by**: the `preservation` theorem (threaded through the `appLam` - beta-reduction case). -* **History**: discharge recipe in link:{discharge-pr}[betlang PR #27]. - -== (b) Budgeted — tested with refutation budget - -No entries. - -== (c) NECESSARY AXIOM - -No entries. (The former `substTop_preserves_typing` axiom was discharged to -a theorem — see §(a).) - -== (d) DEBT — actively to be closed - -No entries. (If the promotion path above is pursued, the §(c) entry -moves to §(a); it does not pass through §(d) because it has a -principled disposition today.) - -== Cross-references - -* link:{policy}[standards#203] — trusted-base reduction policy - (canonical estate-wide definition) -* `proofs/BetLang.lean` — the formal model where the (now proved) - substitution lemma is stated and used -* `docs/semantics.md` — language semantics overview -* link:{discharge-pr}[betlang PR #27] — discharge recipe (optional - promotion to §(a)) +== Proof Debt — betlang + +*Schema*: +https://github.com/hyperpolymath/standards/blob/main/docs/TRUSTED-BASE-REDUCTION-POLICY.adoc[`+hyperpolymath/standards/docs/TRUSTED-BASE-REDUCTION-POLICY.adoc+`] +— the canonical estate-wide policy (standards#203). Companion to the +strategic narrative in link:../PROOF-NEEDS.md[`+../PROOF-NEEDS.md+`]. +Detection script: +https://github.com/hyperpolymath/standards/blob/main/scripts/check-trusted-base.sh[`+scripts/check-trusted-base.sh+`] +(standards#211). + +This file is the schema-conformant per-repo proof-debt index for +`+hyperpolymath/betlang+`. It enumerates every soundness-relevant escape +hatch in `+proofs/BetLang.lean+`. + +=== Marker count (2026-05-27) + +`+check-trusted-base.sh+` reports *4 syntactic hits* under its Lean +regex (`+\bsorry\b|^[[:space:]]*axiom[[:space:]]+`); *1 is a real +declaration* and *3 are comment-line false positives* that the script’s +`+is_comment_line+` heuristic strips. After comment-skip: *1 real +marker* — the single named axiom `+substTop_preserves_typing+`. + +[width="100%",cols="25%,25%,25%,25%",options="header",] +|=== +|File:line |Raw token |Real? |Disposition +|`+proofs/BetLang.lean:19+` |`+sorry+` (inside +`+-- All theorems are fully proved — no \+`sorry`.`+) | false positive (comment) | n/a | |+`proofs/BetLang.lean:387`+|+`axiom`+/+`sorry`+inside+`– +IMPORTANT: This is NOT sorry — it is an +axiom.`+| false positive (comment) | n/a | |+`proofs/BetLang.lean:388`+|+`axioms`+/+`sorry`+inside+`– +axioms are explicit assumptions … whereas +sorry`+| false positive (comment) | n/a | |+`proofs/BetLang.lean:392`+|+`axiom +substTop_preserves_typing :` |*real* |§(d) DEBT +|=== + +=== (a) DISCHARGED in this repo + +_(None yet — entries move here when a marker is removed or replaced by a +total/proven counterpart.)_ + +=== (b) BUDGETED — tested with a refutation budget + +_(None. BetLang’s proof file is pure Lean 4 metatheory; there is no +extraction-boundary surface needing a property-test budget.)_ + +=== (c) NECESSARY AXIOM + +_(None. The only real axiom in this repo (`+substTop_preserves_typing+`) +is *not* a metatheoretic assumption — it is a standard TAPL Ch.9 +substitution-preservation lemma with a known constructive discharge. It +is therefore classified as §(d) DEBT, not §(c) NECESSARY.)_ + +=== (d) DEBT — actively to be closed + +==== `+proofs/BetLang.lean:392+` — `+axiom substTop_preserves_typing+` + +* *Statement*: +`+∀ (Γ : Ctx) (S T : Ty) (body v : Expr), HasType (S :: Γ) body T → HasType Γ v S → HasType Γ (substTop v body) T+`. +* *Why it is §(d) and not §(c)*: this is the standard top-level +substitution-preservation lemma (Pierce, TAPL Ch.9). It is +constructively provable in Lean 4 by induction on the typing derivation, +factored through a generalised `+substAt_preserves_typing+` and three +context-insertion lookup lemmas. It does *not* encode a metatheoretic +assumption (no funExt, no choice, no UIP). It only exists in the source +because a full de Bruijn substitution calculus would, per the inline +doc-comment at L383-385, "`triple the file size`" — a _cost_ judgement, +not a _necessity_ judgement. +* *Owner*: @hyperpolymath +* *Plan* (verbatim from the discharge recipe in PR +https://github.com/hyperpolymath/betlang/pull/27[#27] body, lightly +reformatted): +[arabic] +. *`+Ctx.insertAt+`* — define a context-insertion operator. (`+Ctx+` is +a `+List Ty+` `+abbrev+`, so dot-notation `+Γ.insertAt+` does not +resolve to anything useful; use a top-level +`+def Ctx.insertAt (Γ : Ctx) (k : Nat) (U : Ty) : Ctx := Γ.take k ++ U :: Γ.drop k+`.) +. *Three lookup lemmas*, all needing the `+k ≤ Γ.length+` hypothesis +(without which they are false at empty Γ with k > 0): +** `+lookup_ctxInsertAt_lt : n < k → Ctx.lookup (ctxInsertAt Γ k U) n = Ctx.lookup Γ n+` +** `+lookup_ctxInsertAt_eq : Ctx.lookup (ctxInsertAt Γ k U) k = some U+` +** `+lookup_ctxInsertAt_gt : n > k → Ctx.lookup (ctxInsertAt Γ k U) n = Ctx.lookup Γ (n - 1)+` +. *`+shift_preserves_typing+`* (weakening at `+amount = 1+`): +`+HasType Γ e T → ∀ k U, k ≤ Γ.length → HasType (ctxInsertAt Γ k U) (shift 1 k e) T+`. +Induction on `+HasType+`; binder cases (`+tLam+`, `+tLet+`) bump `+k+` +by 1 and recurse. Variable case needs `+Int.toNat (↑n + 1) = n + 1+`. +. *`+subst_preserves_typing+`* (generalised): +`+HasType (ctxInsertAt Γ k S) e T → HasType Γ v S → HasType (ctxInsertAt Γ k S) (subst k (shift (k+1 : Int) 0 v) e) T+`. +Induction on `+e+`. Binder cases recurse with `+k+1+` after pulling in +the extra `+shift 1 0+`. +. *"`No surviving var k`" tracking lemma*: after `+subst k v e+`, define +a predicate `+freeVarNotIn k e+` and prove it holds. Then prove +`+shift_down+` preserves typing when this predicate holds. +. *Headline*: `+substTop_preserves_typing Γ S T body v hb hv := ...+` +combining the chain. Then delete `+axiom substTop_preserves_typing+` at +L392. Verify `+preservation+` still typechecks. +* *Estimated size*: ~300-400 LoC of standard TAPL Ch.9 mechanisation. +The `+Int.toNat+` clamping in BetLang’s `+shift+` (signed `+amount+`) +adds non-trivial arithmetic friction at every `+var+` case. +* *Status*: a partial implementation lives on the local working branch +`+proofs/discharge-substTop-axiom-23+` (commit `+8fa128d+` "`proofs: +discharge substTop_preserves_typing axiom (closes #23)`"), not yet +opened as a PR. Issue +https://github.com/hyperpolymath/betlang/issues/23[#23] was closed +optimistically when PR #27 merged the build-fix half of the work; the +axiom itself remained in source. +* *Deadline*: INDEFINITE — single-maintainer repo; discharge is bounded +in scope but not currently scheduled. Re-open issue #23 (or file a +successor) when work resumes. + +=== Cross-language survey (Idris2) + +The repo also contains Idris2 test scaffolding under `+tests/idris2/**+` +and `+playground/tests/idris2/**+`. `+check-trusted-base.sh+` finds +*zero* `+believe_me+` / `+really_believe_me+` / `+assert_total+` / +top-level `+partial+` markers in those files. No §(b)/(c)/(d) entries +needed. + +=== How to update this file + +When markers in `+proofs/BetLang.lean+` (or any future proof-bearing +file) change: + +[arabic] +. Re-run `+bash scripts/check-trusted-base.sh .+` (the script is +delivered from `+hyperpolymath/standards+` and called from the +governance bundle). +. If the count drops, move the resolved entries from §(d) → §(a). +. If new markers appear, add them to §(d) with an owner + (possibly +INDEFINITE) deadline. +. The `+trusted-base+` CI job (`+governance-reusable.yml+`, landed in +standards#211) will fail on un-annotated AND un-enumerated markers — so +every change in source MUST be reflected here. + +=== Companion documents + +* link:../PROOF-NEEDS.md[`+../PROOF-NEEDS.md+`] — the strategic +proof-debt narrative for betlang. +* https://github.com/hyperpolymath/standards/pull/203[hyperpolymath/standards#203] +— the policy this file conforms to. +* https://github.com/hyperpolymath/standards/pull/211[hyperpolymath/standards#211] +— `+scripts/check-trusted-base.sh+` and the `+trusted-base+` job. +* https://github.com/hyperpolymath/standards/pull/222[hyperpolymath/standards#222] +— the fill-in PR for the policy doc. +* https://github.com/hyperpolymath/betlang/issues/23[hyperpolymath/betlang#23] +— the (closed) tracking issue for discharging +`+substTop_preserves_typing+`. +* https://github.com/hyperpolymath/betlang/pull/27[hyperpolymath/betlang#27] +— the merged PR whose body contains the canonical discharge recipe. + +''''' + +Initial seed by Claude Code, 2026-05-27, applying the standards#203 +schema to betlang per the standards#222 per-repo triage pass. diff --git a/docs/proof-debt.md b/docs/proof-debt.md deleted file mode 100644 index f6a8dff..0000000 --- a/docs/proof-debt.md +++ /dev/null @@ -1,147 +0,0 @@ - -# Proof Debt — betlang - -**Schema**: [`hyperpolymath/standards/docs/TRUSTED-BASE-REDUCTION-POLICY.adoc`](https://github.com/hyperpolymath/standards/blob/main/docs/TRUSTED-BASE-REDUCTION-POLICY.adoc) -— the canonical estate-wide policy (standards#203). Companion to the -strategic narrative in [`../PROOF-NEEDS.md`](../PROOF-NEEDS.md). Detection -script: [`scripts/check-trusted-base.sh`](https://github.com/hyperpolymath/standards/blob/main/scripts/check-trusted-base.sh) -(standards#211). - -This file is the schema-conformant per-repo proof-debt index for -`hyperpolymath/betlang`. It enumerates every soundness-relevant escape -hatch in `proofs/BetLang.lean`. - -## Marker count (2026-05-27) - -`check-trusted-base.sh` reports **4 syntactic hits** under its Lean regex -(`\bsorry\b|^[[:space:]]*axiom[[:space:]]`); **1 is a real declaration** -and **3 are comment-line false positives** that the script's -`is_comment_line` heuristic strips. After comment-skip: **1 real marker** -— the single named axiom `substTop_preserves_typing`. - -| File:line | Raw token | Real? | Disposition | -|---|---|---|---| -| `proofs/BetLang.lean:19` | `sorry` (inside `-- All theorems are fully proved — no \`sorry\`.`) | false positive (comment) | n/a | -| `proofs/BetLang.lean:387` | `axiom` / `sorry` inside `-- IMPORTANT: This is NOT sorry — it is an axiom.` | false positive (comment) | n/a | -| `proofs/BetLang.lean:388` | `axioms` / `sorry` inside `-- axioms are explicit assumptions ... whereas sorry` | false positive (comment) | n/a | -| `proofs/BetLang.lean:392` | `axiom substTop_preserves_typing :` | **real** | §(d) DEBT | - -## (a) DISCHARGED in this repo - -*(None yet — entries move here when a marker is removed or replaced by a -total/proven counterpart.)* - -## (b) BUDGETED — tested with a refutation budget - -*(None. BetLang's proof file is pure Lean 4 metatheory; there is no -extraction-boundary surface needing a property-test budget.)* - -## (c) NECESSARY AXIOM - -*(None. The only real axiom in this repo (`substTop_preserves_typing`) -is **not** a metatheoretic assumption — it is a standard TAPL Ch.9 -substitution-preservation lemma with a known constructive discharge. -It is therefore classified as §(d) DEBT, not §(c) NECESSARY.)* - -## (d) DEBT — actively to be closed - -### `proofs/BetLang.lean:392` — `axiom substTop_preserves_typing` - -- **Statement**: `∀ (Γ : Ctx) (S T : Ty) (body v : Expr), HasType (S :: Γ) body T → HasType Γ v S → HasType Γ (substTop v body) T`. -- **Why it is §(d) and not §(c)**: this is the standard top-level - substitution-preservation lemma (Pierce, TAPL Ch.9). It is constructively - provable in Lean 4 by induction on the typing derivation, factored - through a generalised `substAt_preserves_typing` and three - context-insertion lookup lemmas. It does **not** encode a - metatheoretic assumption (no funExt, no choice, no UIP). It only - exists in the source because a full de Bruijn substitution calculus - would, per the inline doc-comment at L383-385, "triple the file size" - — a *cost* judgement, not a *necessity* judgement. -- **Owner**: @hyperpolymath -- **Plan** (verbatim from the discharge recipe in PR - [#27](https://github.com/hyperpolymath/betlang/pull/27) body, lightly - reformatted): - 1. **`Ctx.insertAt`** — define a context-insertion operator. (`Ctx` - is a `List Ty` `abbrev`, so dot-notation `Γ.insertAt` does not - resolve to anything useful; use a top-level - `def Ctx.insertAt (Γ : Ctx) (k : Nat) (U : Ty) : Ctx := - Γ.take k ++ U :: Γ.drop k`.) - 2. **Three lookup lemmas**, all needing the `k ≤ Γ.length` - hypothesis (without which they are false at empty Γ with k > 0): - - `lookup_ctxInsertAt_lt : n < k → Ctx.lookup (ctxInsertAt Γ k U) n = Ctx.lookup Γ n` - - `lookup_ctxInsertAt_eq : Ctx.lookup (ctxInsertAt Γ k U) k = some U` - - `lookup_ctxInsertAt_gt : n > k → Ctx.lookup (ctxInsertAt Γ k U) n = Ctx.lookup Γ (n - 1)` - 3. **`shift_preserves_typing`** (weakening at `amount = 1`): - `HasType Γ e T → ∀ k U, k ≤ Γ.length → HasType (ctxInsertAt Γ k U) (shift 1 k e) T`. - Induction on `HasType`; binder cases (`tLam`, `tLet`) bump `k` by 1 - and recurse. Variable case needs `Int.toNat (↑n + 1) = n + 1`. - 4. **`subst_preserves_typing`** (generalised): - `HasType (ctxInsertAt Γ k S) e T → HasType Γ v S → HasType (ctxInsertAt Γ k S) (subst k (shift (k+1 : Int) 0 v) e) T`. - Induction on `e`. Binder cases recurse with `k+1` after pulling in - the extra `shift 1 0`. - 5. **"No surviving var k" tracking lemma**: after `subst k v e`, define - a predicate `freeVarNotIn k e` and prove it holds. Then prove - `shift_down` preserves typing when this predicate holds. - 6. **Headline**: - `substTop_preserves_typing Γ S T body v hb hv := ...` combining the - chain. Then delete `axiom substTop_preserves_typing` at L392. - Verify `preservation` still typechecks. -- **Estimated size**: ~300-400 LoC of standard TAPL Ch.9 mechanisation. - The `Int.toNat` clamping in BetLang's `shift` (signed `amount`) adds - non-trivial arithmetic friction at every `var` case. -- **Status**: a partial implementation lives on the local working - branch `proofs/discharge-substTop-axiom-23` (commit `8fa128d` - "proofs: discharge substTop_preserves_typing axiom (closes #23)"), - not yet opened as a PR. Issue - [#23](https://github.com/hyperpolymath/betlang/issues/23) was closed - optimistically when PR #27 merged the build-fix half of the work; - the axiom itself remained in source. -- **Deadline**: INDEFINITE — single-maintainer repo; discharge is bounded - in scope but not currently scheduled. Re-open issue #23 (or file a - successor) when work resumes. - -## Cross-language survey (Idris2) - -The repo also contains Idris2 test scaffolding under `tests/idris2/**` -and `playground/tests/idris2/**`. `check-trusted-base.sh` finds **zero** -`believe_me` / `really_believe_me` / `assert_total` / top-level -`partial` markers in those files. No §(b)/(c)/(d) entries needed. - -## How to update this file - -When markers in `proofs/BetLang.lean` (or any future proof-bearing file) -change: - -1. Re-run `bash scripts/check-trusted-base.sh .` (the script is - delivered from `hyperpolymath/standards` and called from the - governance bundle). -2. If the count drops, move the resolved entries from §(d) → §(a). -3. If new markers appear, add them to §(d) with an owner + (possibly - INDEFINITE) deadline. -4. The `trusted-base` CI job (`governance-reusable.yml`, landed in - standards#211) will fail on un-annotated AND un-enumerated markers - — so every change in source MUST be reflected here. - -## Companion documents - -- [`../PROOF-NEEDS.md`](../PROOF-NEEDS.md) — the strategic proof-debt - narrative for betlang. -- [hyperpolymath/standards#203](https://github.com/hyperpolymath/standards/pull/203) - — the policy this file conforms to. -- [hyperpolymath/standards#211](https://github.com/hyperpolymath/standards/pull/211) - — `scripts/check-trusted-base.sh` and the `trusted-base` job. -- [hyperpolymath/standards#222](https://github.com/hyperpolymath/standards/pull/222) - — the fill-in PR for the policy doc. -- [hyperpolymath/betlang#23](https://github.com/hyperpolymath/betlang/issues/23) - — the (closed) tracking issue for discharging - `substTop_preserves_typing`. -- [hyperpolymath/betlang#27](https://github.com/hyperpolymath/betlang/pull/27) - — the merged PR whose body contains the canonical discharge recipe. - ---- - -Initial seed by Claude Code, 2026-05-27, applying the standards#203 -schema to betlang per the standards#222 per-repo triage pass. diff --git a/docs/semantics.adoc b/docs/semantics.adoc new file mode 100644 index 0000000..9f4ea76 --- /dev/null +++ b/docs/semantics.adoc @@ -0,0 +1,394 @@ +== betlang Semantics + +Betlang is a ternary DSL for probabilistic modeling and symbolic wagers, +implemented in Racket. It provides a rich set of primitives for +expressing and analyzing probabilistic computations. + +=== Core Philosophy + +The language is built around the ternary principle - all fundamental +operations involve three choices, inspired by musical ternary form +(A–B–A). This design creates a natural framework for modeling +uncertainty and probabilistic decision-making. + +=== Core Forms + +==== Basic Bet + +[source,racket] +---- +(bet A B C) → randomly selects one of three values +---- + +The fundamental primitive of betlang. Each value has equal probability +(1/3) of being selected. + +*Examples:* + +[source,racket] +---- +(bet 1 2 3) ; Returns 1, 2, or 3 +(bet 'win 'draw 'lose) ; Returns one of three symbols +(bet "A" "B" "C") ; Returns one of three strings +---- + +*Formal Semantics:* + +.... +⟦bet A B C⟧ = A with probability 1/3 + = B with probability 1/3 + = C with probability 1/3 +.... + +==== Weighted Bet + +[source,racket] +---- +(bet/weighted '(A weight-a) '(B weight-b) '(C weight-c)) +---- + +Allows non-uniform probability distributions where probabilities are +proportional to weights. + +*Examples:* + +[source,racket] +---- +(bet/weighted '(rare 1) '(uncommon 3) '(common 6)) +; P(rare) = 0.1, P(uncommon) = 0.3, P(common) = 0.6 +---- + +*Formal Semantics:* + +.... +⟦bet/weighted (A w₁) (B w₂) (C w₃)⟧ = A with probability w₁/(w₁+w₂+w₃) + = B with probability w₂/(w₁+w₂+w₃) + = C with probability w₃/(w₁+w₂+w₃) +.... + +==== Conditional Bet + +[source,racket] +---- +(bet/conditional predicate A B C) +---- + +Deterministically returns A if predicate is true, otherwise performs a +bet between B and C (with fallback to A). + +*Examples:* + +[source,racket] +---- +(bet/conditional (> x 10) 'large 'medium 'small) +---- + +*Formal Semantics:* + +.... +⟦bet/conditional pred A B C⟧ = A if pred = true + = ⟦bet B C A⟧ if pred = false +.... + +==== Lazy Bet + +[source,racket] +---- +(bet/lazy thunk-a thunk-b thunk-c) +---- + +Delays evaluation - only the selected branch is computed, avoiding +unnecessary computation. + +*Examples:* + +[source,racket] +---- +(bet/lazy + (lambda () (expensive-computation-a)) + (lambda () (expensive-computation-b)) + (lambda () (expensive-computation-c))) +---- + +=== Composition Operators + +==== Bet Chain + +[source,racket] +---- +(bet-chain n f init) +---- + +Chains n probabilistic transformations together, threading results +through function f. + +*Formal Semantics:* + +.... +⟦bet-chain 0 f init⟧ = init +⟦bet-chain n f init⟧ = ⟦bet-chain (n-1) f (f init)⟧ +.... + +==== Bet Compose + +[source,racket] +---- +(bet-compose f g h) +---- + +Creates a higher-order function that randomly selects one of three +functions to apply. + +*Examples:* + +[source,racket] +---- +(define process (bet-compose add1 identity sub1)) +(process 10) ; Returns 11, 10, or 9 +---- + +==== Bet Map + +[source,racket] +---- +(bet-map f lst) +---- + +Probabilistically applies a function to list elements. + +==== Bet Fold + +[source,racket] +---- +(bet-fold f init lst) +---- + +Fold with probabilistic choices at each step. + +=== Parallel Operations + +==== Bet Parallel + +[source,racket] +---- +(bet-parallel n A B C) +---- + +Runs n independent trials of the bet and returns all results as a list. + +*Examples:* + +[source,racket] +---- +(bet-parallel 100 'heads 'tails 'edge) +; Returns list of 100 coin flips +---- + +==== Bet Sequence + +[source,racket] +---- +(bet-sequence bet1 bet2 bet3 ...) +---- + +Executes multiple bets in sequence and returns all results. + +=== Control Flow + +==== Bet Until + +[source,racket] +---- +(bet-until predicate thunk) +---- + +Repeatedly executes bet until predicate is satisfied. + +*Formal Semantics:* + +.... +⟦bet-until pred thunk⟧ = result where result is first value from thunk satisfying pred +.... + +==== Bet Repeat + +[source,racket] +---- +(bet-repeat n thunk) +---- + +Executes bet n times and collects results. + +=== Statistical Utilities + +==== Bet Probability + +[source,racket] +---- +(bet-probability n predicate A B C) +---- + +Estimates probability that predicate holds by running n trials. + +*Examples:* + +[source,racket] +---- +(bet-probability 10000 (lambda (x) (equal? x 'A)) 'A 'B 'C) +; Returns approximately 0.333 +---- + +==== Bet Entropy + +[source,racket] +---- +(bet-entropy samples) +---- + +Calculates Shannon entropy of bet outcomes in bits. + +*Formula:* + +.... +H(X) = -Σ P(xᵢ) log₂ P(xᵢ) +.... + +For uniform ternary bet: H = log₂(3) ≈ 1.585 bits + +==== Bet Expect + +[source,racket] +---- +(bet-expect n f A B C) +---- + +Calculates expected value of function f over n trials. + +*Formula:* + +.... +E[f(X)] ≈ (1/n) Σ f(xᵢ) +.... + +=== Determinism and Reproducibility + +==== Bet With Seed + +[source,racket] +---- +(bet-with-seed seed thunk) +---- + +Executes bet with a specific random seed for reproducible results. + +*Examples:* + +[source,racket] +---- +(bet-with-seed 42 (lambda () (bet 1 2 3))) +; Always returns the same value for seed 42 +---- + +=== Type System (Informal) + +Betlang is dynamically typed through Racket, but conceptually: + +.... +Bet[A, B, C] :: Type A → Type B → Type C → Bet (A | B | C) +.... + +Where `+Bet T+` represents a probabilistic value of type T. + +=== Probability Monad (Informal) + +The bet operations form a monad-like structure: + +[source,racket] +---- +return x = (bet x x x) ; Pure/deterministic value +bind m f = (bet (f (bet A B C)) ; Monadic bind + (f (bet A B C)) + (f (bet A B C))) +---- + +More formally available through `+bet-pure+` and `+bet-bind+` in +combinators. + +=== Equational Properties + +==== Symmetry + +For uniform bet: `+(bet A B C) ≡ (bet B C A) ≡ (bet C A B)+` +(distributionally) + +==== Idempotence + +`+(bet X X X) = X+` (deterministic) + +==== Commutativity (distributional) + +`+(bet A B C)+` and `+(bet B A C)+` have same distribution up to +permutation + +=== Error Semantics + +All bet operations propagate Racket exceptions. Invalid arguments raise +`+exn:fail+` exceptions. + +=== Memory Semantics + +Bets are evaluated eagerly by default (except `+bet/lazy+`). Results are +not memoized unless explicitly using `+bet-memoize+` combinator. + +=== Concurrency Semantics + +Bet operations are not thread-safe by default. Use Racket’s +synchronization primitives for concurrent access. + +=== Integration with Racket + +Betlang is implemented as a Racket library. All Racket primitives and +libraries are available. Bets can be freely mixed with Racket code: + +[source,racket] +---- +(define result (bet 1 2 3)) +(if (> result 2) + (displayln "Large!") + (displayln "Small!")) +---- + +=== Advanced Semantics + +==== Probability Distributions + +Through `+lib/distributions.rkt+`, betlang provides: - Discrete: +binomial, geometric, Poisson, multinomial - Continuous: normal, +exponential, gamma, beta - Stochastic processes: random walks, Brownian +motion + +==== Markov Chains + +Through `+lib/markov.rkt+`, betlang supports: - Discrete-time Markov +chains - Transition matrix learning - Stationary distribution estimation +- Hidden Markov Models (simplified) + +==== Statistical Inference + +Through `+lib/statistics.rkt+`: - Descriptive statistics - Hypothesis +testing - Resampling methods (bootstrap, jackknife) - Time series +analysis + +=== Notation Conventions + +Throughout this documentation: - `+⟦expr⟧+` denotes semantic +interpretation - `+A ≡ B+` denotes distributional equivalence - +`+P(event)+` denotes probability - `+E[X]+` denotes expected value - +`+H(X)+` denotes entropy + +=== Future Extensions + +Potential semantic extensions under consideration: - Continuous +probability support - Bayesian inference primitives - Automatic +differentiation for probabilistic programs - Parallel/distributed +execution model diff --git a/docs/semantics.md b/docs/semantics.md deleted file mode 100644 index 8f914de..0000000 --- a/docs/semantics.md +++ /dev/null @@ -1,343 +0,0 @@ - -# betlang Semantics - -Betlang is a ternary DSL for probabilistic modeling and symbolic wagers, implemented in Racket. It provides a rich set of primitives for expressing and analyzing probabilistic computations. - -## Core Philosophy - -The language is built around the ternary principle - all fundamental operations involve three choices, inspired by musical ternary form (A–B–A). This design creates a natural framework for modeling uncertainty and probabilistic decision-making. - -## Core Forms - -### Basic Bet - -```racket -(bet A B C) → randomly selects one of three values -``` - -The fundamental primitive of betlang. Each value has equal probability (1/3) of being selected. - -**Examples:** -```racket -(bet 1 2 3) ; Returns 1, 2, or 3 -(bet 'win 'draw 'lose) ; Returns one of three symbols -(bet "A" "B" "C") ; Returns one of three strings -``` - -**Formal Semantics:** -``` -⟦bet A B C⟧ = A with probability 1/3 - = B with probability 1/3 - = C with probability 1/3 -``` - -### Weighted Bet - -```racket -(bet/weighted '(A weight-a) '(B weight-b) '(C weight-c)) -``` - -Allows non-uniform probability distributions where probabilities are proportional to weights. - -**Examples:** -```racket -(bet/weighted '(rare 1) '(uncommon 3) '(common 6)) -; P(rare) = 0.1, P(uncommon) = 0.3, P(common) = 0.6 -``` - -**Formal Semantics:** -``` -⟦bet/weighted (A w₁) (B w₂) (C w₃)⟧ = A with probability w₁/(w₁+w₂+w₃) - = B with probability w₂/(w₁+w₂+w₃) - = C with probability w₃/(w₁+w₂+w₃) -``` - -### Conditional Bet - -```racket -(bet/conditional predicate A B C) -``` - -Deterministically returns A if predicate is true, otherwise performs a bet between B and C (with fallback to A). - -**Examples:** -```racket -(bet/conditional (> x 10) 'large 'medium 'small) -``` - -**Formal Semantics:** -``` -⟦bet/conditional pred A B C⟧ = A if pred = true - = ⟦bet B C A⟧ if pred = false -``` - -### Lazy Bet - -```racket -(bet/lazy thunk-a thunk-b thunk-c) -``` - -Delays evaluation - only the selected branch is computed, avoiding unnecessary computation. - -**Examples:** -```racket -(bet/lazy - (lambda () (expensive-computation-a)) - (lambda () (expensive-computation-b)) - (lambda () (expensive-computation-c))) -``` - -## Composition Operators - -### Bet Chain - -```racket -(bet-chain n f init) -``` - -Chains n probabilistic transformations together, threading results through function f. - -**Formal Semantics:** -``` -⟦bet-chain 0 f init⟧ = init -⟦bet-chain n f init⟧ = ⟦bet-chain (n-1) f (f init)⟧ -``` - -### Bet Compose - -```racket -(bet-compose f g h) -``` - -Creates a higher-order function that randomly selects one of three functions to apply. - -**Examples:** -```racket -(define process (bet-compose add1 identity sub1)) -(process 10) ; Returns 11, 10, or 9 -``` - -### Bet Map - -```racket -(bet-map f lst) -``` - -Probabilistically applies a function to list elements. - -### Bet Fold - -```racket -(bet-fold f init lst) -``` - -Fold with probabilistic choices at each step. - -## Parallel Operations - -### Bet Parallel - -```racket -(bet-parallel n A B C) -``` - -Runs n independent trials of the bet and returns all results as a list. - -**Examples:** -```racket -(bet-parallel 100 'heads 'tails 'edge) -; Returns list of 100 coin flips -``` - -### Bet Sequence - -```racket -(bet-sequence bet1 bet2 bet3 ...) -``` - -Executes multiple bets in sequence and returns all results. - -## Control Flow - -### Bet Until - -```racket -(bet-until predicate thunk) -``` - -Repeatedly executes bet until predicate is satisfied. - -**Formal Semantics:** -``` -⟦bet-until pred thunk⟧ = result where result is first value from thunk satisfying pred -``` - -### Bet Repeat - -```racket -(bet-repeat n thunk) -``` - -Executes bet n times and collects results. - -## Statistical Utilities - -### Bet Probability - -```racket -(bet-probability n predicate A B C) -``` - -Estimates probability that predicate holds by running n trials. - -**Examples:** -```racket -(bet-probability 10000 (lambda (x) (equal? x 'A)) 'A 'B 'C) -; Returns approximately 0.333 -``` - -### Bet Entropy - -```racket -(bet-entropy samples) -``` - -Calculates Shannon entropy of bet outcomes in bits. - -**Formula:** -``` -H(X) = -Σ P(xᵢ) log₂ P(xᵢ) -``` - -For uniform ternary bet: H = log₂(3) ≈ 1.585 bits - -### Bet Expect - -```racket -(bet-expect n f A B C) -``` - -Calculates expected value of function f over n trials. - -**Formula:** -``` -E[f(X)] ≈ (1/n) Σ f(xᵢ) -``` - -## Determinism and Reproducibility - -### Bet With Seed - -```racket -(bet-with-seed seed thunk) -``` - -Executes bet with a specific random seed for reproducible results. - -**Examples:** -```racket -(bet-with-seed 42 (lambda () (bet 1 2 3))) -; Always returns the same value for seed 42 -``` - -## Type System (Informal) - -Betlang is dynamically typed through Racket, but conceptually: - -``` -Bet[A, B, C] :: Type A → Type B → Type C → Bet (A | B | C) -``` - -Where `Bet T` represents a probabilistic value of type T. - -## Probability Monad (Informal) - -The bet operations form a monad-like structure: - -```racket -return x = (bet x x x) ; Pure/deterministic value -bind m f = (bet (f (bet A B C)) ; Monadic bind - (f (bet A B C)) - (f (bet A B C))) -``` - -More formally available through `bet-pure` and `bet-bind` in combinators. - -## Equational Properties - -### Symmetry -For uniform bet: `(bet A B C) ≡ (bet B C A) ≡ (bet C A B)` (distributionally) - -### Idempotence -`(bet X X X) = X` (deterministic) - -### Commutativity (distributional) -`(bet A B C)` and `(bet B A C)` have same distribution up to permutation - -## Error Semantics - -All bet operations propagate Racket exceptions. Invalid arguments raise `exn:fail` exceptions. - -## Memory Semantics - -Bets are evaluated eagerly by default (except `bet/lazy`). Results are not memoized unless explicitly using `bet-memoize` combinator. - -## Concurrency Semantics - -Bet operations are not thread-safe by default. Use Racket's synchronization primitives for concurrent access. - -## Integration with Racket - -Betlang is implemented as a Racket library. All Racket primitives and libraries are available. Bets can be freely mixed with Racket code: - -```racket -(define result (bet 1 2 3)) -(if (> result 2) - (displayln "Large!") - (displayln "Small!")) -``` - -## Advanced Semantics - -### Probability Distributions - -Through `lib/distributions.rkt`, betlang provides: -- Discrete: binomial, geometric, Poisson, multinomial -- Continuous: normal, exponential, gamma, beta -- Stochastic processes: random walks, Brownian motion - -### Markov Chains - -Through `lib/markov.rkt`, betlang supports: -- Discrete-time Markov chains -- Transition matrix learning -- Stationary distribution estimation -- Hidden Markov Models (simplified) - -### Statistical Inference - -Through `lib/statistics.rkt`: -- Descriptive statistics -- Hypothesis testing -- Resampling methods (bootstrap, jackknife) -- Time series analysis - -## Notation Conventions - -Throughout this documentation: -- `⟦expr⟧` denotes semantic interpretation -- `A ≡ B` denotes distributional equivalence -- `P(event)` denotes probability -- `E[X]` denotes expected value -- `H(X)` denotes entropy - -## Future Extensions - -Potential semantic extensions under consideration: -- Continuous probability support -- Bayesian inference primitives -- Automatic differentiation for probabilistic programs -- Parallel/distributed execution model diff --git a/docs/tech-debt-2026-05-26.adoc b/docs/tech-debt-2026-05-26.adoc new file mode 100644 index 0000000..3ca0a16 --- /dev/null +++ b/docs/tech-debt-2026-05-26.adoc @@ -0,0 +1,84 @@ +== Tech-Debt Audit — betlang — 2026-05-26 + +*Source:* estate-wide automated scan 2026-05-26. *Companion:* +https://github.com/hyperpolymath/standards/tree/main/docs/audits[`+hyperpolymath/standards+` +2026-05-26-estate-*-debt audits]. *Combined severity:* `+LOW+`. + +This file records the _raw findings_ — it does not by itself fix the +debt. Each section ends with a '`Recommended next move`' line; closing +the debt is follow-up work. + +=== 1. Proof debt + +Scanner counted the following markers in proof-bearing files of this +repo: + +.... +files= 10 | Coq-Axm/Adm= 0 | Lean-srry/ax= 5 | Agda-pst= 0 | Idr-blv= 0 | Idr-prtl= 0 | Fstr-asm= 0 | TODO= 0 | Unsafe= 0 +.... + +*Total markers:* 5. *Severity:* `+>05+`. + +*Marker types* (any non-zero counts above): - Coq `+Axiom+`/`+Admitted+` +— unconditional proof escapes. - Lean `+sorry+`/`+axiom+` — Lean’s +equivalent. - Agda `+postulate+` — accepted axiomatically. - Idris2 +`+believe_me+`/`+assert_total+` — runtime-safe coercion / totality +assumption. - Idris2 top-level `+partial+` — totality-check waived. - F* +`+assume val+`/`+admit_p+` — F* admit. - `+TODO PROOF+` / `+OWED:+` — +self-documented debt markers. - `+unsafePerformIO+`/`+unsafeCoerce+` — +soundness-relevant escape hatches in Haskell/Rust source. + +*Recommended next move:* triage each finding into one of: (a) discharge +by proof, (b) cover with property-tests + a documented refutation +budget, or (c) annotate as a known/necessary axiom (e.g. `+funExt+`) in +`+docs/proof-debt.md+`. + +=== 2. Licence debt + +[cols=",",options="header",] +|=== +|Field |Value +|LICENSE file |`+LICENSE+` +|SPDX header |`+MPL-2.0+` +|Manifest licence |`+MIT OR Apache-2.0+` +|Body classifier |`+Palimp-MPL-2.0+` +|Severity |`+ok+` +|=== + +*Recommended next move:* none for licence. + +=== 3. Documentation debt + +[cols=",",options="header",] +|=== +|Field |Value +|README lines |549 +|`+docs/+` files |10 +|`+docs/+` LoC |2837 +|CHANGELOG.md |Y +|CONTRIBUTING.md |Y +|CODE_OF_CONDUCT.md |Y +|SECURITY.md |Y +|Severity |`+LOW+` +|=== + +*Recommended next move:* `+docs/+` has only 10 file(s). Aim for ≥10 +organised docs (architecture, usage, contributing-guide, +troubleshooting, design-decisions). The user’s bar for a +"`heavily-developed and well-organised wiki`" is ≥10 files with topical +organisation. + +=== Cross-references + +* Estate proof-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-proof-debt.md+` +* Estate licence-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-licence-debt.md+` +* Estate documentation-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-documentation-debt.md+` + +''''' + +🤖 Generated by Claude Code estate-wide tech-debt scan (2026-05-26). +This file is informational — closing the debt is follow-up work owned by +the maintainer. diff --git a/docs/tech-debt-2026-05-26.md b/docs/tech-debt-2026-05-26.md deleted file mode 100644 index a28375d..0000000 --- a/docs/tech-debt-2026-05-26.md +++ /dev/null @@ -1,70 +0,0 @@ - -# Tech-Debt Audit — betlang — 2026-05-26 - -**Source:** estate-wide automated scan 2026-05-26. -**Companion:** [`hyperpolymath/standards` 2026-05-26-estate-*-debt audits](https://github.com/hyperpolymath/standards/tree/main/docs/audits). -**Combined severity:** `LOW`. - -This file records the *raw findings* — it does not by itself fix the debt. Each section ends with a 'Recommended next move' line; closing the debt is follow-up work. - -## 1. Proof debt - -Scanner counted the following markers in proof-bearing files of this repo: - -``` -files= 10 | Coq-Axm/Adm= 0 | Lean-srry/ax= 5 | Agda-pst= 0 | Idr-blv= 0 | Idr-prtl= 0 | Fstr-asm= 0 | TODO= 0 | Unsafe= 0 -``` - -**Total markers:** 5. **Severity:** `>05`. - -**Marker types** (any non-zero counts above): -- Coq `Axiom`/`Admitted` — unconditional proof escapes. -- Lean `sorry`/`axiom` — Lean's equivalent. -- Agda `postulate` — accepted axiomatically. -- Idris2 `believe_me`/`assert_total` — runtime-safe coercion / totality assumption. -- Idris2 top-level `partial` — totality-check waived. -- F\* `assume val`/`admit_p` — F\* admit. -- `TODO PROOF` / `OWED:` — self-documented debt markers. -- `unsafePerformIO`/`unsafeCoerce` — soundness-relevant escape hatches in Haskell/Rust source. - -**Recommended next move:** triage each finding into one of: (a) discharge by proof, (b) cover with property-tests + a documented refutation budget, or (c) annotate as a known/necessary axiom (e.g. `funExt`) in `docs/proof-debt.md`. - -## 2. Licence debt - -| Field | Value | -|---|---| -| LICENSE file | `LICENSE` | -| SPDX header | `MPL-2.0` | -| Manifest licence | `MIT OR Apache-2.0` | -| Body classifier | `Palimp-MPL-2.0` | -| Severity | `ok` | - -**Recommended next move:** none for licence. - -## 3. Documentation debt - -| Field | Value | -|---|---| -| README lines | 549 | -| `docs/` files | 10 | -| `docs/` LoC | 2837 | -| CHANGELOG.md | Y | -| CONTRIBUTING.md | Y | -| CODE_OF_CONDUCT.md | Y | -| SECURITY.md | Y | -| Severity | `LOW` | - -**Recommended next move:** `docs/` has only 10 file(s). Aim for ≥10 organised docs (architecture, usage, contributing-guide, troubleshooting, design-decisions). The user's bar for a "heavily-developed and well-organised wiki" is ≥10 files with topical organisation. - -## Cross-references - -- Estate proof-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-proof-debt.md` -- Estate licence-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-licence-debt.md` -- Estate documentation-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-documentation-debt.md` - ---- - -🤖 Generated by Claude Code estate-wide tech-debt scan (2026-05-26). This file is informational — closing the debt is follow-up work owned by the maintainer. diff --git a/docs/tutorial.md b/docs/tutorial.adoc similarity index 60% rename from docs/tutorial.md rename to docs/tutorial.adoc index 4f554f2..1db3614 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.adoc @@ -1,83 +1,94 @@ - -# betlang Tutorial +== betlang Tutorial -Welcome to betlang! This tutorial will guide you through the ternary probabilistic programming language from basics to advanced topics. +Welcome to betlang! This tutorial will guide you through the ternary +probabilistic programming language from basics to advanced topics. -## Table of Contents +=== Table of Contents -1. [Introduction](#introduction) -2. [Installation & Setup](#installation--setup) -3. [Your First Bet](#your-first-bet) -4. [Understanding Probability](#understanding-probability) -5. [Weighted Bets](#weighted-bets) -6. [Composition & Chaining](#composition--chaining) -7. [Statistical Analysis](#statistical-analysis) -8. [Probability Distributions](#probability-distributions) -9. [Markov Chains](#markov-chains) -10. [Real-World Applications](#real-world-applications) +[arabic] +. link:#introduction[Introduction] +. link:++#installation--setup++[Installation & Setup] +. link:#your-first-bet[Your First Bet] +. link:#understanding-probability[Understanding Probability] +. link:#weighted-bets[Weighted Bets] +. link:++#composition--chaining++[Composition & Chaining] +. link:#statistical-analysis[Statistical Analysis] +. link:#probability-distributions[Probability Distributions] +. link:#markov-chains[Markov Chains] +. link:#real-world-applications[Real-World Applications] -## Introduction +=== Introduction -Betlang is a Domain-Specific Language (DSL) for probabilistic programming built on Racket. Its core primitive is the **ternary bet** - a choice between three values with associated probabilities. This simple primitive enables elegant expression of complex probabilistic models. +Betlang is a Domain-Specific Language (DSL) for probabilistic +programming built on Racket. Its core primitive is the *ternary bet* - a +choice between three values with associated probabilities. This simple +primitive enables elegant expression of complex probabilistic models. -### Why Ternary? +==== Why Ternary? -Most probabilistic languages focus on binary choices. Betlang's ternary approach: -- Models three-way decisions naturally (win/draw/lose, yes/no/maybe) -- Provides richer expressiveness than binary -- Inspired by musical ternary form (A-B-A) -- Creates interesting emergent properties +Most probabilistic languages focus on binary choices. Betlang’s ternary +approach: - Models three-way decisions naturally (win/draw/lose, +yes/no/maybe) - Provides richer expressiveness than binary - Inspired by +musical ternary form (A-B-A) - Creates interesting emergent properties -## Installation & Setup +=== Installation & Setup -### Prerequisites +==== Prerequisites -- Racket 7.0 or later +* Racket 7.0 or later -### Getting Started +==== Getting Started -1. Clone the repository: -```bash +[arabic] +. Clone the repository: + +[source,bash] +---- git clone https://github.com/hyperpolymath/betlang.git cd betlang -``` +---- + +[arabic, start=2] +. Start the REPL: -2. Start the REPL: -```bash +[source,bash] +---- racket repl/shell.rkt -``` +---- + +[arabic, start=3] +. You should see: -3. You should see: -``` +.... ╔══════════════════════════════════════════╗ ║ 🎰 Welcome to betlang REPL v2.0 ║ ║ Ternary Probabilistic Programming ║ ╚══════════════════════════════════════════╝ -``` +.... -## Your First Bet +=== Your First Bet -Let's start with the simplest bet - choosing between three values: +Let’s start with the simplest bet - choosing between three values: -```racket +[source,racket] +---- betlang> (bet 1 2 3) 2 betlang> (bet 1 2 3) 3 betlang> (bet 1 2 3) 1 -``` +---- -Each time you run it, you get one of the three values. Each has equal probability: 1/3. +Each time you run it, you get one of the three values. Each has equal +probability: 1/3. -### Try Different Types +==== Try Different Types Bets work with any values: -```racket +[source,racket] +---- betlang> (bet 'red 'green 'blue) green @@ -86,13 +97,14 @@ betlang> (bet "Win" "Draw" "Lose") betlang> (bet #t #f 'maybe) maybe -``` +---- -### Bets in Expressions +==== Bets in Expressions Bets integrate seamlessly with Racket code: -```racket +[source,racket] +---- betlang> (+ 10 (bet 1 2 3)) 12 @@ -100,16 +112,18 @@ betlang> (if (equal? (bet 'a 'b 'c) 'a) "Got A!" "Not A") "Got A!" -``` +---- -## Keyword-delimited Syntax (preferred) +=== Keyword-delimited Syntax (preferred) -Betlang now exposes a Julia-style keyword syntax where each block ends with `end`. -It desugars directly to the same AST as the brace-based form, but many users find it -easier to read and write. The brace form is still accepted for backwards compatibility, -but the `end` style is the recommended one going forward. +Betlang now exposes a Julia-style keyword syntax where each block ends +with `+end+`. It desugars directly to the same AST as the brace-based +form, but many users find it easier to read and write. The brace form is +still accepted for backwards compatibility, but the `+end+` style is the +recommended one going forward. -```betlang +[source,betlang] +---- if x > 0 then bet "Positive" "Zero" "Negative" else @@ -139,63 +153,70 @@ end parallel 4 do sample normal end -``` +---- -Each keyword form (`bet`, `let … in`, `if … then … else`, `match`, `do`, `parallel`) can resurface general-purpose expressions, and the trailing `end` keeps the structure explicit. Old-style `{}`/`,` grouping is still parsed, but aim for `end` blocks for new code. +Each keyword form (`+bet+`, `+let … in+`, `+if … then … else+`, +`+match+`, `+do+`, `+parallel+`) can resurface general-purpose +expressions, and the trailing `+end+` keeps the structure explicit. +Old-style `+{}+`/`+,+` grouping is still parsed, but aim for `+end+` +blocks for new code. -## Understanding Probability +=== Understanding Probability -### Running Multiple Trials +==== Running Multiple Trials -Use `bet-parallel` to run many trials: +Use `+bet-parallel+` to run many trials: -```racket +[source,racket] +---- betlang> (bet-parallel 10 'heads 'tails 'edge) (heads edge tails heads heads tails heads edge tails heads) -``` +---- -### Estimating Probabilities +==== Estimating Probabilities -Let's verify the 1/3 probability: +Let’s verify the 1/3 probability: -```racket +[source,racket] +---- betlang> (bet-probability 10000 (lambda (x) (equal? x 'heads)) 'heads 'tails 'edge) 0.3337 -``` +---- Close to 0.333 (1/3)! -### Frequency Analysis +==== Frequency Analysis Count how often each outcome appears: -```racket +[source,racket] +---- betlang> (require "../lib/statistics.rkt") betlang> (define results (bet-parallel 1000 'A 'B 'C)) betlang> (frequency-table results) ((A . 334) (B . 333) (C . 333)) -``` +---- -## Weighted Bets +=== Weighted Bets Sometimes you need non-uniform probabilities: -```racket +[source,racket] +---- betlang> (bet/weighted '(common 6) '(uncommon 3) '(rare 1)) -``` +---- -This creates: -- P(common) = 6/10 = 60% -- P(uncommon) = 3/10 = 30% -- P(rare) = 1/10 = 10% +This creates: - P(common) = 6/10 = 60% - P(uncommon) = 3/10 = 30% - +P(rare) = 1/10 = 10% -### Practical Example: Loot Drops +==== Practical Example: Loot Drops Simulating game loot with varying rarity: -```racket +[source,racket] +---- (define (drop-loot) (bet/weighted '("Common Sword" 7) @@ -207,15 +228,16 @@ Simulating game loot with varying rarity: (drop-loot))) (frequency-table drops) -``` +---- -## Composition & Chaining +=== Composition & Chaining -### Bet Composition +==== Bet Composition Create higher-order probabilistic functions: -```racket +[source,racket] +---- betlang> (define random-op (bet-compose add1 identity sub1)) betlang> (random-op 10) 11 @@ -223,38 +245,41 @@ betlang> (random-op 10) 9 betlang> (random-op 10) 10 -``` +---- -### Chaining Bets +==== Chaining Bets Thread values through multiple probabilistic transformations: -```racket +[source,racket] +---- (define (step x) (+ x (bet -1 0 1))) (bet-chain 10 step 0) ; Random walk of 10 steps -``` +---- -### Lazy Evaluation +==== Lazy Evaluation Only compute the branch you need: -```racket +[source,racket] +---- (bet/lazy (lambda () (displayln "Computing A") 'A) (lambda () (displayln "Computing B") 'B) (lambda () (displayln "Computing C") 'C)) ;; Prints only one message! -``` +---- -## Statistical Analysis +=== Statistical Analysis -### Basic Statistics +==== Basic Statistics -```racket +[source,racket] +---- (require "../lib/statistics.rkt") (define data (bet-parallel 1000 1 2 3)) @@ -263,43 +288,47 @@ Only compute the branch you need: (median data) ; 2 (stddev data) ; ~0.816 (variance data) ; ~0.666 -``` +---- -### Entropy +==== Entropy Measure information content: -```racket +[source,racket] +---- (define samples (bet-parallel 1000 'A 'B 'C)) (bet-entropy samples) ; ~1.585 bits (maximum for 3 outcomes) -``` +---- -### Expected Value +==== Expected Value -```racket +[source,racket] +---- (bet-expect 10000 (lambda (x) x) ; identity function 1 2 3) ; ~2.0 (mean of 1, 2, 3) -``` +---- -### Correlation +==== Correlation -```racket +[source,racket] +---- (define x-vals '(1 2 3 4 5)) (define y-vals '(2 4 6 8 10)) (correlation x-vals y-vals) ; 1.0 (perfect positive correlation) -``` +---- -## Probability Distributions +=== Probability Distributions Betlang provides many standard distributions: -### Discrete Distributions +==== Discrete Distributions -```racket +[source,racket] +---- (require "../lib/distributions.rkt") ;; Binomial: n trials, probability p @@ -313,11 +342,12 @@ Betlang provides many standard distributions: ;; Poisson: events in fixed interval (poisson 5.0) ; Number of events with rate 5 -``` +---- -### Continuous Distributions +==== Continuous Distributions -```racket +[source,racket] +---- ;; Normal (Gaussian) (normal 0 1) ; Standard normal @@ -332,11 +362,12 @@ Betlang provides many standard distributions: ;; Gamma (gamma 2 2) ; Waiting time for multiple events -``` +---- -### Stochastic Processes +==== Stochastic Processes -```racket +[source,racket] +---- ;; Random Walk (random-walk 100) ; 100-step random walk @@ -344,13 +375,14 @@ Betlang provides many standard distributions: ;; Brownian Motion (brownian-motion 100 0.01) ; Continuous random walk -``` +---- -## Markov Chains +=== Markov Chains Model systems with state transitions: -```racket +[source,racket] +---- (require "../lib/markov.rkt") ;; Weather model @@ -365,11 +397,12 @@ Model systems with state transitions: ;; Simulate a week (markov-simulate weather-chain 7) ; (sunny sunny cloudy rainy rainy rainy cloudy sunny) -``` +---- -### Learning from Data +==== Learning from Data -```racket +[source,racket] +---- ;; Observe sequence (define data '(A B A C A B C C A B A)) @@ -385,38 +418,41 @@ Model systems with state transitions: ;; Generate new sequences (markov-simulate learned-chain 20) -``` +---- -## Real-World Applications +=== Real-World Applications -### 1. Monte Carlo Simulation +==== 1. Monte Carlo Simulation Estimate π using random sampling: -```racket +[source,racket] +---- (require "../examples/monte-carlo.rkt") (monte-carlo-pi 100000) ; ~3.14159 -``` +---- -### 2. Game Theory +==== 2. Game Theory Simulate repeated games: -```racket +[source,racket] +---- (require "../examples/game-theory.rkt") ;; Rock-Paper-Scissors tournament (define (random-rps) (bet 'rock 'paper 'scissors)) (play-rps random-rps random-rps 1000) -``` +---- -### 3. A/B Testing +==== 3. A/B Testing Compare two strategies: -```racket +[source,racket] +---- (define strategy-a (lambda () (bet/weighted '(success 3) '(neutral 1) '(failure 1)))) @@ -431,13 +467,14 @@ Compare two strategies: (mean (map (lambda (x) (if (equal? x 'success) 1 0)) results-b)) ; Success rate for B -``` +---- -### 4. Risk Analysis +==== 4. Risk Analysis Model uncertain outcomes: -```racket +[source,racket] +---- (define (project-outcome) (bet/weighted '("Success: $1M" 5) @@ -449,11 +486,12 @@ Model uncertain outcomes: ;; Analyze risk (frequency-table outcomes) -``` +---- -### 5. Traffic Simulation +==== 5. Traffic Simulation -```racket +[source,racket] +---- (define (traffic-light-duration) (bet 30 45 60)) ; Seconds @@ -464,15 +502,16 @@ Model uncertain outcomes: (define total-cars (for/sum ([i (in-range 60)]) ; 60 cycles per hour (cars-passing))) -``` +---- -## Advanced Topics +=== Advanced Topics -### Conditional Betting +==== Conditional Betting Make decisions based on conditions: -```racket +[source,racket] +---- (define temperature 75) (bet/conditional @@ -480,13 +519,14 @@ Make decisions based on conditions: 'swim ; If hot 'hike ; If not (random between these) 'read) ; If not -``` +---- -### Bet Combinators +==== Bet Combinators Build complex behaviors from simple parts: -```racket +[source,racket] +---- (require "../lib/combinators.rkt") ;; Retry with fallback @@ -507,11 +547,12 @@ Build complex behaviors from simple parts: ;; Subsequent calls: instant (expensive-bet) ; Returns immediately! -``` +---- -### Analysis Tools +==== Analysis Tools -```racket +[source,racket] +---- (require "../tools/analyzer.rkt") ;; Comprehensive analysis @@ -524,72 +565,72 @@ Build complex behaviors from simple parts: (lambda () (bet 'heads 'tails 'edge)) 'heads 10000) -``` +---- -## Best Practices +=== Best Practices -### 1. Use Seeds for Reproducibility +==== 1. Use Seeds for Reproducibility -```racket +[source,racket] +---- (bet-with-seed 42 (lambda () (bet-parallel 10 1 2 3))) ; Same result every time with seed 42 -``` +---- -### 2. Verify Distributions +==== 2. Verify Distributions Always check your probability distribution makes sense: -```racket +[source,racket] +---- (define results (bet-parallel 10000 'A 'B 'C)) (frequency-table results) ; Should be roughly equal -``` +---- -### 3. Test with Large Samples +==== 3. Test with Large Samples Probabilistic code needs statistical validation: -```racket +[source,racket] +---- (define prob (bet-probability 100000 (lambda (x) (equal? x 'rare)) 'common 'uncommon 'rare)) (check-true (< 0.30 prob 0.36)) ; Should be ~1/3 -``` +---- -### 4. Profile Performance +==== 4. Profile Performance -```racket +[source,racket] +---- (time (bet-parallel 1000000 1 2 3)) ; Measure execution time -``` +---- -## Next Steps +=== Next Steps -- Explore the [API Reference](api-reference.md) for complete function documentation -- Read the [Semantics](semantics.md) for formal specifications -- Check out [examples/](../examples/) for more complex programs -- Run tests with `racket tests/basics.rkt` -- Try the analyzer: `racket tools/analyzer.rkt` +* Explore the link:api-reference.md[API Reference] for complete function +documentation +* Read the link:semantics.md[Semantics] for formal specifications +* Check out link:../examples/[examples/] for more complex programs +* Run tests with `+racket tests/basics.rkt+` +* Try the analyzer: `+racket tools/analyzer.rkt+` -## Getting Help +=== Getting Help -- Type `:help` in the REPL for command reference -- Type `:examples` for quick examples -- Read the source code - it's well-commented! -- Check out the examples in `examples/` directory +* Type `+:help+` in the REPL for command reference +* Type `+:examples+` for quick examples +* Read the source code - it’s well-commented! +* Check out the examples in `+examples/+` directory -## Summary +=== Summary -You've learned: -- ✓ Basic bet syntax and usage -- ✓ Weighted probability distributions -- ✓ Composition and chaining -- ✓ Statistical analysis -- ✓ Probability distributions -- ✓ Markov chains -- ✓ Real-world applications +You’ve learned: - ✓ Basic bet syntax and usage - ✓ Weighted probability +distributions - ✓ Composition and chaining - ✓ Statistical analysis - ✓ +Probability distributions - ✓ Markov chains - ✓ Real-world applications Happy betting! 🎲 diff --git a/docs/v3-optimization-workflow.adoc b/docs/v3-optimization-workflow.adoc new file mode 100644 index 0000000..56ab41d --- /dev/null +++ b/docs/v3-optimization-workflow.adoc @@ -0,0 +1,38 @@ +== Betlang v3 optimization workflow + +=== Purpose + +This workflow keeps the Betlang core fast and stable while giving you a +repeatable path to exercised "`optimized`" artifacts. + +=== Steps + +[arabic] +. *Configure Rust toolchain* +* Run `+rustup default stable+` (or your preferred channel). The +optimized build uses the Rust compiler, so a working toolchain is +required before you run `+just build-v3+`. +. *Run the optimization driver* +* Execute `+just build-v3+`. That currently delegates to +`+cargo build --release+` via `+build-tooling+` and is the entry point +for v3 release artifacts. +* When the toolchain is available, this command produces release-mode +Rust binaries that you can ship or plug into other tools. +. *Verify tests (optional but recommended)* +* Run `+cargo test+`/`+cargo test --release+` in `+compiler/bet-parse+` +to make sure your optimized binaries are still correct. +* Continue running Racket smoke tests (`+just test+`, +`+racket lib/number-systems.rkt+`, `+examples/safety-features.rkt+`) as +part of your nightly verification. +. *Document results* +* After each optimized build, note the toolchain version and outcomes in +your release notes or logs (e.g., `+logs/v3-optimized-YYYYMMDD.md+`). + +=== Notes + +* The `+build-v3+` target is intentionally lightweight: it acknowledges +Rust’s release build as the proxy for optimization while keeping the +Racket core unchanged. +* If you add other optimized artifacts (e.g., precompiled Racket files +or Julia caches), extend this workflow and `+Justfile+` recipe +accordingly. diff --git a/docs/v3-optimization-workflow.md b/docs/v3-optimization-workflow.md deleted file mode 100644 index 626dfd7..0000000 --- a/docs/v3-optimization-workflow.md +++ /dev/null @@ -1,30 +0,0 @@ - -# Betlang v3 optimization workflow - -## Purpose - -This workflow keeps the Betlang core fast and stable while giving you a repeatable path to exercised "optimized" artifacts. - -## Steps - -1. **Configure Rust toolchain** - - Run `rustup default stable` (or your preferred channel). The optimized build uses the Rust compiler, so a working toolchain is required before you run `just build-v3`. - -2. **Run the optimization driver** - - Execute `just build-v3`. That currently delegates to `cargo build --release` via `build-tooling` and is the entry point for v3 release artifacts. - - When the toolchain is available, this command produces release-mode Rust binaries that you can ship or plug into other tools. - -3. **Verify tests (optional but recommended)** - - Run `cargo test`/`cargo test --release` in `compiler/bet-parse` to make sure your optimized binaries are still correct. - - Continue running Racket smoke tests (`just test`, `racket lib/number-systems.rkt`, `examples/safety-features.rkt`) as part of your nightly verification. - -4. **Document results** - - After each optimized build, note the toolchain version and outcomes in your release notes or logs (e.g., `logs/v3-optimized-YYYYMMDD.md`). - -## Notes - -- The `build-v3` target is intentionally lightweight: it acknowledges Rust’s release build as the proxy for optimization while keeping the Racket core unchanged. -- If you add other optimized artifacts (e.g., precompiled Racket files or Julia caches), extend this workflow and `Justfile` recipe accordingly. diff --git a/editors/neovim/README.adoc b/editors/neovim/README.adoc new file mode 100644 index 0000000..84d1551 --- /dev/null +++ b/editors/neovim/README.adoc @@ -0,0 +1,123 @@ +== Betlang Neovim/Vim Plugin + +Language support for Betlang in Neovim (with Lua) and Vim. + +=== Features + +* Syntax highlighting +* LSP integration (via bet-lsp) +* Integrated REPL +* Run files directly +* Send code to REPL + +=== Installation + +==== Using lazy.nvim + +[source,lua] +---- +{ + "hyperpolymath/betlang", + config = function() + require("betlang").setup({ + lsp = { + enabled = true, + }, + repl = { + split = "horizontal", + size = 15, + }, + }) + end, + ft = { "betlang", "bet" }, +} +---- + +==== Using packer.nvim + +[source,lua] +---- +use { + "hyperpolymath/betlang", + config = function() + require("betlang").setup() + end, + ft = { "betlang", "bet" }, +} +---- + +==== Using vim-plug + +[source,vim] +---- +Plug 'hyperpolymath/betlang' +---- + +==== Manual Installation + +Copy the contents of this directory to: - Neovim: +`+~/.config/nvim/pack/plugins/start/betlang/+` - Vim: +`+~/.vim/pack/plugins/start/betlang/+` + +=== Configuration + +[source,lua] +---- +require("betlang").setup({ + -- LSP settings + lsp = { + enabled = true, -- Enable LSP + cmd = nil, -- Custom LSP command (auto-detect if nil) + settings = {}, -- LSP settings + }, + -- REPL settings + repl = { + cmd = nil, -- Custom REPL command (auto-detect if nil) + split = "horizontal", -- "horizontal", "vertical", or "float" + size = 15, -- Split size + }, + -- Highlighting + highlight = { + enabled = true, + }, +}) +---- + +=== Commands + +[cols=",",options="header",] +|=== +|Command |Description +|`+:BetlangRepl+` |Open REPL +|`+:BetlangReplClose+` |Close REPL +|`+:BetlangReplToggle+` |Toggle REPL +|`+:BetlangSendLine+` |Send current line to REPL +|`+:BetlangSendSelection+` |Send selection to REPL +|`+:BetlangRun+` |Run current file +|=== + +=== Key Mappings + +Default mappings (buffer-local for .bet files): + +[cols=",",options="header",] +|=== +|Mapping |Description +|`+br+` |Open REPL +|`+bt+` |Toggle REPL +|`+bl+` |Send line to REPL +|`+bs+` |Send selection to REPL (visual mode) +|`+bx+` |Run current file +|=== + +=== Requirements + +* Neovim 0.9+ (for full Lua support) or Vim 8+ +* https://github.com/neovim/nvim-lspconfig[nvim-lspconfig] (for LSP +support) +* bet-lsp (for language server features) +* bet-cli or Racket (for REPL) + +=== License + +MIT OR Apache-2.0 diff --git a/editors/neovim/README.md b/editors/neovim/README.md deleted file mode 100644 index 4484014..0000000 --- a/editors/neovim/README.md +++ /dev/null @@ -1,118 +0,0 @@ - -# Betlang Neovim/Vim Plugin - -Language support for Betlang in Neovim (with Lua) and Vim. - -## Features - -- Syntax highlighting -- LSP integration (via bet-lsp) -- Integrated REPL -- Run files directly -- Send code to REPL - -## Installation - -### Using lazy.nvim - -```lua -{ - "hyperpolymath/betlang", - config = function() - require("betlang").setup({ - lsp = { - enabled = true, - }, - repl = { - split = "horizontal", - size = 15, - }, - }) - end, - ft = { "betlang", "bet" }, -} -``` - -### Using packer.nvim - -```lua -use { - "hyperpolymath/betlang", - config = function() - require("betlang").setup() - end, - ft = { "betlang", "bet" }, -} -``` - -### Using vim-plug - -```vim -Plug 'hyperpolymath/betlang' -``` - -### Manual Installation - -Copy the contents of this directory to: -- Neovim: `~/.config/nvim/pack/plugins/start/betlang/` -- Vim: `~/.vim/pack/plugins/start/betlang/` - -## Configuration - -```lua -require("betlang").setup({ - -- LSP settings - lsp = { - enabled = true, -- Enable LSP - cmd = nil, -- Custom LSP command (auto-detect if nil) - settings = {}, -- LSP settings - }, - -- REPL settings - repl = { - cmd = nil, -- Custom REPL command (auto-detect if nil) - split = "horizontal", -- "horizontal", "vertical", or "float" - size = 15, -- Split size - }, - -- Highlighting - highlight = { - enabled = true, - }, -}) -``` - -## Commands - -| Command | Description | -|---------|-------------| -| `:BetlangRepl` | Open REPL | -| `:BetlangReplClose` | Close REPL | -| `:BetlangReplToggle` | Toggle REPL | -| `:BetlangSendLine` | Send current line to REPL | -| `:BetlangSendSelection` | Send selection to REPL | -| `:BetlangRun` | Run current file | - -## Key Mappings - -Default mappings (buffer-local for .bet files): - -| Mapping | Description | -|---------|-------------| -| `br` | Open REPL | -| `bt` | Toggle REPL | -| `bl` | Send line to REPL | -| `bs` | Send selection to REPL (visual mode) | -| `bx` | Run current file | - -## Requirements - -- Neovim 0.9+ (for full Lua support) or Vim 8+ -- [nvim-lspconfig](https://github.com/neovim/nvim-lspconfig) (for LSP support) -- bet-lsp (for language server features) -- bet-cli or Racket (for REPL) - -## License - -MIT OR Apache-2.0 diff --git a/editors/vscode/README.adoc b/editors/vscode/README.adoc new file mode 100644 index 0000000..6f4b85e --- /dev/null +++ b/editors/vscode/README.adoc @@ -0,0 +1,43 @@ +== Betlang for Visual Studio Code + +Language support for https://github.com/hyperpolymath/betlang[Betlang] — +a ternary probabilistic programming language. + +=== Features + +* Syntax highlighting (TextMate grammar, +`+syntaxes/betlang.tmLanguage.json+`) +* Language Server Protocol client — spawns `+bet-lsp+` for diagnostics, +completion, and hover +* Commands: Start/Stop REPL, Evaluate Selection, Restart Language Server + +=== Implementation + +The extension is written in *AffineScript* (`+src/extension.affine+`), +per the hyperpolymath language policy under which AffineScript replaces +AffineScript/TypeScript for editor tooling. It compiles to +`+out/extension.cjs+`, which `+package.json+`’s `+main+` field points +to: + +[source,bash] +---- +affinescript compile src/extension.affine -o out/extension.cjs --vscode-extension +# or +npm run compile +---- + +Requires the https://github.com/hyperpolymath/affinescript[AffineScript +compiler]. + +____ +History: the previous AffineScript implementation (`+Extension.res+` / +`+VSCode.res+`) was removed 2026-06-02 when the extension was migrated +to AffineScript. +____ + +=== Settings + +* `+betlang.lspPath+` — path to the `+bet-lsp+` executable (default: +`+bet-lsp+`) +* `+betlang.enableDiagnostics+`, `+betlang.enableCompletion+`, +`+betlang.enableHover+` diff --git a/editors/vscode/README.md b/editors/vscode/README.md deleted file mode 100644 index 0b6ff90..0000000 --- a/editors/vscode/README.md +++ /dev/null @@ -1,37 +0,0 @@ - - - -# Betlang for Visual Studio Code - -Language support for [Betlang](https://github.com/hyperpolymath/betlang) — a -ternary probabilistic programming language. - -## Features - -- Syntax highlighting (TextMate grammar, `syntaxes/betlang.tmLanguage.json`) -- Language Server Protocol client — spawns `bet-lsp` for diagnostics, - completion, and hover -- Commands: Start/Stop REPL, Evaluate Selection, Restart Language Server - -## Implementation - -The extension is written in **AffineScript** (`src/extension.affine`), per the -hyperpolymath language policy under which AffineScript replaces -AffineScript/TypeScript for editor tooling. It compiles to `out/extension.cjs`, -which `package.json`'s `main` field points to: - -```bash -affinescript compile src/extension.affine -o out/extension.cjs --vscode-extension -# or -npm run compile -``` - -Requires the [AffineScript compiler](https://github.com/hyperpolymath/affinescript). - -> History: the previous AffineScript implementation (`Extension.res` / `VSCode.res`) -> was removed 2026-06-02 when the extension was migrated to AffineScript. - -## Settings - -- `betlang.lspPath` — path to the `bet-lsp` executable (default: `bet-lsp`) -- `betlang.enableDiagnostics`, `betlang.enableCompletion`, `betlang.enableHover` diff --git a/julia-backend/README.adoc b/julia-backend/README.adoc new file mode 100644 index 0000000..953c654 --- /dev/null +++ b/julia-backend/README.adoc @@ -0,0 +1,268 @@ +== BetLang Julia Backend (v0.8.0-dev) + +High-performance Julia backend for the betlang probabilistic programming +language. + +=== Status + +*Phase:* Step 1 - Minimal Viable Backend ✅ *Completion:* 20% (basic bet +operations working) + +==== What Works + +* ✅ Core bet primitives (`+bet+`, `+bet_weighted+`, `+bet_parallel+`) +* ✅ Statistical utilities (`+bet_probability+`, `+bet_entropy+`) +* ✅ Conditional bets +* ✅ Deterministic (seeded) bets +* ✅ Test suite (8/8 passing) +* ✅ Basic compiler (betlang → Julia) + +==== What’s Next + +* [ ] Standard library functions (statistics, distributions) +* [ ] Number systems (14 types) +* [ ] Safety features (Dutch book, risk-of-ruin, cool-off) +* [ ] Full Racket compatibility + +=== Quick Start + +==== Prerequisites + +* Julia 1.9+ +* Racket (for compiler) +* just (task runner) + +==== Installation + +[source,bash] +---- +cd julia-backend +just install +---- + +==== Run Tests + +[source,bash] +---- +just test +---- + +==== Run Example + +[source,bash] +---- +just example +---- + +==== Compile BetLang Source + +[source,bash] +---- +# Compile .bet file to .jl +just compile examples/test.bet + +# Compile and run +just run examples/test.bet +---- + +=== Usage + +==== Direct Julia API + +[source,julia] +---- +using BetLang + +# Basic ternary bet +result = bet("heads", "tails", "edge") + +# Weighted bet +result = bet_weighted([("common", 0.7), ("rare", 0.3)]) + +# Parallel trials +results = bet_parallel(100, "win", "draw", "lose") + +# Probability estimation +prob = bet_probability(1000, x -> x == "heads", "heads", "tails", "edge") + +# Entropy calculation +H = bet_entropy(10000, "A", "B", "C") +---- + +==== Compiled BetLang + +Write betlang source (`+example.bet+`): + +[source,racket] +---- +;; Simple bet program +(define result (bet "heads" "tails" "edge")) +(display result) +---- + +Compile and run: + +[source,bash] +---- +just run example.bet +---- + +=== Architecture + +.... +BetLang (Racket) Source + ↓ + Compiler (Racket) + ↓ + Julia Code (Generated) + ↓ + BetLang.jl Runtime + ↓ + Distributions.jl +.... + +=== Performance + +Expected speedup vs Racket: - Monte Carlo (1M samples): *75x faster* - +Distribution sampling: *100x faster* - Statistical operations: *50x +faster* + +=== Integration with Julia Ecosystem + +==== Tier 0 (Foundation) - Current Focus + +* *Distributions.jl* ✅ - In use for Categorical distributions +* *StatsBase.jl* ✅ - Dependencies declared +* *Random.jl* ✅ - Used for seeded RNG +* Turing.jl ⏳ - Planned for Bayesian inference +* IntervalArithmetic.jl ⏳ - Planned for AffineNumber +* Measurements.jl ⏳ - Planned for DistnumberNormal + +==== Tier 1 (Extensions) - Future + +* BowtieRisk.jl - Risk analysis integration +* ZeroProb.jl - Zero-probability event handling +* Causals.jl - Causal inference +* Cliometrics.jl, Cliodynamics.jl, Cladistics.jl - Domain applications + +See `+docs/julia-backend-design.md+` for complete integration plan. + +=== Development + +==== Project Structure + +.... +julia-backend/ +├── Project.toml # Julia package metadata +├── src/ +│ └── BetLang.jl # Core module +├── test/ +│ └── runtests.jl # Test suite +├── examples/ +│ ├── basic.jl # Julia examples +│ └── test.bet # BetLang source +├── compiler/ +│ └── betlang-to-julia.rkt # Compiler +└── Justfile # Build recipes +.... + +==== Adding New Features + +[arabic] +. *Add to BetLang.jl*: Implement in `+src/BetLang.jl+` +. *Add tests*: Update `+test/runtests.jl+` +. *Update compiler*: Add translation in +`+compiler/betlang-to-julia.rkt+` +. *Run tests*: `+just test+` + +==== Running REPL + +[source,bash] +---- +just repl +---- + +Then: + +[source,julia] +---- +julia> using BetLang +julia> bet("a", "b", "c") +---- + +=== Comparison to Racket + +[cols=",,,",options="header",] +|=== +|Feature |Racket |Julia |Status +|Core bets |✅ |✅ |Complete +|Weighted bets |✅ |✅ |Complete +|Conditional |✅ |✅ |Complete +|Statistics |✅ |⏳ |Partial +|Distributions |✅ |⏳ |Planned +|Number systems |✅ |⏳ |Planned +|Safety features |✅ |⏳ |Planned +|Performance |1x |50-100x |✅ +|=== + +=== Roadmap + +==== ✅ Step 1: Minimal Viable Backend (Current) + +* Basic bet operations +* Simple compiler +* Test suite + +==== Step 2: Core Language (4 weeks) + +* All bet primitives +* Function definitions +* Control flow +* List operations + +==== Step 3: Standard Library (6 weeks) + +* Statistics functions +* Distribution operations +* Bayesian inference +* Optimization + +==== Step 4: Number Systems (4 weeks) + +* All 14 uncertainty-aware types +* Automatic error propagation +* Julia ecosystem integration + +==== Step 5: Safety Features (2 weeks) + +* Dutch book prevention +* Risk-of-ruin protection +* Cool-off mechanism + +==== Step 6: Optimization (4 weeks) + +* Type stability +* GPU acceleration +* Multi-threading +* Precompilation + +*Total Timeline:* 22 weeks (~5 months) + +=== Contributing + +See `+../CONTRIBUTING.adoc+` for contribution guidelines. + +=== License + +MPL-2.0 (MPL-2.0) + +=== Authors + +Jonathan D.A. Jewell j.d.a.jewell@open.ac.uk + +=== References + +* Main betlang docs: `+../README.adoc+` +* Julia backend design: `+../docs/julia-backend-design.md+` +* Hyperpolymath integration: +`+../docs/hyperpolymath-julia-integration.md+` diff --git a/julia-backend/README.md b/julia-backend/README.md deleted file mode 100644 index 0e3535e..0000000 --- a/julia-backend/README.md +++ /dev/null @@ -1,244 +0,0 @@ - -# BetLang Julia Backend (v0.8.0-dev) - -High-performance Julia backend for the betlang probabilistic programming language. - -## Status - -**Phase:** Step 1 - Minimal Viable Backend ✅ -**Completion:** 20% (basic bet operations working) - -### What Works -- ✅ Core bet primitives (`bet`, `bet_weighted`, `bet_parallel`) -- ✅ Statistical utilities (`bet_probability`, `bet_entropy`) -- ✅ Conditional bets -- ✅ Deterministic (seeded) bets -- ✅ Test suite (8/8 passing) -- ✅ Basic compiler (betlang → Julia) - -### What's Next -- [ ] Standard library functions (statistics, distributions) -- [ ] Number systems (14 types) -- [ ] Safety features (Dutch book, risk-of-ruin, cool-off) -- [ ] Full Racket compatibility - -## Quick Start - -### Prerequisites -- Julia 1.9+ -- Racket (for compiler) -- just (task runner) - -### Installation - -```bash -cd julia-backend -just install -``` - -### Run Tests - -```bash -just test -``` - -### Run Example - -```bash -just example -``` - -### Compile BetLang Source - -```bash -# Compile .bet file to .jl -just compile examples/test.bet - -# Compile and run -just run examples/test.bet -``` - -## Usage - -### Direct Julia API - -```julia -using BetLang - -# Basic ternary bet -result = bet("heads", "tails", "edge") - -# Weighted bet -result = bet_weighted([("common", 0.7), ("rare", 0.3)]) - -# Parallel trials -results = bet_parallel(100, "win", "draw", "lose") - -# Probability estimation -prob = bet_probability(1000, x -> x == "heads", "heads", "tails", "edge") - -# Entropy calculation -H = bet_entropy(10000, "A", "B", "C") -``` - -### Compiled BetLang - -Write betlang source (`example.bet`): -```racket -;; Simple bet program -(define result (bet "heads" "tails" "edge")) -(display result) -``` - -Compile and run: -```bash -just run example.bet -``` - -## Architecture - -``` -BetLang (Racket) Source - ↓ - Compiler (Racket) - ↓ - Julia Code (Generated) - ↓ - BetLang.jl Runtime - ↓ - Distributions.jl -``` - -## Performance - -Expected speedup vs Racket: -- Monte Carlo (1M samples): **75x faster** -- Distribution sampling: **100x faster** -- Statistical operations: **50x faster** - -## Integration with Julia Ecosystem - -### Tier 0 (Foundation) - Current Focus -- **Distributions.jl** ✅ - In use for Categorical distributions -- **StatsBase.jl** ✅ - Dependencies declared -- **Random.jl** ✅ - Used for seeded RNG -- Turing.jl ⏳ - Planned for Bayesian inference -- IntervalArithmetic.jl ⏳ - Planned for AffineNumber -- Measurements.jl ⏳ - Planned for DistnumberNormal - -### Tier 1 (Extensions) - Future -- BowtieRisk.jl - Risk analysis integration -- ZeroProb.jl - Zero-probability event handling -- Causals.jl - Causal inference -- Cliometrics.jl, Cliodynamics.jl, Cladistics.jl - Domain applications - -See `docs/julia-backend-design.md` for complete integration plan. - -## Development - -### Project Structure - -``` -julia-backend/ -├── Project.toml # Julia package metadata -├── src/ -│ └── BetLang.jl # Core module -├── test/ -│ └── runtests.jl # Test suite -├── examples/ -│ ├── basic.jl # Julia examples -│ └── test.bet # BetLang source -├── compiler/ -│ └── betlang-to-julia.rkt # Compiler -└── Justfile # Build recipes -``` - -### Adding New Features - -1. **Add to BetLang.jl**: Implement in `src/BetLang.jl` -2. **Add tests**: Update `test/runtests.jl` -3. **Update compiler**: Add translation in `compiler/betlang-to-julia.rkt` -4. **Run tests**: `just test` - -### Running REPL - -```bash -just repl -``` - -Then: -```julia -julia> using BetLang -julia> bet("a", "b", "c") -``` - -## Comparison to Racket - -| Feature | Racket | Julia | Status | -|---------|--------|-------|--------| -| Core bets | ✅ | ✅ | Complete | -| Weighted bets | ✅ | ✅ | Complete | -| Conditional | ✅ | ✅ | Complete | -| Statistics | ✅ | ⏳ | Partial | -| Distributions | ✅ | ⏳ | Planned | -| Number systems | ✅ | ⏳ | Planned | -| Safety features | ✅ | ⏳ | Planned | -| Performance | 1x | 50-100x | ✅ | - -## Roadmap - -### ✅ Step 1: Minimal Viable Backend (Current) -- Basic bet operations -- Simple compiler -- Test suite - -### Step 2: Core Language (4 weeks) -- All bet primitives -- Function definitions -- Control flow -- List operations - -### Step 3: Standard Library (6 weeks) -- Statistics functions -- Distribution operations -- Bayesian inference -- Optimization - -### Step 4: Number Systems (4 weeks) -- All 14 uncertainty-aware types -- Automatic error propagation -- Julia ecosystem integration - -### Step 5: Safety Features (2 weeks) -- Dutch book prevention -- Risk-of-ruin protection -- Cool-off mechanism - -### Step 6: Optimization (4 weeks) -- Type stability -- GPU acceleration -- Multi-threading -- Precompilation - -**Total Timeline:** 22 weeks (~5 months) - -## Contributing - -See `../CONTRIBUTING.adoc` for contribution guidelines. - -## License - -MPL-2.0 (MPL-2.0) - -## Authors - -Jonathan D.A. Jewell - -## References - -- Main betlang docs: `../README.adoc` -- Julia backend design: `../docs/julia-backend-design.md` -- Hyperpolymath integration: `../docs/hyperpolymath-julia-integration.md` diff --git a/proofs/README.adoc b/proofs/README.adoc new file mode 100644 index 0000000..e9c43b9 --- /dev/null +++ b/proofs/README.adoc @@ -0,0 +1,256 @@ +== Betlang Academic Proofs and Documentation + +=== Overview + +This directory contains comprehensive academic documentation for +betlang, including mathematical foundations, formal semantics, theorem +proofs, formal verification specifications, and academic papers. + +''''' + +=== Document Index + +==== Core Mathematical Foundations + +[width="100%",cols="34%,41%,25%",options="header",] +|=== +|Document |Description |Status +|link:./mathematical-foundations.md[mathematical-foundations.md] +|Probability spaces, measure theory basics, algebraic structures |✅ +Complete + +|link:./formal-semantics.md[formal-semantics.md] |Operational, +denotational, and axiomatic semantics |✅ Complete +|=== + +==== Theorems and Proofs + +[width="100%",cols="34%,41%,25%",options="header",] +|=== +|Document |Description |Status +|link:./theorems/type-theory.md[theorems/type-theory.md] |Probability +monad, functor/applicative laws, type soundness |✅ Complete + +|link:./theorems/soundness-completeness.md[theorems/soundness-completeness.md] +|Type soundness, semantic adequacy, Hoare logic |✅ Complete + +|link:./theorems/convergence-statistics.md[theorems/convergence-statistics.md] +|LLN, CLT, Monte Carlo convergence, bootstrap |✅ Complete + +|link:./theorems/termination-analysis.md[theorems/termination-analysis.md] +|Termination classes, ranking functions, expected time |✅ Complete + +|link:./theorems/information-theory.md[theorems/information-theory.md] +|Entropy, mutual information, channel capacity |✅ Complete + +|link:./theorems/category-theory.md[theorems/category-theory.md] +|Functors, monads, Kleisli category, Lawvere theory |✅ Complete + +|link:./theorems/measure-theory.md[theorems/measure-theory.md] +|σ-algebras, Lebesgue integration, martingales |✅ Complete + +|link:./theorems/mcmc-inference.md[theorems/mcmc-inference.md] |MH, +Gibbs, HMC correctness; SMC, ABC |✅ Complete +|=== + +==== Computational Analysis + +[width="100%",cols="34%,41%,25%",options="header",] +|=== +|Document |Description |Status +|link:./complexity/computational-complexity.md[complexity/computational-complexity.md] +|Time/space complexity, BPP simulation |✅ Complete +|=== + +==== Formal Verification + +[width="100%",cols="34%,41%,25%",options="header",] +|=== +|Document |Description |Status +|link:./verification/formal-verification.md[verification/formal-verification.md] +|Specifications, Hoare logic, refinement types |✅ Complete +|=== + +==== Academic Papers + +[width="100%",cols="34%,41%,25%",options="header",] +|=== +|Document |Description |Status +|link:./papers/betlang-whitepaper.md[papers/betlang-whitepaper.md] +|Complete language specification and design |✅ Complete + +|link:./papers/ppl-comparison.md[papers/ppl-comparison.md] |Comparison +with Church, Stan, Pyro, etc. |✅ Complete +|=== + +''''' + +=== TODOs and Known Gaps + +==== High Priority (Required for Academic Rigor) + +[width="100%",cols="19%,46%,35%",options="header",] +|=== +|Gap |Description |Document +|⚠️ *Continuous semantics* |Full measure-theoretic semantics for +continuous distributions |formal-semantics.md + +|⚠️ *Automatic differentiation* |Gradient computation for probabilistic +programs |Implementation gap + +|⚠️ *Variational inference* |ELBO bounds, VI correctness +|mcmc-inference.md + +|⚠️ *Numerical stability* |Floating-point error bounds +|computational-complexity.md + +|⚠️ *PDF/CDF correctness* |Formal proofs for all distributions +|soundness-completeness.md +|=== + +==== Medium Priority (Recommended) + +[width="100%",cols="19%,46%,35%",options="header",] +|=== +|Gap |Description |Document +|🔶 *Concurrency semantics* |Parallel bet execution model +|formal-semantics.md + +|🔶 *Adaptive MCMC* |Ergodicity of adaptive algorithms +|mcmc-inference.md + +|🔶 *Convergence diagnostics* |R-hat, ESS implementation proofs +|mcmc-inference.md + +|🔶 *Dependent types* |Probability-indexed types |type-theory.md + +|🔶 *Mechanized proofs* |Coq/Lean formalization |All +|=== + +==== Lower Priority (Extensions) + +[width="100%",cols="19%,46%,35%",options="header",] +|=== +|Gap |Description |Document +|🔷 *Quantum connections* |Qutrit relationships |information-theory.md + +|🔷 *Optimal transport* |Wasserstein distances |measure-theory.md + +|🔷 *Deep learning* |Neural network integration |ppl-comparison.md + +|🔷 *Distributed computing* |Multi-node sampling +|computational-complexity.md +|=== + +''''' + +=== Key Theorems Summary + +==== Foundational Theorems + +[arabic] +. *Type Soundness* (soundness-completeness.md, Thm 1.1): Well-typed +programs don’t get stuck +. *Monad Laws* (type-theory.md, Thm 1.1-1.3): bet-pure and bet-bind +satisfy monad laws +. *Semantic Adequacy* (soundness-completeness.md, Thm 2.1): Operational +matches denotational semantics +. *Full Abstraction* (soundness-completeness.md, Thm 2.2): Observational +≡ denotational equivalence + +==== Statistical Theorems + +[arabic, start=5] +. *SLLN for Bets* (convergence-statistics.md, Thm 1.2): Sample mean +converges a.s. to expectation +. *CLT for Bets* (convergence-statistics.md, Thm 2.1): Standardized mean +is asymptotically normal +. *Maximum Entropy* (information-theory.md, Thm 1.1): Uniform ternary +achieves log₂(3) bits + +==== Inference Theorems + +[arabic, start=8] +. *MH Detailed Balance* (mcmc-inference.md, Thm 1.1): MH satisfies +detailed balance +. *Gibbs Invariance* (mcmc-inference.md, Thm 2.1): Joint is invariant +under Gibbs +. *Rejection Correctness* (mcmc-inference.md, Thm 4.1): Rejection +sampling is exact + +==== Complexity Theorems + +[arabic, start=11] +. *Bet Termination* (termination-analysis.md, Thm 2.1): Basic bet +terminates in O(1) +. *bet-until PAST* (termination-analysis.md, Thm 3.3): Almost-sure +termination with finite expectation +. *BPP Simulation* (computational-complexity.md, Thm 13.1): Betlang +simulates BPP + +''''' + +=== How to Read These Documents + +==== For Computer Scientists + +Start with: 1. formal-semantics.md (operational semantics) 2. +type-theory.md (monad structure) 3. computational-complexity.md +(efficiency) + +==== For Mathematicians + +Start with: 1. mathematical-foundations.md (probability spaces) 2. +measure-theory.md (Lebesgue integration) 3. category-theory.md +(categorical semantics) + +==== For Statisticians + +Start with: 1. convergence-statistics.md (limit theorems) 2. +mcmc-inference.md (sampling methods) 3. information-theory.md (entropy) + +==== For Practitioners + +Start with: 1. papers/betlang-whitepaper.md (overview) 2. +papers/ppl-comparison.md (comparison with alternatives) 3. +verification/formal-verification.md (correctness guarantees) + +''''' + +=== Citation + +If you use betlang or this documentation in academic work, please cite: + +[source,bibtex] +---- +@misc{betlang2024, + title={Betlang: A Ternary Probabilistic Programming Language}, + author={Betlang Development Team}, + year={2024}, + howpublished={\\url{https://github.com/betlang/betlang}} +} +---- + +''''' + +=== Contributing + +To contribute proofs or documentation: 1. Follow the existing format +(LaTeX-style math in Markdown) 2. Include clear theorem statements and +proofs 3. Reference standard results with citations 4. Mark incomplete +sections with *TODO* + +''''' + +=== Document Statistics + +* Total documents: 14 +* Total theorems proven: ~150 +* Coverage areas: 9 (probability, statistics, types, complexity, +verification, information theory, category theory, measure theory, +inference) +* Known gaps: 13 (documented above) + +''''' + +_Last updated: 2024_ diff --git a/proofs/README.md b/proofs/README.md deleted file mode 100644 index 0cede7b..0000000 --- a/proofs/README.md +++ /dev/null @@ -1,180 +0,0 @@ - -# Betlang Academic Proofs and Documentation - -## Overview - -This directory contains comprehensive academic documentation for betlang, including mathematical foundations, formal semantics, theorem proofs, formal verification specifications, and academic papers. - ---- - -## Document Index - -### Core Mathematical Foundations - -| Document | Description | Status | -|----------|-------------|--------| -| [mathematical-foundations.md](./mathematical-foundations.md) | Probability spaces, measure theory basics, algebraic structures | ✅ Complete | -| [formal-semantics.md](./formal-semantics.md) | Operational, denotational, and axiomatic semantics | ✅ Complete | - -### Theorems and Proofs - -| Document | Description | Status | -|----------|-------------|--------| -| [theorems/type-theory.md](./theorems/type-theory.md) | Probability monad, functor/applicative laws, type soundness | ✅ Complete | -| [theorems/soundness-completeness.md](./theorems/soundness-completeness.md) | Type soundness, semantic adequacy, Hoare logic | ✅ Complete | -| [theorems/convergence-statistics.md](./theorems/convergence-statistics.md) | LLN, CLT, Monte Carlo convergence, bootstrap | ✅ Complete | -| [theorems/termination-analysis.md](./theorems/termination-analysis.md) | Termination classes, ranking functions, expected time | ✅ Complete | -| [theorems/information-theory.md](./theorems/information-theory.md) | Entropy, mutual information, channel capacity | ✅ Complete | -| [theorems/category-theory.md](./theorems/category-theory.md) | Functors, monads, Kleisli category, Lawvere theory | ✅ Complete | -| [theorems/measure-theory.md](./theorems/measure-theory.md) | σ-algebras, Lebesgue integration, martingales | ✅ Complete | -| [theorems/mcmc-inference.md](./theorems/mcmc-inference.md) | MH, Gibbs, HMC correctness; SMC, ABC | ✅ Complete | - -### Computational Analysis - -| Document | Description | Status | -|----------|-------------|--------| -| [complexity/computational-complexity.md](./complexity/computational-complexity.md) | Time/space complexity, BPP simulation | ✅ Complete | - -### Formal Verification - -| Document | Description | Status | -|----------|-------------|--------| -| [verification/formal-verification.md](./verification/formal-verification.md) | Specifications, Hoare logic, refinement types | ✅ Complete | - -### Academic Papers - -| Document | Description | Status | -|----------|-------------|--------| -| [papers/betlang-whitepaper.md](./papers/betlang-whitepaper.md) | Complete language specification and design | ✅ Complete | -| [papers/ppl-comparison.md](./papers/ppl-comparison.md) | Comparison with Church, Stan, Pyro, etc. | ✅ Complete | - ---- - -## TODOs and Known Gaps - -### High Priority (Required for Academic Rigor) - -| Gap | Description | Document | -|-----|-------------|----------| -| ⚠️ **Continuous semantics** | Full measure-theoretic semantics for continuous distributions | formal-semantics.md | -| ⚠️ **Automatic differentiation** | Gradient computation for probabilistic programs | Implementation gap | -| ⚠️ **Variational inference** | ELBO bounds, VI correctness | mcmc-inference.md | -| ⚠️ **Numerical stability** | Floating-point error bounds | computational-complexity.md | -| ⚠️ **PDF/CDF correctness** | Formal proofs for all distributions | soundness-completeness.md | - -### Medium Priority (Recommended) - -| Gap | Description | Document | -|-----|-------------|----------| -| 🔶 **Concurrency semantics** | Parallel bet execution model | formal-semantics.md | -| 🔶 **Adaptive MCMC** | Ergodicity of adaptive algorithms | mcmc-inference.md | -| 🔶 **Convergence diagnostics** | R-hat, ESS implementation proofs | mcmc-inference.md | -| 🔶 **Dependent types** | Probability-indexed types | type-theory.md | -| 🔶 **Mechanized proofs** | Coq/Lean formalization | All | - -### Lower Priority (Extensions) - -| Gap | Description | Document | -|-----|-------------|----------| -| 🔷 **Quantum connections** | Qutrit relationships | information-theory.md | -| 🔷 **Optimal transport** | Wasserstein distances | measure-theory.md | -| 🔷 **Deep learning** | Neural network integration | ppl-comparison.md | -| 🔷 **Distributed computing** | Multi-node sampling | computational-complexity.md | - ---- - -## Key Theorems Summary - -### Foundational Theorems - -1. **Type Soundness** (soundness-completeness.md, Thm 1.1): Well-typed programs don't get stuck -2. **Monad Laws** (type-theory.md, Thm 1.1-1.3): bet-pure and bet-bind satisfy monad laws -3. **Semantic Adequacy** (soundness-completeness.md, Thm 2.1): Operational matches denotational semantics -4. **Full Abstraction** (soundness-completeness.md, Thm 2.2): Observational ≡ denotational equivalence - -### Statistical Theorems - -5. **SLLN for Bets** (convergence-statistics.md, Thm 1.2): Sample mean converges a.s. to expectation -6. **CLT for Bets** (convergence-statistics.md, Thm 2.1): Standardized mean is asymptotically normal -7. **Maximum Entropy** (information-theory.md, Thm 1.1): Uniform ternary achieves log₂(3) bits - -### Inference Theorems - -8. **MH Detailed Balance** (mcmc-inference.md, Thm 1.1): MH satisfies detailed balance -9. **Gibbs Invariance** (mcmc-inference.md, Thm 2.1): Joint is invariant under Gibbs -10. **Rejection Correctness** (mcmc-inference.md, Thm 4.1): Rejection sampling is exact - -### Complexity Theorems - -11. **Bet Termination** (termination-analysis.md, Thm 2.1): Basic bet terminates in O(1) -12. **bet-until PAST** (termination-analysis.md, Thm 3.3): Almost-sure termination with finite expectation -13. **BPP Simulation** (computational-complexity.md, Thm 13.1): Betlang simulates BPP - ---- - -## How to Read These Documents - -### For Computer Scientists -Start with: -1. formal-semantics.md (operational semantics) -2. type-theory.md (monad structure) -3. computational-complexity.md (efficiency) - -### For Mathematicians -Start with: -1. mathematical-foundations.md (probability spaces) -2. measure-theory.md (Lebesgue integration) -3. category-theory.md (categorical semantics) - -### For Statisticians -Start with: -1. convergence-statistics.md (limit theorems) -2. mcmc-inference.md (sampling methods) -3. information-theory.md (entropy) - -### For Practitioners -Start with: -1. papers/betlang-whitepaper.md (overview) -2. papers/ppl-comparison.md (comparison with alternatives) -3. verification/formal-verification.md (correctness guarantees) - ---- - -## Citation - -If you use betlang or this documentation in academic work, please cite: - -```bibtex -@misc{betlang2024, - title={Betlang: A Ternary Probabilistic Programming Language}, - author={Betlang Development Team}, - year={2024}, - howpublished={\\url{https://github.com/betlang/betlang}} -} -``` - ---- - -## Contributing - -To contribute proofs or documentation: -1. Follow the existing format (LaTeX-style math in Markdown) -2. Include clear theorem statements and proofs -3. Reference standard results with citations -4. Mark incomplete sections with **TODO** - ---- - -## Document Statistics - -- Total documents: 14 -- Total theorems proven: ~150 -- Coverage areas: 9 (probability, statistics, types, complexity, verification, information theory, category theory, measure theory, inference) -- Known gaps: 13 (documented above) - ---- - -*Last updated: 2024* diff --git a/proofs/complexity/computational-complexity.adoc b/proofs/complexity/computational-complexity.adoc new file mode 100644 index 0000000..d31a23b --- /dev/null +++ b/proofs/complexity/computational-complexity.adoc @@ -0,0 +1,772 @@ +== Computational Complexity Analysis of Betlang + +=== Abstract + +This document provides a rigorous computational complexity analysis of +betlang operations, establishing time and space bounds, analyzing the +relationship to probabilistic complexity classes, and proving efficiency +properties of the core language constructs and library functions. + +''''' + +=== 1. Complexity Model + +==== 1.1 Computational Model + +We analyze betlang under the *Random Access Machine (RAM) model* with +unit-cost arithmetic and the following additions: + +* *Random oracle*: O(1) cost per random number generation +* *Memory model*: Unbounded memory with O(1) access +* *Word size*: O(log n) bits for input size n + +==== 1.2 Probabilistic Complexity + +For probabilistic analysis, we consider: - *Expected time*: E[T(n)] - +*Worst-case time*: max T(n) - *High-probability bounds*: P(T(n) > t) ≤ δ + +''''' + +=== 2. Core Primitives + +==== 2.1 Basic Bet + +*Operation*: `+(bet A B C)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(1) +|Space |O(1) +|Random bits consumed |O(log 3) ≈ 1.585 bits +|=== + +_Proof_: + +.... +(bet A B C) = + 1. Generate random r ∈ [0,1) ; O(1) + 2. Compute i = floor(3r) ; O(1) + 3. Return [A,B,C][i] ; O(1) +Total: O(1) +.... + +∎ + +==== 2.2 Weighted Bet + +*Operation*: `+(bet/weighted '(A wₐ) '(B w_b) '(C w_c))+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(1) +|Space |O(1) +|Random bits |O(log W) where W = sum of weights +|=== + +_Proof_: + +.... +1. Compute W = wₐ + w_b + w_c ; O(1) +2. Generate r ∈ [0,W) ; O(1) +3. Linear scan of 3 weights ; O(1) - constant 3 items +4. Return selected value ; O(1) +.... + +∎ + +==== 2.3 Conditional Bet + +*Operation*: `+(bet/conditional pred A B C)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(T_pred) +|Space |O(S_pred) +|=== + +where T_pred, S_pred are complexities of evaluating the predicate. + +==== 2.4 Lazy Bet + +*Operation*: `+(bet/lazy thunk_A thunk_B thunk_C)+` + +[cols=",,,",options="header",] +|=== +|Metric |Best Case |Average Case |Worst Case +|Time |O(min T_i) |O((T_A+T_B+T_C)/3) |O(max T_i) +|Space |O(min S_i) |O((S_A+S_B+S_C)/3) |O(max S_i) +|=== + +_Key insight_: Lazy evaluation only computes the selected branch. + +''''' + +=== 3. Iteration Constructs + +==== 3.1 bet-chain + +*Operation*: `+(bet-chain n f init)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n · T_f) +|Space |O(S_f) (tail-recursive) +|=== + +_Proof_: + +[source,racket] +---- +(define (bet-chain n f v) + (if (= n 0) + v + (bet-chain (- n 1) f (f v)))) +---- + +The function calls f exactly n times. If implemented tail-recursively, +space is constant in n. ∎ + +==== 3.2 bet-until + +*Operation*: `+(bet-until pred thunk)+` + +Let p = P(pred(thunk()) = true). + +[cols=",,",options="header",] +|=== +|Metric |Expected |Worst Case +|Time |O(T_thunk/p) |∞ (unbounded) +|Space |O(S_thunk) |O(S_thunk) +|=== + +_Proof_: Number of iterations N ~ Geometric(p). - E[N] = 1/p - Var(N) = +(1-p)/p² - P(N > k) = (1-p)^k + +Expected time: E[N] · (T_thunk + T_pred) = O((T_thunk + T_pred)/p). ∎ + +*Theorem 3.1* (High-Probability Bound). For p > 0: + +[latexmath] +++++ +P(N > \frac{\ln(1/\delta)}{p}) \leq \delta +++++ + +_Proof_: P(N > k) = (1-p)^k ≤ e^\{-pk}. Set e^\{-pk} = δ, solve for k. ∎ + +==== 3.3 bet-repeat + +*Operation*: `+(bet-repeat n thunk)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n · T_thunk) +|Space |O(n) (storing results) +|=== + +==== 3.4 bet-parallel + +*Operation*: `+(bet-parallel n A B C)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n) +|Space |O(n) +|Random bits |O(n · log 3) +|=== + +_Note_: This is sequential simulation of n independent trials, not +actual parallelism. + +''''' + +=== 4. Higher-Order Operations + +==== 4.1 bet-map + +*Operation*: `+(bet-map f lst)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O( +|Space |O( +|=== + +==== 4.2 bet-filter + +*Operation*: `+(bet-filter pred lst)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O( +|Space |O( +|=== + +_Expected output size_: Binomial(|lst|, p) where p = P(pred succeeds). + +==== 4.3 bet-fold + +*Operation*: `+(bet-fold f init lst)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O( +|Space |O(S_f) (tail-recursive) +|=== + +==== 4.4 bet-compose + +*Operation*: `+(bet-compose f g h)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Composition Time |O(1) +|Application Time |O(max(T_f, T_g, T_h)) expected +|Space |O(max(S_f, S_g, S_h)) +|=== + +''''' + +=== 5. Statistical Functions + +==== 5.1 bet-probability + +*Operation*: `+(bet-probability n pred A B C)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n · (T_bet + T_pred)) = O(n · T_pred) +|Space |O(1) +|Statistical precision |O(1/√n) +|=== + +==== 5.2 bet-entropy + +*Operation*: `+(bet-entropy samples)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n log n) +|Space |O(k) where k = # unique values +|=== + +_Proof_: 1. Count frequencies: O(n) with hash table 2. Compute -Σ p log +p: O(k) 3. If using sorting: O(n log n) + +==== 5.3 bet-expect + +*Operation*: `+(bet-expect n f A B C)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n · (1 + T_f)) +|Space |O(1) +|=== + +''''' + +=== 6. Statistical Library Complexity + +==== 6.1 Descriptive Statistics + +[cols=",,",options="header",] +|=== +|Function |Time |Space +|mean |O(n) |O(1) +|median |O(n) or O(n log n) |O(1) or O(n) +|variance |O(n) |O(1) +|stddev |O(n) |O(1) +|percentile |O(n log n) |O(n) +|mode |O(n) |O(k) +|histogram |O(n) |O(bins) +|=== + +==== 6.2 Correlation and Covariance + +[cols=",,",options="header",] +|=== +|Function |Time |Space +|covariance |O(n) |O(1) +|correlation |O(n) |O(1) +|=== + +==== 6.3 Statistical Tests + +[cols=",,",options="header",] +|=== +|Function |Time |Space +|chi-square-test |O(n) |O(k) +|kolmogorov-smirnov |O(n log n) |O(n) +|confidence-interval |O(n) |O(1) +|=== + +''''' + +=== 7. Distribution Sampling + +==== 7.1 Discrete Distributions + +[cols=",,",options="header",] +|=== +|Distribution |Time per sample |Method +|uniform(a,b) |O(1) |Direct +|bernoulli(p) |O(1) |Direct +|binomial(n,p) |O(n) |Sum of Bernoullis +|geometric(p) |O(1) |Inverse transform +|poisson(λ) |O(λ) expected |Knuth’s algorithm +|categorical |O(k) or O(log k) |Linear or binary search +|=== + +*Theorem 7.1* (Binomial Complexity). Binomial(n,p) sampling is O(n) +using Bernoulli sum or O(1) using normal approximation for large n. + +==== 7.2 Continuous Distributions + +[cols=",,",options="header",] +|=== +|Distribution |Time per sample |Method +|normal |O(1) |Box-Muller +|exponential |O(1) |Inverse transform +|gamma(α,β) |O(1) amortized |Marsaglia-Tsang +|beta(α,β) |O(1) |Ratio of gammas +|chi-square(k) |O(1) |Sum of squared normals +|=== + +*Theorem 7.2* (Normal Sampling). Box-Muller generates 2 normal samples +using 2 uniform samples and O(1) arithmetic operations. + +_Proof_: + +.... +Z₁ = √(-2 ln U₁) cos(2π U₂) +Z₂ = √(-2 ln U₁) sin(2π U₂) +.... + +Operations: 1 log, 1 sqrt, 2 trig = O(1). ∎ + +==== 7.3 Stochastic Processes + +[cols=",",options="header",] +|=== +|Process |Time for n steps +|random-walk |O(n) +|brownian-motion |O(n) +|levy-flight |O(n) +|=== + +''''' + +=== 8. MCMC Complexity + +==== 8.1 Metropolis-Hastings + +*Operation*: `+(metropolis-hastings target proposal init n)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n · (T_target + T_proposal)) +|Space |O(n) for samples +|=== + +*Mixing time complexity*: Depends on spectral gap of the chain. + +==== 8.2 Gibbs Sampling + +*Operation*: `+(gibbs-sampler cond-x cond-y init-x init-y n)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(n · (T_cond-x + T_cond-y)) +|Space |O(n) +|=== + +==== 8.3 Hamiltonian Monte Carlo + +*Operation*: `+(hamiltonian-monte-carlo log-prob grad init n ε L)+` + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time per sample |O(L · (T_grad + d)) where d = dimension +|Space |O(d) +|=== + +_Key insight_: L leapfrog steps, each requiring gradient computation. + +''''' + +=== 9. Optimization Complexity + +==== 9.1 Simulated Annealing + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(iterations · T_objective) +|Space |O(d) +|=== + +Convergence guarantee: Logarithmic cooling schedule guarantees +convergence to global optimum but requires exponentially many +iterations. + +==== 9.2 Genetic Algorithm + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(generations · population · (T_objective + gene_length)) +|Space |O(population · gene_length) +|=== + +==== 9.3 Particle Swarm Optimization + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(iterations · particles · (T_objective + d)) +|Space |O(particles · d) +|=== + +==== 9.4 Ternary Search + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(log_\{3/2}(range/ε) · T_f) +|Space |O(1) +|=== + +_Proof_: Each iteration reduces search range by factor of 2/3. ∎ + +''''' + +=== 10. Bayesian Inference Complexity + +==== 10.1 Exact Inference + +For discrete distributions with k states: + +[cols=",",options="header",] +|=== +|Operation |Complexity +|Prior evaluation |O(1) +|Likelihood computation |O(n) for n data points +|Posterior (conjugate) |O(1) +|Evidence (marginal) |O(k) discrete, intractable continuous +|=== + +==== 10.2 Approximate Inference + +[cols=",,",options="header",] +|=== +|Method |Time |Space +|Rejection sampling |O(n · M) where M = bound ratio |O(n) +|Importance sampling |O(n) |O(n) +|ABC |O(n · simulations) |O(n) +|=== + +==== 10.3 Sequential Monte Carlo + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(T · N · (T_transition + T_observation)) +|Space |O(N · d) +|=== + +where T = timesteps, N = particles, d = dimension. + +''''' + +=== 11. Markov Chain Complexity + +==== 11.1 Transition + +[cols=",",options="header",] +|=== +|Operation |Complexity +|markov-step |O(k) for k states +|markov-simulate n steps |O(n · k) +|=== + +With sparse transitions: O(n · average_degree) + +==== 11.2 Stationary Distribution + +[cols=",",options="header",] +|=== +|Method |Complexity +|Power iteration |O(iter · k²) +|Monte Carlo estimate |O(n · k) +|=== + +==== 11.3 Viterbi Algorithm + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Time |O(T · k²) +|Space |O(T · k) +|=== + +where T = sequence length, k = states. + +''''' + +=== 12. Probabilistic Data Structures + +==== 12.1 Skip Lists + +[cols=",",options="header",] +|=== +|Operation |Expected Time +|Search |O(log n) +|Insert |O(log n) +|Delete |O(log n) +|Space |O(n) expected +|=== + +_Proof_: Expected number of levels is O(log n) with geometric +distribution. ∎ + +==== 12.2 Bloom Filters + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Insert |O(k) +|Query |O(k) +|Space |O(m) bits +|=== + +False positive rate: (1 - e^\{-kn/m})^k + +Optimal k = (m/n) ln 2 + +==== 12.3 HyperLogLog + +[cols=",",options="header",] +|=== +|Metric |Complexity +|Insert |O(1) +|Query |O(m) +|Space |O(m) registers +|=== + +Standard error: 1.04/√m + +''''' + +=== 13. Probabilistic Complexity Classes + +==== 13.1 Relationship to Complexity Classes + +Betlang expressions relate to probabilistic complexity classes: + +[cols=",",options="header",] +|=== +|Class |Betlang Correspondence +|BPP |Poly-time with bet, 2/3 majority +|RP |Poly-time, no false positives +|ZPP |Expected poly-time, always correct +|PP |Poly-time, majority correct +|=== + +==== 13.2 Expressiveness + +*Theorem 13.1* (BPP Simulation). Any BPP algorithm can be simulated in +betlang with polynomial overhead. + +_Proof sketch_: 1. BPP uses polynomial random bits 2. Each random bit +can be simulated with 2 bet operations 3. Polynomial composition of +polynomial = polynomial ∎ + +*Theorem 13.2* (Amplification). Probability of error can be reduced from +1/3 to 2^\{-n} using O(n) repetitions and majority vote. + +''''' + +=== 14. Space Complexity Analysis + +==== 14.1 Memory Usage Patterns + +[cols=",,",options="header",] +|=== +|Operation |Stack Depth |Heap Usage +|bet |O(1) |O(1) +|bet-chain n |O(1) if tail-recursive |O(1) +|bet-map |O(1) |O(n) +|bet-parallel n |O(1) |O(n) +|=== + +==== 14.2 Streaming Complexity + +For streaming algorithms on samples: + +[cols=",,",options="header",] +|=== +|Statistic |Space |Update Time +|mean |O(1) |O(1) +|variance |O(1) |O(1) +|median |O(n) or O(log n) approx |varies +|mode |O(k) |O(1) +|entropy |O(k) |O(1) +|=== + +''''' + +=== 15. Amortized Analysis + +==== 15.1 Memoized Operations + +*Operation*: `+(bet-memoize thunk)+` + +[cols=",",options="header",] +|=== +|Call |Time +|First |O(T_thunk) +|Subsequent |O(1) +|Amortized over n calls |O(T_thunk/n) → O(1) +|=== + +==== 15.2 Cached Operations + +*Operation*: `+(bet-cache ttl thunk)+` + +[cols=",",options="header",] +|=== +|Scenario |Time +|Cache hit |O(1) +|Cache miss |O(T_thunk) +|Miss rate r |O(1 + r · T_thunk) +|=== + +''''' + +=== 16. Lower Bounds + +==== 16.1 Random Bit Lower Bound + +*Theorem 16.1* (Entropy Lower Bound). Sampling from a distribution with +entropy H requires at least H random bits on average. + +_Proof_: Shannon’s source coding theorem. ∎ + +*Corollary 16.1* Uniform ternary bet requires ≥ log₂(3) ≈ 1.585 random +bits. + +==== 16.2 Estimation Lower Bounds + +*Theorem 16.2* (Sample Complexity Lower Bound). Estimating a probability +p to within ε requires Ω(1/ε²) samples. + +_Proof_: Cramér-Rao lower bound. ∎ + +==== 16.3 Simulation Lower Bound + +*Theorem 16.3* (Coin Flipping Lower Bound). Generating a fair ternary +outcome from fair coins requires ≥ 1.585 coin flips on average. + +''''' + +=== 17. Parallel Complexity + +==== 17.1 Inherent Parallelism + +[cols=",,",options="header",] +|=== +|Operation |Parallel Time |Span +|bet-parallel n |O(1) |O(1) +|bet-map |O(1) |O(1) +|bet-fold (associative f) |O(log n) |O(log n) +|=== + +==== 17.2 Work-Span Analysis + +[cols=",,,",options="header",] +|=== +|Operation |Work |Span |Parallelism +|bet-parallel |O(n) |O(1) |O(n) +|Monte Carlo n trials |O(n) |O(1) |O(n) +|MCMC |O(n) |O(n) |O(1) - inherently sequential +|=== + +''''' + +=== 18. Complexity Summary Tables + +==== Core Operations + +[cols=",,,",options="header",] +|=== +|Operation |Time |Space |Random Bits +|bet |O(1) |O(1) |1.585 +|bet/weighted |O(1) |O(1) |varies +|bet/lazy |O(T_i) |O(S_i) |1.585 +|bet-chain n |O(n·T_f) |O(1) |n·1.585 +|bet-until |O(T/p) exp |O(S) |1.585/p exp +|=== + +==== Statistical Functions + +[cols=",,",options="header",] +|=== +|Function |Time |Space +|bet-probability n |O(n) |O(1) +|bet-entropy |O(n log n) |O(k) +|bet-expect n |O(n) |O(1) +|bootstrap n B |O(n·B) |O(B) +|=== + +==== Inference Methods + +[cols=",,",options="header",] +|=== +|Method |Time per sample |Total for n samples +|Rejection |O(M·T) |O(n·M·T) +|MH |O(T_target) |O(n·T_target) +|Gibbs |O(d·T_cond) |O(n·d·T_cond) +|HMC |O(L·T_grad) |O(n·L·T_grad) +|=== + +''''' + +=== 19. TODOs and Open Problems + +*TODO*: The following require further complexity analysis: + +[arabic] +. *Tight bounds for MCMC mixing*: Precise mixing time for MH on specific +targets +. *Adaptive algorithm analysis*: Amortized complexity of adaptive MCMC +. *Cache-oblivious bounds*: Memory hierarchy effects on large +simulations +. *Communication complexity*: For distributed betlang implementations +. *Quantum speedups*: Potential for quantum Monte Carlo acceleration + +''''' + +=== References + +[arabic] +. Cormen, T.H., et al. (2009). _Introduction to Algorithms_, 3rd ed. +. Motwani, R. & Raghavan, P. (1995). _Randomized Algorithms_ +. Mitzenmacher, M. & Upfal, E. (2017). _Probability and Computing_ +. Arora, S. & Barak, B. (2009). _Computational Complexity: A Modern +Approach_ +. Robert, C.P. & Casella, G. (2004). _Monte Carlo Statistical Methods_ diff --git a/proofs/complexity/computational-complexity.md b/proofs/complexity/computational-complexity.md deleted file mode 100644 index 8fde2a8..0000000 --- a/proofs/complexity/computational-complexity.md +++ /dev/null @@ -1,656 +0,0 @@ - -# Computational Complexity Analysis of Betlang - -## Abstract - -This document provides a rigorous computational complexity analysis of betlang operations, establishing time and space bounds, analyzing the relationship to probabilistic complexity classes, and proving efficiency properties of the core language constructs and library functions. - ---- - -## 1. Complexity Model - -### 1.1 Computational Model - -We analyze betlang under the **Random Access Machine (RAM) model** with unit-cost arithmetic and the following additions: - -- **Random oracle**: O(1) cost per random number generation -- **Memory model**: Unbounded memory with O(1) access -- **Word size**: O(log n) bits for input size n - -### 1.2 Probabilistic Complexity - -For probabilistic analysis, we consider: -- **Expected time**: E[T(n)] -- **Worst-case time**: max T(n) -- **High-probability bounds**: P(T(n) > t) ≤ δ - ---- - -## 2. Core Primitives - -### 2.1 Basic Bet - -**Operation**: `(bet A B C)` - -| Metric | Complexity | -|--------|------------| -| Time | O(1) | -| Space | O(1) | -| Random bits consumed | O(log 3) ≈ 1.585 bits | - -*Proof*: -``` -(bet A B C) = - 1. Generate random r ∈ [0,1) ; O(1) - 2. Compute i = floor(3r) ; O(1) - 3. Return [A,B,C][i] ; O(1) -Total: O(1) -``` -∎ - -### 2.2 Weighted Bet - -**Operation**: `(bet/weighted '(A wₐ) '(B w_b) '(C w_c))` - -| Metric | Complexity | -|--------|------------| -| Time | O(1) | -| Space | O(1) | -| Random bits | O(log W) where W = sum of weights | - -*Proof*: -``` -1. Compute W = wₐ + w_b + w_c ; O(1) -2. Generate r ∈ [0,W) ; O(1) -3. Linear scan of 3 weights ; O(1) - constant 3 items -4. Return selected value ; O(1) -``` -∎ - -### 2.3 Conditional Bet - -**Operation**: `(bet/conditional pred A B C)` - -| Metric | Complexity | -|--------|------------| -| Time | O(T_pred) | -| Space | O(S_pred) | - -where T_pred, S_pred are complexities of evaluating the predicate. - -### 2.4 Lazy Bet - -**Operation**: `(bet/lazy thunk_A thunk_B thunk_C)` - -| Metric | Best Case | Average Case | Worst Case | -|--------|-----------|--------------|------------| -| Time | O(min T_i) | O((T_A+T_B+T_C)/3) | O(max T_i) | -| Space | O(min S_i) | O((S_A+S_B+S_C)/3) | O(max S_i) | - -*Key insight*: Lazy evaluation only computes the selected branch. - ---- - -## 3. Iteration Constructs - -### 3.1 bet-chain - -**Operation**: `(bet-chain n f init)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n · T_f) | -| Space | O(S_f) (tail-recursive) | - -*Proof*: -```racket -(define (bet-chain n f v) - (if (= n 0) - v - (bet-chain (- n 1) f (f v)))) -``` - -The function calls f exactly n times. If implemented tail-recursively, space is constant in n. ∎ - -### 3.2 bet-until - -**Operation**: `(bet-until pred thunk)` - -Let p = P(pred(thunk()) = true). - -| Metric | Expected | Worst Case | -|--------|----------|------------| -| Time | O(T_thunk/p) | ∞ (unbounded) | -| Space | O(S_thunk) | O(S_thunk) | - -*Proof*: -Number of iterations N ~ Geometric(p). -- E[N] = 1/p -- Var(N) = (1-p)/p² -- P(N > k) = (1-p)^k - -Expected time: E[N] · (T_thunk + T_pred) = O((T_thunk + T_pred)/p). ∎ - -**Theorem 3.1** (High-Probability Bound). For p > 0: -$$P(N > \frac{\ln(1/\delta)}{p}) \leq \delta$$ - -*Proof*: P(N > k) = (1-p)^k ≤ e^{-pk}. Set e^{-pk} = δ, solve for k. ∎ - -### 3.3 bet-repeat - -**Operation**: `(bet-repeat n thunk)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n · T_thunk) | -| Space | O(n) (storing results) | - -### 3.4 bet-parallel - -**Operation**: `(bet-parallel n A B C)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n) | -| Space | O(n) | -| Random bits | O(n · log 3) | - -*Note*: This is sequential simulation of n independent trials, not actual parallelism. - ---- - -## 4. Higher-Order Operations - -### 4.1 bet-map - -**Operation**: `(bet-map f lst)` - -| Metric | Complexity | -|--------|------------| -| Time | O(|lst| · T_f) | -| Space | O(|lst| + S_f) | - -### 4.2 bet-filter - -**Operation**: `(bet-filter pred lst)` - -| Metric | Complexity | -|--------|------------| -| Time | O(|lst| · (T_pred + T_thunk)) | -| Space | O(|lst|) worst case | - -*Expected output size*: Binomial(|lst|, p) where p = P(pred succeeds). - -### 4.3 bet-fold - -**Operation**: `(bet-fold f init lst)` - -| Metric | Complexity | -|--------|------------| -| Time | O(|lst| · T_f) | -| Space | O(S_f) (tail-recursive) | - -### 4.4 bet-compose - -**Operation**: `(bet-compose f g h)` - -| Metric | Complexity | -|--------|------------| -| Composition Time | O(1) | -| Application Time | O(max(T_f, T_g, T_h)) expected | -| Space | O(max(S_f, S_g, S_h)) | - ---- - -## 5. Statistical Functions - -### 5.1 bet-probability - -**Operation**: `(bet-probability n pred A B C)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n · (T_bet + T_pred)) = O(n · T_pred) | -| Space | O(1) | -| Statistical precision | O(1/√n) | - -### 5.2 bet-entropy - -**Operation**: `(bet-entropy samples)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n log n) | -| Space | O(k) where k = # unique values | - -*Proof*: -1. Count frequencies: O(n) with hash table -2. Compute -Σ p log p: O(k) -3. If using sorting: O(n log n) - -### 5.3 bet-expect - -**Operation**: `(bet-expect n f A B C)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n · (1 + T_f)) | -| Space | O(1) | - ---- - -## 6. Statistical Library Complexity - -### 6.1 Descriptive Statistics - -| Function | Time | Space | -|----------|------|-------| -| mean | O(n) | O(1) | -| median | O(n) or O(n log n) | O(1) or O(n) | -| variance | O(n) | O(1) | -| stddev | O(n) | O(1) | -| percentile | O(n log n) | O(n) | -| mode | O(n) | O(k) | -| histogram | O(n) | O(bins) | - -### 6.2 Correlation and Covariance - -| Function | Time | Space | -|----------|------|-------| -| covariance | O(n) | O(1) | -| correlation | O(n) | O(1) | - -### 6.3 Statistical Tests - -| Function | Time | Space | -|----------|------|-------| -| chi-square-test | O(n) | O(k) | -| kolmogorov-smirnov | O(n log n) | O(n) | -| confidence-interval | O(n) | O(1) | - ---- - -## 7. Distribution Sampling - -### 7.1 Discrete Distributions - -| Distribution | Time per sample | Method | -|--------------|-----------------|--------| -| uniform(a,b) | O(1) | Direct | -| bernoulli(p) | O(1) | Direct | -| binomial(n,p) | O(n) | Sum of Bernoullis | -| geometric(p) | O(1) | Inverse transform | -| poisson(λ) | O(λ) expected | Knuth's algorithm | -| categorical | O(k) or O(log k) | Linear or binary search | - -**Theorem 7.1** (Binomial Complexity). Binomial(n,p) sampling is O(n) using Bernoulli sum or O(1) using normal approximation for large n. - -### 7.2 Continuous Distributions - -| Distribution | Time per sample | Method | -|--------------|-----------------|--------| -| normal | O(1) | Box-Muller | -| exponential | O(1) | Inverse transform | -| gamma(α,β) | O(1) amortized | Marsaglia-Tsang | -| beta(α,β) | O(1) | Ratio of gammas | -| chi-square(k) | O(1) | Sum of squared normals | - -**Theorem 7.2** (Normal Sampling). Box-Muller generates 2 normal samples using 2 uniform samples and O(1) arithmetic operations. - -*Proof*: -``` -Z₁ = √(-2 ln U₁) cos(2π U₂) -Z₂ = √(-2 ln U₁) sin(2π U₂) -``` -Operations: 1 log, 1 sqrt, 2 trig = O(1). ∎ - -### 7.3 Stochastic Processes - -| Process | Time for n steps | -|---------|------------------| -| random-walk | O(n) | -| brownian-motion | O(n) | -| levy-flight | O(n) | - ---- - -## 8. MCMC Complexity - -### 8.1 Metropolis-Hastings - -**Operation**: `(metropolis-hastings target proposal init n)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n · (T_target + T_proposal)) | -| Space | O(n) for samples | - -**Mixing time complexity**: Depends on spectral gap of the chain. - -### 8.2 Gibbs Sampling - -**Operation**: `(gibbs-sampler cond-x cond-y init-x init-y n)` - -| Metric | Complexity | -|--------|------------| -| Time | O(n · (T_cond-x + T_cond-y)) | -| Space | O(n) | - -### 8.3 Hamiltonian Monte Carlo - -**Operation**: `(hamiltonian-monte-carlo log-prob grad init n ε L)` - -| Metric | Complexity | -|--------|------------| -| Time per sample | O(L · (T_grad + d)) where d = dimension | -| Space | O(d) | - -*Key insight*: L leapfrog steps, each requiring gradient computation. - ---- - -## 9. Optimization Complexity - -### 9.1 Simulated Annealing - -| Metric | Complexity | -|--------|------------| -| Time | O(iterations · T_objective) | -| Space | O(d) | - -Convergence guarantee: Logarithmic cooling schedule guarantees convergence to global optimum but requires exponentially many iterations. - -### 9.2 Genetic Algorithm - -| Metric | Complexity | -|--------|------------| -| Time | O(generations · population · (T_objective + gene_length)) | -| Space | O(population · gene_length) | - -### 9.3 Particle Swarm Optimization - -| Metric | Complexity | -|--------|------------| -| Time | O(iterations · particles · (T_objective + d)) | -| Space | O(particles · d) | - -### 9.4 Ternary Search - -| Metric | Complexity | -|--------|------------| -| Time | O(log_{3/2}(range/ε) · T_f) | -| Space | O(1) | - -*Proof*: Each iteration reduces search range by factor of 2/3. ∎ - ---- - -## 10. Bayesian Inference Complexity - -### 10.1 Exact Inference - -For discrete distributions with k states: - -| Operation | Complexity | -|-----------|------------| -| Prior evaluation | O(1) | -| Likelihood computation | O(n) for n data points | -| Posterior (conjugate) | O(1) | -| Evidence (marginal) | O(k) discrete, intractable continuous | - -### 10.2 Approximate Inference - -| Method | Time | Space | -|--------|------|-------| -| Rejection sampling | O(n · M) where M = bound ratio | O(n) | -| Importance sampling | O(n) | O(n) | -| ABC | O(n · simulations) | O(n) | - -### 10.3 Sequential Monte Carlo - -| Metric | Complexity | -|--------|------------| -| Time | O(T · N · (T_transition + T_observation)) | -| Space | O(N · d) | - -where T = timesteps, N = particles, d = dimension. - ---- - -## 11. Markov Chain Complexity - -### 11.1 Transition - -| Operation | Complexity | -|-----------|------------| -| markov-step | O(k) for k states | -| markov-simulate n steps | O(n · k) | - -With sparse transitions: O(n · average_degree) - -### 11.2 Stationary Distribution - -| Method | Complexity | -|--------|------------| -| Power iteration | O(iter · k²) | -| Monte Carlo estimate | O(n · k) | - -### 11.3 Viterbi Algorithm - -| Metric | Complexity | -|--------|------------| -| Time | O(T · k²) | -| Space | O(T · k) | - -where T = sequence length, k = states. - ---- - -## 12. Probabilistic Data Structures - -### 12.1 Skip Lists - -| Operation | Expected Time | -|-----------|---------------| -| Search | O(log n) | -| Insert | O(log n) | -| Delete | O(log n) | -| Space | O(n) expected | - -*Proof*: Expected number of levels is O(log n) with geometric distribution. ∎ - -### 12.2 Bloom Filters - -| Metric | Complexity | -|--------|------------| -| Insert | O(k) | -| Query | O(k) | -| Space | O(m) bits | - -False positive rate: (1 - e^{-kn/m})^k - -Optimal k = (m/n) ln 2 - -### 12.3 HyperLogLog - -| Metric | Complexity | -|--------|------------| -| Insert | O(1) | -| Query | O(m) | -| Space | O(m) registers | - -Standard error: 1.04/√m - ---- - -## 13. Probabilistic Complexity Classes - -### 13.1 Relationship to Complexity Classes - -Betlang expressions relate to probabilistic complexity classes: - -| Class | Betlang Correspondence | -|-------|----------------------| -| BPP | Poly-time with bet, 2/3 majority | -| RP | Poly-time, no false positives | -| ZPP | Expected poly-time, always correct | -| PP | Poly-time, majority correct | - -### 13.2 Expressiveness - -**Theorem 13.1** (BPP Simulation). Any BPP algorithm can be simulated in betlang with polynomial overhead. - -*Proof sketch*: -1. BPP uses polynomial random bits -2. Each random bit can be simulated with 2 bet operations -3. Polynomial composition of polynomial = polynomial ∎ - -**Theorem 13.2** (Amplification). Probability of error can be reduced from 1/3 to 2^{-n} using O(n) repetitions and majority vote. - ---- - -## 14. Space Complexity Analysis - -### 14.1 Memory Usage Patterns - -| Operation | Stack Depth | Heap Usage | -|-----------|-------------|------------| -| bet | O(1) | O(1) | -| bet-chain n | O(1) if tail-recursive | O(1) | -| bet-map | O(1) | O(n) | -| bet-parallel n | O(1) | O(n) | - -### 14.2 Streaming Complexity - -For streaming algorithms on samples: - -| Statistic | Space | Update Time | -|-----------|-------|-------------| -| mean | O(1) | O(1) | -| variance | O(1) | O(1) | -| median | O(n) or O(log n) approx | varies | -| mode | O(k) | O(1) | -| entropy | O(k) | O(1) | - ---- - -## 15. Amortized Analysis - -### 15.1 Memoized Operations - -**Operation**: `(bet-memoize thunk)` - -| Call | Time | -|------|------| -| First | O(T_thunk) | -| Subsequent | O(1) | -| Amortized over n calls | O(T_thunk/n) → O(1) | - -### 15.2 Cached Operations - -**Operation**: `(bet-cache ttl thunk)` - -| Scenario | Time | -|----------|------| -| Cache hit | O(1) | -| Cache miss | O(T_thunk) | -| Miss rate r | O(1 + r · T_thunk) | - ---- - -## 16. Lower Bounds - -### 16.1 Random Bit Lower Bound - -**Theorem 16.1** (Entropy Lower Bound). Sampling from a distribution with entropy H requires at least H random bits on average. - -*Proof*: Shannon's source coding theorem. ∎ - -**Corollary 16.1** Uniform ternary bet requires ≥ log₂(3) ≈ 1.585 random bits. - -### 16.2 Estimation Lower Bounds - -**Theorem 16.2** (Sample Complexity Lower Bound). Estimating a probability p to within ε requires Ω(1/ε²) samples. - -*Proof*: Cramér-Rao lower bound. ∎ - -### 16.3 Simulation Lower Bound - -**Theorem 16.3** (Coin Flipping Lower Bound). Generating a fair ternary outcome from fair coins requires ≥ 1.585 coin flips on average. - ---- - -## 17. Parallel Complexity - -### 17.1 Inherent Parallelism - -| Operation | Parallel Time | Span | -|-----------|---------------|------| -| bet-parallel n | O(1) | O(1) | -| bet-map | O(1) | O(1) | -| bet-fold (associative f) | O(log n) | O(log n) | - -### 17.2 Work-Span Analysis - -| Operation | Work | Span | Parallelism | -|-----------|------|------|-------------| -| bet-parallel | O(n) | O(1) | O(n) | -| Monte Carlo n trials | O(n) | O(1) | O(n) | -| MCMC | O(n) | O(n) | O(1) - inherently sequential | - ---- - -## 18. Complexity Summary Tables - -### Core Operations - -| Operation | Time | Space | Random Bits | -|-----------|------|-------|-------------| -| bet | O(1) | O(1) | 1.585 | -| bet/weighted | O(1) | O(1) | varies | -| bet/lazy | O(T_i) | O(S_i) | 1.585 | -| bet-chain n | O(n·T_f) | O(1) | n·1.585 | -| bet-until | O(T/p) exp | O(S) | 1.585/p exp | - -### Statistical Functions - -| Function | Time | Space | -|----------|------|-------| -| bet-probability n | O(n) | O(1) | -| bet-entropy | O(n log n) | O(k) | -| bet-expect n | O(n) | O(1) | -| bootstrap n B | O(n·B) | O(B) | - -### Inference Methods - -| Method | Time per sample | Total for n samples | -|--------|-----------------|---------------------| -| Rejection | O(M·T) | O(n·M·T) | -| MH | O(T_target) | O(n·T_target) | -| Gibbs | O(d·T_cond) | O(n·d·T_cond) | -| HMC | O(L·T_grad) | O(n·L·T_grad) | - ---- - -## 19. TODOs and Open Problems - -**TODO**: The following require further complexity analysis: - -1. **Tight bounds for MCMC mixing**: Precise mixing time for MH on specific targets -2. **Adaptive algorithm analysis**: Amortized complexity of adaptive MCMC -3. **Cache-oblivious bounds**: Memory hierarchy effects on large simulations -4. **Communication complexity**: For distributed betlang implementations -5. **Quantum speedups**: Potential for quantum Monte Carlo acceleration - ---- - -## References - -1. Cormen, T.H., et al. (2009). *Introduction to Algorithms*, 3rd ed. -2. Motwani, R. & Raghavan, P. (1995). *Randomized Algorithms* -3. Mitzenmacher, M. & Upfal, E. (2017). *Probability and Computing* -4. Arora, S. & Barak, B. (2009). *Computational Complexity: A Modern Approach* -5. Robert, C.P. & Casella, G. (2004). *Monte Carlo Statistical Methods* diff --git a/proofs/formal-semantics.adoc b/proofs/formal-semantics.adoc new file mode 100644 index 0000000..69b5011 --- /dev/null +++ b/proofs/formal-semantics.adoc @@ -0,0 +1,607 @@ +== Formal Semantics of Betlang + +=== Abstract + +This document provides a complete formal semantics for betlang, +including operational semantics (small-step and big-step), denotational +semantics, and axiomatic semantics. We establish the semantic +foundations required for formal reasoning about betlang programs. + +''''' + +=== 1. Syntax + +==== 1.1 Abstract Grammar + +.... +e ::= v ; values + | (bet e e e) ; ternary choice + | (bet/weighted p p p) ; weighted choice + | (bet/conditional e e e e) ; conditional choice + | (bet/lazy e e e) ; lazy choice + | (bet-chain e e e) ; iteration + | (bet-compose e e e) ; function composition + | (e e) ; application + | (λ (x) e) ; abstraction + | (if e e e) ; conditional + | x ; variable + +v ::= n ; numbers + | b ; booleans + | 'sym ; symbols + | (λ (x) e) ; closures + | () ; unit + +p ::= '(e w) ; weighted pair (value, weight) + +w ::= n where n ≥ 0 ; non-negative weight +.... + +==== 1.2 Syntactic Sugar + +.... +(bet A B C) ≡ (bet/weighted '(A 1) '(B 1) '(C 1)) +.... + +''''' + +=== 2. Operational Semantics + +==== 2.1 Configuration + +A configuration is a triple ⟨e, σ, ω⟩ where: - e: expression being +evaluated - σ: environment mapping variables to values - ω: random +stream (infinite sequence of values in [0,1)) + +==== 2.2 Small-Step Semantics (→) + +===== Values + +[latexmath] +++++ +\frac{}{⟨v, σ, ω⟩ → ⟨v, σ, ω⟩} \text{ (V-VALUE)} +++++ + +===== Uniform Bet + +[latexmath] +++++ +\frac{r = \text{head}(ω) \quad ω' = \text{tail}(ω) \quad i = \lfloor 3r \rfloor}{⟨\text{(bet } v_0 \ v_1 \ v_2\text{)}, σ, ω⟩ → ⟨v_i, σ, ω'⟩} \text{ (E-BET)} +++++ + +===== Weighted Bet + +[latexmath] +++++ +\frac{r = \text{head}(ω) \quad ω' = \text{tail}(ω) \quad W = w_0 + w_1 + w_2 \quad i = \text{select}(r \cdot W, [w_0, w_1, w_2])}{⟨\text{(bet/weighted } '(v_0 \ w_0) \ '(v_1 \ w_1) \ '(v_2 \ w_2)\text{)}, σ, ω⟩ → ⟨v_i, σ, ω'⟩} \text{ (E-WBET)} +++++ + +where select(x, [w₀, w₁, w₂]) returns: - 0 if x < w₀ - 1 if w₀ ≤ x < w₀ ++ w₁ - 2 otherwise + +===== Conditional Bet + +[latexmath] +++++ +\frac{⟨e_{pred}, σ, ω⟩ →^* ⟨\text{true}, σ, ω'⟩}{⟨\text{(bet/conditional } e_{pred} \ e_a \ e_b \ e_c\text{)}, σ, ω⟩ → ⟨e_a, σ, ω'⟩} \text{ (E-COND-T)} +++++ + +[latexmath] +++++ +\frac{⟨e_{pred}, σ, ω⟩ →^* ⟨\text{false}, σ, ω'⟩}{⟨\text{(bet/conditional } e_{pred} \ e_a \ e_b \ e_c\text{)}, σ, ω⟩ → ⟨\text{(bet } e_b \ e_c \ e_a\text{)}, σ, ω'⟩} \text{ (E-COND-F)} +++++ + +===== Lazy Bet + +[latexmath] +++++ +\frac{r = \text{head}(ω) \quad ω' = \text{tail}(ω) \quad i = \lfloor 3r \rfloor \quad ⟨(thunk_i), σ, ω'⟩ →^* ⟨v, σ, ω''⟩}{⟨\text{(bet/lazy } thunk_0 \ thunk_1 \ thunk_2\text{)}, σ, ω⟩ → ⟨v, σ, ω''⟩} \text{ (E-LAZY)} +++++ + +===== Function Application + +[latexmath] +++++ +\frac{⟨e_1, σ, ω⟩ →^* ⟨(λ (x) e), σ, ω'⟩ \quad ⟨e_2, σ, ω'⟩ →^* ⟨v, σ, ω''⟩ \quad ⟨e, σ[x ↦ v], ω''⟩ →^* ⟨v', σ, ω'''⟩}{⟨(e_1 \ e_2), σ, ω⟩ → ⟨v', σ, ω'''⟩} \text{ (E-APP)} +++++ + +===== Bet Chain + +[latexmath] +++++ +\frac{n = 0}{⟨\text{(bet-chain } 0 \ f \ v\text{)}, σ, ω⟩ → ⟨v, σ, ω⟩} \text{ (E-CHAIN-0)} +++++ + +[latexmath] +++++ +\frac{n > 0 \quad ⟨(f \ v), σ, ω⟩ →^* ⟨v', σ, ω'⟩ \quad ⟨\text{(bet-chain } (n-1) \ f \ v'\text{)}, σ, ω'⟩ →^* ⟨v'', σ, ω''⟩}{⟨\text{(bet-chain } n \ f \ v\text{)}, σ, ω⟩ → ⟨v'', σ, ω''⟩} \text{ (E-CHAIN-N)} +++++ + +==== 2.3 Big-Step Semantics (⇓) + +===== Values + +[latexmath] +++++ +\frac{}{⟨v, σ, ω⟩ ⇓ ⟨v, ω⟩} \text{ (B-VALUE)} +++++ + +===== Uniform Bet + +[latexmath] +++++ +\frac{⟨e_0, σ, ω⟩ ⇓ ⟨v_0, ω_0⟩ \quad ⟨e_1, σ, ω_0⟩ ⇓ ⟨v_1, ω_1⟩ \quad ⟨e_2, σ, ω_1⟩ ⇓ ⟨v_2, ω_2⟩ \quad r = \text{head}(ω_2) \quad i = \lfloor 3r \rfloor}{⟨\text{(bet } e_0 \ e_1 \ e_2\text{)}, σ, ω⟩ ⇓ ⟨v_i, \text{tail}(ω_2)⟩} \text{ (B-BET)} +++++ + +*Note*: This evaluates all three branches before selecting (eager +evaluation). For lazy semantics, see B-LAZY below. + +===== Lazy Bet (Big-Step) + +[latexmath] +++++ +\frac{r = \text{head}(ω) \quad i = \lfloor 3r \rfloor \quad ⟨(thunk_i), σ, \text{tail}(ω)⟩ ⇓ ⟨v, ω'⟩}{⟨\text{(bet/lazy } thunk_0 \ thunk_1 \ thunk_2\text{)}, σ, ω⟩ ⇓ ⟨v, ω'⟩} \text{ (B-LAZY)} +++++ + +''''' + +=== 3. Denotational Semantics + +==== 3.1 Semantic Domains + +*Domain D* (semantic values): + +.... +D = ℤ + ℝ + Bool + Symbol + (D → Dist(D)) + Unit + Error +.... + +*Distribution Domain* Dist(D): + +.... +Dist(D) = D → [0, 1] such that Σ_d P(d) = 1 +.... + +==== 3.2 Semantic Function + +[latexmath] +++++ +⟦ \cdot ⟧ : \text{Expr} → \text{Env} → \text{Dist}(D) +++++ + +===== Literals + +[latexmath] +++++ +⟦ n ⟧ρ = δ_n +++++ +(Dirac distribution at n) + +[latexmath] +++++ +⟦ \text{true} ⟧ρ = δ_{\text{true}} +++++ + +[latexmath] +++++ +⟦ 'sym ⟧ρ = δ_{sym} +++++ + +===== Variables + +[latexmath] +++++ +⟦ x ⟧ρ = δ_{ρ(x)} +++++ + +===== Uniform Bet + +[latexmath] +++++ +⟦ \text{(bet } e_0 \ e_1 \ e_2\text{)} ⟧ρ = \frac{1}{3}⟦e_0⟧ρ + \frac{1}{3}⟦e_1⟧ρ + \frac{1}{3}⟦e_2⟧ρ +++++ + +where + denotes mixture of distributions. + +===== Weighted Bet + +[latexmath] +++++ +⟦ \text{(bet/weighted } '(e_0 \ w_0) \ '(e_1 \ w_1) \ '(e_2 \ w_2)\text{)} ⟧ρ = \sum_{i=0}^{2} \frac{w_i}{W} ⟦e_i⟧ρ +++++ + +where W = w₀ + w₁ + w₂. + +===== Conditional + +[latexmath] +++++ +⟦ \text{(if } e_c \ e_t \ e_f\text{)} ⟧ρ = ⟦e_c⟧ρ \rhd λb. \begin{cases} ⟦e_t⟧ρ & \text{if } b = \text{true} \\ ⟦e_f⟧ρ & \text{if } b = \text{false} \end{cases} +++++ + +where ▷ is monadic bind for distributions. + +===== Abstraction + +[latexmath] +++++ +⟦ (λ (x) e) ⟧ρ = δ_{λv. ⟦e⟧ρ[x ↦ v]} +++++ + +===== Application + +[latexmath] +++++ +⟦ (e_1 \ e_2) ⟧ρ = ⟦e_1⟧ρ \rhd λf. ⟦e_2⟧ρ \rhd λv. f(v) +++++ + +==== 3.3 Distribution Operations + +*Mixture*: For distributions μ, ν and weight p ∈ [0,1]: + +[latexmath] +++++ +(p \cdot μ + (1-p) \cdot ν)(x) = p \cdot μ(x) + (1-p) \cdot ν(x) +++++ + +*Bind* (▷): For distribution μ and function f: D → Dist(D): + +[latexmath] +++++ +(μ \rhd f)(y) = \sum_{x} μ(x) \cdot f(x)(y) +++++ + +*Expected Value*: + +[latexmath] +++++ +\mathbb{E}[⟦e⟧ρ] = \sum_{v} v \cdot ⟦e⟧ρ(v) +++++ + +''''' + +=== 4. Axiomatic Semantics + +==== 4.1 Probabilistic Hoare Logic + +We extend Hoare triples to probabilistic settings: + +[latexmath] +++++ +\{P\} \ e \ \{Q\}_p +++++ + +meaning: if precondition P holds, then after evaluating e, postcondition +Q holds with probability at least p. + +==== 4.2 Axioms + +===== Bet Axiom + +[latexmath] +++++ +\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{Q\}_{1} \quad \text{if} \ P \Rightarrow Q[A/x] \land Q[B/x] \land Q[C/x] +++++ + +"`If Q holds for all possible outcomes, it holds with certainty.`" + +===== Probabilistic Bet Axiom + +[latexmath] +++++ +\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{x = A\}_{1/3} +++++ + +[latexmath] +++++ +\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{x = B\}_{1/3} +++++ + +[latexmath] +++++ +\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{x = C\}_{1/3} +++++ + +===== Weighted Bet Axiom + +[latexmath] +++++ +\{P\} \ \text{(bet/weighted } '(A \ w_A) \ '(B \ w_B) \ '(C \ w_C)\text{)} \ \{x = A\}_{w_A/W} +++++ + +where W = wₐ + w_B + w_C. + +===== Conditional Axiom + +[latexmath] +++++ +\frac{\{P ∧ b\} \ e_1 \ \{Q\}_p \quad \{P ∧ ¬b\} \ e_2 \ \{Q\}_p}{\{P\} \ \text{(if } b \ e_1 \ e_2\text{)} \ \{Q\}_p} +++++ + +===== Sequence Rule + +[latexmath] +++++ +\frac{\{P\} \ e_1 \ \{R\}_{p_1} \quad \{R\} \ e_2 \ \{Q\}_{p_2}}{\{P\} \ e_1; e_2 \ \{Q\}_{p_1 \cdot p_2}} +++++ + +===== Consequence Rule + +[latexmath] +++++ +\frac{P' \Rightarrow P \quad \{P\} \ e \ \{Q\}_p \quad Q \Rightarrow Q' \quad p \leq p'}{\{P'\} \ e \ \{Q'\}_{p'}} +++++ + +==== 4.3 Expectation Calculus + +For numeric expressions, we can reason about expected values: + +[latexmath] +++++ +\text{wp}[\text{(bet } A \ B \ C\text{)}](f) = \frac{1}{3}(f(A) + f(B) + f(C)) +++++ + +where wp is the weakest precondition transformer and f is a +post-expectation. + +*Theorem 4.1* (Expectation Soundness). If ⟦e⟧ρ = μ, then: + +[latexmath] +++++ +\mathbb{E}_{x \sim μ}[f(x)] = \text{wp}[e](f) +++++ + +''''' + +=== 5. Semantic Properties + +==== 5.1 Determinism and Confluence + +*Theorem 5.1* (Probabilistic Confluence). Betlang is confluent up to +distribution: + +If ⟨e, σ, ω⟩ →* ⟨v₁, σ, ω₁⟩ and ⟨e, σ, ω⟩ →* ⟨v₂, σ, ω₂⟩ with the same +random stream ω, then v₁ = v₂. + +_Proof_: The operational semantics is deterministic given a fixed random +stream. ∎ + +*Corollary 5.1* (Distribution Determinism). The distribution ⟦e⟧ρ is +uniquely determined by e and ρ. + +==== 5.2 Adequacy + +*Theorem 5.2* (Computational Adequacy). For closed expressions e: + +[latexmath] +++++ +P(⟨e, ∅, ω⟩ ⇓ ⟨v, ω'⟩) = ⟦e⟧∅(v) +++++ + +where the left side is probability over random streams ω. + +_Proof sketch_: By structural induction on e, showing operational and +denotational semantics agree. ∎ + +==== 5.3 Compositionality + +*Theorem 5.3* (Compositionality). The denotational semantics is +compositional: + +[latexmath] +++++ +⟦e⟧ρ \text{ depends only on } ⟦e_i⟧ρ \text{ for immediate subexpressions } e_i +++++ + +_Proof_: By inspection of the semantic equations. ∎ + +''''' + +=== 6. Type System + +==== 6.1 Types + +.... +τ ::= Int | Real | Bool | Symbol | Unit + | τ → τ ; function type + | Dist(τ) ; distribution type + | ∀α. τ ; polymorphism +.... + +==== 6.2 Typing Rules + +[latexmath] +++++ +\frac{}{Γ ⊢ n : \text{Int}} \text{ (T-INT)} +++++ + +[latexmath] +++++ +\frac{}{Γ ⊢ r : \text{Real}} \text{ (T-REAL)} +++++ + +[latexmath] +++++ +\frac{Γ ⊢ e_0 : τ \quad Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ}{Γ ⊢ \text{(bet } e_0 \ e_1 \ e_2\text{)} : \text{Dist}(τ)} \text{ (T-BET)} +++++ + +[latexmath] +++++ +\frac{Γ, x : τ_1 ⊢ e : τ_2}{Γ ⊢ (λ (x) e) : τ_1 → τ_2} \text{ (T-ABS)} +++++ + +[latexmath] +++++ +\frac{Γ ⊢ e_1 : τ_1 → τ_2 \quad Γ ⊢ e_2 : τ_1}{Γ ⊢ (e_1 \ e_2) : τ_2} \text{ (T-APP)} +++++ + +[latexmath] +++++ +\frac{Γ ⊢ e : \text{Dist}(τ_1) \quad Γ ⊢ f : τ_1 → \text{Dist}(τ_2)}{Γ ⊢ \text{(bind } e \ f\text{)} : \text{Dist}(τ_2)} \text{ (T-BIND)} +++++ + +==== 6.3 Type Soundness + +*Theorem 6.1* (Type Soundness). If Γ ⊢ e : τ and ⟨e, σ, ω⟩ ⇓ ⟨v, ω’⟩ +where σ ⊨ Γ, then v : τ. + +_Proof_: By induction on the typing derivation and evaluation +derivation. ∎ + +''''' + +=== 7. Semantic Equivalences + +==== 7.1 Observational Equivalence + +*Definition 7.1* (Observational Equivalence). e₁ ≅ e₂ iff for all +contexts C[·]: + +[latexmath] +++++ +⟦C[e_1]⟧∅ = ⟦C[e_2]⟧∅ +++++ + +==== 7.2 Distribution Equivalence + +*Definition 7.2* (Distribution Equivalence). e₁ ≡_d e₂ iff: + +[latexmath] +++++ +⟦e_1⟧ρ = ⟦e_2⟧ρ +++++ +for all ρ. + +*Theorem 7.1* (Bet Symmetry). For any permutation π of \{0,1,2}: + +[latexmath] +++++ +(bet \ e_0 \ e_1 \ e_2) ≡_d (bet \ e_{π(0)} \ e_{π(1)} \ e_{π(2)}) +++++ + +_Proof_: Both expressions produce the same uniform distribution over +\{⟦e₀⟧ρ, ⟦e₁⟧ρ, ⟦e₂⟧ρ}. ∎ + +*Theorem 7.2* (Idempotence). + +[latexmath] +++++ +(bet \ e \ e \ e) ≡_d e +++++ + +_Proof_: ⟦(bet e e e)⟧ρ = (1/3)⟦e⟧ρ + (1/3)⟦e⟧ρ + (1/3)⟦e⟧ρ = ⟦e⟧ρ. ∎ + +*Theorem 7.3* (Linearity of Expectation). + +[latexmath] +++++ +\mathbb{E}[\text{(bet } A \ B \ C\text{)}] = \frac{1}{3}(\mathbb{E}[A] + \mathbb{E}[B] + \mathbb{E}[C]) +++++ + +''''' + +=== 8. Fixed Points and Recursion + +==== 8.1 Domain-Theoretic Foundation + +*Definition 8.1* (CPO of Distributions). Dist(D) forms a complete +partial order under: + +[latexmath] +++++ +μ ⊑ ν \iff ∀x. μ(x) ≤ ν(x) +++++ + +*Theorem 8.1* (Kleene Fixed Point). Recursive bet definitions have least +fixed points: + +For F: Dist(D) → Dist(D) continuous, there exists least fixed point μ* = +⊔ₙ Fⁿ(⊥). + +==== 8.2 Recursive Bets + +The `+bet-chain+` and `+bet-until+` constructs define recursive +computations: + +[latexmath] +++++ +⟦\text{(bet-until } p \ t\text{)}⟧ρ = \mu X. (⟦t⟧ρ \rhd λv. \text{if } p(v) \text{ then } δ_v \text{ else } X) +++++ + +''''' + +=== 9. Game Semantics (Optional Advanced Framework) + +==== 9.1 Games for Probabilistic Choice + +A probabilistic game consists of: - Positions P - Player/Opponent moves +at each position - A probability distribution over nature’s moves + +*Definition 9.1* (Bet Game). The game for (bet A B C): - Initial +position: ? - Nature moves: selects branch with probability 1/3 each - +Player responds: plays the selected subgame + +==== 9.2 Strategy Composition + +Strategies compose via: - Sequential: play one game then another - +Probabilistic: nature selects which strategy + +''''' + +=== 10. Semantic Gaps and Future Work + +==== 10.1 Currently Unformalized + +*TODO*: The following require additional formalization: + +[arabic] +. *Continuous distributions*: Current semantics handle discrete +distributions only +. *Measure-theoretic continuous semantics*: Need σ-algebra over function +spaces +. *Concurrency semantics*: No formal model for parallel bet execution +. *Effect handlers*: Exception propagation not fully formalized + +==== 10.2 Proposed Extensions + +[arabic] +. *Probabilistic Effect System*: Track probabilistic effects in types +. *Gradual Typing*: Allow dynamic probabilistic types +. *Dependent Types*: Types depending on probability values + +''''' + +=== Appendix: Inference Rules Summary + +==== Operational Rules + +[cols=",",options="header",] +|=== +|Rule |Form +|E-BET |Uniform random selection +|E-WBET |Weighted random selection +|E-COND-T/F |Conditional branching +|E-LAZY |Lazy evaluation +|E-CHAIN |Iterative chaining +|=== + +==== Typing Rules + +[cols=",",options="header",] +|=== +|Rule |Judgment +|T-BET |(bet e e e) : Dist(τ) +|T-BIND |(bind e f) : Dist(τ₂) +|T-ABS |(λ x e) : τ₁ → τ₂ +|=== + +''''' + +=== References + +[arabic] +. Plotkin, G. (1981). "`A Structural Approach to Operational Semantics`" +. Moggi, E. (1991). "`Notions of Computation and Monads`" +. Kozen, D. (1981). "`Semantics of Probabilistic Programs`" +. McIver, A. & Morgan, C. (2005). "`Abstraction, Refinement and Proof +for Probabilistic Systems`" +. Ramsey, N. & Pfeffer, A. (2002). "`Stochastic Lambda Calculus`" diff --git a/proofs/formal-semantics.md b/proofs/formal-semantics.md deleted file mode 100644 index 19c273b..0000000 --- a/proofs/formal-semantics.md +++ /dev/null @@ -1,392 +0,0 @@ - -# Formal Semantics of Betlang - -## Abstract - -This document provides a complete formal semantics for betlang, including operational semantics (small-step and big-step), denotational semantics, and axiomatic semantics. We establish the semantic foundations required for formal reasoning about betlang programs. - ---- - -## 1. Syntax - -### 1.1 Abstract Grammar - -``` -e ::= v ; values - | (bet e e e) ; ternary choice - | (bet/weighted p p p) ; weighted choice - | (bet/conditional e e e e) ; conditional choice - | (bet/lazy e e e) ; lazy choice - | (bet-chain e e e) ; iteration - | (bet-compose e e e) ; function composition - | (e e) ; application - | (λ (x) e) ; abstraction - | (if e e e) ; conditional - | x ; variable - -v ::= n ; numbers - | b ; booleans - | 'sym ; symbols - | (λ (x) e) ; closures - | () ; unit - -p ::= '(e w) ; weighted pair (value, weight) - -w ::= n where n ≥ 0 ; non-negative weight -``` - -### 1.2 Syntactic Sugar - -``` -(bet A B C) ≡ (bet/weighted '(A 1) '(B 1) '(C 1)) -``` - ---- - -## 2. Operational Semantics - -### 2.1 Configuration - -A configuration is a triple ⟨e, σ, ω⟩ where: -- e: expression being evaluated -- σ: environment mapping variables to values -- ω: random stream (infinite sequence of values in [0,1)) - -### 2.2 Small-Step Semantics (→) - -#### Values -$$\frac{}{⟨v, σ, ω⟩ → ⟨v, σ, ω⟩} \text{ (V-VALUE)}$$ - -#### Uniform Bet -$$\frac{r = \text{head}(ω) \quad ω' = \text{tail}(ω) \quad i = \lfloor 3r \rfloor}{⟨\text{(bet } v_0 \ v_1 \ v_2\text{)}, σ, ω⟩ → ⟨v_i, σ, ω'⟩} \text{ (E-BET)}$$ - -#### Weighted Bet -$$\frac{r = \text{head}(ω) \quad ω' = \text{tail}(ω) \quad W = w_0 + w_1 + w_2 \quad i = \text{select}(r \cdot W, [w_0, w_1, w_2])}{⟨\text{(bet/weighted } '(v_0 \ w_0) \ '(v_1 \ w_1) \ '(v_2 \ w_2)\text{)}, σ, ω⟩ → ⟨v_i, σ, ω'⟩} \text{ (E-WBET)}$$ - -where select(x, [w₀, w₁, w₂]) returns: -- 0 if x < w₀ -- 1 if w₀ ≤ x < w₀ + w₁ -- 2 otherwise - -#### Conditional Bet -$$\frac{⟨e_{pred}, σ, ω⟩ →^* ⟨\text{true}, σ, ω'⟩}{⟨\text{(bet/conditional } e_{pred} \ e_a \ e_b \ e_c\text{)}, σ, ω⟩ → ⟨e_a, σ, ω'⟩} \text{ (E-COND-T)}$$ - -$$\frac{⟨e_{pred}, σ, ω⟩ →^* ⟨\text{false}, σ, ω'⟩}{⟨\text{(bet/conditional } e_{pred} \ e_a \ e_b \ e_c\text{)}, σ, ω⟩ → ⟨\text{(bet } e_b \ e_c \ e_a\text{)}, σ, ω'⟩} \text{ (E-COND-F)}$$ - -#### Lazy Bet -$$\frac{r = \text{head}(ω) \quad ω' = \text{tail}(ω) \quad i = \lfloor 3r \rfloor \quad ⟨(thunk_i), σ, ω'⟩ →^* ⟨v, σ, ω''⟩}{⟨\text{(bet/lazy } thunk_0 \ thunk_1 \ thunk_2\text{)}, σ, ω⟩ → ⟨v, σ, ω''⟩} \text{ (E-LAZY)}$$ - -#### Function Application -$$\frac{⟨e_1, σ, ω⟩ →^* ⟨(λ (x) e), σ, ω'⟩ \quad ⟨e_2, σ, ω'⟩ →^* ⟨v, σ, ω''⟩ \quad ⟨e, σ[x ↦ v], ω''⟩ →^* ⟨v', σ, ω'''⟩}{⟨(e_1 \ e_2), σ, ω⟩ → ⟨v', σ, ω'''⟩} \text{ (E-APP)}$$ - -#### Bet Chain -$$\frac{n = 0}{⟨\text{(bet-chain } 0 \ f \ v\text{)}, σ, ω⟩ → ⟨v, σ, ω⟩} \text{ (E-CHAIN-0)}$$ - -$$\frac{n > 0 \quad ⟨(f \ v), σ, ω⟩ →^* ⟨v', σ, ω'⟩ \quad ⟨\text{(bet-chain } (n-1) \ f \ v'\text{)}, σ, ω'⟩ →^* ⟨v'', σ, ω''⟩}{⟨\text{(bet-chain } n \ f \ v\text{)}, σ, ω⟩ → ⟨v'', σ, ω''⟩} \text{ (E-CHAIN-N)}$$ - -### 2.3 Big-Step Semantics (⇓) - -#### Values -$$\frac{}{⟨v, σ, ω⟩ ⇓ ⟨v, ω⟩} \text{ (B-VALUE)}$$ - -#### Uniform Bet -$$\frac{⟨e_0, σ, ω⟩ ⇓ ⟨v_0, ω_0⟩ \quad ⟨e_1, σ, ω_0⟩ ⇓ ⟨v_1, ω_1⟩ \quad ⟨e_2, σ, ω_1⟩ ⇓ ⟨v_2, ω_2⟩ \quad r = \text{head}(ω_2) \quad i = \lfloor 3r \rfloor}{⟨\text{(bet } e_0 \ e_1 \ e_2\text{)}, σ, ω⟩ ⇓ ⟨v_i, \text{tail}(ω_2)⟩} \text{ (B-BET)}$$ - -**Note**: This evaluates all three branches before selecting (eager evaluation). For lazy semantics, see B-LAZY below. - -#### Lazy Bet (Big-Step) -$$\frac{r = \text{head}(ω) \quad i = \lfloor 3r \rfloor \quad ⟨(thunk_i), σ, \text{tail}(ω)⟩ ⇓ ⟨v, ω'⟩}{⟨\text{(bet/lazy } thunk_0 \ thunk_1 \ thunk_2\text{)}, σ, ω⟩ ⇓ ⟨v, ω'⟩} \text{ (B-LAZY)}$$ - ---- - -## 3. Denotational Semantics - -### 3.1 Semantic Domains - -**Domain D** (semantic values): -``` -D = ℤ + ℝ + Bool + Symbol + (D → Dist(D)) + Unit + Error -``` - -**Distribution Domain** Dist(D): -``` -Dist(D) = D → [0, 1] such that Σ_d P(d) = 1 -``` - -### 3.2 Semantic Function - -$$⟦ \cdot ⟧ : \text{Expr} → \text{Env} → \text{Dist}(D)$$ - -#### Literals -$$⟦ n ⟧ρ = δ_n$$ (Dirac distribution at n) -$$⟦ \text{true} ⟧ρ = δ_{\text{true}}$$ -$$⟦ 'sym ⟧ρ = δ_{sym}$$ - -#### Variables -$$⟦ x ⟧ρ = δ_{ρ(x)}$$ - -#### Uniform Bet -$$⟦ \text{(bet } e_0 \ e_1 \ e_2\text{)} ⟧ρ = \frac{1}{3}⟦e_0⟧ρ + \frac{1}{3}⟦e_1⟧ρ + \frac{1}{3}⟦e_2⟧ρ$$ - -where + denotes mixture of distributions. - -#### Weighted Bet -$$⟦ \text{(bet/weighted } '(e_0 \ w_0) \ '(e_1 \ w_1) \ '(e_2 \ w_2)\text{)} ⟧ρ = \sum_{i=0}^{2} \frac{w_i}{W} ⟦e_i⟧ρ$$ - -where W = w₀ + w₁ + w₂. - -#### Conditional -$$⟦ \text{(if } e_c \ e_t \ e_f\text{)} ⟧ρ = ⟦e_c⟧ρ \rhd λb. \begin{cases} ⟦e_t⟧ρ & \text{if } b = \text{true} \\ ⟦e_f⟧ρ & \text{if } b = \text{false} \end{cases}$$ - -where ▷ is monadic bind for distributions. - -#### Abstraction -$$⟦ (λ (x) e) ⟧ρ = δ_{λv. ⟦e⟧ρ[x ↦ v]}$$ - -#### Application -$$⟦ (e_1 \ e_2) ⟧ρ = ⟦e_1⟧ρ \rhd λf. ⟦e_2⟧ρ \rhd λv. f(v)$$ - -### 3.3 Distribution Operations - -**Mixture**: For distributions μ, ν and weight p ∈ [0,1]: -$$(p \cdot μ + (1-p) \cdot ν)(x) = p \cdot μ(x) + (1-p) \cdot ν(x)$$ - -**Bind** (▷): For distribution μ and function f: D → Dist(D): -$$(μ \rhd f)(y) = \sum_{x} μ(x) \cdot f(x)(y)$$ - -**Expected Value**: -$$\mathbb{E}[⟦e⟧ρ] = \sum_{v} v \cdot ⟦e⟧ρ(v)$$ - ---- - -## 4. Axiomatic Semantics - -### 4.1 Probabilistic Hoare Logic - -We extend Hoare triples to probabilistic settings: - -$$\{P\} \ e \ \{Q\}_p$$ - -meaning: if precondition P holds, then after evaluating e, postcondition Q holds with probability at least p. - -### 4.2 Axioms - -#### Bet Axiom -$$\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{Q\}_{1} \quad \text{if} \ P \Rightarrow Q[A/x] \land Q[B/x] \land Q[C/x]$$ - -"If Q holds for all possible outcomes, it holds with certainty." - -#### Probabilistic Bet Axiom -$$\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{x = A\}_{1/3}$$ -$$\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{x = B\}_{1/3}$$ -$$\{P\} \ \text{(bet } A \ B \ C\text{)} \ \{x = C\}_{1/3}$$ - -#### Weighted Bet Axiom -$$\{P\} \ \text{(bet/weighted } '(A \ w_A) \ '(B \ w_B) \ '(C \ w_C)\text{)} \ \{x = A\}_{w_A/W}$$ - -where W = wₐ + w_B + w_C. - -#### Conditional Axiom -$$\frac{\{P ∧ b\} \ e_1 \ \{Q\}_p \quad \{P ∧ ¬b\} \ e_2 \ \{Q\}_p}{\{P\} \ \text{(if } b \ e_1 \ e_2\text{)} \ \{Q\}_p}$$ - -#### Sequence Rule -$$\frac{\{P\} \ e_1 \ \{R\}_{p_1} \quad \{R\} \ e_2 \ \{Q\}_{p_2}}{\{P\} \ e_1; e_2 \ \{Q\}_{p_1 \cdot p_2}}$$ - -#### Consequence Rule -$$\frac{P' \Rightarrow P \quad \{P\} \ e \ \{Q\}_p \quad Q \Rightarrow Q' \quad p \leq p'}{\{P'\} \ e \ \{Q'\}_{p'}}$$ - -### 4.3 Expectation Calculus - -For numeric expressions, we can reason about expected values: - -$$\text{wp}[\text{(bet } A \ B \ C\text{)}](f) = \frac{1}{3}(f(A) + f(B) + f(C))$$ - -where wp is the weakest precondition transformer and f is a post-expectation. - -**Theorem 4.1** (Expectation Soundness). If ⟦e⟧ρ = μ, then: -$$\mathbb{E}_{x \sim μ}[f(x)] = \text{wp}[e](f)$$ - ---- - -## 5. Semantic Properties - -### 5.1 Determinism and Confluence - -**Theorem 5.1** (Probabilistic Confluence). Betlang is confluent up to distribution: - -If ⟨e, σ, ω⟩ →* ⟨v₁, σ, ω₁⟩ and ⟨e, σ, ω⟩ →* ⟨v₂, σ, ω₂⟩ with the same random stream ω, then v₁ = v₂. - -*Proof*: The operational semantics is deterministic given a fixed random stream. ∎ - -**Corollary 5.1** (Distribution Determinism). The distribution ⟦e⟧ρ is uniquely determined by e and ρ. - -### 5.2 Adequacy - -**Theorem 5.2** (Computational Adequacy). For closed expressions e: - -$$P(⟨e, ∅, ω⟩ ⇓ ⟨v, ω'⟩) = ⟦e⟧∅(v)$$ - -where the left side is probability over random streams ω. - -*Proof sketch*: By structural induction on e, showing operational and denotational semantics agree. ∎ - -### 5.3 Compositionality - -**Theorem 5.3** (Compositionality). The denotational semantics is compositional: - -$$⟦e⟧ρ \text{ depends only on } ⟦e_i⟧ρ \text{ for immediate subexpressions } e_i$$ - -*Proof*: By inspection of the semantic equations. ∎ - ---- - -## 6. Type System - -### 6.1 Types - -``` -τ ::= Int | Real | Bool | Symbol | Unit - | τ → τ ; function type - | Dist(τ) ; distribution type - | ∀α. τ ; polymorphism -``` - -### 6.2 Typing Rules - -$$\frac{}{Γ ⊢ n : \text{Int}} \text{ (T-INT)}$$ - -$$\frac{}{Γ ⊢ r : \text{Real}} \text{ (T-REAL)}$$ - -$$\frac{Γ ⊢ e_0 : τ \quad Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ}{Γ ⊢ \text{(bet } e_0 \ e_1 \ e_2\text{)} : \text{Dist}(τ)} \text{ (T-BET)}$$ - -$$\frac{Γ, x : τ_1 ⊢ e : τ_2}{Γ ⊢ (λ (x) e) : τ_1 → τ_2} \text{ (T-ABS)}$$ - -$$\frac{Γ ⊢ e_1 : τ_1 → τ_2 \quad Γ ⊢ e_2 : τ_1}{Γ ⊢ (e_1 \ e_2) : τ_2} \text{ (T-APP)}$$ - -$$\frac{Γ ⊢ e : \text{Dist}(τ_1) \quad Γ ⊢ f : τ_1 → \text{Dist}(τ_2)}{Γ ⊢ \text{(bind } e \ f\text{)} : \text{Dist}(τ_2)} \text{ (T-BIND)}$$ - -### 6.3 Type Soundness - -**Theorem 6.1** (Type Soundness). If Γ ⊢ e : τ and ⟨e, σ, ω⟩ ⇓ ⟨v, ω'⟩ where σ ⊨ Γ, then v : τ. - -*Proof*: By induction on the typing derivation and evaluation derivation. ∎ - ---- - -## 7. Semantic Equivalences - -### 7.1 Observational Equivalence - -**Definition 7.1** (Observational Equivalence). e₁ ≅ e₂ iff for all contexts C[·]: -$$⟦C[e_1]⟧∅ = ⟦C[e_2]⟧∅$$ - -### 7.2 Distribution Equivalence - -**Definition 7.2** (Distribution Equivalence). e₁ ≡_d e₂ iff: -$$⟦e_1⟧ρ = ⟦e_2⟧ρ$$ for all ρ. - -**Theorem 7.1** (Bet Symmetry). For any permutation π of {0,1,2}: -$$(bet \ e_0 \ e_1 \ e_2) ≡_d (bet \ e_{π(0)} \ e_{π(1)} \ e_{π(2)})$$ - -*Proof*: Both expressions produce the same uniform distribution over {⟦e₀⟧ρ, ⟦e₁⟧ρ, ⟦e₂⟧ρ}. ∎ - -**Theorem 7.2** (Idempotence). -$$(bet \ e \ e \ e) ≡_d e$$ - -*Proof*: ⟦(bet e e e)⟧ρ = (1/3)⟦e⟧ρ + (1/3)⟦e⟧ρ + (1/3)⟦e⟧ρ = ⟦e⟧ρ. ∎ - -**Theorem 7.3** (Linearity of Expectation). -$$\mathbb{E}[\text{(bet } A \ B \ C\text{)}] = \frac{1}{3}(\mathbb{E}[A] + \mathbb{E}[B] + \mathbb{E}[C])$$ - ---- - -## 8. Fixed Points and Recursion - -### 8.1 Domain-Theoretic Foundation - -**Definition 8.1** (CPO of Distributions). Dist(D) forms a complete partial order under: -$$μ ⊑ ν \iff ∀x. μ(x) ≤ ν(x)$$ - -**Theorem 8.1** (Kleene Fixed Point). Recursive bet definitions have least fixed points: - -For F: Dist(D) → Dist(D) continuous, there exists least fixed point μ* = ⊔ₙ Fⁿ(⊥). - -### 8.2 Recursive Bets - -The `bet-chain` and `bet-until` constructs define recursive computations: - -$$⟦\text{(bet-until } p \ t\text{)}⟧ρ = \mu X. (⟦t⟧ρ \rhd λv. \text{if } p(v) \text{ then } δ_v \text{ else } X)$$ - ---- - -## 9. Game Semantics (Optional Advanced Framework) - -### 9.1 Games for Probabilistic Choice - -A probabilistic game consists of: -- Positions P -- Player/Opponent moves at each position -- A probability distribution over nature's moves - -**Definition 9.1** (Bet Game). The game for (bet A B C): -- Initial position: ? -- Nature moves: selects branch with probability 1/3 each -- Player responds: plays the selected subgame - -### 9.2 Strategy Composition - -Strategies compose via: -- Sequential: play one game then another -- Probabilistic: nature selects which strategy - ---- - -## 10. Semantic Gaps and Future Work - -### 10.1 Currently Unformalized - -**TODO**: The following require additional formalization: - -1. **Continuous distributions**: Current semantics handle discrete distributions only -2. **Measure-theoretic continuous semantics**: Need σ-algebra over function spaces -3. **Concurrency semantics**: No formal model for parallel bet execution -4. **Effect handlers**: Exception propagation not fully formalized - -### 10.2 Proposed Extensions - -1. **Probabilistic Effect System**: Track probabilistic effects in types -2. **Gradual Typing**: Allow dynamic probabilistic types -3. **Dependent Types**: Types depending on probability values - ---- - -## Appendix: Inference Rules Summary - -### Operational Rules -| Rule | Form | -|------|------| -| E-BET | Uniform random selection | -| E-WBET | Weighted random selection | -| E-COND-T/F | Conditional branching | -| E-LAZY | Lazy evaluation | -| E-CHAIN | Iterative chaining | - -### Typing Rules -| Rule | Judgment | -|------|----------| -| T-BET | (bet e e e) : Dist(τ) | -| T-BIND | (bind e f) : Dist(τ₂) | -| T-ABS | (λ x e) : τ₁ → τ₂ | - ---- - -## References - -1. Plotkin, G. (1981). "A Structural Approach to Operational Semantics" -2. Moggi, E. (1991). "Notions of Computation and Monads" -3. Kozen, D. (1981). "Semantics of Probabilistic Programs" -4. McIver, A. & Morgan, C. (2005). "Abstraction, Refinement and Proof for Probabilistic Systems" -5. Ramsey, N. & Pfeffer, A. (2002). "Stochastic Lambda Calculus" diff --git a/proofs/mathematical-foundations.adoc b/proofs/mathematical-foundations.adoc new file mode 100644 index 0000000..6233cac --- /dev/null +++ b/proofs/mathematical-foundations.adoc @@ -0,0 +1,522 @@ +== Mathematical Foundations of Betlang + +=== Abstract + +This document establishes the rigorous mathematical foundations +underlying betlang, a ternary probabilistic programming language. We +formalize the probability spaces, measure-theoretic foundations, and +algebraic structures that govern the language semantics. + +''''' + +=== 1. Probability Space Foundation + +==== 1.1 Sample Space Definition + +*Definition 1.1* (Ternary Sample Space). For any bet expression +`+(bet A B C)+`, the sample space is: + +[latexmath] +++++ +\Omega = \{A, B, C\} +++++ + +where latexmath:[|\Omega| = 3] (the ternary cardinality). + +*Definition 1.2* (σ-algebra). The σ-algebra over Ω is the power set: + +[latexmath] +++++ +\mathcal{F} = \mathcal{P}(\Omega) = \{\emptyset, \{A\}, \{B\}, \{C\}, \{A,B\}, \{A,C\}, \{B,C\}, \{A,B,C\}\} +++++ + +with latexmath:[|\mathcal{F}| = 2^3 = 8] events. + +==== 1.2 Probability Measure + +*Definition 1.3* (Uniform Ternary Measure). The probability measure P: F +→ [0,1] for uniform bet is: + +[latexmath] +++++ +P(\{x\}) = \frac{1}{3} \quad \forall x \in \Omega +++++ + +*Theorem 1.1* (Measure Axioms). The betlang probability measure +satisfies Kolmogorov’s axioms: + +[arabic] +. *Non-negativity*: latexmath:[P(E) \geq 0] for all +latexmath:[E \in \mathcal{F}] +. *Unitarity*: latexmath:[P(\Omega) = P(\{A,B,C\}) = 1] +. *σ-additivity*: For disjoint events latexmath:[E_i]: +latexmath:[P(\bigcup_i E_i) = \sum_i P(E_i)] + +_Proof_: 1. By construction, latexmath:[P(\{x\}) = 1/3 \geq 0] and +additivity preserves non-negativity. 2. +latexmath:[P(\Omega) = P(\{A\}) + P(\{B\}) + P(\{C\}) = 1/3 + 1/3 + 1/3 = 1] +∎ 3. For finite discrete spaces, σ-additivity reduces to finite +additivity, which holds by definition of P on compound events. ∎ + +==== 1.3 Weighted Probability Measure + +*Definition 1.4* (Weighted Ternary Measure). For +`+(bet/weighted '(A wₐ) '(B w_b) '(C w_c))+`: + +[latexmath] +++++ +P(\{x\}) = \frac{w_x}{\sum_{y \in \Omega} w_y} +++++ + +*Theorem 1.2* (Weighted Measure Validity). The weighted measure +satisfies probability axioms iff all weights are non-negative and at +least one is positive. + +_Proof_: - If latexmath:[w_x \geq 0] for all x and +latexmath:[\sum w_x > 0], then latexmath:[P(\{x\}) \geq 0] and +latexmath:[\sum P(\{x\}) = 1]. - Conversely, if any latexmath:[w_x < 0], +then latexmath:[P(\{x\})] may be negative, violating axiom 1. ∎ + +''''' + +=== 2. Measure-Theoretic Framework + +==== 2.1 Random Variables + +*Definition 2.1* (Bet Random Variable). A bet expression defines a +random variable: + +[latexmath] +++++ +X: \Omega \to E +++++ + +where E is the outcome space (which may differ from Ω when outcomes are +transformed). + +*Definition 2.2* (Pushforward Measure). For a function f: E → F, the +pushforward measure is: + +[latexmath] +++++ +P_f(B) = P(f^{-1}(B)) = P(\{\omega : f(X(\omega)) \in B\}) +++++ + +This formalizes `+bet-map+` operations. + +==== 2.2 Independence + +*Definition 2.3* (Independent Bets). Two bet expressions X and Y are +independent iff: + +[latexmath] +++++ +P(X = x, Y = y) = P(X = x) \cdot P(Y = y) +++++ + +*Theorem 2.1* (bet-parallel Independence). Each trial in +`+(bet-parallel n A B C)+` is independent. + +_Proof_: Each trial uses an independent random number generator call. By +construction of the pseudorandom number generator (assuming proper +seeding), successive calls produce independent uniform samples. ∎ + +*Remark*: True independence requires cryptographic-quality randomness. +Racket’s `+random+` provides pseudorandom independence sufficient for +simulation purposes. + +==== 2.3 Conditional Probability + +*Definition 2.4* (Conditional Bet). For +`+(bet/conditional pred A B C)+`: + +[latexmath] +++++ +P(X = x | \text{pred}) = \begin{cases} 1 & \text{if pred is true and } x = A \\ P_{\text{bet}}(x) & \text{if pred is false} \end{cases} +++++ + +where latexmath:[P_{\text{bet}}] is the standard ternary measure over +\{B, C, A} when pred is false. + +''''' + +=== 3. Expected Value Theory + +==== 3.1 Expectation Definition + +*Definition 3.1* (Expected Value). For a bet random variable X with +outcomes \{a, b, c}: + +[latexmath] +++++ +\mathbb{E}[X] = \frac{1}{3}(a + b + c) +++++ + +when a, b, c are numeric and uniformly weighted. + +*Definition 3.2* (Weighted Expectation). For weighted bets: + +[latexmath] +++++ +\mathbb{E}[X] = \sum_{x \in \{a,b,c\}} P(\{x\}) \cdot x = \frac{w_a \cdot a + w_b \cdot b + w_c \cdot c}{w_a + w_b + w_c} +++++ + +==== 3.2 Moments + +*Theorem 3.1* (Variance Formula). For uniform ternary bet: + +[latexmath] +++++ +\text{Var}(X) = \mathbb{E}[X^2] - \mathbb{E}[X]^2 = \frac{a^2 + b^2 + c^2}{3} - \left(\frac{a+b+c}{3}\right)^2 +++++ + +_Proof_: Direct application of the variance definition with discrete +uniform distribution. ∎ + +*Corollary 3.1* (Symmetric Bet Variance). If a = -d, b = 0, c = d for +some d ≠ 0: + +[latexmath] +++++ +\text{Var}(X) = \frac{d^2 + 0 + d^2}{3} - 0 = \frac{2d^2}{3} +++++ + +''''' + +=== 4. Information Theory + +==== 4.1 Shannon Entropy + +*Definition 4.1* (Entropy of Ternary Bet). The Shannon entropy of a bet +distribution: + +[latexmath] +++++ +H(X) = -\sum_{x \in \Omega} P(x) \log_2 P(x) +++++ + +*Theorem 4.1* (Maximum Entropy). The uniform ternary bet achieves +maximum entropy: + +[latexmath] +++++ +H_{\max} = \log_2(3) \approx 1.585 \text{ bits} +++++ + +_Proof_: By the maximum entropy theorem, entropy is maximized when all +outcomes are equiprobable. For uniform distribution P(x) = 1/3: + +[latexmath] +++++ +H = -3 \cdot \frac{1}{3} \log_2\frac{1}{3} = \log_2(3) ∎ +++++ + +*Theorem 4.2* (Entropy Bounds). For any ternary bet: + +[latexmath] +++++ +0 \leq H(X) \leq \log_2(3) +++++ + +with H = 0 iff one outcome has probability 1 (deterministic), and H = +log₂(3) iff uniform. + +==== 4.2 Mutual Information + +*Definition 4.2* (Mutual Information). For two bet expressions X and Y: + +[latexmath] +++++ +I(X; Y) = H(X) + H(Y) - H(X, Y) +++++ + +*Theorem 4.3* (Independent Bets). If X and Y are independent: + +[latexmath] +++++ +I(X; Y) = 0 +++++ + +_Proof_: For independent variables, H(X,Y) = H(X) + H(Y), so I(X;Y) = 0. +∎ + +''''' + +=== 5. Algebraic Structure + +==== 5.1 Probability Monad + +*Definition 5.1* (Bet Monad). The bet construction forms a monad (Bet, +return, bind): + +*return* (unit): latexmath:[\eta: A \to \text{Bet}(A)] + +.... +return x = (bet x x x) +.... + +*bind* (>>=): +latexmath:[\mu: \text{Bet}(A) \to (A \to \text{Bet}(B)) \to \text{Bet}(B)] + +.... +bind m f = evaluate m, then apply f to result +.... + +*Theorem 5.1* (Monad Laws). The bet monad satisfies: + +[arabic] +. *Left identity*: `+(bind (return x) f) = (f x)+` +. *Right identity*: `+(bind m return) = m+` +. *Associativity*: +`+(bind (bind m f) g) = (bind m (λ x. bind (f x) g))+` + +_Proof_: 1. `+(bind (bet x x x) f)+` always returns `+(f x)+` since all +outcomes are x. ∎ 2. `+(bind m return)+` returns `+(bet m m m)+` which +equals m distributionally. ∎ 3. Follows from associativity of function +composition over probabilistic choice. ∎ + +==== 5.2 Kleisli Category + +*Definition 5.2* (Kleisli Morphisms). The Kleisli category for Bet has: +- Objects: Types A, B, C, … - Morphisms: Functions A → Bet(B) - +Composition: Kleisli composition via bind + +*Theorem 5.2* (Category Laws). Kleisli composition satisfies identity +and associativity laws by the monad laws. + +==== 5.3 Semiring Structure + +*Definition 5.3* (Probability Semiring). Bet probabilities form a +semiring (ℝ≥0, +, ×, 0, 1): + +* Addition: corresponds to disjoint union of events +* Multiplication: corresponds to independent conjunction + +''''' + +=== 6. Ternary Logic Foundation + +==== 6.1 Kleene’s Three-Valued Logic + +*Definition 6.1* (Truth Values). The ternary truth domain: + +[latexmath] +++++ +\mathbb{T}_3 = \{0, \frac{1}{2}, 1\} +++++ + +representing FALSE, UNKNOWN, TRUE respectively. + +*Definition 6.2* (Logical Operations). + +[cols=",",options="header",] +|=== +|Operation |Definition +|AND(a,b,c) |min(a, b, c) +|OR(a,b,c) |max(a, b, c) +|NOT(a) |1 - a +|=== + +*Theorem 6.1* (De Morgan’s Laws). For ternary logic: + +[latexmath] +++++ +\neg(a \land b \land c) = \neg a \lor \neg b \lor \neg c +++++ + +[latexmath] +++++ +\neg(a \lor b \lor c) = \neg a \land \neg b \land \neg c +++++ + +_Proof_: - NOT(min(a,b,c)) = 1 - min(a,b,c) = max(1-a, 1-b, 1-c) = +max(NOT a, NOT b, NOT c) ∎ + +==== 6.2 Relationship to Probability + +*Theorem 6.2* (Probabilistic Interpretation). Ternary truth values can +be interpreted as probabilities: + +* TRUE (1): Certain to hold +* FALSE (0): Certainly does not hold +* UNKNOWN (1/2): Maximum uncertainty (entropy = 1 bit) + +The ternary AND/OR operations correspond to: - AND: Probability of all +events (under positive correlation assumption: min) - OR: Probability of +at least one event (under positive correlation: max) + +''''' + +=== 7. Convergence Theory + +==== 7.1 Law of Large Numbers + +*Theorem 7.1* (Weak LLN for Bets). For i.i.d. bet trials X₁, X₂, …, Xₙ: + +[latexmath] +++++ +\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i \xrightarrow{P} \mathbb{E}[X] +++++ + +as n → ∞. + +_Proof_: Follows directly from the classical Weak LLN since bet outcomes +are i.i.d. with finite expectation. ∎ + +*Theorem 7.2* (Strong LLN for Bets). Under the same conditions: + +[latexmath] +++++ +\bar{X}_n \xrightarrow{a.s.} \mathbb{E}[X] +++++ + +_Proof_: Follows from the Strong LLN since Var(X) is finite for ternary +bets. ∎ + +==== 7.2 Central Limit Theorem + +*Theorem 7.3* (CLT for Bets). For standardized sample mean: + +[latexmath] +++++ +\sqrt{n}\frac{\bar{X}_n - \mu}{\sigma} \xrightarrow{d} N(0, 1) +++++ + +where μ = E[X] and σ² = Var(X). + +_Proof_: Standard CLT applies since bet outcomes have finite variance. ∎ + +*Corollary 7.1* (Confidence Intervals). For large n, approximate 95% CI +for true probability: + +[latexmath] +++++ +\hat{p} \pm 1.96\sqrt{\frac{\hat{p}(1-\hat{p})}{n}} +++++ + +This justifies the `+bet-probability+` function’s convergence. + +''''' + +=== 8. Martingale Theory + +==== 8.1 Bet Sequences as Martingales + +*Definition 8.1* (Cumulative Bet Sum). Let Sₙ = Σᵢ₌₁ⁿ Xᵢ where Xᵢ are +centered i.i.d. bets. + +*Theorem 8.1* (Martingale Property). If E[Xᵢ] = 0, then \{Sₙ} is a +martingale: + +[latexmath] +++++ +\mathbb{E}[S_{n+1} | S_1, ..., S_n] = S_n +++++ + +_Proof_: + +[latexmath] +++++ +\mathbb{E}[S_{n+1} | S_n] = \mathbb{E}[S_n + X_{n+1} | S_n] = S_n + \mathbb{E}[X_{n+1}] = S_n + 0 = S_n ∎ +++++ + +==== 8.2 Optional Stopping + +*Theorem 8.2* (Optional Stopping for bet-until). For +`+(bet-until pred thunk)+` with stopping time τ: + +If E[τ] < ∞ and the bet sum is a martingale, then E[S_τ] = E[S₀] = 0. + +''''' + +=== 9. Measure-Theoretic Probability + +==== 9.1 Lebesgue Integration + +*Definition 9.1* (Expectation as Lebesgue Integral). For bet random +variable X: + +[latexmath] +++++ +\mathbb{E}[X] = \int_\Omega X(\omega) \, dP(\omega) +++++ + +For discrete ternary space, this reduces to the sum: + +[latexmath] +++++ +\mathbb{E}[X] = \sum_{\omega \in \Omega} X(\omega) P(\{\omega\}) +++++ + +==== 9.2 Radon-Nikodym Derivative + +*Theorem 9.1* (Density for Weighted Bets). Given uniform measure Q and +weighted measure P: + +[latexmath] +++++ +\frac{dP}{dQ}(\omega) = \frac{3w_\omega}{\sum w} +++++ + +This is the likelihood ratio used in importance sampling. + +''''' + +=== 10. Category-Theoretic Foundations + +==== 10.1 The Giry Monad + +*Definition 10.1* (Connection to Giry Monad). Betlang’s probability +monad is a discrete instantiation of the Giry monad on the category of +measurable spaces. + +The Giry monad G: *Meas* → *Meas* maps: - Objects: Measurable space (X, +Σ) ↦ (G(X), Σ_G) - Morphisms: Measurable f ↦ G(f) via pushforward + +*Theorem 10.1*. The discrete Bet monad is the restriction of the Giry +monad to finite discrete spaces. + +==== 10.2 Lawvere Theory + +*Definition 10.2* (Ternary Lawvere Theory). The algebraic theory of +betlang is generated by: - One ternary operation: bet(-, -, -) - Subject +to idempotency: bet(x, x, x) = x - Symmetry under outcome permutation +(distributionally) + +''''' + +=== Appendix A: Notation Summary + +[cols=",",options="header",] +|=== +|Symbol |Meaning +|Ω |Sample space \{A, B, C} +|F |σ-algebra (power set) +|P |Probability measure +|E[X] |Expected value +|Var(X) |Variance +|H(X) |Shannon entropy +|Bet(A) |Probability monad over type A +|T₃ |Ternary truth domain \{0, ½, 1} +|=== + +=== Appendix B: Key Results Summary + +[arabic] +. Uniform bet achieves maximum entropy log₂(3) ≈ 1.585 bits +. Bet monad satisfies all three monad laws +. LLN and CLT apply to bet sequences +. Ternary logic satisfies De Morgan’s laws +. Weighted bets define valid probability measures iff weights are +non-negative + +''''' + +=== References + +[arabic] +. Kolmogorov, A.N. (1933). _Foundations of the Theory of Probability_ +. Kleene, S.C. (1952). _Introduction to Metamathematics_ +. Giry, M. (1982). "`A categorical approach to probability theory`" +. Lawvere, F.W. (1963). "`Functorial Semantics of Algebraic Theories`" +. Shannon, C.E. (1948). "`A Mathematical Theory of Communication`" diff --git a/proofs/mathematical-foundations.md b/proofs/mathematical-foundations.md deleted file mode 100644 index a4a387d..0000000 --- a/proofs/mathematical-foundations.md +++ /dev/null @@ -1,376 +0,0 @@ - -# Mathematical Foundations of Betlang - -## Abstract - -This document establishes the rigorous mathematical foundations underlying betlang, a ternary probabilistic programming language. We formalize the probability spaces, measure-theoretic foundations, and algebraic structures that govern the language semantics. - ---- - -## 1. Probability Space Foundation - -### 1.1 Sample Space Definition - -**Definition 1.1** (Ternary Sample Space). For any bet expression `(bet A B C)`, the sample space is: - -$$\Omega = \{A, B, C\}$$ - -where $|\Omega| = 3$ (the ternary cardinality). - -**Definition 1.2** (σ-algebra). The σ-algebra over Ω is the power set: - -$$\mathcal{F} = \mathcal{P}(\Omega) = \{\emptyset, \{A\}, \{B\}, \{C\}, \{A,B\}, \{A,C\}, \{B,C\}, \{A,B,C\}\}$$ - -with $|\mathcal{F}| = 2^3 = 8$ events. - -### 1.2 Probability Measure - -**Definition 1.3** (Uniform Ternary Measure). The probability measure P: F → [0,1] for uniform bet is: - -$$P(\{x\}) = \frac{1}{3} \quad \forall x \in \Omega$$ - -**Theorem 1.1** (Measure Axioms). The betlang probability measure satisfies Kolmogorov's axioms: - -1. **Non-negativity**: $P(E) \geq 0$ for all $E \in \mathcal{F}$ -2. **Unitarity**: $P(\Omega) = P(\{A,B,C\}) = 1$ -3. **σ-additivity**: For disjoint events $E_i$: $P(\bigcup_i E_i) = \sum_i P(E_i)$ - -*Proof*: -1. By construction, $P(\{x\}) = 1/3 \geq 0$ and additivity preserves non-negativity. -2. $P(\Omega) = P(\{A\}) + P(\{B\}) + P(\{C\}) = 1/3 + 1/3 + 1/3 = 1$ ∎ -3. For finite discrete spaces, σ-additivity reduces to finite additivity, which holds by definition of P on compound events. ∎ - -### 1.3 Weighted Probability Measure - -**Definition 1.4** (Weighted Ternary Measure). For `(bet/weighted '(A wₐ) '(B w_b) '(C w_c))`: - -$$P(\{x\}) = \frac{w_x}{\sum_{y \in \Omega} w_y}$$ - -**Theorem 1.2** (Weighted Measure Validity). The weighted measure satisfies probability axioms iff all weights are non-negative and at least one is positive. - -*Proof*: -- If $w_x \geq 0$ for all x and $\sum w_x > 0$, then $P(\{x\}) \geq 0$ and $\sum P(\{x\}) = 1$. -- Conversely, if any $w_x < 0$, then $P(\{x\})$ may be negative, violating axiom 1. ∎ - ---- - -## 2. Measure-Theoretic Framework - -### 2.1 Random Variables - -**Definition 2.1** (Bet Random Variable). A bet expression defines a random variable: - -$$X: \Omega \to E$$ - -where E is the outcome space (which may differ from Ω when outcomes are transformed). - -**Definition 2.2** (Pushforward Measure). For a function f: E → F, the pushforward measure is: - -$$P_f(B) = P(f^{-1}(B)) = P(\{\omega : f(X(\omega)) \in B\})$$ - -This formalizes `bet-map` operations. - -### 2.2 Independence - -**Definition 2.3** (Independent Bets). Two bet expressions X and Y are independent iff: - -$$P(X = x, Y = y) = P(X = x) \cdot P(Y = y)$$ - -**Theorem 2.1** (bet-parallel Independence). Each trial in `(bet-parallel n A B C)` is independent. - -*Proof*: Each trial uses an independent random number generator call. By construction of the pseudorandom number generator (assuming proper seeding), successive calls produce independent uniform samples. ∎ - -**Remark**: True independence requires cryptographic-quality randomness. Racket's `random` provides pseudorandom independence sufficient for simulation purposes. - -### 2.3 Conditional Probability - -**Definition 2.4** (Conditional Bet). For `(bet/conditional pred A B C)`: - -$$P(X = x | \text{pred}) = \begin{cases} 1 & \text{if pred is true and } x = A \\ P_{\text{bet}}(x) & \text{if pred is false} \end{cases}$$ - -where $P_{\text{bet}}$ is the standard ternary measure over {B, C, A} when pred is false. - ---- - -## 3. Expected Value Theory - -### 3.1 Expectation Definition - -**Definition 3.1** (Expected Value). For a bet random variable X with outcomes {a, b, c}: - -$$\mathbb{E}[X] = \frac{1}{3}(a + b + c)$$ - -when a, b, c are numeric and uniformly weighted. - -**Definition 3.2** (Weighted Expectation). For weighted bets: - -$$\mathbb{E}[X] = \sum_{x \in \{a,b,c\}} P(\{x\}) \cdot x = \frac{w_a \cdot a + w_b \cdot b + w_c \cdot c}{w_a + w_b + w_c}$$ - -### 3.2 Moments - -**Theorem 3.1** (Variance Formula). For uniform ternary bet: - -$$\text{Var}(X) = \mathbb{E}[X^2] - \mathbb{E}[X]^2 = \frac{a^2 + b^2 + c^2}{3} - \left(\frac{a+b+c}{3}\right)^2$$ - -*Proof*: Direct application of the variance definition with discrete uniform distribution. ∎ - -**Corollary 3.1** (Symmetric Bet Variance). If a = -d, b = 0, c = d for some d ≠ 0: - -$$\text{Var}(X) = \frac{d^2 + 0 + d^2}{3} - 0 = \frac{2d^2}{3}$$ - ---- - -## 4. Information Theory - -### 4.1 Shannon Entropy - -**Definition 4.1** (Entropy of Ternary Bet). The Shannon entropy of a bet distribution: - -$$H(X) = -\sum_{x \in \Omega} P(x) \log_2 P(x)$$ - -**Theorem 4.1** (Maximum Entropy). The uniform ternary bet achieves maximum entropy: - -$$H_{\max} = \log_2(3) \approx 1.585 \text{ bits}$$ - -*Proof*: By the maximum entropy theorem, entropy is maximized when all outcomes are equiprobable. For uniform distribution P(x) = 1/3: - -$$H = -3 \cdot \frac{1}{3} \log_2\frac{1}{3} = \log_2(3) ∎$$ - -**Theorem 4.2** (Entropy Bounds). For any ternary bet: - -$$0 \leq H(X) \leq \log_2(3)$$ - -with H = 0 iff one outcome has probability 1 (deterministic), and H = log₂(3) iff uniform. - -### 4.2 Mutual Information - -**Definition 4.2** (Mutual Information). For two bet expressions X and Y: - -$$I(X; Y) = H(X) + H(Y) - H(X, Y)$$ - -**Theorem 4.3** (Independent Bets). If X and Y are independent: - -$$I(X; Y) = 0$$ - -*Proof*: For independent variables, H(X,Y) = H(X) + H(Y), so I(X;Y) = 0. ∎ - ---- - -## 5. Algebraic Structure - -### 5.1 Probability Monad - -**Definition 5.1** (Bet Monad). The bet construction forms a monad (Bet, return, bind): - -**return** (unit): $\eta: A \to \text{Bet}(A)$ -``` -return x = (bet x x x) -``` - -**bind** (>>=): $\mu: \text{Bet}(A) \to (A \to \text{Bet}(B)) \to \text{Bet}(B)$ -``` -bind m f = evaluate m, then apply f to result -``` - -**Theorem 5.1** (Monad Laws). The bet monad satisfies: - -1. **Left identity**: `(bind (return x) f) = (f x)` -2. **Right identity**: `(bind m return) = m` -3. **Associativity**: `(bind (bind m f) g) = (bind m (λ x. bind (f x) g))` - -*Proof*: -1. `(bind (bet x x x) f)` always returns `(f x)` since all outcomes are x. ∎ -2. `(bind m return)` returns `(bet m m m)` which equals m distributionally. ∎ -3. Follows from associativity of function composition over probabilistic choice. ∎ - -### 5.2 Kleisli Category - -**Definition 5.2** (Kleisli Morphisms). The Kleisli category for Bet has: -- Objects: Types A, B, C, ... -- Morphisms: Functions A → Bet(B) -- Composition: Kleisli composition via bind - -**Theorem 5.2** (Category Laws). Kleisli composition satisfies identity and associativity laws by the monad laws. - -### 5.3 Semiring Structure - -**Definition 5.3** (Probability Semiring). Bet probabilities form a semiring (ℝ≥0, +, ×, 0, 1): - -- Addition: corresponds to disjoint union of events -- Multiplication: corresponds to independent conjunction - ---- - -## 6. Ternary Logic Foundation - -### 6.1 Kleene's Three-Valued Logic - -**Definition 6.1** (Truth Values). The ternary truth domain: - -$$\mathbb{T}_3 = \{0, \frac{1}{2}, 1\}$$ - -representing FALSE, UNKNOWN, TRUE respectively. - -**Definition 6.2** (Logical Operations). - -| Operation | Definition | -|-----------|------------| -| AND(a,b,c) | min(a, b, c) | -| OR(a,b,c) | max(a, b, c) | -| NOT(a) | 1 - a | - -**Theorem 6.1** (De Morgan's Laws). For ternary logic: - -$$\neg(a \land b \land c) = \neg a \lor \neg b \lor \neg c$$ -$$\neg(a \lor b \lor c) = \neg a \land \neg b \land \neg c$$ - -*Proof*: -- NOT(min(a,b,c)) = 1 - min(a,b,c) = max(1-a, 1-b, 1-c) = max(NOT a, NOT b, NOT c) ∎ - -### 6.2 Relationship to Probability - -**Theorem 6.2** (Probabilistic Interpretation). Ternary truth values can be interpreted as probabilities: - -- TRUE (1): Certain to hold -- FALSE (0): Certainly does not hold -- UNKNOWN (1/2): Maximum uncertainty (entropy = 1 bit) - -The ternary AND/OR operations correspond to: -- AND: Probability of all events (under positive correlation assumption: min) -- OR: Probability of at least one event (under positive correlation: max) - ---- - -## 7. Convergence Theory - -### 7.1 Law of Large Numbers - -**Theorem 7.1** (Weak LLN for Bets). For i.i.d. bet trials X₁, X₂, ..., Xₙ: - -$$\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i \xrightarrow{P} \mathbb{E}[X]$$ - -as n → ∞. - -*Proof*: Follows directly from the classical Weak LLN since bet outcomes are i.i.d. with finite expectation. ∎ - -**Theorem 7.2** (Strong LLN for Bets). Under the same conditions: - -$$\bar{X}_n \xrightarrow{a.s.} \mathbb{E}[X]$$ - -*Proof*: Follows from the Strong LLN since Var(X) is finite for ternary bets. ∎ - -### 7.2 Central Limit Theorem - -**Theorem 7.3** (CLT for Bets). For standardized sample mean: - -$$\sqrt{n}\frac{\bar{X}_n - \mu}{\sigma} \xrightarrow{d} N(0, 1)$$ - -where μ = E[X] and σ² = Var(X). - -*Proof*: Standard CLT applies since bet outcomes have finite variance. ∎ - -**Corollary 7.1** (Confidence Intervals). For large n, approximate 95% CI for true probability: - -$$\hat{p} \pm 1.96\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$ - -This justifies the `bet-probability` function's convergence. - ---- - -## 8. Martingale Theory - -### 8.1 Bet Sequences as Martingales - -**Definition 8.1** (Cumulative Bet Sum). Let Sₙ = Σᵢ₌₁ⁿ Xᵢ where Xᵢ are centered i.i.d. bets. - -**Theorem 8.1** (Martingale Property). If E[Xᵢ] = 0, then {Sₙ} is a martingale: - -$$\mathbb{E}[S_{n+1} | S_1, ..., S_n] = S_n$$ - -*Proof*: -$$\mathbb{E}[S_{n+1} | S_n] = \mathbb{E}[S_n + X_{n+1} | S_n] = S_n + \mathbb{E}[X_{n+1}] = S_n + 0 = S_n ∎$$ - -### 8.2 Optional Stopping - -**Theorem 8.2** (Optional Stopping for bet-until). For `(bet-until pred thunk)` with stopping time τ: - -If E[τ] < ∞ and the bet sum is a martingale, then E[S_τ] = E[S₀] = 0. - ---- - -## 9. Measure-Theoretic Probability - -### 9.1 Lebesgue Integration - -**Definition 9.1** (Expectation as Lebesgue Integral). For bet random variable X: - -$$\mathbb{E}[X] = \int_\Omega X(\omega) \, dP(\omega)$$ - -For discrete ternary space, this reduces to the sum: - -$$\mathbb{E}[X] = \sum_{\omega \in \Omega} X(\omega) P(\{\omega\})$$ - -### 9.2 Radon-Nikodym Derivative - -**Theorem 9.1** (Density for Weighted Bets). Given uniform measure Q and weighted measure P: - -$$\frac{dP}{dQ}(\omega) = \frac{3w_\omega}{\sum w}$$ - -This is the likelihood ratio used in importance sampling. - ---- - -## 10. Category-Theoretic Foundations - -### 10.1 The Giry Monad - -**Definition 10.1** (Connection to Giry Monad). Betlang's probability monad is a discrete instantiation of the Giry monad on the category of measurable spaces. - -The Giry monad G: **Meas** → **Meas** maps: -- Objects: Measurable space (X, Σ) ↦ (G(X), Σ_G) -- Morphisms: Measurable f ↦ G(f) via pushforward - -**Theorem 10.1**. The discrete Bet monad is the restriction of the Giry monad to finite discrete spaces. - -### 10.2 Lawvere Theory - -**Definition 10.2** (Ternary Lawvere Theory). The algebraic theory of betlang is generated by: -- One ternary operation: bet(-, -, -) -- Subject to idempotency: bet(x, x, x) = x -- Symmetry under outcome permutation (distributionally) - ---- - -## Appendix A: Notation Summary - -| Symbol | Meaning | -|--------|---------| -| Ω | Sample space {A, B, C} | -| F | σ-algebra (power set) | -| P | Probability measure | -| E[X] | Expected value | -| Var(X) | Variance | -| H(X) | Shannon entropy | -| Bet(A) | Probability monad over type A | -| T₃ | Ternary truth domain {0, ½, 1} | - -## Appendix B: Key Results Summary - -1. Uniform bet achieves maximum entropy log₂(3) ≈ 1.585 bits -2. Bet monad satisfies all three monad laws -3. LLN and CLT apply to bet sequences -4. Ternary logic satisfies De Morgan's laws -5. Weighted bets define valid probability measures iff weights are non-negative - ---- - -## References - -1. Kolmogorov, A.N. (1933). *Foundations of the Theory of Probability* -2. Kleene, S.C. (1952). *Introduction to Metamathematics* -3. Giry, M. (1982). "A categorical approach to probability theory" -4. Lawvere, F.W. (1963). "Functorial Semantics of Algebraic Theories" -5. Shannon, C.E. (1948). "A Mathematical Theory of Communication" diff --git a/proofs/papers/betlang-whitepaper.adoc b/proofs/papers/betlang-whitepaper.adoc new file mode 100644 index 0000000..a3ee24f --- /dev/null +++ b/proofs/papers/betlang-whitepaper.adoc @@ -0,0 +1,461 @@ +== Betlang: A Ternary Probabilistic Programming Language + +=== White Paper v1.0 + +==== Authors + +Betlang Development Team + +==== Abstract + +We present *betlang*, a domain-specific language for probabilistic +modeling and symbolic wagers built on a novel ternary primitive. Unlike +binary probabilistic choice, betlang’s fundamental `+(bet A B C)+` +operation selects uniformly among three alternatives, drawing +inspiration from musical ternary form (A–B–A). This paper formalizes the +language semantics, establishes its mathematical foundations, proves key +correctness properties, and demonstrates its expressiveness for +statistical computation, Bayesian inference, and stochastic simulation. +We show that the ternary primitive offers both theoretical elegance and +practical utility, enabling natural expression of three-way decisions +common in real-world probabilistic modeling. + +''''' + +=== 1. Introduction + +==== 1.1 Motivation + +Probabilistic programming has emerged as a powerful paradigm for +expressing and reasoning about uncertainty. Existing languages typically +provide binary choice (coin flip) or continuous sampling as primitives. +We observe that many real-world decisions naturally involve three +alternatives: + +* *Market conditions*: Bull, Bear, Sideways +* *Classification outcomes*: Positive, Negative, Neutral +* *Game theory*: Cooperate, Defect, Abstain +* *Quantum states*: |0⟩, |1⟩, |+⟩ in certain bases +* *Musical structure*: Exposition, Development, Recapitulation + +==== 1.2 Contributions + +[arabic] +. *Novel primitive*: The ternary bet `+(bet A B C)+` as the fundamental +probabilistic operation +. *Complete semantics*: Operational, denotational, and axiomatic +semantics +. *Type-theoretic foundation*: Probability monad with proven monad laws +. *Rich ecosystem*: Libraries for distributions, inference, Markov +chains, and optimization +. *Correctness proofs*: Soundness, completeness, and convergence +theorems + +==== 1.3 Design Principles + +[arabic] +. *Ternary philosophy*: All operations fundamentally involve three +choices +. *Functional purity*: Randomness is explicit, side effects controlled +. *Compositional*: Monadic structure enables modular probabilistic +programs +. *Practical*: Comprehensive library for real statistical computation + +''''' + +=== 2. Language Overview + +==== 2.1 Core Syntax + +.... +e ::= (bet e₁ e₂ e₃) ; uniform ternary choice + | (bet/weighted p₁ p₂ p₃) ; weighted ternary choice + | (bet/conditional pred A B C) ; conditional choice + | (bet/lazy t₁ t₂ t₃) ; lazy evaluation + | (bet-chain n f init) ; iterative application + | (bet-compose f g h) ; function choice + | (λ (x) e) ; abstraction + | (e₁ e₂) ; application + | v ; values +.... + +==== 2.2 Example Programs + +*Basic probability estimation*: + +[source,racket] +---- +(define (estimate-heads-probability n) + (bet-probability n + (λ (x) (equal? x 'heads)) + 'heads 'tails 'edge)) +---- + +*Monte Carlo integration*: + +[source,racket] +---- +(define (integrate f a b n) + (let ([samples (bet-repeat n + (λ () (+ a (* (random) (- b a)))))]) + (* (- b a) (mean (map f samples))))) +---- + +*Bayesian inference*: + +[source,racket] +---- +(define posterior + (conjugate-beta-binomial + 1 1 ; uniform prior + 7 3)) ; 7 successes, 3 failures +---- + +''''' + +=== 3. Formal Semantics + +==== 3.1 Probability Space + +*Definition 3.1* (Ternary Sample Space). For `+(bet A B C)+`: - Ω = \{A, +B, C} - F = P(Ω) (power set) - P(\{x}) = 1/3 for uniform bet + +==== 3.2 Denotational Semantics + +*Definition 3.2* (Semantic Function). ⟦·⟧ : Expr → Env → Dist(D) + +[latexmath] +++++ +⟦\text{bet } e_1 \ e_2 \ e_3⟧ρ = \frac{1}{3}⟦e_1⟧ρ + \frac{1}{3}⟦e_2⟧ρ + \frac{1}{3}⟦e_3⟧ρ +++++ + +==== 3.3 Operational Semantics + +*Rule E-BET*: + +[latexmath] +++++ +\frac{r = \text{random}() \quad i = \lfloor 3r \rfloor}{⟨\text{bet } v_0 \ v_1 \ v_2, σ, ω⟩ → ⟨v_i, σ, \text{tail}(ω)⟩} +++++ + +==== 3.4 Axiomatic Semantics (Probabilistic Hoare Logic) + +[latexmath] +++++ +\{P\} \ \text{bet } A \ B \ C \ \{Q\}_{p} +++++ + +means: if P holds, Q holds with probability at least p after evaluation. + +''''' + +=== 4. Type Theory + +==== 4.1 Probability Monad + +*Definition 4.1* (Monad Operations). + +[source,racket] +---- +(define (bet-pure x) (bet x x x)) ; return +(define (bet-bind m f) (f (m))) ; bind +---- + +*Theorem 4.1* (Monad Laws). bet-pure and bet-bind satisfy: 1. Left +identity: `+(bet-bind (bet-pure x) f) ≡ (f x)+` 2. Right identity: +`+(bet-bind m bet-pure) ≡ m+` 3. Associativity: +`+(bet-bind (bet-bind m f) g) ≡ (bet-bind m (λ x. bet-bind (f x) g))+` + +==== 4.2 Type System + +*Typing rule for bet*: + +[latexmath] +++++ +\frac{Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ \quad Γ ⊢ e_3 : τ}{Γ ⊢ \text{bet } e_1 \ e_2 \ e_3 : \text{Dist}(τ)} +++++ + +*Theorem 4.2* (Type Soundness). Well-typed betlang programs do not get +stuck. + +''''' + +=== 5. Mathematical Properties + +==== 5.1 Information Theory + +*Theorem 5.1* (Maximum Entropy). Uniform ternary bet achieves maximum +entropy: + +[latexmath] +++++ +H_{\max} = \log_2(3) ≈ 1.585 \text{ bits} +++++ + +This exceeds the 1 bit of binary choice, providing more "`information +content`" per operation. + +==== 5.2 Convergence + +*Theorem 5.2* (Strong Law of Large Numbers). For i.i.d. samples from +bet: + +[latexmath] +++++ +\bar{X}_n \xrightarrow{a.s.} μ = \frac{A + B + C}{3} +++++ + +*Theorem 5.3* (Central Limit Theorem). The standardized mean converges +to normal: + +[latexmath] +++++ +\sqrt{n}\frac{\bar{X}_n - μ}{σ} \xrightarrow{d} N(0, 1) +++++ + +==== 5.3 Computational Properties + +*Theorem 5.4* (BPP Simulation). Betlang can simulate any BPP algorithm +with polynomial overhead. + +''''' + +=== 6. Library Ecosystem + +==== 6.1 Probability Distributions + +[width="100%",cols="40%,60%",options="header",] +|=== +|Category |Distributions +|Discrete |Bernoulli, Binomial, Poisson, Geometric, Categorical, +Multinomial, Zipf + +|Continuous |Normal, Exponential, Gamma, Beta, Chi-square, Student-t, F, +Weibull, Pareto, Cauchy, Laplace + +|Multivariate |Dirichlet + +|Processes |Random walk, Brownian motion, Lévy flight +|=== + +==== 6.2 Statistical Functions + +* Descriptive: mean, median, mode, variance, stddev, percentile +* Inference: confidence intervals, hypothesis tests +* Resampling: bootstrap, jackknife +* Time series: moving average, exponential smoothing + +==== 6.3 Bayesian Inference + +* Conjugate priors: Beta-Binomial, Normal-Normal +* MCMC: Metropolis-Hastings, Gibbs sampling, HMC +* Approximate: Rejection sampling, importance sampling, ABC + +==== 6.4 Markov Chains + +* Construction and simulation +* Stationary distribution estimation +* Hidden Markov Models (Viterbi algorithm) +* Text generation + +==== 6.5 Optimization + +* Simulated annealing +* Genetic algorithms +* Particle swarm optimization +* Cross-entropy method +* Ternary search + +''''' + +=== 7. Case Studies + +==== 7.1 Financial Modeling + +Portfolio allocation under three market regimes: + +[source,racket] +---- +(define (market-scenario) + (bet 'bull 'bear 'sideways)) + +(define (portfolio-return scenario allocation) + (case scenario + [(bull) (* 1.15 allocation)] + [(bear) (* 0.85 allocation)] + [(sideways) (* 1.02 allocation)])) + +(define (simulate-portfolio n allocation) + (mean (bet-repeat n + (λ () (portfolio-return (market-scenario) allocation))))) +---- + +==== 7.2 Game Theory + +Rock-Paper-Scissors with ternary natural encoding: + +[source,racket] +---- +(define (play-rps strategy-a strategy-b) + (let ([move-a (strategy-a)] + [move-b (strategy-b)]) + (match* (move-a move-b) + [('rock 'scissors) 1] + [('scissors 'paper) 1] + [('paper 'rock) 1] + [(_ _) (if (equal? move-a move-b) 0 -1)]))) + +(define (nash-equilibrium) + (bet 'rock 'paper 'scissors)) ; Uniform is Nash equilibrium +---- + +==== 7.3 Probabilistic Data Structures + +Skip list level generation: + +[source,racket] +---- +(define (skip-level) + (let loop ([level 1]) + (if (equal? (bet 'up 'stop 'stop) 'up) ; P(up) = 1/3 + (loop (+ level 1)) + level))) +---- + +''''' + +=== 8. Implementation + +==== 8.1 Architecture + +.... +betlang/ +├── core/betlang.rkt ; Core primitives +├── lib/ +│ ├── ternary.rkt ; Ternary logic +│ ├── distributions.rkt ; Probability distributions +│ ├── statistics.rkt ; Statistical functions +│ ├── bayesian.rkt ; Bayesian inference +│ ├── markov.rkt ; Markov chains +│ └── optimization.rkt ; Optimization algorithms +├── repl/shell.rkt ; Interactive REPL +└── tests/basics.rkt ; Test suite +.... + +==== 8.2 Performance + +Benchmarks on reference hardware (specific to implementation): + +[cols=",,",options="header",] +|=== +|Operation |Time (ms) |Throughput +|Basic bet |0.01 |100K ops/sec +|Normal sample |0.1 |10K samples/sec +|MH step |varies |depends on target +|=== + +==== 8.3 Extensibility + +New distributions and inference methods can be added by: 1. Implementing +the sampling interface 2. Providing PDF/CDF when available 3. +Registering with the library + +''''' + +=== 9. Related Work + +==== 9.1 Probabilistic Programming Languages + +[cols=",,,",options="header",] +|=== +|Language |Primitive |Paradigm |Inference +|Church |`+flip+` (binary) |Functional |Sampling +|Stan |Continuous |Imperative |HMC/NUTS +|Pyro |PyTorch-based |Deep learning |VI/MCMC +|*Betlang* |`+bet+` (ternary) |Functional |Monte Carlo +|=== + +==== 9.2 Distinguishing Features + +[arabic] +. *Ternary primitive*: Unique among PPLs +. *Pure Racket*: Seamless integration with host language +. *Educational focus*: Clear semantics for teaching +. *Lightweight*: No external dependencies + +''''' + +=== 10. Future Work + +==== 10.1 Planned Extensions + +[arabic] +. *Automatic differentiation*: For gradient-based inference +. *Parallel execution*: True concurrent sampling +. *Continuous semantics*: Measure-theoretic foundations for continuous +distributions +. *Type inference*: Hindley-Milner for Dist types +. *Probabilistic effects*: Effect system for tracking randomness + +==== 10.2 Research Directions + +[arabic] +. *Ternary quantum computing*: Connection to qutrit systems +. *Game-theoretic applications*: Three-player games +. *Information geometry*: Ternary simplex structures + +''''' + +=== 11. Conclusion + +Betlang demonstrates that ternary probabilistic choice offers a viable +and elegant foundation for probabilistic programming. The +`+(bet A B C)+` primitive naturally expresses three-way uncertainty +while enabling all standard probabilistic computations through +composition. Our formal development provides: + +* Complete semantics (operational, denotational, axiomatic) +* Proven type soundness and monad laws +* Convergence guarantees for statistical estimators +* Comprehensive verification specifications + +The language is suitable for education, research, and practical +probabilistic modeling where ternary structure is natural. + +''''' + +=== References + +[arabic] +. Goodman, N.D., et al. (2008). "`Church: A language for generative +models.`" +. Carpenter, B., et al. (2017). "`Stan: A probabilistic programming +language.`" +. Bingham, E., et al. (2019). "`Pyro: Deep universal probabilistic +programming.`" +. Kozen, D. (1981). "`Semantics of probabilistic programs.`" +. Ramsey, N., & Pfeffer, A. (2002). "`Stochastic lambda calculus and +monads of probability distributions.`" +. Giry, M. (1982). "`A categorical approach to probability theory.`" + +''''' + +=== Appendix A: Complete API Reference + +See `+docs/api-reference.md+` for complete function signatures. + +=== Appendix B: Proofs + +See `+proofs/+` directory for: - `+mathematical-foundations.md+`: +Measure theory and probability - `+formal-semantics.md+`: Complete +semantic definitions - `+theorems/+`: Type theory, soundness, +convergence proofs - `+verification/+`: Formal specifications + +=== Appendix C: Benchmarks + +See `+benchmarks/performance.rkt+` for reproducible performance +measurements. + +''''' + +_Document version: 1.0_ _Last updated: 2024_ diff --git a/proofs/papers/betlang-whitepaper.md b/proofs/papers/betlang-whitepaper.md deleted file mode 100644 index bfe3dda..0000000 --- a/proofs/papers/betlang-whitepaper.md +++ /dev/null @@ -1,378 +0,0 @@ - -# Betlang: A Ternary Probabilistic Programming Language - -## White Paper v1.0 - -### Authors -Betlang Development Team - -### Abstract - -We present **betlang**, a domain-specific language for probabilistic modeling and symbolic wagers built on a novel ternary primitive. Unlike binary probabilistic choice, betlang's fundamental `(bet A B C)` operation selects uniformly among three alternatives, drawing inspiration from musical ternary form (A–B–A). This paper formalizes the language semantics, establishes its mathematical foundations, proves key correctness properties, and demonstrates its expressiveness for statistical computation, Bayesian inference, and stochastic simulation. We show that the ternary primitive offers both theoretical elegance and practical utility, enabling natural expression of three-way decisions common in real-world probabilistic modeling. - ---- - -## 1. Introduction - -### 1.1 Motivation - -Probabilistic programming has emerged as a powerful paradigm for expressing and reasoning about uncertainty. Existing languages typically provide binary choice (coin flip) or continuous sampling as primitives. We observe that many real-world decisions naturally involve three alternatives: - -- **Market conditions**: Bull, Bear, Sideways -- **Classification outcomes**: Positive, Negative, Neutral -- **Game theory**: Cooperate, Defect, Abstain -- **Quantum states**: |0⟩, |1⟩, |+⟩ in certain bases -- **Musical structure**: Exposition, Development, Recapitulation - -### 1.2 Contributions - -1. **Novel primitive**: The ternary bet `(bet A B C)` as the fundamental probabilistic operation -2. **Complete semantics**: Operational, denotational, and axiomatic semantics -3. **Type-theoretic foundation**: Probability monad with proven monad laws -4. **Rich ecosystem**: Libraries for distributions, inference, Markov chains, and optimization -5. **Correctness proofs**: Soundness, completeness, and convergence theorems - -### 1.3 Design Principles - -1. **Ternary philosophy**: All operations fundamentally involve three choices -2. **Functional purity**: Randomness is explicit, side effects controlled -3. **Compositional**: Monadic structure enables modular probabilistic programs -4. **Practical**: Comprehensive library for real statistical computation - ---- - -## 2. Language Overview - -### 2.1 Core Syntax - -``` -e ::= (bet e₁ e₂ e₃) ; uniform ternary choice - | (bet/weighted p₁ p₂ p₃) ; weighted ternary choice - | (bet/conditional pred A B C) ; conditional choice - | (bet/lazy t₁ t₂ t₃) ; lazy evaluation - | (bet-chain n f init) ; iterative application - | (bet-compose f g h) ; function choice - | (λ (x) e) ; abstraction - | (e₁ e₂) ; application - | v ; values -``` - -### 2.2 Example Programs - -**Basic probability estimation**: -```racket -(define (estimate-heads-probability n) - (bet-probability n - (λ (x) (equal? x 'heads)) - 'heads 'tails 'edge)) -``` - -**Monte Carlo integration**: -```racket -(define (integrate f a b n) - (let ([samples (bet-repeat n - (λ () (+ a (* (random) (- b a)))))]) - (* (- b a) (mean (map f samples))))) -``` - -**Bayesian inference**: -```racket -(define posterior - (conjugate-beta-binomial - 1 1 ; uniform prior - 7 3)) ; 7 successes, 3 failures -``` - ---- - -## 3. Formal Semantics - -### 3.1 Probability Space - -**Definition 3.1** (Ternary Sample Space). For `(bet A B C)`: -- Ω = {A, B, C} -- F = P(Ω) (power set) -- P({x}) = 1/3 for uniform bet - -### 3.2 Denotational Semantics - -**Definition 3.2** (Semantic Function). ⟦·⟧ : Expr → Env → Dist(D) - -$$⟦\text{bet } e_1 \ e_2 \ e_3⟧ρ = \frac{1}{3}⟦e_1⟧ρ + \frac{1}{3}⟦e_2⟧ρ + \frac{1}{3}⟦e_3⟧ρ$$ - -### 3.3 Operational Semantics - -**Rule E-BET**: -$$\frac{r = \text{random}() \quad i = \lfloor 3r \rfloor}{⟨\text{bet } v_0 \ v_1 \ v_2, σ, ω⟩ → ⟨v_i, σ, \text{tail}(ω)⟩}$$ - -### 3.4 Axiomatic Semantics (Probabilistic Hoare Logic) - -$$\{P\} \ \text{bet } A \ B \ C \ \{Q\}_{p}$$ - -means: if P holds, Q holds with probability at least p after evaluation. - ---- - -## 4. Type Theory - -### 4.1 Probability Monad - -**Definition 4.1** (Monad Operations). -```racket -(define (bet-pure x) (bet x x x)) ; return -(define (bet-bind m f) (f (m))) ; bind -``` - -**Theorem 4.1** (Monad Laws). bet-pure and bet-bind satisfy: -1. Left identity: `(bet-bind (bet-pure x) f) ≡ (f x)` -2. Right identity: `(bet-bind m bet-pure) ≡ m` -3. Associativity: `(bet-bind (bet-bind m f) g) ≡ (bet-bind m (λ x. bet-bind (f x) g))` - -### 4.2 Type System - -**Typing rule for bet**: -$$\frac{Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ \quad Γ ⊢ e_3 : τ}{Γ ⊢ \text{bet } e_1 \ e_2 \ e_3 : \text{Dist}(τ)}$$ - -**Theorem 4.2** (Type Soundness). Well-typed betlang programs do not get stuck. - ---- - -## 5. Mathematical Properties - -### 5.1 Information Theory - -**Theorem 5.1** (Maximum Entropy). Uniform ternary bet achieves maximum entropy: -$$H_{\max} = \log_2(3) ≈ 1.585 \text{ bits}$$ - -This exceeds the 1 bit of binary choice, providing more "information content" per operation. - -### 5.2 Convergence - -**Theorem 5.2** (Strong Law of Large Numbers). For i.i.d. samples from bet: -$$\bar{X}_n \xrightarrow{a.s.} μ = \frac{A + B + C}{3}$$ - -**Theorem 5.3** (Central Limit Theorem). The standardized mean converges to normal: -$$\sqrt{n}\frac{\bar{X}_n - μ}{σ} \xrightarrow{d} N(0, 1)$$ - -### 5.3 Computational Properties - -**Theorem 5.4** (BPP Simulation). Betlang can simulate any BPP algorithm with polynomial overhead. - ---- - -## 6. Library Ecosystem - -### 6.1 Probability Distributions - -| Category | Distributions | -|----------|---------------| -| Discrete | Bernoulli, Binomial, Poisson, Geometric, Categorical, Multinomial, Zipf | -| Continuous | Normal, Exponential, Gamma, Beta, Chi-square, Student-t, F, Weibull, Pareto, Cauchy, Laplace | -| Multivariate | Dirichlet | -| Processes | Random walk, Brownian motion, Lévy flight | - -### 6.2 Statistical Functions - -- Descriptive: mean, median, mode, variance, stddev, percentile -- Inference: confidence intervals, hypothesis tests -- Resampling: bootstrap, jackknife -- Time series: moving average, exponential smoothing - -### 6.3 Bayesian Inference - -- Conjugate priors: Beta-Binomial, Normal-Normal -- MCMC: Metropolis-Hastings, Gibbs sampling, HMC -- Approximate: Rejection sampling, importance sampling, ABC - -### 6.4 Markov Chains - -- Construction and simulation -- Stationary distribution estimation -- Hidden Markov Models (Viterbi algorithm) -- Text generation - -### 6.5 Optimization - -- Simulated annealing -- Genetic algorithms -- Particle swarm optimization -- Cross-entropy method -- Ternary search - ---- - -## 7. Case Studies - -### 7.1 Financial Modeling - -Portfolio allocation under three market regimes: - -```racket -(define (market-scenario) - (bet 'bull 'bear 'sideways)) - -(define (portfolio-return scenario allocation) - (case scenario - [(bull) (* 1.15 allocation)] - [(bear) (* 0.85 allocation)] - [(sideways) (* 1.02 allocation)])) - -(define (simulate-portfolio n allocation) - (mean (bet-repeat n - (λ () (portfolio-return (market-scenario) allocation))))) -``` - -### 7.2 Game Theory - -Rock-Paper-Scissors with ternary natural encoding: - -```racket -(define (play-rps strategy-a strategy-b) - (let ([move-a (strategy-a)] - [move-b (strategy-b)]) - (match* (move-a move-b) - [('rock 'scissors) 1] - [('scissors 'paper) 1] - [('paper 'rock) 1] - [(_ _) (if (equal? move-a move-b) 0 -1)]))) - -(define (nash-equilibrium) - (bet 'rock 'paper 'scissors)) ; Uniform is Nash equilibrium -``` - -### 7.3 Probabilistic Data Structures - -Skip list level generation: - -```racket -(define (skip-level) - (let loop ([level 1]) - (if (equal? (bet 'up 'stop 'stop) 'up) ; P(up) = 1/3 - (loop (+ level 1)) - level))) -``` - ---- - -## 8. Implementation - -### 8.1 Architecture - -``` -betlang/ -├── core/betlang.rkt ; Core primitives -├── lib/ -│ ├── ternary.rkt ; Ternary logic -│ ├── distributions.rkt ; Probability distributions -│ ├── statistics.rkt ; Statistical functions -│ ├── bayesian.rkt ; Bayesian inference -│ ├── markov.rkt ; Markov chains -│ └── optimization.rkt ; Optimization algorithms -├── repl/shell.rkt ; Interactive REPL -└── tests/basics.rkt ; Test suite -``` - -### 8.2 Performance - -Benchmarks on reference hardware (specific to implementation): - -| Operation | Time (ms) | Throughput | -|-----------|-----------|------------| -| Basic bet | 0.01 | 100K ops/sec | -| Normal sample | 0.1 | 10K samples/sec | -| MH step | varies | depends on target | - -### 8.3 Extensibility - -New distributions and inference methods can be added by: -1. Implementing the sampling interface -2. Providing PDF/CDF when available -3. Registering with the library - ---- - -## 9. Related Work - -### 9.1 Probabilistic Programming Languages - -| Language | Primitive | Paradigm | Inference | -|----------|-----------|----------|-----------| -| Church | `flip` (binary) | Functional | Sampling | -| Stan | Continuous | Imperative | HMC/NUTS | -| Pyro | PyTorch-based | Deep learning | VI/MCMC | -| **Betlang** | `bet` (ternary) | Functional | Monte Carlo | - -### 9.2 Distinguishing Features - -1. **Ternary primitive**: Unique among PPLs -2. **Pure Racket**: Seamless integration with host language -3. **Educational focus**: Clear semantics for teaching -4. **Lightweight**: No external dependencies - ---- - -## 10. Future Work - -### 10.1 Planned Extensions - -1. **Automatic differentiation**: For gradient-based inference -2. **Parallel execution**: True concurrent sampling -3. **Continuous semantics**: Measure-theoretic foundations for continuous distributions -4. **Type inference**: Hindley-Milner for Dist types -5. **Probabilistic effects**: Effect system for tracking randomness - -### 10.2 Research Directions - -1. **Ternary quantum computing**: Connection to qutrit systems -2. **Game-theoretic applications**: Three-player games -3. **Information geometry**: Ternary simplex structures - ---- - -## 11. Conclusion - -Betlang demonstrates that ternary probabilistic choice offers a viable and elegant foundation for probabilistic programming. The `(bet A B C)` primitive naturally expresses three-way uncertainty while enabling all standard probabilistic computations through composition. Our formal development provides: - -- Complete semantics (operational, denotational, axiomatic) -- Proven type soundness and monad laws -- Convergence guarantees for statistical estimators -- Comprehensive verification specifications - -The language is suitable for education, research, and practical probabilistic modeling where ternary structure is natural. - ---- - -## References - -1. Goodman, N.D., et al. (2008). "Church: A language for generative models." -2. Carpenter, B., et al. (2017). "Stan: A probabilistic programming language." -3. Bingham, E., et al. (2019). "Pyro: Deep universal probabilistic programming." -4. Kozen, D. (1981). "Semantics of probabilistic programs." -5. Ramsey, N., & Pfeffer, A. (2002). "Stochastic lambda calculus and monads of probability distributions." -6. Giry, M. (1982). "A categorical approach to probability theory." - ---- - -## Appendix A: Complete API Reference - -See `docs/api-reference.md` for complete function signatures. - -## Appendix B: Proofs - -See `proofs/` directory for: -- `mathematical-foundations.md`: Measure theory and probability -- `formal-semantics.md`: Complete semantic definitions -- `theorems/`: Type theory, soundness, convergence proofs -- `verification/`: Formal specifications - -## Appendix C: Benchmarks - -See `benchmarks/performance.rkt` for reproducible performance measurements. - ---- - -*Document version: 1.0* -*Last updated: 2024* diff --git a/proofs/papers/betlang-whitepaper.md.invariants.adoc b/proofs/papers/betlang-whitepaper.md.invariants.adoc new file mode 100644 index 0000000..d3159fb --- /dev/null +++ b/proofs/papers/betlang-whitepaper.md.invariants.adoc @@ -0,0 +1,26 @@ +== Invariant Path Scan: betlang-whitepaper.md + +=== Invariant: ip-30723e8b73ccb799 + +⚠️ *ISSUE DETECTED / 🔍 REVIEW REQUIRED* + +*Source Text:* This paper formalizes the language semantics, establishes +its mathematical foundations + +*Target Text:* key correctness properties, and demonstrates its +expressiveness for statistical computation, Bayesian inference, and +stochastic simulation + +*Invariant Type:* provenance + +*Notes:* auto-generated heuristic suggestion; editable + +[width="5%",cols="100%",] +|=== +|## Invariant: ip-26fd2cfd51e56465 +|⚠️ *ISSUE DETECTED / 🔍 REVIEW REQUIRED* +|*Source Text:* - Convergence +|*Target Text:* for statistical estimators +|*Invariant Type:* provenance +|*Notes:* auto-generated heuristic suggestion; editable +|=== diff --git a/proofs/papers/betlang-whitepaper.md.invariants.md b/proofs/papers/betlang-whitepaper.md.invariants.md deleted file mode 100644 index 5ba0e90..0000000 --- a/proofs/papers/betlang-whitepaper.md.invariants.md +++ /dev/null @@ -1,32 +0,0 @@ - -# Invariant Path Scan: betlang-whitepaper.md - -## Invariant: ip-30723e8b73ccb799 - -⚠️ **ISSUE DETECTED / 🔍 REVIEW REQUIRED** - -**Source Text:** This paper formalizes the language semantics, establishes its mathematical foundations - -**Target Text:** key correctness properties, and demonstrates its expressiveness for statistical computation, Bayesian inference, and stochastic simulation - -**Invariant Type:** provenance - -**Notes:** auto-generated heuristic suggestion; editable - ---- -## Invariant: ip-26fd2cfd51e56465 - -⚠️ **ISSUE DETECTED / 🔍 REVIEW REQUIRED** - -**Source Text:** - Convergence - -**Target Text:** for statistical estimators - -**Invariant Type:** provenance - -**Notes:** auto-generated heuristic suggestion; editable - ---- diff --git a/proofs/papers/ppl-comparison.adoc b/proofs/papers/ppl-comparison.adoc new file mode 100644 index 0000000..d1a48a6 --- /dev/null +++ b/proofs/papers/ppl-comparison.adoc @@ -0,0 +1,598 @@ +== Comparative Analysis: Betlang and Probabilistic Programming Languages + +=== Abstract + +This document provides a rigorous comparison of betlang with established +probabilistic programming languages (PPLs) including Church, Anglican, +Stan, Pyro, Edward, Gen, and Turing. We analyze semantic foundations, +expressiveness, inference capabilities, and theoretical properties. + +''''' + +=== 1. Taxonomy of Probabilistic Programming Languages + +==== 1.1 Classification Dimensions + +[cols=",",options="header",] +|=== +|Dimension |Categories +|Paradigm |Functional, Imperative, Declarative +|Typing |Static, Dynamic, Gradual +|Primitive |Binary flip, Continuous sample, Ternary bet +|Inference |Exact, Sampling, Variational +|Host language |Standalone, Embedded +|=== + +==== 1.2 Language Overview + +[cols=",,,,",options="header",] +|=== +|Language |Year |Host |Paradigm |Primary Inference +|BUGS |1989 |Standalone |Declarative |Gibbs +|Church |2008 |Scheme |Functional |MH, Enumeration +|Stan |2012 |Standalone |Imperative |HMC/NUTS +|Anglican |2014 |Clojure |Functional |SMC, MCMC +|Edward |2016 |Python/TF |Functional |VI +|Pyro |2017 |Python/PyTorch |Functional |VI, MCMC +|Gen |2019 |Julia |Functional |Programmable +|Turing |2018 |Julia |Functional |MCMC +|*Betlang* |2024 |Racket |Functional |Monte Carlo +|=== + +''''' + +=== 2. Primitive Operations + +==== 2.1 Comparison of Primitives + +*Church* (Binary): + +[source,scheme] +---- +(flip 0.5) ; Bernoulli(0.5) +(uniform 0 1) ; Uniform continuous +---- + +*Stan* (Continuous): + +[source,stan] +---- +y ~ normal(0, 1); ; Sampling statement +target += normal_lpdf(y | 0, 1); ; Log-prob +---- + +*Pyro* (Tensor-based): + +[source,python] +---- +pyro.sample("x", dist.Normal(0, 1)) +---- + +*Betlang* (Ternary): + +[source,racket] +---- +(bet A B C) ; Uniform ternary +(bet/weighted '(A 1) '(B 2) '(C 1)) ; Weighted +---- + +==== 2.2 Expressiveness Comparison + +*Theorem 2.1* (Inter-reducibility). The following primitives are +inter-reducible: 1. Binary flip ⟺ Ternary bet (with constant overhead) +2. Discrete uniform ⟺ Binary flip (logarithmic overhead) 3. Continuous +uniform ⟺ Binary flip (infinite precision) + +*Proof* (Binary from Ternary): + +[source,racket] +---- +(define (flip p) + (let ([x (bet 0 1 2)]) ; x ∈ {0, 1, 2} + (if (< x (* 3 p)) ; Scale to [0,3) + #t + #f))) +---- + +For p = 0.5: P(x < 1.5) = P(x ∈ \{0, 1}) = 2/3 ≠ 0.5 + +Better construction using rejection: + +[source,racket] +---- +(define (flip-from-bet p) + (let loop () + (let ([x (bet 0 1 2)]) + (cond [(= x 0) #t] + [(= x 1) #f] + [else (loop)])))) ; P(heads) = P(tails) = 0.5 +---- + +*Proof* (Ternary from Binary): + +[source,racket] +---- +(define (bet-from-flip a b c) + (let loop () + (let ([b1 (flip 0.5)] + [b2 (flip 0.5)]) + (cond [(and (not b1) (not b2)) a] ; 00 → A (1/4) + [(and (not b1) b2) b] ; 01 → B (1/4) + [(and b1 (not b2)) c] ; 10 → C (1/4) + [else (loop)])))) ; 11 → reject (1/4) +---- + +Expected flips: 8/3 ≈ 2.67 per ternary sample. ∎ + +==== 2.3 Information Efficiency + +[cols=",,",options="header",] +|=== +|Primitive |Entropy/operation |Random bits +|flip |1 bit |1 +|bet |log₂(3) ≈ 1.585 bits |1.585 +|uniform(0,1) |∞ (continuous) |32-64 +|=== + +*Theorem 2.2* (Entropy Efficiency). Ternary bet is the most efficient +primitive for 3-way decisions. + +For n-way uniform choice: - Using binary flips: ⌈log₂(n)⌉ flips, +efficiency = log₂(n)/⌈log₂(n)⌉ - Using ternary bets: ⌈log₃(n)⌉ bets, +efficiency = log₃(n)/⌈log₃(n)⌉ + +For n = 3: bet efficiency = 1.0, flip efficiency = log₂(3)/2 ≈ 0.79 + +''''' + +=== 3. Semantic Foundations + +==== 3.1 Denotational Semantics Comparison + +*Church/Scheme-based*: + +[latexmath] +++++ +⟦\text{flip } p⟧ = p \cdot δ_{\text{true}} + (1-p) \cdot δ_{\text{false}} +++++ + +*Stan* (Log-density semantics): + +[latexmath] +++++ +⟦y \sim D⟧(σ) = σ[y ↦ v], \text{score} += \log p_D(v) +++++ + +*Betlang*: + +[latexmath] +++++ +⟦\text{bet } A \ B \ C⟧ = \frac{1}{3}δ_A + \frac{1}{3}δ_B + \frac{1}{3}δ_C +++++ + +==== 3.2 Monadic Structure + +[cols=",,,",options="header",] +|=== +|Language |Monad |bind |return +|Church |Probability |stochastic function |deterministic +|Haskell PPLs |Prob a |>>= |return +|Betlang |Dist τ |bet-bind |bet-pure +|=== + +*Theorem 3.1* All listed PPLs satisfy monad laws (with appropriate +interpretation). + +==== 3.3 Operational Semantics + +*Church* (Call-by-value, eager): - Expressions evaluated before random +choice - Memoization through `+mem+` + +*Stan* (Two-phase): - Data block: deterministic - Model block: +stochastic + +*Betlang* (Call-by-value with lazy option): - Default: eager evaluation +- `+bet/lazy+`: only selected branch evaluated + +''''' + +=== 4. Inference Methods + +==== 4.1 Inference Comparison + +[cols=",,,,",options="header",] +|=== +|Method |Church |Stan |Pyro |Betlang +|Exact enumeration |✓ |✗ |✗ |✗ +|Rejection sampling |✓ |✗ |✓ |✓ +|Importance sampling |✓ |✗ |✓ |✓ +|MH |✓ |✓ |✓ |✓ +|HMC/NUTS |✗ |✓ |✓ |✓ (basic) +|SMC |✓ |✗ |✓ |✓ +|Variational |✗ |✓ |✓ |✗ +|ABC |✗ |✗ |✗ |✓ +|=== + +==== 4.2 Inference Correctness + +*Definition 4.1* (Inference Correctness). An inference algorithm is +correct if its output converges to the true posterior. + +*Theorem 4.1* (MH Correctness - All PPLs). Metropolis-Hastings is +correct if: 1. Detailed balance is satisfied 2. Chain is irreducible 3. +Chain is aperiodic + +*Betlang-specific*: The `+metropolis-hastings+` implementation satisfies +these conditions for well-posed target distributions. + +==== 4.3 Convergence Diagnostics + +[cols=",,,",options="header",] +|=== +|Diagnostic |Stan |Pyro |Betlang +|R̂ (Gelman-Rubin) |✓ |✗ |✗ +|ESS |✓ |✓ |✗ +|Trace plots |✓ |✓ |Via analyzer +|ELBO |✓ |✓ |✗ +|=== + +*TODO*: Implement standard convergence diagnostics in Betlang. + +''''' + +=== 5. Type Systems + +==== 5.1 Static vs Dynamic Typing + +[cols=",,",options="header",] +|=== +|Language |Typing |Probabilistic Types +|Church |Dynamic |No +|Stan |Static |Implicit (bounded types) +|Pyro |Dynamic |Tensor shapes +|Gen |Gradual |Trace types +|Betlang |Dynamic |Dist τ (informal) +|=== + +==== 5.2 Type Soundness + +*Stan*: Compile-time type checking prevents runtime errors. + +*Pyro*: Shape errors detected at runtime; optional mypy support. + +*Betlang*: Racket’s contracts provide optional runtime checking. Type +soundness theorem holds for the idealized type system. + +==== 5.3 Dependent Probability Types + +Advanced PPLs explore dependent types: + +*Hakaru*: Dependent types for measure-theoretic semantics + +*Betlang* (potential extension): + +[source,racket] +---- +;; Dependent probability type (future work) +(: bet (∀ (A) (→ A A A (Dist_{1/3} A)))) +---- + +''''' + +=== 6. Expressiveness + +==== 6.1 Turing Completeness + +*Theorem 6.1*. All listed PPLs are Turing complete. + +_Proof sketch_: Each can encode lambda calculus with fixed-point +combinators. ∎ + +==== 6.2 Stochastic Lambda Calculus + +All functional PPLs are instances of stochastic lambda calculus: + +[latexmath] +++++ +\Lambda_\text{prob} ::= x \mid λx.M \mid M N \mid \text{sample}(D) +++++ + +*Betlang addition*: + +[latexmath] +++++ +\Lambda_\text{bet} ::= ... \mid \text{bet}(M_1, M_2, M_3) +++++ + +==== 6.3 Higher-Order Probability + +[cols=",,",options="header",] +|=== +|Language |Higher-order distributions |Distribution-valued returns +|Church |✓ |✓ +|Stan |✗ |✗ +|Pyro |✓ |✓ +|Betlang |✓ |✓ +|=== + +*Example* (Betlang): + +[source,racket] +---- +(define (distribution-of-distributions) + (bet (λ () (normal 0 1)) + (λ () (exponential 1)) + (λ () (uniform 0 1)))) +---- + +''''' + +=== 7. Implementation Comparison + +==== 7.1 Compilation Strategies + +[cols=",,",options="header",] +|=== +|Language |Strategy |Target +|Stan |Transpile |C++ +|Pyro |JIT |PyTorch +|Betlang |Interpret |Racket VM +|=== + +==== 7.2 Performance Characteristics + +[cols=",,,",options="header",] +|=== +|Language |Compilation |Inference |Strengths +|Stan |Slow |Fast |Production HMC +|Pyro |N/A |GPU |Deep learning integration +|Betlang |N/A |Moderate |Simplicity, education +|=== + +==== 7.3 Memory Model + +*Stan*: Static memory allocation based on data block declarations. + +*Pyro*: PyTorch tensor memory management; GPU support. + +*Betlang*: Racket garbage collection; functional data structures. + +''''' + +=== 8. Modeling Capabilities + +==== 8.1 Model Expressiveness + +[cols=",,,,",options="header",] +|=== +|Model Type |Church |Stan |Pyro |Betlang +|Discrete |✓ |✓ |✓ |✓ +|Continuous |✓ |✓ |✓ |✓ +|Hierarchical |✓ |✓ |✓ |✓ +|Non-parametric |✓ |Limited |✓ |Limited +|Time series |✓ |✓ |✓ |✓ +|Neural networks |✗ |✗ |✓ |✗ +|=== + +==== 8.2 Conjugate Prior Support + +[cols=",,",options="header",] +|=== +|Prior-Likelihood |Stan |Betlang +|Beta-Binomial |Auto |Manual (provided) +|Normal-Normal |Auto |Manual (provided) +|Gamma-Poisson |Auto |✗ +|Dirichlet-Categorical |Auto |✗ +|=== + +*TODO*: Implement additional conjugate pairs in Betlang. + +==== 8.3 Mixture Models + +*Stan*: + +[source,stan] +---- +target += log_mix(theta, normal_lpdf(y | mu1, sigma1), + normal_lpdf(y | mu2, sigma2)); +---- + +*Betlang*: + +[source,racket] +---- +(define (mixture-sample) + (let ([component (bet 'comp1 'comp2 'comp3)]) + (case component + [(comp1) (normal 0 1)] + [(comp2) (normal 5 2)] + [(comp3) (exponential 1)]))) +---- + +Natural ternary structure for 3-component mixtures. + +''''' + +=== 9. Theoretical Contributions + +==== 9.1 Unique to Betlang + +[arabic] +. *Ternary primitive*: No other PPL uses ternary as fundamental +. *Musical inspiration*: A-B-A form connection +. *Ternary logic library*: Kleene three-valued logic integration +. *Ternary-optimized algorithms*: ternary search, skip lists with +P(up)=1/3 + +==== 9.2 Contributions from Other PPLs Incorporated + +From *Church*: Stochastic lambda calculus semantics From *Stan*: +Probabilistic Hoare logic approach From *Pyro*: Effect handling concepts +(implicit) + +==== 9.3 Open Questions + +[arabic] +. *Ternary quantum*: Connection to qutrit quantum computing +. *Optimal ternary inference*: Specialized inference for ternary +structure +. *Ternary information geometry*: Geometry of ternary simplex + +''''' + +=== 10. Case Study: Expressing the Same Model + +==== 10.1 Bayesian Linear Regression + +*Stan*: + +[source,stan] +---- +data { + int N; + vector[N] x; + vector[N] y; +} +parameters { + real alpha; + real beta; + real sigma; +} +model { + alpha ~ normal(0, 10); + beta ~ normal(0, 10); + sigma ~ exponential(1); + y ~ normal(alpha + beta * x, sigma); +} +---- + +*Pyro*: + +[source,python] +---- +def model(x, y): + alpha = pyro.sample("alpha", dist.Normal(0, 10)) + beta = pyro.sample("beta", dist.Normal(0, 10)) + sigma = pyro.sample("sigma", dist.Exponential(1)) + with pyro.plate("data", len(x)): + pyro.sample("obs", dist.Normal(alpha + beta * x, sigma), obs=y) +---- + +*Betlang*: + +[source,racket] +---- +(define (linear-regression-model x-data y-data) + (define alpha (normal 0 10)) + (define beta (normal 0 10)) + (define sigma (exponential 1)) + (define (log-likelihood) + (for/sum ([x x-data] [y y-data]) + (normal-log-pdf y (+ alpha (* beta x)) sigma))) + (list alpha beta sigma (log-likelihood))) +---- + +==== 10.2 Mixture Model with Three Components + +*Betlang* (natural): + +[source,racket] +---- +(define (three-component-mixture) + (bet (normal 0 1) + (normal 5 1) + (normal 10 1))) +---- + +*Stan* (requires indexing): + +[source,stan] +---- +simplex[3] theta; +ordered[3] mu; +y ~ normal(mu[categorical_rng(theta)], sigma); +---- + +''''' + +=== 11. Limitations and TODOs + +==== 11.1 Current Betlang Limitations + +[cols=",,",options="header",] +|=== +|Feature |Status |Priority +|Automatic differentiation |Missing |High +|Variational inference |Missing |High +|GPU acceleration |Missing |Medium +|Distributed computing |Missing |Medium +|Advanced diagnostics |Missing |Medium +|Continuous semantics |Incomplete |High +|=== + +==== 11.2 Comparison Summary + +[cols=",,",options="header",] +|=== +|Criterion |Best |Betlang Position +|Production inference |Stan |Developing +|Deep learning |Pyro |Not targeted +|Theoretical elegance |Church |Comparable +|Education |Church |Strong +|Ternary modeling |*Betlang* |Leader +|=== + +''''' + +=== 12. Recommendations + +==== 12.1 When to Use Betlang + +[arabic] +. *Three-way decisions*: Natural ternary structure +. *Education*: Clear, minimal semantics +. *Racket ecosystem*: Integration needed +. *Monte Carlo*: Simple sampling applications +. *Research*: Novel probabilistic semantics + +==== 12.2 When to Use Alternatives + +[arabic] +. *Production HMC*: Stan +. *Deep learning*: Pyro +. *Program synthesis*: Church +. *Flexible inference*: Gen + +''''' + +=== 13. Conclusion + +Betlang occupies a unique position in the PPL landscape with its ternary +primitive. While not intended to replace production systems like Stan or +Pyro, it offers: + +[arabic] +. *Theoretical novelty*: First ternary-based PPL +. *Clean semantics*: Ideal for formal study +. *Natural fit*: Three-way uncertainty modeling + +The comparison reveals that betlang is expressively equivalent to other +PPLs while offering unique structure for ternary problems. + +''''' + +=== References + +[arabic] +. Goodman, N.D., et al. (2008). "`Church: A language for generative +models.`" +. Carpenter, B., et al. (2017). "`Stan: A probabilistic programming +language.`" +. Bingham, E., et al. (2019). "`Pyro: Deep universal probabilistic +programming.`" +. Cusumano-Towner, M.F., et al. (2019). "`Gen: A general-purpose +probabilistic programming system.`" +. Ge, H., Xu, K., & Ghahramani, Z. (2018). "`Turing: A language for +flexible probabilistic inference.`" +. Wood, F., van de Meent, J.W., & Mansinghka, V. (2014). "`A new +approach to probabilistic programming inference.`" diff --git a/proofs/papers/ppl-comparison.md b/proofs/papers/ppl-comparison.md deleted file mode 100644 index c0a17b1..0000000 --- a/proofs/papers/ppl-comparison.md +++ /dev/null @@ -1,501 +0,0 @@ - -# Comparative Analysis: Betlang and Probabilistic Programming Languages - -## Abstract - -This document provides a rigorous comparison of betlang with established probabilistic programming languages (PPLs) including Church, Anglican, Stan, Pyro, Edward, Gen, and Turing. We analyze semantic foundations, expressiveness, inference capabilities, and theoretical properties. - ---- - -## 1. Taxonomy of Probabilistic Programming Languages - -### 1.1 Classification Dimensions - -| Dimension | Categories | -|-----------|------------| -| Paradigm | Functional, Imperative, Declarative | -| Typing | Static, Dynamic, Gradual | -| Primitive | Binary flip, Continuous sample, Ternary bet | -| Inference | Exact, Sampling, Variational | -| Host language | Standalone, Embedded | - -### 1.2 Language Overview - -| Language | Year | Host | Paradigm | Primary Inference | -|----------|------|------|----------|-------------------| -| BUGS | 1989 | Standalone | Declarative | Gibbs | -| Church | 2008 | Scheme | Functional | MH, Enumeration | -| Stan | 2012 | Standalone | Imperative | HMC/NUTS | -| Anglican | 2014 | Clojure | Functional | SMC, MCMC | -| Edward | 2016 | Python/TF | Functional | VI | -| Pyro | 2017 | Python/PyTorch | Functional | VI, MCMC | -| Gen | 2019 | Julia | Functional | Programmable | -| Turing | 2018 | Julia | Functional | MCMC | -| **Betlang** | 2024 | Racket | Functional | Monte Carlo | - ---- - -## 2. Primitive Operations - -### 2.1 Comparison of Primitives - -**Church** (Binary): -```scheme -(flip 0.5) ; Bernoulli(0.5) -(uniform 0 1) ; Uniform continuous -``` - -**Stan** (Continuous): -```stan -y ~ normal(0, 1); ; Sampling statement -target += normal_lpdf(y | 0, 1); ; Log-prob -``` - -**Pyro** (Tensor-based): -```python -pyro.sample("x", dist.Normal(0, 1)) -``` - -**Betlang** (Ternary): -```racket -(bet A B C) ; Uniform ternary -(bet/weighted '(A 1) '(B 2) '(C 1)) ; Weighted -``` - -### 2.2 Expressiveness Comparison - -**Theorem 2.1** (Inter-reducibility). The following primitives are inter-reducible: -1. Binary flip ⟺ Ternary bet (with constant overhead) -2. Discrete uniform ⟺ Binary flip (logarithmic overhead) -3. Continuous uniform ⟺ Binary flip (infinite precision) - -**Proof** (Binary from Ternary): -```racket -(define (flip p) - (let ([x (bet 0 1 2)]) ; x ∈ {0, 1, 2} - (if (< x (* 3 p)) ; Scale to [0,3) - #t - #f))) -``` - -For p = 0.5: P(x < 1.5) = P(x ∈ {0, 1}) = 2/3 ≠ 0.5 - -Better construction using rejection: -```racket -(define (flip-from-bet p) - (let loop () - (let ([x (bet 0 1 2)]) - (cond [(= x 0) #t] - [(= x 1) #f] - [else (loop)])))) ; P(heads) = P(tails) = 0.5 -``` - -**Proof** (Ternary from Binary): -```racket -(define (bet-from-flip a b c) - (let loop () - (let ([b1 (flip 0.5)] - [b2 (flip 0.5)]) - (cond [(and (not b1) (not b2)) a] ; 00 → A (1/4) - [(and (not b1) b2) b] ; 01 → B (1/4) - [(and b1 (not b2)) c] ; 10 → C (1/4) - [else (loop)])))) ; 11 → reject (1/4) -``` - -Expected flips: 8/3 ≈ 2.67 per ternary sample. ∎ - -### 2.3 Information Efficiency - -| Primitive | Entropy/operation | Random bits | -|-----------|-------------------|-------------| -| flip | 1 bit | 1 | -| bet | log₂(3) ≈ 1.585 bits | 1.585 | -| uniform(0,1) | ∞ (continuous) | 32-64 | - -**Theorem 2.2** (Entropy Efficiency). Ternary bet is the most efficient primitive for 3-way decisions. - -For n-way uniform choice: -- Using binary flips: ⌈log₂(n)⌉ flips, efficiency = log₂(n)/⌈log₂(n)⌉ -- Using ternary bets: ⌈log₃(n)⌉ bets, efficiency = log₃(n)/⌈log₃(n)⌉ - -For n = 3: bet efficiency = 1.0, flip efficiency = log₂(3)/2 ≈ 0.79 - ---- - -## 3. Semantic Foundations - -### 3.1 Denotational Semantics Comparison - -**Church/Scheme-based**: -$$⟦\text{flip } p⟧ = p \cdot δ_{\text{true}} + (1-p) \cdot δ_{\text{false}}$$ - -**Stan** (Log-density semantics): -$$⟦y \sim D⟧(σ) = σ[y ↦ v], \text{score} += \log p_D(v)$$ - -**Betlang**: -$$⟦\text{bet } A \ B \ C⟧ = \frac{1}{3}δ_A + \frac{1}{3}δ_B + \frac{1}{3}δ_C$$ - -### 3.2 Monadic Structure - -| Language | Monad | bind | return | -|----------|-------|------|--------| -| Church | Probability | stochastic function | deterministic | -| Haskell PPLs | Prob a | >>= | return | -| Betlang | Dist τ | bet-bind | bet-pure | - -**Theorem 3.1** All listed PPLs satisfy monad laws (with appropriate interpretation). - -### 3.3 Operational Semantics - -**Church** (Call-by-value, eager): -- Expressions evaluated before random choice -- Memoization through `mem` - -**Stan** (Two-phase): -- Data block: deterministic -- Model block: stochastic - -**Betlang** (Call-by-value with lazy option): -- Default: eager evaluation -- `bet/lazy`: only selected branch evaluated - ---- - -## 4. Inference Methods - -### 4.1 Inference Comparison - -| Method | Church | Stan | Pyro | Betlang | -|--------|--------|------|------|---------| -| Exact enumeration | ✓ | ✗ | ✗ | ✗ | -| Rejection sampling | ✓ | ✗ | ✓ | ✓ | -| Importance sampling | ✓ | ✗ | ✓ | ✓ | -| MH | ✓ | ✓ | ✓ | ✓ | -| HMC/NUTS | ✗ | ✓ | ✓ | ✓ (basic) | -| SMC | ✓ | ✗ | ✓ | ✓ | -| Variational | ✗ | ✓ | ✓ | ✗ | -| ABC | ✗ | ✗ | ✗ | ✓ | - -### 4.2 Inference Correctness - -**Definition 4.1** (Inference Correctness). An inference algorithm is correct if its output converges to the true posterior. - -**Theorem 4.1** (MH Correctness - All PPLs). Metropolis-Hastings is correct if: -1. Detailed balance is satisfied -2. Chain is irreducible -3. Chain is aperiodic - -**Betlang-specific**: The `metropolis-hastings` implementation satisfies these conditions for well-posed target distributions. - -### 4.3 Convergence Diagnostics - -| Diagnostic | Stan | Pyro | Betlang | -|------------|------|------|---------| -| R̂ (Gelman-Rubin) | ✓ | ✗ | ✗ | -| ESS | ✓ | ✓ | ✗ | -| Trace plots | ✓ | ✓ | Via analyzer | -| ELBO | ✓ | ✓ | ✗ | - -**TODO**: Implement standard convergence diagnostics in Betlang. - ---- - -## 5. Type Systems - -### 5.1 Static vs Dynamic Typing - -| Language | Typing | Probabilistic Types | -|----------|--------|---------------------| -| Church | Dynamic | No | -| Stan | Static | Implicit (bounded types) | -| Pyro | Dynamic | Tensor shapes | -| Gen | Gradual | Trace types | -| Betlang | Dynamic | Dist τ (informal) | - -### 5.2 Type Soundness - -**Stan**: Compile-time type checking prevents runtime errors. - -**Pyro**: Shape errors detected at runtime; optional mypy support. - -**Betlang**: Racket's contracts provide optional runtime checking. Type soundness theorem holds for the idealized type system. - -### 5.3 Dependent Probability Types - -Advanced PPLs explore dependent types: - -**Hakaru**: Dependent types for measure-theoretic semantics - -**Betlang** (potential extension): -```racket -;; Dependent probability type (future work) -(: bet (∀ (A) (→ A A A (Dist_{1/3} A)))) -``` - ---- - -## 6. Expressiveness - -### 6.1 Turing Completeness - -**Theorem 6.1**. All listed PPLs are Turing complete. - -*Proof sketch*: Each can encode lambda calculus with fixed-point combinators. ∎ - -### 6.2 Stochastic Lambda Calculus - -All functional PPLs are instances of stochastic lambda calculus: - -$$\Lambda_\text{prob} ::= x \mid λx.M \mid M N \mid \text{sample}(D)$$ - -**Betlang addition**: -$$\Lambda_\text{bet} ::= ... \mid \text{bet}(M_1, M_2, M_3)$$ - -### 6.3 Higher-Order Probability - -| Language | Higher-order distributions | Distribution-valued returns | -|----------|---------------------------|----------------------------| -| Church | ✓ | ✓ | -| Stan | ✗ | ✗ | -| Pyro | ✓ | ✓ | -| Betlang | ✓ | ✓ | - -**Example** (Betlang): -```racket -(define (distribution-of-distributions) - (bet (λ () (normal 0 1)) - (λ () (exponential 1)) - (λ () (uniform 0 1)))) -``` - ---- - -## 7. Implementation Comparison - -### 7.1 Compilation Strategies - -| Language | Strategy | Target | -|----------|----------|--------| -| Stan | Transpile | C++ | -| Pyro | JIT | PyTorch | -| Betlang | Interpret | Racket VM | - -### 7.2 Performance Characteristics - -| Language | Compilation | Inference | Strengths | -|----------|-------------|-----------|-----------| -| Stan | Slow | Fast | Production HMC | -| Pyro | N/A | GPU | Deep learning integration | -| Betlang | N/A | Moderate | Simplicity, education | - -### 7.3 Memory Model - -**Stan**: Static memory allocation based on data block declarations. - -**Pyro**: PyTorch tensor memory management; GPU support. - -**Betlang**: Racket garbage collection; functional data structures. - ---- - -## 8. Modeling Capabilities - -### 8.1 Model Expressiveness - -| Model Type | Church | Stan | Pyro | Betlang | -|------------|--------|------|------|---------| -| Discrete | ✓ | ✓ | ✓ | ✓ | -| Continuous | ✓ | ✓ | ✓ | ✓ | -| Hierarchical | ✓ | ✓ | ✓ | ✓ | -| Non-parametric | ✓ | Limited | ✓ | Limited | -| Time series | ✓ | ✓ | ✓ | ✓ | -| Neural networks | ✗ | ✗ | ✓ | ✗ | - -### 8.2 Conjugate Prior Support - -| Prior-Likelihood | Stan | Betlang | -|------------------|------|---------| -| Beta-Binomial | Auto | Manual (provided) | -| Normal-Normal | Auto | Manual (provided) | -| Gamma-Poisson | Auto | ✗ | -| Dirichlet-Categorical | Auto | ✗ | - -**TODO**: Implement additional conjugate pairs in Betlang. - -### 8.3 Mixture Models - -**Stan**: -```stan -target += log_mix(theta, normal_lpdf(y | mu1, sigma1), - normal_lpdf(y | mu2, sigma2)); -``` - -**Betlang**: -```racket -(define (mixture-sample) - (let ([component (bet 'comp1 'comp2 'comp3)]) - (case component - [(comp1) (normal 0 1)] - [(comp2) (normal 5 2)] - [(comp3) (exponential 1)]))) -``` - -Natural ternary structure for 3-component mixtures. - ---- - -## 9. Theoretical Contributions - -### 9.1 Unique to Betlang - -1. **Ternary primitive**: No other PPL uses ternary as fundamental -2. **Musical inspiration**: A-B-A form connection -3. **Ternary logic library**: Kleene three-valued logic integration -4. **Ternary-optimized algorithms**: ternary search, skip lists with P(up)=1/3 - -### 9.2 Contributions from Other PPLs Incorporated - -From **Church**: Stochastic lambda calculus semantics -From **Stan**: Probabilistic Hoare logic approach -From **Pyro**: Effect handling concepts (implicit) - -### 9.3 Open Questions - -1. **Ternary quantum**: Connection to qutrit quantum computing -2. **Optimal ternary inference**: Specialized inference for ternary structure -3. **Ternary information geometry**: Geometry of ternary simplex - ---- - -## 10. Case Study: Expressing the Same Model - -### 10.1 Bayesian Linear Regression - -**Stan**: -```stan -data { - int N; - vector[N] x; - vector[N] y; -} -parameters { - real alpha; - real beta; - real sigma; -} -model { - alpha ~ normal(0, 10); - beta ~ normal(0, 10); - sigma ~ exponential(1); - y ~ normal(alpha + beta * x, sigma); -} -``` - -**Pyro**: -```python -def model(x, y): - alpha = pyro.sample("alpha", dist.Normal(0, 10)) - beta = pyro.sample("beta", dist.Normal(0, 10)) - sigma = pyro.sample("sigma", dist.Exponential(1)) - with pyro.plate("data", len(x)): - pyro.sample("obs", dist.Normal(alpha + beta * x, sigma), obs=y) -``` - -**Betlang**: -```racket -(define (linear-regression-model x-data y-data) - (define alpha (normal 0 10)) - (define beta (normal 0 10)) - (define sigma (exponential 1)) - (define (log-likelihood) - (for/sum ([x x-data] [y y-data]) - (normal-log-pdf y (+ alpha (* beta x)) sigma))) - (list alpha beta sigma (log-likelihood))) -``` - -### 10.2 Mixture Model with Three Components - -**Betlang** (natural): -```racket -(define (three-component-mixture) - (bet (normal 0 1) - (normal 5 1) - (normal 10 1))) -``` - -**Stan** (requires indexing): -```stan -simplex[3] theta; -ordered[3] mu; -y ~ normal(mu[categorical_rng(theta)], sigma); -``` - ---- - -## 11. Limitations and TODOs - -### 11.1 Current Betlang Limitations - -| Feature | Status | Priority | -|---------|--------|----------| -| Automatic differentiation | Missing | High | -| Variational inference | Missing | High | -| GPU acceleration | Missing | Medium | -| Distributed computing | Missing | Medium | -| Advanced diagnostics | Missing | Medium | -| Continuous semantics | Incomplete | High | - -### 11.2 Comparison Summary - -| Criterion | Best | Betlang Position | -|-----------|------|------------------| -| Production inference | Stan | Developing | -| Deep learning | Pyro | Not targeted | -| Theoretical elegance | Church | Comparable | -| Education | Church | Strong | -| Ternary modeling | **Betlang** | Leader | - ---- - -## 12. Recommendations - -### 12.1 When to Use Betlang - -1. **Three-way decisions**: Natural ternary structure -2. **Education**: Clear, minimal semantics -3. **Racket ecosystem**: Integration needed -4. **Monte Carlo**: Simple sampling applications -5. **Research**: Novel probabilistic semantics - -### 12.2 When to Use Alternatives - -1. **Production HMC**: Stan -2. **Deep learning**: Pyro -3. **Program synthesis**: Church -4. **Flexible inference**: Gen - ---- - -## 13. Conclusion - -Betlang occupies a unique position in the PPL landscape with its ternary primitive. While not intended to replace production systems like Stan or Pyro, it offers: - -1. **Theoretical novelty**: First ternary-based PPL -2. **Clean semantics**: Ideal for formal study -3. **Natural fit**: Three-way uncertainty modeling - -The comparison reveals that betlang is expressively equivalent to other PPLs while offering unique structure for ternary problems. - ---- - -## References - -1. Goodman, N.D., et al. (2008). "Church: A language for generative models." -2. Carpenter, B., et al. (2017). "Stan: A probabilistic programming language." -3. Bingham, E., et al. (2019). "Pyro: Deep universal probabilistic programming." -4. Cusumano-Towner, M.F., et al. (2019). "Gen: A general-purpose probabilistic programming system." -5. Ge, H., Xu, K., & Ghahramani, Z. (2018). "Turing: A language for flexible probabilistic inference." -6. Wood, F., van de Meent, J.W., & Mansinghka, V. (2014). "A new approach to probabilistic programming inference." diff --git a/proofs/theorems/category-theory.adoc b/proofs/theorems/category-theory.adoc new file mode 100644 index 0000000..f53a097 --- /dev/null +++ b/proofs/theorems/category-theory.adoc @@ -0,0 +1,596 @@ +== Category-Theoretic Foundations of Betlang + +=== Abstract + +This document develops the category-theoretic foundations of betlang, +formalizing the probability monad, establishing its relationship to the +Giry monad, proving functoriality properties, and developing the Kleisli +category structure that underlies compositional probabilistic +programming. + +''''' + +=== 1. Categorical Preliminaries + +==== 1.1 Category Definition + +*Definition 1.1* (Category). A category C consists of: - Objects: Ob(C) +- Morphisms: for each A, B ∈ Ob(C), a set Hom(A, B) - Composition: ∘ : +Hom(B, C) × Hom(A, B) → Hom(A, C) - Identity: id_A ∈ Hom(A, A) for each +A + +satisfying associativity and identity laws. + +==== 1.2 Relevant Categories + +[cols=",,",options="header",] +|=== +|Category |Objects |Morphisms +|*Set* |Sets |Functions +|*Meas* |Measurable spaces |Measurable functions +|*Prob* |Probability spaces |Measure-preserving maps +|*Bet* |Ternary value spaces |Probabilistic functions +|=== + +''''' + +=== 2. Functors + +==== 2.1 Definition + +*Definition 2.1* (Functor). A functor F: C → D consists of: - Object +mapping: F: Ob(C) → Ob(D) - Morphism mapping: F: Hom_C(A, B) → +Hom_D(F(A), F(B)) + +preserving identity and composition. + +==== 2.2 Dist Functor + +*Definition 2.2* (Distribution Functor). Dist: *Set* → *Set*: - Objects: +Dist(A) = \{probability distributions over A} - Morphisms: For f: A → B, +Dist(f): Dist(A) → Dist(B) is pushforward + +[latexmath] +++++ +(\text{Dist}(f))(μ)(B) = μ(f^{-1}(B)) +++++ + +*Theorem 2.1* (Dist is a Functor). 1. Dist(id_A) = id_\{Dist(A)} 2. +Dist(g ∘ f) = Dist(g) ∘ Dist(f) + +_Proof_: 1. Pushforward by identity is identity on distributions 2. +(Dist(g ∘ f))(μ)(C) = μ((g ∘ f)⁻¹(C)) = μ(f⁻¹(g⁻¹(C))) = +(Dist(g)(Dist(f)(μ)))(C) ∎ + +==== 2.3 bet-map as Functor Action + +[source,racket] +---- +(bet-map f distribution) ≡ Dist(f)(distribution) +---- + +This is the functorial action on morphisms. + +''''' + +=== 3. Monads + +==== 3.1 Definition + +*Definition 3.1* (Monad). A monad on category C is a triple (T, η, μ) +where: - T: C → C is an endofunctor - η: Id → T is a natural +transformation (unit) - μ: T² → T is a natural transformation +(multiplication) + +satisfying: - μ ∘ Tμ = μ ∘ μT (associativity) - μ ∘ ηT = μ ∘ Tη = id +(unit laws) + +==== 3.2 Probability Monad (Giry Monad) + +*Definition 3.2* (Giry Monad). On *Meas*: - G(X) = probability measures +on X with weak topology - η_X: X → G(X) by η_X(x) = δ_x (Dirac measure) +- μ_X: G(G(X)) → G(X) by μ_X(Φ)(A) = ∫ μ(A) dΦ(μ) + +==== 3.3 Betlang Monad + +*Definition 3.3* (Bet Monad). On *Set* (discrete case): - Dist(A) = +finitely-supported probability distributions on A - η_A(a) = δ_a = Dirac +at a - μ_A(Φ) = ∫ μ dΦ = Σ_μ Φ(μ) · μ + +*Theorem 3.1* (Bet Monad Laws). The triple (Dist, η, μ) satisfies monad +laws. + +_Proof of associativity_: + +[latexmath] +++++ +μ ∘ \text{Dist}(μ) = μ ∘ μ_{\text{Dist}} +++++ + +For Φ ∈ Dist(Dist(Dist(A))): + +[latexmath] +++++ +LHS = μ(\text{Dist}(μ)(Φ)) = \sum_μ (\text{Dist}(μ)(Φ))(μ) · μ +++++ + +Both sides yield the same flattening of nested distributions. ∎ + +_Proof of unit laws_: + +[latexmath] +++++ +μ ∘ η_{\text{Dist}} = id \quad \text{and} \quad μ ∘ \text{Dist}(η) = id +++++ + +Inserting Dirac and then flattening returns original distribution. ∎ + +''''' + +=== 4. Kleisli Category + +==== 4.1 Definition + +*Definition 4.1* (Kleisli Category). For monad (T, η, μ) on C, the +Kleisli category C_T has: - Objects: same as C - Morphisms: +Hom_\{C_T}(A, B) = Hom_C(A, T(B)) - Composition: g ∘_K f = μ_C ∘ T(g) ∘ +f - Identity: η_A + +==== 4.2 Kleisli Category for Bet + +*Definition 4.2* (Kleisli Morphisms for Bet). + +[latexmath] +++++ +\text{Hom}_{\text{Bet}}(A, B) = A → \text{Dist}(B) +++++ + +These are "`stochastic functions`" or "`probabilistic arrows.`" + +*Composition*: + +[latexmath] +++++ +(g ∘_K f)(a) = \sum_{b} f(a)(b) \cdot g(b) +++++ + +==== 4.3 bet-bind as Kleisli Composition + +[source,racket] +---- +(bet-bind m f) = (m >>=_K f) +---- + +where >>=_K is Kleisli bind: + +[latexmath] +++++ +(m >>= f) = μ ∘ \text{Dist}(f) ∘ m +++++ + +For sampled m: + +[source,racket] +---- +(bet-bind m f) = (f (sample m)) +---- + +''''' + +=== 5. Natural Transformations + +==== 5.1 Definition + +*Definition 5.1* (Natural Transformation). α: F ⇒ G is a family of +morphisms: + +[latexmath] +++++ +α_A: F(A) → G(A) +++++ + +such that for f: A → B: + +[latexmath] +++++ +α_B ∘ F(f) = G(f) ∘ α_A +++++ + +==== 5.2 Unit as Natural Transformation + +*Theorem 5.1* (η is Natural). For f: A → B: + +[latexmath] +++++ +η_B ∘ f = \text{Dist}(f) ∘ η_A +++++ + +_Proof_: + +[latexmath] +++++ +(η_B ∘ f)(a) = δ_{f(a)} +++++ + +[latexmath] +++++ +(\text{Dist}(f) ∘ η_A)(a) = \text{Dist}(f)(δ_a) = δ_{f(a)} +++++ + +Equal. ∎ + +==== 5.3 Multiplication as Natural Transformation + +*Theorem 5.2* (μ is Natural). For f: A → B: + +[latexmath] +++++ +μ_B ∘ \text{Dist}(\text{Dist}(f)) = \text{Dist}(f) ∘ μ_A +++++ + +''''' + +=== 6. Monoidal Structure + +==== 6.1 Monoidal Category + +*Definition 6.1* (Monoidal Category). A monoidal category (C, ⊗, I) has: +- Bifunctor ⊗: C × C → C - Unit object I - Natural isomorphisms for +associativity and unit + +==== 6.2 Product of Distributions + +*Definition 6.2* (Distribution Product). For μ ∈ Dist(A), ν ∈ Dist(B): + +[latexmath] +++++ +μ ⊗ ν ∈ \text{Dist}(A × B) +++++ + +[latexmath] +++++ +(μ ⊗ ν)(a, b) = μ(a) · ν(b) +++++ + +This is the product measure (independence). + +==== 6.3 Dist as Monoidal Functor + +*Theorem 6.1* (Dist is Lax Monoidal). + +[latexmath] +++++ +\text{Dist}(A) × \text{Dist}(B) → \text{Dist}(A × B) +++++ + +by the product operation. + +''''' + +=== 7. Commutative Monad + +==== 7.1 Definition + +*Definition 7.1* (Commutative Monad). A monad T is commutative if: + +[latexmath] +++++ +\text{strength}: T(A) × B → T(A × B) +++++ + +satisfies commutativity conditions. + +==== 7.2 Commutativity of Bet Monad + +*Theorem 7.1* (Bet Monad is Commutative). The order of sampling doesn’t +affect joint distribution: + +[source,racket] +---- +(bet-bind m₁ (λ (x) (bet-bind m₂ (λ (y) (pair x y))))) +≡ +(bet-bind m₂ (λ (y) (bet-bind m₁ (λ (x) (pair x y))))) +---- + +_Proof_: Both produce the product measure m₁ ⊗ m₂. ∎ + +''''' + +=== 8. Algebras and Eilenberg-Moore Category + +==== 8.1 T-Algebra + +*Definition 8.1* (T-Algebra). For monad T, a T-algebra is (A, h) where: +- A is an object - h: T(A) → A is the structure map + +satisfying: - h ∘ η_A = id_A - h ∘ μ_A = h ∘ T(h) + +==== 8.2 Dist-Algebras + +*Theorem 8.1* (Dist-Algebras are Convex Spaces). A Dist-algebra is +equivalently a convex set: a set with affine combinations. + +For ternary: ability to form weighted averages of three elements. + +==== 8.3 Free Algebra + +*Definition 8.2* (Free Dist-Algebra). The free Dist-algebra on set A is +(Dist(A), μ_A). + +''''' + +=== 9. Adjunctions + +==== 9.1 Definition + +*Definition 9.1* (Adjunction). F: C → D is left adjoint to G: D → C (F ⊣ +G) if: + +[latexmath] +++++ +\text{Hom}_D(F(A), B) ≅ \text{Hom}_C(A, G(B)) +++++ + +naturally in A and B. + +==== 9.2 Monad from Adjunction + +Every monad arises from an adjunction: + +[latexmath] +++++ +T = G ∘ F +++++ + +[latexmath] +++++ +η = \text{unit of adjunction} +++++ + +[latexmath] +++++ +μ = G(ε_F) \text{ where } ε \text{ is counit} +++++ + +==== 9.3 Kleisli Adjunction + +*Theorem 9.1* (Kleisli Adjunction). There is an adjunction: + +[latexmath] +++++ +F_T: C → C_T, \quad G_T: C_T → C +++++ + +where F_T(A) = A and G_T(A) = T(A). + +''''' + +=== 10. Strength and Enrichment + +==== 10.1 Strength + +*Definition 10.1* (Strength). A strength for monad T is: + +[latexmath] +++++ +\text{st}_{A,B}: A × T(B) → T(A × B) +++++ + +natural in A, B, satisfying coherence conditions. + +==== 10.2 Strength for Dist + +[source,racket] +---- +(define (strength a dist-b) + (bet-map (λ (b) (cons a b)) dist-b)) +---- + +This pairs a fixed value with all outcomes. + +==== 10.3 Enrichment in Probability + +The Kleisli category for Dist is enriched over probability spaces: +hom-sets carry probabilistic structure. + +''''' + +=== 11. Lawvere Theory + +==== 11.1 Definition + +*Definition 11.1* (Lawvere Theory). A Lawvere theory L is a category +with: - Objects: natural numbers 0, 1, 2, … - n = 1 + 1 + … + 1 (n +times) - Product: n × m = n + m + +==== 11.2 Theory of Bet + +*Definition 11.2* (Ternary Theory). The Lawvere theory for betlang has: +- One ternary operation: bet: 3 → 1 - Equations: idempotency, symmetry +(up to distribution) + +*Ternary operation*: + +[latexmath] +++++ +\text{bet}: A × A × A → \text{Dist}(A) +++++ + +==== 11.3 Models + +A model of this theory is a set A with: - bet: A³ → Dist(A) - satisfying +idempotency: bet(a, a, a) = δ_a + +''''' + +=== 12. Operad Structure + +==== 12.1 Definition + +*Definition 12.1* (Operad). An operad P consists of: - Sets P(n) for n ≥ +0 (n-ary operations) - Composition: P(n) × P(k₁) × … × P(kₙ) → P(k₁ + … ++ kₙ) - Identity in P(1) + +==== 12.2 Bet Operad + +*Definition 12.2* (Ternary Probability Operad). - P(1) = \{id} - P(3) = +\{bet} ∪ weighted versions - P(n) = compositions of ternary bets + +==== 12.3 Algebra over Operad + +An algebra over this operad is a set with well-defined ternary +probabilistic choice. + +''''' + +=== 13. Categorical Probability + +==== 13.1 Markov Categories + +*Definition 13.1* (Markov Category). A Markov category is a symmetric +monoidal category where: - Each object has a "`copy`" morphism - Copy is +natural w.r.t. deterministic morphisms + +==== 13.2 Betlang as Markov Category + +The Kleisli category for Dist is a Markov category: - Monoidal: via +distribution product - Copying: μ ↦ μ ⊗ μ (independent copies) + +==== 13.3 Conditional Independence + +*Definition 13.2* (Conditional Independence). X ⊥ Y | Z iff: + +[latexmath] +++++ +P(X, Y | Z) = P(X | Z) · P(Y | Z) +++++ + +Categorically: factorization through Z. + +''''' + +=== 14. String Diagrams + +==== 14.1 Graphical Calculus + +String diagrams for the Dist monad: + +.... + A Dist(A) + | | + η | + ↓ | + Dist(A) = | +.... + +.... +Dist(Dist(A)) Dist(A) + | | + μ | + ↓ | + Dist(A) = | +.... + +==== 14.2 Bet Diagram + +.... + A A A + \ | / + \ | / + bet + | + Dist(A) +.... + +Represents ternary mixing. + +''''' + +=== 15. Higher Category Theory + +==== 15.1 2-Category of Monads + +Monads form a 2-category: - 0-cells: Categories - 1-cells: Monads - +2-cells: Monad morphisms + +==== 15.2 Dist in the 2-Category + +The Dist monad is related to other probability monads by: - Embedding: +Dist ↪ Giry (discrete into continuous) - Quotient: Giry → Dist +(discretization) + +==== 15.3 Monad Transformers + +*Definition 15.1* (Monad Transformer). Monad transformer T lifts monads: + +[latexmath] +++++ +T(M) \text{ is a monad when } M \text{ is} +++++ + +DistT: adding probabilistic effects to other monads. + +''''' + +=== 16. Topos-Theoretic Aspects + +==== 16.1 Presheaf Topos + +*Definition 16.1* (Presheaf). A presheaf on C is a functor: + +[latexmath] +++++ +F: C^{op} → \textbf{Set} +++++ + +==== 16.2 Probability Distributions as Presheaves + +Distributions can be viewed as certain presheaves satisfying +normalization. + +==== 16.3 Internal Logic + +The internal logic of a topos with probability monads yields +probabilistic reasoning principles. + +''''' + +=== 17. Summary: Categorical Structure of Betlang + +[cols=",",options="header",] +|=== +|Structure |Betlang Instance +|Functor |Dist: Set → Set +|Monad |(Dist, η, μ) +|Kleisli Category |Stochastic functions +|Monoidal |Product distributions +|Commutative |Order-independent sampling +|Lawvere Theory |Ternary operations +|Markov Category |With copying +|=== + +''''' + +=== 18. TODOs + +*TODO*: Further categorical development: + +[arabic] +. *∞-categorical structure*: Higher homotopy in probabilistic +programming +. *Profunctors*: Probabilistic relations +. *Comonads*: For Bayesian updates +. *Polynomial functors*: For recursive types + +''''' + +=== References + +[arabic] +. Giry, M. (1982). "`A categorical approach to probability theory.`" +. Lawvere, F.W. (1963). "`Functorial Semantics of Algebraic Theories.`" +. Moggi, E. (1991). "`Notions of Computation and Monads.`" +. Fritz, T. (2020). "`A synthetic approach to Markov kernels, +conditional independence and theorems on sufficient statistics.`" +. Perrone, P. (2021). "`Markov categories and entropy.`" +. Jacobs, B. (2019). "`Categorical Probability.`" diff --git a/proofs/theorems/category-theory.md b/proofs/theorems/category-theory.md deleted file mode 100644 index 1342002..0000000 --- a/proofs/theorems/category-theory.md +++ /dev/null @@ -1,483 +0,0 @@ - -# Category-Theoretic Foundations of Betlang - -## Abstract - -This document develops the category-theoretic foundations of betlang, formalizing the probability monad, establishing its relationship to the Giry monad, proving functoriality properties, and developing the Kleisli category structure that underlies compositional probabilistic programming. - ---- - -## 1. Categorical Preliminaries - -### 1.1 Category Definition - -**Definition 1.1** (Category). A category C consists of: -- Objects: Ob(C) -- Morphisms: for each A, B ∈ Ob(C), a set Hom(A, B) -- Composition: ∘ : Hom(B, C) × Hom(A, B) → Hom(A, C) -- Identity: id_A ∈ Hom(A, A) for each A - -satisfying associativity and identity laws. - -### 1.2 Relevant Categories - -| Category | Objects | Morphisms | -|----------|---------|-----------| -| **Set** | Sets | Functions | -| **Meas** | Measurable spaces | Measurable functions | -| **Prob** | Probability spaces | Measure-preserving maps | -| **Bet** | Ternary value spaces | Probabilistic functions | - ---- - -## 2. Functors - -### 2.1 Definition - -**Definition 2.1** (Functor). A functor F: C → D consists of: -- Object mapping: F: Ob(C) → Ob(D) -- Morphism mapping: F: Hom_C(A, B) → Hom_D(F(A), F(B)) - -preserving identity and composition. - -### 2.2 Dist Functor - -**Definition 2.2** (Distribution Functor). Dist: **Set** → **Set**: -- Objects: Dist(A) = {probability distributions over A} -- Morphisms: For f: A → B, Dist(f): Dist(A) → Dist(B) is pushforward - -$$(\text{Dist}(f))(μ)(B) = μ(f^{-1}(B))$$ - -**Theorem 2.1** (Dist is a Functor). -1. Dist(id_A) = id_{Dist(A)} -2. Dist(g ∘ f) = Dist(g) ∘ Dist(f) - -*Proof*: -1. Pushforward by identity is identity on distributions -2. (Dist(g ∘ f))(μ)(C) = μ((g ∘ f)⁻¹(C)) = μ(f⁻¹(g⁻¹(C))) = (Dist(g)(Dist(f)(μ)))(C) ∎ - -### 2.3 bet-map as Functor Action - -```racket -(bet-map f distribution) ≡ Dist(f)(distribution) -``` - -This is the functorial action on morphisms. - ---- - -## 3. Monads - -### 3.1 Definition - -**Definition 3.1** (Monad). A monad on category C is a triple (T, η, μ) where: -- T: C → C is an endofunctor -- η: Id → T is a natural transformation (unit) -- μ: T² → T is a natural transformation (multiplication) - -satisfying: -- μ ∘ Tμ = μ ∘ μT (associativity) -- μ ∘ ηT = μ ∘ Tη = id (unit laws) - -### 3.2 Probability Monad (Giry Monad) - -**Definition 3.2** (Giry Monad). On **Meas**: -- G(X) = probability measures on X with weak topology -- η_X: X → G(X) by η_X(x) = δ_x (Dirac measure) -- μ_X: G(G(X)) → G(X) by μ_X(Φ)(A) = ∫ μ(A) dΦ(μ) - -### 3.3 Betlang Monad - -**Definition 3.3** (Bet Monad). On **Set** (discrete case): -- Dist(A) = finitely-supported probability distributions on A -- η_A(a) = δ_a = Dirac at a -- μ_A(Φ) = ∫ μ dΦ = Σ_μ Φ(μ) · μ - -**Theorem 3.1** (Bet Monad Laws). The triple (Dist, η, μ) satisfies monad laws. - -*Proof of associativity*: -$$μ ∘ \text{Dist}(μ) = μ ∘ μ_{\text{Dist}}$$ - -For Φ ∈ Dist(Dist(Dist(A))): -$$LHS = μ(\text{Dist}(μ)(Φ)) = \sum_μ (\text{Dist}(μ)(Φ))(μ) · μ$$ - -Both sides yield the same flattening of nested distributions. ∎ - -*Proof of unit laws*: -$$μ ∘ η_{\text{Dist}} = id \quad \text{and} \quad μ ∘ \text{Dist}(η) = id$$ - -Inserting Dirac and then flattening returns original distribution. ∎ - ---- - -## 4. Kleisli Category - -### 4.1 Definition - -**Definition 4.1** (Kleisli Category). For monad (T, η, μ) on C, the Kleisli category C_T has: -- Objects: same as C -- Morphisms: Hom_{C_T}(A, B) = Hom_C(A, T(B)) -- Composition: g ∘_K f = μ_C ∘ T(g) ∘ f -- Identity: η_A - -### 4.2 Kleisli Category for Bet - -**Definition 4.2** (Kleisli Morphisms for Bet). -$$\text{Hom}_{\text{Bet}}(A, B) = A → \text{Dist}(B)$$ - -These are "stochastic functions" or "probabilistic arrows." - -**Composition**: -$$(g ∘_K f)(a) = \sum_{b} f(a)(b) \cdot g(b)$$ - -### 4.3 bet-bind as Kleisli Composition - -```racket -(bet-bind m f) = (m >>=_K f) -``` - -where >>=_K is Kleisli bind: -$$(m >>= f) = μ ∘ \text{Dist}(f) ∘ m$$ - -For sampled m: -```racket -(bet-bind m f) = (f (sample m)) -``` - ---- - -## 5. Natural Transformations - -### 5.1 Definition - -**Definition 5.1** (Natural Transformation). α: F ⇒ G is a family of morphisms: -$$α_A: F(A) → G(A)$$ - -such that for f: A → B: -$$α_B ∘ F(f) = G(f) ∘ α_A$$ - -### 5.2 Unit as Natural Transformation - -**Theorem 5.1** (η is Natural). For f: A → B: -$$η_B ∘ f = \text{Dist}(f) ∘ η_A$$ - -*Proof*: -$$(η_B ∘ f)(a) = δ_{f(a)}$$ -$$(\text{Dist}(f) ∘ η_A)(a) = \text{Dist}(f)(δ_a) = δ_{f(a)}$$ - -Equal. ∎ - -### 5.3 Multiplication as Natural Transformation - -**Theorem 5.2** (μ is Natural). For f: A → B: -$$μ_B ∘ \text{Dist}(\text{Dist}(f)) = \text{Dist}(f) ∘ μ_A$$ - ---- - -## 6. Monoidal Structure - -### 6.1 Monoidal Category - -**Definition 6.1** (Monoidal Category). A monoidal category (C, ⊗, I) has: -- Bifunctor ⊗: C × C → C -- Unit object I -- Natural isomorphisms for associativity and unit - -### 6.2 Product of Distributions - -**Definition 6.2** (Distribution Product). For μ ∈ Dist(A), ν ∈ Dist(B): -$$μ ⊗ ν ∈ \text{Dist}(A × B)$$ -$$(μ ⊗ ν)(a, b) = μ(a) · ν(b)$$ - -This is the product measure (independence). - -### 6.3 Dist as Monoidal Functor - -**Theorem 6.1** (Dist is Lax Monoidal). -$$\text{Dist}(A) × \text{Dist}(B) → \text{Dist}(A × B)$$ - -by the product operation. - ---- - -## 7. Commutative Monad - -### 7.1 Definition - -**Definition 7.1** (Commutative Monad). A monad T is commutative if: -$$\text{strength}: T(A) × B → T(A × B)$$ - -satisfies commutativity conditions. - -### 7.2 Commutativity of Bet Monad - -**Theorem 7.1** (Bet Monad is Commutative). The order of sampling doesn't affect joint distribution: - -```racket -(bet-bind m₁ (λ (x) (bet-bind m₂ (λ (y) (pair x y))))) -≡ -(bet-bind m₂ (λ (y) (bet-bind m₁ (λ (x) (pair x y))))) -``` - -*Proof*: Both produce the product measure m₁ ⊗ m₂. ∎ - ---- - -## 8. Algebras and Eilenberg-Moore Category - -### 8.1 T-Algebra - -**Definition 8.1** (T-Algebra). For monad T, a T-algebra is (A, h) where: -- A is an object -- h: T(A) → A is the structure map - -satisfying: -- h ∘ η_A = id_A -- h ∘ μ_A = h ∘ T(h) - -### 8.2 Dist-Algebras - -**Theorem 8.1** (Dist-Algebras are Convex Spaces). A Dist-algebra is equivalently a convex set: a set with affine combinations. - -For ternary: ability to form weighted averages of three elements. - -### 8.3 Free Algebra - -**Definition 8.2** (Free Dist-Algebra). The free Dist-algebra on set A is (Dist(A), μ_A). - ---- - -## 9. Adjunctions - -### 9.1 Definition - -**Definition 9.1** (Adjunction). F: C → D is left adjoint to G: D → C (F ⊣ G) if: -$$\text{Hom}_D(F(A), B) ≅ \text{Hom}_C(A, G(B))$$ - -naturally in A and B. - -### 9.2 Monad from Adjunction - -Every monad arises from an adjunction: -$$T = G ∘ F$$ -$$η = \text{unit of adjunction}$$ -$$μ = G(ε_F) \text{ where } ε \text{ is counit}$$ - -### 9.3 Kleisli Adjunction - -**Theorem 9.1** (Kleisli Adjunction). There is an adjunction: -$$F_T: C → C_T, \quad G_T: C_T → C$$ - -where F_T(A) = A and G_T(A) = T(A). - ---- - -## 10. Strength and Enrichment - -### 10.1 Strength - -**Definition 10.1** (Strength). A strength for monad T is: -$$\text{st}_{A,B}: A × T(B) → T(A × B)$$ - -natural in A, B, satisfying coherence conditions. - -### 10.2 Strength for Dist - -```racket -(define (strength a dist-b) - (bet-map (λ (b) (cons a b)) dist-b)) -``` - -This pairs a fixed value with all outcomes. - -### 10.3 Enrichment in Probability - -The Kleisli category for Dist is enriched over probability spaces: hom-sets carry probabilistic structure. - ---- - -## 11. Lawvere Theory - -### 11.1 Definition - -**Definition 11.1** (Lawvere Theory). A Lawvere theory L is a category with: -- Objects: natural numbers 0, 1, 2, ... -- n = 1 + 1 + ... + 1 (n times) -- Product: n × m = n + m - -### 11.2 Theory of Bet - -**Definition 11.2** (Ternary Theory). The Lawvere theory for betlang has: -- One ternary operation: bet: 3 → 1 -- Equations: idempotency, symmetry (up to distribution) - -**Ternary operation**: -$$\text{bet}: A × A × A → \text{Dist}(A)$$ - -### 11.3 Models - -A model of this theory is a set A with: -- bet: A³ → Dist(A) -- satisfying idempotency: bet(a, a, a) = δ_a - ---- - -## 12. Operad Structure - -### 12.1 Definition - -**Definition 12.1** (Operad). An operad P consists of: -- Sets P(n) for n ≥ 0 (n-ary operations) -- Composition: P(n) × P(k₁) × ... × P(kₙ) → P(k₁ + ... + kₙ) -- Identity in P(1) - -### 12.2 Bet Operad - -**Definition 12.2** (Ternary Probability Operad). -- P(1) = {id} -- P(3) = {bet} ∪ weighted versions -- P(n) = compositions of ternary bets - -### 12.3 Algebra over Operad - -An algebra over this operad is a set with well-defined ternary probabilistic choice. - ---- - -## 13. Categorical Probability - -### 13.1 Markov Categories - -**Definition 13.1** (Markov Category). A Markov category is a symmetric monoidal category where: -- Each object has a "copy" morphism -- Copy is natural w.r.t. deterministic morphisms - -### 13.2 Betlang as Markov Category - -The Kleisli category for Dist is a Markov category: -- Monoidal: via distribution product -- Copying: μ ↦ μ ⊗ μ (independent copies) - -### 13.3 Conditional Independence - -**Definition 13.2** (Conditional Independence). X ⊥ Y | Z iff: -$$P(X, Y | Z) = P(X | Z) · P(Y | Z)$$ - -Categorically: factorization through Z. - ---- - -## 14. String Diagrams - -### 14.1 Graphical Calculus - -String diagrams for the Dist monad: - -``` - A Dist(A) - | | - η | - ↓ | - Dist(A) = | -``` - -``` -Dist(Dist(A)) Dist(A) - | | - μ | - ↓ | - Dist(A) = | -``` - -### 14.2 Bet Diagram - -``` - A A A - \ | / - \ | / - bet - | - Dist(A) -``` - -Represents ternary mixing. - ---- - -## 15. Higher Category Theory - -### 15.1 2-Category of Monads - -Monads form a 2-category: -- 0-cells: Categories -- 1-cells: Monads -- 2-cells: Monad morphisms - -### 15.2 Dist in the 2-Category - -The Dist monad is related to other probability monads by: -- Embedding: Dist ↪ Giry (discrete into continuous) -- Quotient: Giry → Dist (discretization) - -### 15.3 Monad Transformers - -**Definition 15.1** (Monad Transformer). Monad transformer T lifts monads: -$$T(M) \text{ is a monad when } M \text{ is}$$ - -DistT: adding probabilistic effects to other monads. - ---- - -## 16. Topos-Theoretic Aspects - -### 16.1 Presheaf Topos - -**Definition 16.1** (Presheaf). A presheaf on C is a functor: -$$F: C^{op} → \textbf{Set}$$ - -### 16.2 Probability Distributions as Presheaves - -Distributions can be viewed as certain presheaves satisfying normalization. - -### 16.3 Internal Logic - -The internal logic of a topos with probability monads yields probabilistic reasoning principles. - ---- - -## 17. Summary: Categorical Structure of Betlang - -| Structure | Betlang Instance | -|-----------|------------------| -| Functor | Dist: Set → Set | -| Monad | (Dist, η, μ) | -| Kleisli Category | Stochastic functions | -| Monoidal | Product distributions | -| Commutative | Order-independent sampling | -| Lawvere Theory | Ternary operations | -| Markov Category | With copying | - ---- - -## 18. TODOs - -**TODO**: Further categorical development: - -1. **∞-categorical structure**: Higher homotopy in probabilistic programming -2. **Profunctors**: Probabilistic relations -3. **Comonads**: For Bayesian updates -4. **Polynomial functors**: For recursive types - ---- - -## References - -1. Giry, M. (1982). "A categorical approach to probability theory." -2. Lawvere, F.W. (1963). "Functorial Semantics of Algebraic Theories." -3. Moggi, E. (1991). "Notions of Computation and Monads." -4. Fritz, T. (2020). "A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics." -5. Perrone, P. (2021). "Markov categories and entropy." -6. Jacobs, B. (2019). "Categorical Probability." diff --git a/proofs/theorems/convergence-statistics.adoc b/proofs/theorems/convergence-statistics.adoc new file mode 100644 index 0000000..8f837c1 --- /dev/null +++ b/proofs/theorems/convergence-statistics.adoc @@ -0,0 +1,660 @@ +== Convergence and Statistical Proofs for Betlang + +=== Abstract + +This document provides rigorous proofs for the statistical properties of +betlang, including limit theorems, convergence rates, error bounds, and +the theoretical foundations of the statistical inference library. + +''''' + +=== 1. Law of Large Numbers + +==== 1.1 Weak Law of Large Numbers + +*Theorem 1.1* (WLLN for Ternary Bets). Let X₁, X₂, …, Xₙ be i.i.d. +samples from (bet A B C) where A, B, C ∈ ℝ. Then: + +[latexmath] +++++ +\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i \xrightarrow{P} \mu = \frac{A + B + C}{3} +++++ + +_Proof_: By Chebyshev’s inequality: + +[latexmath] +++++ +P(|\bar{X}_n - \mu| \geq \epsilon) \leq \frac{\text{Var}(\bar{X}_n)}{\epsilon^2} = \frac{\sigma^2}{n\epsilon^2} +++++ + +where σ² = Var(X₁) = (A² + B² + C²)/3 - μ² < ∞. + +As n → ∞, this bound → 0, so latexmath:[\bar{X}_n \xrightarrow{P} \mu]. +∎ + +==== 1.2 Strong Law of Large Numbers + +*Theorem 1.2* (SLLN for Ternary Bets). Under the same conditions: + +[latexmath] +++++ +\bar{X}_n \xrightarrow{a.s.} \mu +++++ + +_Proof_: Since E[X₁⁴] < ∞ (finite fourth moment for bounded discrete +distribution), Kolmogorov’s SLLN applies directly. ∎ + +==== 1.3 Weighted Bet Extension + +*Theorem 1.3* (SLLN for Weighted Bets). For weighted bet with +probabilities (p_A, p_B, p_C): + +[latexmath] +++++ +\bar{X}_n \xrightarrow{a.s.} p_A \cdot A + p_B \cdot B + p_C \cdot C +++++ + +_Proof_: Same as Theorem 1.2 with modified expectation. ∎ + +''''' + +=== 2. Central Limit Theorem + +==== 2.1 Basic CLT + +*Theorem 2.1* (CLT for Ternary Bets). For i.i.d. samples from (bet A B +C): + +[latexmath] +++++ +\sqrt{n}\frac{\bar{X}_n - \mu}{\sigma} \xrightarrow{d} N(0, 1) +++++ + +where μ = (A+B+C)/3 and σ² = E[X²] - μ². + +_Proof_: The moment generating function of Xᵢ exists and is finite +(bounded support). By Lévy’s CLT, the standardized sum converges to +standard normal. ∎ + +==== 2.2 Berry-Esseen Bound + +*Theorem 2.2* (Convergence Rate). The CLT convergence rate is: + +[latexmath] +++++ +\sup_x |P(\sqrt{n}(\bar{X}_n - \mu)/\sigma \leq x) - \Phi(x)| \leq \frac{C \cdot \rho}{\sigma^3 \sqrt{n}} +++++ + +where ρ = E[|X - μ|³] and C ≤ 0.4748 (Berry-Esseen constant). + +_Proof_: For ternary bets, the third absolute moment is bounded: + +[latexmath] +++++ +\rho = E[|X - \mu|^3] \leq \max(|A-\mu|, |B-\mu|, |C-\mu|)^3 < \infty +++++ + +The Berry-Esseen theorem applies directly. ∎ + +*Corollary 2.1* (Practical Bound). For uniform (bet A B C) with A, B, C +∈ [-1, 1]: + +[latexmath] +++++ +\sup_x |F_n(x) - \Phi(x)| \leq \frac{0.95}{\sqrt{n}} +++++ + +This means n ≥ 1000 gives ≤ 3% approximation error. + +==== 2.3 Multivariate CLT + +*Theorem 2.3* (Multivariate CLT). For d-dimensional bet outcomes: + +[latexmath] +++++ +\sqrt{n}(\bar{\mathbf{X}}_n - \boldsymbol{\mu}) \xrightarrow{d} N_d(\mathbf{0}, \boldsymbol{\Sigma}) +++++ + +where Σ is the covariance matrix. + +''''' + +=== 3. Probability Estimation Convergence + +==== 3.1 Frequency Estimator + +*Theorem 3.1* (Frequency Convergence). The `+bet-probability+` +estimator: + +[latexmath] +++++ +\hat{p}_n = \frac{1}{n}\sum_{i=1}^{n} \mathbf{1}[\text{pred}(X_i)] +++++ + +converges at rate O(1/√n). + +_Proof_: + +[latexmath] +++++ +\text{MSE}(\hat{p}_n) = \text{Var}(\hat{p}_n) = \frac{p(1-p)}{n} +++++ + +Standard error: SE = √(p(1-p)/n) = O(1/√n). ∎ + +==== 3.2 Confidence Interval Width + +*Theorem 3.2* (CI Width). The 95% confidence interval width is: + +[latexmath] +++++ +W_n = 2 \cdot 1.96 \cdot \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} = O\left(\frac{1}{\sqrt{n}}\right) +++++ + +*Corollary 3.1* (Sample Size for Precision). To achieve margin of error +ε: + +[latexmath] +++++ +n \geq \frac{z_{\alpha/2}^2 \cdot p(1-p)}{\epsilon^2} +++++ + +For ε = 0.01 and 95% confidence: n ≥ 9604 (using p = 0.5 +conservatively). + +==== 3.3 Hoeffding Bound + +*Theorem 3.3* (Finite-Sample Bound). For any ε > 0: + +[latexmath] +++++ +P(|\hat{p}_n - p| \geq \epsilon) \leq 2\exp(-2n\epsilon^2) +++++ + +_Proof_: Direct application of Hoeffding’s inequality for bounded random +variables in [0,1]. ∎ + +*Corollary 3.2* (Sample Size for High Confidence). For P(error > ε) ≤ δ: + +[latexmath] +++++ +n \geq \frac{\ln(2/\delta)}{2\epsilon^2} +++++ + +''''' + +=== 4. Entropy Estimation + +==== 4.1 Plugin Estimator Bias + +*Theorem 4.1* (Entropy Estimator Bias). The plugin entropy estimator: + +[latexmath] +++++ +\hat{H} = -\sum_{x} \hat{p}(x) \log_2 \hat{p}(x) +++++ + +has negative bias: + +[latexmath] +++++ +\mathbb{E}[\hat{H}] = H - \frac{|S| - 1}{2n\ln 2} + O(n^{-2}) +++++ + +where |S| is the support size and H is the true entropy. + +_Proof_: By Taylor expansion of -p log p around the true probability. ∎ + +*Corollary 4.1* (Bias for Ternary Bet). For uniform ternary bet (|S| = +3): + +[latexmath] +++++ +\text{Bias} \approx -\frac{1}{n\ln 2} \approx -\frac{1.44}{n} +++++ + +==== 4.2 Miller-Madow Correction + +*Theorem 4.2* (Bias-Corrected Entropy). The Miller-Madow estimator: + +[latexmath] +++++ +\hat{H}_{MM} = \hat{H} + \frac{|S| - 1}{2n\ln 2} +++++ + +is approximately unbiased to O(n⁻²). + +==== 4.3 Entropy Convergence Rate + +*Theorem 4.3* (Entropy RMSE). The RMSE of the plugin estimator is: + +[latexmath] +++++ +\text{RMSE}(\hat{H}) = O\left(\frac{1}{\sqrt{n}}\right) +++++ + +_Proof_: Using the delta method on the entropy functional. ∎ + +''''' + +=== 5. Distribution Convergence + +==== 5.1 Empirical Distribution + +*Theorem 5.1* (Glivenko-Cantelli). The empirical CDF converges +uniformly: + +[latexmath] +++++ +\sup_x |\hat{F}_n(x) - F(x)| \xrightarrow{a.s.} 0 +++++ + +For ternary bets (discrete), this is: + +[latexmath] +++++ +\max_{x \in \{A,B,C\}} |\hat{p}(x) - p(x)| \xrightarrow{a.s.} 0 +++++ + +==== 5.2 DKW Inequality + +*Theorem 5.2* (Finite-Sample CDF Bound). + +[latexmath] +++++ +P\left(\sup_x |\hat{F}_n(x) - F(x)| > \epsilon\right) \leq 2e^{-2n\epsilon^2} +++++ + +_Proof_: Dvoretzky-Kiefer-Wolfowitz inequality. ∎ + +==== 5.3 Kolmogorov-Smirnov Test Validity + +*Theorem 5.3* (KS Test Asymptotics). For the KS statistic D_n: + +[latexmath] +++++ +\sqrt{n} D_n \xrightarrow{d} K +++++ + +where K is the Kolmogorov distribution. + +The `+kolmogorov-smirnov+` function uses this for goodness-of-fit +testing. + +''''' + +=== 6. Markov Chain Convergence + +==== 6.1 Ergodic Theorem + +*Theorem 6.1* (MCMC Ergodicity). For an irreducible, aperiodic Markov +chain with stationary distribution π: + +[latexmath] +++++ +\frac{1}{n}\sum_{i=1}^{n} f(X_i) \xrightarrow{a.s.} \mathbb{E}_\pi[f] +++++ + +for any f with E_π[|f|] < ∞. + +_Proof_: By the Markov chain ergodic theorem. ∎ + +==== 6.2 Mixing Time + +*Definition 6.1* (Mixing Time). The ε-mixing time is: + +[latexmath] +++++ +t_{mix}(\epsilon) = \min\{t : \max_x \|P^t(x, \cdot) - \pi\|_{TV} \leq \epsilon\} +++++ + +*Theorem 6.2* (Mixing Time Bound). For the Metropolis-Hastings chain: + +[latexmath] +++++ +t_{mix}(\epsilon) \leq C \cdot \frac{\ln(1/(\epsilon \cdot \pi_{min}))}{\text{gap}} +++++ + +where gap is the spectral gap of the transition matrix. + +==== 6.3 Geometric Ergodicity + +*Theorem 6.3* (Geometric Convergence). If the MH chain is geometrically +ergodic: + +[latexmath] +++++ +\|P^n(x, \cdot) - \pi\|_{TV} \leq M(x) \cdot \rho^n +++++ + +for some ρ < 1 and function M. + +*Corollary 6.1* (Burn-in Recommendation). Use burn-in of: + +[latexmath] +++++ +B \geq \frac{\ln(M \cdot \epsilon)}{\ln(1/\rho)} +++++ + +samples to achieve TV distance ≤ ε. + +''''' + +=== 7. Monte Carlo Integration + +==== 7.1 Consistency + +*Theorem 7.1* (MC Integration Consistency). For Monte Carlo estimate: + +[latexmath] +++++ +\hat{I}_n = \frac{1}{n}\sum_{i=1}^{n} f(X_i) +++++ + +where Xᵢ ~ p, we have: + +[latexmath] +++++ +\hat{I}_n \xrightarrow{a.s.} \int f(x) p(x) dx = \mathbb{E}_p[f] +++++ + +_Proof_: SLLN for i.i.d. samples. ∎ + +==== 7.2 Variance Reduction + +*Theorem 7.2* (Importance Sampling Variance). With importance +distribution q: + +[latexmath] +++++ +\hat{I}_{IS} = \frac{1}{n}\sum_{i=1}^{n} \frac{f(X_i) p(X_i)}{q(X_i)}, \quad X_i \sim q +++++ + +Variance: + +[latexmath] +++++ +\text{Var}(\hat{I}_{IS}) = \frac{1}{n}\left[\int \frac{f(x)^2 p(x)^2}{q(x)} dx - I^2\right] +++++ + +Optimal q* ∝ |f(x)| p(x). + +==== 7.3 Antithetic Variates + +*Theorem 7.3* (Antithetic Variance Reduction). For antithetic pairs (X, +X’): + +[latexmath] +++++ +\text{Var}\left(\frac{f(X) + f(X')}{2}\right) = \frac{\text{Var}(f(X))}{2}(1 + \text{Corr}(f(X), f(X'))) +++++ + +If Corr < 0 (negatively correlated), variance is reduced. + +*Corollary 7.1* (Ternary Antithetic). For symmetric bet A, 0, -A with +f(x) = x: + +[latexmath] +++++ +\text{Corr}(f(A), f(-A)) = -1 +++++ + +giving maximum variance reduction. + +''''' + +=== 8. Bootstrap Convergence + +==== 8.1 Bootstrap Consistency + +*Theorem 8.1* (Bootstrap Validity). For the empirical bootstrap: + +[latexmath] +++++ +\hat{F}^*_n = \frac{1}{n}\sum_{i=1}^{n} \delta_{X_i} +++++ + +the bootstrap distribution of √n(θ̂* - θ̂) converges to the true sampling +distribution of √n(θ̂ - θ) for smooth functionals θ. + +_Proof_: Efron’s theorem on bootstrap consistency. ∎ + +==== 8.2 Bootstrap Confidence Intervals + +*Theorem 8.2* (Percentile Bootstrap). The percentile bootstrap interval: + +[latexmath] +++++ +[\hat{\theta}^*_{(\alpha/2)}, \hat{\theta}^*_{(1-\alpha/2)}] +++++ + +has asymptotically correct coverage for symmetric distributions. + +==== 8.3 Bootstrap Variance Estimation + +*Theorem 8.3* (Bootstrap Variance Consistency). + +[latexmath] +++++ +\hat{\text{Var}}_{boot}(\hat{\theta}) \xrightarrow{P} \text{Var}(\hat{\theta}) +++++ + +The `+bootstrap+` function in lib/statistics.rkt provides consistent +variance estimates. + +''''' + +=== 9. Bayesian Convergence + +==== 9.1 Posterior Consistency + +*Theorem 9.1* (Bernstein-von Mises). Under regularity conditions, the +posterior converges to a normal distribution: + +[latexmath] +++++ +\Pi_n(\cdot | X_1, ..., X_n) \xrightarrow{d} N(\hat{\theta}_{MLE}, I(\theta_0)^{-1}/n) +++++ + +where I(θ₀) is the Fisher information. + +==== 9.2 Bayesian Consistency + +*Theorem 9.2* (Doob’s Consistency). For "`most`" priors (in a +measure-theoretic sense), the posterior concentrates around the true +parameter: + +[latexmath] +++++ +\Pi_n(|\theta - \theta_0| > \epsilon | \text{data}) \xrightarrow{a.s.} 0 +++++ + +==== 9.3 Credible Interval Coverage + +*Theorem 9.3* (Frequentist Coverage of Bayesian CIs). For regular +problems: + +[latexmath] +++++ +P_{\theta_0}(\theta_0 \in CI_{1-\alpha}^{Bayes}) \to 1 - \alpha +++++ + +as n → ∞. The Bayesian credible interval has asymptotically correct +frequentist coverage. + +''''' + +=== 10. Random Walk Analysis + +==== 10.1 Simple Random Walk + +*Theorem 10.1* (Random Walk Expectation). For `+(random-walk n)+` with +steps \{-1, 0, +1}: + +[latexmath] +++++ +\mathbb{E}[S_n] = 0 +++++ + +[latexmath] +++++ +\text{Var}(S_n) = \frac{2n}{3} +++++ + +_Proof_: - E[step] = (-1 + 0 + 1)/3 = 0 - E[step²] = (1 + 0 + 1)/3 = 2/3 +- Var(step) = 2/3 - 0² = 2/3 - By independence, Var(Sₙ) = n · Var(step) += 2n/3 ∎ + +==== 10.2 CLT for Random Walk + +*Theorem 10.2* (Diffusive Scaling). + +[latexmath] +++++ +\frac{S_n}{\sqrt{n}} \xrightarrow{d} N\left(0, \frac{2}{3}\right) +++++ + +==== 10.3 Hitting Time + +*Theorem 10.3* (Expected Hitting Time). For hitting level k starting +from 0: + +[latexmath] +++++ +\mathbb{E}[T_k] = \infty +++++ +(for symmetric walk) + +but + +[latexmath] +++++ +P(T_k < \infty) = 1 +++++ +(recurrence) + +''''' + +=== 11. Stochastic Process Convergence + +==== 11.1 Brownian Motion Approximation + +*Theorem 11.1* (Donsker’s Theorem). The rescaled random walk converges +to Brownian motion: + +[latexmath] +++++ +\left(\frac{S_{\lfloor nt \rfloor}}{\sqrt{n}}\right)_{t \in [0,1]} \xrightarrow{d} \sigma B_t +++++ + +where B_t is standard Brownian motion and σ² = 2/3. + +==== 11.2 Lévy Flight Heavy Tails + +*Theorem 11.2* (Stable Law Convergence). Lévy flights with index α +converge to α-stable distributions: + +[latexmath] +++++ +\frac{S_n}{n^{1/\alpha}} \xrightarrow{d} S_\alpha +++++ + +where S_α is the α-stable distribution. + +''''' + +=== 12. Complexity-Convergence Tradeoffs + +==== 12.1 Sample Complexity + +*Theorem 12.1* (Sample Complexity for ε-δ Estimation). To estimate a +probability p within ε with confidence 1-δ: + +[latexmath] +++++ +n = O\left(\frac{\log(1/\delta)}{\epsilon^2}\right) +++++ + +==== 12.2 Variance-Computation Tradeoff + +*Theorem 12.2* (Bias-Variance-Computation). For time budget T and cost c +per sample: + +[latexmath] +++++ +n = T/c +++++ + +[latexmath] +++++ +\text{MSE} = \frac{\text{Var}}{n} + \text{Bias}^2 = \frac{c \cdot \text{Var}}{T} + \text{Bias}^2 +++++ + +Optimal allocation balances these terms. + +''''' + +=== 13. Error Analysis + +==== 13.1 Numerical Precision + +*Theorem 13.1* (Floating-Point Error). For entropy calculation with n +samples: + +[latexmath] +++++ +|\hat{H}_{computed} - \hat{H}_{exact}| \leq O(n \cdot \epsilon_{machine}) +++++ + +where ε_machine ≈ 2.2 × 10⁻¹⁶ for IEEE 754 double precision. + +==== 13.2 Pseudorandom Quality + +*Theorem 13.2* (PRNG Period Effect). For Racket’s PRNG with period P: + +If n > P, samples become correlated, violating independence assumptions. + +For Racket’s random, P ≈ 2⁶¹, so this is not a practical concern for +typical simulation sizes. + +''''' + +=== 14. Summary of Convergence Rates + +[cols=",,",options="header",] +|=== +|Quantity |Rate |Theorem +|Sample mean |O(1/√n) |CLT +|Probability estimate |O(1/√n) |Thm 3.1 +|Entropy estimate |O(1/√n) |Thm 4.3 +|MCMC convergence |O(ρⁿ), ρ<1 |Thm 6.3 +|Bootstrap variance |O(1/√n) |Thm 8.3 +|KS statistic |O(1/√n) |Thm 5.2 +|=== + +''''' + +=== 15. TODOs and Open Problems + +*TODO*: The following need further investigation: + +[arabic] +. *Optimal importance sampling*: Derive optimal proposal for +betlang-specific computations +. *Adaptive MCMC convergence*: Prove convergence for adaptive MH in +lib/bayesian.rkt +. *Sequential Monte Carlo*: Prove particle filter convergence rate +. *Non-asymptotic Bayesian bounds*: Finite-sample posterior +concentration + +''''' + +=== References + +[arabic] +. Durrett, R. (2019). _Probability: Theory and Examples_ +. Van der Vaart, A.W. (1998). _Asymptotic Statistics_ +. Robert, C.P. & Casella, G. (2004). _Monte Carlo Statistical Methods_ +. Efron, B. & Tibshirani, R. (1993). _An Introduction to the Bootstrap_ +. Meyn, S.P. & Tweedie, R.L. (2009). _Markov Chains and Stochastic +Stability_ diff --git a/proofs/theorems/convergence-statistics.md b/proofs/theorems/convergence-statistics.md deleted file mode 100644 index b0e292f..0000000 --- a/proofs/theorems/convergence-statistics.md +++ /dev/null @@ -1,424 +0,0 @@ - -# Convergence and Statistical Proofs for Betlang - -## Abstract - -This document provides rigorous proofs for the statistical properties of betlang, including limit theorems, convergence rates, error bounds, and the theoretical foundations of the statistical inference library. - ---- - -## 1. Law of Large Numbers - -### 1.1 Weak Law of Large Numbers - -**Theorem 1.1** (WLLN for Ternary Bets). Let X₁, X₂, ..., Xₙ be i.i.d. samples from (bet A B C) where A, B, C ∈ ℝ. Then: - -$$\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i \xrightarrow{P} \mu = \frac{A + B + C}{3}$$ - -*Proof*: -By Chebyshev's inequality: -$$P(|\bar{X}_n - \mu| \geq \epsilon) \leq \frac{\text{Var}(\bar{X}_n)}{\epsilon^2} = \frac{\sigma^2}{n\epsilon^2}$$ - -where σ² = Var(X₁) = (A² + B² + C²)/3 - μ² < ∞. - -As n → ∞, this bound → 0, so $\bar{X}_n \xrightarrow{P} \mu$. ∎ - -### 1.2 Strong Law of Large Numbers - -**Theorem 1.2** (SLLN for Ternary Bets). Under the same conditions: - -$$\bar{X}_n \xrightarrow{a.s.} \mu$$ - -*Proof*: -Since E[X₁⁴] < ∞ (finite fourth moment for bounded discrete distribution), Kolmogorov's SLLN applies directly. ∎ - -### 1.3 Weighted Bet Extension - -**Theorem 1.3** (SLLN for Weighted Bets). For weighted bet with probabilities (p_A, p_B, p_C): - -$$\bar{X}_n \xrightarrow{a.s.} p_A \cdot A + p_B \cdot B + p_C \cdot C$$ - -*Proof*: Same as Theorem 1.2 with modified expectation. ∎ - ---- - -## 2. Central Limit Theorem - -### 2.1 Basic CLT - -**Theorem 2.1** (CLT for Ternary Bets). For i.i.d. samples from (bet A B C): - -$$\sqrt{n}\frac{\bar{X}_n - \mu}{\sigma} \xrightarrow{d} N(0, 1)$$ - -where μ = (A+B+C)/3 and σ² = E[X²] - μ². - -*Proof*: -The moment generating function of Xᵢ exists and is finite (bounded support). By Lévy's CLT, the standardized sum converges to standard normal. ∎ - -### 2.2 Berry-Esseen Bound - -**Theorem 2.2** (Convergence Rate). The CLT convergence rate is: - -$$\sup_x |P(\sqrt{n}(\bar{X}_n - \mu)/\sigma \leq x) - \Phi(x)| \leq \frac{C \cdot \rho}{\sigma^3 \sqrt{n}}$$ - -where ρ = E[|X - μ|³] and C ≤ 0.4748 (Berry-Esseen constant). - -*Proof*: -For ternary bets, the third absolute moment is bounded: -$$\rho = E[|X - \mu|^3] \leq \max(|A-\mu|, |B-\mu|, |C-\mu|)^3 < \infty$$ - -The Berry-Esseen theorem applies directly. ∎ - -**Corollary 2.1** (Practical Bound). For uniform (bet A B C) with A, B, C ∈ [-1, 1]: - -$$\sup_x |F_n(x) - \Phi(x)| \leq \frac{0.95}{\sqrt{n}}$$ - -This means n ≥ 1000 gives ≤ 3% approximation error. - -### 2.3 Multivariate CLT - -**Theorem 2.3** (Multivariate CLT). For d-dimensional bet outcomes: - -$$\sqrt{n}(\bar{\mathbf{X}}_n - \boldsymbol{\mu}) \xrightarrow{d} N_d(\mathbf{0}, \boldsymbol{\Sigma})$$ - -where Σ is the covariance matrix. - ---- - -## 3. Probability Estimation Convergence - -### 3.1 Frequency Estimator - -**Theorem 3.1** (Frequency Convergence). The `bet-probability` estimator: -$$\hat{p}_n = \frac{1}{n}\sum_{i=1}^{n} \mathbf{1}[\text{pred}(X_i)]$$ - -converges at rate O(1/√n). - -*Proof*: -$$\text{MSE}(\hat{p}_n) = \text{Var}(\hat{p}_n) = \frac{p(1-p)}{n}$$ - -Standard error: SE = √(p(1-p)/n) = O(1/√n). ∎ - -### 3.2 Confidence Interval Width - -**Theorem 3.2** (CI Width). The 95% confidence interval width is: - -$$W_n = 2 \cdot 1.96 \cdot \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} = O\left(\frac{1}{\sqrt{n}}\right)$$ - -**Corollary 3.1** (Sample Size for Precision). To achieve margin of error ε: -$$n \geq \frac{z_{\alpha/2}^2 \cdot p(1-p)}{\epsilon^2}$$ - -For ε = 0.01 and 95% confidence: n ≥ 9604 (using p = 0.5 conservatively). - -### 3.3 Hoeffding Bound - -**Theorem 3.3** (Finite-Sample Bound). For any ε > 0: - -$$P(|\hat{p}_n - p| \geq \epsilon) \leq 2\exp(-2n\epsilon^2)$$ - -*Proof*: Direct application of Hoeffding's inequality for bounded random variables in [0,1]. ∎ - -**Corollary 3.2** (Sample Size for High Confidence). For P(error > ε) ≤ δ: -$$n \geq \frac{\ln(2/\delta)}{2\epsilon^2}$$ - ---- - -## 4. Entropy Estimation - -### 4.1 Plugin Estimator Bias - -**Theorem 4.1** (Entropy Estimator Bias). The plugin entropy estimator: -$$\hat{H} = -\sum_{x} \hat{p}(x) \log_2 \hat{p}(x)$$ - -has negative bias: -$$\mathbb{E}[\hat{H}] = H - \frac{|S| - 1}{2n\ln 2} + O(n^{-2})$$ - -where |S| is the support size and H is the true entropy. - -*Proof*: By Taylor expansion of -p log p around the true probability. ∎ - -**Corollary 4.1** (Bias for Ternary Bet). For uniform ternary bet (|S| = 3): -$$\text{Bias} \approx -\frac{1}{n\ln 2} \approx -\frac{1.44}{n}$$ - -### 4.2 Miller-Madow Correction - -**Theorem 4.2** (Bias-Corrected Entropy). The Miller-Madow estimator: -$$\hat{H}_{MM} = \hat{H} + \frac{|S| - 1}{2n\ln 2}$$ - -is approximately unbiased to O(n⁻²). - -### 4.3 Entropy Convergence Rate - -**Theorem 4.3** (Entropy RMSE). The RMSE of the plugin estimator is: -$$\text{RMSE}(\hat{H}) = O\left(\frac{1}{\sqrt{n}}\right)$$ - -*Proof*: Using the delta method on the entropy functional. ∎ - ---- - -## 5. Distribution Convergence - -### 5.1 Empirical Distribution - -**Theorem 5.1** (Glivenko-Cantelli). The empirical CDF converges uniformly: -$$\sup_x |\hat{F}_n(x) - F(x)| \xrightarrow{a.s.} 0$$ - -For ternary bets (discrete), this is: -$$\max_{x \in \{A,B,C\}} |\hat{p}(x) - p(x)| \xrightarrow{a.s.} 0$$ - -### 5.2 DKW Inequality - -**Theorem 5.2** (Finite-Sample CDF Bound). -$$P\left(\sup_x |\hat{F}_n(x) - F(x)| > \epsilon\right) \leq 2e^{-2n\epsilon^2}$$ - -*Proof*: Dvoretzky-Kiefer-Wolfowitz inequality. ∎ - -### 5.3 Kolmogorov-Smirnov Test Validity - -**Theorem 5.3** (KS Test Asymptotics). For the KS statistic D_n: -$$\sqrt{n} D_n \xrightarrow{d} K$$ - -where K is the Kolmogorov distribution. - -The `kolmogorov-smirnov` function uses this for goodness-of-fit testing. - ---- - -## 6. Markov Chain Convergence - -### 6.1 Ergodic Theorem - -**Theorem 6.1** (MCMC Ergodicity). For an irreducible, aperiodic Markov chain with stationary distribution π: -$$\frac{1}{n}\sum_{i=1}^{n} f(X_i) \xrightarrow{a.s.} \mathbb{E}_\pi[f]$$ - -for any f with E_π[|f|] < ∞. - -*Proof*: By the Markov chain ergodic theorem. ∎ - -### 6.2 Mixing Time - -**Definition 6.1** (Mixing Time). The ε-mixing time is: -$$t_{mix}(\epsilon) = \min\{t : \max_x \|P^t(x, \cdot) - \pi\|_{TV} \leq \epsilon\}$$ - -**Theorem 6.2** (Mixing Time Bound). For the Metropolis-Hastings chain: -$$t_{mix}(\epsilon) \leq C \cdot \frac{\ln(1/(\epsilon \cdot \pi_{min}))}{\text{gap}}$$ - -where gap is the spectral gap of the transition matrix. - -### 6.3 Geometric Ergodicity - -**Theorem 6.3** (Geometric Convergence). If the MH chain is geometrically ergodic: -$$\|P^n(x, \cdot) - \pi\|_{TV} \leq M(x) \cdot \rho^n$$ - -for some ρ < 1 and function M. - -**Corollary 6.1** (Burn-in Recommendation). Use burn-in of: -$$B \geq \frac{\ln(M \cdot \epsilon)}{\ln(1/\rho)}$$ - -samples to achieve TV distance ≤ ε. - ---- - -## 7. Monte Carlo Integration - -### 7.1 Consistency - -**Theorem 7.1** (MC Integration Consistency). For Monte Carlo estimate: -$$\hat{I}_n = \frac{1}{n}\sum_{i=1}^{n} f(X_i)$$ - -where Xᵢ ~ p, we have: -$$\hat{I}_n \xrightarrow{a.s.} \int f(x) p(x) dx = \mathbb{E}_p[f]$$ - -*Proof*: SLLN for i.i.d. samples. ∎ - -### 7.2 Variance Reduction - -**Theorem 7.2** (Importance Sampling Variance). With importance distribution q: -$$\hat{I}_{IS} = \frac{1}{n}\sum_{i=1}^{n} \frac{f(X_i) p(X_i)}{q(X_i)}, \quad X_i \sim q$$ - -Variance: -$$\text{Var}(\hat{I}_{IS}) = \frac{1}{n}\left[\int \frac{f(x)^2 p(x)^2}{q(x)} dx - I^2\right]$$ - -Optimal q* ∝ |f(x)| p(x). - -### 7.3 Antithetic Variates - -**Theorem 7.3** (Antithetic Variance Reduction). For antithetic pairs (X, X'): -$$\text{Var}\left(\frac{f(X) + f(X')}{2}\right) = \frac{\text{Var}(f(X))}{2}(1 + \text{Corr}(f(X), f(X')))$$ - -If Corr < 0 (negatively correlated), variance is reduced. - -**Corollary 7.1** (Ternary Antithetic). For symmetric bet A, 0, -A with f(x) = x: -$$\text{Corr}(f(A), f(-A)) = -1$$ - -giving maximum variance reduction. - ---- - -## 8. Bootstrap Convergence - -### 8.1 Bootstrap Consistency - -**Theorem 8.1** (Bootstrap Validity). For the empirical bootstrap: -$$\hat{F}^*_n = \frac{1}{n}\sum_{i=1}^{n} \delta_{X_i}$$ - -the bootstrap distribution of √n(θ̂* - θ̂) converges to the true sampling distribution of √n(θ̂ - θ) for smooth functionals θ. - -*Proof*: Efron's theorem on bootstrap consistency. ∎ - -### 8.2 Bootstrap Confidence Intervals - -**Theorem 8.2** (Percentile Bootstrap). The percentile bootstrap interval: -$$[\hat{\theta}^*_{(\alpha/2)}, \hat{\theta}^*_{(1-\alpha/2)}]$$ - -has asymptotically correct coverage for symmetric distributions. - -### 8.3 Bootstrap Variance Estimation - -**Theorem 8.3** (Bootstrap Variance Consistency). -$$\hat{\text{Var}}_{boot}(\hat{\theta}) \xrightarrow{P} \text{Var}(\hat{\theta})$$ - -The `bootstrap` function in lib/statistics.rkt provides consistent variance estimates. - ---- - -## 9. Bayesian Convergence - -### 9.1 Posterior Consistency - -**Theorem 9.1** (Bernstein-von Mises). Under regularity conditions, the posterior converges to a normal distribution: -$$\Pi_n(\cdot | X_1, ..., X_n) \xrightarrow{d} N(\hat{\theta}_{MLE}, I(\theta_0)^{-1}/n)$$ - -where I(θ₀) is the Fisher information. - -### 9.2 Bayesian Consistency - -**Theorem 9.2** (Doob's Consistency). For "most" priors (in a measure-theoretic sense), the posterior concentrates around the true parameter: -$$\Pi_n(|\theta - \theta_0| > \epsilon | \text{data}) \xrightarrow{a.s.} 0$$ - -### 9.3 Credible Interval Coverage - -**Theorem 9.3** (Frequentist Coverage of Bayesian CIs). For regular problems: -$$P_{\theta_0}(\theta_0 \in CI_{1-\alpha}^{Bayes}) \to 1 - \alpha$$ - -as n → ∞. The Bayesian credible interval has asymptotically correct frequentist coverage. - ---- - -## 10. Random Walk Analysis - -### 10.1 Simple Random Walk - -**Theorem 10.1** (Random Walk Expectation). For `(random-walk n)` with steps {-1, 0, +1}: -$$\mathbb{E}[S_n] = 0$$ -$$\text{Var}(S_n) = \frac{2n}{3}$$ - -*Proof*: -- E[step] = (-1 + 0 + 1)/3 = 0 -- E[step²] = (1 + 0 + 1)/3 = 2/3 -- Var(step) = 2/3 - 0² = 2/3 -- By independence, Var(Sₙ) = n · Var(step) = 2n/3 ∎ - -### 10.2 CLT for Random Walk - -**Theorem 10.2** (Diffusive Scaling). -$$\frac{S_n}{\sqrt{n}} \xrightarrow{d} N\left(0, \frac{2}{3}\right)$$ - -### 10.3 Hitting Time - -**Theorem 10.3** (Expected Hitting Time). For hitting level k starting from 0: -$$\mathbb{E}[T_k] = \infty$$ (for symmetric walk) - -but -$$P(T_k < \infty) = 1$$ (recurrence) - ---- - -## 11. Stochastic Process Convergence - -### 11.1 Brownian Motion Approximation - -**Theorem 11.1** (Donsker's Theorem). The rescaled random walk converges to Brownian motion: -$$\left(\frac{S_{\lfloor nt \rfloor}}{\sqrt{n}}\right)_{t \in [0,1]} \xrightarrow{d} \sigma B_t$$ - -where B_t is standard Brownian motion and σ² = 2/3. - -### 11.2 Lévy Flight Heavy Tails - -**Theorem 11.2** (Stable Law Convergence). Lévy flights with index α converge to α-stable distributions: -$$\frac{S_n}{n^{1/\alpha}} \xrightarrow{d} S_\alpha$$ - -where S_α is the α-stable distribution. - ---- - -## 12. Complexity-Convergence Tradeoffs - -### 12.1 Sample Complexity - -**Theorem 12.1** (Sample Complexity for ε-δ Estimation). To estimate a probability p within ε with confidence 1-δ: -$$n = O\left(\frac{\log(1/\delta)}{\epsilon^2}\right)$$ - -### 12.2 Variance-Computation Tradeoff - -**Theorem 12.2** (Bias-Variance-Computation). For time budget T and cost c per sample: -$$n = T/c$$ -$$\text{MSE} = \frac{\text{Var}}{n} + \text{Bias}^2 = \frac{c \cdot \text{Var}}{T} + \text{Bias}^2$$ - -Optimal allocation balances these terms. - ---- - -## 13. Error Analysis - -### 13.1 Numerical Precision - -**Theorem 13.1** (Floating-Point Error). For entropy calculation with n samples: -$$|\hat{H}_{computed} - \hat{H}_{exact}| \leq O(n \cdot \epsilon_{machine})$$ - -where ε_machine ≈ 2.2 × 10⁻¹⁶ for IEEE 754 double precision. - -### 13.2 Pseudorandom Quality - -**Theorem 13.2** (PRNG Period Effect). For Racket's PRNG with period P: - -If n > P, samples become correlated, violating independence assumptions. - -For Racket's random, P ≈ 2⁶¹, so this is not a practical concern for typical simulation sizes. - ---- - -## 14. Summary of Convergence Rates - -| Quantity | Rate | Theorem | -|----------|------|---------| -| Sample mean | O(1/√n) | CLT | -| Probability estimate | O(1/√n) | Thm 3.1 | -| Entropy estimate | O(1/√n) | Thm 4.3 | -| MCMC convergence | O(ρⁿ), ρ<1 | Thm 6.3 | -| Bootstrap variance | O(1/√n) | Thm 8.3 | -| KS statistic | O(1/√n) | Thm 5.2 | - ---- - -## 15. TODOs and Open Problems - -**TODO**: The following need further investigation: - -1. **Optimal importance sampling**: Derive optimal proposal for betlang-specific computations -2. **Adaptive MCMC convergence**: Prove convergence for adaptive MH in lib/bayesian.rkt -3. **Sequential Monte Carlo**: Prove particle filter convergence rate -4. **Non-asymptotic Bayesian bounds**: Finite-sample posterior concentration - ---- - -## References - -1. Durrett, R. (2019). *Probability: Theory and Examples* -2. Van der Vaart, A.W. (1998). *Asymptotic Statistics* -3. Robert, C.P. & Casella, G. (2004). *Monte Carlo Statistical Methods* -4. Efron, B. & Tibshirani, R. (1993). *An Introduction to the Bootstrap* -5. Meyn, S.P. & Tweedie, R.L. (2009). *Markov Chains and Stochastic Stability* diff --git a/proofs/theorems/information-theory.adoc b/proofs/theorems/information-theory.adoc new file mode 100644 index 0000000..e577963 --- /dev/null +++ b/proofs/theorems/information-theory.adoc @@ -0,0 +1,641 @@ +== Information-Theoretic Analysis of Betlang + +=== Abstract + +This document provides a comprehensive information-theoretic analysis of +betlang, covering Shannon entropy, mutual information, channel capacity, +rate-distortion theory, and the information content of probabilistic +programs. + +''''' + +=== 1. Entropy Fundamentals + +==== 1.1 Shannon Entropy + +*Definition 1.1* (Entropy). For random variable X with PMF p: + +[latexmath] +++++ +H(X) = -\sum_{x} p(x) \log_2 p(x) +++++ + +*Theorem 1.1* (Uniform Ternary Entropy). For `+(bet A B C)+`: + +[latexmath] +++++ +H = \log_2(3) \approx 1.5849625 \text{ bits} +++++ + +_Proof_: + +[latexmath] +++++ +H = -3 \cdot \frac{1}{3} \log_2 \frac{1}{3} = -\log_2 \frac{1}{3} = \log_2 3 \approx 1.585 ∎ +++++ + +*Theorem 1.2* (Maximum Entropy Principle). Among all ternary +distributions, uniform achieves maximum entropy. + +_Proof_: By Lagrange multipliers on H subject to Σp = 1: + +[latexmath] +++++ +\frac{\partial}{\partial p_i}\left[-\sum_j p_j \log p_j + λ(\sum_j p_j - 1)\right] = 0 +++++ + +[latexmath] +++++ +-\log p_i - 1 + λ = 0 \Rightarrow p_i = e^{λ-1} +++++ + +All probabilities equal ⟹ uniform distribution. ∎ + +==== 1.2 Weighted Bet Entropy + +*Theorem 1.3* (Weighted Entropy). For +`+(bet/weighted '(A wₐ) '(B w_b) '(C w_c))+`: + +[latexmath] +++++ +H = -\sum_{i \in \{A,B,C\}} \frac{w_i}{W} \log_2 \frac{w_i}{W} +++++ + +where W = wₐ + w_b + w_c. + +*Example*: Weights (2, 1, 1): - P(A) = 1/2, P(B) = P(C) = 1/4 - H = -1/2 +log(1/2) - 2·(1/4)log(1/4) = 1/2 + 1 = 1.5 bits + +==== 1.3 Entropy Bounds + +*Theorem 1.4* (Entropy Bounds for Ternary). + +[latexmath] +++++ +0 \leq H(X) \leq \log_2(3) +++++ + +* H = 0 ⟺ deterministic (one outcome has probability 1) +* H = log₂(3) ⟺ uniform + +''''' + +=== 2. Conditional Entropy and Mutual Information + +==== 2.1 Conditional Entropy + +*Definition 2.1* (Conditional Entropy). + +[latexmath] +++++ +H(X|Y) = \sum_y p(y) H(X|Y=y) = -\sum_{x,y} p(x,y) \log_2 p(x|y) +++++ + +*Theorem 2.1* (Independence and Conditional Entropy). For independent +bets X, Y: + +[latexmath] +++++ +H(X|Y) = H(X) +++++ + +_Proof_: Independence implies p(x|y) = p(x), so: + +[latexmath] +++++ +H(X|Y) = -\sum_{x,y} p(x,y) \log_2 p(x) = -\sum_x p(x) \log_2 p(x) = H(X) ∎ +++++ + +==== 2.2 Mutual Information + +*Definition 2.2* (Mutual Information). + +[latexmath] +++++ +I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = H(X) + H(Y) - H(X,Y) +++++ + +*Theorem 2.2* (Independent Bets). For independent `+(bet A B C)+` and +`+(bet D E F)+`: + +[latexmath] +++++ +I(X; Y) = 0 +++++ + +*Theorem 2.3* (Mutual Information Bounds). + +[latexmath] +++++ +0 \leq I(X; Y) \leq \min(H(X), H(Y)) +++++ + +==== 2.3 Data Processing Inequality + +*Theorem 2.4* (Data Processing for Bets). For Markov chain X → Y → Z: + +[latexmath] +++++ +I(X; Z) \leq I(X; Y) +++++ + +Processing cannot increase mutual information. + +''''' + +=== 3. Entropy Rate + +==== 3.1 Definition + +*Definition 3.1* (Entropy Rate). For stochastic process \{Xₙ}: + +[latexmath] +++++ +H_\infty = \lim_{n \to \infty} \frac{1}{n} H(X_1, ..., X_n) +++++ + +*Theorem 3.1* (i.i.d. Bet Sequence). For i.i.d. `+(bet A B C)+` +sequence: + +[latexmath] +++++ +H_\infty = H(X_1) = \log_2(3) \approx 1.585 \text{ bits/symbol} +++++ + +==== 3.2 Markov Chain Entropy Rate + +*Theorem 3.2* (Markov Entropy Rate). For Markov chain with transition +matrix P and stationary distribution π: + +[latexmath] +++++ +H_\infty = -\sum_i π_i \sum_j P_{ij} \log_2 P_{ij} +++++ + +*Application*: Ternary Markov chain with uniform stationary +distribution: + +[latexmath] +++++ +H_\infty = H(\text{transition from any state}) +++++ + +==== 3.3 Random Walk Entropy + +*Theorem 3.3* (Ternary Random Walk Entropy). For walk with steps \{-1, +0, +1}: + +[latexmath] +++++ +H_\infty = \log_2(3) \approx 1.585 \text{ bits/step} +++++ + +Each step is an independent ternary choice. + +''''' + +=== 4. Source Coding + +==== 4.1 Optimal Code Length + +*Theorem 4.1* (Shannon’s Source Coding Theorem). For source X with +entropy H(X), the optimal expected code length L* satisfies: + +[latexmath] +++++ +H(X) \leq L^* < H(X) + 1 +++++ + +*Corollary 4.1* (Ternary Bet Coding). To encode outcomes of (bet A B C): +- Minimum: 1.585 bits average - Achievable: 2 bits (fixed length) or +1.585 bits (optimal variable length) + +==== 4.2 Huffman Coding for Ternary + +For uniform ternary (p = 1/3 each): + +*Optimal code*: Not possible with binary prefix codes at exactly log₂(3) +bits. Best binary Huffman: one outcome gets 1 bit, two get 2 bits. - +Average: 1·(1/3) + 2·(1/3) + 2·(1/3) = 5/3 ≈ 1.67 bits + +*Efficiency*: 1.585/1.67 ≈ 95% + +==== 4.3 Arithmetic Coding + +*Theorem 4.2* (Arithmetic Coding Optimality). Arithmetic coding achieves +rate H + ε for any ε > 0. + +For ternary bets, arithmetic coding achieves ~1.585 bits/symbol +asymptotically. + +''''' + +=== 5. Channel Capacity + +==== 5.1 Ternary Channel + +*Definition 5.1* (Ternary Symmetric Channel). Input/output alphabet \{0, +1, 2} with crossover probability ε: + +[latexmath] +++++ +P(Y=j|X=i) = \begin{cases} 1-ε & i=j \\ ε/2 & i≠j \end{cases} +++++ + +*Theorem 5.1* (TSC Capacity). + +[latexmath] +++++ +C_{TSC} = \log_2(3) - H(\epsilon, \epsilon/2, \epsilon/2) +++++ + +where H(·) is the entropy function. + +For ε = 0 (noiseless): C = log₂(3) ≈ 1.585 bits. + +==== 5.2 Channel Coding with Bets + +*Theorem 5.2* (Achievability). For BSC with capacity C, we can reliably +transmit at rate R < C using ternary-encoded messages decoded with +probabilistic algorithms. + +==== 5.3 Noisy Bet Channel + +*Model*: `+(noisy-bet A B C ε)+` - intended outcome corrupted with +probability ε. + +*Capacity*: + +[latexmath] +++++ +C = \log_2(3) \cdot (1 - H_2(\epsilon)) +++++ + +where H₂ is binary entropy. + +''''' + +=== 6. Rate-Distortion Theory + +==== 6.1 Distortion Measure + +*Definition 6.1* (Hamming Distortion for Ternary). + +[latexmath] +++++ +d(x, \hat{x}) = \begin{cases} 0 & x = \hat{x} \\ 1 & x ≠ \hat{x} \end{cases} +++++ + +==== 6.2 Rate-Distortion Function + +*Theorem 6.1* (Ternary Rate-Distortion). For uniform ternary source: + +[latexmath] +++++ +R(D) = \log_2(3) - H(D, (1-D)/2, (1-D)/2) \text{ for } D \leq 2/3 +++++ + +==== 6.3 Compression of Bet Sequences + +To compress sequence of bets to rate R bits/symbol with distortion D: - +R = 0: D = 2/3 (always guess one outcome) - R = log₂(3): D = 0 +(lossless) + +''''' + +=== 7. Kullback-Leibler Divergence + +==== 7.1 Definition + +*Definition 7.1* (KL Divergence). + +[latexmath] +++++ +D_{KL}(P || Q) = \sum_x P(x) \log_2 \frac{P(x)}{Q(x)} +++++ + +==== 7.2 Uniform vs Weighted Bet + +*Theorem 7.1* (Divergence from Uniform). For weighted bet P vs uniform +Q: + +[latexmath] +++++ +D_{KL}(P || Q) = \log_2(3) - H(P) +++++ + +_Proof_: + +[latexmath] +++++ +D_{KL}(P || Q) = \sum_x P(x) \log_2 \frac{P(x)}{1/3} = \sum_x P(x) \log_2 P(x) + \log_2 3 = -H(P) + \log_2 3 ∎ +++++ + +==== 7.3 Information Geometry + +The space of ternary distributions forms a 2-simplex: + +[latexmath] +++++ +\Delta_2 = \{(p_1, p_2, p_3) : p_i \geq 0, \sum p_i = 1\} +++++ + +This is an information-geometric manifold with Fisher metric. + +''''' + +=== 8. Fisher Information + +==== 8.1 Definition + +*Definition 8.1* (Fisher Information). For parametric family p(x;θ): + +[latexmath] +++++ +I(\theta) = E\left[\left(\frac{\partial}{\partial θ} \log p(X;θ)\right)^2\right] +++++ + +==== 8.2 Ternary Parametric Family + +For ternary distribution parameterized by (θ₁, θ₂): + +[latexmath] +++++ +P(X=1) = θ_1, \quad P(X=2) = θ_2, \quad P(X=3) = 1-θ_1-θ_2 +++++ + +*Fisher Information Matrix*: + +[latexmath] +++++ +I_{ij} = \frac{∂}{∂θ_i} \frac{∂}{∂θ_j} D_{KL}(P_θ || P_{θ'}) |_{θ'=θ} +++++ + +==== 8.3 Cramér-Rao Bound + +*Theorem 8.1* (Cramér-Rao for Ternary Estimation). For unbiased +estimator θ̂ of ternary parameter: + +[latexmath] +++++ +\text{Var}(\hat{θ}) \geq \frac{1}{n \cdot I(θ)} +++++ + +''''' + +=== 9. Entropy of Programs + +==== 9.1 Program Entropy + +*Definition 9.1* (Program Entropy). For probabilistic program P +producing distribution D: + +[latexmath] +++++ +H(P) = H(D) = -\sum_x D(x) \log_2 D(x) +++++ + +==== 9.2 Entropy Composition + +*Theorem 9.1* (Sequential Composition). For independent programs P₁, P₂: + +[latexmath] +++++ +H(P_1 ; P_2) = H(P_1) + H(P_2) +++++ + +*Theorem 9.2* (Bet Composition). + +[latexmath] +++++ +H(\text{bet } P_1 \ P_2 \ P_3) = \log_2(3) + \frac{1}{3}(H(P_1) + H(P_2) + H(P_3)) +++++ + +==== 9.3 Entropy of bet-chain + +*Theorem 9.3* (Chain Entropy). For `+(bet-chain n f init)+` where f is +deterministic: + +[latexmath] +++++ +H(\text{bet-chain } n \ f \ \text{init}) = 0 +++++ + +If f involves randomness: + +[latexmath] +++++ +H(\text{bet-chain } n \ f \ \text{init}) = n \cdot H_f +++++ + +where Hf is the entropy contribution per iteration. + +''''' + +=== 10. Entropy Estimation in Betlang + +==== 10.1 Plugin Estimator + +The `+bet-entropy+` function uses: + +[latexmath] +++++ +\hat{H} = -\sum_x \hat{p}(x) \log_2 \hat{p}(x) +++++ + +where p̂(x) = count(x)/n. + +*Theorem 10.1* (Bias of Plugin Estimator). + +[latexmath] +++++ +E[\hat{H}] = H - \frac{|S| - 1}{2n \ln 2} + O(n^{-2}) +++++ + +For ternary (|S| = 3): Bias ≈ -1.44/n + +==== 10.2 Miller-Madow Correction + +*Corrected estimator*: + +[latexmath] +++++ +\hat{H}_{MM} = \hat{H} + \frac{|\hat{S}| - 1}{2n \ln 2} +++++ + +==== 10.3 Confidence Interval for Entropy + +*Theorem 10.2* (Asymptotic Variance). + +[latexmath] +++++ +\text{Var}(\hat{H}) \approx \frac{1}{n}\left[\sum_x p(x) (\log p(x))^2 - H^2\right] +++++ + +''''' + +=== 11. Information in Statistical Inference + +==== 11.1 Bayesian Information + +*Definition 11.1* (Information Gain). + +[latexmath] +++++ +I_{gain} = H(\text{prior}) - H(\text{posterior}) +++++ + +For conjugate updates, this quantifies learning. + +==== 11.2 Expected Information Gain + +*Theorem 11.1* (Expected IG). For observation X: + +[latexmath] +++++ +E[I_{gain}] = I(\Theta; X) +++++ + +mutual information between parameter and observation. + +==== 11.3 Sequential Experimental Design + +*Criterion*: Choose experiment maximizing expected information gain. + +For ternary outcomes, optimal design balances: + +[latexmath] +++++ +\max_{\text{design}} E[H(\text{prior}) - H(\text{posterior} | X)] +++++ + +''''' + +=== 12. Minimum Description Length + +==== 12.1 MDL Principle + +*Definition 12.1* (Two-Part MDL). Model complexity: + +[latexmath] +++++ +L(M) + L(D|M) +++++ + +where L(M) = model description length, L(D|M) = data given model. + +==== 12.2 Ternary Model Selection + +For selecting between ternary distributions: - Uniform: L(M) = 0 (no +parameters) - Weighted (2 params): L(M) = 2 · precision bits - Full: +L(M) = (|S|-1) · precision bits + +==== 12.3 Normalized Maximum Likelihood + +*Definition 12.2* (NML). The NML distribution: + +[latexmath] +++++ +p_{NML}(x) = \frac{p(x | \hat{θ}_{ML}(x))}{∑_{x'} p(x' | \hat{θ}_{ML}(x'))} +++++ + +''''' + +=== 13. Typicality and AEP + +==== 13.1 Asymptotic Equipartition Property + +*Theorem 13.1* (AEP for Ternary i.i.d.). For X₁, …, Xₙ i.i.d. uniform +ternary: + +[latexmath] +++++ +-\frac{1}{n} \log_2 p(X_1, ..., X_n) \xrightarrow{P} H(X) = \log_2(3) +++++ + +==== 13.2 Typical Set + +*Definition 13.1* (Typical Set). + +[latexmath] +++++ +A_\epsilon^{(n)} = \{(x_1,...,x_n) : |{-}\frac{1}{n} \log_2 p - H| < \epsilon\} +++++ + +*Theorem 13.2* (Typical Set Properties). 1. P(A_ε^\{(n)}) → 1 as n → ∞ +2. |A_ε^\{(n)}| ≈ 2^\{nH} 3. All elements have probability ≈ 2^\{-nH} + +For ternary: |A_ε^\{(n)}| ≈ 3^n + +''''' + +=== 14. Information-Theoretic Security + +==== 14.1 Perfect Secrecy + +*Theorem 14.1* (Shannon Secrecy). Perfect secrecy requires: + +[latexmath] +++++ +H(K) \geq H(M) +++++ + +For ternary message space, need ternary key. + +==== 14.2 Ternary One-Time Pad + +*Definition 14.1* (Ternary OTP). + +[latexmath] +++++ +C = (M + K) \mod 3 +++++ + +*Theorem 14.2* (Ternary OTP Security). If K ~ Uniform\{0,1,2} +independent of M: + +[latexmath] +++++ +I(M; C) = 0 +++++ + +Perfect secrecy. + +''''' + +=== 15. Summary of Key Results + +[cols=",,",options="header",] +|=== +|Quantity |Ternary Value |Binary Comparison +|Maximum entropy |log₂(3) ≈ 1.585 |1 bit +|Entropy rate (i.i.d.) |1.585 bits/symbol |1 bit/symbol +|Optimal code length |≥ 1.585 bits |≥ 1 bit +|TSC capacity (noiseless) |1.585 bits |1 bit +|Fisher information (uniform) |3 |4 +|=== + +''''' + +=== 16. TODOs + +*TODO*: Further development needed: + +[arabic] +. *Continuous entropy*: Differential entropy for continuous +distributions +. *Multivariate information*: Information in joint ternary distributions +. *Network information theory*: Multiple ternary sources/channels +. *Quantum information*: Connection to qutrit systems + +''''' + +=== References + +[arabic] +. Cover, T.M. & Thomas, J.A. (2006). _Elements of Information Theory_, +2nd ed. +. MacKay, D.J.C. (2003). _Information Theory, Inference, and Learning +Algorithms_ +. Shannon, C.E. (1948). "`A Mathematical Theory of Communication`" +. Rissanen, J. (1978). "`Modeling by shortest data description`" diff --git a/proofs/theorems/information-theory.md b/proofs/theorems/information-theory.md deleted file mode 100644 index a43887a..0000000 --- a/proofs/theorems/information-theory.md +++ /dev/null @@ -1,409 +0,0 @@ - -# Information-Theoretic Analysis of Betlang - -## Abstract - -This document provides a comprehensive information-theoretic analysis of betlang, covering Shannon entropy, mutual information, channel capacity, rate-distortion theory, and the information content of probabilistic programs. - ---- - -## 1. Entropy Fundamentals - -### 1.1 Shannon Entropy - -**Definition 1.1** (Entropy). For random variable X with PMF p: -$$H(X) = -\sum_{x} p(x) \log_2 p(x)$$ - -**Theorem 1.1** (Uniform Ternary Entropy). For `(bet A B C)`: -$$H = \log_2(3) \approx 1.5849625 \text{ bits}$$ - -*Proof*: -$$H = -3 \cdot \frac{1}{3} \log_2 \frac{1}{3} = -\log_2 \frac{1}{3} = \log_2 3 \approx 1.585 ∎$$ - -**Theorem 1.2** (Maximum Entropy Principle). Among all ternary distributions, uniform achieves maximum entropy. - -*Proof*: By Lagrange multipliers on H subject to Σp = 1: -$$\frac{\partial}{\partial p_i}\left[-\sum_j p_j \log p_j + λ(\sum_j p_j - 1)\right] = 0$$ -$$-\log p_i - 1 + λ = 0 \Rightarrow p_i = e^{λ-1}$$ - -All probabilities equal ⟹ uniform distribution. ∎ - -### 1.2 Weighted Bet Entropy - -**Theorem 1.3** (Weighted Entropy). For `(bet/weighted '(A wₐ) '(B w_b) '(C w_c))`: -$$H = -\sum_{i \in \{A,B,C\}} \frac{w_i}{W} \log_2 \frac{w_i}{W}$$ - -where W = wₐ + w_b + w_c. - -**Example**: Weights (2, 1, 1): -- P(A) = 1/2, P(B) = P(C) = 1/4 -- H = -1/2 log(1/2) - 2·(1/4)log(1/4) = 1/2 + 1 = 1.5 bits - -### 1.3 Entropy Bounds - -**Theorem 1.4** (Entropy Bounds for Ternary). -$$0 \leq H(X) \leq \log_2(3)$$ - -- H = 0 ⟺ deterministic (one outcome has probability 1) -- H = log₂(3) ⟺ uniform - ---- - -## 2. Conditional Entropy and Mutual Information - -### 2.1 Conditional Entropy - -**Definition 2.1** (Conditional Entropy). -$$H(X|Y) = \sum_y p(y) H(X|Y=y) = -\sum_{x,y} p(x,y) \log_2 p(x|y)$$ - -**Theorem 2.1** (Independence and Conditional Entropy). For independent bets X, Y: -$$H(X|Y) = H(X)$$ - -*Proof*: Independence implies p(x|y) = p(x), so: -$$H(X|Y) = -\sum_{x,y} p(x,y) \log_2 p(x) = -\sum_x p(x) \log_2 p(x) = H(X) ∎$$ - -### 2.2 Mutual Information - -**Definition 2.2** (Mutual Information). -$$I(X; Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) = H(X) + H(Y) - H(X,Y)$$ - -**Theorem 2.2** (Independent Bets). For independent `(bet A B C)` and `(bet D E F)`: -$$I(X; Y) = 0$$ - -**Theorem 2.3** (Mutual Information Bounds). -$$0 \leq I(X; Y) \leq \min(H(X), H(Y))$$ - -### 2.3 Data Processing Inequality - -**Theorem 2.4** (Data Processing for Bets). For Markov chain X → Y → Z: -$$I(X; Z) \leq I(X; Y)$$ - -Processing cannot increase mutual information. - ---- - -## 3. Entropy Rate - -### 3.1 Definition - -**Definition 3.1** (Entropy Rate). For stochastic process {Xₙ}: -$$H_\infty = \lim_{n \to \infty} \frac{1}{n} H(X_1, ..., X_n)$$ - -**Theorem 3.1** (i.i.d. Bet Sequence). For i.i.d. `(bet A B C)` sequence: -$$H_\infty = H(X_1) = \log_2(3) \approx 1.585 \text{ bits/symbol}$$ - -### 3.2 Markov Chain Entropy Rate - -**Theorem 3.2** (Markov Entropy Rate). For Markov chain with transition matrix P and stationary distribution π: -$$H_\infty = -\sum_i π_i \sum_j P_{ij} \log_2 P_{ij}$$ - -**Application**: Ternary Markov chain with uniform stationary distribution: -$$H_\infty = H(\text{transition from any state})$$ - -### 3.3 Random Walk Entropy - -**Theorem 3.3** (Ternary Random Walk Entropy). For walk with steps {-1, 0, +1}: -$$H_\infty = \log_2(3) \approx 1.585 \text{ bits/step}$$ - -Each step is an independent ternary choice. - ---- - -## 4. Source Coding - -### 4.1 Optimal Code Length - -**Theorem 4.1** (Shannon's Source Coding Theorem). For source X with entropy H(X), the optimal expected code length L* satisfies: -$$H(X) \leq L^* < H(X) + 1$$ - -**Corollary 4.1** (Ternary Bet Coding). To encode outcomes of (bet A B C): -- Minimum: 1.585 bits average -- Achievable: 2 bits (fixed length) or 1.585 bits (optimal variable length) - -### 4.2 Huffman Coding for Ternary - -For uniform ternary (p = 1/3 each): - -**Optimal code**: Not possible with binary prefix codes at exactly log₂(3) bits. Best binary Huffman: one outcome gets 1 bit, two get 2 bits. -- Average: 1·(1/3) + 2·(1/3) + 2·(1/3) = 5/3 ≈ 1.67 bits - -**Efficiency**: 1.585/1.67 ≈ 95% - -### 4.3 Arithmetic Coding - -**Theorem 4.2** (Arithmetic Coding Optimality). Arithmetic coding achieves rate H + ε for any ε > 0. - -For ternary bets, arithmetic coding achieves ~1.585 bits/symbol asymptotically. - ---- - -## 5. Channel Capacity - -### 5.1 Ternary Channel - -**Definition 5.1** (Ternary Symmetric Channel). Input/output alphabet {0, 1, 2} with crossover probability ε: -$$P(Y=j|X=i) = \begin{cases} 1-ε & i=j \\ ε/2 & i≠j \end{cases}$$ - -**Theorem 5.1** (TSC Capacity). -$$C_{TSC} = \log_2(3) - H(\epsilon, \epsilon/2, \epsilon/2)$$ - -where H(·) is the entropy function. - -For ε = 0 (noiseless): C = log₂(3) ≈ 1.585 bits. - -### 5.2 Channel Coding with Bets - -**Theorem 5.2** (Achievability). For BSC with capacity C, we can reliably transmit at rate R < C using ternary-encoded messages decoded with probabilistic algorithms. - -### 5.3 Noisy Bet Channel - -**Model**: `(noisy-bet A B C ε)` - intended outcome corrupted with probability ε. - -**Capacity**: -$$C = \log_2(3) \cdot (1 - H_2(\epsilon))$$ - -where H₂ is binary entropy. - ---- - -## 6. Rate-Distortion Theory - -### 6.1 Distortion Measure - -**Definition 6.1** (Hamming Distortion for Ternary). -$$d(x, \hat{x}) = \begin{cases} 0 & x = \hat{x} \\ 1 & x ≠ \hat{x} \end{cases}$$ - -### 6.2 Rate-Distortion Function - -**Theorem 6.1** (Ternary Rate-Distortion). For uniform ternary source: -$$R(D) = \log_2(3) - H(D, (1-D)/2, (1-D)/2) \text{ for } D \leq 2/3$$ - -### 6.3 Compression of Bet Sequences - -To compress sequence of bets to rate R bits/symbol with distortion D: -- R = 0: D = 2/3 (always guess one outcome) -- R = log₂(3): D = 0 (lossless) - ---- - -## 7. Kullback-Leibler Divergence - -### 7.1 Definition - -**Definition 7.1** (KL Divergence). -$$D_{KL}(P || Q) = \sum_x P(x) \log_2 \frac{P(x)}{Q(x)}$$ - -### 7.2 Uniform vs Weighted Bet - -**Theorem 7.1** (Divergence from Uniform). For weighted bet P vs uniform Q: -$$D_{KL}(P || Q) = \log_2(3) - H(P)$$ - -*Proof*: -$$D_{KL}(P || Q) = \sum_x P(x) \log_2 \frac{P(x)}{1/3} = \sum_x P(x) \log_2 P(x) + \log_2 3 = -H(P) + \log_2 3 ∎$$ - -### 7.3 Information Geometry - -The space of ternary distributions forms a 2-simplex: -$$\Delta_2 = \{(p_1, p_2, p_3) : p_i \geq 0, \sum p_i = 1\}$$ - -This is an information-geometric manifold with Fisher metric. - ---- - -## 8. Fisher Information - -### 8.1 Definition - -**Definition 8.1** (Fisher Information). For parametric family p(x;θ): -$$I(\theta) = E\left[\left(\frac{\partial}{\partial θ} \log p(X;θ)\right)^2\right]$$ - -### 8.2 Ternary Parametric Family - -For ternary distribution parameterized by (θ₁, θ₂): -$$P(X=1) = θ_1, \quad P(X=2) = θ_2, \quad P(X=3) = 1-θ_1-θ_2$$ - -**Fisher Information Matrix**: -$$I_{ij} = \frac{∂}{∂θ_i} \frac{∂}{∂θ_j} D_{KL}(P_θ || P_{θ'}) |_{θ'=θ}$$ - -### 8.3 Cramér-Rao Bound - -**Theorem 8.1** (Cramér-Rao for Ternary Estimation). For unbiased estimator θ̂ of ternary parameter: -$$\text{Var}(\hat{θ}) \geq \frac{1}{n \cdot I(θ)}$$ - ---- - -## 9. Entropy of Programs - -### 9.1 Program Entropy - -**Definition 9.1** (Program Entropy). For probabilistic program P producing distribution D: -$$H(P) = H(D) = -\sum_x D(x) \log_2 D(x)$$ - -### 9.2 Entropy Composition - -**Theorem 9.1** (Sequential Composition). For independent programs P₁, P₂: -$$H(P_1 ; P_2) = H(P_1) + H(P_2)$$ - -**Theorem 9.2** (Bet Composition). -$$H(\text{bet } P_1 \ P_2 \ P_3) = \log_2(3) + \frac{1}{3}(H(P_1) + H(P_2) + H(P_3))$$ - -### 9.3 Entropy of bet-chain - -**Theorem 9.3** (Chain Entropy). For `(bet-chain n f init)` where f is deterministic: -$$H(\text{bet-chain } n \ f \ \text{init}) = 0$$ - -If f involves randomness: -$$H(\text{bet-chain } n \ f \ \text{init}) = n \cdot H_f$$ - -where Hf is the entropy contribution per iteration. - ---- - -## 10. Entropy Estimation in Betlang - -### 10.1 Plugin Estimator - -The `bet-entropy` function uses: -$$\hat{H} = -\sum_x \hat{p}(x) \log_2 \hat{p}(x)$$ - -where p̂(x) = count(x)/n. - -**Theorem 10.1** (Bias of Plugin Estimator). -$$E[\hat{H}] = H - \frac{|S| - 1}{2n \ln 2} + O(n^{-2})$$ - -For ternary (|S| = 3): Bias ≈ -1.44/n - -### 10.2 Miller-Madow Correction - -**Corrected estimator**: -$$\hat{H}_{MM} = \hat{H} + \frac{|\hat{S}| - 1}{2n \ln 2}$$ - -### 10.3 Confidence Interval for Entropy - -**Theorem 10.2** (Asymptotic Variance). -$$\text{Var}(\hat{H}) \approx \frac{1}{n}\left[\sum_x p(x) (\log p(x))^2 - H^2\right]$$ - ---- - -## 11. Information in Statistical Inference - -### 11.1 Bayesian Information - -**Definition 11.1** (Information Gain). -$$I_{gain} = H(\text{prior}) - H(\text{posterior})$$ - -For conjugate updates, this quantifies learning. - -### 11.2 Expected Information Gain - -**Theorem 11.1** (Expected IG). For observation X: -$$E[I_{gain}] = I(\Theta; X)$$ - -mutual information between parameter and observation. - -### 11.3 Sequential Experimental Design - -**Criterion**: Choose experiment maximizing expected information gain. - -For ternary outcomes, optimal design balances: -$$\max_{\text{design}} E[H(\text{prior}) - H(\text{posterior} | X)]$$ - ---- - -## 12. Minimum Description Length - -### 12.1 MDL Principle - -**Definition 12.1** (Two-Part MDL). Model complexity: -$$L(M) + L(D|M)$$ - -where L(M) = model description length, L(D|M) = data given model. - -### 12.2 Ternary Model Selection - -For selecting between ternary distributions: -- Uniform: L(M) = 0 (no parameters) -- Weighted (2 params): L(M) = 2 · precision bits -- Full: L(M) = (|S|-1) · precision bits - -### 12.3 Normalized Maximum Likelihood - -**Definition 12.2** (NML). The NML distribution: -$$p_{NML}(x) = \frac{p(x | \hat{θ}_{ML}(x))}{∑_{x'} p(x' | \hat{θ}_{ML}(x'))}$$ - ---- - -## 13. Typicality and AEP - -### 13.1 Asymptotic Equipartition Property - -**Theorem 13.1** (AEP for Ternary i.i.d.). For X₁, ..., Xₙ i.i.d. uniform ternary: -$$-\frac{1}{n} \log_2 p(X_1, ..., X_n) \xrightarrow{P} H(X) = \log_2(3)$$ - -### 13.2 Typical Set - -**Definition 13.1** (Typical Set). -$$A_\epsilon^{(n)} = \{(x_1,...,x_n) : |{-}\frac{1}{n} \log_2 p - H| < \epsilon\}$$ - -**Theorem 13.2** (Typical Set Properties). -1. P(A_ε^{(n)}) → 1 as n → ∞ -2. |A_ε^{(n)}| ≈ 2^{nH} -3. All elements have probability ≈ 2^{-nH} - -For ternary: |A_ε^{(n)}| ≈ 3^n - ---- - -## 14. Information-Theoretic Security - -### 14.1 Perfect Secrecy - -**Theorem 14.1** (Shannon Secrecy). Perfect secrecy requires: -$$H(K) \geq H(M)$$ - -For ternary message space, need ternary key. - -### 14.2 Ternary One-Time Pad - -**Definition 14.1** (Ternary OTP). -$$C = (M + K) \mod 3$$ - -**Theorem 14.2** (Ternary OTP Security). If K ~ Uniform{0,1,2} independent of M: -$$I(M; C) = 0$$ - -Perfect secrecy. - ---- - -## 15. Summary of Key Results - -| Quantity | Ternary Value | Binary Comparison | -|----------|---------------|-------------------| -| Maximum entropy | log₂(3) ≈ 1.585 | 1 bit | -| Entropy rate (i.i.d.) | 1.585 bits/symbol | 1 bit/symbol | -| Optimal code length | ≥ 1.585 bits | ≥ 1 bit | -| TSC capacity (noiseless) | 1.585 bits | 1 bit | -| Fisher information (uniform) | 3 | 4 | - ---- - -## 16. TODOs - -**TODO**: Further development needed: - -1. **Continuous entropy**: Differential entropy for continuous distributions -2. **Multivariate information**: Information in joint ternary distributions -3. **Network information theory**: Multiple ternary sources/channels -4. **Quantum information**: Connection to qutrit systems - ---- - -## References - -1. Cover, T.M. & Thomas, J.A. (2006). *Elements of Information Theory*, 2nd ed. -2. MacKay, D.J.C. (2003). *Information Theory, Inference, and Learning Algorithms* -3. Shannon, C.E. (1948). "A Mathematical Theory of Communication" -4. Rissanen, J. (1978). "Modeling by shortest data description" diff --git a/proofs/theorems/mcmc-inference.adoc b/proofs/theorems/mcmc-inference.adoc new file mode 100644 index 0000000..ee690a3 --- /dev/null +++ b/proofs/theorems/mcmc-inference.adoc @@ -0,0 +1,612 @@ +== MCMC and Inference Correctness Proofs + +=== Abstract + +This document provides rigorous proofs of correctness for the Markov +Chain Monte Carlo (MCMC) and inference methods implemented in betlang. +We establish convergence guarantees, prove detailed balance, and analyze +the statistical properties of the samplers. + +''''' + +=== 1. Metropolis-Hastings Algorithm + +==== 1.1 Algorithm Definition + +*Algorithm 1.1* (Metropolis-Hastings): + +.... +Input: target π, proposal q, initial x₀, iterations n +Output: samples x₁, ..., xₙ + +for i = 1 to n: + y ~ q(·|xᵢ₋₁) # Propose + α = min(1, π(y)q(xᵢ₋₁|y) / (π(xᵢ₋₁)q(y|xᵢ₋₁))) # Accept ratio + u ~ Uniform(0,1) + if u < α: + xᵢ = y # Accept + else: + xᵢ = xᵢ₋₁ # Reject +.... + +==== 1.2 Detailed Balance + +*Definition 1.1* (Detailed Balance). A transition kernel P satisfies +detailed balance w.r.t. π if: + +[latexmath] +++++ +π(x) P(x, y) = π(y) P(y, x) \quad ∀x, y +++++ + +*Theorem 1.1* (MH Satisfies Detailed Balance). The MH transition kernel +satisfies detailed balance with target π. + +_Proof_: The MH kernel is: + +[latexmath] +++++ +P(x, y) = q(y|x) · α(x, y) + r(x) · δ_x(y) +++++ + +where α(x,y) = min(1, π(y)q(x|y)/(π(x)q(y|x))) and r(x) = 1 - ∫ +q(y|x)α(x,y)dy. + +For x ≠ y: + +[latexmath] +++++ +π(x) P(x, y) = π(x) q(y|x) α(x, y) +++++ + +Case 1: α(x,y) = 1 (i.e., π(y)q(x|y) ≥ π(x)q(y|x)): + +[latexmath] +++++ +π(x) P(x, y) = π(x) q(y|x) +++++ + +[latexmath] +++++ +π(y) P(y, x) = π(y) q(x|y) α(y, x) = π(y) q(x|y) · \frac{π(x)q(y|x)}{π(y)q(x|y)} = π(x) q(y|x) +++++ + +Case 2: α(x,y) < 1: + +[latexmath] +++++ +π(x) P(x, y) = π(x) q(y|x) · \frac{π(y)q(x|y)}{π(x)q(y|x)} = π(y) q(x|y) +++++ + +[latexmath] +++++ +π(y) P(y, x) = π(y) q(x|y) · 1 = π(y) q(x|y) +++++ + +Both cases: π(x)P(x,y) = π(y)P(y,x). ∎ + +==== 1.3 Invariance + +*Theorem 1.2* (π is Invariant). If P satisfies detailed balance with π, +then π is invariant: + +[latexmath] +++++ +π P = π +++++ + +_Proof_: + +[latexmath] +++++ +(\pi P)(y) = \int π(x) P(x, y) dx = \int π(y) P(y, x) dx = π(y) \int P(y, x) dx = π(y) · 1 = π(y) ∎ +++++ + +==== 1.4 Ergodicity + +*Theorem 1.3* (MH Ergodicity). If the MH chain is irreducible and +aperiodic, then: + +[latexmath] +++++ +\frac{1}{n} \sum_{i=1}^n f(X_i) \xrightarrow{a.s.} \mathbb{E}_π[f] +++++ + +_Proof_: By the ergodic theorem for Markov chains with invariant +distribution. ∎ + +==== 1.5 Sufficient Conditions for Irreducibility + +*Lemma 1.1* (Irreducibility). MH is irreducible if: + +[latexmath] +++++ +\forall x, y: \exists \text{ path } x = x_0, x_1, ..., x_k = y \text{ with } \prod_i q(x_{i+1}|x_i) > 0 +++++ + +*Lemma 1.2* (Aperiodicity). If q(x|x) > 0 or there exist states with +positive probability of rejection, MH is aperiodic. + +''''' + +=== 2. Gibbs Sampling + +==== 2.1 Algorithm + +*Algorithm 2.1* (Gibbs Sampler for (X, Y)): + +.... +Input: conditionals p(x|y), p(y|x), initial (x₀, y₀), iterations n +Output: samples (x₁, y₁), ..., (xₙ, yₙ) + +for i = 1 to n: + xᵢ ~ p(x | yᵢ₋₁) + yᵢ ~ p(y | xᵢ) +.... + +==== 2.2 Correctness + +*Theorem 2.1* (Gibbs Preserves Joint). If p(x|y) and p(y|x) are the true +conditionals of joint p(x,y), then Gibbs sampling has p(x,y) as +invariant distribution. + +_Proof_: The transition kernel is: + +[latexmath] +++++ +P((x,y), (x',y')) = p(x'|y) · p(y'|x') +++++ + +For invariance, need: + +[latexmath] +++++ +\int\int p(x,y) P((x,y),(x',y')) dx dy = p(x',y') +++++ + +[latexmath] +++++ +\int\int p(x,y) p(x'|y) p(y'|x') dx dy +++++ + +[latexmath] +++++ += \int p(y) p(x'|y) dy · p(y'|x') +++++ +(integrating out x using ∫p(x|y)dx = 1) + +[latexmath] +++++ += p(x') · p(y'|x') +++++ +(using ∫p(x’|y)p(y)dy = p(x’)) + +[latexmath] +++++ += p(x',y') +++++ +∎ + +==== 2.3 Convergence Rate + +*Theorem 2.2* (Geometric Convergence). Under regularity conditions, +Gibbs sampling converges geometrically: + +[latexmath] +++++ +\|P^n((x_0, y_0), ·) - π\|_{TV} \leq C · ρ^n +++++ + +for some ρ < 1. + +''''' + +=== 3. Hamiltonian Monte Carlo + +==== 3.1 Algorithm + +*Algorithm 3.1* (HMC): + +.... +Input: log-prob U, gradient ∇U, initial q₀, step size ε, path length L +Output: samples q₁, ..., qₙ + +for i = 1 to n: + p ~ N(0, I) # Sample momentum + (q', p') = leapfrog(qᵢ₋₁, p, ε, L) # Integrate + α = min(1, exp(-H(q', p') + H(qᵢ₋₁, p))) + u ~ Uniform(0,1) + if u < α: + qᵢ = q' + else: + qᵢ = qᵢ₋₁ +.... + +==== 3.2 Leapfrog Integrator + +*Algorithm 3.2* (Leapfrog): + +.... +p ← p - (ε/2)∇U(q) +for l = 1 to L: + q ← q + ε · p + if l < L: + p ← p - ε∇U(q) +p ← p - (ε/2)∇U(q) +return (q, -p) +.... + +==== 3.3 Symplectic Property + +*Theorem 3.1* (Volume Preservation). Leapfrog is symplectic (preserves +volume in phase space). + +_Proof_: Each step is a shear transformation with unit Jacobian +determinant. Composition preserves this property. ∎ + +==== 3.4 Detailed Balance + +*Theorem 3.2* (HMC Detailed Balance). HMC satisfies detailed balance +with joint distribution: + +[latexmath] +++++ +π(q, p) ∝ \exp(-U(q) - \frac{1}{2}p^T p) +++++ + +_Proof_: 1. Momentum flip: (q, p) ↔ (q, -p) is self-inverse 2. Leapfrog: +approximately preserves H(q,p) = U(q) + K(p) 3. MH correction: accounts +for numerical integration error ∎ + +==== 3.5 No-U-Turn Sampler (NUTS) + +*Theorem 3.3* (NUTS Correctness). NUTS maintains detailed balance while +adaptively choosing path length. + +_Proof sketch_: The doubling procedure and slice sampling maintain +reversibility. ∎ + +''''' + +=== 4. Rejection Sampling + +==== 4.1 Algorithm + +*Algorithm 4.1* (Rejection Sampling): + +.... +Input: target f, proposal g, bound M (where f(x) ≤ M·g(x)) +Output: sample from f + +repeat: + x ~ g + u ~ Uniform(0, 1) +until u < f(x) / (M·g(x)) +return x +.... + +==== 4.2 Correctness + +*Theorem 4.1* (Rejection Sampling Correctness). The output follows +distribution f. + +_Proof_: P(accept x ∈ A) = P(X ∈ A, U < f(X)/(M·g(X)) | accept) + +By Bayes: + +[latexmath] +++++ += \frac{P(X ∈ A, U < f(X)/(M·g(X)))}{P(\text{accept})} +++++ + +[latexmath] +++++ += \frac{\int_A g(x) · \frac{f(x)}{M·g(x)} dx}{1/M} +++++ + +[latexmath] +++++ += \frac{(1/M)\int_A f(x) dx}{1/M} +++++ + +[latexmath] +++++ += \int_A f(x) dx +++++ + +where we used P(accept) = (1/M)∫f(x)dx = 1/M for normalized f. ∎ + +==== 4.3 Efficiency + +*Theorem 4.2* (Acceptance Rate). The acceptance probability is 1/M. + +*Corollary 4.1* (Expected Samples). Expected number of proposals until +acceptance: M. + +''''' + +=== 5. Importance Sampling + +==== 5.1 Estimator + +*Definition 5.1* (Importance Sampling Estimator). For target p, proposal +q: + +[latexmath] +++++ +\hat{I}_{IS} = \frac{1}{n} \sum_{i=1}^n \frac{f(X_i) p(X_i)}{q(X_i)}, \quad X_i \sim q +++++ + +==== 5.2 Unbiasedness + +*Theorem 5.1* (IS Unbiasedness). + +[latexmath] +++++ +\mathbb{E}_q[\hat{I}_{IS}] = \mathbb{E}_p[f] +++++ + +_Proof_: + +[latexmath] +++++ +\mathbb{E}_q\left[\frac{f(X) p(X)}{q(X)}\right] = \int \frac{f(x) p(x)}{q(x)} q(x) dx = \int f(x) p(x) dx = \mathbb{E}_p[f] ∎ +++++ + +==== 5.3 Variance + +*Theorem 5.2* (IS Variance). + +[latexmath] +++++ +\text{Var}_q\left[\frac{f(X)p(X)}{q(X)}\right] = \mathbb{E}_q\left[\left(\frac{f(X)p(X)}{q(X)}\right)^2\right] - \left(\mathbb{E}_p[f]\right)^2 +++++ + +==== 5.4 Optimal Proposal + +*Theorem 5.3* (Optimal IS Proposal). The variance-minimizing proposal +is: + +[latexmath] +++++ +q^*(x) ∝ |f(x)| p(x) +++++ + +_Proof_: By Lagrange multipliers on variance subject to ∫q = 1. ∎ + +''''' + +=== 6. Sequential Monte Carlo + +==== 6.1 Algorithm + +*Algorithm 6.1* (Particle Filter): + +.... +Input: prior p₀, transition p(xₜ|xₜ₋₁), observation p(yₜ|xₜ), observations y₁:T +Output: particles {xₜ^{(i)}} + +Initialize: x₀^{(i)} ~ p₀ for i = 1..N +for t = 1 to T: + Propagate: x̃ₜ^{(i)} ~ p(xₜ | xₜ₋₁^{(i)}) + Weight: wₜ^{(i)} ∝ p(yₜ | x̃ₜ^{(i)}) + Resample: xₜ^{(i)} ~ Categorical({x̃ₜ^{(j)}}, {wₜ^{(j)}}) +.... + +==== 6.2 Correctness + +*Theorem 6.1* (Particle Filter Consistency). As N → ∞: + +[latexmath] +++++ +\sum_{i=1}^N w_t^{(i)} δ_{x_t^{(i)}} \xrightarrow{d} p(x_t | y_{1:t}) +++++ + +_Proof_: By sequential application of importance sampling with +resampling correction. ∎ + +==== 6.3 Effective Sample Size + +*Definition 6.1* (ESS). + +[latexmath] +++++ +\text{ESS} = \frac{(\sum_i w_i)^2}{\sum_i w_i^2} +++++ + +*Theorem 6.2* (ESS Bound). 1 ≤ ESS ≤ N with equality at N iff uniform +weights. + +''''' + +=== 7. Approximate Bayesian Computation + +==== 7.1 Algorithm + +*Algorithm 7.1* (ABC Rejection): + +.... +Input: simulator p(D|θ), prior π(θ), observed D_obs, threshold ε +Output: approximate posterior samples + +repeat: + θ ~ π(θ) + D_sim ~ p(D|θ) +until d(D_sim, D_obs) < ε +return θ +.... + +==== 7.2 Consistency + +*Theorem 7.1* (ABC Consistency). As ε → 0: + +[latexmath] +++++ +π_{ABC}(θ | d(D_{sim}, D_{obs}) < ε) → π(θ | D = D_{obs}) +++++ + +under regularity conditions. + +_Proof_: As ε → 0, the acceptance region shrinks to \{D = D_obs}, +recovering exact posterior. ∎ + +==== 7.3 Approximation Error + +*Theorem 7.2* (ABC Error Bound). The total variation distance: + +[latexmath] +++++ +\|π_{ABC} - π_{true}\|_{TV} = O(ε) +++++ + +for smooth likelihoods. + +''''' + +=== 8. Convergence Diagnostics + +==== 8.1 Potential Scale Reduction Factor + +*Definition 8.1* (R-hat). For m chains of length n: + +[latexmath] +++++ +\hat{R} = \sqrt{\frac{\hat{V}}{W}} +++++ + +where W = within-chain variance, V̂ = pooled variance estimate. + +*Theorem 8.1* (Convergence Criterion). R̂ → 1 as n → ∞ for converged +chains. + +==== 8.2 Effective Sample Size + +*Definition 8.2* (MCMC ESS). + +[latexmath] +++++ +\text{ESS} = \frac{n}{1 + 2\sum_{k=1}^∞ ρ_k} +++++ + +where ρ_k is the lag-k autocorrelation. + +==== 8.3 Geweke Diagnostic + +*Theorem 8.2* (Geweke Test). Under convergence, for first 10% and last +50% of chain: + +[latexmath] +++++ +\frac{\bar{X}_A - \bar{X}_B}{\sqrt{S_A^2/n_A + S_B^2/n_B}} \xrightarrow{d} N(0, 1) +++++ + +''''' + +=== 9. Betlang-Specific Implementation Details + +==== 9.1 metropolis-hastings Implementation + +[source,racket] +---- +(define (metropolis-hastings target-log-density proposal initial n-samples) + (let loop ([current initial] + [samples '()] + [i 0]) + (if (>= i n-samples) + (reverse samples) + (let* ([proposed ((proposal current))] + [log-alpha (- (target-log-density proposed) + (target-log-density current))] + [accept? (< (log (random)) log-alpha)]) + (loop (if accept? proposed current) + (cons (if accept? proposed current) samples) + (+ i 1)))))) +---- + +*Correctness*: Follows Theorem 1.1 (detailed balance) assuming symmetric +proposal. + +==== 9.2 gibbs-sampler Implementation + +[source,racket] +---- +(define (gibbs-sampler conditional-x conditional-y init-x init-y n-samples) + (let loop ([x init-x] [y init-y] [samples '()] [i 0]) + (if (>= i n-samples) + (reverse samples) + (let* ([new-x (conditional-x y)] + [new-y (conditional-y new-x)]) + (loop new-x new-y + (cons (list new-x new-y) samples) + (+ i 1)))))) +---- + +*Correctness*: Follows Theorem 2.1 given correct conditionals. + +''''' + +=== 10. Theoretical Guarantees Summary + +[cols=",,",options="header",] +|=== +|Method |Condition |Guarantee +|MH |Irreducible, aperiodic |Ergodic convergence +|Gibbs |Correct conditionals |Invariant = joint +|HMC |Symplectic integrator |Volume preservation +|Rejection |f ≤ Mg |Exact samples +|IS |q > 0 where p > 0 |Unbiased estimator +|SMC |Proper resampling |Consistent filtering +|ABC |ε → 0 |Posterior convergence +|=== + +''''' + +=== 11. Error Bounds + +==== 11.1 MCMC Standard Error + +*Theorem 11.1* (MCMC CLT). Under geometric ergodicity: + +[latexmath] +++++ +\sqrt{n}(\bar{X}_n - \mu) \xrightarrow{d} N(0, σ^2_{eff}) +++++ + +where σ²_eff = σ²(1 + 2Σρₖ) is the effective variance. + +==== 11.2 Burn-in Selection + +*Theorem 11.2* (Burn-in Bound). To achieve TV distance ε from +stationarity: + +[latexmath] +++++ +B \geq \frac{\log(1/ε)}{\log(1/ρ)} +++++ + +where ρ < 1 is the convergence rate. + +''''' + +=== 12. TODOs + +*TODO*: Further proofs needed: + +[arabic] +. *Adaptive MCMC*: Prove ergodicity of adaptive algorithms +. *Parallel tempering*: Prove mixing time bounds +. *Gradient estimation*: Bias/variance of stochastic gradients +. *Variational inference*: Approximation bounds + +''''' + +=== References + +[arabic] +. Robert, C.P. & Casella, G. (2004). _Monte Carlo Statistical Methods_ +. Brooks, S., et al. (2011). _Handbook of Markov Chain Monte Carlo_ +. Doucet, A., de Freitas, N., & Gordon, N. (2001). _Sequential Monte +Carlo Methods in Practice_ +. Neal, R.M. (2011). "`MCMC using Hamiltonian dynamics`" +. Marin, J.-M., et al. (2012). "`Approximate Bayesian computational +methods`" diff --git a/proofs/theorems/mcmc-inference.md b/proofs/theorems/mcmc-inference.md deleted file mode 100644 index 4b99118..0000000 --- a/proofs/theorems/mcmc-inference.md +++ /dev/null @@ -1,426 +0,0 @@ - -# MCMC and Inference Correctness Proofs - -## Abstract - -This document provides rigorous proofs of correctness for the Markov Chain Monte Carlo (MCMC) and inference methods implemented in betlang. We establish convergence guarantees, prove detailed balance, and analyze the statistical properties of the samplers. - ---- - -## 1. Metropolis-Hastings Algorithm - -### 1.1 Algorithm Definition - -**Algorithm 1.1** (Metropolis-Hastings): -``` -Input: target π, proposal q, initial x₀, iterations n -Output: samples x₁, ..., xₙ - -for i = 1 to n: - y ~ q(·|xᵢ₋₁) # Propose - α = min(1, π(y)q(xᵢ₋₁|y) / (π(xᵢ₋₁)q(y|xᵢ₋₁))) # Accept ratio - u ~ Uniform(0,1) - if u < α: - xᵢ = y # Accept - else: - xᵢ = xᵢ₋₁ # Reject -``` - -### 1.2 Detailed Balance - -**Definition 1.1** (Detailed Balance). A transition kernel P satisfies detailed balance w.r.t. π if: -$$π(x) P(x, y) = π(y) P(y, x) \quad ∀x, y$$ - -**Theorem 1.1** (MH Satisfies Detailed Balance). The MH transition kernel satisfies detailed balance with target π. - -*Proof*: -The MH kernel is: -$$P(x, y) = q(y|x) · α(x, y) + r(x) · δ_x(y)$$ - -where α(x,y) = min(1, π(y)q(x|y)/(π(x)q(y|x))) and r(x) = 1 - ∫ q(y|x)α(x,y)dy. - -For x ≠ y: -$$π(x) P(x, y) = π(x) q(y|x) α(x, y)$$ - -Case 1: α(x,y) = 1 (i.e., π(y)q(x|y) ≥ π(x)q(y|x)): -$$π(x) P(x, y) = π(x) q(y|x)$$ -$$π(y) P(y, x) = π(y) q(x|y) α(y, x) = π(y) q(x|y) · \frac{π(x)q(y|x)}{π(y)q(x|y)} = π(x) q(y|x)$$ - -Case 2: α(x,y) < 1: -$$π(x) P(x, y) = π(x) q(y|x) · \frac{π(y)q(x|y)}{π(x)q(y|x)} = π(y) q(x|y)$$ -$$π(y) P(y, x) = π(y) q(x|y) · 1 = π(y) q(x|y)$$ - -Both cases: π(x)P(x,y) = π(y)P(y,x). ∎ - -### 1.3 Invariance - -**Theorem 1.2** (π is Invariant). If P satisfies detailed balance with π, then π is invariant: -$$π P = π$$ - -*Proof*: -$$(\pi P)(y) = \int π(x) P(x, y) dx = \int π(y) P(y, x) dx = π(y) \int P(y, x) dx = π(y) · 1 = π(y) ∎$$ - -### 1.4 Ergodicity - -**Theorem 1.3** (MH Ergodicity). If the MH chain is irreducible and aperiodic, then: -$$\frac{1}{n} \sum_{i=1}^n f(X_i) \xrightarrow{a.s.} \mathbb{E}_π[f]$$ - -*Proof*: By the ergodic theorem for Markov chains with invariant distribution. ∎ - -### 1.5 Sufficient Conditions for Irreducibility - -**Lemma 1.1** (Irreducibility). MH is irreducible if: -$$\forall x, y: \exists \text{ path } x = x_0, x_1, ..., x_k = y \text{ with } \prod_i q(x_{i+1}|x_i) > 0$$ - -**Lemma 1.2** (Aperiodicity). If q(x|x) > 0 or there exist states with positive probability of rejection, MH is aperiodic. - ---- - -## 2. Gibbs Sampling - -### 2.1 Algorithm - -**Algorithm 2.1** (Gibbs Sampler for (X, Y)): -``` -Input: conditionals p(x|y), p(y|x), initial (x₀, y₀), iterations n -Output: samples (x₁, y₁), ..., (xₙ, yₙ) - -for i = 1 to n: - xᵢ ~ p(x | yᵢ₋₁) - yᵢ ~ p(y | xᵢ) -``` - -### 2.2 Correctness - -**Theorem 2.1** (Gibbs Preserves Joint). If p(x|y) and p(y|x) are the true conditionals of joint p(x,y), then Gibbs sampling has p(x,y) as invariant distribution. - -*Proof*: -The transition kernel is: -$$P((x,y), (x',y')) = p(x'|y) · p(y'|x')$$ - -For invariance, need: -$$\int\int p(x,y) P((x,y),(x',y')) dx dy = p(x',y')$$ - -$$\int\int p(x,y) p(x'|y) p(y'|x') dx dy$$ -$$= \int p(y) p(x'|y) dy · p(y'|x')$$ (integrating out x using ∫p(x|y)dx = 1) -$$= p(x') · p(y'|x')$$ (using ∫p(x'|y)p(y)dy = p(x')) -$$= p(x',y')$$ ∎ - -### 2.3 Convergence Rate - -**Theorem 2.2** (Geometric Convergence). Under regularity conditions, Gibbs sampling converges geometrically: -$$\|P^n((x_0, y_0), ·) - π\|_{TV} \leq C · ρ^n$$ - -for some ρ < 1. - ---- - -## 3. Hamiltonian Monte Carlo - -### 3.1 Algorithm - -**Algorithm 3.1** (HMC): -``` -Input: log-prob U, gradient ∇U, initial q₀, step size ε, path length L -Output: samples q₁, ..., qₙ - -for i = 1 to n: - p ~ N(0, I) # Sample momentum - (q', p') = leapfrog(qᵢ₋₁, p, ε, L) # Integrate - α = min(1, exp(-H(q', p') + H(qᵢ₋₁, p))) - u ~ Uniform(0,1) - if u < α: - qᵢ = q' - else: - qᵢ = qᵢ₋₁ -``` - -### 3.2 Leapfrog Integrator - -**Algorithm 3.2** (Leapfrog): -``` -p ← p - (ε/2)∇U(q) -for l = 1 to L: - q ← q + ε · p - if l < L: - p ← p - ε∇U(q) -p ← p - (ε/2)∇U(q) -return (q, -p) -``` - -### 3.3 Symplectic Property - -**Theorem 3.1** (Volume Preservation). Leapfrog is symplectic (preserves volume in phase space). - -*Proof*: Each step is a shear transformation with unit Jacobian determinant. Composition preserves this property. ∎ - -### 3.4 Detailed Balance - -**Theorem 3.2** (HMC Detailed Balance). HMC satisfies detailed balance with joint distribution: -$$π(q, p) ∝ \exp(-U(q) - \frac{1}{2}p^T p)$$ - -*Proof*: -1. Momentum flip: (q, p) ↔ (q, -p) is self-inverse -2. Leapfrog: approximately preserves H(q,p) = U(q) + K(p) -3. MH correction: accounts for numerical integration error ∎ - -### 3.5 No-U-Turn Sampler (NUTS) - -**Theorem 3.3** (NUTS Correctness). NUTS maintains detailed balance while adaptively choosing path length. - -*Proof sketch*: The doubling procedure and slice sampling maintain reversibility. ∎ - ---- - -## 4. Rejection Sampling - -### 4.1 Algorithm - -**Algorithm 4.1** (Rejection Sampling): -``` -Input: target f, proposal g, bound M (where f(x) ≤ M·g(x)) -Output: sample from f - -repeat: - x ~ g - u ~ Uniform(0, 1) -until u < f(x) / (M·g(x)) -return x -``` - -### 4.2 Correctness - -**Theorem 4.1** (Rejection Sampling Correctness). The output follows distribution f. - -*Proof*: -P(accept x ∈ A) = P(X ∈ A, U < f(X)/(M·g(X)) | accept) - -By Bayes: -$$= \frac{P(X ∈ A, U < f(X)/(M·g(X)))}{P(\text{accept})}$$ -$$= \frac{\int_A g(x) · \frac{f(x)}{M·g(x)} dx}{1/M}$$ -$$= \frac{(1/M)\int_A f(x) dx}{1/M}$$ -$$= \int_A f(x) dx$$ - -where we used P(accept) = (1/M)∫f(x)dx = 1/M for normalized f. ∎ - -### 4.3 Efficiency - -**Theorem 4.2** (Acceptance Rate). The acceptance probability is 1/M. - -**Corollary 4.1** (Expected Samples). Expected number of proposals until acceptance: M. - ---- - -## 5. Importance Sampling - -### 5.1 Estimator - -**Definition 5.1** (Importance Sampling Estimator). For target p, proposal q: -$$\hat{I}_{IS} = \frac{1}{n} \sum_{i=1}^n \frac{f(X_i) p(X_i)}{q(X_i)}, \quad X_i \sim q$$ - -### 5.2 Unbiasedness - -**Theorem 5.1** (IS Unbiasedness). -$$\mathbb{E}_q[\hat{I}_{IS}] = \mathbb{E}_p[f]$$ - -*Proof*: -$$\mathbb{E}_q\left[\frac{f(X) p(X)}{q(X)}\right] = \int \frac{f(x) p(x)}{q(x)} q(x) dx = \int f(x) p(x) dx = \mathbb{E}_p[f] ∎$$ - -### 5.3 Variance - -**Theorem 5.2** (IS Variance). -$$\text{Var}_q\left[\frac{f(X)p(X)}{q(X)}\right] = \mathbb{E}_q\left[\left(\frac{f(X)p(X)}{q(X)}\right)^2\right] - \left(\mathbb{E}_p[f]\right)^2$$ - -### 5.4 Optimal Proposal - -**Theorem 5.3** (Optimal IS Proposal). The variance-minimizing proposal is: -$$q^*(x) ∝ |f(x)| p(x)$$ - -*Proof*: By Lagrange multipliers on variance subject to ∫q = 1. ∎ - ---- - -## 6. Sequential Monte Carlo - -### 6.1 Algorithm - -**Algorithm 6.1** (Particle Filter): -``` -Input: prior p₀, transition p(xₜ|xₜ₋₁), observation p(yₜ|xₜ), observations y₁:T -Output: particles {xₜ^{(i)}} - -Initialize: x₀^{(i)} ~ p₀ for i = 1..N -for t = 1 to T: - Propagate: x̃ₜ^{(i)} ~ p(xₜ | xₜ₋₁^{(i)}) - Weight: wₜ^{(i)} ∝ p(yₜ | x̃ₜ^{(i)}) - Resample: xₜ^{(i)} ~ Categorical({x̃ₜ^{(j)}}, {wₜ^{(j)}}) -``` - -### 6.2 Correctness - -**Theorem 6.1** (Particle Filter Consistency). As N → ∞: -$$\sum_{i=1}^N w_t^{(i)} δ_{x_t^{(i)}} \xrightarrow{d} p(x_t | y_{1:t})$$ - -*Proof*: By sequential application of importance sampling with resampling correction. ∎ - -### 6.3 Effective Sample Size - -**Definition 6.1** (ESS). -$$\text{ESS} = \frac{(\sum_i w_i)^2}{\sum_i w_i^2}$$ - -**Theorem 6.2** (ESS Bound). 1 ≤ ESS ≤ N with equality at N iff uniform weights. - ---- - -## 7. Approximate Bayesian Computation - -### 7.1 Algorithm - -**Algorithm 7.1** (ABC Rejection): -``` -Input: simulator p(D|θ), prior π(θ), observed D_obs, threshold ε -Output: approximate posterior samples - -repeat: - θ ~ π(θ) - D_sim ~ p(D|θ) -until d(D_sim, D_obs) < ε -return θ -``` - -### 7.2 Consistency - -**Theorem 7.1** (ABC Consistency). As ε → 0: -$$π_{ABC}(θ | d(D_{sim}, D_{obs}) < ε) → π(θ | D = D_{obs})$$ - -under regularity conditions. - -*Proof*: As ε → 0, the acceptance region shrinks to {D = D_obs}, recovering exact posterior. ∎ - -### 7.3 Approximation Error - -**Theorem 7.2** (ABC Error Bound). The total variation distance: -$$\|π_{ABC} - π_{true}\|_{TV} = O(ε)$$ - -for smooth likelihoods. - ---- - -## 8. Convergence Diagnostics - -### 8.1 Potential Scale Reduction Factor - -**Definition 8.1** (R-hat). For m chains of length n: -$$\hat{R} = \sqrt{\frac{\hat{V}}{W}}$$ - -where W = within-chain variance, V̂ = pooled variance estimate. - -**Theorem 8.1** (Convergence Criterion). R̂ → 1 as n → ∞ for converged chains. - -### 8.2 Effective Sample Size - -**Definition 8.2** (MCMC ESS). -$$\text{ESS} = \frac{n}{1 + 2\sum_{k=1}^∞ ρ_k}$$ - -where ρ_k is the lag-k autocorrelation. - -### 8.3 Geweke Diagnostic - -**Theorem 8.2** (Geweke Test). Under convergence, for first 10% and last 50% of chain: -$$\frac{\bar{X}_A - \bar{X}_B}{\sqrt{S_A^2/n_A + S_B^2/n_B}} \xrightarrow{d} N(0, 1)$$ - ---- - -## 9. Betlang-Specific Implementation Details - -### 9.1 metropolis-hastings Implementation - -```racket -(define (metropolis-hastings target-log-density proposal initial n-samples) - (let loop ([current initial] - [samples '()] - [i 0]) - (if (>= i n-samples) - (reverse samples) - (let* ([proposed ((proposal current))] - [log-alpha (- (target-log-density proposed) - (target-log-density current))] - [accept? (< (log (random)) log-alpha)]) - (loop (if accept? proposed current) - (cons (if accept? proposed current) samples) - (+ i 1)))))) -``` - -**Correctness**: Follows Theorem 1.1 (detailed balance) assuming symmetric proposal. - -### 9.2 gibbs-sampler Implementation - -```racket -(define (gibbs-sampler conditional-x conditional-y init-x init-y n-samples) - (let loop ([x init-x] [y init-y] [samples '()] [i 0]) - (if (>= i n-samples) - (reverse samples) - (let* ([new-x (conditional-x y)] - [new-y (conditional-y new-x)]) - (loop new-x new-y - (cons (list new-x new-y) samples) - (+ i 1)))))) -``` - -**Correctness**: Follows Theorem 2.1 given correct conditionals. - ---- - -## 10. Theoretical Guarantees Summary - -| Method | Condition | Guarantee | -|--------|-----------|-----------| -| MH | Irreducible, aperiodic | Ergodic convergence | -| Gibbs | Correct conditionals | Invariant = joint | -| HMC | Symplectic integrator | Volume preservation | -| Rejection | f ≤ Mg | Exact samples | -| IS | q > 0 where p > 0 | Unbiased estimator | -| SMC | Proper resampling | Consistent filtering | -| ABC | ε → 0 | Posterior convergence | - ---- - -## 11. Error Bounds - -### 11.1 MCMC Standard Error - -**Theorem 11.1** (MCMC CLT). Under geometric ergodicity: -$$\sqrt{n}(\bar{X}_n - \mu) \xrightarrow{d} N(0, σ^2_{eff})$$ - -where σ²_eff = σ²(1 + 2Σρₖ) is the effective variance. - -### 11.2 Burn-in Selection - -**Theorem 11.2** (Burn-in Bound). To achieve TV distance ε from stationarity: -$$B \geq \frac{\log(1/ε)}{\log(1/ρ)}$$ - -where ρ < 1 is the convergence rate. - ---- - -## 12. TODOs - -**TODO**: Further proofs needed: - -1. **Adaptive MCMC**: Prove ergodicity of adaptive algorithms -2. **Parallel tempering**: Prove mixing time bounds -3. **Gradient estimation**: Bias/variance of stochastic gradients -4. **Variational inference**: Approximation bounds - ---- - -## References - -1. Robert, C.P. & Casella, G. (2004). *Monte Carlo Statistical Methods* -2. Brooks, S., et al. (2011). *Handbook of Markov Chain Monte Carlo* -3. Doucet, A., de Freitas, N., & Gordon, N. (2001). *Sequential Monte Carlo Methods in Practice* -4. Neal, R.M. (2011). "MCMC using Hamiltonian dynamics" -5. Marin, J.-M., et al. (2012). "Approximate Bayesian computational methods" diff --git a/proofs/theorems/measure-theory.adoc b/proofs/theorems/measure-theory.adoc new file mode 100644 index 0000000..5535699 --- /dev/null +++ b/proofs/theorems/measure-theory.adoc @@ -0,0 +1,612 @@ +== Measure-Theoretic Foundations of Betlang + +=== Abstract + +This document provides the rigorous measure-theoretic foundations +underlying betlang’s probability theory. We develop the theory from +σ-algebras through the Lebesgue integral, establishing the mathematical +framework for both discrete and continuous probability in probabilistic +programming. + +''''' + +=== 1. Measurable Spaces + +==== 1.1 σ-Algebra + +*Definition 1.1* (σ-Algebra). A σ-algebra F on set Ω is a collection of +subsets satisfying: 1. Ω ∈ F 2. A ∈ F ⟹ Aᶜ ∈ F (closure under +complement) 3. A₁, A₂, … ∈ F ⟹ ∪ᵢAᵢ ∈ F (closure under countable union) + +*Definition 1.2* (Measurable Space). A measurable space is a pair (Ω, +F). + +==== 1.2 Ternary Measurable Space + +*Definition 1.3* (Ternary Space). For `+(bet A B C)+`: + +[latexmath] +++++ +Ω = \{A, B, C\} +++++ + +[latexmath] +++++ +\mathcal{F} = \mathcal{P}(Ω) = 2^Ω +++++ + +This is the discrete σ-algebra (power set). + +*Properties*: - |F| = 2³ = 8 - Every subset is measurable - Generated by +singletons: F = σ(\{A}, \{B}, \{C}) + +==== 1.3 Product σ-Algebra + +*Definition 1.4* (Product Space). For spaces (Ω₁, F₁) and (Ω₂, F₂): + +[latexmath] +++++ +Ω_1 × Ω_2, \quad \mathcal{F}_1 ⊗ \mathcal{F}_2 = σ(\{A × B : A ∈ \mathcal{F}_1, B ∈ \mathcal{F}_2\}) +++++ + +*Application*: Joint distribution of independent bets. + +''''' + +=== 2. Measure Theory + +==== 2.1 Measure + +*Definition 2.1* (Measure). A measure μ on (Ω, F) is a function μ: F → +[0, ∞] satisfying: 1. μ(∅) = 0 2. Countable additivity: For disjoint Aᵢ: +μ(∪ᵢAᵢ) = Σᵢμ(Aᵢ) + +*Definition 2.2* (Probability Measure). A probability measure P +satisfies P(Ω) = 1. + +==== 2.2 Ternary Probability Measure + +*Definition 2.3* (Uniform Ternary Measure). For (Ω, F) with Ω = \{A, B, +C}: + +[latexmath] +++++ +P(\{x\}) = \frac{1}{3} \quad ∀x ∈ Ω +++++ + +*Theorem 2.1* (Kolmogorov Axioms). This measure satisfies: 1. P(E) ≥ 0 +for all E ∈ F 2. P(Ω) = 1 3. For disjoint E₁, E₂, …: P(∪ᵢEᵢ) = Σᵢ P(Eᵢ) + +_Proof_: 1. Sum of non-negative terms is non-negative 2. P(Ω) = P(\{A}) ++ P(\{B}) + P(\{C}) = 1/3 + 1/3 + 1/3 = 1 3. For finite case in discrete +space, follows from definition ∎ + +==== 2.3 Weighted Measure + +*Definition 2.4* (Weighted Ternary Measure). For weights w = (wₐ, w_b, +w_c): + +[latexmath] +++++ +P(\{x\}) = \frac{w_x}{\sum_y w_y} +++++ + +*Theorem 2.2* (Well-Defined Measure). This defines a probability measure +iff: 1. wₓ ≥ 0 for all x 2. Σₓwₓ > 0 + +''''' + +=== 3. Random Variables + +==== 3.1 Definition + +*Definition 3.1* (Random Variable). A random variable X on (Ω, F, P) is +a measurable function: + +[latexmath] +++++ +X: Ω → E +++++ + +where (E, ε) is a measurable space, and X⁻¹(B) ∈ F for all B ∈ ε. + +==== 3.2 Bet as Random Variable + +*Definition 3.2* (Bet Random Variable). For `+(bet A B C)+`: + +[latexmath] +++++ +X: Ω → E +++++ + +[latexmath] +++++ +X(ω) = ω +++++ + +where E = \{A, B, C} with discrete σ-algebra. + +==== 3.3 Distribution + +*Definition 3.3* (Distribution/Law). The distribution of X is the +pushforward measure: + +[latexmath] +++++ +P_X(B) = P(X^{-1}(B)) = P(\{ω : X(ω) ∈ B\}) +++++ + +For uniform bet: Pₓ = Uniform(\{A, B, C}). + +==== 3.4 Measurability of Compositions + +*Theorem 3.1* (Composition Measurability). If X is measurable and f is +measurable, then f ∘ X is measurable. + +_Proof_: + +[latexmath] +++++ +(f ∘ X)^{-1}(B) = X^{-1}(f^{-1}(B)) ∈ \mathcal{F} +++++ + +since f⁻¹(B) ∈ ε and X is measurable. ∎ + +*Application*: `+bet-map+` preserves measurability. + +''''' + +=== 4. Integration + +==== 4.1 Simple Functions + +*Definition 4.1* (Simple Function). A simple function is: + +[latexmath] +++++ +s = \sum_{i=1}^n a_i \mathbf{1}_{A_i} +++++ + +where 1_\{Aᵢ} is the indicator function. + +*Definition 4.2* (Integral of Simple Function). + +[latexmath] +++++ +\int s \, dP = \sum_{i=1}^n a_i P(A_i) +++++ + +==== 4.2 Lebesgue Integral + +*Definition 4.3* (Lebesgue Integral). For non-negative measurable f: + +[latexmath] +++++ +\int f \, dP = \sup\left\{\int s \, dP : s \text{ simple}, s ≤ f\right\} +++++ + +For general f: ∫f dP = ∫f⁺ dP - ∫f⁻ dP where f = f⁺ - f⁻. + +==== 4.3 Expectation + +*Definition 4.4* (Expectation). + +[latexmath] +++++ +\mathbb{E}[X] = \int_Ω X(ω) \, dP(ω) +++++ + +*Theorem 4.1* (Ternary Expectation). For uniform `+(bet A B C)+` with +numeric outcomes: + +[latexmath] +++++ +\mathbb{E}[X] = \frac{A + B + C}{3} +++++ + +_Proof_: + +[latexmath] +++++ +\mathbb{E}[X] = \sum_{x ∈ Ω} x · P(\{x\}) = A · \frac{1}{3} + B · \frac{1}{3} + C · \frac{1}{3} = \frac{A+B+C}{3} ∎ +++++ + +==== 4.4 Properties of Integration + +*Theorem 4.2* (Linearity). + +[latexmath] +++++ +\mathbb{E}[aX + bY] = a\mathbb{E}[X] + b\mathbb{E}[Y] +++++ + +*Theorem 4.3* (Monotone Convergence). If 0 ≤ Xₙ ↑ X: + +[latexmath] +++++ +\mathbb{E}[X_n] ↑ \mathbb{E}[X] +++++ + +*Theorem 4.4* (Dominated Convergence). If Xₙ → X and |Xₙ| ≤ Y with E[Y] +< ∞: + +[latexmath] +++++ +\mathbb{E}[X_n] → \mathbb{E}[X] +++++ + +''''' + +=== 5. Product Measures + +==== 5.1 Definition + +*Definition 5.1* (Product Measure). For (Ω₁, F₁, P₁) and (Ω₂, F₂, P₂): + +[latexmath] +++++ +(P_1 × P_2)(A × B) = P_1(A) · P_2(B) +++++ + +Extended to F₁ ⊗ F₂ by Carathéodory. + +==== 5.2 Independence + +*Definition 5.2* (Independence). X and Y are independent if: + +[latexmath] +++++ +P(X ∈ A, Y ∈ B) = P(X ∈ A) · P(Y ∈ B) +++++ + +for all measurable A, B. + +*Theorem 5.1* (bet-parallel Independence). In +`+(bet-parallel n A B C)+`, the n trials are independent. + +_Proof_: Each trial samples independently from the probability space. ∎ + +==== 5.3 Fubini’s Theorem + +*Theorem 5.2* (Fubini). For integrable f on product space: + +[latexmath] +++++ +\int_{Ω_1 × Ω_2} f \, d(P_1 × P_2) = \int_{Ω_1} \left(\int_{Ω_2} f(x,y) \, dP_2(y)\right) dP_1(x) +++++ + +*Application*: Computing expectations of functions of independent bets. + +''''' + +=== 6. Conditional Probability + +==== 6.1 Definition + +*Definition 6.1* (Conditional Probability). + +[latexmath] +++++ +P(A | B) = \frac{P(A ∩ B)}{P(B)} +++++ + +for P(B) > 0. + +==== 6.2 Conditional Expectation + +*Definition 6.2* (Conditional Expectation). E[X | G] for sub-σ-algebra G +is the G-measurable random variable satisfying: + +[latexmath] +++++ +\int_A \mathbb{E}[X | \mathcal{G}] \, dP = \int_A X \, dP \quad ∀A ∈ \mathcal{G} +++++ + +==== 6.3 Properties + +*Theorem 6.1* (Tower Property). + +[latexmath] +++++ +\mathbb{E}[\mathbb{E}[X | \mathcal{G}]] = \mathbb{E}[X] +++++ + +*Theorem 6.2* (Independence). If X ⊥ G: + +[latexmath] +++++ +\mathbb{E}[X | \mathcal{G}] = \mathbb{E}[X] +++++ + +''''' + +=== 7. Characteristic Functions + +==== 7.1 Definition + +*Definition 7.1* (Characteristic Function). + +[latexmath] +++++ +\phi_X(t) = \mathbb{E}[e^{itX}] +++++ + +==== 7.2 Ternary Characteristic Function + +*Theorem 7.1* (Uniform Ternary CF). For uniform bet on \{-1, 0, 1}: + +[latexmath] +++++ +\phi_X(t) = \frac{1}{3}(e^{-it} + 1 + e^{it}) = \frac{1 + 2\cos(t)}{3} +++++ + +==== 7.3 Uniqueness + +*Theorem 7.2* (Uniqueness). The characteristic function uniquely +determines the distribution. + +''''' + +=== 8. Convergence + +==== 8.1 Types of Convergence + +*Definition 8.1* (Convergence Types). 1. *Almost sure*: P(Xₙ → X) = 1 2. +*In probability*: ∀ε > 0: P(|Xₙ - X| > ε) → 0 3. *In distribution*: +E[f(Xₙ)] → E[f(X)] for bounded continuous f 4. *In Lᵖ*: E[|Xₙ - X|ᵖ] → 0 + +==== 8.2 Relationships + +*Theorem 8.1* (Convergence Hierarchy). + +[latexmath] +++++ +\text{a.s.} ⟹ \text{in prob.} ⟹ \text{in dist.} +++++ + +[latexmath] +++++ +L^p ⟹ \text{in prob.} \text{ (for } p ≥ 1\text{)} +++++ + +==== 8.3 SLLN (Measure-Theoretic) + +*Theorem 8.2* (Strong Law). For i.i.d. Xᵢ with E[|X|] < ∞: + +[latexmath] +++++ +\frac{1}{n}\sum_{i=1}^n X_i \xrightarrow{a.s.} \mathbb{E}[X] +++++ + +_Proof_: By Kolmogorov’s strong law using truncation and Borel-Cantelli. +∎ + +''''' + +=== 9. Radon-Nikodym Theorem + +==== 9.1 Absolute Continuity + +*Definition 9.1* (Absolute Continuity). μ ≪ ν if ν(A) = 0 ⟹ μ(A) = 0. + +==== 9.2 Radon-Nikodym Derivative + +*Theorem 9.1* (Radon-Nikodym). If μ ≪ ν for σ-finite measures, ∃ +measurable f ≥ 0: + +[latexmath] +++++ +μ(A) = \int_A f \, dν +++++ + +f is the Radon-Nikodym derivative dμ/dν. + +==== 9.3 Application: Importance Sampling + +For target P and proposal Q with P ≪ Q: + +[latexmath] +++++ +\frac{dP}{dQ}(x) = \frac{p(x)}{q(x)} +++++ + +is the importance weight. + +''''' + +=== 10. Probability Kernels + +==== 10.1 Definition + +*Definition 10.1* (Markov Kernel). A Markov kernel from (Ω₁, F₁) to (Ω₂, +F₂) is: + +[latexmath] +++++ +K: Ω_1 × \mathcal{F}_2 → [0, 1] +++++ + +such that: 1. K(ω, ·) is a probability measure for each ω 2. K(·, B) is +measurable for each B + +==== 10.2 Composition + +*Definition 10.2* (Kernel Composition). + +[latexmath] +++++ +(K_1 K_2)(x, C) = \int K_1(x, dy) K_2(y, C) +++++ + +==== 10.3 Betlang Stochastic Functions + +*Theorem 10.1*. A betlang function f: A → Dist(B) is a Markov kernel +from (A, P(A)) to (B, P(B)). + +''''' + +=== 11. Polish Spaces and Regular Conditional Distributions + +==== 11.1 Polish Space + +*Definition 11.1* (Polish Space). A topological space that is: - +Separable (has countable dense subset) - Completely metrizable + +==== 11.2 Regular Conditional Distribution + +*Theorem 11.1* (Existence). On Polish spaces, regular conditional +distributions exist. + +For random variable X and σ-algebra G, there exists kernel P^X|G such +that: + +[latexmath] +++++ +P^{X|\mathcal{G}}(ω, B) = P(X ∈ B | \mathcal{G})(ω) \text{ a.s.} +++++ + +''''' + +=== 12. Giry Monad (Measure-Theoretic) + +==== 12.1 Definition + +*Definition 12.1* (Giry Monad on Meas). - Objects: Measurable spaces (X, +Σ) - G(X) = probability measures on X with weak topology - η_X: x ↦ δ_x +(Dirac) - μ_X: ∫ P dΦ for Φ ∈ G(G(X)) + +==== 12.2 Relationship to Betlang + +*Theorem 12.1*. The discrete Dist functor is the restriction of Giry to +finite discrete spaces with counting measure. + +''''' + +=== 13. Continuous Distributions in Betlang + +==== 13.1 Lebesgue Measure + +*Definition 13.1* (Lebesgue Measure). On (ℝ, B(ℝ)): + +[latexmath] +++++ +λ([a, b]) = b - a +++++ + +Extended to Borel σ-algebra. + +==== 13.2 Density + +*Definition 13.2* (Probability Density). f is a density for P w.r.t. λ +if: + +[latexmath] +++++ +P(A) = \int_A f(x) \, dλ(x) +++++ + +==== 13.3 Betlang Continuous Distributions + +*Normal*: + +[latexmath] +++++ +f(x) = \frac{1}{\sqrt{2πσ^2}} \exp\left(-\frac{(x-μ)^2}{2σ^2}\right) +++++ + +*Exponential*: + +[latexmath] +++++ +f(x) = λ e^{-λx} \mathbf{1}_{x ≥ 0} +++++ + +''''' + +=== 14. Martingales + +==== 14.1 Definition + +*Definition 14.1* (Martingale). A sequence (Xₙ, Fₙ) is a martingale if: +1. Xₙ is Fₙ-measurable 2. E[|Xₙ|] < ∞ 3. E[Xₙ₊₁ | Fₙ] = Xₙ a.s. + +==== 14.2 Optional Stopping + +*Theorem 14.1* (Optional Stopping). For bounded stopping time τ: + +[latexmath] +++++ +\mathbb{E}[X_τ] = \mathbb{E}[X_0] +++++ + +==== 14.3 Application: bet-until Analysis + +For centered bet-until sum Sₙ = Σ(Xᵢ - E[Xᵢ]): - (Sₙ, Fₙ) is a +martingale - Optional stopping applies to analyze hitting times + +''''' + +=== 15. Weak Convergence + +==== 15.1 Definition + +*Definition 15.1* (Weak Convergence). Pₙ ⇒ P if: + +[latexmath] +++++ +\int f \, dP_n → \int f \, dP +++++ + +for all bounded continuous f. + +==== 15.2 Portmanteau Theorem + +*Theorem 15.1* (Portmanteau). TFAE: 1. Pₙ ⇒ P 2. ∫f dPₙ → ∫f dP for +bounded Lipschitz f 3. lim sup Pₙ(F) ≤ P(F) for closed F 4. lim inf +Pₙ(G) ≥ P(G) for open G + +==== 15.3 Prokhorov’s Theorem + +*Theorem 15.2* (Prokhorov). On Polish spaces, tightness ⟺ relative +compactness in weak topology. + +''''' + +=== 16. Summary: Measure Theory in Betlang + +[cols=",",options="header",] +|=== +|Concept |Betlang Realization +|σ-algebra |Power set (discrete) +|Measure |Probability distribution +|Random variable |bet expression +|Integration |Expectation (bet-expect) +|Product measure |Independent bets +|Kernel |Stochastic function +|Density |PDF for continuous distributions +|=== + +''''' + +=== 17. TODOs + +*TODO*: The following need measure-theoretic development: + +[arabic] +. *Disintegration theorem*: For conditional distributions +. *Prokhorov metric*: For convergence of distributions +. *Central limit theorem*: Measure-theoretic proof +. *Ergodic theory*: For MCMC convergence +. *Optimal transport*: Wasserstein distances + +''''' + +=== References + +[arabic] +. Billingsley, P. (1995). _Probability and Measure_, 3rd ed. +. Durrett, R. (2019). _Probability: Theory and Examples_, 5th ed. +. Klenke, A. (2014). _Probability Theory: A Comprehensive Course_, 2nd +ed. +. Williams, D. (1991). _Probability with Martingales_ +. Giry, M. (1982). "`A categorical approach to probability theory`" diff --git a/proofs/theorems/measure-theory.md b/proofs/theorems/measure-theory.md deleted file mode 100644 index fcebe23..0000000 --- a/proofs/theorems/measure-theory.md +++ /dev/null @@ -1,436 +0,0 @@ - -# Measure-Theoretic Foundations of Betlang - -## Abstract - -This document provides the rigorous measure-theoretic foundations underlying betlang's probability theory. We develop the theory from σ-algebras through the Lebesgue integral, establishing the mathematical framework for both discrete and continuous probability in probabilistic programming. - ---- - -## 1. Measurable Spaces - -### 1.1 σ-Algebra - -**Definition 1.1** (σ-Algebra). A σ-algebra F on set Ω is a collection of subsets satisfying: -1. Ω ∈ F -2. A ∈ F ⟹ Aᶜ ∈ F (closure under complement) -3. A₁, A₂, ... ∈ F ⟹ ∪ᵢAᵢ ∈ F (closure under countable union) - -**Definition 1.2** (Measurable Space). A measurable space is a pair (Ω, F). - -### 1.2 Ternary Measurable Space - -**Definition 1.3** (Ternary Space). For `(bet A B C)`: -$$Ω = \{A, B, C\}$$ -$$\mathcal{F} = \mathcal{P}(Ω) = 2^Ω$$ - -This is the discrete σ-algebra (power set). - -**Properties**: -- |F| = 2³ = 8 -- Every subset is measurable -- Generated by singletons: F = σ({A}, {B}, {C}) - -### 1.3 Product σ-Algebra - -**Definition 1.4** (Product Space). For spaces (Ω₁, F₁) and (Ω₂, F₂): -$$Ω_1 × Ω_2, \quad \mathcal{F}_1 ⊗ \mathcal{F}_2 = σ(\{A × B : A ∈ \mathcal{F}_1, B ∈ \mathcal{F}_2\})$$ - -**Application**: Joint distribution of independent bets. - ---- - -## 2. Measure Theory - -### 2.1 Measure - -**Definition 2.1** (Measure). A measure μ on (Ω, F) is a function μ: F → [0, ∞] satisfying: -1. μ(∅) = 0 -2. Countable additivity: For disjoint Aᵢ: μ(∪ᵢAᵢ) = Σᵢμ(Aᵢ) - -**Definition 2.2** (Probability Measure). A probability measure P satisfies P(Ω) = 1. - -### 2.2 Ternary Probability Measure - -**Definition 2.3** (Uniform Ternary Measure). For (Ω, F) with Ω = {A, B, C}: -$$P(\{x\}) = \frac{1}{3} \quad ∀x ∈ Ω$$ - -**Theorem 2.1** (Kolmogorov Axioms). This measure satisfies: -1. P(E) ≥ 0 for all E ∈ F -2. P(Ω) = 1 -3. For disjoint E₁, E₂, ...: P(∪ᵢEᵢ) = Σᵢ P(Eᵢ) - -*Proof*: -1. Sum of non-negative terms is non-negative -2. P(Ω) = P({A}) + P({B}) + P({C}) = 1/3 + 1/3 + 1/3 = 1 -3. For finite case in discrete space, follows from definition ∎ - -### 2.3 Weighted Measure - -**Definition 2.4** (Weighted Ternary Measure). For weights w = (wₐ, w_b, w_c): -$$P(\{x\}) = \frac{w_x}{\sum_y w_y}$$ - -**Theorem 2.2** (Well-Defined Measure). This defines a probability measure iff: -1. wₓ ≥ 0 for all x -2. Σₓwₓ > 0 - ---- - -## 3. Random Variables - -### 3.1 Definition - -**Definition 3.1** (Random Variable). A random variable X on (Ω, F, P) is a measurable function: -$$X: Ω → E$$ - -where (E, ε) is a measurable space, and X⁻¹(B) ∈ F for all B ∈ ε. - -### 3.2 Bet as Random Variable - -**Definition 3.2** (Bet Random Variable). For `(bet A B C)`: -$$X: Ω → E$$ -$$X(ω) = ω$$ - -where E = {A, B, C} with discrete σ-algebra. - -### 3.3 Distribution - -**Definition 3.3** (Distribution/Law). The distribution of X is the pushforward measure: -$$P_X(B) = P(X^{-1}(B)) = P(\{ω : X(ω) ∈ B\})$$ - -For uniform bet: Pₓ = Uniform({A, B, C}). - -### 3.4 Measurability of Compositions - -**Theorem 3.1** (Composition Measurability). If X is measurable and f is measurable, then f ∘ X is measurable. - -*Proof*: -$$(f ∘ X)^{-1}(B) = X^{-1}(f^{-1}(B)) ∈ \mathcal{F}$$ - -since f⁻¹(B) ∈ ε and X is measurable. ∎ - -**Application**: `bet-map` preserves measurability. - ---- - -## 4. Integration - -### 4.1 Simple Functions - -**Definition 4.1** (Simple Function). A simple function is: -$$s = \sum_{i=1}^n a_i \mathbf{1}_{A_i}$$ - -where 1_{Aᵢ} is the indicator function. - -**Definition 4.2** (Integral of Simple Function). -$$\int s \, dP = \sum_{i=1}^n a_i P(A_i)$$ - -### 4.2 Lebesgue Integral - -**Definition 4.3** (Lebesgue Integral). For non-negative measurable f: -$$\int f \, dP = \sup\left\{\int s \, dP : s \text{ simple}, s ≤ f\right\}$$ - -For general f: ∫f dP = ∫f⁺ dP - ∫f⁻ dP where f = f⁺ - f⁻. - -### 4.3 Expectation - -**Definition 4.4** (Expectation). -$$\mathbb{E}[X] = \int_Ω X(ω) \, dP(ω)$$ - -**Theorem 4.1** (Ternary Expectation). For uniform `(bet A B C)` with numeric outcomes: -$$\mathbb{E}[X] = \frac{A + B + C}{3}$$ - -*Proof*: -$$\mathbb{E}[X] = \sum_{x ∈ Ω} x · P(\{x\}) = A · \frac{1}{3} + B · \frac{1}{3} + C · \frac{1}{3} = \frac{A+B+C}{3} ∎$$ - -### 4.4 Properties of Integration - -**Theorem 4.2** (Linearity). -$$\mathbb{E}[aX + bY] = a\mathbb{E}[X] + b\mathbb{E}[Y]$$ - -**Theorem 4.3** (Monotone Convergence). If 0 ≤ Xₙ ↑ X: -$$\mathbb{E}[X_n] ↑ \mathbb{E}[X]$$ - -**Theorem 4.4** (Dominated Convergence). If Xₙ → X and |Xₙ| ≤ Y with E[Y] < ∞: -$$\mathbb{E}[X_n] → \mathbb{E}[X]$$ - ---- - -## 5. Product Measures - -### 5.1 Definition - -**Definition 5.1** (Product Measure). For (Ω₁, F₁, P₁) and (Ω₂, F₂, P₂): -$$(P_1 × P_2)(A × B) = P_1(A) · P_2(B)$$ - -Extended to F₁ ⊗ F₂ by Carathéodory. - -### 5.2 Independence - -**Definition 5.2** (Independence). X and Y are independent if: -$$P(X ∈ A, Y ∈ B) = P(X ∈ A) · P(Y ∈ B)$$ - -for all measurable A, B. - -**Theorem 5.1** (bet-parallel Independence). In `(bet-parallel n A B C)`, the n trials are independent. - -*Proof*: Each trial samples independently from the probability space. ∎ - -### 5.3 Fubini's Theorem - -**Theorem 5.2** (Fubini). For integrable f on product space: -$$\int_{Ω_1 × Ω_2} f \, d(P_1 × P_2) = \int_{Ω_1} \left(\int_{Ω_2} f(x,y) \, dP_2(y)\right) dP_1(x)$$ - -**Application**: Computing expectations of functions of independent bets. - ---- - -## 6. Conditional Probability - -### 6.1 Definition - -**Definition 6.1** (Conditional Probability). -$$P(A | B) = \frac{P(A ∩ B)}{P(B)}$$ - -for P(B) > 0. - -### 6.2 Conditional Expectation - -**Definition 6.2** (Conditional Expectation). E[X | G] for sub-σ-algebra G is the G-measurable random variable satisfying: -$$\int_A \mathbb{E}[X | \mathcal{G}] \, dP = \int_A X \, dP \quad ∀A ∈ \mathcal{G}$$ - -### 6.3 Properties - -**Theorem 6.1** (Tower Property). -$$\mathbb{E}[\mathbb{E}[X | \mathcal{G}]] = \mathbb{E}[X]$$ - -**Theorem 6.2** (Independence). If X ⊥ G: -$$\mathbb{E}[X | \mathcal{G}] = \mathbb{E}[X]$$ - ---- - -## 7. Characteristic Functions - -### 7.1 Definition - -**Definition 7.1** (Characteristic Function). -$$\phi_X(t) = \mathbb{E}[e^{itX}]$$ - -### 7.2 Ternary Characteristic Function - -**Theorem 7.1** (Uniform Ternary CF). For uniform bet on {-1, 0, 1}: -$$\phi_X(t) = \frac{1}{3}(e^{-it} + 1 + e^{it}) = \frac{1 + 2\cos(t)}{3}$$ - -### 7.3 Uniqueness - -**Theorem 7.2** (Uniqueness). The characteristic function uniquely determines the distribution. - ---- - -## 8. Convergence - -### 8.1 Types of Convergence - -**Definition 8.1** (Convergence Types). -1. **Almost sure**: P(Xₙ → X) = 1 -2. **In probability**: ∀ε > 0: P(|Xₙ - X| > ε) → 0 -3. **In distribution**: E[f(Xₙ)] → E[f(X)] for bounded continuous f -4. **In Lᵖ**: E[|Xₙ - X|ᵖ] → 0 - -### 8.2 Relationships - -**Theorem 8.1** (Convergence Hierarchy). -$$\text{a.s.} ⟹ \text{in prob.} ⟹ \text{in dist.}$$ -$$L^p ⟹ \text{in prob.} \text{ (for } p ≥ 1\text{)}$$ - -### 8.3 SLLN (Measure-Theoretic) - -**Theorem 8.2** (Strong Law). For i.i.d. Xᵢ with E[|X|] < ∞: -$$\frac{1}{n}\sum_{i=1}^n X_i \xrightarrow{a.s.} \mathbb{E}[X]$$ - -*Proof*: By Kolmogorov's strong law using truncation and Borel-Cantelli. ∎ - ---- - -## 9. Radon-Nikodym Theorem - -### 9.1 Absolute Continuity - -**Definition 9.1** (Absolute Continuity). μ ≪ ν if ν(A) = 0 ⟹ μ(A) = 0. - -### 9.2 Radon-Nikodym Derivative - -**Theorem 9.1** (Radon-Nikodym). If μ ≪ ν for σ-finite measures, ∃ measurable f ≥ 0: -$$μ(A) = \int_A f \, dν$$ - -f is the Radon-Nikodym derivative dμ/dν. - -### 9.3 Application: Importance Sampling - -For target P and proposal Q with P ≪ Q: -$$\frac{dP}{dQ}(x) = \frac{p(x)}{q(x)}$$ - -is the importance weight. - ---- - -## 10. Probability Kernels - -### 10.1 Definition - -**Definition 10.1** (Markov Kernel). A Markov kernel from (Ω₁, F₁) to (Ω₂, F₂) is: -$$K: Ω_1 × \mathcal{F}_2 → [0, 1]$$ - -such that: -1. K(ω, ·) is a probability measure for each ω -2. K(·, B) is measurable for each B - -### 10.2 Composition - -**Definition 10.2** (Kernel Composition). -$$(K_1 K_2)(x, C) = \int K_1(x, dy) K_2(y, C)$$ - -### 10.3 Betlang Stochastic Functions - -**Theorem 10.1**. A betlang function f: A → Dist(B) is a Markov kernel from (A, P(A)) to (B, P(B)). - ---- - -## 11. Polish Spaces and Regular Conditional Distributions - -### 11.1 Polish Space - -**Definition 11.1** (Polish Space). A topological space that is: -- Separable (has countable dense subset) -- Completely metrizable - -### 11.2 Regular Conditional Distribution - -**Theorem 11.1** (Existence). On Polish spaces, regular conditional distributions exist. - -For random variable X and σ-algebra G, there exists kernel P^X|G such that: -$$P^{X|\mathcal{G}}(ω, B) = P(X ∈ B | \mathcal{G})(ω) \text{ a.s.}$$ - ---- - -## 12. Giry Monad (Measure-Theoretic) - -### 12.1 Definition - -**Definition 12.1** (Giry Monad on Meas). -- Objects: Measurable spaces (X, Σ) -- G(X) = probability measures on X with weak topology -- η_X: x ↦ δ_x (Dirac) -- μ_X: ∫ P dΦ for Φ ∈ G(G(X)) - -### 12.2 Relationship to Betlang - -**Theorem 12.1**. The discrete Dist functor is the restriction of Giry to finite discrete spaces with counting measure. - ---- - -## 13. Continuous Distributions in Betlang - -### 13.1 Lebesgue Measure - -**Definition 13.1** (Lebesgue Measure). On (ℝ, B(ℝ)): -$$λ([a, b]) = b - a$$ - -Extended to Borel σ-algebra. - -### 13.2 Density - -**Definition 13.2** (Probability Density). f is a density for P w.r.t. λ if: -$$P(A) = \int_A f(x) \, dλ(x)$$ - -### 13.3 Betlang Continuous Distributions - -**Normal**: -$$f(x) = \frac{1}{\sqrt{2πσ^2}} \exp\left(-\frac{(x-μ)^2}{2σ^2}\right)$$ - -**Exponential**: -$$f(x) = λ e^{-λx} \mathbf{1}_{x ≥ 0}$$ - ---- - -## 14. Martingales - -### 14.1 Definition - -**Definition 14.1** (Martingale). A sequence (Xₙ, Fₙ) is a martingale if: -1. Xₙ is Fₙ-measurable -2. E[|Xₙ|] < ∞ -3. E[Xₙ₊₁ | Fₙ] = Xₙ a.s. - -### 14.2 Optional Stopping - -**Theorem 14.1** (Optional Stopping). For bounded stopping time τ: -$$\mathbb{E}[X_τ] = \mathbb{E}[X_0]$$ - -### 14.3 Application: bet-until Analysis - -For centered bet-until sum Sₙ = Σ(Xᵢ - E[Xᵢ]): -- (Sₙ, Fₙ) is a martingale -- Optional stopping applies to analyze hitting times - ---- - -## 15. Weak Convergence - -### 15.1 Definition - -**Definition 15.1** (Weak Convergence). Pₙ ⇒ P if: -$$\int f \, dP_n → \int f \, dP$$ - -for all bounded continuous f. - -### 15.2 Portmanteau Theorem - -**Theorem 15.1** (Portmanteau). TFAE: -1. Pₙ ⇒ P -2. ∫f dPₙ → ∫f dP for bounded Lipschitz f -3. lim sup Pₙ(F) ≤ P(F) for closed F -4. lim inf Pₙ(G) ≥ P(G) for open G - -### 15.3 Prokhorov's Theorem - -**Theorem 15.2** (Prokhorov). On Polish spaces, tightness ⟺ relative compactness in weak topology. - ---- - -## 16. Summary: Measure Theory in Betlang - -| Concept | Betlang Realization | -|---------|---------------------| -| σ-algebra | Power set (discrete) | -| Measure | Probability distribution | -| Random variable | bet expression | -| Integration | Expectation (bet-expect) | -| Product measure | Independent bets | -| Kernel | Stochastic function | -| Density | PDF for continuous distributions | - ---- - -## 17. TODOs - -**TODO**: The following need measure-theoretic development: - -1. **Disintegration theorem**: For conditional distributions -2. **Prokhorov metric**: For convergence of distributions -3. **Central limit theorem**: Measure-theoretic proof -4. **Ergodic theory**: For MCMC convergence -5. **Optimal transport**: Wasserstein distances - ---- - -## References - -1. Billingsley, P. (1995). *Probability and Measure*, 3rd ed. -2. Durrett, R. (2019). *Probability: Theory and Examples*, 5th ed. -3. Klenke, A. (2014). *Probability Theory: A Comprehensive Course*, 2nd ed. -4. Williams, D. (1991). *Probability with Martingales* -5. Giry, M. (1982). "A categorical approach to probability theory" diff --git a/proofs/theorems/soundness-completeness.adoc b/proofs/theorems/soundness-completeness.adoc new file mode 100644 index 0000000..582feff --- /dev/null +++ b/proofs/theorems/soundness-completeness.adoc @@ -0,0 +1,550 @@ +== Soundness and Completeness Theorems for Betlang + +=== Abstract + +This document establishes the fundamental correctness properties of +betlang: soundness (the system only proves true things) and completeness +(the system can prove all true things within its scope). We address +these properties for the type system, probabilistic logic, and +statistical inference. + +''''' + +=== 1. Type System Soundness + +==== 1.1 Statement + +*Theorem 1.1* (Type Soundness). If e is a closed, well-typed expression +(⊢ e : τ), then evaluation of e does not get stuck. + +More precisely, if ⊢ e : τ and e →* e’, then either: 1. e’ is a value, +or 2. ∃e’'`. e`' → e’’ + +==== 1.2 Proof via Progress and Preservation + +*Lemma 1.1* (Progress). If ⊢ e : τ, then either e is a value or ∃e’. e → +e’. + +_Proof by structural induction on the typing derivation:_ + +*Case T-VAR*: Γ ⊢ x : τ - Impossible for closed terms (Γ = ∅) + +*Case T-INT, T-REAL, T-BOOL, T-SYM*: - e is already a value ✓ + +*Case T-ABS*: Γ ⊢ (λ x. e’) : τ₁ → τ₂ - Lambda is a value ✓ + +*Case T-APP*: Γ ⊢ (e₁ e₂) : τ₂ from Γ ⊢ e₁ : τ₁ → τ₂ and Γ ⊢ e₂ : τ₁ - +By IH on e₁: either e₁ is a value or e₁ → e₁’ - If e₁ → e₁’, then (e₁ +e₂) → (e₁’ e₂) by congruence ✓ - If e₁ is a value, by canonical forms, +e₁ = λx.e₁’ - By IH on e₂: either e₂ is a value or e₂ → e₂’ - If e₂ → +e₂’, then ((λx.e₁’) e₂) → ((λx.e₁’) e₂’) ✓ - If e₂ is a value v, then +((λx.e₁’) v) → e₁’[x ↦ v] by β ✓ + +*Case T-BET*: Γ ⊢ (bet e₁ e₂ e₃) : Dist τ - By IH, each eᵢ either is a +value or steps - If any eᵢ steps, the bet steps by congruence ✓ - If all +are values v₁, v₂, v₃, then (bet v₁ v₂ v₃) → vᵢ for random i ✓ + +*Case T-COND*: Γ ⊢ (if e₁ e₂ e₃) : τ - By IH on e₁ - If e₁ → e₁’, then +(if e₁ e₂ e₃) → (if e₁’ e₂ e₃) ✓ - If e₁ = true, then (if true e₂ e₃) → +e₂ ✓ - If e₁ = false, then (if false e₂ e₃) → e₃ ✓ + +∎ + +*Lemma 1.2* (Preservation). If Γ ⊢ e : τ and e → e’, then Γ ⊢ e’ : τ. + +_Proof by induction on the evaluation derivation:_ + +*Case E-BET*: (bet v₁ v₂ v₃) → vᵢ - By inversion of T-BET: Γ ⊢ vᵢ : τ +for each i - Hence Γ ⊢ vᵢ : τ ✓ + +*Case E-BETA*: ((λx.e) v) → e[x ↦ v] - By inversion: Γ ⊢ (λx.e) : τ₁ → +τ₂ and Γ ⊢ v : τ₁ - From T-ABS: Γ, x:τ₁ ⊢ e : τ₂ - By substitution +lemma: Γ ⊢ e[x ↦ v] : τ₂ ✓ + +*Case E-IF-TRUE*: (if true e₂ e₃) → e₂ - By inversion: Γ ⊢ e₂ : τ ✓ + +*Case E-IF-FALSE*: (if false e₂ e₃) → e₃ - By inversion: Γ ⊢ e₃ : τ ✓ + +*Congruence cases*: Follow by IH. + +∎ + +*Proof of Theorem 1.1*: By induction on the length of e →* e’, using +Progress and Preservation. ∎ + +''''' + +=== 2. Semantic Soundness + +==== 2.1 Operational-Denotational Correspondence + +*Theorem 2.1* (Adequacy). The operational semantics is adequate with +respect to the denotational semantics: + +For closed expression e, the probability that e evaluates to value v +equals the denotational probability: + +[latexmath] +++++ +P(⟨e, ∅, ω⟩ ⇓ ⟨v, ω'⟩ \text{ for some } ω') = ⟦e⟧∅(v) +++++ + +where the probability is over random streams ω. + +_Proof sketch_: By structural induction on e. + +*Base case (values)*: - ⟦v⟧∅ = δᵥ (Dirac at v) - Operationally, v ⇓ v +with probability 1 - Both give P(v) = 1 ✓ + +*Case (bet e₁ e₂ e₃)*: - Denotationally: ⟦bet e₁ e₂ e₃⟧∅ = (1/3)(⟦e₁⟧∅ + +⟦e₂⟧∅ + ⟦e₃⟧∅) - Operationally: uniform random choice over evaluated +branches - By IH, each ⟦eᵢ⟧∅ correctly represents eᵢ’s distribution - +The operational random choice is uniform, matching (1/3) weights ✓ + +∎ + +==== 2.2 Full Abstraction + +*Definition 2.1* (Observational Equivalence). e₁ ≃_obs e₂ iff for all +contexts C[·]: + +[latexmath] +++++ +P(C[e_1] ⇓ \text{true}) = P(C[e_2] ⇓ \text{true}) +++++ + +*Definition 2.2* (Denotational Equivalence). e₁ ≃_den e₂ iff: + +[latexmath] +++++ +⟦e_1⟧ρ = ⟦e_2⟧ρ \text{ for all } ρ +++++ + +*Theorem 2.2* (Full Abstraction). e₁ ≃_obs e₂ ⟺ e₁ ≃_den e₂ + +_Proof_: + +*Soundness* (≃_den ⟹ ≃_obs): If denotationally equal, compositionality +of ⟦·⟧ ensures: + +[latexmath] +++++ +⟦C[e_1]⟧ = ⟦C[e_2]⟧ +++++ +By adequacy, observational behavior matches. ∎ + +*Completeness* (≃_obs ⟹ ≃_den): We construct a distinguishing context +for any denotationally different expressions. + +If ⟦e₁⟧ρ(v) ≠ ⟦e₂⟧ρ(v) for some v, construct: + +.... +C[·] = (let x = · in (if (equal? x v) true false)) +.... + +This context has different termination probabilities for e₁ vs e₂. ∎ + +''''' + +=== 3. Probabilistic Logic Soundness + +==== 3.1 Probabilistic Hoare Logic + +*Syntax*: \{P} e \{Q}_p means "`if P holds, then after e, Q holds with +probability ≥ p`" + +*Theorem 3.1* (Soundness of Probabilistic Hoare Logic). If \{P} e \{Q}_p +is derivable, then for all states σ satisfying P: + +[latexmath] +++++ +P(\{ω : ⟨e, σ, ω⟩ ⇓ ⟨v, ω'⟩ \text{ and } Q[v]\}) ≥ p +++++ + +_Proof_: By induction on the derivation. + +*Case BET-AXIOM*: \{P} (bet A B C) \{x = A}_1/3 - Uniform bet selects A +with probability exactly 1/3 - Hence P(x = A) = 1/3 ≥ 1/3 ✓ + +*Case CONSEQUENCE*: From P’ ⇒ P, \{P} e \{Q}_p, Q ⇒ Q’, p ≤ p’ - By IH, +P(Q) ≥ p - Since Q ⇒ Q’, P(Q’) ≥ P(Q) ≥ p ≥ p’ would require p’ ≤ p… - +Actually the rule says p ≤ p’, so we weaken: P(Q’) ≥ P(Q) ≥ p ≥ p’ if we +had p ≥ p’ - Correction: The consequence rule allows weakening +probability, so P(Q’) ≥ p’ when p’ ≤ p ✓ + +*Case SEQUENCE*: \{P} e₁ \{R}_\{p₁}, \{R} e₂ \{Q}_\{p₂} implies \{P} +e₁;e₂ \{Q}_\{p₁·p₂} - By IH, P(R after e₁) ≥ p₁ - Given R, P(Q after e₂) +≥ p₂ - By independence (or conditional probability), P(Q after e₁;e₂) ≥ +p₁·p₂ ✓ + +∎ + +==== 3.2 Completeness of Probabilistic Logic + +*Theorem 3.2* (Relative Completeness). The probabilistic Hoare logic is +complete relative to an oracle for the underlying probability theory: + +If \{P} e \{Q}_p is semantically valid, then it is derivable (given +arithmetic facts as axioms). + +_Proof sketch_: We can construct derivations using the weakest +precondition: + +[latexmath] +++++ +\text{wp}[e](Q) = \lambda σ. P(Q \text{ after } e \text{ from } σ) +++++ + +For any valid triple, wplink:Q[e] ≥ p when P holds, and we can derive +this using consequence from the exact probability. ∎ + +''''' + +=== 4. Statistical Inference Soundness + +==== 4.1 Monte Carlo Estimator Soundness + +*Theorem 4.1* (Unbiasedness of bet-probability). The estimator: + +[source,racket] +---- +(bet-probability n pred A B C) +---- + +is an unbiased estimator of P(pred(X) = true) where X ~ Uniform\{A, B, +C}. + +_Proof_: Let X₁, …, Xₙ be i.i.d. samples from (bet A B C). Let Yᵢ = 1 if +pred(Xᵢ) else 0. + +The estimator computes: + +[latexmath] +++++ +\hat{p} = \frac{1}{n}\sum_{i=1}^{n} Y_i +++++ + +Expected value: + +[latexmath] +++++ +\mathbb{E}[\hat{p}] = \frac{1}{n}\sum_{i=1}^{n} \mathbb{E}[Y_i] = \mathbb{E}[Y_1] = P(\text{pred}(X) = \text{true}) +++++ + +Hence unbiased. ∎ + +*Theorem 4.2* (Consistency of bet-probability). As n → ∞: + +[latexmath] +++++ +\hat{p} \xrightarrow{a.s.} P(\text{pred}(X) = \text{true}) +++++ + +_Proof_: By the Strong Law of Large Numbers for i.i.d. bounded random +variables. ∎ + +==== 4.2 Confidence Interval Validity + +*Theorem 4.3* (Asymptotic Coverage). The confidence interval from +`+confidence-interval+`: + +[latexmath] +++++ +\hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} +++++ + +has asymptotic coverage probability (1 - α). + +_Proof_: By CLT: + +[latexmath] +++++ +\sqrt{n}\frac{\hat{p} - p}{\sqrt{p(1-p)}} \xrightarrow{d} N(0, 1) +++++ + +Using Slutsky’s theorem with latexmath:[\hat{p} \xrightarrow{P} p]: + +[latexmath] +++++ +\sqrt{n}\frac{\hat{p} - p}{\sqrt{\hat{p}(1-\hat{p})}} \xrightarrow{d} N(0, 1) +++++ + +Hence: + +[latexmath] +++++ +P\left(-z_{\alpha/2} \leq \sqrt{n}\frac{\hat{p} - p}{\sqrt{\hat{p}(1-\hat{p})}} \leq z_{\alpha/2}\right) \to 1 - \alpha +++++ +∎ + +==== 4.3 Bayesian Inference Correctness + +*Theorem 4.4* (Posterior Correctness). For `+conjugate-beta-binomial+`: + +Given prior Beta(α, β) and data (k successes, n-k failures), the +posterior is Beta(α+k, β+n-k). + +_Proof_: + +[latexmath] +++++ +\text{Prior: } p(\theta) \propto \theta^{\alpha-1}(1-\theta)^{\beta-1} +++++ + +[latexmath] +++++ +\text{Likelihood: } p(D|\theta) \propto \theta^k(1-\theta)^{n-k} +++++ + +[latexmath] +++++ +\text{Posterior: } p(\theta|D) \propto \theta^{\alpha+k-1}(1-\theta)^{\beta+n-k-1} +++++ + +This is the kernel of Beta(α+k, β+n-k). ∎ + +==== 4.4 MCMC Correctness + +*Theorem 4.5* (Metropolis-Hastings Convergence). The +`+metropolis-hastings+` implementation converges to the target +distribution. + +_Proof sketch_: 1. *Detailed balance*: The M-H acceptance ratio: + +[latexmath] +++++ +\alpha(x, y) = \min\left(1, \frac{\pi(y)q(x|y)}{\pi(x)q(y|x)}\right) +++++ +ensures detailed balance: π(x)P(x,y) = π(y)P(y,x) + +[arabic, start=2] +. *Irreducibility*: Assuming the proposal distribution has full support +. *Aperiodicity*: Positive rejection probability ensures aperiodicity + +By the ergodic theorem for Markov chains: + +[latexmath] +++++ +\frac{1}{n}\sum_{i=1}^{n} f(X_i) \xrightarrow{a.s.} \mathbb{E}_\pi[f] +++++ +∎ + +''''' + +=== 5. Distribution Implementation Correctness + +==== 5.1 Normal Distribution + +*Theorem 5.1* (Box-Muller Correctness). The `+normal+` function produces +N(μ, σ²) samples. + +_Proof_: Let U₁, U₂ ~ Uniform(0,1) independent. Define: + +[latexmath] +++++ +Z_1 = \sqrt{-2\ln U_1}\cos(2\pi U_2) +++++ + +[latexmath] +++++ +Z_2 = \sqrt{-2\ln U_1}\sin(2\pi U_2) +++++ + +Then Z₁, Z₂ are independent N(0,1). + +Transformation X = μ + σZ gives X ~ N(μ, σ²). ∎ + +==== 5.2 Gamma Distribution + +*Theorem 5.2* (Marsaglia-Tsang Correctness). The gamma sampler produces +Gamma(α, β) samples for α ≥ 1. + +_Proof_: The Marsaglia-Tsang method uses rejection sampling with: - +Proposal based on shifted Gaussian - Acceptance ratio ensuring correct +target + +See Marsaglia & Tsang (2000) for the full proof of correctness. ∎ + +==== 5.3 Entropy Calculation + +*Theorem 5.3* (Entropy Correctness). The `+bet-entropy+` function +computes Shannon entropy correctly. + +_Proof_: Given samples, the function: 1. Computes empirical frequencies +f(x) = count(x)/n 2. Computes -Σ f(x) log₂ f(x) + +As n → ∞, f(x) → P(x) by LLN, so the empirical entropy converges to true +entropy. ∎ + +''''' + +=== 6. Termination + +==== 6.1 Guaranteed Termination + +*Theorem 6.1* (Basic Bet Termination). (bet A B C) terminates in O(1) +time. + +_Proof_: The bet operation: 1. Generates one random number (O(1)) 2. +Selects one of three branches (O(1)) 3. Returns the selected value +(O(1)) + +Total: O(1). ∎ + +*Theorem 6.2* (bet-chain Termination). (bet-chain n f init) terminates +for finite n. + +_Proof_: By induction on n. - Base: n = 0 returns init immediately - +Step: n+1 calls f once and recurses with n + +Total: n+1 function calls, each terminating (by assumption on f). ∎ + +==== 6.2 Almost-Sure Termination + +*Theorem 6.3* (bet-until Almost-Sure Termination). If P(pred(X)) > 0 for +X ~ bet(A,B,C), then: + +[source,racket] +---- +(bet-until pred (λ () (bet A B C))) +---- + +terminates with probability 1. + +_Proof_: Let p = P(pred(X) = true) > 0. The number of iterations N +follows Geometric(p). P(N < ∞) = 1 since Σₙ p(1-p)ⁿ = 1. + +Expected iterations: E[N] = 1/p < ∞. ∎ + +*Corollary 6.1* (Expected Termination Time). E[iterations] = 1/p for +bet-until. + +==== 6.3 Non-Termination Cases + +*Theorem 6.4* (Non-Termination Condition). (bet-until pred thunk) may +not terminate if: - P(pred(thunk())) = 0, or - thunk itself doesn’t +terminate + +These are the only non-termination cases for bet-until. + +''''' + +=== 7. Correctness of Combinators + +==== 7.1 bet-compose + +*Theorem 7.1* (bet-compose Correctness). For functions f, g, h: + +[source,racket] +---- +(define composed (bet-compose f g h)) +(composed x) +---- + +returns f(x), g(x), or h(x) each with probability 1/3. + +_Proof_: By definition: + +[source,racket] +---- +(bet-compose f g h) = (λ (x) ((bet f g h) x)) +---- + +* (bet f g h) returns one of f, g, h uniformly +* Applying to x gives f(x), g(x), or h(x) ✓ ∎ + +==== 7.2 bet-map + +*Theorem 7.2* (bet-map Preservation). bet-map preserves list structure: + +[source,racket] +---- +(length (bet-map f lst)) = (length lst) +---- + +_Proof_: bet-map applies f to each element, preserving list length. ∎ + +==== 7.3 bet-fold Associativity + +*Theorem 7.3* (bet-fold Order Independence). For associative, +commutative f with identity init: + +[latexmath] +++++ +\mathbb{E}[\text{bet-fold } f \text{ init } lst] = f(\text{init}, \prod_{x \in lst} x) +++++ + +where the product is in terms of f. + +_Proof sketch_: By induction on list length, using associativity and +commutativity to reorder fold operations. ∎ + +''''' + +=== 8. Compositional Soundness + +==== 8.1 Modular Reasoning + +*Theorem 8.1* (Compositionality). If: - ⊢ e₁ : τ₁ with property P₁ - ⊢ +e₂ : τ₂ with property P₂ + +Then ⊢ (bet e₁ e₂ e₃) satisfies a property derivable from P₁, P₂, P₃. + +This enables modular verification of betlang programs. + +==== 8.2 Refinement + +*Definition 8.1* (Probabilistic Refinement). e₁ ⊑ e₂ iff: For all +postconditions Q: wplink:Q[e₂] ≤ wplink:Q[e₁] + +"`e₁ refines e₂ if it’s at least as good for any property.`" + +*Theorem 8.2* (Refinement Soundness). If e₁ ⊑ e₂, then e₁ can safely +replace e₂ in any context. + +''''' + +=== 9. Summary of Key Results + +[cols=",,",options="header",] +|=== +|Property |Theorem |Status +|Type Soundness |Theorem 1.1 |✓ Proved +|Semantic Adequacy |Theorem 2.1 |✓ Proved +|Full Abstraction |Theorem 2.2 |✓ Proved +|Hoare Logic Soundness |Theorem 3.1 |✓ Proved +|Hoare Logic Completeness |Theorem 3.2 |✓ Relative +|Monte Carlo Unbiasedness |Theorem 4.1 |✓ Proved +|Monte Carlo Consistency |Theorem 4.2 |✓ Proved +|CI Coverage |Theorem 4.3 |✓ Asymptotic +|MCMC Convergence |Theorem 4.5 |✓ Sketch +|Termination |Theorems 6.1-6.3 |✓ Proved +|=== + +''''' + +=== 10. Open Problems and TODOs + +*TODO*: The following require further work: + +[arabic] +. *Continuous distribution soundness*: Formalize correctness for all +distributions in lib/distributions.rkt +. *Parallel execution soundness*: Prove bet-parallel produces +independent samples +. *Numerical stability*: Prove bounds on floating-point errors in +statistical functions +. *Complexity-theoretic soundness*: Relate betlang to probabilistic +complexity classes + +''''' + +=== References + +[arabic] +. Wright, A. & Felleisen, M. (1994). "`A Syntactic Approach to Type +Soundness`" +. McIver, A. & Morgan, C. (2005). "`Abstraction, Refinement and Proof +for Probabilistic Systems`" +. Kozen, D. (1985). "`A Probabilistic PDL`" +. Robert, C.P. & Casella, G. (2004). "`Monte Carlo Statistical Methods`" diff --git a/proofs/theorems/soundness-completeness.md b/proofs/theorems/soundness-completeness.md deleted file mode 100644 index 6da6c25..0000000 --- a/proofs/theorems/soundness-completeness.md +++ /dev/null @@ -1,442 +0,0 @@ - -# Soundness and Completeness Theorems for Betlang - -## Abstract - -This document establishes the fundamental correctness properties of betlang: soundness (the system only proves true things) and completeness (the system can prove all true things within its scope). We address these properties for the type system, probabilistic logic, and statistical inference. - ---- - -## 1. Type System Soundness - -### 1.1 Statement - -**Theorem 1.1** (Type Soundness). If e is a closed, well-typed expression (⊢ e : τ), then evaluation of e does not get stuck. - -More precisely, if ⊢ e : τ and e →* e', then either: -1. e' is a value, or -2. ∃e''. e' → e'' - -### 1.2 Proof via Progress and Preservation - -**Lemma 1.1** (Progress). If ⊢ e : τ, then either e is a value or ∃e'. e → e'. - -*Proof by structural induction on the typing derivation:* - -**Case T-VAR**: Γ ⊢ x : τ -- Impossible for closed terms (Γ = ∅) - -**Case T-INT, T-REAL, T-BOOL, T-SYM**: -- e is already a value ✓ - -**Case T-ABS**: Γ ⊢ (λ x. e') : τ₁ → τ₂ -- Lambda is a value ✓ - -**Case T-APP**: Γ ⊢ (e₁ e₂) : τ₂ from Γ ⊢ e₁ : τ₁ → τ₂ and Γ ⊢ e₂ : τ₁ -- By IH on e₁: either e₁ is a value or e₁ → e₁' - - If e₁ → e₁', then (e₁ e₂) → (e₁' e₂) by congruence ✓ -- If e₁ is a value, by canonical forms, e₁ = λx.e₁' -- By IH on e₂: either e₂ is a value or e₂ → e₂' - - If e₂ → e₂', then ((λx.e₁') e₂) → ((λx.e₁') e₂') ✓ - - If e₂ is a value v, then ((λx.e₁') v) → e₁'[x ↦ v] by β ✓ - -**Case T-BET**: Γ ⊢ (bet e₁ e₂ e₃) : Dist τ -- By IH, each eᵢ either is a value or steps -- If any eᵢ steps, the bet steps by congruence ✓ -- If all are values v₁, v₂, v₃, then (bet v₁ v₂ v₃) → vᵢ for random i ✓ - -**Case T-COND**: Γ ⊢ (if e₁ e₂ e₃) : τ -- By IH on e₁ -- If e₁ → e₁', then (if e₁ e₂ e₃) → (if e₁' e₂ e₃) ✓ -- If e₁ = true, then (if true e₂ e₃) → e₂ ✓ -- If e₁ = false, then (if false e₂ e₃) → e₃ ✓ - -∎ - -**Lemma 1.2** (Preservation). If Γ ⊢ e : τ and e → e', then Γ ⊢ e' : τ. - -*Proof by induction on the evaluation derivation:* - -**Case E-BET**: (bet v₁ v₂ v₃) → vᵢ -- By inversion of T-BET: Γ ⊢ vᵢ : τ for each i -- Hence Γ ⊢ vᵢ : τ ✓ - -**Case E-BETA**: ((λx.e) v) → e[x ↦ v] -- By inversion: Γ ⊢ (λx.e) : τ₁ → τ₂ and Γ ⊢ v : τ₁ -- From T-ABS: Γ, x:τ₁ ⊢ e : τ₂ -- By substitution lemma: Γ ⊢ e[x ↦ v] : τ₂ ✓ - -**Case E-IF-TRUE**: (if true e₂ e₃) → e₂ -- By inversion: Γ ⊢ e₂ : τ ✓ - -**Case E-IF-FALSE**: (if false e₂ e₃) → e₃ -- By inversion: Γ ⊢ e₃ : τ ✓ - -**Congruence cases**: Follow by IH. - -∎ - -**Proof of Theorem 1.1**: By induction on the length of e →* e', using Progress and Preservation. ∎ - ---- - -## 2. Semantic Soundness - -### 2.1 Operational-Denotational Correspondence - -**Theorem 2.1** (Adequacy). The operational semantics is adequate with respect to the denotational semantics: - -For closed expression e, the probability that e evaluates to value v equals the denotational probability: - -$$P(⟨e, ∅, ω⟩ ⇓ ⟨v, ω'⟩ \text{ for some } ω') = ⟦e⟧∅(v)$$ - -where the probability is over random streams ω. - -*Proof sketch*: By structural induction on e. - -**Base case (values)**: -- ⟦v⟧∅ = δᵥ (Dirac at v) -- Operationally, v ⇓ v with probability 1 -- Both give P(v) = 1 ✓ - -**Case (bet e₁ e₂ e₃)**: -- Denotationally: ⟦bet e₁ e₂ e₃⟧∅ = (1/3)(⟦e₁⟧∅ + ⟦e₂⟧∅ + ⟦e₃⟧∅) -- Operationally: uniform random choice over evaluated branches -- By IH, each ⟦eᵢ⟧∅ correctly represents eᵢ's distribution -- The operational random choice is uniform, matching (1/3) weights ✓ - -∎ - -### 2.2 Full Abstraction - -**Definition 2.1** (Observational Equivalence). e₁ ≃_obs e₂ iff for all contexts C[·]: -$$P(C[e_1] ⇓ \text{true}) = P(C[e_2] ⇓ \text{true})$$ - -**Definition 2.2** (Denotational Equivalence). e₁ ≃_den e₂ iff: -$$⟦e_1⟧ρ = ⟦e_2⟧ρ \text{ for all } ρ$$ - -**Theorem 2.2** (Full Abstraction). e₁ ≃_obs e₂ ⟺ e₁ ≃_den e₂ - -*Proof*: - -**Soundness** (≃_den ⟹ ≃_obs): If denotationally equal, compositionality of ⟦·⟧ ensures: -$$⟦C[e_1]⟧ = ⟦C[e_2]⟧$$ -By adequacy, observational behavior matches. ∎ - -**Completeness** (≃_obs ⟹ ≃_den): -We construct a distinguishing context for any denotationally different expressions. - -If ⟦e₁⟧ρ(v) ≠ ⟦e₂⟧ρ(v) for some v, construct: -``` -C[·] = (let x = · in (if (equal? x v) true false)) -``` -This context has different termination probabilities for e₁ vs e₂. ∎ - ---- - -## 3. Probabilistic Logic Soundness - -### 3.1 Probabilistic Hoare Logic - -**Syntax**: {P} e {Q}_p means "if P holds, then after e, Q holds with probability ≥ p" - -**Theorem 3.1** (Soundness of Probabilistic Hoare Logic). If {P} e {Q}_p is derivable, then for all states σ satisfying P: - -$$P(\{ω : ⟨e, σ, ω⟩ ⇓ ⟨v, ω'⟩ \text{ and } Q[v]\}) ≥ p$$ - -*Proof*: By induction on the derivation. - -**Case BET-AXIOM**: {P} (bet A B C) {x = A}_1/3 -- Uniform bet selects A with probability exactly 1/3 -- Hence P(x = A) = 1/3 ≥ 1/3 ✓ - -**Case CONSEQUENCE**: From P' ⇒ P, {P} e {Q}_p, Q ⇒ Q', p ≤ p' -- By IH, P(Q) ≥ p -- Since Q ⇒ Q', P(Q') ≥ P(Q) ≥ p ≥ p' would require p' ≤ p... -- Actually the rule says p ≤ p', so we weaken: P(Q') ≥ P(Q) ≥ p ≥ p' if we had p ≥ p' -- Correction: The consequence rule allows weakening probability, so P(Q') ≥ p' when p' ≤ p ✓ - -**Case SEQUENCE**: {P} e₁ {R}_{p₁}, {R} e₂ {Q}_{p₂} implies {P} e₁;e₂ {Q}_{p₁·p₂} -- By IH, P(R after e₁) ≥ p₁ -- Given R, P(Q after e₂) ≥ p₂ -- By independence (or conditional probability), P(Q after e₁;e₂) ≥ p₁·p₂ ✓ - -∎ - -### 3.2 Completeness of Probabilistic Logic - -**Theorem 3.2** (Relative Completeness). The probabilistic Hoare logic is complete relative to an oracle for the underlying probability theory: - -If {P} e {Q}_p is semantically valid, then it is derivable (given arithmetic facts as axioms). - -*Proof sketch*: We can construct derivations using the weakest precondition: -$$\text{wp}[e](Q) = \lambda σ. P(Q \text{ after } e \text{ from } σ)$$ - -For any valid triple, wp[e](Q) ≥ p when P holds, and we can derive this using consequence from the exact probability. ∎ - ---- - -## 4. Statistical Inference Soundness - -### 4.1 Monte Carlo Estimator Soundness - -**Theorem 4.1** (Unbiasedness of bet-probability). The estimator: -```racket -(bet-probability n pred A B C) -``` -is an unbiased estimator of P(pred(X) = true) where X ~ Uniform{A, B, C}. - -*Proof*: -Let X₁, ..., Xₙ be i.i.d. samples from (bet A B C). -Let Yᵢ = 1 if pred(Xᵢ) else 0. - -The estimator computes: -$$\hat{p} = \frac{1}{n}\sum_{i=1}^{n} Y_i$$ - -Expected value: -$$\mathbb{E}[\hat{p}] = \frac{1}{n}\sum_{i=1}^{n} \mathbb{E}[Y_i] = \mathbb{E}[Y_1] = P(\text{pred}(X) = \text{true})$$ - -Hence unbiased. ∎ - -**Theorem 4.2** (Consistency of bet-probability). As n → ∞: -$$\hat{p} \xrightarrow{a.s.} P(\text{pred}(X) = \text{true})$$ - -*Proof*: By the Strong Law of Large Numbers for i.i.d. bounded random variables. ∎ - -### 4.2 Confidence Interval Validity - -**Theorem 4.3** (Asymptotic Coverage). The confidence interval from `confidence-interval`: -$$\hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$ - -has asymptotic coverage probability (1 - α). - -*Proof*: By CLT: -$$\sqrt{n}\frac{\hat{p} - p}{\sqrt{p(1-p)}} \xrightarrow{d} N(0, 1)$$ - -Using Slutsky's theorem with $\hat{p} \xrightarrow{P} p$: -$$\sqrt{n}\frac{\hat{p} - p}{\sqrt{\hat{p}(1-\hat{p})}} \xrightarrow{d} N(0, 1)$$ - -Hence: -$$P\left(-z_{\alpha/2} \leq \sqrt{n}\frac{\hat{p} - p}{\sqrt{\hat{p}(1-\hat{p})}} \leq z_{\alpha/2}\right) \to 1 - \alpha$$ -∎ - -### 4.3 Bayesian Inference Correctness - -**Theorem 4.4** (Posterior Correctness). For `conjugate-beta-binomial`: - -Given prior Beta(α, β) and data (k successes, n-k failures), the posterior is Beta(α+k, β+n-k). - -*Proof*: -$$\text{Prior: } p(\theta) \propto \theta^{\alpha-1}(1-\theta)^{\beta-1}$$ -$$\text{Likelihood: } p(D|\theta) \propto \theta^k(1-\theta)^{n-k}$$ -$$\text{Posterior: } p(\theta|D) \propto \theta^{\alpha+k-1}(1-\theta)^{\beta+n-k-1}$$ - -This is the kernel of Beta(α+k, β+n-k). ∎ - -### 4.4 MCMC Correctness - -**Theorem 4.5** (Metropolis-Hastings Convergence). The `metropolis-hastings` implementation converges to the target distribution. - -*Proof sketch*: -1. **Detailed balance**: The M-H acceptance ratio: - $$\alpha(x, y) = \min\left(1, \frac{\pi(y)q(x|y)}{\pi(x)q(y|x)}\right)$$ - ensures detailed balance: π(x)P(x,y) = π(y)P(y,x) - -2. **Irreducibility**: Assuming the proposal distribution has full support - -3. **Aperiodicity**: Positive rejection probability ensures aperiodicity - -By the ergodic theorem for Markov chains: -$$\frac{1}{n}\sum_{i=1}^{n} f(X_i) \xrightarrow{a.s.} \mathbb{E}_\pi[f]$$ -∎ - ---- - -## 5. Distribution Implementation Correctness - -### 5.1 Normal Distribution - -**Theorem 5.1** (Box-Muller Correctness). The `normal` function produces N(μ, σ²) samples. - -*Proof*: -Let U₁, U₂ ~ Uniform(0,1) independent. -Define: -$$Z_1 = \sqrt{-2\ln U_1}\cos(2\pi U_2)$$ -$$Z_2 = \sqrt{-2\ln U_1}\sin(2\pi U_2)$$ - -Then Z₁, Z₂ are independent N(0,1). - -Transformation X = μ + σZ gives X ~ N(μ, σ²). ∎ - -### 5.2 Gamma Distribution - -**Theorem 5.2** (Marsaglia-Tsang Correctness). The gamma sampler produces Gamma(α, β) samples for α ≥ 1. - -*Proof*: The Marsaglia-Tsang method uses rejection sampling with: -- Proposal based on shifted Gaussian -- Acceptance ratio ensuring correct target - -See Marsaglia & Tsang (2000) for the full proof of correctness. ∎ - -### 5.3 Entropy Calculation - -**Theorem 5.3** (Entropy Correctness). The `bet-entropy` function computes Shannon entropy correctly. - -*Proof*: -Given samples, the function: -1. Computes empirical frequencies f(x) = count(x)/n -2. Computes -Σ f(x) log₂ f(x) - -As n → ∞, f(x) → P(x) by LLN, so the empirical entropy converges to true entropy. ∎ - ---- - -## 6. Termination - -### 6.1 Guaranteed Termination - -**Theorem 6.1** (Basic Bet Termination). (bet A B C) terminates in O(1) time. - -*Proof*: The bet operation: -1. Generates one random number (O(1)) -2. Selects one of three branches (O(1)) -3. Returns the selected value (O(1)) - -Total: O(1). ∎ - -**Theorem 6.2** (bet-chain Termination). (bet-chain n f init) terminates for finite n. - -*Proof*: By induction on n. -- Base: n = 0 returns init immediately -- Step: n+1 calls f once and recurses with n - -Total: n+1 function calls, each terminating (by assumption on f). ∎ - -### 6.2 Almost-Sure Termination - -**Theorem 6.3** (bet-until Almost-Sure Termination). If P(pred(X)) > 0 for X ~ bet(A,B,C), then: - -```racket -(bet-until pred (λ () (bet A B C))) -``` - -terminates with probability 1. - -*Proof*: -Let p = P(pred(X) = true) > 0. -The number of iterations N follows Geometric(p). -P(N < ∞) = 1 since Σₙ p(1-p)ⁿ = 1. - -Expected iterations: E[N] = 1/p < ∞. ∎ - -**Corollary 6.1** (Expected Termination Time). E[iterations] = 1/p for bet-until. - -### 6.3 Non-Termination Cases - -**Theorem 6.4** (Non-Termination Condition). (bet-until pred thunk) may not terminate if: -- P(pred(thunk())) = 0, or -- thunk itself doesn't terminate - -These are the only non-termination cases for bet-until. - ---- - -## 7. Correctness of Combinators - -### 7.1 bet-compose - -**Theorem 7.1** (bet-compose Correctness). For functions f, g, h: -```racket -(define composed (bet-compose f g h)) -(composed x) -``` -returns f(x), g(x), or h(x) each with probability 1/3. - -*Proof*: By definition: -```racket -(bet-compose f g h) = (λ (x) ((bet f g h) x)) -``` -- (bet f g h) returns one of f, g, h uniformly -- Applying to x gives f(x), g(x), or h(x) ✓ ∎ - -### 7.2 bet-map - -**Theorem 7.2** (bet-map Preservation). bet-map preserves list structure: -```racket -(length (bet-map f lst)) = (length lst) -``` - -*Proof*: bet-map applies f to each element, preserving list length. ∎ - -### 7.3 bet-fold Associativity - -**Theorem 7.3** (bet-fold Order Independence). For associative, commutative f with identity init: - -$$\mathbb{E}[\text{bet-fold } f \text{ init } lst] = f(\text{init}, \prod_{x \in lst} x)$$ - -where the product is in terms of f. - -*Proof sketch*: By induction on list length, using associativity and commutativity to reorder fold operations. ∎ - ---- - -## 8. Compositional Soundness - -### 8.1 Modular Reasoning - -**Theorem 8.1** (Compositionality). If: -- ⊢ e₁ : τ₁ with property P₁ -- ⊢ e₂ : τ₂ with property P₂ - -Then ⊢ (bet e₁ e₂ e₃) satisfies a property derivable from P₁, P₂, P₃. - -This enables modular verification of betlang programs. - -### 8.2 Refinement - -**Definition 8.1** (Probabilistic Refinement). e₁ ⊑ e₂ iff: -For all postconditions Q: wp[e₂](Q) ≤ wp[e₁](Q) - -"e₁ refines e₂ if it's at least as good for any property." - -**Theorem 8.2** (Refinement Soundness). If e₁ ⊑ e₂, then e₁ can safely replace e₂ in any context. - ---- - -## 9. Summary of Key Results - -| Property | Theorem | Status | -|----------|---------|--------| -| Type Soundness | Theorem 1.1 | ✓ Proved | -| Semantic Adequacy | Theorem 2.1 | ✓ Proved | -| Full Abstraction | Theorem 2.2 | ✓ Proved | -| Hoare Logic Soundness | Theorem 3.1 | ✓ Proved | -| Hoare Logic Completeness | Theorem 3.2 | ✓ Relative | -| Monte Carlo Unbiasedness | Theorem 4.1 | ✓ Proved | -| Monte Carlo Consistency | Theorem 4.2 | ✓ Proved | -| CI Coverage | Theorem 4.3 | ✓ Asymptotic | -| MCMC Convergence | Theorem 4.5 | ✓ Sketch | -| Termination | Theorems 6.1-6.3 | ✓ Proved | - ---- - -## 10. Open Problems and TODOs - -**TODO**: The following require further work: - -1. **Continuous distribution soundness**: Formalize correctness for all distributions in lib/distributions.rkt -2. **Parallel execution soundness**: Prove bet-parallel produces independent samples -3. **Numerical stability**: Prove bounds on floating-point errors in statistical functions -4. **Complexity-theoretic soundness**: Relate betlang to probabilistic complexity classes - ---- - -## References - -1. Wright, A. & Felleisen, M. (1994). "A Syntactic Approach to Type Soundness" -2. McIver, A. & Morgan, C. (2005). "Abstraction, Refinement and Proof for Probabilistic Systems" -3. Kozen, D. (1985). "A Probabilistic PDL" -4. Robert, C.P. & Casella, G. (2004). "Monte Carlo Statistical Methods" diff --git a/proofs/theorems/termination-analysis.adoc b/proofs/theorems/termination-analysis.adoc new file mode 100644 index 0000000..77a89d2 --- /dev/null +++ b/proofs/theorems/termination-analysis.adoc @@ -0,0 +1,610 @@ +== Termination Analysis for Betlang + +=== Abstract + +This document provides a comprehensive analysis of termination +properties in betlang, distinguishing between deterministic termination, +almost-sure termination, and positive almost-sure termination. We +establish termination theorems for all core constructs and analyze the +termination behavior of loops and recursive structures. + +''''' + +=== 1. Termination Classes + +==== 1.1 Definitions + +*Definition 1.1* (Termination). A program e terminates (e↓) if +evaluation reaches a value. + +*Definition 1.2* (Deterministic Termination). e↓_det iff e terminates +for all random streams. + +*Definition 1.3* (Almost-Sure Termination). e↓_\{a.s.} iff P(e↓) = 1. + +*Definition 1.4* (Positive Almost-Sure Termination - PAST). e↓_\{PAST} +iff E[termination time] < ∞. + +*Definition 1.5* (Bounded Termination). e↓_b iff ∃B. termination time ≤ +B. + +==== 1.2 Hierarchy + +[latexmath] +++++ +\text{Bounded} \Rightarrow \text{Deterministic} \Rightarrow \text{PAST} \Rightarrow \text{Almost-Sure} +++++ + +All implications are strict: - Deterministic ⇏ Bounded: (bet-chain n f +init) for large n - PAST ⇏ Deterministic: bet-until with p < 1 - +Almost-Sure ⇏ PAST: Possible in theory (requires careful construction) + +''''' + +=== 2. Core Primitives + +==== 2.1 bet + +*Theorem 2.1* (bet Bounded Termination). For any values A, B, C: + +.... +(bet A B C) ↓_b with bound B = O(1) +.... + +_Proof_: The bet operation: 1. Generate random number: O(1) 2. Index +selection: O(1) 3. Value return: O(1) + +No loops, no recursion. Deterministic O(1) termination. ∎ + +==== 2.2 bet/weighted + +*Theorem 2.2* (bet/weighted Bounded Termination). For non-negative +weights with positive sum: + +.... +(bet/weighted '(A wₐ) '(B w_b) '(C w_c)) ↓_b +.... + +_Proof_: Same structure as bet, O(1) operations. ∎ + +*Precondition violation*: If all weights are 0 or negative weights +produce negative probabilities, behavior is undefined. + +==== 2.3 bet/conditional + +*Theorem 2.3* (bet/conditional Termination). If pred terminates and +selected branch terminates: + +.... +pred↓ ∧ (pred = true ⟹ A↓) ∧ (pred = false ⟹ (bet B C A)↓) ⟹ (bet/conditional pred A B C)↓ +.... + +_Proof_: Follows from sequential composition of terminating operations. +∎ + +==== 2.4 bet/lazy + +*Theorem 2.4* (bet/lazy Termination). If the selected thunk terminates: + +.... +∀i. thunkᵢ↓ ⟹ (bet/lazy thunk₀ thunk₁ thunk₂)↓ +.... + +_Proof_: Only one thunk is evaluated. If all thunks terminate, the +selected one terminates. ∎ + +*Corollary 2.1* (Non-terminating branches). If exactly one thunk may not +terminate: + +.... +P((bet/lazy thunk₀ thunk₁ thunk₂)↓) ≥ 2/3 +.... + +''''' + +=== 3. Iteration Constructs + +==== 3.1 bet-chain + +*Theorem 3.1* (bet-chain Termination). + +.... +n ∈ ℕ ∧ (∀v. f(v)↓) ⟹ (bet-chain n f init)↓_b +.... + +with bound B = O(n · Tf) where Tf is the time for f. + +_Proof_: By induction on n: - Base: n = 0 returns init immediately - +Step: n+1 = one call to f plus recursive call with n + +Total: exactly n calls to f. ∎ + +*Corollary 3.1* (bet-chain Complexity). Time = Θ(n · Tf). + +==== 3.2 bet-until + +*Theorem 3.2* (bet-until Almost-Sure Termination). If P(pred(thunk())) = +p > 0: + +.... +(bet-until pred thunk) ↓_{a.s.} +.... + +_Proof_: Let N = number of iterations. N ~ Geometric(p). - P(N = k) = +(1-p)^\{k-1} · p - P(N < ∞) = Σₖ P(N = k) = 1 for p > 0 + +Therefore, almost-sure termination. ∎ + +*Theorem 3.3* (bet-until PAST). If p > 0: + +.... +(bet-until pred thunk) ↓_{PAST} +.... + +with E[iterations] = 1/p. + +_Proof_: E[N] = Σₖ k · (1-p)^\{k-1} · p = 1/p < ∞ ∎ + +*Theorem 3.4* (bet-until Variance). + +.... +Var(N) = (1-p)/p² +.... + +*Theorem 3.5* (bet-until Concentration). For k > 0: + +.... +P(N > k · E[N]) = P(N > k/p) = (1-p)^{k/p} ≤ e^{-k} +.... + +*Corollary 3.2* (High-Probability Bound). With probability 1 - δ: + +.... +N ≤ (ln(1/δ))/p +.... + +==== 3.3 bet-repeat + +*Theorem 3.6* (bet-repeat Termination). + +.... +n ∈ ℕ ∧ thunk↓ ⟹ (bet-repeat n thunk)↓_b +.... + +with bound B = O(n · T_thunk). + +_Proof_: Exactly n evaluations of thunk. ∎ + +==== 3.4 bet-parallel + +*Theorem 3.7* (bet-parallel Termination). + +.... +(bet-parallel n A B C)↓_b +.... + +with bound B = O(n). + +_Proof_: n independent bet operations, each O(1). ∎ + +''''' + +=== 4. Higher-Order Operations + +==== 4.1 bet-map + +*Theorem 4.1* (bet-map Termination). + +.... +(∀x ∈ lst. f(x)↓) ⟹ (bet-map f lst)↓ +.... + +_Proof_: Maps f over each element. If f terminates on each, map +terminates. ∎ + +*Probabilistic case*: If f involves randomness but terminates a.s., +bet-map terminates a.s. + +==== 4.2 bet-filter + +*Theorem 4.2* (bet-filter Termination). + +.... +(∀x ∈ lst. pred(x)↓) ⟹ (bet-filter pred lst)↓ +.... + +==== 4.3 bet-fold + +*Theorem 4.3* (bet-fold Termination). + +.... +(∀acc, x. f(acc, x)↓) ⟹ (bet-fold f init lst)↓_b +.... + +with bound B = O(|lst| · Tf). + +''''' + +=== 5. Composition Operators + +==== 5.1 bet-compose + +*Theorem 5.1* (bet-compose Termination). Composition creates terminating +function if components terminate: + +.... +(∀x. f(x)↓ ∧ g(x)↓ ∧ h(x)↓) ⟹ (∀x. ((bet-compose f g h) x)↓) +.... + +_Proof_: Selects one of f, g, h uniformly, applies to x. Selected +function terminates. ∎ + +==== 5.2 bet-sequence + +*Theorem 5.2* (bet-sequence Termination). + +.... +(∀i. betᵢ↓) ⟹ (bet-sequence bet₁ bet₂ ... betₙ)↓ +.... + +_Proof_: Sequential execution of n terminating operations. ∎ + +''''' + +=== 6. Probabilistic Loops + +==== 6.1 Random Walk Analysis + +*Theorem 6.1* (Random Walk Recurrence). For symmetric random walk on ℤ: + +.... +P(return to 0) = 1 (recurrence) +E[return time] = ∞ (null recurrence) +.... + +*Application to bet-until*: + +[source,racket] +---- +(bet-until (λ (pos) (= pos 0)) + (λ () (+ pos (bet -1 0 1)))) +---- + +This terminates a.s. but NOT with finite expected time. + +==== 6.2 Biased Random Walk + +*Theorem 6.2* (Biased Walk Termination). For rightward bias: + +.... +P(step = +1) > P(step = -1) +.... + +Then E[hitting time for target > current] < ∞. + +==== 6.3 Martingale Stopping + +*Theorem 6.3* (Optional Stopping). If \{Sₙ} is a martingale and τ is a +stopping time with E[τ] < ∞: + +.... +E[S_τ] = E[S₀] +.... + +''''' + +=== 7. MCMC Termination + +==== 7.1 Fixed Iterations + +*Theorem 7.1* (MH Fixed Termination). + +.... +(metropolis-hastings target proposal init n)↓_b +.... + +with bound B = O(n · (T_target + T_proposal)). + +_Proof_: Fixed n iterations, each involving target and proposal +evaluation. ∎ + +==== 7.2 Burn-in and Convergence + +*Note*: MCMC "`convergence`" refers to distributional convergence, not +program termination. The programs terminate after fixed iterations; +convergence is a statistical property. + +''''' + +=== 8. Optimization Termination + +==== 8.1 Simulated Annealing + +*Theorem 8.1* (SA Termination). + +.... +(simulated-annealing objective init schedule max-iter neighbor)↓_b +.... + +with bound B = O(max-iter · (T_objective + T_neighbor)). + +==== 8.2 Genetic Algorithm + +*Theorem 8.2* (GA Termination). + +.... +(genetic-algorithm objective pop generations ...)↓_b +.... + +with bound B = O(generations · pop · T_objective). + +==== 8.3 Ternary Search + +*Theorem 8.3* (Ternary Search Termination). + +.... +(ternary-search f left right epsilon)↓_b +.... + +with bound B = O(log_\{3/2}((right - left)/epsilon) · T_f). + +_Proof_: Search interval shrinks by factor 2/3 each iteration. ∎ + +''''' + +=== 9. Distribution Sampling + +==== 9.1 Rejection Sampling + +*Theorem 9.1* (Rejection Sampling PAST). For acceptance rate p > 0: + +.... +(rejection-sampling target proposal M max-attempts)↓_{a.s.} +.... + +if max-attempts = ∞. With finite max-attempts, bounded termination. + +*Expected samples needed*: 1/p where p = E[acceptance probability]. + +==== 9.2 Inverse Transform + +*Theorem 9.2* (Inverse Transform Termination). For continuous +distributions with explicit quantile: + +.... +(inverse-transform-sample cdf)↓_b +.... + +==== 9.3 Box-Muller + +*Theorem 9.3* (Normal Sampling Termination). + +.... +(normal μ σ)↓_b +.... + +_Proof_: Box-Muller requires: 1. Two uniform samples: O(1) 2. Arithmetic +(log, sqrt, trig): O(1) + +Total: O(1). ∎ + +''''' + +=== 10. Bayesian Inference Termination + +==== 10.1 Conjugate Updates + +*Theorem 10.1* (Conjugate Prior Termination). All conjugate prior +updates terminate in O(1): + +.... +(conjugate-beta-binomial α β k n-k)↓_b +(conjugate-normal μ₀ σ₀² σ² observations)↓_b +.... + +==== 10.2 ABC + +*Theorem 10.2* (ABC Termination). ABC may not terminate if threshold is +too strict: + +.... +(abc-algorithm simulator data distance threshold n) +.... + +Terminates if P(distance(simulation, data) < threshold) > 0. + +''''' + +=== 11. Markov Chain Termination + +==== 11.1 Simulation + +*Theorem 11.1* (Markov Simulation Termination). + +.... +(markov-simulate chain n)↓_b +.... + +with bound B = O(n · k) where k = number of states. + +==== 11.2 Stationary Distribution + +*Theorem 11.2* (Stationary Estimation Termination). + +.... +(markov-stationary chain n-simulations)↓_b +.... + +''''' + +=== 12. Ranking Functions + +==== 12.1 Definition + +*Definition 12.1* (Ranking Function). A ranking function r: States → ℕ +satisfies: 1. r(s) = 0 ⟹ s is terminal 2. r(s) > 0 ⟹ r(s’) < r(s) after +transition with positive probability + +==== 12.2 Ranking Function for bet-until + +*Theorem 12.1* (bet-until Ranking). For `+(bet-until pred thunk)+`: + +If pred has probability p > 0 of success, define: + +.... +r(state) = 1 if ¬pred(state) +r(state) = 0 if pred(state) +.... + +P(r decreases) = p > 0, satisfying ranking function conditions. + +==== 12.3 Lexicographic Ranking + +For nested loops, use lexicographic ranking: + +.... +r(n₁, n₂) = (n₁, n₂) with lexicographic order +.... + +*Example* (nested bet-chain): + +[source,racket] +---- +(bet-chain n₁ (λ (x) (bet-chain n₂ f x)) init) +---- + +Ranking: (n₁, n₂) decreases lexicographically. + +''''' + +=== 13. Non-Termination Analysis + +==== 13.1 Identifying Non-Termination + +*Theorem 13.1* (Non-Termination Condition). `+(bet-until pred thunk)+` +does not terminate a.s. iff: + +.... +P(pred(thunk()) = true) = 0 +.... + +==== 13.2 Partial Termination + +*Definition 13.1* (Partial Termination). e↓_p iff P(e↓) = p for some p ∈ +(0, 1). + +*Example*: + +[source,racket] +---- +(bet-until (λ (x) (= x 'target)) + (λ () (bet 'target 'other 'loop))) +---- + +If ’loop causes infinite recursion in 1/3 of cases, P(↓) = 2/3. + +==== 13.3 Detecting Infinite Loops + +Static analysis can detect: 1. Unconditional recursion without base case +2. bet-until with impossible predicate 3. Infinite bet-chain (n = ∞) + +''''' + +=== 14. Expected Time Analysis + +==== 14.1 Expected Time Formulas + +[cols=",",options="header",] +|=== +|Construct |Expected Time +|bet |O(1) +|bet-chain n |O(n · T_f) +|bet-until (p > 0) |O(T_thunk/p) +|bet-repeat n |O(n · T_thunk) +|bet-parallel n |O(n) +|=== + +==== 14.2 Variance of Execution Time + +[cols=",",options="header",] +|=== +|Construct |Variance +|Deterministic |0 +|bet-until |O((1-p)/p² · T²_thunk) +|=== + +==== 14.3 Tail Bounds + +*Theorem 14.1* (Execution Time Tail Bound). For bet-until with success +probability p: + +.... +P(time > t) ≤ (1-p)^{t/T_thunk} +.... + +''''' + +=== 15. Termination Checking Algorithm + +==== 15.1 Conservative Static Analysis + +.... +terminate?(e) = match e with + | (bet A B C) → TRUE + | (bet-chain n f init) → n ∈ ℕ ∧ terminate?(f applied) + | (bet-until pred thunk) → MAYBE ; Cannot decide statically + | (f e) → terminate?(f) ∧ terminate?(e) + | ... +.... + +==== 15.2 Probabilistic Termination Verification + +For bet-until, verify: 1. Compute/estimate p = P(pred(thunk()) = true) +2. If p > 0: almost-sure termination 3. If p = 0: non-termination + +''''' + +=== 16. Summary + +[cols=",,",options="header",] +|=== +|Construct |Termination Class |Condition +|bet |Bounded |Always +|bet/weighted |Bounded |Weights valid +|bet/lazy |Bounded |Thunks terminate +|bet-chain |Bounded |n ∈ ℕ, f terminates +|bet-until |Almost-Sure/PAST |P(success) > 0 +|bet-repeat |Bounded |n ∈ ℕ, thunk terminates +|bet-parallel |Bounded |Always +|bet-map |Bounded |f terminates on elements +|bet-fold |Bounded |f terminates +|MCMC |Bounded |Fixed iterations +|rejection-sampling |PAST |Accept prob > 0 +|=== + +''''' + +=== 17. TODOs + +*TODO*: The following need formalization: + +[arabic] +. *Probabilistic termination logic*: Formal proof system for termination +. *Expected time type system*: Types indexed by expected time +. *Amortized analysis*: For cached/memoized operations +. *Concurrent termination*: For parallel bet extensions + +''''' + +=== References + +[arabic] +. McIver, A. & Morgan, C. (2005). _Abstraction, Refinement and Proof for +Probabilistic Systems_ +. Bournez, O. & Garnier, F. (2005). "`Proving positive almost sure +termination`" +. Chatterjee, K., et al. (2016). "`Algorithmic analysis of qualitative +and quantitative termination problems for affine probabilistic +programs`" +. Fioriti, L.M.F. & Hermanns, H. (2015). "`Probabilistic termination: +Soundness, completeness, and compositionality`" diff --git a/proofs/theorems/termination-analysis.md b/proofs/theorems/termination-analysis.md deleted file mode 100644 index 137c105..0000000 --- a/proofs/theorems/termination-analysis.md +++ /dev/null @@ -1,541 +0,0 @@ - -# Termination Analysis for Betlang - -## Abstract - -This document provides a comprehensive analysis of termination properties in betlang, distinguishing between deterministic termination, almost-sure termination, and positive almost-sure termination. We establish termination theorems for all core constructs and analyze the termination behavior of loops and recursive structures. - ---- - -## 1. Termination Classes - -### 1.1 Definitions - -**Definition 1.1** (Termination). A program e terminates (e↓) if evaluation reaches a value. - -**Definition 1.2** (Deterministic Termination). e↓_det iff e terminates for all random streams. - -**Definition 1.3** (Almost-Sure Termination). e↓_{a.s.} iff P(e↓) = 1. - -**Definition 1.4** (Positive Almost-Sure Termination - PAST). e↓_{PAST} iff E[termination time] < ∞. - -**Definition 1.5** (Bounded Termination). e↓_b iff ∃B. termination time ≤ B. - -### 1.2 Hierarchy - -$$\text{Bounded} \Rightarrow \text{Deterministic} \Rightarrow \text{PAST} \Rightarrow \text{Almost-Sure}$$ - -All implications are strict: -- Deterministic ⇏ Bounded: (bet-chain n f init) for large n -- PAST ⇏ Deterministic: bet-until with p < 1 -- Almost-Sure ⇏ PAST: Possible in theory (requires careful construction) - ---- - -## 2. Core Primitives - -### 2.1 bet - -**Theorem 2.1** (bet Bounded Termination). For any values A, B, C: -``` -(bet A B C) ↓_b with bound B = O(1) -``` - -*Proof*: -The bet operation: -1. Generate random number: O(1) -2. Index selection: O(1) -3. Value return: O(1) - -No loops, no recursion. Deterministic O(1) termination. ∎ - -### 2.2 bet/weighted - -**Theorem 2.2** (bet/weighted Bounded Termination). For non-negative weights with positive sum: -``` -(bet/weighted '(A wₐ) '(B w_b) '(C w_c)) ↓_b -``` - -*Proof*: Same structure as bet, O(1) operations. ∎ - -**Precondition violation**: If all weights are 0 or negative weights produce negative probabilities, behavior is undefined. - -### 2.3 bet/conditional - -**Theorem 2.3** (bet/conditional Termination). If pred terminates and selected branch terminates: -``` -pred↓ ∧ (pred = true ⟹ A↓) ∧ (pred = false ⟹ (bet B C A)↓) ⟹ (bet/conditional pred A B C)↓ -``` - -*Proof*: Follows from sequential composition of terminating operations. ∎ - -### 2.4 bet/lazy - -**Theorem 2.4** (bet/lazy Termination). If the selected thunk terminates: -``` -∀i. thunkᵢ↓ ⟹ (bet/lazy thunk₀ thunk₁ thunk₂)↓ -``` - -*Proof*: Only one thunk is evaluated. If all thunks terminate, the selected one terminates. ∎ - -**Corollary 2.1** (Non-terminating branches). If exactly one thunk may not terminate: -``` -P((bet/lazy thunk₀ thunk₁ thunk₂)↓) ≥ 2/3 -``` - ---- - -## 3. Iteration Constructs - -### 3.1 bet-chain - -**Theorem 3.1** (bet-chain Termination). -``` -n ∈ ℕ ∧ (∀v. f(v)↓) ⟹ (bet-chain n f init)↓_b -``` - -with bound B = O(n · Tf) where Tf is the time for f. - -*Proof*: -By induction on n: -- Base: n = 0 returns init immediately -- Step: n+1 = one call to f plus recursive call with n - -Total: exactly n calls to f. ∎ - -**Corollary 3.1** (bet-chain Complexity). Time = Θ(n · Tf). - -### 3.2 bet-until - -**Theorem 3.2** (bet-until Almost-Sure Termination). If P(pred(thunk())) = p > 0: -``` -(bet-until pred thunk) ↓_{a.s.} -``` - -*Proof*: -Let N = number of iterations. N ~ Geometric(p). -- P(N = k) = (1-p)^{k-1} · p -- P(N < ∞) = Σₖ P(N = k) = 1 for p > 0 - -Therefore, almost-sure termination. ∎ - -**Theorem 3.3** (bet-until PAST). If p > 0: -``` -(bet-until pred thunk) ↓_{PAST} -``` - -with E[iterations] = 1/p. - -*Proof*: -E[N] = Σₖ k · (1-p)^{k-1} · p = 1/p < ∞ ∎ - -**Theorem 3.4** (bet-until Variance). -``` -Var(N) = (1-p)/p² -``` - -**Theorem 3.5** (bet-until Concentration). For k > 0: -``` -P(N > k · E[N]) = P(N > k/p) = (1-p)^{k/p} ≤ e^{-k} -``` - -**Corollary 3.2** (High-Probability Bound). With probability 1 - δ: -``` -N ≤ (ln(1/δ))/p -``` - -### 3.3 bet-repeat - -**Theorem 3.6** (bet-repeat Termination). -``` -n ∈ ℕ ∧ thunk↓ ⟹ (bet-repeat n thunk)↓_b -``` - -with bound B = O(n · T_thunk). - -*Proof*: Exactly n evaluations of thunk. ∎ - -### 3.4 bet-parallel - -**Theorem 3.7** (bet-parallel Termination). -``` -(bet-parallel n A B C)↓_b -``` - -with bound B = O(n). - -*Proof*: n independent bet operations, each O(1). ∎ - ---- - -## 4. Higher-Order Operations - -### 4.1 bet-map - -**Theorem 4.1** (bet-map Termination). -``` -(∀x ∈ lst. f(x)↓) ⟹ (bet-map f lst)↓ -``` - -*Proof*: Maps f over each element. If f terminates on each, map terminates. ∎ - -**Probabilistic case**: If f involves randomness but terminates a.s., bet-map terminates a.s. - -### 4.2 bet-filter - -**Theorem 4.2** (bet-filter Termination). -``` -(∀x ∈ lst. pred(x)↓) ⟹ (bet-filter pred lst)↓ -``` - -### 4.3 bet-fold - -**Theorem 4.3** (bet-fold Termination). -``` -(∀acc, x. f(acc, x)↓) ⟹ (bet-fold f init lst)↓_b -``` - -with bound B = O(|lst| · Tf). - ---- - -## 5. Composition Operators - -### 5.1 bet-compose - -**Theorem 5.1** (bet-compose Termination). Composition creates terminating function if components terminate: -``` -(∀x. f(x)↓ ∧ g(x)↓ ∧ h(x)↓) ⟹ (∀x. ((bet-compose f g h) x)↓) -``` - -*Proof*: Selects one of f, g, h uniformly, applies to x. Selected function terminates. ∎ - -### 5.2 bet-sequence - -**Theorem 5.2** (bet-sequence Termination). -``` -(∀i. betᵢ↓) ⟹ (bet-sequence bet₁ bet₂ ... betₙ)↓ -``` - -*Proof*: Sequential execution of n terminating operations. ∎ - ---- - -## 6. Probabilistic Loops - -### 6.1 Random Walk Analysis - -**Theorem 6.1** (Random Walk Recurrence). For symmetric random walk on ℤ: -``` -P(return to 0) = 1 (recurrence) -E[return time] = ∞ (null recurrence) -``` - -**Application to bet-until**: -```racket -(bet-until (λ (pos) (= pos 0)) - (λ () (+ pos (bet -1 0 1)))) -``` - -This terminates a.s. but NOT with finite expected time. - -### 6.2 Biased Random Walk - -**Theorem 6.2** (Biased Walk Termination). For rightward bias: -``` -P(step = +1) > P(step = -1) -``` - -Then E[hitting time for target > current] < ∞. - -### 6.3 Martingale Stopping - -**Theorem 6.3** (Optional Stopping). If {Sₙ} is a martingale and τ is a stopping time with E[τ] < ∞: -``` -E[S_τ] = E[S₀] -``` - ---- - -## 7. MCMC Termination - -### 7.1 Fixed Iterations - -**Theorem 7.1** (MH Fixed Termination). -``` -(metropolis-hastings target proposal init n)↓_b -``` - -with bound B = O(n · (T_target + T_proposal)). - -*Proof*: Fixed n iterations, each involving target and proposal evaluation. ∎ - -### 7.2 Burn-in and Convergence - -**Note**: MCMC "convergence" refers to distributional convergence, not program termination. The programs terminate after fixed iterations; convergence is a statistical property. - ---- - -## 8. Optimization Termination - -### 8.1 Simulated Annealing - -**Theorem 8.1** (SA Termination). -``` -(simulated-annealing objective init schedule max-iter neighbor)↓_b -``` - -with bound B = O(max-iter · (T_objective + T_neighbor)). - -### 8.2 Genetic Algorithm - -**Theorem 8.2** (GA Termination). -``` -(genetic-algorithm objective pop generations ...)↓_b -``` - -with bound B = O(generations · pop · T_objective). - -### 8.3 Ternary Search - -**Theorem 8.3** (Ternary Search Termination). -``` -(ternary-search f left right epsilon)↓_b -``` - -with bound B = O(log_{3/2}((right - left)/epsilon) · T_f). - -*Proof*: Search interval shrinks by factor 2/3 each iteration. ∎ - ---- - -## 9. Distribution Sampling - -### 9.1 Rejection Sampling - -**Theorem 9.1** (Rejection Sampling PAST). For acceptance rate p > 0: -``` -(rejection-sampling target proposal M max-attempts)↓_{a.s.} -``` - -if max-attempts = ∞. With finite max-attempts, bounded termination. - -**Expected samples needed**: 1/p where p = E[acceptance probability]. - -### 9.2 Inverse Transform - -**Theorem 9.2** (Inverse Transform Termination). For continuous distributions with explicit quantile: -``` -(inverse-transform-sample cdf)↓_b -``` - -### 9.3 Box-Muller - -**Theorem 9.3** (Normal Sampling Termination). -``` -(normal μ σ)↓_b -``` - -*Proof*: Box-Muller requires: -1. Two uniform samples: O(1) -2. Arithmetic (log, sqrt, trig): O(1) - -Total: O(1). ∎ - ---- - -## 10. Bayesian Inference Termination - -### 10.1 Conjugate Updates - -**Theorem 10.1** (Conjugate Prior Termination). All conjugate prior updates terminate in O(1): -``` -(conjugate-beta-binomial α β k n-k)↓_b -(conjugate-normal μ₀ σ₀² σ² observations)↓_b -``` - -### 10.2 ABC - -**Theorem 10.2** (ABC Termination). ABC may not terminate if threshold is too strict: -``` -(abc-algorithm simulator data distance threshold n) -``` - -Terminates if P(distance(simulation, data) < threshold) > 0. - ---- - -## 11. Markov Chain Termination - -### 11.1 Simulation - -**Theorem 11.1** (Markov Simulation Termination). -``` -(markov-simulate chain n)↓_b -``` - -with bound B = O(n · k) where k = number of states. - -### 11.2 Stationary Distribution - -**Theorem 11.2** (Stationary Estimation Termination). -``` -(markov-stationary chain n-simulations)↓_b -``` - ---- - -## 12. Ranking Functions - -### 12.1 Definition - -**Definition 12.1** (Ranking Function). A ranking function r: States → ℕ satisfies: -1. r(s) = 0 ⟹ s is terminal -2. r(s) > 0 ⟹ r(s') < r(s) after transition with positive probability - -### 12.2 Ranking Function for bet-until - -**Theorem 12.1** (bet-until Ranking). For `(bet-until pred thunk)`: - -If pred has probability p > 0 of success, define: -``` -r(state) = 1 if ¬pred(state) -r(state) = 0 if pred(state) -``` - -P(r decreases) = p > 0, satisfying ranking function conditions. - -### 12.3 Lexicographic Ranking - -For nested loops, use lexicographic ranking: -``` -r(n₁, n₂) = (n₁, n₂) with lexicographic order -``` - -**Example** (nested bet-chain): -```racket -(bet-chain n₁ (λ (x) (bet-chain n₂ f x)) init) -``` - -Ranking: (n₁, n₂) decreases lexicographically. - ---- - -## 13. Non-Termination Analysis - -### 13.1 Identifying Non-Termination - -**Theorem 13.1** (Non-Termination Condition). `(bet-until pred thunk)` does not terminate a.s. iff: -``` -P(pred(thunk()) = true) = 0 -``` - -### 13.2 Partial Termination - -**Definition 13.1** (Partial Termination). e↓_p iff P(e↓) = p for some p ∈ (0, 1). - -**Example**: -```racket -(bet-until (λ (x) (= x 'target)) - (λ () (bet 'target 'other 'loop))) -``` - -If 'loop causes infinite recursion in 1/3 of cases, P(↓) = 2/3. - -### 13.3 Detecting Infinite Loops - -Static analysis can detect: -1. Unconditional recursion without base case -2. bet-until with impossible predicate -3. Infinite bet-chain (n = ∞) - ---- - -## 14. Expected Time Analysis - -### 14.1 Expected Time Formulas - -| Construct | Expected Time | -|-----------|---------------| -| bet | O(1) | -| bet-chain n | O(n · T_f) | -| bet-until (p > 0) | O(T_thunk/p) | -| bet-repeat n | O(n · T_thunk) | -| bet-parallel n | O(n) | - -### 14.2 Variance of Execution Time - -| Construct | Variance | -|-----------|----------| -| Deterministic | 0 | -| bet-until | O((1-p)/p² · T²_thunk) | - -### 14.3 Tail Bounds - -**Theorem 14.1** (Execution Time Tail Bound). For bet-until with success probability p: -``` -P(time > t) ≤ (1-p)^{t/T_thunk} -``` - ---- - -## 15. Termination Checking Algorithm - -### 15.1 Conservative Static Analysis - -``` -terminate?(e) = match e with - | (bet A B C) → TRUE - | (bet-chain n f init) → n ∈ ℕ ∧ terminate?(f applied) - | (bet-until pred thunk) → MAYBE ; Cannot decide statically - | (f e) → terminate?(f) ∧ terminate?(e) - | ... -``` - -### 15.2 Probabilistic Termination Verification - -For bet-until, verify: -1. Compute/estimate p = P(pred(thunk()) = true) -2. If p > 0: almost-sure termination -3. If p = 0: non-termination - ---- - -## 16. Summary - -| Construct | Termination Class | Condition | -|-----------|-------------------|-----------| -| bet | Bounded | Always | -| bet/weighted | Bounded | Weights valid | -| bet/lazy | Bounded | Thunks terminate | -| bet-chain | Bounded | n ∈ ℕ, f terminates | -| bet-until | Almost-Sure/PAST | P(success) > 0 | -| bet-repeat | Bounded | n ∈ ℕ, thunk terminates | -| bet-parallel | Bounded | Always | -| bet-map | Bounded | f terminates on elements | -| bet-fold | Bounded | f terminates | -| MCMC | Bounded | Fixed iterations | -| rejection-sampling | PAST | Accept prob > 0 | - ---- - -## 17. TODOs - -**TODO**: The following need formalization: - -1. **Probabilistic termination logic**: Formal proof system for termination -2. **Expected time type system**: Types indexed by expected time -3. **Amortized analysis**: For cached/memoized operations -4. **Concurrent termination**: For parallel bet extensions - ---- - -## References - -1. McIver, A. & Morgan, C. (2005). *Abstraction, Refinement and Proof for Probabilistic Systems* -2. Bournez, O. & Garnier, F. (2005). "Proving positive almost sure termination" -3. Chatterjee, K., et al. (2016). "Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs" -4. Fioriti, L.M.F. & Hermanns, H. (2015). "Probabilistic termination: Soundness, completeness, and compositionality" diff --git a/proofs/theorems/type-theory.adoc b/proofs/theorems/type-theory.adoc new file mode 100644 index 0000000..c4b5515 --- /dev/null +++ b/proofs/theorems/type-theory.adoc @@ -0,0 +1,599 @@ +== Type Theory and Probability Monad Proofs + +=== Abstract + +This document establishes the type-theoretic foundations of betlang, +formalizing the probability monad structure, proving the monad laws, and +establishing type safety. We present a gradual type system suitable for +a dynamically-typed host language (Racket) with optional static typing +guarantees. + +''''' + +=== 1. The Probability Monad + +==== 1.1 Monad Definition + +*Definition 1.1* (Probability Monad). The triple (Dist, η, μ) forms a +monad where: + +* *Dist*: Type → Type is the distribution type constructor +* *η*: A → Dist(A) is the unit (return) +* *μ*: Dist(Dist(A)) → Dist(A) is the join (flatten) + +Equivalently, using Kleisli presentation with bind: - *return*: A → +Dist(A) - *>>=* (bind): Dist(A) → (A → Dist(B)) → Dist(B) + +==== 1.2 Concrete Implementation + +[source,racket] +---- +;; return : A → Dist(A) +(define (bet-pure x) + (bet x x x)) ; Dirac distribution at x + +;; bind : Dist(A) → (A → Dist(B)) → Dist(B) +(define (bet-bind m f) + (f (m))) ; Sample from m, then apply f +---- + +==== 1.3 Monad Laws + +*Theorem 1.1* (Left Identity). For all x and f: + +.... +(bet-bind (bet-pure x) f) ≡ (f x) +.... + +_Proof_: + +.... + (bet-bind (bet-pure x) f) += (bet-bind (bet x x x) f) += (f (bet x x x)) += (f x) ; since (bet x x x) = x with probability 1 +.... + +∎ + +*Theorem 1.2* (Right Identity). For all m: + +.... +(bet-bind m bet-pure) ≡ m +.... + +_Proof_: + +.... + (bet-bind m bet-pure) += (bet-pure (m)) += (bet (m) (m) (m)) +.... + +This is distributionally equivalent to m since bet-pure maps each +outcome back to itself. More formally: + +Let m produce value v with probability P(v). Then: - bet-bind m bet-pure +produces (bet v v v) for each sampled v - (bet v v v) = v with +probability 1 - Therefore the output distribution equals the input +distribution ∎ + +*Theorem 1.3* (Associativity). For all m, f, g: + +.... +(bet-bind (bet-bind m f) g) ≡ (bet-bind m (λ x. bet-bind (f x) g)) +.... + +_Proof_: + +.... +LHS = (bet-bind (bet-bind m f) g) + = (g ((bet-bind m f))) + = (g ((f (m)))) + +RHS = (bet-bind m (λ x. bet-bind (f x) g)) + = ((λ x. bet-bind (f x) g) (m)) + = (bet-bind (f (m)) g) + = (g ((f (m)))) + +LHS = RHS +.... + +∎ + +''''' + +=== 2. Functor Laws + +==== 2.1 Definition + +*Definition 2.1* (bet-map as Functor). The map operation: + +[source,racket] +---- +(define (bet-fmap f m) + (bet-bind m (λ x. bet-pure (f x)))) +---- + +==== 2.2 Functor Laws + +*Theorem 2.1* (Identity). + +.... +(bet-fmap identity m) ≡ m +.... + +_Proof_: + +.... + (bet-fmap identity m) += (bet-bind m (λ x. bet-pure (identity x))) += (bet-bind m (λ x. bet-pure x)) += (bet-bind m bet-pure) += m ; by right identity +.... + +∎ + +*Theorem 2.2* (Composition). + +.... +(bet-fmap (compose f g) m) ≡ (bet-fmap f (bet-fmap g m)) +.... + +_Proof_: + +.... +LHS = (bet-bind m (λ x. bet-pure ((compose f g) x))) + = (bet-bind m (λ x. bet-pure (f (g x)))) + +RHS = (bet-fmap f (bet-bind m (λ x. bet-pure (g x)))) + = (bet-bind (bet-bind m (λ x. bet-pure (g x))) (λ y. bet-pure (f y))) + = (bet-bind m (λ x. bet-bind (bet-pure (g x)) (λ y. bet-pure (f y)))) ; assoc + = (bet-bind m (λ x. (λ y. bet-pure (f y)) (g x))) ; left identity + = (bet-bind m (λ x. bet-pure (f (g x)))) + +LHS = RHS +.... + +∎ + +''''' + +=== 3. Applicative Functor Laws + +==== 3.1 Definition + +*Definition 3.1* (Applicative Operations). + +[source,racket] +---- +(define (bet-ap mf mx) + (bet-bind mf (λ f. bet-bind mx (λ x. bet-pure (f x))))) + +(define bet-pure bet-pure) ; same as monad return +---- + +==== 3.2 Applicative Laws + +*Theorem 3.1* (Identity). + +.... +(bet-ap (bet-pure identity) m) ≡ m +.... + +_Proof_: + +.... + (bet-ap (bet-pure identity) m) += (bet-bind (bet-pure identity) (λ f. bet-bind m (λ x. bet-pure (f x)))) += (bet-bind m (λ x. bet-pure (identity x))) ; left identity += (bet-bind m bet-pure) += m ; right identity +.... + +∎ + +*Theorem 3.2* (Homomorphism). + +.... +(bet-ap (bet-pure f) (bet-pure x)) ≡ (bet-pure (f x)) +.... + +_Proof_: + +.... + (bet-ap (bet-pure f) (bet-pure x)) += (bet-bind (bet-pure f) (λ g. bet-bind (bet-pure x) (λ y. bet-pure (g y)))) += (bet-bind (bet-pure x) (λ y. bet-pure (f y))) ; left identity += (bet-pure (f x)) ; left identity +.... + +∎ + +*Theorem 3.3* (Interchange). + +.... +(bet-ap mf (bet-pure x)) ≡ (bet-ap (bet-pure (λ f. f x)) mf) +.... + +_Proof_: By expansion using monad laws and function application. ∎ + +*Theorem 3.4* (Composition). + +.... +(bet-ap (bet-ap (bet-ap (bet-pure compose) mf) mg) mx) +≡ (bet-ap mf (bet-ap mg mx)) +.... + +_Proof_: Follows from monad associativity. ∎ + +''''' + +=== 4. Kleisli Category + +==== 4.1 Objects and Morphisms + +*Definition 4.1* (Kleisli Category for Dist). - *Objects*: Types A, B, +C, … - *Morphisms*: Kleisli arrows A ⇝ B = A → Dist(B) - *Identity*: η : +A → Dist(A) - *Composition*: (g ∘_K f) = λx. bind (f x) g + +==== 4.2 Category Laws + +*Theorem 4.1* (Left Unit). + +.... +η ∘_K f ≡ f +.... + +_Proof_: + +.... +(η ∘_K f)(x) = bind (f x) η = f x ; by right identity +.... + +∎ + +*Theorem 4.2* (Right Unit). + +.... +f ∘_K η ≡ f +.... + +_Proof_: + +.... +(f ∘_K η)(x) = bind (η x) f = f x ; by left identity +.... + +∎ + +*Theorem 4.3* (Associativity). + +.... +(h ∘_K g) ∘_K f ≡ h ∘_K (g ∘_K f) +.... + +_Proof_: Follows from monad associativity. ∎ + +''''' + +=== 5. Type System + +==== 5.1 Base Types + +.... +BaseType ::= Int | Real | Bool | Symbol | Unit | Void +.... + +==== 5.2 Compound Types + +.... +Type τ ::= BaseType + | τ → τ ; Function + | Dist τ ; Distribution + | List τ ; Homogeneous list + | (τ₁ × τ₂ × ... × τₙ) ; Product/tuple + | ∀α. τ ; Universal quantification +.... + +==== 5.3 Typing Judgments + +*Judgment Form*: Γ ⊢ e : τ + +where Γ is a typing context mapping variables to types. + +==== 5.4 Core Typing Rules + +*Literals* + +[latexmath] +++++ +\frac{n ∈ ℤ}{Γ ⊢ n : \text{Int}} \quad \frac{r ∈ ℝ}{Γ ⊢ r : \text{Real}} \quad \frac{b ∈ \{\text{true}, \text{false}\}}{Γ ⊢ b : \text{Bool}} +++++ + +*Variables* + +[latexmath] +++++ +\frac{x : τ ∈ Γ}{Γ ⊢ x : τ} +++++ + +*Abstraction* + +[latexmath] +++++ +\frac{Γ, x : τ_1 ⊢ e : τ_2}{Γ ⊢ (λ (x) e) : τ_1 → τ_2} +++++ + +*Application* + +[latexmath] +++++ +\frac{Γ ⊢ e_1 : τ_1 → τ_2 \quad Γ ⊢ e_2 : τ_1}{Γ ⊢ (e_1 \ e_2) : τ_2} +++++ + +*Bet* + +[latexmath] +++++ +\frac{Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ \quad Γ ⊢ e_3 : τ}{Γ ⊢ (\text{bet } e_1 \ e_2 \ e_3) : \text{Dist } τ} +++++ + +*Bet-Pure* + +[latexmath] +++++ +\frac{Γ ⊢ e : τ}{Γ ⊢ (\text{bet-pure } e) : \text{Dist } τ} +++++ + +*Bet-Bind* + +[latexmath] +++++ +\frac{Γ ⊢ m : \text{Dist } τ_1 \quad Γ ⊢ f : τ_1 → \text{Dist } τ_2}{Γ ⊢ (\text{bet-bind } m \ f) : \text{Dist } τ_2} +++++ + +*Sample* (extract from distribution) + +[latexmath] +++++ +\frac{Γ ⊢ e : \text{Dist } τ}{Γ ⊢ (\text{sample } e) : τ} +++++ + +''''' + +=== 6. Type Safety + +==== 6.1 Progress + +*Theorem 6.1* (Progress). If ⊢ e : τ (closed, well-typed), then either: +1. e is a value, or 2. ∃e’. e → e’ (e can take a step) + +_Proof_: By induction on the typing derivation. + +*Case* bet e₁ e₂ e₃: - By IH, each eᵢ either is a value or steps - If +all are values, the bet expression steps via E-BET - If any eᵢ steps, +the whole expression steps by congruence + +*Case* application (e₁ e₂): - By IH on e₁ and e₂ - If e₁ = λx.e and e₂ = +v, step by β-reduction - Otherwise, step a subexpression + +Other cases follow standard patterns. ∎ + +==== 6.2 Preservation + +*Theorem 6.2* (Preservation). If Γ ⊢ e : τ and e → e’, then Γ ⊢ e’ : τ. + +_Proof_: By induction on the evaluation derivation. + +*Case* E-BET: (bet v₁ v₂ v₃) → vᵢ - By inversion, Γ ⊢ vᵢ : τ for all i - +Therefore Γ ⊢ vᵢ : τ ✓ + +*Case* β-reduction: ((λx.e) v) → e[x ↦ v] - By inversion, Γ ⊢ λx.e : τ₁ +→ τ₂ and Γ ⊢ v : τ₁ - So Γ, x:τ₁ ⊢ e : τ₂ - By substitution lemma, Γ ⊢ +e[x ↦ v] : τ₂ ✓ + +∎ + +==== 6.3 Type Soundness + +*Theorem 6.3* (Type Soundness). Well-typed programs don’t go wrong. + +If ⊢ e : τ and e →* e’ where e’ is irreducible, then e’ is a value of +type τ. + +_Proof_: By induction using Progress and Preservation. ∎ + +''''' + +=== 7. Subtyping for Distributions + +==== 7.1 Distribution Subtyping + +*Definition 7.1* (Distribution Subtyping). + +[latexmath] +++++ +\frac{τ_1 <: τ_2}{\text{Dist } τ_1 <: \text{Dist } τ_2} +++++ + +Distributions are covariant in their element type. + +==== 7.2 Numeric Subtyping + +[latexmath] +++++ +\text{Int} <: \text{Real} +++++ + +*Theorem 7.1* (Subsumption). + +[latexmath] +++++ +\frac{Γ ⊢ e : τ_1 \quad τ_1 <: τ_2}{Γ ⊢ e : τ_2} +++++ + +''''' + +=== 8. Effect System for Randomness + +==== 8.1 Effect Annotations + +We can track probabilistic effects in types: + +.... +τ ::= ... | τ !ε ; τ with effect ε +ε ::= Pure | Prob | ε₁ ∪ ε₂ +.... + +==== 8.2 Effect Rules + +*Bet produces Prob effect*: + +[latexmath] +++++ +\frac{Γ ⊢ e_i : τ \ !\varepsilon_i}{Γ ⊢ (\text{bet } e_1 \ e_2 \ e_3) : τ \ !(\text{Prob} ∪ \varepsilon_1 ∪ \varepsilon_2 ∪ \varepsilon_3)} +++++ + +*Pure computations*: + +[latexmath] +++++ +\frac{Γ ⊢ e : τ \ !\text{Pure}}{e \text{ is deterministic}} +++++ + +==== 8.3 Effect Masking + +[latexmath] +++++ +\frac{Γ ⊢ e : τ \ !\varepsilon \quad \text{seed fixed}}{Γ ⊢ (\text{bet-with-seed } s \ e) : τ \ !\text{Pure}} +++++ + +Fixing the random seed converts probabilistic to deterministic. + +''''' + +=== 9. Parametricity + +==== 9.1 Free Theorems + +*Theorem 9.1* (Parametricity for bet-map). For bet-map with type: + +.... +∀α β. (α → β) → Dist α → Dist β +.... + +For any f: A → B and g: B → C: + +.... +bet-map (g ∘ f) ≡ bet-map g ∘ bet-map f +.... + +This is the free theorem derived from the type. + +==== 9.2 Naturality + +*Theorem 9.2* (Natural Transformation). bet-pure is a natural +transformation: + +.... +Id → Dist +.... + +For any f: A → B: + +.... +bet-pure ∘ f ≡ bet-map f ∘ bet-pure +.... + +_Proof_: + +.... +(bet-pure ∘ f)(x) = bet-pure (f x) +(bet-map f ∘ bet-pure)(x) = bet-map f (bet-pure x) + = bet-bind (bet-pure x) (λ y. bet-pure (f y)) + = bet-pure (f x) ; by left identity +.... + +∎ + +''''' + +=== 10. Dependent Types (Extension) + +==== 10.1 Indexed Distributions + +For advanced type safety, we can index by probability: + +.... +Dist_{p} τ ; Distribution with probability p of success +.... + +==== 10.2 Dependent Bet + +[latexmath] +++++ +\frac{Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ \quad Γ ⊢ e_3 : τ}{Γ ⊢ (\text{bet } e_1 \ e_2 \ e_3) : \text{Dist}_{1/3} τ × \text{Dist}_{1/3} τ × \text{Dist}_{1/3} τ} +++++ + +*TODO*: Full dependent type system requires more infrastructure: - Π +types for probability-indexed functions - Σ types for existential +probability bounds - Refinement types for probabilistic assertions + +''''' + +=== 11. Probability Algebra + +==== 11.1 Convex Combination + +*Definition 11.1* (Convex Space). Dist(A) forms a convex space: + +For p ∈ [0,1] and distributions μ, ν: + +[latexmath] +++++ +p \cdot μ + (1-p) \cdot ν +++++ + +is a valid distribution. + +*Theorem 11.1* (Bet as Convex Combination). + +.... +(bet A B C) = (1/3)·δ_A + (1/3)·δ_B + (1/3)·δ_C +.... + +==== 11.2 Barycentric Algebra + +The set of distributions satisfies: 1. *Idempotence*: p·μ + (1-p)·μ = μ +2. *Skew-commutativity*: p·μ + (1-p)·ν = (1-p)·ν + p·μ 3. +*Skew-associativity*: Nesting of mixtures associates properly + +''''' + +=== 12. Categorical Semantics Summary + +[cols=",",options="header",] +|=== +|Structure |Betlang Instantiation +|Category |Kleisli(Dist) +|Functor |bet-map +|Natural Transformation |bet-pure +|Monad |(Dist, bet-pure, bet-bind) +|Applicative |(Dist, bet-pure, bet-ap) +|Convex Space |probability mixtures +|=== + +''''' + +=== References + +[arabic] +. Wadler, P. (1995). "`Monads for Functional Programming`" +. Ramsey, N. & Pfeffer, A. (2002). "`Stochastic Lambda Calculus and +Monads of Probability Distributions`" +. Moggi, E. (1991). "`Notions of Computation and Monads`" +. Heunen, C., Kammar, O., et al. (2017). "`A Convenient Category for +Higher-Order Probability Theory`" +. Pierce, B.C. (2002). "`Types and Programming Languages`" diff --git a/proofs/theorems/type-theory.md b/proofs/theorems/type-theory.md deleted file mode 100644 index 6c55ab6..0000000 --- a/proofs/theorems/type-theory.md +++ /dev/null @@ -1,500 +0,0 @@ - -# Type Theory and Probability Monad Proofs - -## Abstract - -This document establishes the type-theoretic foundations of betlang, formalizing the probability monad structure, proving the monad laws, and establishing type safety. We present a gradual type system suitable for a dynamically-typed host language (Racket) with optional static typing guarantees. - ---- - -## 1. The Probability Monad - -### 1.1 Monad Definition - -**Definition 1.1** (Probability Monad). The triple (Dist, η, μ) forms a monad where: - -- **Dist**: Type → Type is the distribution type constructor -- **η**: A → Dist(A) is the unit (return) -- **μ**: Dist(Dist(A)) → Dist(A) is the join (flatten) - -Equivalently, using Kleisli presentation with bind: -- **return**: A → Dist(A) -- **>>=** (bind): Dist(A) → (A → Dist(B)) → Dist(B) - -### 1.2 Concrete Implementation - -```racket -;; return : A → Dist(A) -(define (bet-pure x) - (bet x x x)) ; Dirac distribution at x - -;; bind : Dist(A) → (A → Dist(B)) → Dist(B) -(define (bet-bind m f) - (f (m))) ; Sample from m, then apply f -``` - -### 1.3 Monad Laws - -**Theorem 1.1** (Left Identity). For all x and f: -``` -(bet-bind (bet-pure x) f) ≡ (f x) -``` - -*Proof*: -``` - (bet-bind (bet-pure x) f) -= (bet-bind (bet x x x) f) -= (f (bet x x x)) -= (f x) ; since (bet x x x) = x with probability 1 -``` -∎ - -**Theorem 1.2** (Right Identity). For all m: -``` -(bet-bind m bet-pure) ≡ m -``` - -*Proof*: -``` - (bet-bind m bet-pure) -= (bet-pure (m)) -= (bet (m) (m) (m)) -``` - -This is distributionally equivalent to m since bet-pure maps each outcome back to itself. More formally: - -Let m produce value v with probability P(v). Then: -- bet-bind m bet-pure produces (bet v v v) for each sampled v -- (bet v v v) = v with probability 1 -- Therefore the output distribution equals the input distribution -∎ - -**Theorem 1.3** (Associativity). For all m, f, g: -``` -(bet-bind (bet-bind m f) g) ≡ (bet-bind m (λ x. bet-bind (f x) g)) -``` - -*Proof*: -``` -LHS = (bet-bind (bet-bind m f) g) - = (g ((bet-bind m f))) - = (g ((f (m)))) - -RHS = (bet-bind m (λ x. bet-bind (f x) g)) - = ((λ x. bet-bind (f x) g) (m)) - = (bet-bind (f (m)) g) - = (g ((f (m)))) - -LHS = RHS -``` -∎ - ---- - -## 2. Functor Laws - -### 2.1 Definition - -**Definition 2.1** (bet-map as Functor). The map operation: -```racket -(define (bet-fmap f m) - (bet-bind m (λ x. bet-pure (f x)))) -``` - -### 2.2 Functor Laws - -**Theorem 2.1** (Identity). -``` -(bet-fmap identity m) ≡ m -``` - -*Proof*: -``` - (bet-fmap identity m) -= (bet-bind m (λ x. bet-pure (identity x))) -= (bet-bind m (λ x. bet-pure x)) -= (bet-bind m bet-pure) -= m ; by right identity -``` -∎ - -**Theorem 2.2** (Composition). -``` -(bet-fmap (compose f g) m) ≡ (bet-fmap f (bet-fmap g m)) -``` - -*Proof*: -``` -LHS = (bet-bind m (λ x. bet-pure ((compose f g) x))) - = (bet-bind m (λ x. bet-pure (f (g x)))) - -RHS = (bet-fmap f (bet-bind m (λ x. bet-pure (g x)))) - = (bet-bind (bet-bind m (λ x. bet-pure (g x))) (λ y. bet-pure (f y))) - = (bet-bind m (λ x. bet-bind (bet-pure (g x)) (λ y. bet-pure (f y)))) ; assoc - = (bet-bind m (λ x. (λ y. bet-pure (f y)) (g x))) ; left identity - = (bet-bind m (λ x. bet-pure (f (g x)))) - -LHS = RHS -``` -∎ - ---- - -## 3. Applicative Functor Laws - -### 3.1 Definition - -**Definition 3.1** (Applicative Operations). -```racket -(define (bet-ap mf mx) - (bet-bind mf (λ f. bet-bind mx (λ x. bet-pure (f x))))) - -(define bet-pure bet-pure) ; same as monad return -``` - -### 3.2 Applicative Laws - -**Theorem 3.1** (Identity). -``` -(bet-ap (bet-pure identity) m) ≡ m -``` - -*Proof*: -``` - (bet-ap (bet-pure identity) m) -= (bet-bind (bet-pure identity) (λ f. bet-bind m (λ x. bet-pure (f x)))) -= (bet-bind m (λ x. bet-pure (identity x))) ; left identity -= (bet-bind m bet-pure) -= m ; right identity -``` -∎ - -**Theorem 3.2** (Homomorphism). -``` -(bet-ap (bet-pure f) (bet-pure x)) ≡ (bet-pure (f x)) -``` - -*Proof*: -``` - (bet-ap (bet-pure f) (bet-pure x)) -= (bet-bind (bet-pure f) (λ g. bet-bind (bet-pure x) (λ y. bet-pure (g y)))) -= (bet-bind (bet-pure x) (λ y. bet-pure (f y))) ; left identity -= (bet-pure (f x)) ; left identity -``` -∎ - -**Theorem 3.3** (Interchange). -``` -(bet-ap mf (bet-pure x)) ≡ (bet-ap (bet-pure (λ f. f x)) mf) -``` - -*Proof*: By expansion using monad laws and function application. ∎ - -**Theorem 3.4** (Composition). -``` -(bet-ap (bet-ap (bet-ap (bet-pure compose) mf) mg) mx) -≡ (bet-ap mf (bet-ap mg mx)) -``` - -*Proof*: Follows from monad associativity. ∎ - ---- - -## 4. Kleisli Category - -### 4.1 Objects and Morphisms - -**Definition 4.1** (Kleisli Category for Dist). -- **Objects**: Types A, B, C, ... -- **Morphisms**: Kleisli arrows A ⇝ B = A → Dist(B) -- **Identity**: η : A → Dist(A) -- **Composition**: (g ∘_K f) = λx. bind (f x) g - -### 4.2 Category Laws - -**Theorem 4.1** (Left Unit). -``` -η ∘_K f ≡ f -``` - -*Proof*: -``` -(η ∘_K f)(x) = bind (f x) η = f x ; by right identity -``` -∎ - -**Theorem 4.2** (Right Unit). -``` -f ∘_K η ≡ f -``` - -*Proof*: -``` -(f ∘_K η)(x) = bind (η x) f = f x ; by left identity -``` -∎ - -**Theorem 4.3** (Associativity). -``` -(h ∘_K g) ∘_K f ≡ h ∘_K (g ∘_K f) -``` - -*Proof*: Follows from monad associativity. ∎ - ---- - -## 5. Type System - -### 5.1 Base Types - -``` -BaseType ::= Int | Real | Bool | Symbol | Unit | Void -``` - -### 5.2 Compound Types - -``` -Type τ ::= BaseType - | τ → τ ; Function - | Dist τ ; Distribution - | List τ ; Homogeneous list - | (τ₁ × τ₂ × ... × τₙ) ; Product/tuple - | ∀α. τ ; Universal quantification -``` - -### 5.3 Typing Judgments - -**Judgment Form**: Γ ⊢ e : τ - -where Γ is a typing context mapping variables to types. - -### 5.4 Core Typing Rules - -**Literals** -$$\frac{n ∈ ℤ}{Γ ⊢ n : \text{Int}} \quad \frac{r ∈ ℝ}{Γ ⊢ r : \text{Real}} \quad \frac{b ∈ \{\text{true}, \text{false}\}}{Γ ⊢ b : \text{Bool}}$$ - -**Variables** -$$\frac{x : τ ∈ Γ}{Γ ⊢ x : τ}$$ - -**Abstraction** -$$\frac{Γ, x : τ_1 ⊢ e : τ_2}{Γ ⊢ (λ (x) e) : τ_1 → τ_2}$$ - -**Application** -$$\frac{Γ ⊢ e_1 : τ_1 → τ_2 \quad Γ ⊢ e_2 : τ_1}{Γ ⊢ (e_1 \ e_2) : τ_2}$$ - -**Bet** -$$\frac{Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ \quad Γ ⊢ e_3 : τ}{Γ ⊢ (\text{bet } e_1 \ e_2 \ e_3) : \text{Dist } τ}$$ - -**Bet-Pure** -$$\frac{Γ ⊢ e : τ}{Γ ⊢ (\text{bet-pure } e) : \text{Dist } τ}$$ - -**Bet-Bind** -$$\frac{Γ ⊢ m : \text{Dist } τ_1 \quad Γ ⊢ f : τ_1 → \text{Dist } τ_2}{Γ ⊢ (\text{bet-bind } m \ f) : \text{Dist } τ_2}$$ - -**Sample** (extract from distribution) -$$\frac{Γ ⊢ e : \text{Dist } τ}{Γ ⊢ (\text{sample } e) : τ}$$ - ---- - -## 6. Type Safety - -### 6.1 Progress - -**Theorem 6.1** (Progress). If ⊢ e : τ (closed, well-typed), then either: -1. e is a value, or -2. ∃e'. e → e' (e can take a step) - -*Proof*: By induction on the typing derivation. - -**Case** bet e₁ e₂ e₃: -- By IH, each eᵢ either is a value or steps -- If all are values, the bet expression steps via E-BET -- If any eᵢ steps, the whole expression steps by congruence - -**Case** application (e₁ e₂): -- By IH on e₁ and e₂ -- If e₁ = λx.e and e₂ = v, step by β-reduction -- Otherwise, step a subexpression - -Other cases follow standard patterns. ∎ - -### 6.2 Preservation - -**Theorem 6.2** (Preservation). If Γ ⊢ e : τ and e → e', then Γ ⊢ e' : τ. - -*Proof*: By induction on the evaluation derivation. - -**Case** E-BET: (bet v₁ v₂ v₃) → vᵢ -- By inversion, Γ ⊢ vᵢ : τ for all i -- Therefore Γ ⊢ vᵢ : τ ✓ - -**Case** β-reduction: ((λx.e) v) → e[x ↦ v] -- By inversion, Γ ⊢ λx.e : τ₁ → τ₂ and Γ ⊢ v : τ₁ -- So Γ, x:τ₁ ⊢ e : τ₂ -- By substitution lemma, Γ ⊢ e[x ↦ v] : τ₂ ✓ - -∎ - -### 6.3 Type Soundness - -**Theorem 6.3** (Type Soundness). Well-typed programs don't go wrong. - -If ⊢ e : τ and e →* e' where e' is irreducible, then e' is a value of type τ. - -*Proof*: By induction using Progress and Preservation. ∎ - ---- - -## 7. Subtyping for Distributions - -### 7.1 Distribution Subtyping - -**Definition 7.1** (Distribution Subtyping). -$$\frac{τ_1 <: τ_2}{\text{Dist } τ_1 <: \text{Dist } τ_2}$$ - -Distributions are covariant in their element type. - -### 7.2 Numeric Subtyping - -$$\text{Int} <: \text{Real}$$ - -**Theorem 7.1** (Subsumption). -$$\frac{Γ ⊢ e : τ_1 \quad τ_1 <: τ_2}{Γ ⊢ e : τ_2}$$ - ---- - -## 8. Effect System for Randomness - -### 8.1 Effect Annotations - -We can track probabilistic effects in types: - -``` -τ ::= ... | τ !ε ; τ with effect ε -ε ::= Pure | Prob | ε₁ ∪ ε₂ -``` - -### 8.2 Effect Rules - -**Bet produces Prob effect**: -$$\frac{Γ ⊢ e_i : τ \ !\varepsilon_i}{Γ ⊢ (\text{bet } e_1 \ e_2 \ e_3) : τ \ !(\text{Prob} ∪ \varepsilon_1 ∪ \varepsilon_2 ∪ \varepsilon_3)}$$ - -**Pure computations**: -$$\frac{Γ ⊢ e : τ \ !\text{Pure}}{e \text{ is deterministic}}$$ - -### 8.3 Effect Masking - -$$\frac{Γ ⊢ e : τ \ !\varepsilon \quad \text{seed fixed}}{Γ ⊢ (\text{bet-with-seed } s \ e) : τ \ !\text{Pure}}$$ - -Fixing the random seed converts probabilistic to deterministic. - ---- - -## 9. Parametricity - -### 9.1 Free Theorems - -**Theorem 9.1** (Parametricity for bet-map). For bet-map with type: -``` -∀α β. (α → β) → Dist α → Dist β -``` - -For any f: A → B and g: B → C: -``` -bet-map (g ∘ f) ≡ bet-map g ∘ bet-map f -``` - -This is the free theorem derived from the type. - -### 9.2 Naturality - -**Theorem 9.2** (Natural Transformation). bet-pure is a natural transformation: -``` -Id → Dist -``` - -For any f: A → B: -``` -bet-pure ∘ f ≡ bet-map f ∘ bet-pure -``` - -*Proof*: -``` -(bet-pure ∘ f)(x) = bet-pure (f x) -(bet-map f ∘ bet-pure)(x) = bet-map f (bet-pure x) - = bet-bind (bet-pure x) (λ y. bet-pure (f y)) - = bet-pure (f x) ; by left identity -``` -∎ - ---- - -## 10. Dependent Types (Extension) - -### 10.1 Indexed Distributions - -For advanced type safety, we can index by probability: - -``` -Dist_{p} τ ; Distribution with probability p of success -``` - -### 10.2 Dependent Bet - -$$\frac{Γ ⊢ e_1 : τ \quad Γ ⊢ e_2 : τ \quad Γ ⊢ e_3 : τ}{Γ ⊢ (\text{bet } e_1 \ e_2 \ e_3) : \text{Dist}_{1/3} τ × \text{Dist}_{1/3} τ × \text{Dist}_{1/3} τ}$$ - -**TODO**: Full dependent type system requires more infrastructure: -- Π types for probability-indexed functions -- Σ types for existential probability bounds -- Refinement types for probabilistic assertions - ---- - -## 11. Probability Algebra - -### 11.1 Convex Combination - -**Definition 11.1** (Convex Space). Dist(A) forms a convex space: - -For p ∈ [0,1] and distributions μ, ν: -$$p \cdot μ + (1-p) \cdot ν$$ - -is a valid distribution. - -**Theorem 11.1** (Bet as Convex Combination). -``` -(bet A B C) = (1/3)·δ_A + (1/3)·δ_B + (1/3)·δ_C -``` - -### 11.2 Barycentric Algebra - -The set of distributions satisfies: -1. **Idempotence**: p·μ + (1-p)·μ = μ -2. **Skew-commutativity**: p·μ + (1-p)·ν = (1-p)·ν + p·μ -3. **Skew-associativity**: Nesting of mixtures associates properly - ---- - -## 12. Categorical Semantics Summary - -| Structure | Betlang Instantiation | -|-----------|----------------------| -| Category | Kleisli(Dist) | -| Functor | bet-map | -| Natural Transformation | bet-pure | -| Monad | (Dist, bet-pure, bet-bind) | -| Applicative | (Dist, bet-pure, bet-ap) | -| Convex Space | probability mixtures | - ---- - -## References - -1. Wadler, P. (1995). "Monads for Functional Programming" -2. Ramsey, N. & Pfeffer, A. (2002). "Stochastic Lambda Calculus and Monads of Probability Distributions" -3. Moggi, E. (1991). "Notions of Computation and Monads" -4. Heunen, C., Kammar, O., et al. (2017). "A Convenient Category for Higher-Order Probability Theory" -5. Pierce, B.C. (2002). "Types and Programming Languages" diff --git a/proofs/verification/formal-verification.md b/proofs/verification/formal-verification.adoc similarity index 65% rename from proofs/verification/formal-verification.md rename to proofs/verification/formal-verification.adoc index 1268a0a..eff4da2 100644 --- a/proofs/verification/formal-verification.md +++ b/proofs/verification/formal-verification.adoc @@ -1,22 +1,22 @@ - -# Formal Verification Specifications for Betlang +== Formal Verification Specifications for Betlang -## Abstract +=== Abstract -This document provides formal specifications for verifying betlang programs using established verification techniques: Hoare logic, refinement types, separation logic, and probabilistic model checking. We define invariants, pre/postconditions, and correctness criteria suitable for automated and interactive theorem provers. +This document provides formal specifications for verifying betlang +programs using established verification techniques: Hoare logic, +refinement types, separation logic, and probabilistic model checking. We +define invariants, pre/postconditions, and correctness criteria suitable +for automated and interactive theorem provers. ---- +''''' -## 1. Verification Framework +=== 1. Verification Framework -### 1.1 Specification Language +==== 1.1 Specification Language We use a probabilistic specification language extending Hoare logic: -``` +.... Spec ::= {P} e {Q} ; Total correctness | {P} e {Q}_p ; Probabilistic guarantee | {P} e {Q}_≥p ; Lower bound @@ -24,11 +24,11 @@ Spec ::= {P} e {Q} ; Total correctness | ⟨P⟩ e ⟨Q⟩ ; Partial correctness | e ↓ ; Termination | e ↓_p ; Probabilistic termination -``` +.... -### 1.2 Assertion Language +==== 1.2 Assertion Language -``` +.... P, Q ::= true | false | e₁ = e₂ | e₁ < e₂ | e₁ ≤ e₂ | P ∧ Q | P ∨ Q | ¬P | P ⟹ Q @@ -38,16 +38,17 @@ P, Q ::= true | false | Var(X) ≤ σ² ; Variance bound | H(X) ≤ h ; Entropy bound | X ~ D ; Distribution assertion -``` +.... ---- +''''' -## 2. Core Primitive Specifications +=== 2. Core Primitive Specifications -### 2.1 Uniform Bet +==== 2.1 Uniform Bet -**Specification**: -``` +*Specification*: + +.... {true} (bet A B C) {result ∈ {A, B, C}}₁ @@ -63,17 +64,19 @@ P, Q ::= true | false {true} (bet A B C) {result = C}₁/₃ -``` +.... + +*Distribution Specification*: -**Distribution Specification**: -``` +.... {true} let X = (bet A B C) {X ~ Uniform({A, B, C})} -``` +.... + +*Expected Value Specification* (for numeric A, B, C): -**Expected Value Specification** (for numeric A, B, C): -``` +.... {A, B, C ∈ ℝ} let X = (bet A B C) {E[X] = (A + B + C)/3} @@ -81,10 +84,11 @@ P, Q ::= true | false {A, B, C ∈ ℝ} let X = (bet A B C) {Var(X) = (A² + B² + C²)/3 - ((A + B + C)/3)²} -``` +.... -**Entropy Specification**: -``` +*Entropy Specification*: + +.... {A ≠ B ∨ B ≠ C} let X = (bet A B C) {H(X) = log₂(3)} @@ -92,28 +96,31 @@ P, Q ::= true | false {A = B = C} let X = (bet A B C) {H(X) = 0} -``` +.... + +==== 2.2 Weighted Bet -### 2.2 Weighted Bet +*Specification*: -**Specification**: -``` +.... {wₐ ≥ 0 ∧ w_b ≥ 0 ∧ w_c ≥ 0 ∧ wₐ + w_b + w_c > 0} (bet/weighted '(A wₐ) '(B w_b) '(C w_c)) {result = A}_{wₐ/(wₐ+w_b+w_c)} -``` +.... + +*Normalization Invariant*: -**Normalization Invariant**: -``` +.... {W = wₐ + w_b + w_c > 0} let X = (bet/weighted ...) {P(X = A) + P(X = B) + P(X = C) = 1} -``` +.... -### 2.3 Conditional Bet +==== 2.3 Conditional Bet -**Specification**: -``` +*Specification*: + +.... {true} (bet/conditional pred A B C) {pred ⟹ result = A}₁ @@ -121,12 +128,13 @@ P, Q ::= true | false {¬pred} (bet/conditional pred A B C) {result ∈ {B, C, A}}₁ -``` +.... + +==== 2.4 Lazy Bet -### 2.4 Lazy Bet +*Specification*: -**Specification**: -``` +.... {true} (bet/lazy thunk_A thunk_B thunk_C) {result = (thunk_A) ∨ result = (thunk_B) ∨ result = (thunk_C)}₁ @@ -135,40 +143,44 @@ P, Q ::= true | false {true} (bet/lazy thunk_A thunk_B thunk_C) {exactly_one_evaluated(thunk_A, thunk_B, thunk_C)}₁ -``` +.... + +''''' ---- +=== 3. Iteration Specifications -## 3. Iteration Specifications +==== 3.1 bet-chain -### 3.1 bet-chain +*Specification*: -**Specification**: -``` +.... {n ≥ 0} (bet-chain n f init) {result = fⁿ(init)} -``` +.... where fⁿ denotes n-fold application of f. -**Loop Invariant**: -``` +*Loop Invariant*: + +.... ;; Invariant: after k iterations, state = f^k(init) {n ≥ 0 ∧ ∀k ∈ [0,n]. f^k(init) is defined} (bet-chain n f init) {result = fⁿ(init)} -``` +.... + +*Termination*: -**Termination**: -``` +.... (bet-chain n f init) ↓ ⟺ n ∈ ℕ ∧ f terminates on all intermediate values -``` +.... -### 3.2 bet-until +==== 3.2 bet-until -**Specification**: -``` +*Specification*: + +.... {P(pred(thunk())) = p > 0} (bet-until pred thunk) {pred(result)}₁ @@ -176,39 +188,43 @@ where fⁿ denotes n-fold application of f. {P(pred(thunk())) = p > 0} (bet-until pred thunk) ↓₁ ;; Almost sure termination -``` +.... + +*Expected Iterations*: -**Expected Iterations**: -``` +.... {P(pred(thunk())) = p} let N = iterations in (bet-until pred thunk) {E[N] = 1/p} -``` +.... + +==== 3.3 bet-repeat -### 3.3 bet-repeat +*Specification*: -**Specification**: -``` +.... {n ≥ 0} (bet-repeat n thunk) {|result| = n ∧ ∀i. result[i] = some evaluation of thunk} -``` +.... -**Independence**: -``` +*Independence*: + +.... {n ≥ 0} (bet-repeat n thunk) {result[i] ⊥ result[j] for i ≠ j} ;; Independent samples -``` +.... + +''''' ---- +=== 4. Probability Estimation Specifications -## 4. Probability Estimation Specifications +==== 4.1 bet-probability -### 4.1 bet-probability +*Specification*: -**Specification**: -``` +.... {n > 0 ∧ p = P(pred(bet A B C))} let p̂ = (bet-probability n pred A B C) {E[p̂] = p} ;; Unbiased @@ -220,12 +236,13 @@ where fⁿ denotes n-fold application of f. {n > 0 ∧ ε > 0} let p̂ = (bet-probability n pred A B C) {|p̂ - p| < ε}_{1 - 2exp(-2nε²)} ;; Hoeffding bound -``` +.... + +==== 4.2 bet-expect -### 4.2 bet-expect +*Specification*: -**Specification**: -``` +.... {n > 0 ∧ μ = E[f(bet A B C)]} let μ̂ = (bet-expect n f A B C) {E[μ̂] = μ} ;; Unbiased @@ -233,12 +250,13 @@ where fⁿ denotes n-fold application of f. {n > 0} let μ̂ = (bet-expect n f A B C) {μ̂ → μ as n → ∞} ;; Consistency -``` +.... -### 4.3 bet-entropy +==== 4.3 bet-entropy -**Specification**: -``` +*Specification*: + +.... {n > 0 ∧ samples from distribution with entropy H} let Ĥ = (bet-entropy samples) {E[Ĥ] ≈ H - (k-1)/(2n ln 2)} ;; Bias formula @@ -246,16 +264,17 @@ where fⁿ denotes n-fold application of f. {n → ∞} let Ĥ = (bet-entropy samples) {Ĥ → H} ;; Consistency -``` +.... + +''''' ---- +=== 5. Statistical Function Specifications -## 5. Statistical Function Specifications +==== 5.1 Descriptive Statistics -### 5.1 Descriptive Statistics +*Mean*: -**Mean**: -``` +.... {samples ≠ ∅} let μ̂ = (mean samples) {μ̂ = Σᵢ samples[i] / |samples|} @@ -263,10 +282,11 @@ where fⁿ denotes n-fold application of f. {samples are i.i.d. with mean μ} let μ̂ = (mean samples) {E[μ̂] = μ ∧ μ̂ → μ a.s.} -``` +.... + +*Variance*: -**Variance**: -``` +.... {|samples| > 1} let σ̂² = (variance samples) {σ̂² = Σᵢ (samples[i] - mean)² / (|samples| - 1)} ;; Bessel correction @@ -274,20 +294,22 @@ where fⁿ denotes n-fold application of f. {samples are i.i.d. with variance σ²} let σ̂² = (variance samples) {E[σ̂²] = σ²} ;; Unbiased -``` +.... -**Median**: -``` +*Median*: + +.... {samples ≠ ∅} let m = (median samples) {|{x ∈ samples : x ≤ m}| ≥ |samples|/2 ∧ |{x ∈ samples : x ≥ m}| ≥ |samples|/2} -``` +.... + +==== 5.2 Statistical Tests -### 5.2 Statistical Tests +*Chi-Square Test*: -**Chi-Square Test**: -``` +.... {observed, expected have same length ∧ expected[i] > 0 ∀i} let (χ², p) = (chi-square-test observed expected) {χ² = Σᵢ (observed[i] - expected[i])² / expected[i]} @@ -295,23 +317,25 @@ where fⁿ denotes n-fold application of f. {H₀: observed ~ expected} let (χ², p) = (chi-square-test observed expected) {p = P(χ²_{k-1} > χ² | H₀)} -``` +.... -**Kolmogorov-Smirnov Test**: -``` +*Kolmogorov-Smirnov Test*: + +.... {samples₁, samples₂ ≠ ∅} let D = (kolmogorov-smirnov samples₁ samples₂) {D = sup_x |F₁(x) - F₂(x)|} ;; Two-sample KS statistic -``` +.... + +''''' ---- +=== 6. Distribution Specifications -## 6. Distribution Specifications +==== 6.1 Continuous Distributions -### 6.1 Continuous Distributions +*Normal Distribution*: -**Normal Distribution**: -``` +.... {σ > 0} let X = (normal μ σ) {X ~ N(μ, σ²)} @@ -319,82 +343,91 @@ where fⁿ denotes n-fold application of f. {true} let X = (normal μ σ) {E[X] = μ ∧ Var(X) = σ²} -``` +.... + +*Exponential Distribution*: -**Exponential Distribution**: -``` +.... {λ > 0} let X = (exponential λ) {X ~ Exp(λ) ∧ E[X] = 1/λ ∧ Var(X) = 1/λ²} -``` +.... -**Gamma Distribution**: -``` +*Gamma Distribution*: + +.... {α > 0 ∧ β > 0} let X = (gamma α β) {X ~ Gamma(α, β) ∧ E[X] = αβ ∧ Var(X) = αβ²} -``` +.... + +==== 6.2 Discrete Distributions -### 6.2 Discrete Distributions +*Binomial Distribution*: -**Binomial Distribution**: -``` +.... {n ∈ ℕ ∧ 0 ≤ p ≤ 1} let X = (binomial n p) {X ~ Binomial(n, p) ∧ E[X] = np ∧ Var(X) = np(1-p)} -``` +.... + +*Poisson Distribution*: -**Poisson Distribution**: -``` +.... {λ > 0} let X = (poisson λ) {X ~ Poisson(λ) ∧ E[X] = λ ∧ Var(X) = λ} -``` +.... ---- +''''' -## 7. MCMC Specifications +=== 7. MCMC Specifications -### 7.1 Metropolis-Hastings +==== 7.1 Metropolis-Hastings -**Detailed Balance**: -``` +*Detailed Balance*: + +.... {π is target distribution, q is proposal} (metropolis-hastings π q x₀ n) {∀x,y. π(x)P(x→y) = π(y)P(y→x)} ;; Detailed balance -``` +.... + +*Convergence*: -**Convergence**: -``` +.... {chain is irreducible and aperiodic} let samples = (metropolis-hastings π q x₀ n) {empirical_distribution(samples) → π as n → ∞} -``` +.... + +*Acceptance Rate*: -**Acceptance Rate**: -``` +.... {true} let (samples, accepts) = (metropolis-hastings π q x₀ n) {E[accepts/n] depends on proposal tuning} -``` +.... -### 7.2 Gibbs Sampling +==== 7.2 Gibbs Sampling -**Full Conditional Correctness**: -``` +*Full Conditional Correctness*: + +.... {p(x|y) and p(y|x) are correct full conditionals of joint p(x,y)} (gibbs-sampler cond-x cond-y x₀ y₀ n) {samples → p(x,y) as n → ∞} -``` +.... + +''''' ---- +=== 8. Bayesian Inference Specifications -## 8. Bayesian Inference Specifications +==== 8.1 Bayes Theorem -### 8.1 Bayes Theorem +*Correctness*: -**Correctness**: -``` +.... {prior_prob > 0 ∧ evidence_prob > 0} let posterior = (bayes-theorem prior_prob likelihood evidence_prob) {posterior = (prior_prob × likelihood) / evidence_prob} @@ -402,134 +435,141 @@ where fⁿ denotes n-fold application of f. {prior_prob × likelihood / evidence_prob ∈ [0,1]} (bayes-theorem prior_prob likelihood evidence_prob) {result ∈ [0, 1]} ;; Valid probability -``` +.... -### 8.2 Conjugate Priors +==== 8.2 Conjugate Priors -**Beta-Binomial**: -``` +*Beta-Binomial*: + +.... {α > 0 ∧ β > 0 ∧ k ≥ 0 ∧ n ≥ 0} let posterior = (conjugate-beta-binomial α β k (n-k)) {posterior ~ Beta(α + k, β + n - k)} -``` +.... + +*Normal-Normal*: -**Normal-Normal**: -``` +.... {σ₀² > 0 ∧ σ² > 0} let posterior = (conjugate-normal μ₀ σ₀² σ² observations) {posterior ~ N(μ_n, σ_n²) where μ_n, σ_n² are posterior parameters} -``` +.... -### 8.3 Credible Intervals +==== 8.3 Credible Intervals -``` +.... {0 < α < 1} let (lo, hi) = (credible-interval samples α) {P(lo ≤ θ ≤ hi | data) = 1 - α} -``` +.... + +''''' ---- +=== 9. Markov Chain Specifications -## 9. Markov Chain Specifications +==== 9.1 Transition Matrix -### 9.1 Transition Matrix +*Stochasticity*: -**Stochasticity**: -``` +.... {transitions is a valid transition specification} let chain = (make-markov-chain states transitions initial) {∀s. Σ_s' P(s→s') = 1} ;; Rows sum to 1 -``` +.... -### 9.2 Simulation +==== 9.2 Simulation -**Markov Property**: -``` +*Markov Property*: + +.... {chain is a valid Markov chain} let path = (markov-simulate chain n) {P(path[k+1] | path[0..k]) = P(path[k+1] | path[k])} -``` +.... + +*Stationary Distribution*: -**Stationary Distribution**: -``` +.... {chain is irreducible and aperiodic} let π̂ = (markov-stationary chain n) {π̂ → π as n → ∞ where π is the unique stationary distribution} -``` +.... ---- +''''' -## 10. Refinement Types +=== 10. Refinement Types -### 10.1 Probability-Indexed Types +==== 10.1 Probability-Indexed Types -``` +.... Prob_p T ;; Type T with associated probability p -``` +.... -**Subtyping**: -``` +*Subtyping*: + +.... p ≤ q ⟹ Prob_p T <: Prob_q T -``` +.... -### 10.2 Refined Bet Types +==== 10.2 Refined Bet Types -``` +.... bet : ∀A. A → A → A → Prob_{1/3} A bet/weighted : ∀A. (A, w₁:ℕ) → (A, w₂:ℕ) → (A, w₃:ℕ) → Prob_{w₁/(w₁+w₂+w₃)} A -``` +.... -### 10.3 Dependent Probability Types +==== 10.3 Dependent Probability Types -``` +.... ;; Type depending on probability type BetResult[p : Prob] = { x : Value | P(selected) = p } bet : A → B → C → Σ(i : Fin 3). BetResult[1/3] -``` +.... ---- +''''' -## 11. Separation Logic for Probabilistic State +=== 11. Separation Logic for Probabilistic State -### 11.1 Probabilistic Heap Assertions +==== 11.1 Probabilistic Heap Assertions -``` +.... P ::= ... | x ↦ v ;; x points to value v | x ↦_p v ;; x points to v with probability p | P * Q ;; Separating conjunction | P -* Q ;; Magic wand -``` +.... -### 11.2 Frame Rule +==== 11.2 Frame Rule -``` +.... {P} e {Q} ───────────────── {P * R} e {Q * R} -``` +.... The frame rule preserves separate resources. -### 11.3 Random State Assertions +==== 11.3 Random State Assertions -``` +.... {rng_state = s} (bet A B C) {rng_state = next(s) ∧ result = select(s, A, B, C)} -``` +.... + +''''' ---- +=== 12. Verification Conditions -## 12. Verification Conditions +==== 12.1 Weakest Precondition -### 12.1 Weakest Precondition +*Definition*: -**Definition**: -``` +.... wp[bet A B C](Q) = (1/3)Q[A] + (1/3)Q[B] + (1/3)Q[C] wp[bet/weighted (A,wₐ) (B,w_b) (C,w_c)](Q) = @@ -539,97 +579,102 @@ wp[if b then e₁ else e₂](Q) = (b ⟹ wp[e₁](Q)) ∧ (¬b ⟹ wp[e₂](Q)) wp[let x = e₁ in e₂](Q) = wp[e₁](λv. wp[e₂[x↦v]](Q)) -``` +.... -### 12.2 Verification Condition Generation +==== 12.2 Verification Condition Generation For probabilistic programs: -``` +.... VC({P} e {Q}_p) = P ⟹ wp[e](Q) ≥ p -``` +.... -### 12.3 Automated Verification +==== 12.3 Automated Verification -Decidable fragments: -- Linear arithmetic assertions -- Polynomial probability expressions -- Finite-state distributions +Decidable fragments: - Linear arithmetic assertions - Polynomial +probability expressions - Finite-state distributions ---- +''''' -## 13. Model Checking Specifications +=== 13. Model Checking Specifications -### 13.1 PCTL Properties +==== 13.1 PCTL Properties Probabilistic Computation Tree Logic for betlang: -``` +.... φ ::= true | a | ¬φ | φ ∧ ψ | P_{∼p}[ψ] ;; Probability operator | X φ ;; Next | φ U ψ ;; Until | F φ ;; Eventually | G φ ;; Always -``` +.... + +==== 13.2 Example Properties -### 13.2 Example Properties +*Termination*: -**Termination**: -``` +.... P_{=1}[F terminated] ;; Almost sure termination -``` +.... -**Reachability**: -``` +*Reachability*: + +.... P_{≥0.9}[F goal] ;; Reach goal with prob ≥ 0.9 -``` +.... + +*Safety*: -**Safety**: -``` +.... P_{=1}[G ¬error] ;; Never reach error state -``` +.... -### 13.3 Continuous Stochastic Logic +==== 13.3 Continuous Stochastic Logic For continuous-time extensions: -``` +.... φ ::= ... | P_{∼p}[φ U^{≤t} ψ] ;; Time-bounded until -``` +.... ---- +''''' -## 14. Proof Obligations +=== 14. Proof Obligations -### 14.1 For Each Core Primitive +==== 14.1 For Each Core Primitive -1. **bet**: Uniformity, independence, termination -2. **bet/weighted**: Normalization, non-negativity, termination -3. **bet/conditional**: Determinism under true predicate -4. **bet/lazy**: Lazy evaluation correctness +[arabic] +. *bet*: Uniformity, independence, termination +. *bet/weighted*: Normalization, non-negativity, termination +. *bet/conditional*: Determinism under true predicate +. *bet/lazy*: Lazy evaluation correctness -### 14.2 For Statistical Functions +==== 14.2 For Statistical Functions -1. **Unbiasedness**: E[estimator] = true value -2. **Consistency**: Convergence to true value -3. **Efficiency**: Variance bounds (Cramér-Rao) -4. **Robustness**: Behavior under model misspecification +[arabic] +. *Unbiasedness*: E[estimator] = true value +. *Consistency*: Convergence to true value +. *Efficiency*: Variance bounds (Cramér-Rao) +. *Robustness*: Behavior under model misspecification -### 14.3 For MCMC Methods +==== 14.3 For MCMC Methods -1. **Detailed balance**: Reversibility -2. **Irreducibility**: All states reachable -3. **Aperiodicity**: No cycling -4. **Geometric ergodicity**: Convergence rate bounds +[arabic] +. *Detailed balance*: Reversibility +. *Irreducibility*: All states reachable +. *Aperiodicity*: No cycling +. *Geometric ergodicity*: Convergence rate bounds ---- +''''' -## 15. Mechanized Proofs +=== 15. Mechanized Proofs -### 15.1 Coq Encoding (Sketch) +==== 15.1 Coq Encoding (Sketch) -```coq +[source,coq] +---- Inductive BetExpr (A : Type) : Type := | Bet : A -> A -> A -> BetExpr A | BetWeighted : (A * nat) -> (A * nat) -> (A * nat) -> BetExpr A. @@ -650,37 +695,43 @@ Theorem bet_uniform : forall A (a b c : A), bet_probability (Bet a b c) b = 1/3 /\ bet_probability (Bet a b c) c = 1/3. Proof. ... Qed. -``` +---- -### 15.2 Lean Encoding (Sketch) +==== 15.2 Lean Encoding (Sketch) -```lean +[source,lean] +---- def bet_dist (a b c : α) : Pmf α := Pmf.ofFinset {a, b, c} (by simp) theorem bet_uniform (a b c : α) (h : a ≠ b ∧ b ≠ c ∧ a ≠ c) : (bet_dist a b c) a = 1/3 ∧ (bet_dist a b c) b = 1/3 ∧ (bet_dist a b c) c = 1/3 := by ... -``` +---- ---- +''''' -## 16. TODOs: Incomplete Specifications +=== 16. TODOs: Incomplete Specifications -**TODO**: The following require formal specification: +*TODO*: The following require formal specification: -1. **Continuous distribution correctness**: PDF matching for all distributions -2. **Numerical stability**: Floating-point error bounds for all functions -3. **Concurrency**: Specifications for parallel/concurrent bet operations -4. **Streaming**: Specifications for online/streaming statistics -5. **Approximate inference**: Error bounds for ABC, variational methods +[arabic] +. *Continuous distribution correctness*: PDF matching for all +distributions +. *Numerical stability*: Floating-point error bounds for all functions +. *Concurrency*: Specifications for parallel/concurrent bet operations +. *Streaming*: Specifications for online/streaming statistics +. *Approximate inference*: Error bounds for ABC, variational methods ---- +''''' -## References +=== References -1. Morgan, C., McIver, A., & Seidel, K. (1996). "Probabilistic Predicate Transformers" -2. Barthe, G., et al. (2012). "Probabilistic Relational Reasoning" -3. Batz, K., et al. (2021). "Foundations of Probabilistic Programming" -4. Hurd, J. (2003). "Formal Verification of Probabilistic Algorithms" -5. Kwiatkowska, M., Norman, G., & Parker, D. (2011). "PRISM 4.0: Verification of Probabilistic Real-Time Systems" +[arabic] +. Morgan, C., McIver, A., & Seidel, K. (1996). "`Probabilistic Predicate +Transformers`" +. Barthe, G., et al. (2012). "`Probabilistic Relational Reasoning`" +. Batz, K., et al. (2021). "`Foundations of Probabilistic Programming`" +. Hurd, J. (2003). "`Formal Verification of Probabilistic Algorithms`" +. Kwiatkowska, M., Norman, G., & Parker, D. (2011). "`PRISM 4.0: +Verification of Probabilistic Real-Time Systems`" diff --git a/site/index.adoc b/site/index.adoc new file mode 100644 index 0000000..54994a7 --- /dev/null +++ b/site/index.adoc @@ -0,0 +1,26 @@ +== betlang + +The public web home for this project is +https://betlang.org[betlang.org]. + +A ternary probabilistic programming language for modeling uncertainty + +betlang is a Domain-Specific Language (DSL) for probabilistic +programming built on Racket. Its core primitive is the ternary bet - a +choice between three values with associated probabilities. This simple +yet powerful concept enables elegant expression of complex probabilistic +models. + +=== Project Links + +* Website: https://betlang.org[betlang.org] +* Source: https://github.com/hyperpolymath/betlang +* README: +https://github.com/hyperpolymath/betlang/blob/main/README.adoc[project +overview] +* Docs: +https://github.com/hyperpolymath/betlang/tree/main/docs[documentation +directory] + +This page is a lightweight landing point for the repository and will +grow with the project. diff --git a/site/index.md b/site/index.md deleted file mode 100644 index b86c766..0000000 --- a/site/index.md +++ /dev/null @@ -1,25 +0,0 @@ - ---- -title: betlang -date: 2026-03-31 ---- - -# betlang - -The public web home for this project is [betlang.org](https://betlang.org). - -A ternary probabilistic programming language for modeling uncertainty - -betlang is a Domain-Specific Language (DSL) for probabilistic programming built on Racket. Its core primitive is the ternary bet - a choice between three values with associated probabilities. This simple yet powerful concept enables elegant expression of complex probabilistic models. - -## Project Links - -- Website: [betlang.org](https://betlang.org) -- Source: [https://github.com/hyperpolymath/betlang](https://github.com/hyperpolymath/betlang) -- README: [project overview](https://github.com/hyperpolymath/betlang/blob/main/README.adoc) -- Docs: [documentation directory](https://github.com/hyperpolymath/betlang/tree/main/docs) - -This page is a lightweight landing point for the repository and will grow with the project. diff --git a/spec/system-specs.adoc b/spec/system-specs.adoc new file mode 100644 index 0000000..685a686 --- /dev/null +++ b/spec/system-specs.adoc @@ -0,0 +1,168 @@ +== SPDX-License-Identifier: CC-BY-SA-4.0 + +== Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) j.d.a.jewell@open.ac.uk + +== BetLang System Specifications + +BetLang is a probabilistic programming language designed for Bayesian +inference and stochastic modelling. Implementation stack: Rust compiler +(LALRPOP parser), Racket runtime for symbolic computation and inference. + +''''' + +=== Memory Model + +BetLang’s memory model spans two runtime layers: the Rust compiler and +the Racket execution environment. + +==== Rust Compiler Layer + +* The LALRPOP-generated parser produces an owned AST (`+Box+`, +`+Vec+`). +* All compiler passes (desugaring, type inference, IR generation) +operate on owned data with standard Rust move semantics. +* No reference counting or garbage collection in the compiler pipeline. +* Intermediate representations are allocated on the heap and freed when +each pass completes. + +==== Racket Runtime Layer + +* Racket’s precise generational garbage collector manages all runtime +values. +* Probabilistic samples are boxed Racket values subject to GC. +* Distribution objects (Normal, Beta, Bernoulli, etc.) are Racket +structs allocated on the managed heap. +* Trace storage for inference algorithms (particle histories, MCMC +chains) uses Racket vectors, collected when inference completes. + +==== Probability Monad Storage + +* Each probabilistic program executes within an inference monad that +maintains: +** *Log-weight accumulator*: running log-probability of the current +trace. +** *Sample store*: mapping from sample site names to drawn values. +** *Observation log*: list of observed values with their likelihoods. +* The sample store is ephemeral per inference iteration and GC-eligible +after. + +''''' + +=== Concurrency Model + +BetLang exploits the statistical independence of probabilistic samples +to enable safe parallelism. + +==== Probabilistic Parallelism + +* *Particle-level parallelism*: In Sequential Monte Carlo (SMC) +inference, each particle runs an independent copy of the program. +Particles share no mutable state and can execute on separate Racket +places. +* *Chain-level parallelism*: Multiple MCMC chains run independently. +Each chain maintains its own sample store and log-weight. + +==== Racket Places + +* Racket’s `+place+` construct provides OS-thread-level parallelism with +message-passing communication (no shared memory). +* Each inference particle or chain runs in its own place. +* Results are collected via place channels after inference completes. + +==== Synchronisation Points + +* *Resampling barriers*: In SMC, particles synchronise at resampling +steps. Weights are collected from all places, resampling occurs +centrally, and new particle assignments are distributed. +* No locks or shared mutable state — all coordination is message-based. + +''''' + +=== Effect System + +BetLang’s effect system tracks probabilistic operations as first-class +effects within an inference monad. + +==== Probabilistic Effects + +[width="100%",cols="20%,17%,63%",options="header",] +|=== +|Effect |Operation |Description +|`+Sample+` |`+sample d+` |Draw a value from distribution `+d+` + +|`+Observe+` |`+observe d v+` |Condition on value `+v+` under +distribution `+d+` + +|`+Condition+` |`+condition b+` |Hard constraint — reject trace if `+b+` +is false + +|`+Score+` |`+score w+` |Manually adjust trace log-weight by `+w+` +|=== + +==== Inference Monad + +* All probabilistic effects are interpreted by an inference backend. +* The monad signature: +`+Infer a = TraceState -> (a, TraceState, LogWeight)+`. +* Different inference algorithms provide different effect handlers: +** *Importance sampling*: `+sample+` draws from prior, `+observe+` +updates weight. +** *MH (Metropolis-Hastings)*: `+sample+` proposes from kernel, +accept/reject. +** *SMC*: `+sample+` draws from prior, `+observe+` triggers resampling. + +==== Deterministic Subset + +* Programs using no probabilistic effects are pure and execute +deterministically. +* The type system distinguishes `+Pure a+` from `+Prob a+` at the top +level. +* Pure functions can be called from probabilistic contexts but not vice +versa. + +==== Effect Composition + +* Probabilistic effects compose with standard effects (IO, State) via +monad transformers in the Racket runtime. +* IO effects are restricted to the outermost layer — inference internals +are pure with respect to IO. + +''''' + +=== Module System + +BetLang uses a Racket-style module system for code organisation. + +==== Module Declaration + +* Each file is a module: `+#lang betlang+` at the top. +* Exports are explicit: `+(provide func1 func2 DistributionType)+`. +* Imports use `+(require "path/to/module.bet")+` or +`+(require betlang/stdlib)+`. + +==== Standard Library Modules + +[width="100%",cols="33%,67%",options="header",] +|=== +|Module |Contents +|`+betlang/distributions+` |Normal, Beta, Bernoulli, Poisson, etc. +|`+betlang/inference+` |SMC, MH, importance sampling, enumeration +|`+betlang/combinators+` |Probabilistic combinators (mixture, product) +|`+betlang/plotting+` |Posterior visualisation utilities +|`+betlang/data+` |Data loading and observation helpers +|=== + +==== Racket Interoperability + +* BetLang modules can import standard Racket libraries via +`+(require racket/*)+`. +* Racket modules can consume BetLang exports as standard Racket values. +* Distribution objects implement Racket’s `+gen:custom-write+` for REPL +display. + +==== Compiler Integration + +* The Rust compiler (`+betlang-compiler+` crate) parses `+.bet+` files +and emits Racket source (`+.rkt+`) for execution. +* Module resolution happens at the Racket level after compilation. +* Circular imports are detected and rejected by Racket’s module system. diff --git a/spec/system-specs.md b/spec/system-specs.md deleted file mode 100644 index 63eece1..0000000 --- a/spec/system-specs.md +++ /dev/null @@ -1,147 +0,0 @@ - -# SPDX-License-Identifier: CC-BY-SA-4.0 -# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) - -# BetLang System Specifications - -BetLang is a probabilistic programming language designed for Bayesian inference -and stochastic modelling. Implementation stack: Rust compiler (LALRPOP parser), -Racket runtime for symbolic computation and inference. - ---- - -## Memory Model - -BetLang's memory model spans two runtime layers: the Rust compiler and the -Racket execution environment. - -### Rust Compiler Layer - -- The LALRPOP-generated parser produces an owned AST (`Box`, `Vec`). -- All compiler passes (desugaring, type inference, IR generation) operate on - owned data with standard Rust move semantics. -- No reference counting or garbage collection in the compiler pipeline. -- Intermediate representations are allocated on the heap and freed when each - pass completes. - -### Racket Runtime Layer - -- Racket's precise generational garbage collector manages all runtime values. -- Probabilistic samples are boxed Racket values subject to GC. -- Distribution objects (Normal, Beta, Bernoulli, etc.) are Racket structs - allocated on the managed heap. -- Trace storage for inference algorithms (particle histories, MCMC chains) - uses Racket vectors, collected when inference completes. - -### Probability Monad Storage - -- Each probabilistic program executes within an inference monad that maintains: - - **Log-weight accumulator**: running log-probability of the current trace. - - **Sample store**: mapping from sample site names to drawn values. - - **Observation log**: list of observed values with their likelihoods. -- The sample store is ephemeral per inference iteration and GC-eligible after. - ---- - -## Concurrency Model - -BetLang exploits the statistical independence of probabilistic samples to -enable safe parallelism. - -### Probabilistic Parallelism - -- **Particle-level parallelism**: In Sequential Monte Carlo (SMC) inference, - each particle runs an independent copy of the program. Particles share no - mutable state and can execute on separate Racket places. -- **Chain-level parallelism**: Multiple MCMC chains run independently. Each - chain maintains its own sample store and log-weight. - -### Racket Places - -- Racket's `place` construct provides OS-thread-level parallelism with - message-passing communication (no shared memory). -- Each inference particle or chain runs in its own place. -- Results are collected via place channels after inference completes. - -### Synchronisation Points - -- **Resampling barriers**: In SMC, particles synchronise at resampling steps. - Weights are collected from all places, resampling occurs centrally, and - new particle assignments are distributed. -- No locks or shared mutable state — all coordination is message-based. - ---- - -## Effect System - -BetLang's effect system tracks probabilistic operations as first-class effects -within an inference monad. - -### Probabilistic Effects - -| Effect | Operation | Description | -|---------------|-------------|------------------------------------------------| -| `Sample` | `sample d` | Draw a value from distribution `d` | -| `Observe` | `observe d v` | Condition on value `v` under distribution `d`| -| `Condition` | `condition b` | Hard constraint — reject trace if `b` is false| -| `Score` | `score w` | Manually adjust trace log-weight by `w` | - -### Inference Monad - -- All probabilistic effects are interpreted by an inference backend. -- The monad signature: `Infer a = TraceState -> (a, TraceState, LogWeight)`. -- Different inference algorithms provide different effect handlers: - - **Importance sampling**: `sample` draws from prior, `observe` updates weight. - - **MH (Metropolis-Hastings)**: `sample` proposes from kernel, accept/reject. - - **SMC**: `sample` draws from prior, `observe` triggers resampling. - -### Deterministic Subset - -- Programs using no probabilistic effects are pure and execute deterministically. -- The type system distinguishes `Pure a` from `Prob a` at the top level. -- Pure functions can be called from probabilistic contexts but not vice versa. - -### Effect Composition - -- Probabilistic effects compose with standard effects (IO, State) via monad - transformers in the Racket runtime. -- IO effects are restricted to the outermost layer — inference internals are - pure with respect to IO. - ---- - -## Module System - -BetLang uses a Racket-style module system for code organisation. - -### Module Declaration - -- Each file is a module: `#lang betlang` at the top. -- Exports are explicit: `(provide func1 func2 DistributionType)`. -- Imports use `(require "path/to/module.bet")` or `(require betlang/stdlib)`. - -### Standard Library Modules - -| Module | Contents | -|-----------------------|-----------------------------------------------| -| `betlang/distributions` | Normal, Beta, Bernoulli, Poisson, etc. | -| `betlang/inference` | SMC, MH, importance sampling, enumeration | -| `betlang/combinators` | Probabilistic combinators (mixture, product) | -| `betlang/plotting` | Posterior visualisation utilities | -| `betlang/data` | Data loading and observation helpers | - -### Racket Interoperability - -- BetLang modules can import standard Racket libraries via `(require racket/*)`. -- Racket modules can consume BetLang exports as standard Racket values. -- Distribution objects implement Racket's `gen:custom-write` for REPL display. - -### Compiler Integration - -- The Rust compiler (`betlang-compiler` crate) parses `.bet` files and emits - Racket source (`.rkt`) for execution. -- Module resolution happens at the Racket level after compilation. -- Circular imports are detected and rejected by Racket's module system.