Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
b0b07d1
Add a differential test harness and regression tests
salmans Oct 18, 2026
dcdb880
Fix store_view failure handling, stabilization flag reset, and error …
salmans Oct 18, 2026
68220e6
Add CI workflow and the correctness review log
salmans Oct 18, 2026
fbed841
Remove redundant slice rebindings in the join helpers
salmans Oct 18, 2026
7883acd
Silence the last clippy lint in the instance tests
salmans Oct 18, 2026
6cd4739
Drop the trailing semicolon from the create-relation query macro arm
salmans Oct 18, 2026
18342a7
Fix unused macro imports and run CI with all features
salmans Oct 18, 2026
8286453
Weighted batches: retractions, incremental Difference, and Distinct
salmans Oct 18, 2026
ba5e397
Document weighted storage, retractions, and cheap clones
salmans Oct 18, 2026
b4c9dcb
Benchmark incremental step and clone cost
salmans Oct 18, 2026
3fa9250
Format the benchmark
salmans Oct 19, 2026
b737306
Add CLAUDE.md with commands, architecture, and code shape criteria
salmans Oct 19, 2026
6954c72
Tag view references with the id of the database that stored them
salmans Oct 19, 2026
3d0068d
Make expression closures Fn and drop their shared RefCell state
salmans Oct 19, 2026
cf0c570
Fix stale docs on view validation, weighted changes, and the distinct…
salmans Oct 19, 2026
13254d3
Compute expression dependencies through the shared helpers
salmans Oct 19, 2026
6c20614
Replace the instance set-semantics flag with a Weights enum
salmans Oct 19, 2026
9a181ae
Benchmark the join view itself and record the measured costs
salmans Oct 19, 2026
76ae47b
Log deferred findings from the review
salmans Oct 19, 2026
009d40d
Remove DerefMut from Tuples and build batches from sets without re-so…
salmans Oct 19, 2026
cfb569a
Add the improvement list
salmans Oct 19, 2026
c36e1d4
Keep the improvement list out of the repository
salmans Oct 19, 2026
d39e240
Name the CI job build so the branch protection check can report
salmans Oct 19, 2026
5dc15bd
Take the join key as impl Fn in the keyed helper
salmans Oct 19, 2026
7593fc8
Merge branch 'main' into revival
salmans Oct 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: CI

on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy, rustfmt
- name: Format
run: cargo fmt --all -- --check
- name: Clippy
run: cargo clippy --all-targets --all-features -- -D warnings
- name: Test
run: cargo test --all-features
- name: Differential harness (long run)
run: PROPTEST_CASES=20000 cargo test --test differential codd_agrees
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
/target
Cargo.lock
CLAUDE.md
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# codd

[`codd`](https://en.wikipedia.org/wiki/Edgar_F._Codd) is a library for evaluating *typed* relational expressions in a monotonically growing minimal database in memory. `codd` is primarily developed to support an implementation of [`razor`](https://github.com/salmans/rusty-razor) based on relational algebra, however, its design is consistent with common concepts of database theory and may be used as a minimal general purpose database.
[`codd`](https://en.wikipedia.org/wiki/Edgar_F._Codd) is a library for evaluating *typed* relational expressions in a minimal in-memory database with incrementally maintained views. `codd` is primarily developed to support an implementation of [`razor`](https://github.com/salmans/rusty-razor) based on relational algebra, however, its design is consistent with common concepts of database theory and may be used as a minimal general purpose database.

The implementation of database instances in `codd` is borrowed from [`datafrog`](https://github.com/rust-lang/datafrog):
* `Instance<T>` (`Variable<T>` in `datafrog`) contains tuples of type `T`,
* Incremental view maintenance is implemented by maintaining tuples of `Instance<T>` in three sets of `to_add` (candidate tuples to be inserted), `recent` (recently added tuples), and `stable` (old tuples that have been reflected in all views).
Storage and incremental view maintenance follow [`datafrog`](https://github.com/rust-lang/datafrog) and [DBSP](https://github.com/feldera/feldera):
* Every relation and view is an `Instance<T>` holding sorted, immutable batches of *weighted* tuples (a Z-set). Relations have set semantics (weights are kept at zero or one, so `insert` of a present tuple and `retract` of an absent one are no-ops); views keep the number of derivations of each tuple as its weight.
* Changes arrive in `to_add`, are folded in order into one `recent` change, propagated to dependent views as weighted deltas, and then merged into the `stable` batches. Linear operators pass deltas through, bilinear operators (join, product, intersect) use `ΔL⋈R + L⋈ΔR + ΔL⋈ΔR`, and the non-linear ones (`Difference` as an antijoin, `Distinct`) recompute the affected tuples from the current counts of their operands.
* Batches are shared between a database and its clones, so cloning is cheap and clones diverge independently, including by retraction.

In contrast, `codd` distinguishes relation instances from views and offers the trait `Expression<T>` and types that implement `Expression<T>` to query a database.

Expand Down
9 changes: 8 additions & 1 deletion core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,11 @@ thiserror = "^1.0"
either = "^1.6"

[features]
unstable = []
unstable = []
[dev-dependencies]
proptest = "1"
criterion = "0.5"

[[bench]]
name = "incremental"
harness = false
66 changes: 66 additions & 0 deletions core/benches/incremental.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
//! Cost of one incremental step and of cloning, as a function of the stable size.
use codd::{
expression::{Join, Relation, View},
Database,
};
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};

type Row = (u32, u32);
type JoinView = View<Row, Join<u32, Row, Row, Relation<Row>, Relation<Row>, Row>>;

/// A database with a stored join view, the relation the benchmark inserts into, and the view.
struct Fixture {
db: Database,
r: Relation<Row>,
view: JoinView,
}

/// A database with `n` tuples in two relations and a stored join view over them.
fn setup(n: u32) -> Fixture {
let mut db = Database::new();
let r = db.add_relation::<Row>("r").unwrap();
let s = db.add_relation::<Row>("s").unwrap();
let join = Join::new(
r.clone(),
s.clone(),
|t: &Row| t.0,
|t: &Row| t.0,
|_: &u32, l: &Row, r: &Row| (l.1, r.1),
);
let view = db.store_view(join).unwrap();
db.insert(&r, (0..n).map(|i| (i % 1024, i)).into()).unwrap();
db.insert(&s, (0..n).map(|i| (i % 1024, i + n)).into())
.unwrap();
db.evaluate(&view).unwrap();
Fixture { db, r, view }
}

fn incremental_step(c: &mut Criterion) {
let mut group = c.benchmark_group("join_view_insert_one");
for n in [1_000u32, 10_000, 100_000] {
let Fixture { db, r, view } = setup(n);
let mut next = n;
group.bench_with_input(BenchmarkId::from_parameter(n), &n, |b, _| {
b.iter(|| {
next += 1;
db.insert(&r, vec![(next % 1024, next)].into()).unwrap();
db.evaluate(&view).unwrap();
})
});
}
group.finish();
}

fn clone_database(c: &mut Criterion) {
let mut group = c.benchmark_group("clone");
for n in [1_000u32, 10_000, 100_000] {
let Fixture { db, .. } = setup(n);
group.bench_with_input(BenchmarkId::from_parameter(n), &n, |b, _| {
b.iter(|| db.clone())
});
}
group.finish();
}

criterion_group!(benches, incremental_step, clone_database);
criterion_main!(benches);
Loading
Loading