Skip to content

Reduce Git contention and benchmark multi-gateway Git and LFS workloads #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
forhappy merged 12 commits into from
Sep 30, 2026
Merged

Reduce Git contention and benchmark multi-gateway Git and LFS workloads #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
cd0f0d9
Reduce Git fetch contention and bound per-account waiting
forhappy Sep 29, 2026
ed6f270
Receive concurrent push uploads before serial publication
forhappy Sep 29, 2026
c9a553c
Benchmark stock Git transfers and unique-ref pushes across gateways
forhappy Sep 29, 2026
15d88c7
Allocate Git object write coordination only for hydrating caches
forhappy Sep 29, 2026
6dc8e07
Benchmark incremental fetch and pull with versioned corpus
forhappy Sep 29, 2026
0440308
Benchmark bounded LFS transfers across gateways
forhappy Sep 29, 2026
2ab4b2c
Pin Cellule renewal scheduler for dense repository ownership
forhappy Sep 29, 2026
1ee42ed
Record renewal density evidence and add memory diagnostic mode
forhappy Sep 30, 2026
e57d149
Skip empty ref scans in initial mirror publication
forhappy Sep 30, 2026
2b51b14
Make cold-residency fault fixtures establish eviction
forhappy Sep 30, 2026
97e9ddd
Fail large-transfer qualification early on undersized volumes
forhappy Sep 30, 2026
a00d474
Document cold-scan movement budget bound
forhappy Sep 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ axum = "0.8.9"
base64 = "0.22"
blake3 = "1.8"
bytes = "1.11"
cellule-app = { git = "https://github.com/crabbuild/cellule.git", rev = "a28de7bc09ce36d87e642adc4f4b6be50d6fcb69" }
cellule-host = { git = "https://github.com/crabbuild/cellule.git", rev = "a28de7bc09ce36d87e642adc4f4b6be50d6fcb69" }
cellule-ltx = { git = "https://github.com/crabbuild/cellule.git", rev = "a28de7bc09ce36d87e642adc4f4b6be50d6fcb69", features = ["replica"] }
cellule-runtime = { git = "https://github.com/crabbuild/cellule.git", rev = "a28de7bc09ce36d87e642adc4f4b6be50d6fcb69" }
cellule-store = { git = "https://github.com/crabbuild/cellule.git", rev = "a28de7bc09ce36d87e642adc4f4b6be50d6fcb69" }
cellule-app = { git = "https://github.com/crabbuild/cellule.git", rev = "47a302b79962ee16c698e121315cf4e85ec49549" }
cellule-host = { git = "https://github.com/crabbuild/cellule.git", rev = "47a302b79962ee16c698e121315cf4e85ec49549" }
cellule-ltx = { git = "https://github.com/crabbuild/cellule.git", rev = "47a302b79962ee16c698e121315cf4e85ec49549", features = ["replica"] }
cellule-runtime = { git = "https://github.com/crabbuild/cellule.git", rev = "47a302b79962ee16c698e121315cf4e85ec49549" }
cellule-store = { git = "https://github.com/crabbuild/cellule.git", rev = "47a302b79962ee16c698e121315cf4e85ec49549" }
ed25519-dalek = "2"
flate2 = "1.1"
futures-core = "0.3"
Expand Down
273 changes: 250 additions & 23 deletions docs/performance-plan.md
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 10 additions & 2 deletions examples/benchmark_idle.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@ type Result<T = ()> = std::result::Result<T, Error>;

#[derive(Debug, thiserror::Error)]
enum Error {
#[error("usage: benchmark_idle <s3-url> <new-work-dir> <counts-csv> <window-seconds>=10")]
#[error(
"usage: benchmark_idle <s3-url|memory:///> <new-work-dir> <counts-csv> <window-seconds>=10"
)]
Usage,
#[error("benchmark invariant failed: {0}")]
Invalid(&'static str),
Expand Down Expand Up @@ -98,7 +100,7 @@ async fn run() -> Result {
return Err(Error::Usage);
}
let url = url::Url::parse(&args[0])?;
if url.scheme() != "s3"
if !matches!(url.scheme(), "s3" | "memory")
|| !url.username().is_empty()
|| url.password().is_some()
|| url.query().is_some()
Expand Down Expand Up @@ -146,7 +148,13 @@ async fn run() -> Result {
token: format!("cnp_{}", hex::encode(Sha256::digest(key))),
active_limit,
};
let provider_kind = if url.scheme() == "memory" {
"in-memory scheduler diagnostic; not a real-store qualification"
} else {
"S3-compatible provider"
};
let mut report = json!({"passed":false,"cell_capabilities":"SQL only","requested_counts":counts,"window_seconds":seconds,
"provider_kind":provider_kind,
"binary_sha256":hex::encode(Sha256::digest(std::fs::read(std::env::current_exe()?)?)),"store_prefix":fixture.prefix.to_string(),
"counting_boundary":"object_store API, not provider HTTP attempts; list/delete/multipart parts are not counted",
"windows":[],"seeded_repositories":[],"shutdown_passed":false});
Expand Down
Loading
Loading