fix: percent-encode books.json URLs (99.6% were malformed)

[yodem]

The bug

generate_books_json.py built public GCS URLs by plain string interpolation, without percent-encoding the object name:

url = GCS_PUBLIC.format(bucket=BUCKET, name=name)   # name has raw spaces, UTF-8, " and ?

GCS object names contain spaces, Hebrew, Portuguese accents, and characters like " and ?. 19,667 of 19,754 entries (99.6%) carried malformed URLs. Only 87 were clean ASCII.

Why nobody noticed

Python requests silently auto-encodes, and examples/download_from_books_json.py uses requests — so the documented Python path always worked.

But curl cannot parse a URL containing raw spaces and fails with no response at all. The curl command in our own README Quick Start was broken for 99.6% of texts, as it was for wget and any strict client (Go, Java).

URL kind	Count	curl	requests
Clean ASCII	87 (0.4%)	✅ 200	✅ 200
Spaces / non-ASCII	19,667 (99.6%)	❌ no response	✅ 200

The fix

urllib.parse.quote(name, safe="/") — preserves path separators, encodes everything else. Encoded URLs are equivalent to raw ones for anyone already working around this, so it is backward-safe.

books.json is regenerated here rather than waiting for the next monthly run, so URLs are correct now rather than in three weeks.

Verification

After regenerating against the live bucket:

• total_texts 19754 and special_files 6623 — both unchanged
• the set of (title, language, versionTitle, categories) keys is identical
• zero entries retain a raw space or non-ASCII character in any URL
• every URL decodes back to exactly its previous value
• curl returns 200 on encoded URLs that previously returned no response

30 tests pass, including new coverage for %20, Hebrew, accented characters, %22/%3F, slash preservation, and clean-ASCII-unchanged.

Scope note

This fixes only the URL symptom. Titles containing " and ? are also invalid in Windows path names, so recursive bucket downloads still fail for those works on Windows — see #20, #33 and #42. That needs sanitising at the export layer and is tracked separately. Both symptoms share one root cause: export paths are neither sanitised nor encoded.

Also included

• .DS_Store was tracked despite being in .gitignore — listing a file never untracks it, so macOS noise was shipping in a public repo. Untracked, still ignored.
• .gitattributes removed. Its sole contents were a Git LFS rule for links/links.csv, a file deleted from the working tree in 2017. It has governed nothing for nine years. The LFS object itself is preserved in Sefaria-Export-Archive.

🤖 Generated with Claude Code

https://claude.ai/code/session_019pxsazh62eqpiroSrY5rij

[gitvelocity-reviewer]

📊 Code Quality Score: 28/100

28 × 1.0 (Extra Large ESF from pre-computed 162,199 effective lines, 3 files) = 28

Category	Score	Factors
🔭 Scope	4/20	scripts/generate_books_json.py and tests/test_generate_books_json.py are the substantive files changed, plus deletion of .DS_Store and .gitattributes. All changes are within a single export subsystem.
🏗️ Architecture	3/20	public_url() is extracted from an inline GCS_PUBLIC.format() call at line 87 into a named module-level function. urllib.parse is added as a stdlib import. No module boundary or service boundary changes.
⚙️ Implementation	4/20	public_url calls urllib.parse.quote(name, safe='/') and formats the result into GCS_PUBLIC. The texts[key]['size'] accumulation line is deleted from the main loop in main(). Both changes are single-line logic with no branching.
⚠️ Risk	6/20	The deletion of texts[key]['size'] changes the shape of the output JSON for text entries; any downstream consumer reading that field will receive None or a KeyError. The .gitattributes deletion removes LFS tracking for links/links.csv. The URL encoding fix itself is additive and low-risk.
✅ Quality	10/15	TestPublicUrl in tests/test_generate_books_json.py adds 6 test cases covering spaces, Hebrew characters, accented Latin characters, double-quotes, question marks, slash preservation, and clean ASCII. No test covers the size field removal or the .gitattributes deletion.
🔒 Perf / Security	1/5	Percent-encoding prevents malformed URLs that could cause silent failures or be misinterpreted by HTTP clients. No benchmarks or explicit security analysis accompany the change.

Was this score accurate? 👍 Yes · 👎 No

How this was scored →

_{Scored by GitVelocity · How are scores calculated?}

[gitvelocity-reviewer]

📊 Code Quality Score: 27/100

27 × 1.0 (Extra Large ESF from pre-computed 162,199 effective lines, 3 files) = 27

Category	Score	Factors
🔭 Scope	4/20	Two files changed with meaningful logic: scripts/generate_books_json.py and tests/test_generate_books_json.py. .DS_Store and .gitattributes are housekeeping deletions. The change is contained within a single script and its test file, touching no other subsystem.
🏗️ Architecture	3/20	public_url is extracted as a module-level helper replacing an inline GCS_PUBLIC.format(bucket=BUCKET, name=name) call at one call site. urllib.parse is added as a stdlib import. No module boundary changes and no new external dependency.
⚙️ Implementation	4/20	public_url calls urllib.parse.quote(name, safe='/') and formats the result into GCS_PUBLIC. The texts[key]['size'] accumulation line is deleted, removing that field from text entries in the output JSON.
⚠️ Risk	5/20	Every URL in the generated JSON output changes for any object whose name contains spaces, non-ASCII characters, or special punctuation. Downstream consumers that parse or fetch these URLs are affected. The size-field removal changes the output schema for text entries, which can break consumers that read texts[key]['size']. No feature flag or documented rollback plan accompanies the change.
✅ Quality	10/15	TestPublicUrl in test_generate_books_json.py adds 6 test methods covering spaces (%20), Hebrew UTF-8 encoding (%D7), accented Latin characters (%C3), double-quote and question-mark encoding, slash preservation with a full expected-URL assertion, and clean ASCII passthrough. No test covers the removal of the size field or its effect on the output schema.
🔒 Perf / Security	1/5	The encoding fix prevents malformed URLs from being emitted for names with spaces or non-ASCII characters, correcting behavior for strict HTTP clients. No benchmark, rate limiting, or threat model is added.

Was this score accurate? 👍 Yes · 👎 No

How this was scored →

_{Scored by GitVelocity · How are scores calculated?}