diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/.env.example b/use-cases/iashutoshyadav/api-reference-guide-authoring/.env.example new file mode 100644 index 00000000..c5869346 --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/.env.example @@ -0,0 +1,7 @@ +# Copy to .env and fill in. Nothing in this file is a real secret. + +# Get a free key at https://use.superdocs.app -> Settings -> API Keys -> Create. +# Leave empty to run against the offline fake client (no network, no quota - what tests use by default). +SUPERDOCS_API_KEY= + +PORT=8020 diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/.gitignore b/use-cases/iashutoshyadav/api-reference-guide-authoring/.gitignore new file mode 100644 index 00000000..f5a85b7a --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/.gitignore @@ -0,0 +1,7 @@ +.env +__pycache__/ +*.pyc +.pytest_cache/ +*.db +.venv/ +venv/ diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/README.md b/use-cases/iashutoshyadav/api-reference-guide-authoring/README.md new file mode 100644 index 00000000..81785a18 --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/README.md @@ -0,0 +1,164 @@ +# API-reference & Guide Authoring + +Built by **Ashutosh Yadav** for the SuperDocs engineering hiring round. + +## What it does + +A chat-driven authoring tool for developer-facing teams, two modes: + +1. **Markdown round-trip authoring** - paste a markdown page (a code block, a diagram reference), convert + it into a real SuperDocs styled document, then back to markdown. The code block and the diagram + reference must survive unchanged - that's the whole point. Optionally attach a real image file, which + gets genuinely uploaded via SuperDocs's own image endpoint (`POST /v1/documents/images/upload-base64`) + and embedded in place of the placeholder diagram path - not just a markdown text reference passed + through unchanged. +2. **Getting-started guide generation** - describe an API or feature in plain language, get back a numbered + onboarding guide, written by SuperDocs from a single instruction (not by us generating it ourselves). + +## How to run it + +```bash +python -m venv .venv +source .venv/bin/activate # .venv\Scripts\activate on Windows +pip install -r requirements.txt +cp .env.example .env # leave SUPERDOCS_API_KEY empty to run against the offline fake client +uvicorn app.web:app --port 8020 +``` + +Open `http://localhost:8020` - both modes are on one page, pre-filled with a sample. + +## Tests (no live key needed) + +```bash +pytest +``` + +6 tests, offline against a fake SuperDocs client covering both modes' pipelines: round-trip preservation +of a code block and an image reference, numbered-guide generation from a feature description, and a +regression test capturing the exact real-API finding described below (language tag lost, content intact). + +**Verified against the real API, not just the fake client** - `verify_real_api.py` (not part of the +pytest suite, run manually once with a real key) exercised both modes against the live product: + +- **Diagram/image reference: preserved exactly.** `![sequence diagram](./diagrams/create-widget-flow.png)` + survived the round trip unchanged. +- **Code block: content preserved exactly, language tag not.** The code inside the fence came back + byte-for-byte identical - nothing a reader would copy-paste changed. What *did* change: a fenced block + opened as `` ```python `` came back as a bare `` ``` `` - the language tag used for syntax highlighting + is not preserved through this round trip. `RoundTripResult` reports this precisely + (`code_blocks_preserved=False`, `code_content_preserved=True`) rather than collapsing it into one + blunt pass/fail - a reviewer sees exactly what changed and what didn't, and the UI shows both signals + separately. Captured as a permanent regression test + (`test_a_real_finding_captured_as_a_permanent_check_language_tag_lost_but_code_content_intact`), not left + as a one-off observation. +- **An extra blank line appears before a code fence's closing ` ``` `.** Found by comparing exact newline + counts between the original and round-tripped markdown (22 vs 23) after a visual anomaly in the browser + turned out to be a display-only artifact, not a data bug - the newline-count check was what actually + confirmed a real, tiny difference existed and pinned down exactly where. It doesn't affect what a reader + would copy (the content comparison strips trailing whitespace before comparing, so + `code_content_preserved` correctly stays `True`), but it's a precise, real behavior worth recording + alongside the language-tag finding rather than left undocumented because it happened to be harmless. + Captured as a permanent regression test + (`test_a_real_finding_extra_blank_line_before_closing_fence_does_not_affect_content_preservation`). +- **The "stable public URL" `POST /v1/documents/images/upload-base64` returns is never actually + accessible.** Found the hard way: after embedding it, clicking the "real image uploaded" link this + build shows produced a live Google Cloud Storage `403 AccessDenied` - "Anonymous caller does not have + storage.objects.get access." Confirmed precisely by GET-ing both response fields with no auth headers: + `url` (the one the docs describe as "a stable public URL for embedding via ``") returns + `403` every time, for everyone; only `view_url` (the signed, time-limited link in the same response) + returns `200`. The docs' own description of `url` is simply wrong - it isn't public at all. Fixed by + switching `upload_image()` to return `view_url` instead. This turned out to also explain the finding + below: SuperDocs's export was never "downgrading" a working stable URL into a signed one - the signed + URL was the only one that ever worked, so once this build started uploading `view_url` from the start, + the round trip started preserving the reference byte-for-byte (`image_refs_preserved` now `True`, not + just `image_target_preserved`). +- **A real uploaded image's URL is time-limited (24h), even once you're using the working field.** The + Google Cloud Storage signed URL SuperDocs exports carries `X-Goog-Expires=86400` - a document + re-exported after the signature expires could show a broken image. Worth flagging to SuperDocs + directly: neither response field is a truly durable, permanently-public embed URL. Captured as a + permanent regression test + (`test_a_real_finding_signed_url_rewrite_does_not_count_as_the_image_changing`), which still holds as a + useful check even after the field-name fix, since a fresh signature is still technically a different + string than whatever was embedded moments earlier. +- **Getting-started guide generation: works well.** The real output is a properly structured guide with + headers, numbered steps, bold text, and a working code example - noticeably better than this project's + own offline simulation, which only exists to keep the pipeline testable without a live key. +- **Guide generation is honest about missing specifics, not silent about them.** Given a deliberately + generic feature description with no named SDK or package, the real output inserted visible + `Please fill: [Name of SDK]`-style placeholders instead of inventing a plausible-looking but fake package + name. The right behavior for an AI-authoring tool - guessing wrong here would be worse than an obvious + placeholder a human has to fill in. +- **The first prompt let the model add unrequested flourishes** - a fabricated copyright line, a + "Developer Relations Team" attribution, and a publication date, none of which were in the feature + description or asked for. Fixed by tightening the prompt (`app/guide_generator.py`) to explicitly + instruct against inventing attribution/version/date metadata, on top of the existing "flag what's + missing" instruction. Verified against the real API with the identical input before and after: the + fabricated footer is gone, and the guide is now *more* honest than before, not just quieter - it also + started flagging two more unstated details (Node.js version, API base URL) that the earlier prompt had + silently guessed at rather than flagging. +- **A generated "working example" wasn't actually runnable.** An OAuth2 guide's code sample defined a + `getAccessToken()` function but never called it, then referenced a `tokenData` variable that was never + declared - copy-pasting it would throw `ReferenceError: tokenData is not defined`. Fixed by adding an + explicit instruction that the working example must be self-contained: every function defined must be + called, every variable referenced must be assigned first. Verified against the real API with the + identical OAuth2 input: the re-generated example properly chains `response = requests.post(...)` → + `token_data = response.json()` → `access_token = token_data['access_token']` before using + `access_token`, with no dangling references. + +**A real bug that broke a request outright, not a quality issue:** both `/roundtrip` and `/guide` reused a +single hardcoded `session_id` ("roundtrip-session", "guide-session") across every request for the entire +life of the running process. Every unrelated request kept piling another document upload and chat turn +into the same SuperDocs session, and after enough real requests during this round's own live testing, that +session's accumulated state broke - `export_document` started returning a real `404`, and the app returned +a `500` to the browser. Fixed in `app/web.py` by generating a fresh `uuid.uuid4()`-suffixed session ID per +request, since each request is logically independent and has no reason to share session state with the +last one. `renewal-true-up-engine` (the sibling project) doesn't have this bug - it already scopes +`session_id` per customer, not globally. + +**A real bug, not a product-behavior finding:** a real `SUPERDOCS_API_KEY` set in `.env` was silently +ignored - `python-dotenv` was a listed dependency but `load_dotenv()` was never actually called, so the app +kept reading an empty environment and running against the fake client regardless of what `.env` said, with +only an easy-to-miss "offline fake client" banner as any signal (no error, no crash). Found live on +2026-08-19 in the sibling `renewal-true-up-engine` project first, then confirmed here too - same missing +call, same fix (`load_dotenv()` at module load, before anything reads the environment). + +**A second real bug, found by testing through the actual browser form rather than `curl`:** the same +round-trip input intermittently reported "code content changed" when submitted from the page, but always +reported "preserved exactly" when sent programmatically with identical-looking content. Root cause: an +HTML ` + +
+ + + {% if roundtrip_result %} +

Result

+
+ Image/diagram reference: + {% if roundtrip_result.image_refs_preserved %}preserved{% else %}changed{% endif %} + + {% if roundtrip_result.real_image_url %} + Same underlying image: + {% if roundtrip_result.image_target_preserved %}preserved{% else %}changed{% endif %} + + {% endif %} + Code fence (tag + content): + {% if roundtrip_result.code_blocks_preserved %}preserved exactly{% else %}changed{% endif %} + + Code content only: + {% if roundtrip_result.code_content_preserved %}preserved exactly{% else %}changed{% endif %} + +
+ {% if roundtrip_result.real_image_url %} +
+ Uploaded diagram preview +
+

Real image uploaded to SuperDocs

+ View full size → +
+
+ {% endif %} +
+
+

Original

+
+ + 
{{ roundtrip_result.original_markdown }}
+
+
+
+

After round-trip

+
+ + 
{{ roundtrip_result.round_tripped_markdown }}
+
+
+
+ {% endif %} + + +
+

Mode 2 · Getting-started guide generator

+

Describe an API or feature in plain language — get a numbered onboarding guide.

+
+ +
+
+ + {% if guide_result %} +

Generated guide

+
+ + 
{{ guide_result }}
+
+ {% endif %} +
+ + + + diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/app/web.py b/use-cases/iashutoshyadav/api-reference-guide-authoring/app/web.py new file mode 100644 index 00000000..82090e61 --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/app/web.py @@ -0,0 +1,94 @@ +"""Chat-driven authoring app, two modes, per the task card. In-memory state for the same reason as the +sibling project: this is a single-operator authoring session, not a multi-user persistent system. +""" + +from __future__ import annotations + +import os +import uuid + +from dotenv import load_dotenv +from fastapi import FastAPI, File, Form, Request, UploadFile +from fastapi.responses import HTMLResponse +from fastapi.templating import Jinja2Templates + +from app.guide_generator import generate_getting_started_guide +from app.markdown_roundtrip import RoundTripResult, round_trip_markdown +from app.superdocs_client import make_client + +# Same real bug found in the sibling project (renewal-true-up-engine/app/web.py, 2026-08-19): +# python-dotenv doesn't load .env into os.environ on its own - without this call, a real +# SUPERDOCS_API_KEY set in .env was silently ignored with no error, just an unnoticed fake-client badge. +load_dotenv() + +app = FastAPI(title="API-reference & Guide Authoring") +templates = Jinja2Templates(directory="app/templates") + +_state: dict[str, object] = {"client": None, "roundtrip_result": None, "guide_result": None} + +_SAMPLE_MARKDOWN = """# API Reference: createWidget + +Creates a new widget. + +```python +import widgets + +client = widgets.Client(api_key="...") +result = client.create_widget(name="my-widget") +``` + +![sequence diagram](./diagrams/create-widget-flow.png) +""" + + +def _get_client(): + if _state["client"] is None: + _state["client"] = make_client(os.environ.get("SUPERDOCS_API_KEY")) + return _state["client"] + + +@app.get("/", response_class=HTMLResponse) +def dashboard(request: Request): + using_fake = os.environ.get("SUPERDOCS_API_KEY", "") == "" + return templates.TemplateResponse( + request, + "dashboard.html", + { + "using_fake": using_fake, + "sample_markdown": _SAMPLE_MARKDOWN, + "roundtrip_result": _state["roundtrip_result"], + "guide_result": _state["guide_result"], + }, + ) + + +@app.post("/roundtrip", response_class=HTMLResponse) +def roundtrip( + request: Request, + markdown_text: str = Form(...), + diagram_image: UploadFile | None = File(default=None), +): + client = _get_client() + uploaded_image = None + if diagram_image is not None and diagram_image.filename: + uploaded_image = (diagram_image.file.read(), diagram_image.content_type or "image/png") + # Real bug, found live on 2026-08-20: a fixed session_id here meant every round-trip request for + # the process's entire lifetime piled another upload into the SAME SuperDocs session. That session + # eventually accumulated enough state to break - export_document started returning a real 404. Each + # request is logically independent (a fresh markdown paste has nothing to do with the last one), so + # each gets its own fresh session instead of sharing one that grows without bound. + result: RoundTripResult = round_trip_markdown( + client, markdown_text, session_id=f"roundtrip-{uuid.uuid4()}", uploaded_image=uploaded_image + ) + _state["roundtrip_result"] = result + return dashboard(request) + + +@app.post("/guide", response_class=HTMLResponse) +def guide(request: Request, feature_description: str = Form(...)): + client = _get_client() + # Same fix as roundtrip() above, same real failure mode: this is what actually 404'd first. + _state["guide_result"] = generate_getting_started_guide( + client, feature_description, session_id=f"guide-{uuid.uuid4()}" + ) + return dashboard(request) diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/requirements.txt b/use-cases/iashutoshyadav/api-reference-guide-authoring/requirements.txt new file mode 100644 index 00000000..de28bc05 --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/requirements.txt @@ -0,0 +1,8 @@ +fastapi==0.141.1 +uvicorn[standard]==0.52.3 +httpx==0.28.1 +pydantic==2.13.4 +jinja2==3.1.6 +python-dotenv==1.2.2 +python-multipart==0.0.32 +pytest==9.1.1 diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/__init__.py b/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/__init__.py new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/__init__.py @@ -0,0 +1 @@ + diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/test_guide_generator.py b/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/test_guide_generator.py new file mode 100644 index 00000000..3f72c92c --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/test_guide_generator.py @@ -0,0 +1,29 @@ +from app.guide_generator import generate_getting_started_guide +from app.superdocs_client import FakeSuperDocsClient + + +def test_generates_a_numbered_guide_from_a_feature_description(): + client = FakeSuperDocsClient() + + guide = generate_getting_started_guide( + client, + "Sign up for an account. Create an API key in Settings. Install the SDK with pip install widgets-sdk. " + "Call client.create_widget to make your first widget.", + session_id="guide-1", + ) + + assert "1." in guide + assert "Sign up for an account" in guide + assert "Call client.create_widget" in guide + + +def test_each_sentence_becomes_its_own_numbered_step(): + client = FakeSuperDocsClient() + + guide = generate_getting_started_guide( + client, "Step one happens. Step two happens. Step three happens.", session_id="guide-2", + ) + + assert "1. Step one happens." in guide + assert "2. Step two happens." in guide + assert "3. Step three happens." in guide diff --git a/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/test_markdown_roundtrip.py b/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/test_markdown_roundtrip.py new file mode 100644 index 00000000..d93727c1 --- /dev/null +++ b/use-cases/iashutoshyadav/api-reference-guide-authoring/tests/test_markdown_roundtrip.py @@ -0,0 +1,174 @@ +"""Runs against FakeSuperDocsClient (byte-identical passthrough export) - proves this module's own +pipeline (upload -> styled export -> markdown export -> compare) is wired correctly. Does NOT prove +SuperDocs's real product preserves fidelity - that needs a live key, flagged explicitly in +markdown_roundtrip.py's own docstring, not silently assumed here.""" + +from app.markdown_roundtrip import round_trip_markdown +from app.superdocs_client import FakeSuperDocsClient + +SAMPLE_MARKDOWN = """# API Reference: createWidget + +Creates a new widget. + +```python +import widgets + +client = widgets.Client(api_key="...") +result = client.create_widget(name="my-widget") +``` + +Architecture: + +![sequence diagram](./diagrams/create-widget-flow.png) + +See also the [full reference](./reference.md). +""" + + +def test_round_trip_preserves_the_code_block_and_the_image_reference(): + client = FakeSuperDocsClient() + + result = round_trip_markdown(client, SAMPLE_MARKDOWN, session_id="rt-1") + + assert result.code_blocks_preserved + assert result.image_refs_preserved + assert result.fully_preserved + assert len(result.original_code_blocks) == 1 + assert "import widgets" in result.original_code_blocks[0] + assert result.original_image_refs == ["![sequence diagram](./diagrams/create-widget-flow.png)"] + + +def test_round_trip_result_reports_exact_content_not_just_a_boolean(): + """The pipeline must expose what it actually compared, not just pass/fail - a reviewer (or this + project's own demo) needs to see the before/after, not take a boolean's word for it.""" + client = FakeSuperDocsClient() + + result = round_trip_markdown(client, SAMPLE_MARKDOWN, session_id="rt-2") + + assert result.original_code_blocks == result.round_tripped_code_blocks + assert result.original_image_refs == result.round_tripped_image_refs + assert result.original_markdown == SAMPLE_MARKDOWN + + +def test_a_document_with_no_code_or_images_still_reports_preserved_trivially(): + client = FakeSuperDocsClient() + plain_markdown = "# Just a heading\n\nSome plain prose, nothing fancy.\n" + + result = round_trip_markdown(client, plain_markdown, session_id="rt-3") + + assert result.fully_preserved + assert result.original_code_blocks == [] + assert result.original_image_refs == [] + + +class _LanguageTagStrippingClient(FakeSuperDocsClient): + """Simulates a real, specific finding from running this build against the live SuperDocs API: the + code's own content survives a round-trip byte-for-byte, but the fence's language tag (```python -> + ```) is stripped. Not a hypothetical edge case - this is what the real product actually did when + checked (verify_real_api.py), captured here as a permanent regression check rather than left as a + one-off observation that could silently stop being true (or silently start being worse) later.""" + + def export_document(self, session_id: str, format: str = "markdown") -> str: + content = super().export_document(session_id, format) + return content.replace("```python\n", "```\n") + + +class _ExtraBlankLineBeforeClosingFenceClient(FakeSuperDocsClient): + """Simulates a second, smaller real finding from the live product (2026-08-20): an extra blank line + appears right before a code fence's closing ``` that wasn't in the original. Found by comparing exact + newline counts (22 vs 23) after a visual anomaly in a real browser turned out to be a rendering-only + artifact - this newline-count check was what actually confirmed a real difference existed.""" + + def export_document(self, session_id: str, format: str = "markdown") -> str: + content = super().export_document(session_id, format) + return content.replace("```python\n", "```\n").replace("```\n\n", "\n```\n\n") + + +def test_a_real_finding_extra_blank_line_before_closing_fence_does_not_affect_content_preservation(): + """The extra blank line sits between the code's last line and the closing fence - outside the code + itself. code_content_preserved must stay True (nothing a reader would copy-paste changed; the content + comparison strips trailing whitespace before comparing), even though code_blocks_preserved correctly + stays False (the fence text itself did change).""" + client = _ExtraBlankLineBeforeClosingFenceClient() + + result = round_trip_markdown(client, SAMPLE_MARKDOWN, session_id="rt-6") + + assert result.code_blocks_preserved is False + assert result.code_content_preserved is True + assert result.image_refs_preserved is True + + +def test_a_real_finding_captured_as_a_permanent_check_language_tag_lost_but_code_content_intact(): + """This is the precise, honest signal this build reports: code_blocks_preserved is False (the fence + changed), but code_content_preserved is True (nothing a reader would copy-paste actually changed). A + single blunt boolean would either hide this real limitation or overstate it as total failure - neither + is the honest answer.""" + client = _LanguageTagStrippingClient() + + result = round_trip_markdown(client, SAMPLE_MARKDOWN, session_id="rt-4") + + assert result.code_blocks_preserved is False + assert result.code_content_preserved is True + assert result.image_refs_preserved is True + assert result.fully_preserved is False # strict check correctly still fails + + +def test_a_browser_submitted_CRLF_original_is_not_falsely_reported_as_changed(): + """Regression test for a real bug found live on 2026-08-20: HTML