Skip to content

docs: dynamically generate llms.txt and llms-full.txt#10598

Open
luarss wants to merge 4 commits into
The-OpenROAD-Project:masterfrom
luarss:llms-txt
Open

docs: dynamically generate llms.txt and llms-full.txt#10598
luarss wants to merge 4 commits into
The-OpenROAD-Project:masterfrom
luarss:llms-txt

Conversation

@luarss
Copy link
Copy Markdown
Contributor

@luarss luarss commented Jun 5, 2026

Summary

Add support for llms.txt and llms-full.txt to the ReadTheDocs documentation site using the sphinx-llm extension. These files follow the llmstxt.org standard and make OpenROAD's documentation directly consumable by LLMs without HTML scraping.

Changes:

  • Add sphinx_llm.txt to the Sphinx extensions list in docs/conf.py — the extension generates both files as a native build step, writing directly into the HTML output
  • Add sphinx-llm to docs/requirements.txt
  • bump python version 3.7 -> 3.11

Once merged and built, files will be served at:

  • https://openroad.readthedocs.io/llms.txt
  • https://openroad.readthedocs.io/llms-full.txt

Type of Change

  • Documentation update

Impact

No change to tool behavior. Adds two new URL endpoints to the ReadTheDocs site after the next documentation build.

Verification

  • I have verified that the local build succeeds (./etc/Build.sh).
  • I have run the relevant tests and they pass.
  • My code follows the repository's formatting guidelines.
  • I have signed my commits (DCO).

Related Issues

N/A

@luarss luarss requested a review from a team as a code owner June 5, 2026 14:10
@luarss luarss requested a review from maliberty June 5, 2026 14:10
@github-actions github-actions Bot added the size/M label Jun 5, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces automated generation of llms.txt and llms-full.txt files to facilitate LLM documentation discovery. It adds a new Python script generate_llms_txt.py that parses the documentation structure from toc.yml to generate these files, integrates this script into the Sphinx configuration, and adds pyyaml to the documentation requirements. The review feedback suggests replacing legacy os.popen with subprocess.run to properly handle execution failures, explicitly specifying encoding="utf-8" for all file read and write operations to ensure cross-platform compatibility, and adding robust fallback handling for empty or malformed YAML parsing.

Comment thread docs/conf.py Outdated
Comment thread docs/generate_llms_txt.py Outdated
Comment thread docs/generate_llms_txt.py Outdated
Comment thread docs/generate_llms_txt.py Outdated
Comment thread docs/generate_llms_txt.py Outdated
Comment thread docs/generate_llms_txt.py Outdated
Comment thread docs/generate_llms_txt.py Outdated
@github-actions github-actions Bot added size/XS and removed size/M labels Jun 5, 2026
@luarss
Copy link
Copy Markdown
Contributor Author

luarss commented Jun 5, 2026

luarss added a commit to luarss/OpenROAD-flow-scripts that referenced this pull request Jun 5, 2026
Mirrors The-OpenROAD-Project/OpenROAD#10598. Adds sphinx-llm extension
to Sphinx config and requirements, bumps Python to 3.11 in readthedocs.

Signed-off-by: Jack Luar <jluar@precisioninno.com>
luarss added a commit to luarss/OpenROAD-flow-scripts that referenced this pull request Jun 6, 2026
Mirrors The-OpenROAD-Project/OpenROAD#10598. Adds sphinx-llm extension
to Sphinx config and requirements, bumps Python to 3.11 in readthedocs.

Signed-off-by: Jack Luar <jluar@precisioninno.com>
luarss added 3 commits June 6, 2026 13:45
Add docs/generate_llms_txt.py which reads toc.yml and all module
READMEs to produce llms.txt (navigation index) and llms-full.txt
(full inline content) on every Sphinx build. Hook it into conf.py
setup(app) alongside getMessages.py, copy outputs to the HTML root
via html_extra_path, and add pyyaml to docs/requirements.txt.
Both files are gitignored as build artifacts.

Signed-off-by: Jack Luar <39641663+luarss@users.noreply.github.com>
Replace the custom generate_llms_txt.py script with the sphinx-llm
extension, which generates llms.txt and llms-full.txt directly into
the Sphinx HTML output as a first-class build step. Removes the
html_extra_path workaround and the pyyaml dependency.

Signed-off-by: Jack Luar <39641663+luarss@users.noreply.github.com>
sphinx-llm requires Python >=3.9; the previous "3.7" setting caused
pip to find no matching distribution and fail the RTD build.

Signed-off-by: Jack Luar <39641663+luarss@users.noreply.github.com>
@github-actions github-actions Bot added size/L and removed size/XS labels Jun 6, 2026
luarss added a commit to luarss/OpenROAD-flow-scripts that referenced this pull request Jun 6, 2026
Mirrors The-OpenROAD-Project/OpenROAD#10598. Adds sphinx-llm extension
to Sphinx config and requirements, bumps Python to 3.11 in readthedocs.

Signed-off-by: Jack Luar <jluar@precisioninno.com>
…nx-llm

Add sphinx-llm to requirements.in, regenerate requirements.txt via
pip-compile with Python 3.10, and downgrade RTD Python back to 3.10.

Signed-off-by: Jack Luar <39641663+luarss@users.noreply.github.com>
@github-actions github-actions Bot added size/S and removed size/L labels Jun 6, 2026
@luarss luarss requested a review from vvbandeira June 6, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant