Skip to content

[FEATURE] Add RDF/SPARQL graph store with optional Neptune IAM authentication - #405

Draft
KilianTrunk wants to merge 4 commits into
awslabs:mainfrom
Ortecha:feat/native-sparql-rdflib-iam
Draft

[FEATURE] Add RDF/SPARQL graph store with optional Neptune IAM authentication#405
KilianTrunk wants to merge 4 commits into
awslabs:mainfrom
Ortecha:feat/native-sparql-rdflib-iam

Conversation

@KilianTrunk

Copy link
Copy Markdown

Description

This PR adds a new lexical-graph-contrib/sparql package for storing and retrieving the lexical graph through SPARQL 1.1 query and update endpoints.

The package uses RDFLib and supports generic HTTP endpoints. It also provides optional IAM authentication for the Amazon Neptune SPARQL endpoint.

Lexical graph builders and retrievers now pass a GraphOperation value together with the existing graph query. The SPARQL graph store uses this value to select its native SPARQL implementation. Existing graph stores continue to execute their existing queries.

Changes

  • Add the graphrag-toolkit-lexical-graph-sparql contributor package.
  • Add RDFLib and Requests as package dependencies.
  • Add Botocore as an optional dependency for Neptune IAM authentication.
  • Add a graph store factory for sparql, sparql+http, sparql+https, and sparql+neptune connection schemes.
  • Support separate SPARQL query and update endpoints.
  • Support HTTP Basic authentication, custom HTTP headers, and request timeouts for generic endpoints.
  • Add native SPARQL implementations for all 32 graph operations introduced by this PR: 17 update operations and 15 query operations.
  • Represent lexical graph nodes with deterministic RDF IRIs and configurable namespaces.
  • Represent facts as RDF resources connected to their subject, predicate, object, and supporting statement.
  • Store each tenant in a deterministic named graph.
  • Sign Neptune SPARQL requests with AWS Signature Version 4 for the neptune-db service.
  • Resolve Neptune credentials through Botocore for every request so temporary credentials can be refreshed.
  • Pass GraphOperation through retries, batched writes, query trees, and multi-tenant graph stores.
  • Keep the existing query execution as the default for graph stores that do not provide an operation-specific implementation.
  • Add SPARQL unit tests and extend core tests for operation forwarding and graph store factory selection.
  • Add a contributor README and an RDF/SPARQL page to the documentation site.
  • Rename the local development Compose project from local-dev to graphrag-toolkit-rdf-dev.
  • Run the Jupyter reader dependency installation as root, apply fix-permissions to the Conda and notebook-user directories, and then return to the jovyan user.

Problem

The toolkit did not provide a GraphStore implementation for SPARQL query and update endpoints.

Because of this, the lexical graph storage interface could not be used with an RDF store through SPARQL. The toolkit also did not provide an IAM-signed transport for requests to the Amazon Neptune SPARQL endpoint.

This PR adds the missing SPARQL graph store as a contributor package while keeping the current graph stores and query execution paths.

Related issue (if any): N/A

Testing

  • Unit tests added/updated
  • Integration tests added (as appropriate)
  • Existing tests pass (pytest)
  • Tested manually (describe below)

Test results:

  • pytest lexical-graph-contrib/sparql/tests: 108 passed
  • pytest lexical-graph/tests: 1,756 passed and 1 skipped
  • git diff --check upstream/main...HEAD: no errors

Endpoint behavior was tested manually against RDFox, GraphDB, and Amazon Neptune.

Checklist

  • Code follows existing style and conventions
  • License headers present on new files
  • Documentation updated (if applicable)
  • No breaking changes

The new GraphOperation argument is optional. Graph stores without an operation-specific implementation continue to execute their existing query.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

KilianTrunk and others added 3 commits July 16, 2026 11:58
Co-authored-by: Matic <majericmatic@gmail.com>
Co-authored-by: Tom Soru <tom@tommaso-soru.it>
Co-authored-by: Matic <majericmatic@gmail.com>
Co-authored-by: Tom Soru <tom@tommaso-soru.it>
Co-authored-by: Matic <majericmatic@gmail.com>
Co-authored-by: Tom Soru <tom@tommaso-soru.it>

Supported schemes are:

| Scheme | Endpoint transport |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all these bespoke scheme extensions really nessecary? It measn that developers will have to do some string concatination any time they want to pass in a SPARQL endpoint. I would prefer we found another way to specify specific headers for authentication.

…book

Add lexical_graph.ttl, the RDF schema for the lexical-graph vocabulary
(lg:Source, lg:Chunk, lg:Topic, lg:Statement, lg:Fact, lg:Entity, lg:Relation,
lg:SysClass, lg:SysRelation and their properties), inside the sparql package so
it installs and ships with the backend. Verified against a live store: every
term the write path emits is declared. lg:subject and lg:object deliberately
carry no rdfs:range, because each takes either an Entity IRI or a plain literal.
Source and Chunk metadata keys become lg:<key> properties dynamically and are
documented as a note rather than enumerated.

Add 00-SPARQL-Setup.ipynb so the published image can be pointed at any SPARQL
1.1 endpoint without a source checkout. It registers the contrib
SPARQLGraphStoreFactory - the one step environment variables cannot do - wires
the graph and vector stores from GRAPH_STORE/VECTOR_STORE, verifies endpoint
connectivity, loads the ontology into its own named graph so the schema stays
separate from tenant instance data, and documents tenant named-graph scoping.
The other notebooks in the folder target Neo4j.

Add Dockerfile.hub and a repo-root .dockerignore that build the self-contained
image: lexical-graph and the RDF/SPARQL contrib baked in, an embedded empty
pgvector as the vector store, the example notebooks and JupyterLab. The graph
store (RDFox, Fuseki, Neptune, ...) stays external.
@KilianTrunk
KilianTrunk marked this pull request as draft July 23, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants