Skip to content

lightrag-memgraph: kv upsert() TOCTOU race, comment overclaims atomicity #218

Description

@antejavor

What's wrong

MemgraphKVStorage.upsert() in integrations/lightrag-memgraph/src/lightrag_memgraph/kv_impl.py decides create_time vs update_time via a separate filter_keys() round trip before the actual MERGE write:

existing = set(data.keys()) - await self.filter_keys(set(data.keys()))
...
if k in existing:
    value["update_time"] = current_time
else:
    value["create_time"] = current_time
    value["update_time"] = current_time

A code comment claims this "matches JsonKVStorage semantics", but JsonKVStorage holds a lock across both the existence check and the write (non-reentrant, serializes concurrent callers); this implementation's check and write are two independent Memgraph round trips with nothing serializing them — a real TOCTOU race.

Reachable in practice: LightRAG's max_parallel_insert runs multiple documents concurrently with no keyed lock on text_chunks/full_docs, and chunk ids are content-hash based (compute_mdhash_id(content, ...)), so two documents sharing identical chunk text (duplicated boilerplate/headers) can both call upsert() for the same key concurrently. Both see the key as "not existing" via filter_keys() before either writes, so both stamp create_time.

Severity: the resulting drift is low-severity bookkeeping (wrong create_time/update_time classification), not data loss — the last-write-wins full-data overwrite this could also cause is not actually a Memgraph-specific regression, since JsonKVStorage has the same last-write-wins behavior on genuine key collisions.

Acceptance criteria

  • upsert() determines create-vs-update in a single round trip (e.g. MERGE ... ON CREATE SET ... ON MATCH SET ...) instead of a separate filter_keys() check
  • Comment updated to accurately describe the actual concurrency guarantee (or lack thereof)
  • Test covering concurrent upsert() calls with an overlapping new key

Blocked by

None - can start immediately

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions