Bug: internal_info / info stored as JSON string by Neo4j sanitizer but never deserialized on read — breaks TextualMemoryItem validation on recall

[bencomputer-vn]

Pre-submission checklist | 提交前检查

• I have searched existing issues and this hasn't been mentioned before | 我已搜索现有问题，确认此问题尚未被提及
• I have read the project documentation and confirmed this issue doesn't already exist | 我已阅读项目文档并确认此问题尚未存在
• This issue is specific to MemOS and not a general software issue | 该问题是针对 MemOS 的，而不是一般软件问题

Bug Description | 问题描述

Neo4jCommunityGraphDB (and Neo4jGraphDB) serialize every dict-typed metadata value into a JSON string before writing, but the read path only deserializes sources. Fields such as internal_info and info therefore come back as str instead of dict, and TextualMemoryItem — which declares them as dict | None — raises ValidationError during recall.

This is the same class of bug already reported for the Postgres backend in #2229 and #2270 (pgvector embedding returned as string breaking TextualMemoryItem), but on the Neo4j write/read path and affecting a different field.

Root cause

Write path — src/memos/graph_dbs/neo4j_community.py:

metadata = _sanitize_neo4j_metadata(metadata)   # stringifies every dict
if metadata.get("sources"):
    for idx in range(len(metadata["sources"])):
        metadata["sources"][idx] = json.dumps(metadata["sources"][idx])

_sanitize_neo4j_value() in src/memos/graph_dbs/neo4j.py:

if isinstance(value, dict):
    return json.dumps(value, ensure_ascii=False, sort_keys=True)

So internal_info: dict becomes internal_info: str.

Read path — _parse_node() and _parse_nodes() in neo4j_community.py only reverse the sources transform. There is no equivalent for internal_info or info. The same asymmetry exists in neo4j.py.

Failure point — internal_info is listed among the returned fields in src/memos/memories/textual/tree_text_memory/retrieve/recall.py, and src/memos/memories/textual/item.py:157 declares internal_info: dict | None. A str value fails validation and recall raises.

Why this is not always visible

internal_info is None for simple conversational memories, and None skips the dict branch in the sanitizer. It is only populated on paths such as mem_reader/multi_modal_struct.py:173 and :426 (document chunking) and dream/enrichment.py:156. A plain POST /product/add with a short chat message therefore works fine; the failure surfaces once documents are ingested or Dream is enabled.

Note that api/handlers/search_handler.py:242, dream/search.py:130 and _coerce_json_dict() in dream/contextualization.py:214 already contain defensive handling for internal_info, suggesting the problem has been worked around downstream rather than fixed at the storage boundary.

Suggested fix

Make the read path symmetric with the write path, in both _parse_node() and _parse_nodes() in neo4j_community.py and the equivalent location in neo4j.py:

for _f in ("internal_info", "info"):
    _v = node.get(_f)
    if isinstance(_v, str) and _v.startswith("{") and _v.endswith("}"):
        try:
            node[_f] = json.loads(_v)
        except (ValueError, TypeError):
            node[_f] = None

A more durable fix would avoid the hardcoded field list — deriving the set of dict-typed fields from the metadata model, or recording which keys were serialized at write time — so future dict fields are handled automatically.

How to Reproduce | 如何重现

1. Deploy with NEO4J_BACKEND=neo4j-community, Neo4j 5.26.6 Community + Qdrant v1.15.3 as vec_db.

2. Ingest content that goes through the chunking path so internal_info is populated (document/file ingestion rather than a one-line chat message).

3. Inspect the stored payload — internal_info is a JSON string:

   curl -s 'http://:6333/collections/neo4j_vec_db/points/scroll'
   -H 'Content-Type: application/json'
   -d '{"limit":2,"with_payload":true}'

4. Call POST /product/search — recall raises ValidationError on TextualMemoryItem.metadata.internal_info (expected dict, got str).

For comparison, sources in the same payload is also stored as a JSON string but is correctly restored on read, which makes the asymmetry visible:

"sources": ["{"type": "chat", "role": "user", ...}"]

Environment | 环境信息

• Python version: 3.11.16 (inside official docker/Dockerfile image)
• Operating System: Ubuntu 24.04 (Docker host), Debian trixie (container base)
• MemOS version: built from main via docker/Dockerfile (not a pip release)
• Graph DB: Neo4j 5.26.6 Community, NEO4J_BACKEND=neo4j-community, MOS_NEO4J_SHARED_DB=true
• Vector DB: Qdrant v1.15.3 (external server, collection neo4j_vec_db)
• Embedder: bge-m3 via Ollama OpenAI-compatible endpoint, EMBEDDING_DIMENSION=1024

Additional Context | 其他信息

Related issues sharing the same underlying pattern (value serialized on write, not deserialized on read, TextualMemoryItem validation fails) on a different backend and field:

• Bug: PostgresGraphDB returns empty search results (pgvector embedding string breaks TextualMemoryItem) #2229 — PostgresGraphDB returns empty search results (pgvector embedding string breaks TextualMemoryItem)
• fix: Postgres reorganizer fails when pgvector embedding is returned as string #2270 / Fix #2270: fix: Postgres reorganizer fails when pgvector embedding is returned as string #2275 — Postgres reorganizer fails when pgvector embedding is returned as string

Since the same failure mode has now appeared on both the Postgres and Neo4j paths, it may be worth addressing the serialize/deserialize contract at the storage boundary rather than patching each field individually.

Willingness to Implement | 实现意愿

• I'm willing to implement this myself | 我愿意自己解决
• I would like someone else to implement this | 我希望其他人来解决

[Memtensor-AI]

🤖 AutoDev has picked up this issue and started working on it.

Task ID: 9d183872ea1d90db
Working branch: bugfix/autodev-2288-20260827081609558
Target branch: latest dev* branch
Workflow: opsp (analysis → coding → testing → PR)

I will post the PR link here once done. If I need more information, I will ask in the comments.

[Memtensor-AI]

✅ AutoDev task 9d183872ea1d90db completed.

Summary: Fixes #2288: Neo4jGraphDB / Neo4jCommunityGraphDB sanitize every dict-typed metadata value into a JSON string on write via _sanitize_neo4j_value, but the read path only reversed the sources transform. internal_info and info therefore came back as str, breaking TextualMemoryItem's dict | None pydantic validation during recall on document-ingest / Dream-enabled deployments.

Adds a small _deserialize_dict_field helper in src/memos/graph_dbs/neo4j.py (symmetric with the write-side sanitizer, module constant _DICT_METADATA_FIELDS = ("internal_info", "info")), and wires it into Neo4jGraphDB._parse_node, Neo4jCommunityGraphDB._parse_node, and Neo4jCommunityGraphDB._parse_nodes. JSON-object-shaped strings deserialize to dict; malformed strings fall back to None so pydantic still validates; dict / None / non-object strings pass through unchanged. The write path and other graph backends (Postgres, PolarDB, Nebular) are intentionally untouched — the boundary contract fix mirrors the analogous Postgres work in #2229 / #2270 without expanding scope.

Coverage: 16 new tests in tests/graph_dbs/test_neo4j_internal_info_roundtrip.py — 7 helper unit tests, 8 _parse_node / _parse_nodes regressions for both editions, and one end-to-end guard that feeds the parser output into TreeNodeTextualMemoryMetadata(**metadata) and asserts pydantic validation succeeds, reproducing and closing the exact failure mode from the issue. python3 -m pytest tests/graph_dbs/ tests/memories/textual/ → 107 passed, 3 skipped (Neo4j integration cases behind an opt-in flag). Ruff check + ruff format both clean.

Base branch: main
Branch: bugfix/autodev-2288-20260827081609558
Commit: 958ee25
PR: #2289
Assigned to: @Wang-Daoji
Reviewers: @wustzdy