Describe the bug
OmegaClaw fails to finish indexing Markdown knowledge priors when running under its default Landlock filesystem policy.
Indexing succeeds for several files, then ChromaDB fails during compaction:
chromadb.errors.InternalError: Error in compaction: Failed to apply logs to the metadata segment
Two conditions contribute to the failure:
- The writable Landlock permissions do not include
AccessFs.REFER, which Chroma compaction requires for filesystem reparent/rename operations.
lib_chromadb initializes Chroma with ./chroma_db, while src/rag.py defaults to /PeTTa/chroma_db. ChromaDB 1.5.9 uses the literal persistence path as its shared-system identifier, so these paths create separate Chroma systems even though they resolve to the same directory. The RAG system is then initialized after Landlock has been applied.
To Reproduce
Steps to reproduce the behavior:
-
Add enough Markdown files to OmegaClaw-Core/knowledge-priors to trigger Chroma compaction. The reproduced case used 33 files totalling approximately 330 KB.
-
Start OmegaClaw with:
- Local embeddings
- The default
profile/policy.yaml
- Persistent Chroma memory at
/PeTTa/chroma_db
-
Follow the container logs:
docker logs --follow omegaclaw 2>&1 |
grep --line-buffered -E 'Embedding type selected|\.md: (indexed|unchanged)|Knowledge(:| init failed)'
-
Observe that several files are indexed before knowledge initialization fails.
Example output:
Embedding type selected is Local
file1.md: indexed 20 chunks
file2.md: indexed 3 chunks
file3.md: indexed 14 chunks
...
Knowledge init failed: Error in compaction: Failed to apply logs to the metadata segment
The underlying traceback ends at collection.upsert():
Traceback (most recent call last):
File "/PeTTa/repos/OmegaClaw-Core/src/rag.py", line 276, in init_knowledge
collection.upsert(
File "/usr/local/lib/python3.11/dist-packages/chromadb/api/models/Collection.py", line 530, in upsert
self._client._upsert(
File "/usr/local/lib/python3.11/dist-packages/chromadb/api/rust.py", line 541, in _upsert
return self.bindings.upsert(
chromadb.errors.InternalError: Error in compaction: Failed to apply logs to the metadata segment
Expected behavior
OmegaClaw should index all available knowledge files and report successful completion:
Knowledge: 33 files (0 unchanged, 33 re-indexed)
Knowledge indexing should work with the default filesystem security policy and persistent memory configuration.
Actual behavior
Knowledge initialization stops partway through indexing with:
Knowledge init failed: Error in compaction: Failed to apply logs to the metadata segment
The Chroma database remains only partially populated. Restarting the container may index additional files, but compaction fails again before initialization completes.
Desktop (please complete the following information):
- OS: N/A — reproduced in an Ubuntu Linux Docker deployment
- Browser: N/A
- Version: N/A
Smartphone (please complete the following information):
- Device: N/A
- OS: N/A
- Browser: N/A
- Version: N/A
Additional context
- OmegaClaw version:
v0.1.15
- ChromaDB version:
1.5.9
- py-landlock version:
0.1.1
- Communication channel: WebSocket
- Embedding provider: Local
- Model provider: ASICloud
The Markdown files themselves are not the cause. Indexing the same 33 files into a fresh Chroma database without the Landlock policy succeeds.
The following two changes were tested together and resolved the issue:
-
Add AccessFs.REFER to FileSystemPolicy.READ_WRITE_DIR_ACCESS:
READ_WRITE_DIR_ACCESS = (
AccessFs.READ_FILE
| AccessFs.READ_DIR
| AccessFs.WRITE_FILE
| AccessFs.TRUNCATE
| AccessFs.MAKE_REG
| AccessFs.MAKE_DIR
| AccessFs.MAKE_SYM
| AccessFs.REMOVE_FILE
| AccessFs.REMOVE_DIR
| AccessFs.MAKE_FIFO
| AccessFs.MAKE_SOCK
| AccessFs.REFER
)
-
Make src/rag.py use the same default persistence-path identifier as lib_chromadb:
DB_PATH = os.environ.get("CHROMA_DB_PATH", "./chroma_db")
After applying both changes, all 33 files indexed successfully, knowledge retrieval returned the expected content, and the container remained running with zero restarts.
Describe the bug
OmegaClaw fails to finish indexing Markdown knowledge priors when running under its default Landlock filesystem policy.
Indexing succeeds for several files, then ChromaDB fails during compaction:
Two conditions contribute to the failure:
AccessFs.REFER, which Chroma compaction requires for filesystem reparent/rename operations.lib_chromadbinitializes Chroma with./chroma_db, whilesrc/rag.pydefaults to/PeTTa/chroma_db. ChromaDB 1.5.9 uses the literal persistence path as its shared-system identifier, so these paths create separate Chroma systems even though they resolve to the same directory. The RAG system is then initialized after Landlock has been applied.To Reproduce
Steps to reproduce the behavior:
Add enough Markdown files to
OmegaClaw-Core/knowledge-priorsto trigger Chroma compaction. The reproduced case used 33 files totalling approximately 330 KB.Start OmegaClaw with:
profile/policy.yaml/PeTTa/chroma_dbFollow the container logs:
Observe that several files are indexed before knowledge initialization fails.
Example output:
The underlying traceback ends at
collection.upsert():Expected behavior
OmegaClaw should index all available knowledge files and report successful completion:
Knowledge indexing should work with the default filesystem security policy and persistent memory configuration.
Actual behavior
Knowledge initialization stops partway through indexing with:
The Chroma database remains only partially populated. Restarting the container may index additional files, but compaction fails again before initialization completes.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
v0.1.151.5.90.1.1The Markdown files themselves are not the cause. Indexing the same 33 files into a fresh Chroma database without the Landlock policy succeeds.
The following two changes were tested together and resolved the issue:
Add
AccessFs.REFERtoFileSystemPolicy.READ_WRITE_DIR_ACCESS:Make
src/rag.pyuse the same default persistence-path identifier aslib_chromadb:After applying both changes, all 33 files indexed successfully, knowledge retrieval returned the expected content, and the container remained running with zero restarts.