Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
1290859
feat(memory): centralize persistent memory access
Bereket-Eshete Aug 17, 2026
b034451
feat(memory): add portable archive import and export
Bereket-Eshete Aug 17, 2026
ed844a6
feat(launcher): add memory transfer startup options
Bereket-Eshete Aug 17, 2026
ba46ca7
feat(channels): add authenticated memory export controls
Bereket-Eshete Aug 17, 2026
813bfcf
docs(memory): document portability workflow
Bereket-Eshete Aug 17, 2026
2a62fcd
build(docker): install memory portability package
Bereket-Eshete Aug 19, 2026
c0e8f1a
refactor(memory): use portability package
Bereket-Eshete Aug 19, 2026
333379e
refactor(channels): centralize memory export dispatch
Bereket-Eshete Aug 19, 2026
10c7ea1
build(memory): use unified portability package
Bereket-Eshete Aug 20, 2026
71ed9e4
refactor(memory-export): process exports in the main loop
Bereket-Eshete Aug 20, 2026
e34fc7f
refactor(memory): remove obsolete gateway wrapper
Bereket-Eshete Aug 20, 2026
0e79d9e
fix(memory-export): isolate completion delivery failures
Bereket-Eshete Aug 20, 2026
d8bb83e
build(memory): Install the tagged v0.2.1 standalone package
Bereket-Eshete Aug 20, 2026
e81329d
feat: integrate memory portability package
Bereket-Eshete Aug 20, 2026
69ac2c8
test: cover synchronous memory export handling
Bereket-Eshete Aug 20, 2026
26819ee
docs: clarify memory portability operations
Bereket-Eshete Aug 20, 2026
d4d4123
fix: notify WebSocket users about unsupported memory export
Bereket-Eshete Aug 21, 2026
5cdc8b9
fix: preserve embedding provider for memory export
Bereket-Eshete Aug 21, 2026
88b2b3c
fix: create missing memory transfer directories
Bereket-Eshete Aug 21, 2026
7e402ad
docs: remove duplicate python bridges reference
Bereket-Eshete Aug 21, 2026
f35efa9
Merge pull request #63 from Bereket-Eshete/feat/user-memory-portability
blackhammer116 Aug 21, 2026
b6ef821
Fix: embedding provider now visible to memory_portability package and…
blackhammer116 Aug 21, 2026
eb1a46e
Merge remote-tracking branch 'core/main' into dev/new-feature
blackhammer116 Aug 21, 2026
8f563ca
Fix: lazily load memory portability for export
blackhammer116 Aug 21, 2026
e830bd9
chore: updated README to include information regarding import memory …
blackhammer116 Aug 22, 2026
a5e47d2
Fix: moved export handler to commChannelReceive in channels.py.
blackhammer116 Aug 22, 2026
89021a8
chore: moved memory_export adapter into src dir
blackhammer116 Aug 22, 2026
cde5828
Merge remote-tracking branch 'core/main' into dev/new-feature
blackhammer116 Aug 22, 2026
d5c6239
chore: updated version for documentation
blackhammer116 Aug 24, 2026
f2a26c6
Feat: enabled support for wsocket channel export and removed redundan…
blackhammer116 Aug 25, 2026
a3f0943
chore: updated the readme
blackhammer116 Aug 25, 2026
e4da3e9
Merge remote-tracking branch 'core/main' into dev/new-feature
blackhammer116 Aug 25, 2026
d862554
chore: updated the import-kb version to the latest patch
blackhammer116 Aug 25, 2026
acf9230
Feat: paths are now implemented as plugin config
blackhammer116 Aug 31, 2026
4a3f1ba
Fix: included version number during export and removed redundant laun…
blackhammer116 Sep 3, 2026
b4d359c
Merge remote-tracking branch 'core/main' into dev/new-feature
blackhammer116 Sep 3, 2026
60fd8a0
chore: renamed OMEGACLAW_VERSION to OMEGA_VERSION
blackhammer116 Sep 3, 2026
7aa0e1e
Merge remote-tracking branch 'core/main' into dev/new-feature
blackhammer116 Sep 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,14 @@ COPY --from=versioned-source /omega-source ${OMEGA_DIR}

RUN cp ${OMEGA_DIR}/run.metta /PeTTa/run.metta \
&& mkdir -p ${MEMORY_DIR}/chroma_db \
&& mkdir -p /memory-transfer \
&& ln -s ${MEMORY_DIR}/chroma_db ./chroma_db \
&& chmod +x ${OMEGA_DIR}/entrypoint.sh \
&& chmod +x ${OMEGA_DIR}/scripts/import_knowledge.sh \
&& chmod +x ${OMEGA_DIR}/scripts/omega \
&& chown -R 65534:65534 ${MEMORY_DIR} \
&& chown 65534:65534 /memory-transfer \
&& chmod 0700 /memory-transfer \
&& find ${MEMORY_DIR} -type f -exec chmod 0644 {} \; \
&& chmod 0444 ${MEMORY_DIR}/prompt.txt \
&& chown -R 65534:65534 /opt/huggingface /opt/sentence_transformers
Expand Down
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,25 @@ To reset Omega's memory:
docker volume rm omega-memory
```

### Memory portability

Memory export is disabled by default. See the [memory portability reference](./docs/reference-memory-portability.md)
for setup, export controls, archive contents, and import modes.

> **Current limitation:** Memory import does not work in a standalone Omega
> run. Import and interrupted-import recovery are supported only through Docker
> using `scripts/omega`, because both operations run from the container
> entrypoint before the agent loop starts.

To restore an archive while upgrading to a tagged image, use the same transfer directory:

```sh
scripts/omega start -d singularitynet/omega:<tag> -p OpenAI -t telegram \
--memory-transfer-dir "$HOME/omega-transfers" \
--memory-import omegaclaw-memory-<timestamp>.tar.gz \
--memory-mode overwrite
```

---

## Usage
Expand Down
8 changes: 6 additions & 2 deletions config/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ wakeupInterval: 600
# Path to the logger configuration file
# See https://docs.python.org/3/library/logging.config.html#configuration-file-format
logConfigPath: ""
# (internal) Path to the memory directory
#memoryDirectory: "./"
# Directory containing persistent memory files such as history.metta.
memoryDirectory: "./repos/Omega/memory"

# Memory

Expand All @@ -34,8 +34,12 @@ maxRecallItems: 20
maxEpisodeRecallLines: 20
# Tail of `memory/history.metta` included in the prompt (chars)
maxHistory: 30000
# ChromaDB persistence directory used for long-term memory portability.
chromaDbPath: "./chroma_db"
# `Local` (Python-side model) or `OpenAI` (requires `OPENAI_API_KEY`)
embeddingprovider: Local
# Enable authenticated operator-triggered /memory-export commands (disabled by default).
memoryExportEnabled: false

# Policy

Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ User-facing MeTTa skills the agent invokes. Each page follows the template **Sig
- [reference-configuration.md](./reference-configuration.md) — `configure` form and all runtime parameters
- [reference-channels.md](./reference-channels.md) — IRC, Telegram, Slack, Mattermost, WebSocket, and websearch adapters plus the channel contract
- [reference-python-bridges.md](./reference-python-bridges.md) — `lib_llm_ext.py`, `src/helper.py`, `src/skills.pl`
- [reference-memory-portability.md](./reference-memory-portability.md) — Operator backup, restore, and archive-transfer workflow

### Plugin API

Expand Down
1 change: 1 addition & 0 deletions docs/reference-channels.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ Minimal JSON chat adapter over a WebSocket connection. Selected with `commchanne
- `stop_websocket()` — stop the listener thread and close the socket.
- Requires the `websockets` Python package.
- When `WS_TOKEN` is set it is sent as an `Authorization: Bearer <token>` header. Unlike the IRC/Telegram/Slack adapters there is no one-time `auth <secret>` gate — trust is established by the endpoint URL and bearer token.
- Supports immediate `/memory-export history|ltm|both` commands when memory export is enabled and `WS_TOKEN` is configured. The export handler uses a SHA-256-derived connection principal and never exposes the bearer token. Protect the endpoint with `wss://` and server-side access controls because WebSocket does not have the per-user ownership gate used by the other channels.
- Reconnects automatically with exponential backoff (1s → 30s, ±20% jitter) and is safe to start once at process startup.

### Frame protocol
Expand Down
4 changes: 3 additions & 1 deletion docs/reference-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ This reads a command-line override via `argk` (`name=value` on the MeTTa command
| `maxRecallItems` | 20 | Items returned by `query`. |
| `maxEpisodeRecallLines` | 20 | Lines returned by `episodes`. |
| `maxHistory` | 30000 (chars) | Tail of `memory/history.metta` included in the prompt. |
| `memoryDirectory` | `./repos/Omega/memory` | Directory containing persistent memory files such as `history.metta`. |
| `chromaDbPath` | `./chroma_db` | ChromaDB persistence directory used for memory backup and restore. |
| `embeddingprovider` | `Local` | `Local` (Python-side model) or `OpenAI`. |

## Channels (`src/channels.metta`, `initChannels`)
Expand Down Expand Up @@ -68,7 +70,7 @@ metta run.metta provider=Anthropic LLM=claude-opus-4-6 commchannel=mattermost
```

Configuration values are resolved in this order: command-line `key=value`,
`OMEGACLAW_<KEY>` environment variable, `config/config.yaml`, then the caller's
`OMEGA_<KEY>` environment variable, `config/config.yaml`, then the caller's
default. `TG_BOT_TOKEN` and `OMEGA_AUTH_SECRET` are read directly from the
environment and must be placed before the `metta`/`petta` command.

Expand Down
78 changes: 78 additions & 0 deletions docs/reference-memory-portability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Reference - Memory Portability

Memory portability lets an operator export persistent user memory from one
deployment and restore it before another agent starts. It is an operator
workflow, not an LLM skill.

## Setup

Choose an absolute host directory for archives. The launcher creates it when
needed and mounts it at the fixed container path `/memory-transfer`; the agent
never accepts arbitrary runtime export paths.

```sh
scripts/omega start -p OpenAI -t telegram \
--memory-transfer-dir "$HOME/omega-transfers" \
--enable-memory-export
```

`--enable-memory-export` is required because export is disabled by default.
The transfer directory must be writable by the container's agent user.

## Export

In the active chat, request one component. The export runs immediately. For IRC,
Telegram, Slack, and Mattermost, the export handler requires the authenticated
user ID persisted by the channel authorization layer. WebSocket export requires
a configured `WS_TOKEN`; the handler derives a non-reversible principal from the
token so the credential itself is never used as an identifier:

```text
/memory-export history
/memory-export ltm
/memory-export both
```

Completion is delivered through the active channel and includes the filename,
record count, size, and SHA-256.

Archives contain selected persistent user memory only:

```text
manifest.json
history/history.metta
vector/collections.json
vector/records.jsonl
```

History is the conversation trace. LTM is logical user-memory records from
ChromaDB. Prompts, credentials, logs, skills, and other operational state are
not exported. SHA-256 detects corruption, not archive authorship.

## Import

Import is an administrative startup operation. The archive argument is a plain
filename in the chosen transfer directory:

```sh
scripts/omega start -d singularitynet/omega:<tag> -p OpenAI -t telegram \
--memory-transfer-dir "$HOME/omega-transfers" \
--memory-import omegaclaw-memory-<timestamp>.tar.gz \
--memory-mode overwrite
```

`overwrite` replaces the selected components after validation and rollback
preparation. `append` preserves existing history and adds imported LTM records
under new IDs. Select a single component with `--only-history` or
`--only-vector`. Without either option, both components are imported.

The importer validates archive paths, checksums, manifest metadata, record
counts, and embedding compatibility before changing live memory. It runs before
the agent loop starts. A receipt prevents a completed archive import from
running again on container restart.

## Security

Archives are private operator data; keep the host transfer directory protected.
Memory export is denied when channel authentication is disabled, no authenticated
channel user has been persisted, or WebSocket has no `WS_TOKEN`.
37 changes: 36 additions & 1 deletion entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,46 @@ if [[ "${IMPORT_KB_ON_START}" == "1" ]]; then
su nobody -s /bin/sh -c "${OMEGA_DIR}/scripts/import_knowledge.sh"
fi

MEMORY_PORTABILITY_PYTHON='import os
from config import init_config
from memory_export import create_memory_store
from memory_portability import MemoryTransfer

init_config([])
transfer = MemoryTransfer(store=create_memory_store())
operation = os.environ["MEMORY_PORTABILITY_OPERATION"]
if operation == "recover":
transfer.recover()
elif operation == "import":
transfer.import_archive(
os.environ["MEMORY_IMPORT_FILE"],
mode=os.environ.get("MEMORY_IMPORT_MODE", "overwrite"),
include_history=os.environ.get("MEMORY_IMPORT_NO_HISTORY") != "1",
include_vectors=os.environ.get("MEMORY_IMPORT_NO_VECTOR") != "1",
)
else:
raise ValueError(f"Unsupported memory portability operation: {operation!r}")'
export MEMORY_PORTABILITY_PYTHON
export PYTHONPATH="${OMEGA_DIR}:${OMEGA_DIR}/src${PYTHONPATH:+:${PYTHONPATH}}"

export MEMORY_PORTABILITY_OPERATION=recover
su nobody -s /bin/sh -c 'exec python3 -c "$MEMORY_PORTABILITY_PYTHON"' \
|| { echo "Memory import recovery failed. Aborting startup." >&2; exit 1; }

if [[ -n "${MEMORY_IMPORT_FILE:-}" ]]; then
echo "memory_portability: importing ${MEMORY_IMPORT_FILE}"
export MEMORY_PORTABILITY_OPERATION=import
su nobody -s /bin/sh -c 'exec python3 -c "$MEMORY_PORTABILITY_PYTHON"' \
|| { echo "Memory import failed. Aborting startup." >&2; exit 1; }
echo "memory_portability: import complete"
fi
unset MEMORY_PORTABILITY_OPERATION MEMORY_PORTABILITY_PYTHON PYTHONPATH

# Scrub environment: only allowlisted vars survive.
SAFE_VARS="HOME USER PATH HOSTNAME TERM LANG LC_ALL \
PYTHONDONTWRITEBYTECODE PYTHONUNBUFFERED \
HF_HOME SENTENCE_TRANSFORMERS_HOME HF_HUB_OFFLINE TRANSFORMERS_OFFLINE \
OMEGA_DIR MEMORY_DIR TEST_SERVER_IP"
CHROMA_DB_PATH EMBEDDING_PROVIDER OMEGA_DIR MEMORY_DIR TEST_SERVER_IP"

env_args=""
for var in $SAFE_VARS; do
Expand Down
1 change: 1 addition & 0 deletions profile/policy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ filesystem_policy:
- /PeTTa
read_write:
- /PeTTa/repos/Omega/memory
- /memory-transfer
- /tmp
- /dev/null
- /opt/huggingface
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ chromadb==1.5.9
openai==2.38.0
transformers==5.8.0
sentence-transformers==5.5.1
import-kb==0.1.8
import-kb==0.2.1
py-landlock==0.1.1
pyyaml==6.0.3
ddgs==9.14.4
Expand Down
Loading
Loading