Skip to content

feat(altk_evolve): entity sharing — public/private visibility#201

Merged
visahak merged 18 commits intoAgentToolkit:mainfrom
visahak:feature/entity-sharing-full-mcp
Apr 28, 2026
Merged

feat(altk_evolve): entity sharing — public/private visibility#201
visahak merged 18 commits intoAgentToolkit:mainfrom
visahak:feature/entity-sharing-full-mcp

Conversation

@visahak
Copy link
Copy Markdown
Collaborator

@visahak visahak commented Apr 19, 2026

Addresses partly issue #182 in the mcp full version.

Summary

  • Adds public/private visibility to entities via metadata fields (owner_id, visibility, published_at) — no schema migration required, all backends unaffected
  • New publish_entity and unpublish_entity MCP tools to control visibility per entity
  • get_entities gains include_public=True to merge public entities from all namespaces, annotated with [public: {owner_id}]
  • create_entity gains owner_id and visibility params; save_trajectory gains owner_id
  • New get_public_entities, get_entity_by_id, patch_entity_metadata on EvolveClient
  • Concrete patch_entity on BaseEntityBackend (+ filesystem override) enables metadata-only patches without changing entity ID or content

Unit Tests (291 total)

File Tests What's covered
test_entity_sharing.py 12 create_entity, publish_entity, unpublish_entity, get_entities(include_public) — all mocked
test_entity_sharing_e2e.py 10 publish/unpublish round-trips, cross-namespace discovery, patch_entity_metadata persistence, type filtering — real FilesystemEntityBackend
test_postgres_backend.py 3 Native UPDATE ... metadata || jsonb, missing entity error, non-numeric ID rejection
test_milvus_backend.py 3 Native milvus.query fetch + upsert merge, missing entity error, non-numeric ID rejection
(pre-existing) 263 All other backend, client, MCP, and pipeline tests

E2E Tests — 30 total (--run-e2e)

File Tests Backends What's covered
test_sharing.py 5 × 2 = 10 filesystem + milvus publish/unpublish via MCP, create_entity(visibility=public), content preservation after metadata patch, not-found error handling
test_mcp.py 10 × 2 = 20 filesystem + milvus Pre-existing MCP tool tests, now parameterized against both backends

Summary by CodeRabbit

  • New Features

    • Entity sharing: public/private visibility, owner and published_at metadata; publish/unpublish operations; create entities with visibility/owner; cross-namespace public listing (include_public).
    • Client APIs: fetch single entity by id, patch entity metadata, and list public entities.
    • MCP tools: create_entity accepts visibility/owner, save_trajectory accepts owner, get_entities supports include_public.
  • Documentation

    • Added sharing docs, examples, and a note about deferred REST/UI work.
  • Tests

    • New unit and e2e tests covering sharing, metadata patching, public discovery, and backend updates.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 19, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds metadata-driven entity sharing (owner_id, visibility, published_at), backend helpers to patch/merge entity metadata, client APIs for public entity discovery and metadata patching, MCP tools to create/publish/unpublish entities, README updates, and new unit and e2e tests across filesystem, Milvus, and Postgres backends.

Changes

Cohort / File(s) Summary
Documentation
README.md
Documented entity sharing: owner_id, visibility, published_at; updated MCP tool signatures (get_entities, create_entity, save_trajectory) and added publish_entity / unpublish_entity usage and examples.
Schema docs
altk_evolve/schema/core.py
Extended Entity docstring to describe optional sharing metadata (owner_id, visibility, published_at).
Backend core
altk_evolve/backend/base.py
Added patch_entity(...) and update_entity_metadata(...) public helpers for in-place metadata merge and RecordedEntity return.
Backend implementations
altk_evolve/backend/filesystem.py, altk_evolve/backend/milvus.py, altk_evolve/backend/postgres.py
Filesystem: implemented patch_entity(...) under lock. Milvus/Postgres: added update_entity_metadata(...) with numeric id validation, metadata merge, persistence (upsert / UPDATE ... RETURNING), post-update flush, and not-found handling.
Client
altk_evolve/frontend/client/evolve_client.py
Added get_entity_by_id, patch_entity_metadata (delegates to backend), and get_public_entities which aggregates public entities across namespaces with optional type filtering and global limit.
MCP server & tools
altk_evolve/frontend/mcp/mcp_server.py
Extended get_entities with include_public and deduplication + public-owner annotation; expanded create_entity to accept/validate visibility and owner_id; added publish_entity and unpublish_entity; updated save_trajectory to accept owner_id.
E2E fixtures & tests
tests/e2e/conftest.py, tests/e2e/test_mcp.py, tests/e2e/test_sharing.py
Moved/expanded mcp fixture into conftest.py to parametrize filesystem and milvus, centralized setup/teardown; removed duplicate fixture; added async e2e sharing tests exercising create/publish/unpublish and cross-namespace discovery.
Unit tests
tests/unit/test_entity_sharing.py, tests/unit/test_entity_sharing_e2e.py, tests/unit/test_milvus_backend.py, tests/unit/test_postgres_backend.py
Added unit tests for MCP sharing logic (mocked), filesystem-backed e2e tests, and backend-specific tests validating update_entity_metadata behavior, numeric id validation, JSONB merge SQL, and not-found error handling.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant MCP_Server
    participant Backend
    participant DB

    Client->>MCP_Server: publish_entity(entity_id, user_id)
    MCP_Server->>Backend: get_entity_by_id(namespace, entity_id)
    Backend-->>MCP_Server: RecordedEntity or None
    alt entity found
        MCP_Server->>Backend: patch_entity(namespace, entity_id, metadata_updates with visibility/published_at)
        Backend->>DB: load entity / validate existence
        DB-->>Backend: entity row
        Backend->>DB: update entity (merge metadata / upsert)
        DB-->>Backend: updated row
        Backend-->>MCP_Server: RecordedEntity (merged metadata)
        MCP_Server-->>Client: JSON response (updated entity)
    else not found
        MCP_Server-->>Client: JSON error
    end
Loading
sequenceDiagram
    participant Client
    participant EvolveClient
    participant NamespaceService
    participant StorageBackends

    Client->>EvolveClient: get_public_entities(query,type,limit)
    EvolveClient->>NamespaceService: search_namespaces()
    NamespaceService-->>EvolveClient: list(namespaces)
    loop per-namespace
        EvolveClient->>StorageBackends: search_entities(filters={'metadata.visibility':'public', ...}, per_ns_limit)
        StorageBackends-->>EvolveClient: list[RecordedEntity]
    end
    EvolveClient-->>Client: aggregated, deduplicated public results
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

  • vinodmut
  • illeatmyhat

Poem

🐰 I hopped through metadata fields so spry,
Owner and visibility waving hi.
Publish a carrot, unpublish a seed,
Shared tips hop out for those in need.
I nibble bugs — now go test and try!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 48.53% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: introducing public/private visibility for entities in altk_evolve, which is the core feature implemented across all files in this PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@visahak visahak force-pushed the feature/entity-sharing-full-mcp branch from df42b97 to 4876121 Compare April 21, 2026 20:34
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
altk_evolve/frontend/mcp/mcp_server.py (1)

274-287: ⚠️ Potential issue | 🟡 Minor

Validate that metadata parses to a JSON object before mutating it.

A valid JSON value like [] or "x" reaches Line 285 and then raises because it is not a dict. Return a structured validation error instead of falling through to the generic server-error path.

Suggested fix
         metadata_dict = {}
         if metadata:
             try:
-                metadata_dict = json.loads(metadata)
+                parsed_metadata = json.loads(metadata)
             except json.JSONDecodeError as e:
                 logger.exception(f"Invalid JSON in metadata parameter: {str(e)}")
                 return json.dumps({"error": "Invalid JSON", "message": f"Failed to parse metadata: {str(e)}", "invalid_metadata": metadata})
+            if not isinstance(parsed_metadata, dict):
+                return json.dumps({"error": "Invalid metadata", "message": "metadata must be a JSON object"})
+            metadata_dict = parsed_metadata
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@altk_evolve/frontend/mcp/mcp_server.py` around lines 274 - 287, The parsed
metadata from json.loads(metadata) must be validated as a JSON object (dict)
before mutating it; in the block around json.loads in mcp_server.py (the
metadata_dict variable and subsequent mutations:
metadata_dict.setdefault("creation_mode", ...), metadata_dict["visibility"],
metadata_dict["owner_id"]), check isinstance(metadata_dict, dict) after parsing
and if it's not a dict return a structured JSON error (e.g., {"error":"Invalid
metadata type","message":"metadata must be a JSON object","invalid_metadata":
metadata}) instead of proceeding, so non-object JSON like arrays or strings are
rejected with a clear response rather than causing later exceptions.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@altk_evolve/backend/base.py`:
- Around line 91-107: The update_entity_metadata implementation uses a new
current timestamp when calling patch_entity, which causes created_at to be
overwritten for metadata-only updates; change update_entity_metadata to reuse
the existing entity timestamp (e.g., entity.created_at or
entity.get("created_at")) instead of computing now, pass that timestamp into
patch_entity, and return RecordedEntity with the merged metadata and the
original timestamp so filesystem/backends won't persist a new created_at; locate
this logic in update_entity_metadata, search_entities result handling, the call
to patch_entity, and the RecordedEntity construction to make the change.

In `@altk_evolve/backend/filesystem.py`:
- Around line 173-176: Wrap the update sequence inside the critical section in a
try/finally so that if any exception occurs after loading namespace data the
instance attribute _active_data (set by _load_namespace_data(namespace_id)) is
cleared in the finally block to avoid stale data; additionally, validate
entity_id before calling _update_entity(namespace_id, entity_id, entity_type,
content_str, timestamp, metadata) and raise a clear exception (e.g., ValueError)
if entity_id is missing/invalid so the operation does not silently no-op; keep
the calls to _load_namespace_data, _update_entity, and _post_update unchanged
but ensure exceptions propagate after cleaning up.

In `@altk_evolve/backend/milvus.py`:
- Around line 307-313: When applying a metadata-only patch in the block that
parses the existing Milvus entity (parse_milvus_entity) and then calls
_update_entity, preserve the original entity creation timestamp instead of
generating a fresh timestamp: detect that only metadata is changing, fetch the
existing timestamp/created_at from the parsed entity (e.g., entity.created_at or
entity.timestamp), pass that original timestamp into _update_entity (instead of
the new int(datetime...timestamp() value) currently assigned to timestamp), and
ensure the returned RecordedEntity constructed via
RecordedEntity(**{**entity.model_dump(), "metadata": merged}) retains the
original created_at value.

In `@altk_evolve/frontend/mcp/mcp_server.py`:
- Around line 327-334: The patch currently sets "owner_id": user_id
unconditionally which will write null when user_id is absent; update the call to
get_client().patch_entity_metadata (the metadata_updates passed for entity_id)
so that "owner_id" is only added to the metadata_updates dict when user_id is
present (i.e., check user_id is not None before inserting owner_id), while still
always setting "visibility" to "public" and "published_at" to
datetime.now(UTC).isoformat().

In `@README.md`:
- Around line 152-172: The fenced code blocks in the README examples lack
language labels (MD040); update each fence to use a language tag `python` for
the examples: modify the blocks containing publish_entity(entity_id="42",
user_id="alice"), unpublish_entity(entity_id="42"), get_entities(task="write
safer code", include_public=True), and create_entity(content="...",
entity_type="guideline", visibility="public", owner_id="alice") so their opening
triple-backticks are changed to ```python to satisfy markdownlint.

In `@tests/e2e/conftest.py`:
- Around line 8-26: The mcp fixture mutates os.environ and
milvus_client_settings.uri without restoring them; capture original values for
EVOLVE_NAMESPACE_ID, EVOLVE_BACKEND, EVOLVE_SQLITE_PATH and EVOLVE_DATA_DIR (if
set) and original_milvus_uri before modifying, then register a teardown via
request.addfinalizer (or a try/finally around operations) that restores
milvus_client_settings.uri to original_milvus_uri (or None) and resets each
EVOLVE_* key to its previous value or deletes the key if it didn’t exist; also
ensure any milvus_db_file created is not left referenced in env after teardown.
- Around line 35-48: The teardown currently only closes the local setup
EvolveClient instance (evolve_client) leaving the actual MCP singleton client
(mcp_server_module._client created via get_client()) open; update the teardown
to locate and close the real singleton backend by calling get_client() or
checking mcp_server_module._client and invoking its backend.close() (with safe
try/except), so the MCP/Milvus Lite resources are properly released in tests.

In `@tests/unit/test_entity_sharing_e2e.py`:
- Around line 1-14: The tests are end-to-end but are incorrectly marked as unit
via the pytestmark variable; update the module-level marker from pytestmark =
pytest.mark.unit to pytestmark = pytest.mark.e2e (or remove and add pytestmark =
pytest.mark.e2e) so the file is classified as e2e tests; alternatively, if you
prefer path-based organization, move the file into the e2e test directory and
ensure the module-level pytestmark uses pytest.mark.e2e (refer to the pytestmark
symbol in this module).

---

Outside diff comments:
In `@altk_evolve/frontend/mcp/mcp_server.py`:
- Around line 274-287: The parsed metadata from json.loads(metadata) must be
validated as a JSON object (dict) before mutating it; in the block around
json.loads in mcp_server.py (the metadata_dict variable and subsequent
mutations: metadata_dict.setdefault("creation_mode", ...),
metadata_dict["visibility"], metadata_dict["owner_id"]), check
isinstance(metadata_dict, dict) after parsing and if it's not a dict return a
structured JSON error (e.g., {"error":"Invalid metadata
type","message":"metadata must be a JSON object","invalid_metadata": metadata})
instead of proceeding, so non-object JSON like arrays or strings are rejected
with a clear response rather than causing later exceptions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 182c2944-da14-4c08-b9e6-60e972a41985

📥 Commits

Reviewing files that changed from the base of the PR and between 6f79732 and 4876121.

📒 Files selected for processing (15)
  • README.md
  • altk_evolve/backend/base.py
  • altk_evolve/backend/filesystem.py
  • altk_evolve/backend/milvus.py
  • altk_evolve/backend/postgres.py
  • altk_evolve/frontend/client/evolve_client.py
  • altk_evolve/frontend/mcp/mcp_server.py
  • altk_evolve/schema/core.py
  • tests/e2e/conftest.py
  • tests/e2e/test_mcp.py
  • tests/e2e/test_sharing.py
  • tests/unit/test_entity_sharing.py
  • tests/unit/test_entity_sharing_e2e.py
  • tests/unit/test_milvus_backend.py
  • tests/unit/test_postgres_backend.py
💤 Files with no reviewable changes (1)
  • tests/e2e/test_mcp.py

Comment thread altk_evolve/backend/base.py
Comment thread altk_evolve/backend/filesystem.py Outdated
Comment thread altk_evolve/backend/milvus.py
Comment thread altk_evolve/frontend/client/evolve_client.py Outdated
Comment thread altk_evolve/frontend/mcp/mcp_server.py Outdated
Comment thread README.md Outdated
Comment thread tests/e2e/conftest.py
Comment thread tests/e2e/conftest.py Outdated
Comment thread tests/unit/test_entity_sharing_e2e.py Outdated
@visahak
Copy link
Copy Markdown
Collaborator Author

visahak commented Apr 21, 2026

This is ready for review. It applies the same sharing technique on the full mcp version. Separate from claude, bob, and codex versions.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@visahak visahak force-pushed the feature/entity-sharing-full-mcp branch from 9ecffbb to f97adfb Compare April 22, 2026 16:32
coderabbitai[bot]

This comment was marked as resolved.

@visahak
Copy link
Copy Markdown
Collaborator Author

visahak commented Apr 22, 2026

Tested with the cuga agent and postgres. ready for review.

coderabbitai[bot]

This comment was marked as resolved.

@visahak visahak force-pushed the feature/entity-sharing-full-mcp branch from 1022d43 to f95b2f3 Compare April 27, 2026 13:44
visahak and others added 16 commits April 27, 2026 09:47
- Fixed test_uninstall_prunes_evolve_only_hook_groups by adding empty group pruning logic to _remove_user_prompt_hook in install.sh
- Fixed test_skips_symlinked_entities by correcting test expectations - symlinks exist in git clones but are filtered by retrieve script
- Added RETRIEVE_SCRIPT constant to test_sync.py for cleaner test code
- All 230 platform integration tests now pass
…ng tests

Fixes failing CI check: check-formatting (3.12)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Unit tests (postgres + milvus):
- Verifies native metadata merge SQL / upsert path
- Missing entity raises EvolveException
- Non-numeric ID rejected early

E2E tests (filesystem + milvus, via --run-e2e):
- publish_entity / unpublish_entity full MCP flow
- create_entity with visibility=public
- Content preserved after metadata patch
- Extracted shared mcp fixture into tests/e2e/conftest.py
  and extended params to include milvus
@visahak visahak force-pushed the feature/entity-sharing-full-mcp branch from f95b2f3 to e7aaa5c Compare April 27, 2026 13:52
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Vatche Isahagian <vatchei@ibm.com>
@visahak
Copy link
Copy Markdown
Collaborator Author

visahak commented Apr 27, 2026

Rebased and resolved conflicts. This PR is ready for review.

@gaodan-fang
Copy link
Copy Markdown
Collaborator

@visahak verified this works by running the mcp and test it with two different namespaced mcp servers, bob create entity, and publish it, alice can get the public entity, bob unpublish it, alice can't get bob's private entity

Copy link
Copy Markdown
Collaborator

@gaodan-fang gaodan-fang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@visahak visahak merged commit 42316f3 into AgentToolkit:main Apr 28, 2026
16 checks passed
@visahak visahak deleted the feature/entity-sharing-full-mcp branch April 28, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants