Skip to content

feat: add Keenable as a configurable internet search backend - #2072

Open
ilya-bogin-keenable wants to merge 18 commits into
MemTensor:mainfrom
keenableai:feat/keenable-web-search
Open

feat: add Keenable as a configurable internet search backend#2072
ilya-bogin-keenable wants to merge 18 commits into
MemTensor:mainfrom
keenableai:feat/keenable-web-search

Conversation

@ilya-bogin-keenable

Copy link
Copy Markdown

Summary

Adds Keenable as a new internet search backend alongside the existing Bocha / Tavily / Google / Bing / Xinyu retrievers, following the Tavily backend pattern (#1357). Additive and opt-in via INTERNET_SEARCH_BACKEND=keenable; existing backends are untouched.

Keenable is a web search API built for AI agents. Unlike the key-required backends it is keyless by default: with no key it calls the public endpoint (rate-limited), and an optional KEENABLE_API_KEY only lifts the cap.

Files changed

  • src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (new): InternetKeenableRetriever. No SDK dependency, a thin requests call. Keyless requests hit /v1/search/public; a configured key switches to /v1/search with an X-API-Key header. Attribution via X-Keenable-Title. Results map into TextualMemoryItem the same way the Tavily retriever does.
  • src/memos/configs/internet_retriever.py: KeenableSearchConfig (API key optional) + registration in InternetRetrieverConfigFactory.
  • src/memos/memories/textual/tree_text_memory/retrieve/internet_retriever_factory.py: register keenable + constructor branch.
  • src/memos/api/config.py: INTERNET_SEARCH_BACKEND=keenable branch (KEENABLE_API_KEY optional).

Testing

  • python -m py_compile on all changed files: passes.
  • ruff check on all changed files: passes.

Bryunyon and others added 2 commits July 7, 2026 16:16
Add Keenable alongside the existing bocha / tavily / google / bing / xinyu
internet retrievers, following the Tavily backend pattern.

- retrieve/keenablesearch.py: InternetKeenableRetriever. Keyless by default
  (no SDK, a thin requests call): with no key it hits /v1/search/public
  (rate-limited); a key switches to /v1/search with an X-API-Key header.
  Attribution via X-Keenable-Title. Results map into TextualMemoryItem
  exactly like the Tavily retriever.
- configs/internet_retriever.py: KeenableSearchConfig (api_key optional) and
  registration in InternetRetrieverConfigFactory.
- retrieve/internet_retriever_factory.py: register "keenable" + constructor.
- api/config.py: INTERNET_SEARCH_BACKEND=keenable branch (KEENABLE_API_KEY
  optional, keyless by default).

py_compile and ruff pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Memtensor-AI Memtensor-AI added area:memory 记忆存储、检索、更新、召回逻辑 area:api 云服务 / FastAPI / OpenAPI / MCP labels Jul 8, 2026
@Memtensor-AI
Memtensor-AI requested a review from bittergreen July 8, 2026 11:43
@ilya-bogin-keenable

Copy link
Copy Markdown
Author

Hey @bittergreen could you please take a look? Thanks!

@Memtensor-AI Memtensor-AI removed the area:api 云服务 / FastAPI / OpenAPI / MCP label Jul 13, 2026
@Memtensor-AI Memtensor-AI added area:api 云服务 / FastAPI / OpenAPI / MCP area:core MOS 编排层 / 框架底座 / 跨模块问题 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 31, 2026
@Memtensor-AI
Memtensor-AI requested a review from WeiminLee July 31, 2026 19:09
@Memtensor-AI

Memtensor-AI commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2072
Task: 6b2de16b7c284ae6
Base: main
Head: feat/keenable-web-search

🔍 OpenCodeReview found 2 issue(s) in this PR.


1. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L297)

If self.embedder.embed([content]) returns an empty list (e.g., due to an embedder error or an unsupported input), the [0] index access will raise an IndexError. This exception propagates to _convert_to_mem_items where it is caught only by the broad except Exception as e and logged, silently dropping the result with no indication of the root cause.

Add a guard on the return value before indexing:

💡 Suggested Change

Before:

                    embedding=self.embedder.embed([content])[0] if content else [],

After:

                    embedding=self.embedder.embed([content])[0]
                    if content and (embeddings := self.embedder.embed([content]))
                    else [],

2. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L195-L199)

requests.post and resp.raise_for_status() are inside the same broad except Exception block as the JSON parsing. This means a 401/403 (bad API key), a 429 (rate-limited), a network timeout, and a JSON decode error all produce the same silent empty-list return with an identical log message. Callers have no way to distinguish a transient error from a permanent misconfiguration.

Consider at minimum separating the HTTP status check so authentication/rate-limit failures are clearly surfaced:

💡 Suggested Change

Before:

        except Exception:
            import traceback

            logger.error(f"Keenable search error: {traceback.format_exc()}")
            return []

After:

        try:
            resp = requests.post(
                f"{KEENABLE_BASE_URL}{path}",
                json={"query": query, "mode": "pro"},
                headers=headers,
                timeout=self.timeout,
            )
            resp.raise_for_status()
        except requests.HTTPError as e:
            logger.error(f"Keenable HTTP error {e.response.status_code}: {e}")
            return []
        except requests.RequestException:
            import traceback
            logger.error(f"Keenable network error: {traceback.format_exc()}")
            return []
        try:
            raw_results = resp.json().get("results", [])[:limit]
        except Exception:
            logger.error("Keenable: failed to parse JSON response")
            return []

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-33586be3c2b3f89c-20260801031509: 107/107 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 31, 2026
- gate jieba behind require_python_package, like the bocha retriever, so an
  English-only install no longer fails at construction
- skip embedding when the content is empty
- lowercase GDP/AI in the keyword lists; the text is lowercased before matching
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 1, 2026
@Memtensor-AI
Memtensor-AI requested a review from WeiminLee August 1, 2026 05:56
@ilya-bogin-keenable

Copy link
Copy Markdown
Author

Аixed 4 of the 5: jieba is now gated behind require_python_package like the bocha retriever (an English-only install used to fail at construction), empty content is no longer embedded, and GDP/AI are lowercased so they can actually match.

Will not fix for #2. Forwarding the local mode to the API would break the default path: the search endpoint returns 400 for mode: "fast", and fast is the default the caller passes. The API's mode and this method's mode are different things, so I added a comment saying so instead.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-b3c50e54d556e024-20260804132254: 86/87 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added the status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 label Aug 4, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-b90c9e651259700e-20260813203716: 140/140 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 13, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 14, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-504091e23c505b41-20260814164055: 173/174 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 14, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 17, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 5s

Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-cebeff6b3ed746cf-20260818124755: 139/140 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 18, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 19, 2026
Keenable returns both fields on every result: snippet carries the page text
and description is the page's meta description, which is empty for most
pages. Reading description alone stored memories with a title and a URL but
no text.
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-d929796b8f2bd83f-20260819141815: 136/136 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git clone --depth 1 --branch feat/keenable-web-search git@github.com:keenableai/MemOS.git /data/test-workspaces/0bb2eb3c8566c1ed/repo
Cloning into '/data/test-workspaces/0bb2eb3c8566c1ed/repo'...
Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git clone --depth 1 --branch feat/keenable-web-search git@github.com:keenableai/MemOS.git /data/test-workspaces/1a3c8d0e9d69e77a/repo
Cloning into '/data/test-workspaces/1a3c8d0e9d69e77a/repo'...
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Branch: feat/keenable-web-search

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:api 云服务 / FastAPI / OpenAPI / MCP area:core MOS 编排层 / 框架底座 / 跨模块问题 area:memory 记忆存储、检索、更新、召回逻辑 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理

Projects

None yet

Development

Successfully merging this pull request may close these issues.