Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,19 @@ on:
pull_request:

jobs:
typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.12"
- name: Install (with dev and type-check extras)
run: uv pip install -e ".[dev,typecheck]"
- name: Type check
run: uv run --no-sync mypy

test:
runs-on: ubuntu-latest
strategy:
Expand Down
44 changes: 29 additions & 15 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ curl -LsSf https://astral.sh/uv/install.sh | sh
# 3. Create a virtual environment and install dev dependencies
uv venv
source .venv/bin/activate # Windows: .venv\Scripts\Activate.ps1
make install # editable install with dev + ingest extras
make install # editable install with dev, ingest, and type-check extras

# 4. Verify everything works
make check # lint
make check # lint + static type checks
make test # run the offline test suite

# 5. See every available command
Expand All @@ -52,9 +52,10 @@ AI agents working in this repo should use the standardized `make` targets:
```bash
make help # See all available targets
make install # Install dev environment (editable)
make check # Quick health check (lint)
make check # Quick health check (lint + static type checks)
make typecheck # Run mypy across the package
make test # Run all unit tests (offline)
make run-ci # Full CI pipeline locally (lint + test)
make run-ci # Full CI pipeline locally (lint + types + test)
make format # Auto-format and fix lint issues
make docs # Regenerate the static docs site
make clean # Remove caches and build artifacts
Expand All @@ -64,7 +65,9 @@ make clean # Remove caches and build artifacts

- Prefer `make` targets over invoking tools directly — they match CI exactly.
- For direct tool calls, use the `uv run --no-sync` prefix (plain `uv run` can trigger a
universal resolve that pulls yanked optional deps).
universal resolve that pulls yanked optional deps). `make install` includes the
`typecheck` extra so adapter inheritance is checked against the real LangChain,
LlamaIndex, and DSPy APIs.
- Run `make run-ci` before declaring a change complete; it is the same pipeline CI runs.
- Prefer editing existing files over creating new ones, and follow the conventions in
neighboring modules.
Expand Down Expand Up @@ -108,15 +111,15 @@ dynavec is a single Python package (not a monorepo):
**Using Make (recommended):**

```bash
make install # uv pip install -e ".[dev,ingest]"
make install # uv pip install -e ".[dev,ingest,typecheck]"
make install-all # everything: all embedders + adapters + dev tools
```

**Manual setup:**

```bash
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]" # add ,ingest / ,all as needed
uv pip install -e ".[dev,ingest,typecheck]" # same environment as make install
```

### Pre-commit hooks (optional but encouraged)
Expand All @@ -136,13 +139,15 @@ pre-commit run --all-files # run across the whole tree once
| Command | What it does |
|---------|--------------|
| `make help` | List all targets |
| `make install` | Editable install with dev + ingest extras |
| `make install` | Editable install with dev, ingest, and strict type-check dependencies |
| `make install-all` | Editable install with **all** extras |
| `make format` | Auto-format (`ruff format`) and auto-fix lint (`ruff --fix`) |
| `make lint` / `make check` | Lint with ruff — mirrors CI exactly |
| `make lint` | Lint with ruff |
| `make typecheck` | Type-check the package with mypy's strict mode |
| `make check` | Run lint and static type checks |
| `make test` | Run the offline unit suite (`pytest -q`) |
| `make test-live` | Opt-in end-to-end test against **real AWS** (costs money) |
| `make run-ci` | The full CI pipeline locally: lint + test |
| `make run-ci` | The full CI pipeline locally: lint + types + test |
| `make docs` | Regenerate the static docs site |
| `make clean` | Remove caches and build artifacts |

Expand All @@ -160,7 +165,7 @@ git checkout -b feat/your-feature # branch off development
# ... make changes ...

make format # tidy up
make check # lint
make check # lint + static type checks
make test # verify
```

Expand Down Expand Up @@ -205,9 +210,18 @@ uv run --no-sync pytest tests/test_cache.py -k "jitter" -v

- **Style/linting:** [ruff](https://docs.astral.sh/ruff/) (config in `pyproject.toml`, rule
sets `E, F, I, UP, B`, line length 100). `make format` fixes most issues automatically.
- **Type hints:** dynavec ships a `py.typed` marker — please add type hints to new public APIs.
- **CI** (`.github/workflows/ci.yml`) runs on every push/PR: **ruff + pytest across Python
3.9, 3.11, and 3.12**. `make run-ci` reproduces it locally.
- **Type hints:** dynavec ships a `py.typed` marker. Mypy checks the package in strict mode
against the optional framework APIs in a dedicated CI job. Run `make install` once,
then `make typecheck` locally. Consumer fixtures in `tests/typing` protect
integration inheritance and the `explain` return contract for client, namespace,
and batch searches. When adding a return-shape option, cover both literal values
and a runtime `bool` across forwarding APIs; do not cast an overloaded callable
to a single result shape to satisfy an executor's type inference. Wrapper
annotations must also preserve live delegation: an embedder such as Ollama can
infer its dimension on the first request, so a cached wrapper must not snapshot
that property during construction.
- **CI** (`.github/workflows/ci.yml`) runs on every push/PR: **mypy**, plus ruff and
pytest across Python 3.9, 3.11, and 3.12. `make run-ci` reproduces it locally.

---

Expand All @@ -233,7 +247,7 @@ ci: add Python 3.13 to the matrix
**Review checklist:**

- [ ] Tests pass (`make test`)
- [ ] Lint passes (`make check`)
- [ ] Lint and static type checks pass (`make check`)
- [ ] New/changed behavior has tests
- [ ] Docs updated where relevant

Expand Down
16 changes: 10 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# dynavec developer commands. Run `make help` to see everything.
#
# These wrap the exact tools CI uses (uv + ruff + pytest), so `make run-ci`
# These wrap the exact tools CI uses (uv + ruff + mypy + pytest), so `make run-ci`
# locally is the same pipeline that runs on your PR.

.DEFAULT_GOAL := help
.PHONY: help install install-all format lint check test test-live docs clean run-ci
.PHONY: help install install-all format lint typecheck check test test-live docs clean run-ci

PY_DIRS := src benchmarks tests
CI_DIRS := src benchmarks # what CI lints (keep in sync with .github/workflows/ci.yml)
Expand All @@ -13,8 +13,8 @@ help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) \
| awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-14s\033[0m %s\n", $$1, $$2}'

install: ## Install dynavec + dev tools in editable mode (recommended)
uv pip install -e ".[dev,ingest]"
install: ## Install dynavec + dev, ingest, and type-check dependencies (recommended)
uv pip install -e ".[dev,ingest,typecheck]"

install-all: ## Install everything: all embedders, adapters, and dev tools
uv pip install -e ".[all,ingest,dev]"
Expand All @@ -26,7 +26,10 @@ format: ## Auto-format and fix lint issues (ruff format + ruff --fix)
lint: ## Lint without changing files (mirrors CI exactly)
uv run --no-sync ruff check $(CI_DIRS)

check: lint ## Quick health check (lint, no tests)
typecheck: ## Check package annotations with mypy
uv run --no-sync mypy

check: lint typecheck ## Quick health check (lint + types, no tests)

test: ## Run the unit test suite (offline, no AWS needed)
uv run --no-sync pytest -q
Expand All @@ -37,8 +40,9 @@ test-live: ## Run the opt-in end-to-end test against real AWS (costs money)
docs: ## Regenerate the static docs site into opensource/dynavec/docs/
uv run --no-sync python tools/build_docs.py

run-ci: ## Run the full CI pipeline locally (lint + test)
run-ci: ## Run the full CI pipeline locally (lint + types + test)
$(MAKE) lint
$(MAKE) typecheck
$(MAKE) test

clean: ## Remove caches and build artifacts
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,10 @@ for h in hits:
print(h.score, h.id, h.text)
```

`search()` returns a list of hits by default. Pass `explain=True` to receive an
`ExplainedSearchResult` containing those hits plus timing and candidate-count
details; the type annotations distinguish the two return shapes.

### Bring your own vectors (no embedder needed)

```python
Expand Down
65 changes: 65 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,13 @@ langchain = ["langchain-core>=0.3"]
llamaindex = ["llama-index-core>=0.11"]
crewai = ["crewai>=0.70,!=1.14.0; python_version >= '3.10'"]
dspy = ["dspy>=3.3; python_version >= '3.10'"]
# Framework APIs imported by strict mypy checks. Keep this in sync with the
# integration adapters covered by tests/typing.
typecheck = [
"langchain-core>=0.3",
"llama-index-core>=0.11",
"dspy>=3.3; python_version >= '3.10'",
]
all = [
"openai>=1.40",
"google-generativeai>=0.8",
Expand All @@ -75,6 +82,7 @@ dev = [
"pytest-cov>=5.0",
"hypothesis>=6.100",
"moto[dynamodb]>=5.0",
"mypy>=1.11,<2.0",
"ruff>=0.6",
"pre-commit>=3.5",
]
Expand Down Expand Up @@ -109,3 +117,60 @@ target-version = "py39"
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B"]
ignore = ["E501"]

[tool.mypy]
python_version = "3.12"
files = ["src/dynavec", "tests/typing"]
mypy_path = "typings"
strict = true
pretty = true
show_error_codes = true

# Optional integrations deliberately remain importable only when their extras are
# installed. Keep those dependency boundaries explicit while still checking all
# dynavec code that uses them.
[[tool.mypy.overrides]]
module = [
"bs4",
"bs4.*",
"boto3",
"boto3.*",
"botocore",
"botocore.*",
"crewai",
"crewai.*",
"docx",
"docx.*",
"dspy",
"dspy.*",
"google",
"google.*",
"langchain_core",
"langchain_core.*",
"llama_index",
"llama_index.*",
"matplotlib",
"matplotlib.*",
"mcp",
"mcp.*",
"mistralai",
"mistralai.*",
"openai",
"openai.*",
"openpyxl",
"openpyxl.*",
"pptx",
"pptx.*",
"pypdf",
"pypdf.*",
"redis",
"redis.*",
"requests",
"requests.*",
"sentence_transformers",
"sentence_transformers.*",
"voyageai",
"voyageai.*",
"yaml",
]
ignore_missing_imports = true
Loading
Loading