Run the integration suite locally against an AWS emulator

[harshsinghhsr]

tests/integration/test_live_aws.py was gated behind DYNAVEC_LIVE=1 and a real AWS account, so in practice it never ran. This points it at a local emulator instead.

docker compose up -d --wait
export AWS_ENDPOINT_URL=http://localhost:4566
pytest tests/integration -v      # 2 passed in 0.76s

No production code changed. boto3 reads AWS_ENDPOINT_URL natively, so the existing session.client(...) calls in stores/ and provisioning.py follow it with no endpoint plumbing. The test gate now accepts AWS_ENDPOINT_URL alongside DYNAVEC_LIVE, defaults dummy credentials when it sees one (setdefault, so a real profile is untouched), and skips the eventual-consistency sleep locally.

floci is used because it is currently the only emulator implementing s3vectors — LocalStack has it in backlog. I verified vector buckets, indexes, put/get/query and metadata filters (including the $and shape) all behave.

It found a bug on its first run

Both searches in test_provision_upsert_search_roundtrip queried the "default" namespace while the upsert wrote to "it". The filter {"_dv_ns": "default"} can never match, so that test could not have passed on real AWS either — it went unnoticed precisely because nobody could run it. Namespace is now a single NS constant threaded through the helper.

Notes

• New integration-local CI job; the compose healthcheck lets --wait block until the emulator is actually serving.
• Storage is in-memory, so docker compose down returns a clean account.
• Drops moto[dynamodb] from dev deps — nothing imports it.
• This is not a replacement for real AWS: the emulator does a cosine scan, not an ANN index, and has none of the eventual-consistency behaviour. DYNAVEC_LIVE=1 stays as the pre-release gate, and the README says so.

[codeforstartups]

Thanks @harshsinghhsr — the local-emulator developer experience here is excellent: the docker-compose.yml, the README 'Local development' section, and honoring AWS_ENDPOINT_URL in the integration test are all great and I'd love to land them. Concerns are with the CI job:

1. The integration-local job runs on every push and would likely turn CI red for everyone. The healthcheck hits http://localhost:4566/_localstack/health — that's LocalStack's endpoint, but the service is floci, so the healthcheck (and the --wait) may never pass. Please point it at floci's actual health endpoint and confirm floci/floci:latest is a stable, pullable image.
2. Until the emulator job is proven green, please make it non-blocking (continue-on-error: true) or trigger it via workflow_dispatch / a label, so an emulator hiccup doesn't block unrelated PRs.
3. Removing moto[dynamodb] from dev extras — is that intentional? There's an open issue for moto-based DynamoDB tests; if floci supersedes that, let's close that issue in this PR so the direction is explicit.

The non-CI parts are ready; if you'd prefer, split them into their own PR and we can merge that immediately while the CI job is sorted out.

[codeforstartups]

👋 The local-dev tooling (docker-compose + README + AWS_ENDPOINT_URL support) is great and I want to land it. To unblock:

1. Fix the healthcheck — it hits /_localstack/health but the service is floci; point it at floci's real health endpoint and confirm floci/floci:latest is pullable.
2. Make integration-local non-blocking (continue-on-error: true) or workflow_dispatch-only until it's proven green, so it can't red-x unrelated PRs.
3. Confirm the moto removal is intentional (close the moto-tests issue if floci supersedes it).

Or split the non-CI parts into their own PR and I'll merge that immediately.

[harshsinghhsr]

Thanks @codeforstartups for the review, all three fixed in the last commit.

Healthcheck: now hits floci's native /_floci/health. (FWIW /_localstack/health wasn't failing, floci serves it as a compat alias, but depending on a shim is fragile, so agreed.)

Image: pinned to floci/floci:2.0.1. latest moves; nightlies ship most days. Cold-pulled to verify: healthy in 8s, 2 passed in 4.08s.

Non-blocking: continue-on-error: true rather than dispatch-only. It still runs on every PR (so it builds a track record) but can't fail the build.

moto: intentional. moto has no s3vectors, so it could only cover the DynamoDB half; floci covers both. Supersedes #89 rather than completing it. Happy to add Closes #89 if you agree.

[codeforstartups]

You addressed all my earlier feedback and this is now a really nice local-dev story:

• integration-local job is non-blocking (continue-on-error: true) with a clear comment on why and when to drop it — exactly right so a third-party image can never red-X an unrelated PR.
• floci image is pinned (floci/floci:2.0.1), and you wait on a real healthcheck (docker compose up -d --wait + the native /_floci/health endpoint) instead of a sleep.
• The README "Local development" section and the docker-compose.yml comments are excellent, and refactoring test_live_aws.py to run against either real AWS or the emulator via AWS_ENDPOINT_URL (no dynavec code change) is the clean approach.
• Removing moto[dynamodb] from dev is fine — nothing in tests/ imports it.

Two mechanical things before it can land (neither is about the code):

1. Rebase needed — it conflicts in pyproject.toml. The dev extra has since gained pytest-asyncio, hypothesis, and pre-commit on development; please git rebase origin/development and keep those while applying your moto removal.
2. Maintainer will merge — because it edits .github/workflows/ci.yml, our automation token cannot merge it (missing workflow scope). Once rebased, @codeforstartups will click merge in the UI.

Approving on substance — thanks @harshsinghhsr, great contribution! 🙌

[harshsinghhsr]

@codeforstartups fixed the conflicts.
hopefully we can merge this now. :)

[codeforstartups]

Thanks @harshsinghhsr — rebased, conflicts resolved, and everything from the review is addressed. This is approved and mergeable; the only reason it is not already merged is that it edits .github/workflows/ci.yml, and our automation token cannot merge workflow-file changes (missing workflow OAuth scope). @codeforstartups will do the final click in the GitHub UI — it is a clean one-click merge. And yes, please add Closes #89 since this supersedes the moto-only approach. 🙌

[harshsinghhsr]

Closes #89