Repository navigation
docs: add a tested Taskiq worker recipe built on FreeBootstrapper - #276
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #274. A new integrations page for Taskiq workers, plus a test that runs the recipe. There is no change under
lite_bootstrap/, and no new extras. Theinstall-isolationand free-threaded CI lists are unchanged.The recipe
The handlers fire on
TaskiqEvents.WORKER_STARTUPandTaskiqEvents.WORKER_SHUTDOWNonly.FreeBootstrapper(FreeConfig(...)), call.bootstrap(), thenTaskiqInstrumentor().instrument_broker(broker)..teardown().The bootstrapper is kept on
TaskiqStateinstead of a module global. The brief only asked to register the handlers; I went further and also build the bootstrapper inside the startup handler. That way the client process never constructs one, so it gets no instrument-readiness or missing-dependency warnings.The page also covers:
otl-httpfor free-threaded workers;PrometheusMiddlewareand its two side effects: the process-widePROMETHEUS_MULTIPROC_DIRand its own HTTP server onserver_port;Test
tests/test_taskiq_recipe.pyregisters the same handlers, with the same calls in the same order, on ataskiq.InMemoryBroker. It then runs the realbroker.startup()/broker.shutdown().InMemoryBrokerfires the client and worker events in both, so the handlers are not called by hand.It asserts that:
OpenTelemetryMiddlewareis inbroker.middlewares;TracerProvider;As a mutation check, removing the
instrument_brokercall makes the test fail.The brief also asked whether
instrument_brokerstill works when called insideWORKER_STARTUP. It does.AsyncBroker.startupruns the event handlers before it walksself.middlewaresfor middleware startup, so the inserted middleware is in place in time.taskiq[opentelemetry]goes in thedevgroup only. It resolves to 0.13.0, which requires Python >= 3.10, so every pytest leg can install it.Verified outside the suite
ValueErroron anInMemoryBrokerwith a Sentry-configuredFreeBootstrapperproduced anerror-level Sentry event from loggertaskiq.receiver.receiver, carrying theValueError.Noticed in Taskiq, not acted on
instrument_brokerinserts the middleware without callingset_broker. As a result, the middleware's worker CPU and memory observations, which checkself.broker.is_worker_process, never report. Tracing and the task counters are unaffected. That would be an upstream Taskiq issue.Checks
just lint-ci,just test-ci(341 passed, 100% coverage),just docs-build(strict), andjust adr-checkall pass.