diff --git a/docs/design-patterns/downstream-rate-limiting.mdx b/docs/design-patterns/downstream-rate-limiting.mdx index c40f2c2b09..31f7cd7819 100644 --- a/docs/design-patterns/downstream-rate-limiting.mdx +++ b/docs/design-patterns/downstream-rate-limiting.mdx @@ -240,7 +240,7 @@ public class MyWorkflowImpl implements MyWorkflow { This pattern is a good fit when your Workflow calls a downstream service with explicit requests-per-second limits, when you need throughput enforcement that holds across many concurrent Workflow instances without per-Activity logic, and when only a subset of Activity types require throttling and others should run without restriction. -It is not a good fit when you need concurrency limits rather than throughput limits (see [Priority Task Queues](/design-patterns/priority-task-queues)), when the downstream system has no rate limit and throughput is bounded only by Workflow logic, or when all Activities require the same limit and a single shared queue suffices. +It is not a good fit when you need concurrency limits rather than throughput limits, when the downstream system has no rate limit and throughput is bounded only by Workflow logic, or when all Activities require the same limit and a single shared queue suffices. ## Benefits and trade-offs @@ -284,8 +284,8 @@ The concurrency slots (`MaxConcurrentActivityExecutionSize`, `MaxConcurrentWorkf ### Patterns -- **[Priority Task Queues](/design-patterns/priority-task-queues)**: Route work to separate queues by urgency, with different concurrency budgets per tier. -- **[Fairness](/design-patterns/fairness)**: Give each tenant an equal throughput share when multiple tenants share capacity. +- **[Priority](/design-patterns/priority-task-queues)**: Order Task dispatches by urgency within a shared Task Queue. +- **[Fairness](/design-patterns/fairness)**: Distribute Task dispatches among tenants that share a Task Queue. - **[Worker-Specific Task Queues](/design-patterns/worker-specific-taskqueue)**: Route Activities to a specific Worker host for resource or data affinity. ### Guides diff --git a/docs/design-patterns/fairness.mdx b/docs/design-patterns/fairness.mdx index 90e3e024dd..85b1274f60 100644 --- a/docs/design-patterns/fairness.mdx +++ b/docs/design-patterns/fairness.mdx @@ -1,301 +1,97 @@ --- id: fairness title: "Fairness" -description: "Distributes Worker capacity evenly across tenants or users so that a burst from one caller does not starve the others." +description: "Distributes Task dispatches across tenants or users so that a burst from one caller does not starve others." --- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - :::info[TLDR] -Assign a `FairnessKey` and weight to Workflows and Activities so each tenant or group receives the **correct proportional share of Worker capacity** on a shared Task Queue. Use this when a high-volume caller would otherwise starve other tenants without requiring separate queues per tenant. +Assign a Fairness key and weight to Workflows and Activities so each tenant or group receives a proportional share of Task dispatches on a shared Task Queue. Use this when a high-volume caller would otherwise starve other tenants, without requiring a separate Task Queue per tenant. ::: ## Overview -The Fairness pattern distributes Worker capacity proportionally across tenants or user groups within a single Task Queue so that a burst from one caller cannot starve others. Each group is assigned a fairness key and an optional weight; the Temporal matching service dispatches tasks in weighted round-robin order across all keys. +The Fairness pattern distributes Task dispatches proportionally across tenants or user groups within a shared Task Queue so that a burst from one caller cannot starve others. Each group has a Fairness key and an optional weight. The Matching Service uses weighted fair dispatch to select the next Task within a Priority level. + +Fairness applies only to Task dispatch. It does not account for Task duration or resource use. ## Problem -When multiple tenants (for example, customers) share a single Task Queue, a high-volume tenant can fill the queue and occupy all Worker slots. Other tenants receive no service until the dominant tenant's backlog drains. This starvation violates throughput guarantees and makes latency for lower-volume tenants unpredictable under burst conditions. +When multiple tenants, such as customers, share a Task Queue, a high-volume tenant can fill the backlog and dominate dispatch. Tasks from other tenants can wait behind that backlog, making their latency unpredictable during bursts. -The classic workaround—assigning one Task Queue per tenant—scales poorly: each new tenant requires a new Worker deployment, idle capacity on low-traffic tenants cannot be used by busy ones, and queue management complexity grows with tenant count. +The classic workaround is one Task Queue per tenant. This adds Task Queue, Worker, and routing configuration for every tenant. It can also strand idle capacity when Workers are dedicated to individual tenants. ## Solution -Temporal's native Fairness feature lets you assign a `FairnessKey` (a string identifier such as a tenant name or tier) and an optional `FairnessWeight` (a positive float, default 1.0) to Workflows, Activities, and Child Workflows. The Temporal matching service creates a virtual queue for each key and dispatches tasks in proportion to their weights. A single shared Worker pool serves all keys; no extra queues or routing logic is required. +Temporal's Fairness feature lets you assign a Fairness key, such as a tenant name or tier, and an optional Fairness weight to Workflows, Activities, and Child Workflows. A useful mental model is one virtual queue per Fairness key, with weighted round-robin dispatch across the queues. Temporal approximates this model with stride scheduling and a count-min sketch for larger key sets. A single shared Worker pool serves all keys, with no extra queues or routing logic. -For example, assigning weights of 5.0, 3.0, and 2.0 to `premium`, `basic`, and `free` tiers causes 50% of dispatched tasks to come from `premium`, 30% from `basic`, and 20% from `free`—regardless of backlog depth. Within a single fairness key, tasks are dispatched in FIFO order. +For example, assigning weights of 5.0, 3.0, and 2.0 causes approximately 50% of dispatched Tasks to come from `premium`, 30% from `basic`, and 20% from `free` when all three groups have backlogged Tasks. Within a Fairness key, Tasks at the same priority are dispatched in first-in-first-out (FIFO) order. ```mermaid flowchart TD - WA["Workflow\nfairness_key=tenant-big\n(weight 1.0)"] --> TQ["my-task-queue"] - WB["Workflow\nfairness_key=tenant-mid\n(weight 1.0)"] --> TQ - WC["Workflow\nfairness_key=tenant-small\n(weight 1.0)"] --> TQ - TQ --> VQ1["Virtual Queue\ntenant-big"] - TQ --> VQ2["Virtual Queue\ntenant-mid"] - TQ --> VQ3["Virtual Queue\ntenant-small"] - VQ1 -->|round-robin| W["Shared Workers"] - VQ2 -->|round-robin| W - VQ3 -->|round-robin| W - W --> DS["Downstream\nService"] + WA["Workflow\nfairness_key=tenant-big\nweight=1.0"] --> TQ["my-task-queue"] + WB["Workflow\nfairness_key=tenant-mid\nweight=1.0"] --> TQ + WC["Workflow\nfairness_key=tenant-small\nweight=1.0"] --> TQ + TQ --> VQ1["Virtual queue\ntenant-big"] + TQ --> VQ2["Virtual queue\ntenant-mid"] + TQ --> VQ3["Virtual queue\ntenant-small"] + VQ1 -->|weighted dispatch| W["Shared Workers"] + VQ2 -->|weighted dispatch| W + VQ3 -->|weighted dispatch| W ``` The following describes each step in the diagram: -1. Workflows start with a `FairnessKey` matching the tenant or group identity. -2. The Temporal matching service routes each task to the corresponding virtual queue inside the single Task Queue. -3. Workers poll the Task Queue and receive tasks in weighted round-robin order across all fairness keys. -4. Tenant-big's large backlog does not prevent tenant-mid or tenant-small from receiving service. +1. Workflows start with a Fairness key that identifies their tenant or group. +2. Tasks with the same Fairness key enter the same virtual queue. +3. When multiple virtual queues have backlogged Tasks, the Matching Service uses weighted round robin to choose between them. +4. One virtual queue can use all available dispatches when the others have no backlogged Tasks. ## Implementation -### Enable fairness - -**Temporal Cloud:** Navigate to the Namespace's Overview page in the UI and activate the Fairness toggle. Fairness is a paid feature in Temporal Cloud. - -**Self-hosted Temporal:** Set `matching.enableFairness` to `true` in the [dynamic config](/temporal-service/configuration#dynamic-configuration) for the relevant Task Queues or Namespaces. - -### Set fairness key and weight at Workflow start - - - - -```python -from temporalio.common import Priority - -handle = await client.start_workflow( - ProcessOrder.run, - id="process-order-wf", - task_queue="my-task-queue", - priority=Priority( - fairness_key="tenant-a", - fairness_weight=2.0, - ), -) -``` - - - - -```go -we, err := c.ExecuteWorkflow( - context.Background(), - client.StartWorkflowOptions{ - ID: "process-order-wf", - TaskQueue: "my-task-queue", - Priority: temporal.Priority{ - FairnessKey: "tenant-a", - FairnessWeight: 2.0, - }, - }, - ProcessOrder, -) -``` - - - - -```java -WorkflowOptions options = WorkflowOptions.newBuilder() - .setWorkflowId("process-order-wf") - .setTaskQueue("my-task-queue") - .setPriority(Priority.newBuilder() - .setFairnessKey("tenant-a") - .setFairnessWeight(2.0f) - .build()) - .build(); -ProcessOrder workflow = client.newWorkflowStub(ProcessOrder.class, options); -WorkflowClient.start(workflow::run); -``` - - - - -### Set fairness key and weight on Activities - -Activities inherit the parent Workflow's fairness key and weight. Override them in `ActivityOptions` when an Activity should belong to a different fairness group than its Workflow. Each field (`priority_key`, `fairness_key`, `fairness_weight`) is resolved independently in this order: Task Queue weight overrides (highest precedence), value set explicitly in the options, value inherited from the calling Workflow, then the default. Workflows started with Continue-As-New inherit the current execution's priority values unless you pass explicit values. See [Inheritance](/develop/task-queue-priority-fairness#inheritance) in the Temporal docs for the full resolution diagram. - - - - -```python -from temporalio.common import Priority - -# inside the workflow -result = await workflow.execute_activity( - process_for_tenant, - tenant_request, - start_to_close_timeout=timedelta(minutes=1), - priority=Priority( - fairness_key="tenant-a", - fairness_weight=2.0, - ), -) -``` - - - - -```go -ao := workflow.ActivityOptions{ - StartToCloseTimeout: time.Minute, - Priority: temporal.Priority{ - FairnessKey: "tenant-a", - FairnessWeight: 2.0, - }, -} -ctx = workflow.WithActivityOptions(ctx, ao) -err := workflow.ExecuteActivity(ctx, ProcessForTenant, req).Get(ctx, nil) -``` - - - - -```java -ActivityOptions options = ActivityOptions.newBuilder() - .setStartToCloseTimeout(Duration.ofMinutes(1)) - .setPriority(Priority.newBuilder() - .setFairnessKey("tenant-a") - .setFairnessWeight(2.0f) - .build()) - .build(); -TenantActivity activity = Workflow.newActivityStub(TenantActivity.class, options); -activity.processForTenant(request); -``` - - - - -### Set queue-level and per-key rate limits via CLI - -You can rate-limit the entire Task Queue and set a default per-fairness-key limit. The per-key limit is scaled by the fairness weight for that key, so a key with weight 2.5 and a default per-key limit of 10 gets an effective limit of 25 tasks/second. - -```sh -temporal task-queue config set \ - --task-queue my-task-queue \ - --task-queue-type activity \ - --namespace my-namespace \ - --queue-rps-limit 500 \ - --queue-rps-limit-reason "overall limit" \ - --fairness-key-rps-limit-default 33.3 \ - --fairness-key-rps-limit-reason "per-key limit" -``` - -### Override fairness weights via CLI - -When it is more convenient to manage weights through configuration than to embed them in client code, you can override weights for up to 1000 keys per Task Queue. Overrides take precedence over the weight attached to a task's options and can be updated without a code deploy. - -```sh -temporal task-queue config set \ - --task-queue my-task-queue \ - --task-queue-type workflow \ - --namespace my-namespace \ - --fairness-key-weight premium=5.0 \ - --fairness-key-weight basic=3.0 \ - --fairness-key-weight free=2.0 -``` - -### Use priority and fairness together - -Priority and Fairness can be combined. Priority determines which sub-queue (1–5) a task enters; Fairness determines the dispatch order within each priority level. Set both `PriorityKey` and `FairnessKey` on the same options object. - - - - -```python -from temporalio.common import Priority - -handle = await client.start_workflow( - ChargeCustomer.run, - id="charge-customer-wf", - task_queue="my-task-queue", - priority=Priority( - priority_key=1, - fairness_key="tenant-a", - fairness_weight=2.0, - ), -) -``` - - - - -```go -we, err := c.ExecuteWorkflow( - context.Background(), - client.StartWorkflowOptions{ - ID: "charge-customer-wf", - TaskQueue: "my-task-queue", - Priority: temporal.Priority{ - PriorityKey: 1, - FairnessKey: "tenant-a", - FairnessWeight: 2.0, - }, - }, - ChargeCustomer, -) -``` - - - - -```java -WorkflowOptions options = WorkflowOptions.newBuilder() - .setWorkflowId("charge-customer-wf") - .setTaskQueue("my-task-queue") - .setPriority(Priority.newBuilder() - .setPriorityKey(1) - .setFairnessKey("tenant-a") - .setFairnessWeight(2.0f) - .build()) - .build(); -``` +Fairness must be enabled for the Namespace. Set Fairness keys and weights on Workflows, Activities, or Child Workflows. Activities and Child Workflows inherit these values unless you override them. - - +See [Task Queue Priority and Fairness](/develop/task-queue-priority-fairness#task-queue-fairness) for setup, SDK examples, inheritance, Task Queue configuration, and limitations. ## When to use -This pattern is a good fit for multi-tenant applications where large tenants should not block small tenants, for workloads that need proportional capacity allocation across groups without hard rate limits, and when the set of tenants or groups is dynamic (new keys can be introduced without deploying new Workers). For a broader look at multi-tenancy strategies in Temporal, see [Multi-Tenant Patterns](/best-practices/multi-tenant-patterns). +This pattern is a good fit for multi-tenant applications where large tenants should not block small tenants, for workloads that need proportional Task dispatch across groups without hard rate limits, and when the set of tenants or groups is dynamic. New Fairness keys can be introduced without deploying new Workers. For a broader look at multi-tenancy strategies in Temporal, see [Multi-Tenant Patterns](/best-practices/multi-tenant-patterns). -It is not a good fit when absolute throughput isolation is required (dedicated queues per tenant remain or [task queue priorities](/design-patterns/priority-task-queues) are the appropriate choice). +Fairness does not provide exact dispatch ratios, concurrency limits, or compute isolation. Use Activity Task Queue rate limits for throughput caps. Use separate Task Queues with dedicated Worker pools and compute resources for hard isolation. Use Priority to order urgent work ahead of less urgent work. ## Benefits and trade-offs -A single Worker pool serves all tenants; idle capacity from a low-traffic tenant automatically benefits high-traffic tenants rather than going to waste. New tenants require no Worker deployment—add a fairness key and Temporal starts dispatching their tasks immediately. Weights can be updated via CLI without redeploying application code. +A single Worker pool serves all tenants, so idle capacity from a low-traffic tenant is available to high-traffic tenants. New tenants require no Worker deployment. Add a Fairness key and Temporal starts dispatching their Tasks. Fairness weights can be updated through Task Queue configuration without redeploying application code. -Fairness requires explicit enablement on Temporal Cloud and self-hosted deployments. Accuracy can degrade with a very large number of fairness keys. Fairness weight applies at schedule time, not dispatch time: changing a weight does not retroactively reorder tasks already in the backlog. +Fairness is best effort. Dispatch ratios can vary across Task Queue partitions, Worker Versioning, and short time windows. Accuracy can degrade with a large number of Fairness keys. Fairness weights apply when Tasks are scheduled, so changing a weight does not reorder Tasks already in the backlog. Tasks with different runtimes can consume different amounts of Worker capacity even when their dispatch shares match their weights. ## Comparison with alternatives -| Approach | Tenant isolation | Dynamic tenants | Shares idle capacity | Complexity | +| Approach | Per-tenant dispatch | Dynamic tenants | Shares idle capacity | Complexity | | :--- | :--- | :--- | :--- | :--- | -| Temporal FairnessKey (native) | Soft | Yes | Yes | Low | -| Dedicated queue per tenant | Hard | No | No | Medium | -| Single shared queue (no control) | None | Yes | Yes | Lowest | -| External queue with per-tenant consumer groups | Hard | Yes | No | High | +| Temporal Fairness (native) | Weighted fair dispatch | Yes | Yes | Low | +| Dedicated Task Queue per tenant | Separate | No | No | Medium | +| Single shared Task Queue (no control) | None | Yes | Yes | Lowest | +| External queue with per-tenant consumer groups | Separate | Yes | No | High | ## Best practices -- **Use stable, consistent naming for fairness keys.** Use account IDs or tenant slugs rather than display names. Key names cannot be changed retroactively on tasks already in the backlog. -- **Combine Priority and Fairness for multi-class, multi-tenant workloads.** Priority separates urgent from batch work; Fairness prevents any single tenant from dominating within each priority level. -- **Monitor queue depth by fairness key.** Sustained backlog growth for a particular key means its weight fraction of Worker capacity cannot drain its submission rate. +- **Use stable, consistent Fairness keys.** Use account identifiers or tenant slugs instead of display names. Key changes do not reorder Tasks already in the backlog. +- **Combine Priority and Fairness for multi-class, multi-tenant workloads.** Priority separates urgent work from batch work. Fairness prevents a single tenant from dominating within each Priority level. ## Common pitfalls -- **Expecting Fairness to reorder the existing backlog.** Fairness weight is evaluated at schedule time. Enabling Fairness on a Namespace with an existing backlog drains that backlog in its original order first; the fairness-aware dispatch mode takes effect only for newly submitted tasks. -- **Using Fairness as a hard rate limiter.** Fairness controls proportional dispatch but does not cap the absolute throughput of any one key. For hard throughput caps, combine Fairness with per-fairness-key RPS limits via the CLI. -- **Unkeyed tasks bypassing Fairness.** Tasks without a `FairnessKey` are grouped under an implicit empty-string key and participate in round-robin dispatch alongside named keys with a weight of 1.0. They do not bypass Fairness and compete as one group. -- **Task Queue partitioning reducing accuracy.** Task Queues are internally partitioned and tasks are distributed to partitions randomly, which can interfere with fair dispatch proportions. If your workload requires higher accuracy, contact Temporal Support to configure a single-partition Task Queue. -- **Assuming Fairness applies across Worker Versioning boundaries.** When using Worker Versioning and moving Workflows between versions, Priority still applies across versions but Fairness is only guaranteed within tasks originally queued on the same Worker version. Tasks moved from one version to another may not dispatch in fairness order relative to tasks on the destination version. -- **Expecting consistent fairness immediately after a server restart.** Fairness ordering is preserved across restarts for the most active keys. Less active keys may briefly dispatch new tasks ahead of their existing backlog until ordering normalizes. -- **Expecting the running task mix to immediately reflect fair dispatch.** Fairness governs which task is dispatched next; it does not account for tasks already running on Workers. The mix of in-flight tasks at any moment may not match the configured weight ratios. +- **Expecting Fairness to reorder the existing backlog.** Fairness weight is evaluated at schedule time. When Fairness is enabled for a Namespace with an existing backlog, that backlog drains in its original order before fairness-aware dispatch applies to new Tasks. +- **Using Fairness as a hard rate limiter.** Fairness by itself doesn't cap throughput. Use [Fairness key rate limits](/develop/task-queue-priority-fairness#set-rate-limits-at-the-task-queue-level) for per-key limits. +- **Unkeyed Tasks bypassing Fairness.** Tasks without a Fairness key are grouped under an implicit empty-string key and participate in weighted fair dispatch alongside named Fairness keys with a default weight of 1.0. They do not bypass Fairness and compete as one group. +- **Task Queue partitioning reducing accuracy.** Task Queues are internally partitioned, and Tasks are distributed to partitions randomly. This can interfere with fair dispatch proportions. If your workload requires higher accuracy, contact Temporal Support to configure a single-partition Task Queue. +- **Assuming Fairness applies across Worker Versioning boundaries.** Worker Deployment Versions have separate backlogs. Fairness applies within each version's backlog. +- **Expecting consistent Fairness immediately after a server restart.** Fairness ordering is preserved across restarts for the most active keys. Less active keys may briefly dispatch new Tasks ahead of their existing backlog until ordering normalizes. +- **Expecting the running Task mix to immediately reflect fair dispatch.** Fairness governs which Task is dispatched next. It does not account for Tasks already running on Workers. A group with longer-running Tasks can consume more Worker time than its dispatch share suggests. ## Related ### Patterns -- **[Priority Task Queues](/design-patterns/priority-task-queues)**: Order tasks by urgency level within the same Task Queue using `PriorityKey`. -- **[Downstream Rate Limiting](/design-patterns/downstream-rate-limiting)**: Cap absolute throughput to a downstream service with a queue RPS setting. +- **[Priority](/design-patterns/priority-task-queues)**: Order Task dispatches by urgency within the same Task Queue using a Priority key. +- **[Downstream Rate Limiting](/design-patterns/downstream-rate-limiting)**: Cap dispatch throughput to a downstream service with a Task Queue RPS setting. - **[Worker-Specific Task Queues](/design-patterns/worker-specific-taskqueue)**: Route Activities to a specific Worker host for resource or data affinity. diff --git a/docs/design-patterns/index.mdx b/docs/design-patterns/index.mdx index 93d19f808f..41b095f002 100644 --- a/docs/design-patterns/index.mdx +++ b/docs/design-patterns/index.mdx @@ -214,14 +214,14 @@ Having these patterns in your toolbox helps you solve recurring problems in a ba { href: "/design-patterns/priority-task-queues", icon: "priority-task-queues-icon.svg", - title: "Priority Task Queues", - description: "Assigns a priority level to Workflows and Activities so that time-sensitive work executes ahead of lower-priority work within a single Task Queue.", + title: "Priority", + description: "Assigns a priority level to Workflows and Activities so that time-sensitive work dispatches ahead of lower-priority work within a single Task Queue.", }, { href: "/design-patterns/fairness", icon: "fairness-icon.svg", title: "Fairness", - description: "Distributes Worker capacity evenly across tenants or users so that a burst from one caller does not starve the others.", + description: "Distributes Task dispatches across tenants or users so that a burst from one caller does not starve the others.", }, ]} /> diff --git a/docs/design-patterns/priority-task-queues.mdx b/docs/design-patterns/priority-task-queues.mdx index 322f76df00..d3bd7e638b 100644 --- a/docs/design-patterns/priority-task-queues.mdx +++ b/docs/design-patterns/priority-task-queues.mdx @@ -1,33 +1,32 @@ --- id: priority-task-queues -title: "Priority Task Queues" -description: "Assigns a priority level to Workflows and Activities so that time-sensitive work executes ahead of lower-priority work within a single Task Queue." +title: "Priority" +description: "Assigns a priority level to Workflows and Activities so that time-sensitive work dispatches ahead of lower-priority work within a single Task Queue." --- -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - :::info[TLDR] -Assign a `PriorityKey` (1–5) to Workflows and Activities so **high priority work executes ahead of lower priority work** on a shared Task Queue. Use this when a flood of batch or background tasks would otherwise delay high-urgency requests. +Assign a Priority key from 1 to 5 to Workflows and Activities so **high-priority work dispatches ahead of lower-priority work** on a shared Task Queue. Use this when a flood of batch or background Tasks would otherwise delay high-urgency requests. ::: ## Overview -The Priority Task Queues pattern assigns a `PriorityKey` to Workflows, Activities, and Child Workflows so that time-sensitive work executes ahead of lower-priority work within a single Task Queue—without requiring separate queues or routing logic. +The Priority pattern assigns a Priority key to Workflows, Activities, and Child Workflows so that time-sensitive work dispatches ahead of lower-priority work within a single Task Queue, without requiring separate queues or routing logic. + +Priority applies to dispatch. It does not preempt running Tasks or reserve Worker capacity. ## Problem -In a shared Task Queue, tasks execute in generally first-in-first-out (FIFO) order. When a large batch of low-priority work—nightly reports, bulk imports, background processing—floods the queue before time-sensitive requests arrive, the higher-priority requests wait behind the entire batch. A single queue with no ordering mechanism gives equal treatment to all tasks regardless of business urgency. +In a shared Task Queue, backlogged Tasks are generally dispatched in first-in-first-out (FIFO) order within a partition. When a large batch of low-priority work, such as nightly reports, bulk imports, or background processing, fills the backlog before time-sensitive requests arrive, the higher-priority requests wait behind the entire batch. A single Task Queue with no ordering mechanism gives the same dispatch preference to all Tasks, regardless of business urgency. ## Solution -Temporal's native Priority feature lets you assign a `PriorityKey` (an integer from 1 to 5, where 1 is the highest priority and 5 is the lowest) to any Workflow, Activity, or Child Workflow. The Temporal matching service maintains a sub-queue for each priority level and exhausts all tasks at a given level before dispatching to the next. Tasks default to priority 3 when no key is set. Activities and Child Workflows inherit the parent Workflow's priority unless they set their own. +Temporal's native Priority feature lets you assign a Priority key (an integer from 1 to 5, where 1 is the highest priority and 5 is the lowest) to any Workflow, Activity, or Child Workflow. The Matching Service maintains a sub-queue for each Priority level and exhausts all backlogged Tasks at a given level before dispatching to the next. Tasks default to Priority `3` when no key is set. Activities and Child Workflows inherit the parent Workflow's Priority unless they set their own. ```mermaid flowchart TD - WF1["Workflow\nPriorityKey=1\n(payment)"] --> TQ["my-task-queue"] - WF2["Workflow\nPriorityKey=3\n(default)"] --> TQ - WF3["Workflow\nPriorityKey=5\n(batch report)"] --> TQ + WF1["Workflow\nPriority 1\n(payment)"] --> TQ["my-task-queue"] + WF2["Workflow\nPriority 3\n(default)"] --> TQ + WF3["Workflow\nPriority 5\n(batch report)"] --> TQ TQ --> P1["Priority 1\nsub-queue"] TQ --> P3["Priority 3\nsub-queue"] TQ --> P5["Priority 5\nsub-queue"] @@ -37,213 +36,59 @@ flowchart TD W --> DS["Downstream\nService"] ``` -The following describes each step in the diagram: +The following describes each step in the diagram. The example assumes all three levels have backlogged Tasks in the same Task Queue partition and Worker Deployment Version. -1. Workflows start with a `PriorityKey` in their start options. Payment workflows use priority 1; routine workflows default to 3; nightly batch reports use priority 5. -2. The Temporal matching service routes each task to the corresponding priority sub-queue inside the single Task Queue. -3. Workers poll the Task Queue and receive tasks in priority order: all priority-1 tasks are dispatched before any priority-2 task, and so on. -4. Activities and Child Workflows inherit the parent Workflow's `PriorityKey` unless they explicitly set their own. +1. Workflows start with a Priority key in their start options. Payment Workflows use Priority `1`. Routine Workflows default to Priority `3`. Nightly batch reports use Priority `5`. +2. The Matching Service routes each Task to the corresponding Priority sub-queue inside the Task Queue. +3. Workers poll the Task Queue and receive the highest-priority backlogged Tasks first. +4. Activities and Child Workflows inherit the parent Workflow's Priority unless they set their own. ## Implementation -Priority is enabled by default in Temporal Cloud and self-hosted Temporal. - -### Set Workflow priority at start - - - - -```python -from temporalio.common import Priority - -handle = await client.start_workflow( - ChargeCustomer.run, - id="charge-customer-wf", - task_queue="my-task-queue", - priority=Priority(priority_key=1), -) -``` - - - - -```go -we, err := c.ExecuteWorkflow( - context.Background(), - client.StartWorkflowOptions{ - ID: "charge-customer-wf", - TaskQueue: "my-task-queue", - Priority: temporal.Priority{PriorityKey: 1}, - }, - ChargeCustomer, -) -``` - - - - -```java -WorkflowOptions options = WorkflowOptions.newBuilder() - .setWorkflowId("charge-customer-wf") - .setTaskQueue("my-task-queue") - .setPriority(Priority.newBuilder().setPriorityKey(1).build()) - .build(); -ChargeCustomer workflow = client.newWorkflowStub(ChargeCustomer.class, options); -WorkflowClient.start(workflow::run); -``` - - - - -### Set Activity priority - -Activities inherit the parent Workflow's priority. Override the `PriorityKey` in `ActivityOptions` when an individual Activity should run at a different level than its Workflow. - - - - -```python -from temporalio.common import Priority - -# inside the workflow -result = await workflow.execute_activity( - process_payment, - start_to_close_timeout=timedelta(minutes=1), - priority=Priority(priority_key=1), -) -``` - - - - -```go -ao := workflow.ActivityOptions{ - StartToCloseTimeout: time.Minute, - Priority: temporal.Priority{PriorityKey: 1}, -} -ctx = workflow.WithActivityOptions(ctx, ao) -err := workflow.ExecuteActivity(ctx, ProcessPayment).Get(ctx, nil) -``` - - - - -```java -ActivityOptions options = ActivityOptions.newBuilder() - .setStartToCloseTimeout(Duration.ofMinutes(1)) - .setPriority(Priority.newBuilder().setPriorityKey(1).build()) - .build(); -PaymentActivities activities = Workflow.newActivityStub(PaymentActivities.class, options); -activities.processPayment(); -``` - - - - -### Set Child Workflow priority - - - - -```python -from temporalio.common import Priority - -# inside the parent workflow -result = await workflow.execute_child_workflow( - ProcessOrder.run, - id="process-order-child", - task_queue="my-task-queue", - priority=Priority(priority_key=2), -) -``` +Priority is enabled by default in Temporal Cloud and self-hosted Temporal. Set a Priority key in Workflow start options or in Activity and Child Workflow options. - - - -```go -cwo := workflow.ChildWorkflowOptions{ - WorkflowID: "process-order-child", - TaskQueue: "my-task-queue", - Priority: temporal.Priority{PriorityKey: 2}, -} -ctx = workflow.WithChildOptions(ctx, cwo) -err := workflow.ExecuteChildWorkflow(ctx, ProcessOrder).Get(ctx, nil) -``` - - - - -```java -ChildWorkflowOptions options = ChildWorkflowOptions.newBuilder() - .setWorkflowId("process-order-child") - .setTaskQueue("my-task-queue") - .setPriority(Priority.newBuilder().setPriorityKey(2).build()) - .build(); -ProcessOrder child = Workflow.newChildWorkflowStub(ProcessOrder.class, options); -child.run(); -``` - - - - -### Set priority via CLI - -```sh -temporal workflow start \ - --type ChargeCustomer \ - --task-queue my-task-queue \ - --workflow-id charge-customer-wf \ - --input '{"customerId":"12345"}' \ - --priority-key 1 -``` +See [Task Queue Priority](/develop/task-queue-priority-fairness#task-queue-priority) for SDK and command-line examples, inheritance behavior, and self-hosted configuration. ## When to use -This pattern is a good fit when your system mixes time-sensitive operations (payment processing, user-facing requests) with background or batch work (reporting, data imports, inventory management), and you want urgent tasks to proceed even during periods of high load. It also works well when you need to mark urgent tasks that should override normal processing—for example, triggering immediate re-runs of failed critical tasks. +This pattern is a good fit when your system mixes time-sensitive operations (payment processing, user-facing requests) with background or batch work (reporting, data imports, inventory management), and you want urgent Tasks to dispatch first during periods of high load. It also works well when you need to mark urgent Tasks that should dispatch ahead of normal processing, for example, triggering immediate reruns of failed critical Tasks. -It is not a good fit when all work is effectively equal in urgency, when a continuously replenished high-priority backlog could starve lower-priority work indefinitely, or when you need hard capacity isolation between tiers (see dedicated queues per tier as a supplementary measure). If your concern is prioritizing work amongst tenants or customers, consider the [Fairness](/design-patterns/fairness) pattern, which distributes capacity proportionally using weighted fairness keys rather than strict ordering. +It is not a good fit when all work is effectively equal in urgency, when a continuously replenished high-priority backlog could starve lower-priority work indefinitely, or when you need hard capacity isolation between tiers. Use separate Task Queues with dedicated Worker pools and compute resources for hard capacity isolation. If your concern is prioritizing work among tenants or customers, consider the [Fairness](/design-patterns/fairness) pattern, which distributes dispatches proportionally using weighted Fairness keys rather than strict ordering. ## Benefits and trade-offs -Native priority requires no extra queues, routing logic, or additional Worker pools. A single pool of Workers serves all priority levels, so idle capacity at low-priority levels is automatically used by higher-priority work without any additional configuration. +Native Priority requires no extra queues, routing logic, or additional Worker pools. A single pool of Workers serves all Priority levels, so idle Worker capacity is available to every level without additional configuration. -Lower-priority tasks are blocked until all higher-priority tasks have started. In an environment with a continuously replenished high-priority backlog, low-priority tasks may be significantly delayed. The built-in `PriorityKey` range is 1–5; if more than five distinct levels are needed, the feature cannot accommodate them. +Lower-priority Tasks are blocked while higher-priority Tasks remain backlogged. In an environment with a continuously replenished high-priority backlog, low-priority Tasks may be significantly delayed. The built-in Priority key range is 1 to 5. The feature does not support more than five levels. ## Comparison with alternatives -| Approach | Isolation | Dynamic priority | Complexity | Scales to many priorities | -| :--- | :--- | :--- | :--- | :--- | -| Temporal PriorityKey (native) | Soft | Yes | Low | Yes (1–5 levels) | -| [Fairness](/design-patterns/fairness) | Soft | Yes | Low | Yes (unlimited keys) | -| Separate Task Queues per tier | Hard | No | Medium | No (static tiers) | -| Single queue (no control) | None | N/A | Lowest | N/A | -| External queue (Kafka, SQS) | Hard | Yes | High | Yes | +| Approach | Backlog dispatch | Shares idle capacity | +| :--- | :--- | :--- | +| Priority on a shared Task Queue | Higher-priority Tasks first | Yes | +| [Fairness](/design-patterns/fairness) on a shared Task Queue | Weighted across groups within a Priority level | Yes | +| Separate Task Queues with shared compute | Independent backlogs | Yes | +| Separate Task Queues with dedicated compute | Independent backlogs | No | ## Best practices -- **Use no more than five priority levels.** The `PriorityKey` range is 1–5. Keep levels coarse—for example, 1 = urgent, 3 = normal, 5 = batch—rather than mapping fine-grained business importance to many values. -- **Reserve priority 1 for genuinely urgent work.** If high priority is the fallback when no priority is specified, the highest level fills with routine work and the feature provides no benefit. The default is 3 when no key is set. -- **Set `PriorityKey` at Workflow start, not inside Workflow code.** Workflow code cannot change its own priority after it starts. Set the priority in the start options before execution begins. -- **Override Activity priority deliberately.** Activities inherit the parent Workflow's priority by default. Override only when a specific Activity must run at a different level than its Workflow. -- **Monitor queue depth per priority level.** Sustained backlog growth at a priority level signals that Worker capacity is insufficient for the submitted load at that level. +- **Use no more than five Priority levels.** The Priority key range is 1 to 5. Keep levels coarse. For example, use `1` for urgent work, `3` for normal work, and `5` for batch work. +- **Reserve Priority `1` for genuinely urgent work.** When every caller uses Priority `1`, the highest level fills with routine work and the feature provides no benefit. The default is `3` when no key is set. +- **Set the initial Priority key at Workflow start.** Set the Priority in the start options before execution begins. Activities and Child Workflows inherit it unless they set their own. +- **Override Activity Priority deliberately.** Activities inherit the parent Workflow's Priority by default. Override it only when a specific Activity must dispatch at a different level than its Workflow. +- **Monitor queue depth per Priority level.** Sustained backlog growth at a level means Tasks are arriving faster than they are being dispatched. ## Common pitfalls -- **Assigning priority 1 to all work by default.** When every caller sets the highest priority, the feature provides no ordering benefit. Establish an explicit policy for which work types qualify for each level. -- **Neglecting low-priority starvation.** Under sustained high load, priority-5 tasks may wait indefinitely. Use `ScheduleToStartTimeout` on low-priority activities to surface starvation as a visible failure. -- **Changing priority after scheduling.** `PriorityKey` is evaluated when a task enters the queue and cannot be changed while it waits. To re-prioritize an already-queued task, cancel it and reschedule with the new priority. -- **Assuming hard isolation between priority levels.** Priority controls dispatch order, not Worker capacity allocation. A priority-5 task may still consume a Worker slot that is then unavailable for a priority-1 task arriving a moment later. +- **Assigning Priority `1` to all work by default.** When every caller sets the highest Priority, the feature provides no ordering benefit. Establish an explicit policy for which work types qualify for each level. +- **Neglecting low-priority starvation.** Under sustained high load, Priority `5` Tasks may wait indefinitely. Use a [Schedule-To-Start Timeout](/encyclopedia/detecting-activity-failures#schedule-to-start-timeout) on low-priority Activities to surface starvation as a visible failure. +- **Changing priority after scheduling.** The Priority key is evaluated when a Task enters the queue and cannot be changed while it waits. To re-prioritize an already-queued Task, cancel it and reschedule with the new priority. +- **Assuming hard isolation between Priority levels.** Priority controls dispatch order, not Worker capacity allocation. A Priority `5` Task may still occupy a Worker slot when a Priority `1` Task arrives. ## Related ### Patterns -- **[Fairness](/design-patterns/fairness)**: Distribute capacity proportionally across tenants within a priority level using fairness keys. -- **[Downstream Rate Limiting](/design-patterns/downstream-rate-limiting)**: Cap absolute throughput to a downstream service regardless of task priority. +- **[Fairness](/design-patterns/fairness)**: Distribute dispatches across tenants within a Priority level. +- **[Downstream Rate Limiting](/design-patterns/downstream-rate-limiting)**: Cap dispatch throughput to a downstream service. - **[Worker-Specific Task Queues](/design-patterns/worker-specific-taskqueue)**: Route Activities to a specific Worker host for resource or data affinity. - -### Sample code - -The official Temporal documentation provides SDK code examples for setting priority keys on Workflows, Activities, and Child Workflows across all supported languages: - -- [Task Queue Priority and Fairness — Temporal docs](/develop/task-queue-priority-fairness#task-queue-priority) diff --git a/docs/design-patterns/qos-throughput-patterns.mdx b/docs/design-patterns/qos-throughput-patterns.mdx index 0e355c1383..95be00b800 100644 --- a/docs/design-patterns/qos-throughput-patterns.mdx +++ b/docs/design-patterns/qos-throughput-patterns.mdx @@ -1,12 +1,12 @@ --- id: qos-throughput-patterns title: "QoS & Throughput Patterns" -description: "Pattern selection guide for controlling execution rate, protecting downstream services from overload, and ensuring fair capacity distribution across tenants." +description: "Pattern selection guide for controlling execution rate, protecting downstream services from overload, and providing fair dispatch across tenants." --- import PatternCards from '@site/src/components/PatternCards'; -These patterns control how fast work executes, protect downstream services from overload, and make sure no single caller or tenant monopolizes Worker capacity at the expense of others. +These patterns control how fast work executes, protect downstream services from overload, and make sure no single caller or tenant dominates dispatch. ## Patterns in this section @@ -20,14 +20,14 @@ These patterns control how fast work executes, protect downstream services from { href: "/design-patterns/priority-task-queues", icon: "priority-task-queues-icon.svg", - title: "Priority Task Queues", - description: "Assigns a priority level to Workflows and Activities so time-sensitive work runs ahead of lower-priority work on the same Task Queue.", + title: "Priority", + description: "Assigns a priority level to Workflows and Activities so time-sensitive work dispatches ahead of lower-priority work on the same Task Queue.", }, { href: "/design-patterns/fairness", icon: "fairness-icon.svg", title: "Fairness", - description: "Distributes Worker capacity evenly across tenants or users so a burst from one caller does not starve the others.", + description: "Distributes Task dispatches across tenants or users so a burst from one caller does not starve the others.", }, ]} /> @@ -35,7 +35,7 @@ These patterns control how fast work executes, protect downstream services from **A downstream dependency has a fixed rate limit**: use [Downstream Rate Limiting](/design-patterns/downstream-rate-limiting) to cap throughput at the Worker. -**Urgent work must not wait behind bulk work**: use [Priority Task Queues](/design-patterns/priority-task-queues). +**Urgent work should move ahead of bulk work**: use [Priority](/design-patterns/priority-task-queues). **Multiple tenants share the same Workers**: use [Fairness](/design-patterns/fairness) to keep one tenant's burst from starving others.