RabbitMQ Queue Migration Plugin

A RabbitMQ plugin for migrating mirrored classic queues to quorum queues in RabbitMQ 3.13.x clusters.

Overview

• Two-phase migration algorithm with message-by-message transfer
• Automatic binding preservation and snapshot-based recovery
• Selective migration by queue name or batch size
• EBS or tar-based snapshots before migration
• HTTP API for control and monitoring

Prerequisites

• RabbitMQ 3.13.x
• Multi-node cluster (3+ nodes recommended)
• rabbitmq_management plugin enabled
• rabbitmq_shovel plugin enabled
• Mnesia metadata store (Khepri must NOT be enabled)

Note: The setting quorum_queue.property_equivalence.relaxed_checks_on_redeclaration = true must be enabled in rabbitmq.conf before starting migration; this is validated during pre-migration checks. See Client Redeclaration for why.

Installation

rabbitmq-plugins enable rabbitmq_queue_migration

This plugin must be enabled on all cluster nodes for consistent behavior.

Known Limitations

Open-Source RabbitMQ 3.13.7

Three upstream RabbitMQ issues are known to affect migrations performed by this plugin when running on open-source RabbitMQ 3.13.7. There are no effective mitigations within the plugin. Migrating empty queues prevents these issues entirely; migrating shorter queues reduces their likelihood. Amazon MQ for RabbitMQ broker builds on the 3.13 series include backports of the fixes.

See OSS 3.13.7 Known Issues for details.

Message TTL

Message TTL is the most common cause of a failed, rolled-back migration, and it takes two forms.

Warning

This plugin cannot detect per-message TTL set by publishers. Messages with the expiration property may expire during migration, causing a message_count_mismatch failure. This is especially common on dead-letter and _error queues.

Warning

A queue-level TTL is caught by the readiness check, but deleting the policy does not make the queue safe to migrate. A queue-level TTL (x-message-ttl argument or message-ttl policy) stamps an absolute expiry onto each message when it is enqueued, and that stamp survives removal of the policy. If you delete the policy to clear the check and migrate immediately, the already-stamped messages still expire mid-migration and the queue fails with message_count_mismatch.

See Message Loss and Verification for why it happens and how to handle it (migrate with allow_message_ttl, set a tolerance, or drain first).

Documentation

Once installed, follow Running a Migration for the full operator path. Otherwise start with the guide for what you are doing:

Plan a migration - understand how it works and what will bite you

• Migration Guide - two-phase process, queue eligibility, argument conversion, policy and client-redeclaration behavior
• Message Loss and Verification - message-count verification, tolerance, and per-message TTL (the most common cause of a failed migration)

Run a migration - the operator path, start to finish

• Running a Migration - check, start, monitor, interrupt, and every start option in one place
• Skip Unsuitable Queues - migrate the majority now and address problem queues later

Recover from a problem

• Troubleshooting - error messages, diagnostics, and recovery

Reference

• HTTP API - complete request/response schemas
• API Examples - practical API usage examples
• Configuration - all configuration parameters and defaults
• Snapshots - snapshot modes and configuration
• EC2 Setup - AWS EC2 and IAM configuration when EBS snapshots are used
• Performance - real-world migration timing data
• OSS 3.13.7 Known Issues - upstream issues affecting open-source builds

See the docs/ directory.

Contributing

See CONTRIBUTING for contribution guidelines.

Security

See CONTRIBUTING for security issue reporting.

License

This project is licensed under the Apache-2.0 License.