Merge/upstream main 2025 10 07 - #17
Merged
Merged
Conversation
Replicates cowprotocol#3599 after reverting it in cowprotocol#3627, which caused some memory issues after updating the lockfile. This version uses downgraded aws lib versions to avoid updating the lockfile. Depends on cowprotocol/ethcontract-rs#982 Tested on mainnet-shadow, looks fine.
# Description Fixes the lints generated [here](https://github.com/cowprotocol/services/pull/3592/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557). # Changes - `as` -> `AS` - remove `CMD` line that only `echo`'s something ## How to test Run CI and check for these lints
# Description Migrates 0x SC to `alloy-rs` bindings. # Changes - [ ] Binds `IZeroEx` SC using `alloy`. - [ ] Removes legacy bindings from the `build.rs` file. - [ ] For e2e tests, a different from the original alloy provider's signer needs to be used. Currently, alloy doesn't provide a concise way to achieve that. An issue was opened: alloy-rs/alloy#2829. This PR contains a workaround that uses a new provider with the desired wallet for 0x bindings. - [ ] Drops GOERLI from the bindings tests. - [ ] Updates alloy to 1.0.30 since CI fails without it(e.g. https://github.com/cowprotocol/services/actions/runs/17409474760/job/49422717398) ## How to test Existing tests --------- Co-authored-by: MartinquaXD <martin.beckmann@protonmail.com>
…atic to linear time complexity by using a HashMap. (cowprotocol#3621) ## Description This function is building a vector of fee policies (`fee_policies`) by looking up orders in an auction. Currently this is doing a linear search for each order ID. The change I'm proposing is to use a HashMap lookup for better performance and also I think slightly cleaner logic. The scale of the performance benefit is a bit context dependent, i.e, average auction size, but I'll share the theoretical perspective. ## The Theoretical Advantage The theoretical advantage here is from an algorithmic perspective, namely O(n²) -> O(n). The current implementation is ``` for order_id in ranking.ranked().flat_map(...).unique() { // m iterations match auction.orders.iter().find(|order| &order.uid == order_id) { // O(n) search // ... } } ``` m = number of unique order IDs from solutions n = number of orders in auction Total complexity: O(m × n) = O(n²) in the worst case (when m ≈ n) Whereas in this case, we'd be doing ``` let order_lookup: HashMap<_, _> = auction.orders.iter() // O(n) to build .map(|order| (order.uid, order)) .collect(); let fee_policies: Vec<_> = ranking.ranked() // m iterations .flat_map(...) .unique() .filter_map(|order_id| { match order_lookup.get(order_id) { // O(1) lookup // ... } }) .collect(); ``` HashMap creation: O(n) Lookups: O(m × 1) = O(m) Total complexity: O(n + m) = O(n) when m ≤ n --------- Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
…ks (cowprotocol#3632) # Description Some slow nodes time out when there are too many pool IDs when we query the uniswap v3 subgraph for `Tick`s. This PR adds the option to limit the number of such items in the query which means running multiple queries with fewer pool IDs in `where` to achieve the same effect. This can't be done in parallel as it overloads these slow nodes. In the end I found a [fast graph](https://thegraph.com/explorer/subgraphs/FiJDXMFCBv88GP17g2TtPh8BcA8jZozn5WRW7hCN7cUT?view=About&chain=arbitrum-one) so this change is not needed, but if this one ever stops working we might be happy to have the option. # Changes - [x] Add a new driver param that limits how many pool ids are added to a single query ## How to test Start driver with graph-url against a slow subgraph (e.g. `F85MNzUGYqgSHSHRGgeVMNsdnW1KtZSVgFULumXRZTw2`) and watch it fail. Then set max pool ids to 10 and see it succeed. --------- Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
Removes the dummy alloy provider since it was overseen that a similar function already exists in `ethrpc::mock::web()` funciton.
# Description To continue with the alloy migration, a deployed block info is required for some of the SC, list GPV2Settlement, BalancerV2.x, others. # Changes Extend the bindings macro and InstanceExt interface with that data. Now, in the bindings! macro, added a way to provide either only address or a tuple with (address, deployment block).
Adds alloy's BlockId and BlockNumber conversions, which are required for further migration.
# Description Adds an ability to retrieve SC function selectors required across the code base: https://github.com/cowprotocol/services/blob/e8fc0ac7e32d9046523c7d051f430e8d9cf2d5be/crates/autopilot/src/domain/settlement/transaction/mod.rs#L202 https://github.com/cowprotocol/services/blob/97ddf0c55b1ab7a41bd026ed5bfe7a95fd86f9ed/crates/contracts/src/vault.rs#L21 # Changes Parse the same ABI JSON used for the SC bindings and get function selectors by its name.
Adds the ability to retrieve SC ABI functions that are required in some cases to encode some inputs/outputs. Will be used in some tests in follow-up PRs.
Co-authored-by: ilya <ilya@cow.fi>
# Description Currently our axum `/solve` handler immediately parses the JSON request body into the DTO object. This is very convenient but also pretty suboptimal. Because the driver was built such that it can support multiple connected solvers the auction pre processing logic was written such that it only happens once per auction and the results get shared by all the solvers. That means all but 1 `/solve` requests per auction get fully parsed and then discarded. Also since that happens on the main tokio runtime (instead of a blocking task) every incoming request blocks the runtime for a considerable amount of time. # Changes - request handler now takes a raw `String` - parsing into the `dto` representation was moved into the auction pre-processor - also parsing happens in a task that is optimized for blocking - to detect whether a request is duplicated we simply to a string comparison on the requests (the autopilot sends identical requests to all drivers). It was determined via a benchmark that this is considerably faster than just parsing out the auction id from the request. - due to not having the `auction_id` available immediately we now init the tracing info span with `auction_id = field::Empty` and fill in the value immediately after parsing the request. There should be no important logs without the auction id present. - the `deadline` type of the `domain::Auction` needed to be changed because each solver may have different timeout settings which does not lend itself nicely to sharing the same auction struct. In fact sharing the same deadlines for all solvers was actually a bug that also gets resolved by this PR. Now we just store the auction deadline in the domain object and each solver can compute their own unique deadlines individually. ## How to test I ran a test on shadow mainnet which reduced the CPU time spent deserializing from `~40%` to `4%`. Before <img width="1920" height="827" alt="Screenshot 2025-09-01 at 14 49 11" src="https://github.com/user-attachments/assets/dbf2bcd6-ca65-4460-9d1b-eb7fc203dc37" /> After <img width="1920" height="955" alt="Screenshot 2025-09-01 at 14 49 24" src="https://github.com/user-attachments/assets/9fa9761f-b30b-461a-a0a7-95626cb2225e" /> Also the metrics show a ~50% reduction in CPU usage of the shadow mainnet driver <img width="765" height="273" alt="Screenshot 2025-09-10 at 07 21 03" src="https://github.com/user-attachments/assets/98db0241-5080-4d7c-8ebf-b7206f47a69e" />
# Description Checking signatures is currently one of the slower auction building steps. This PR implements a couple optimizations to speed things up. # Changes - avoid `Itertools::partition` as that moves around every item even if we only need to move a few (e.g. in this case we don't need to split ALL orders into pre-signed and "unknown" - instead we just work with the orders that are actually relevant for this processing step) - changed `validate_signatures()` to only handle a single check instead of multiple - that made it a lot less awkward to figure out for which order the check failed. - updated internal helper functions to take an owned `SignatureCheck` so we don't have to clone the `signature` and `interaction` bytes twice - used `FuturesUnordered` instead of `join_all` to avoid allocating a vector for all the results - instead we just await each future concurrently and push the invalid orders into the result vector ## How to test updated the existing unit test unfortunate the performance improvement of this PR can only really be measured by deploying to mainnet prod as all the staging environments don't have enough data to matter and the shadow environment does not hit this code path.
# Description Migrates `BalancerV2BasePool` bindings to alloy. Also, starts using `BalancerV2Vault` alloy bindings in the liquidity fetching logic, which doesn't touch the GPv2Settlement SC stuff. This is required to avoid non-pretty workaround to provide alloy provider for the `BalancerV2BasePool` instance here: https://github.com/cowprotocol/services/pull/3642/files#diff-c3c048247f32ab19c889ca44622f8857e743f2901b2ad2b7b51fb1439835962fR64-R65 `BalancerV2Vault` ethcontract instances remain since they are still used in the core SCs logic, which will be migrated much later. ## How to test Existing tests.
…col#3645) Co-authored-by: José Duarte <duarte.gmj@gmail.com>
# Description This PR is really stupid but unfortunately necessary at the moment... Some tokens are weird to route optimally and only few solvers support them. However, anybody can create a uniswap pool for these tokens which may cause any good solver to be able to route these tokens automatically. This is generally a good thing except in one particular edge case: If the "good" solver is not able to provide calldata with their quote that can be verified but the "bad" solver's calldata verifies just fine we'll currently show the quote from the bad solver despite having very high confidence that the other solver is actually better and knows how to route the tokens. To avoid showing terrible prices in this VERY specific scenario we need an escape hatch to conditionally bypass the quote verification. I decided to go for a simple token list. If either the sell or the buy token show up in the list that we define we'll skip the quote verification altogether. That way quote from all solvers (good and bad) will be unverified which levels the playing field and we again pick the quote based on the highest out amount. --------- Co-authored-by: ilya <ilya@cow.fi>
Adds the MutWallet to enable adding signers when needed. This is a "solution" to alloy-rs/alloy#2829 Since Alloy's Provider is built on layers and in the end it seals them up, it becomes impossible to change the signing wallet. The solution is to share the wallet behind the scenes through the classic Arc + lock (Rw in this case) The main challenge here was running tokio's locks in "sync" contexts: During tests we have "current_thread" so we can't run blocking operations (.blocking_read), we can however, run a separate thread and wait for it (ironic because it blocks the thread anyway — though this is ok because it's for tests) Outside tests, we're running the multi-thread runtime, thus spawning an extra thread and waiting is a waste because we're blocked waiting for it anyways, using block_in_place we can signal the runtime that we're in a pickle and we're going to block, evicting other tasks and keeping the system chugging along
# Description Fetching balances is currently the biggest time sink in the auction pre-processing step. Interestingly the autopilot is able to it way faster with many more orders. So the easiest way to speed things up in the driver is to use the same component the autopilot uses for that. Instead of fetching the balances only after we receive the auction we now have a background task that keeps track of used balances and updates them on every new block. If an auction contains a new order that has no balance yet the balance will be fetched on demand. # Changes Use caching account balance fetching in the driver. Unfortunately the code is a bit ugly because the `shared` code uses different types than the `driver` code so there are quite a few conversions needed. By using the same data types we could probably reduce the times a bit more by avoiding expensive conversions and clones. ## How to test ran a test on shadow mainnet and the metrics show significant improvements. Total pre-processing time goes from the 750-1000ms bucket to primarily the 250-500ms bucket <img width="1196" height="362" alt="Screenshot 2025-09-11 at 12 59 06" src="https://github.com/user-attachments/assets/0206bc5b-a72b-4fb2-9d88-33a5d5ef4e2d" /> balance fetching goes from 500-750ms to 50-100ms <img width="1189" height="363" alt="Screenshot 2025-09-11 at 12 59 18" src="https://github.com/user-attachments/assets/794e3f0a-40fa-4d6e-8662-4c52b37f7906" /> solver time to compute solutions goes from 9s to 9.5s <img width="1191" height="340" alt="Screenshot 2025-09-11 at 12 59 25" src="https://github.com/user-attachments/assets/9774ad6c-f65c-4608-ae1f-0f2f5dd20024" />
# Description Currently saving solver competitions is quite slow (100ms-500ms with outliers going over 3s!). AFAICS the way we handle uploading the JSON is very suboptimal: 1. we do `serde_json::to_value()` while `sqlx` later does `serde_json::to_string()` - we might as well directly do the conversion to string ourselves 2. we block the runtime while we serialize the solver competition JSON. Instead we can offload the serialization to a blocking task and let it already do work while we continue with the other DB queries. <img width="1190" height="297" alt="Screenshot 2025-09-12 at 22 03 01" src="https://github.com/user-attachments/assets/963f710e-ac74-4f92-bc68-da4bcad2fd51" /> # Changes Upload the JSON by inserting a JSON string directly. Offload the serialization to a blocking task and do the solver competition at the very end so we can already finish all the other DB queries in the mean time. This required me to take the `SolverCompetition` as an owned value which also enabled me to turn some `.iter()` into `.into_iter()` which gives the optimized more opportunities to avoid cloning memory. ## How to test updated the `roundtrip` test to work with the string approach
# Description This is the first prerequisite for the upcoming PRs that generalize the event handling module, making it possible to integrate with Alloy smart contract instances. Instead of relying on `ethcontract-rs`–specific types in the interface, the trait is now generic and can work with primitives from either `ethcontract-rs` or `alloy`. All the interfaces in the `event_handling.rs` file don't depend on the `ethercontract-rs` crate, meaning we can use any underlying implementation in the future. Some common logic was extracted to the `EthcontractEventRetrieving` trait. This is required to remove `ethcontract-rs` crate dependency from the `EventRetrieving` trait and deduplicate the code. The change looks pretty straightforward, but you can't imagine how many failed attempts were made to reach its current state 🤕
# Description closes cowprotocol#3404 Add RPC request ID logging to HTTP transport layer for improved observability and debugging. # Changes <!-- List of detailed changes (how the change is accomplished) --> - [ ] ... - [ ] ... ## How to test <!--- Include details of how to test your changes, including any pre-requisites. If no unit tests are included, please explain why and how to test manually 1. 2. 3. --> <!-- ## Related Issues Fixes cowprotocol#3404 --> --------- Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com> Co-authored-by: José Duarte <duarte.gmj@gmail.com>
# Description This PR adds a helper contract called BalancerQueries to the contracts crate. Giving a bit more context, currently the Balancer solver relies on SOR quotes only, which might be running on slightly out-of-date data from the Balancer API. Being able to update and validate amounts with an extra on-chain query swap step would be ideal. On Balancer V3 it's possible to query swaps with the BalancerRouter, which is already present in the contracts crate, but on Balancer V2 the best way is to use this BalancerQueries helper contract, that is not yet available. # Changes I simply followed the steps to add a new contract provided in the [contracts readme.md file](https://github.com/cowprotocol/services/blob/main/crates/contracts/README.md) > - In vendor.rs extend ARTIFACTS with the package and contract name. > - Run the vendor binary for example with (cd crates/contracts; cargo run --bin vendor --features bin). This creates a new json file for this contract in the artifacts folder. > - In build.rs add a generate_contract call for the contract. This creates the ethcontract generated rust code file. > - In lib.rs add an include! call for the contract. This imports the rust code into the library. ## How to test Not sure this addition requires tests to be added. If you think it's necessary, please share a template or even some example I can follow, because I couldn't find test examples from the existing contracts. ## Related Issues Gettin this PR merged and released will unblock gnosis/solvers#161 --------- Co-authored-by: José Duarte <duarte.gmj@gmail.com>
# Description I accidentally merged the cowprotocol#3662 PR into the wrong branch, so this PR resurrects all the changes. Migrates `UniswapV3Pool` SC bindings to alloy. ## How to test Existing test + mainnet shadow. --------- Co-authored-by: José Duarte <duarte.gmj@gmail.com>
# Description In cowprotocol#3709 we used the wrong address for the Hooks Trampoline contract on Gnosis.
Co-authored-by: ilya <ilya@cow.fi>
…wprotocol#3727) This reverts commit adb9dc6. # Description After today's deployment we started having issues where transaction hashes are not present for settlements. This PR is a suspect. More context: https://cowservices.slack.com/archives/C0361CDD1FZ/p1759252600242209
Reverts cowprotocol#3707, since the Swapper SC remains in use in the gnosis/solvers repo: https://github.com/gnosis/solvers/blob/main/src/infra/dex/simulator.rs#L43 Also, migrates its rust bindings to alloy. Added a hotfix label to update the gnosis/solvers repo. This hotfix is not intended for deployment. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…col#3451) # Description <!--- Describe your changes to provide context for reviewers, including why it is needed --> ## Context and why is this needed Liquorice is an RFQ system that aggregates quotes from PMMs (Private Market Makers) and provides this liquidity to solvers. PMMs benefit from knowing that their quotes will be used in settlement as early as possible because they can hedge earlier, manage their risks better, and as a result provide better prices. Right now PMMs can find out that their quote was used only after settlement already occurred on-chain. This situation can be improved drastically if the CoW driver were communicating this information before sending transaction on-chain. ## Solution PR introduces generic liquidity sources notification logic in CoW driver and adds [Liquorice](https://liquorice.gitbook.io/liquorice-docs) liquidity source as the first subscriber. Before submitting settlement transaction on-chain, driver sends notification to configured subscribers that the settlement is about to happen. In case of Liquroice, driver extracts Liquorice [RFQ IDS](https://liquorice.gitbook.io/liquorice-docs/for-market-makers/basic-market-making-api#id-3.-receiving-rfq) from the CoW settlement interactions and sends HTTP request to Liquorice API. # Changes ## `crates/contracts` Added Liquorice [settlement contract](https://liquorice.gitbook.io/liquorice-docs/links/smart-contracts) used for interactions matching and calldata decoding ## `crates/driver` Implemented generic functionality used no notify liquidity sources about settlement. This functionality perhaps could've been implemented on top of existing `infra/notify` and `infra/liquidity` modules, however this would require significant refactoring, so for the sake of simplicity it was added as a standalone set of modules - Added new optional configuration for liquidity sources notifiers - Implemented `notifier` module with `liquorice` notifier as the first one. - Added `notifier.settlement(...)` invocation in `crates/driver/src/domain/competition/mod.rs` before settlement submitted onchain ## `crates/e2e` - Added `forked_node_liquidity_source_notification_mainnet` test with mocked Liquorice API and transaction calldata ## How to test <!--- Include details of how to test your changes, including any pre-requisites. If no unit tests are included, please explain why and how to test manually 1. 2. 3. --> 1. `FORK_URL_MAINNET="...." cargo test -p e2e forked_node_liquidity_source_notification_mainnet -- --ignored` 2. Unit tests in `crates/driver/src/infra/notify/liquidity_sources/liquorice/notifier.rs` <!-- ## Related Issues Fixes # --> ## Related Issues Implements cowprotocol#3452 Depends on cowprotocol#3643
# Description Add BNB and Lens to OpenAPI definitions so it appears in Swagger.
# Description Migrates BalancerQueries to alloy bindings. This SC is used in the gnosis solvers repo. That would require some tiny changes in that repo to support this.
…#3723) # Description The recent changes to autopilot and orderbook allow to specify DB_READ_URL and DB_WRITE_URL. The former is used only for non mutating database operations. The plan is to point DB_READ_URL to our read-replica (giving the possibility of slightly outdated data). The DB_WRITE_URL points to the main database. # Changes Added reuse of connection pool in case of DB_READ_URL and DB_WRITE_URL having the same values. ## How to test 1. Run E2E tests 2. Check the deployment on staging --------- Co-authored-by: ilya <ilya@cow.fi>
# Description Migrates ILiquoriceSettlement bindings to alloy.
# Description Some docs for order validation functions # Changes <!-- List of detailed changes (how the change is accomplished) --> - [ ] custom_interactions - [ ] validate_app_data - [ ] simulate - [ ] validate_signature ## How to test <!--- Include details of how to test your changes, including any pre-requisites. If no unit tests are included, please explain why and how to test manually 1. 2. 3. --> <!-- ## Related Issues Fixes # -->
Co-authored-by: Martin Magnus <martin.beckmann@protonmail.com>
# Description Drops the `auction_orders` table, which contains duplicated data in the `competition_auctions` table. The same data is used there: https://github.com/cowprotocol/services/blob/979e924dae38ba18bf398447e399092867edaf3f/crates/autopilot/src/infra/persistence/mod.rs#L302-L306 ## How to test Existing tests. No usages apart from this repo: https://github.com/search?q=org%3Acowprotocol+auction_orders+-repo%3Acowprotocol%2Fservices&type=code The following query returns 0, which should tell how many `auction_orders.auction_id` do not exist in `competition_auctions.id`. ```sql SELECT COUNT(*) FROM auction_orders ao WHERE NOT EXISTS ( SELECT 1 FROM competition_auctions ca WHERE ca.id = ao.auction_id ); ```
|
I have read the CLA Document and I hereby sign the CLA 0 out of 13 committers have signed the CLA. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Changes
How to test