Test reorgs of the binary trie post flip

[kevaundray]

Observation

At some block N, we would switch from the MPT to the PBT. Ideally nodes do not drop all MPT related state because if we reorg back to block N-1 we will be at the MPT block again. Nodes should in the worse case drop state after block N has been finalised.

Suggestion

This is a tracking issue for testing this reorg edge case

[kevaundray]

We should also test reorgs post flip to ensure that the PBT handles reorgs like the MPT did

[kevaundray]

Another case is:

A node is snap syncing with the MPT, and the tip of the chain reaches the pivot block N where we need to switch to PBT.

There are two behaviours that the node could do:

• continue snap syncing the MPT, the MPT will be frozen at block N-1, so we can start serving that block immmediately. The node would then continue converting the mpt to pbt in the background.

• Stop snapsyncing the MPT and immediately start snap syncing the PBT.

I lean towards the latter, however note that if a node snapsyncs to the PBT at block N, and we reorg back to N-1, then they may need to start snapsyncing the MPT again.

Thinking out loud: One solution here is if the node synced the MPT at block N-1 for reorg purposes, and then synced block N. This means that any node that has the PBT also has the MPT state for N-1.