[asp24]

The ForEach functions of Array and Object give the callback its Iter by value. Each call copied the Iter immediately after Advance wrote it. This is the store-forwarding stall that #8 removed from AdvanceIter. This change removes it from ForEach.

Commits

1. Add a benchmark for ForEach. Before this change, no benchmark used ForEach. The new rows IterRead/*/foreach go into all containers of each document through the ForEach callbacks. They do not allocate memory.
2. Read the next element before ForEach calls fn.
   • The copy of the Iter used 16-byte loads on the 8-byte stores of Advance. The store queue cannot forward such loads, and they wait for the cache. On canada, the IPC was 2.4. On the same tape, the iters walk had an IPC of 5.4.
   • Now the loop reads the next element before it calls fn for the current element. fn gets a copy from before that read.
   • The loop does the read itself, without a call to Advance. A version with two calls to Advance for each object member made twitter 8% slower.
   • Because fn can write to the tape, the loop keeps a read only if its words did not change after fn: one word for an element, three words for a member. If a word changed, the old step runs after fn. The old step also does NOPs, errors and the end of the tape.
   • With a key filter, Object.ForEach keeps its old loop. That loop stops at the last key in the filter. A read after that key reads all of the remaining object.

Time, against the parent

Measured with benchmarks/perfgate/ab.sh: four code layouts for each side, six rounds, twelve cores.

document	IterRead/*/foreach
canada	-34.0%
apache_builds	-31.8%
citm_catalog	-30.9%
twitter	-29.6%
instruments	-28.3%
update-center	-27.9%
github_events	-26.9%
geomean	-29.9%

The other rows of the read group change by -1.3% to +3.3%. No row is past its threshold. The allocations do not change.

Testing

foreach_diff_test.go keeps the old loops as references and compares them with the new functions. The walks start from each tag on the tapes of advance_diff_test.go: the corpus, the corpus with deletions, the corpus cut short, the corpus with bad container ends, and edge tapes. A new edge tape has a root as the value of a member.

Some callbacks write to the tape:

• to their own element;
• to the next element, which the loop read before fn;
• to the value bits of the next element, with the same tag;
• to the length word of the next key;
• on the NOPs after the element.

The test compares each Iter that fn gets, the error, each panic and the tape after the walk. The test failed for each of thirteen intentional errors in the new code. These errors include each check in the guard and in the read.

Found, not changed here

Advance and AdvanceIter skip one word too many after a NOP. DeleteElems, SetNull and Deserialize count a NOP skip from the NOP word. AdvanceInto, PeekNext and NextElementBytes read it in the same way. Advance and AdvanceIter count it from the next word. Example: DeleteElems removes 2 from [1,2,3,4], and then Array.ForEach gives only 1. Upstream has the same code. A different PR will correct it. This PR keeps the current behavior of Advance.

🤖 Generated with Claude Code