statedb: Minor fixes to write transactions by joamaki · Pull Request #204 · cilium/statedb

joamaki · 2026-09-25T10:03:25Z

Add tableIndex.abort() so on aborted transactions we drop dirty transaction state instead of holding onto it until next txn
Avoid unnecessary wakeups from a failed CompareAnd{Swap,Delete} by doing a lookup first instead of modify+re-insert. This is very slightly slower though didn't see impact to the reconciler benchmark which uses CompareAndSwap. This does help when we have multiple reconcilers acting on a single object where we do expect CompareAndSwap to often fail (e.g. first reconciler wins and the rest fail and instead do a lookup+insert to just update status) and this is semantically the more correct way.
Allow new tables to be registered during a WriteTxn by not blindily using the table count at time of WriteTxn()

AIL:3

WriteTxn() clones the root and Commit() publishes that clone after refreshing the entries of the tables it did not lock from the current root. If a table was registered while the write transaction was open the current root is longer than the clone and the newly registered table was dropped from the committed root, leading to an index out of range panic when accessing the table. Append the entries of any tables registered after the write transaction was created when committing. AIL:3 Signed-off-by: Jussi Maki <jussi@isovalent.com>

CompareAndSwap() and CompareAndDelete() first modified the primary index and then reverted the change if the guard revision did not match or the object did not exist. The revert left the index transaction dirty and the watch channels of the key and the root marked for closing, waking up watchers on commit even though nothing had changed. Check the guard revision with a lookup before modifying the index. This costs an extra lookup for CompareAndSwap() and CompareAndDelete() but failures now leave the indexes untouched. AIL:3 Signed-off-by: Jussi Maki <jussi@isovalent.com>

The index transaction of a partIndex is stored in the partIndex itself to avoid allocating it. The partIndex is shared with the committed root and Abort() did not clear it. This kept all the cloned nodes and the objects inserted in the aborted write transaction alive until the next write transaction against the index. Add abort() to the tableIndex interface and call it on the indexes of the locked tables on Abort(). partIndexTxn drops its transaction and for the other implementations it is a no-op. AIL:3 Signed-off-by: Jussi Maki <jussi@isovalent.com>

github-actions · 2026-09-25T10:13:12Z

$ make
go build ./...
go: downloading github.com/cilium/hive v1.0.4
go: downloading golang.org/x/time v0.15.0
go: downloading github.com/spf13/cobra v1.10.2
go: downloading github.com/spf13/pflag v1.0.10
go: downloading go.yaml.in/yaml/v3 v3.0.4
go: downloading github.com/cilium/stream v0.0.1
go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
go: downloading github.com/spf13/viper v1.18.2
go: downloading go.uber.org/dig v1.17.1
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading golang.org/x/text v0.14.0
STATEDB_VALIDATE=1 go test ./... -cover -vet=all -test.count 1
go: downloading github.com/stretchr/testify v1.11.1
go: downloading go.uber.org/goleak v1.3.0
go: downloading golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
ok  	github.com/cilium/statedb	308.826s	coverage: 79.7% of statements
ok  	github.com/cilium/statedb/index	0.005s	coverage: 48.6% of statements
ok  	github.com/cilium/statedb/internal	0.024s	coverage: 45.2% of statements
ok  	github.com/cilium/statedb/lpm	3.899s	coverage: 76.1% of statements
ok  	github.com/cilium/statedb/part	79.118s	coverage: 86.8% of statements
ok  	github.com/cilium/statedb/reconciler	0.318s	coverage: 93.3% of statements
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
go test -race ./... -test.count 1
ok  	github.com/cilium/statedb	38.405s
ok  	github.com/cilium/statedb/index	1.014s
ok  	github.com/cilium/statedb/internal	1.023s
ok  	github.com/cilium/statedb/lpm	2.793s
ok  	github.com/cilium/statedb/part	37.061s
ok  	github.com/cilium/statedb/reconciler	1.399s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 9V74 80-Core Processor                
BenchmarkDB_WriteTxn_1-4                      	  944526	      1265 ns/op	    790780 objects/sec	     656 B/op	      13 allocs/op
BenchmarkDB_WriteTxn_10-4                     	 2368598	       503.6 ns/op	   1985795 objects/sec	     352 B/op	       6 allocs/op
BenchmarkDB_WriteTxn_100-4                    	 2928951	       407.0 ns/op	   2456832 objects/sec	     317 B/op	       5 allocs/op
BenchmarkDB_WriteTxn_1000-4                   	 2662443	       456.6 ns/op	   2190199 objects/sec	     318 B/op	       5 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4     	 1606297	       700.9 ns/op	   1426703 objects/sec	     419 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_100Tables-4   	 1000000	      1034 ns/op	    1112 B/op	       4 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_1Table-4      	 1841636	       658.4 ns/op	     224 B/op	       4 allocs/op
BenchmarkDB_NewWriteTxn-4                     	 2048587	       583.9 ns/op	     200 B/op	       3 allocs/op
BenchmarkDB_WriteTxnCommit100-4               	 1000000	      1037 ns/op	    1096 B/op	       4 allocs/op
BenchmarkDB_NewReadTxn-4                      	560549728	         2.119 ns/op	       0 B/op	       0 allocs/op
BenchmarkDB_Modify-4                          	    2326	    508119 ns/op	   1968044 objects/sec	  342509 B/op	    6073 allocs/op
BenchmarkDB_GetInsert-4                       	    2197	    549633 ns/op	   1819396 objects/sec	  326505 B/op	    6073 allocs/op
BenchmarkDB_RandomInsert-4                    	    2575	    472025 ns/op	   2118533 objects/sec	  318501 B/op	    5073 allocs/op
BenchmarkDB_RandomReplace-4                   	    1240	    959739 ns/op	   1041950 objects/sec	  432322 B/op	    8089 allocs/op
BenchmarkDB_SequentialInsert-4                	    2683	    448662 ns/op	   2228851 objects/sec	  318500 B/op	    5073 allocs/op
BenchmarkDB_SequentialInsert_Prefix-4         	     574	   2100633 ns/op	    476047 objects/sec	 2794462 B/op	   41797 allocs/op
BenchmarkDB_Changes_Baseline-4                	    2113	    575301 ns/op	   1738220 objects/sec	  388267 B/op	    6164 allocs/op
BenchmarkDB_Changes-4                         	    1110	   1069696 ns/op	    934845 objects/sec	  604419 B/op	    9332 allocs/op
BenchmarkDB_RandomLookup-4                    	   30691	     39035 ns/op	  25618280 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_SequentialLookup-4                	   31393	     39159 ns/op	  25536960 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4           	    8314	    145412 ns/op	   6877026 objects/sec	  124952 B/op	    1026 allocs/op
BenchmarkDB_FullIteration_All-4               	    1102	   1079429 ns/op	  92641584 objects/sec	     104 B/op	       4 allocs/op
BenchmarkDB_FullIteration_Prefix-4            	    1030	   1138095 ns/op	  87866095 objects/sec	     136 B/op	       5 allocs/op
BenchmarkDB_FullIteration_Get-4               	     256	   4648446 ns/op	  21512566 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_Get_Secondary-4     	     120	   9878751 ns/op	  10122737 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_ReadTxnGet-4        	     258	   4627833 ns/op	  21608387 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4                	  788204	      1370 ns/op	        11.00 50th_µs	        15.00 90th_µs	        33.00 99th_µs	     807 B/op	      15 allocs/op
BenchmarkDB_WriteTxn_100_LPMIndex-4           	  535606	      2261 ns/op	    442296 objects/sec	    1574 B/op	      35 allocs/op
BenchmarkDB_WriteTxn_1_LPMIndex-4             	  146892	     13396 ns/op	     74651 objects/sec	   13830 B/op	      77 allocs/op
BenchmarkDB_LPMIndex_Get-4                    	     373	   3178136 ns/op	   3146498 objects/sec	       0 B/op	       0 allocs/op
BenchmarkWatchSet_4-4                         	 2378498	       498.7 ns/op	     296 B/op	       4 allocs/op
BenchmarkWatchSet_16-4                        	  745540	      1550 ns/op	    1096 B/op	       5 allocs/op
BenchmarkWatchSet_128-4                       	   87532	     13711 ns/op	    8904 B/op	       5 allocs/op
BenchmarkWatchSet_1024-4                      	    8438	    141573 ns/op	   73743 B/op	       5 allocs/op
PASS
ok  	github.com/cilium/statedb	43.920s
PASS
ok  	github.com/cilium/statedb/index	0.004s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/internal
cpu: AMD EPYC 9V74 80-Core Processor                
Benchmark_SortableMutex-4   	 5879108	       203.9 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/internal	1.203s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/lpm
cpu: AMD EPYC 9V74 80-Core Processor                
Benchmark_txn_insert/batchSize=1-4         	    2113	    568172 ns/op	   1760030 objects/sec	  822415 B/op	   13975 allocs/op
Benchmark_txn_insert/batchSize=10-4        	    3585	    358474 ns/op	   2789601 objects/sec	  369197 B/op	    6668 allocs/op
Benchmark_txn_insert/batchSize=100-4       	    3850	    315893 ns/op	   3165633 objects/sec	  329615 B/op	    6027 allocs/op
Benchmark_txn_delete/batchSize=1-4         	    1600	    739043 ns/op	   1353101 objects/sec	 1270473 B/op	   13976 allocs/op
Benchmark_txn_delete/batchSize=10-4        	    3327	    348607 ns/op	   2868562 objects/sec	  356419 B/op	    5769 allocs/op
Benchmark_txn_delete/batchSize=100-4       	    3847	    310499 ns/op	   3220620 objects/sec	  270754 B/op	    5038 allocs/op
Benchmark_LPM_Lookup-4                     	    8518	    139535 ns/op	   7166638 objects/sec	       0 B/op	       0 allocs/op
Benchmark_LPM_All-4                        	  133734	      8969 ns/op	 111497481 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_Prefix-4                     	  131996	      9098 ns/op	 109909679 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_LowerBound-4                 	  248960	      4781 ns/op	 104582115 objects/sec	     288 B/op	       2 allocs/op
PASS
ok  	github.com/cilium/statedb/lpm	12.030s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 9V74 80-Core Processor                
Benchmark_Set_Singleton_Create-4              	22173171	        54.16 ns/op	      24 B/op	       1 allocs/op
Benchmark_Set_Singleton_Has-4                 	98588782	        12.67 ns/op	       0 B/op	       0 allocs/op
Benchmark_StringMap_Txn_Insert-4              	    8836	    130064 ns/op	   7688545 items/sec	   98250 B/op	    1306 allocs/op
Benchmark_Uint64Map_Random-4                  	    2071	    590500 ns/op	   1693481 items/sec	 1321954 B/op	    6042 allocs/op
Benchmark_Uint64Map_Sequential-4              	    2119	    560045 ns/op	   1785572 items/sec	 1703089 B/op	    5753 allocs/op
Benchmark_Uint64Map_Sequential_Insert-4       	    2253	    517531 ns/op	   1932250 items/sec	 1695083 B/op	    4752 allocs/op
Benchmark_Uint64Map_Sequential_Txn_Insert-4   	   10000	    110520 ns/op	   9048120 items/sec	   90464 B/op	    2031 allocs/op
Benchmark_Uint64Map_Random_Insert-4           	    2154	    531351 ns/op	   1881994 items/sec	 1314794 B/op	    5022 allocs/op
Benchmark_Uint64Map_Random_Txn_Insert-4       	    6822	    181463 ns/op	   5510761 items/sec	  117192 B/op	    2411 allocs/op
Benchmark_Insert_RootOnlyWatch-4              	   10000	    108986 ns/op	   9175515 objects/sec	   75552 B/op	    2036 allocs/op
Benchmark_Insert-4                            	   10000	    107308 ns/op	   9318983 objects/sec	   76096 B/op	    2047 allocs/op
Benchmark_WatchReplace-4                      	   12428	     96497 ns/op	  10362999 objects/sec	   57305 B/op	    1006 allocs/op
Benchmark_Modify-4                            	   12769	     93931 ns/op	  10646102 objects/sec	   58056 B/op	    1007 allocs/op
Benchmark_GetInsert-4                         	    9814	    113682 ns/op	   8796499 objects/sec	   58056 B/op	    1007 allocs/op
Benchmark_Replace-4                           	34543525	        34.84 ns/op	  28704243 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4             	34351393	        35.38 ns/op	  28267763 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                             	 6739618	       177.2 ns/op	   5643977 objects/sec	      64 B/op	       3 allocs/op
Benchmark_txn_10-4                            	10707634	       110.6 ns/op	   9038374 objects/sec	      76 B/op	       2 allocs/op
Benchmark_txn_100-4                           	12375864	        98.36 ns/op	  10166353 objects/sec	      67 B/op	       2 allocs/op
Benchmark_txn_1000-4                          	10720042	       110.6 ns/op	   9044817 objects/sec	      65 B/op	       2 allocs/op
Benchmark_txn_delete_1-4                      	 5415940	       220.1 ns/op	   4543674 objects/sec	     632 B/op	       3 allocs/op
Benchmark_txn_delete_10-4                     	12047257	        97.53 ns/op	  10253159 objects/sec	     103 B/op	       1 allocs/op
Benchmark_txn_delete_100-4                    	13430185	        81.94 ns/op	  12203715 objects/sec	      35 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4                   	14846216	        79.51 ns/op	  12576402 objects/sec	      28 B/op	       1 allocs/op
Benchmark_Get-4                               	   61636	     20071 ns/op	  49823670 objects/sec	       0 B/op	       0 allocs/op
Benchmark_GetWatch-4                          	   49698	     24397 ns/op	  40989118 objects/sec	       0 B/op	       0 allocs/op
Benchmark_All-4                               	  172448	      7161 ns/op	 139646532 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_All-4                      	  159562	      7521 ns/op	 132961355 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_Next-4                     	  153591	      7772 ns/op	 128661026 objects/sec	     896 B/op	       1 allocs/op
Benchmark_Hashmap_Insert-4                    	   17277	     69544 ns/op	  14379459 objects/sec	   74264 B/op	      20 allocs/op
Benchmark_Hashmap_Get_Uint64-4                	  153190	      7841 ns/op	 127542572 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4                 	  125588	      9548 ns/op	 104731853 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Delete_Random-4                     	      82	  15257610 ns/op	   6554106 objects/sec	 2539423 B/op	  102756 allocs/op
Benchmark_find16-4                            	243242143	         5.078 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex16-4                       	79016991	        15.93 ns/op	       0 B/op	       0 allocs/op
Benchmark_find64-4                            	284208284	         4.222 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex64_hit-4                   	283987495	         4.223 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex64_miss-4                  	283887812	         4.226 ns/op	       0 B/op	       0 allocs/op
Benchmark_find4-4                             	422426360	         2.836 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex4-4                        	337526710	         3.538 ns/op	       0 B/op	       0 allocs/op
BenchmarkSmallWriteTxn/updates_1-4            	  456516	      2612 ns/op	    3529 B/op	       4 allocs/op
BenchmarkSmallWriteTxn/updates_2-4            	  298098	      3572 ns/op	    4742 B/op	       6 allocs/op
BenchmarkSmallWriteTxn/updates_4-4            	  208902	      5552 ns/op	    7155 B/op	      11 allocs/op
BenchmarkSmallWriteTxn/updates_8-4            	  123474	      9563 ns/op	   11927 B/op	      20 allocs/op
BenchmarkSmallWriteTxn/updates_16-4           	   66771	     17535 ns/op	   21289 B/op	      38 allocs/op
BenchmarkAtomicWatchPointerFirstChannel-4     	20793639	        57.34 ns/op	     120 B/op	       2 allocs/op
BenchmarkAtomicWatchPointerChannel-4          	485718862	         2.466 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/part	57.489s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 1.56 seconds (batch size 1000)
Throughput 640498.42 objects per second
506MB total allocated, 5010325 in-use objects, 231MB bytes in use

joamaki added 3 commits September 25, 2026 11:53

joamaki requested a review from a team as a code owner September 25, 2026 10:03

joamaki requested review from bimmlerd and removed request for a team September 25, 2026 10:03

joamaki force-pushed the pr/joamaki/minor-writetxn-fixes branch from b3dd6da to bf51338 Compare September 25, 2026 10:11

joamaki mentioned this pull request Sep 25, 2026

statedb: Minor WriteTxn optimizations #205

Open

bimmlerd approved these changes Sep 29, 2026

View reviewed changes

joamaki merged commit 02a833a into main Sep 30, 2026
1 check passed

joamaki deleted the pr/joamaki/minor-writetxn-fixes branch September 30, 2026 07:13

Navigation Menu

statedb: Minor fixes to write transactions - #204

statedb: Minor fixes to write transactions#204
joamaki merged 3 commits into
mainfrom
pr/joamaki/minor-writetxn-fixes

joamaki commented Sep 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

joamaki commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joamaki commented Sep 25, 2026 •

edited

Loading

github-actions Bot commented Sep 25, 2026 •

edited

Loading