(dsc) dyld_shared_cache applying multiple images progressively slow

[rani-i]

Version and Platform (required):

• Binary Ninja Version: 5.2.8128-dev Personal, 86ad75d4
• OS: macos
• OS Version: 15.6
• CPU Architecture: arm64

Bug Description:
Loading multiple images from a dyld_shared_cache is progressively slower and eventually stops functioning.
My workflow is to defer initial analysis so I can apply a large number of DSC images first, then later analyze only MLIL. With analysis disabled, I iterate over ~100 images and call apply_image for each. After ~30 images, per-image load time grows from seconds to tens of seconds/minutes and the UI becomes unresponsive/frozen. This reproduces on both the stable and dev builds.
Note: This was discussed with your team on Slack, I was advised to file an issue because this behavior is not intended.

Steps To Reproduce:

1. Open the DSC.

2. Disable analysis (analysis-on-hold). (Screenshot attached.)

3. Apply many images in a loop:

   for idx, image in enumerate(images_to_load):
       print(f'[+] {idx+1}/{len(images_to_load)} Loading {image}')
       if image:
           start = time.time()
           shared_cache.apply_image(bv, image)
           end = time.time()
           total_time += (end - start)
           print(f'Time:{end-start} (Total Time:{total_time})')

4. Observe that by ~30 images, apply_image takes ~30s–minutes per image and BN becomes frozen.

5. Close the BNDB and retry a single problematic image in a fresh session: the same apply_image takes ~0.5s. (Screenshot attached.)

Expected Behavior:
With analysis disabled, applying many DSC images should remain fast and consistent (similar to the ~0.5s single-image timing), without progressive slowdown or freezing.

Screenshots:

Initial analysis disabled

Applying multiple images

Applying single image

Binary:
Dyld shared cache - 23A5297i__iPhone17,2

[WeiN76LQh]

Might be in part caused by the same issue mentioned in #6271

[bdash]

@rani-i Can you share what set of images you're loading and what the specs are of the machine you're using?

I do see the time taken to load each image increasing, but in my testing with the same dev build loading 400 images from the cache I never see it take more than 2-3 seconds per image. 400 images finish loading in 5-6 minutes. This isn't great, but it's nowhere near as bad as what you're seeing. I'd like to understand what's different between how I'm testing to understand why you're seeing it be so much slower than that.

I have been testing with:

images_to_load = shared_cache.images[:400]

It is possible that loading specific images makes the loads of subsequent images significantly slower.

From looking at profiling data,AddUserSection is a contributing factor, but the runtime is dominated by work performed below EndBulkModifySymbols. The expensive part of the work that function does scales with the total number of symbols. Ideally it would scale with the number of symbols that were added/modified/removed since the corresponding BeginBulkModifySymbols. I have some ideas on how to improve that.

That said, I would really like to understand the gap between what I observe and what @rani-i is seeing to make sure I'm looking at the right thing.

[bdash]

This should be much faster in 5.2.8187-dev and newer. I addressed the two main bottlenecks: EndBulkModifySymbols no longer gets progressively slower as more symbols were added, and the Objective-C support no longer repeatedly adds the same types to the view (#7282).

With those changes I see a test scenario of loading 400 images that was previously taking over 8 minutes on my machine now takes around 45 seconds. The remaining bottleneck is now AddUserSection which is tracked by #6271.