Very very slow on Lenovo Yoga Tab Plus, Termux

[merkalev]

Description

Device: Lenovo Yoga Tab Plus (TB520FU)
CPU: Snapdragon 8 Gen 3
RAM: 12 GB LPDDR5X
OS: Android 16 (Termux environment)
Architecture: aarch64 (ARMv8)
Python: 3.14.6
WIMF Version: 2.2.0 (C++ native build)
SIMD: scalar (NEON not active)

Steps to reproduce:

1. Installed WIMF on Termux via pip install -e . --no-build-isolation
2. Patched third_party/zstd/zstd.c to handle Android Bionic's missing qsort_r
3. Built successfully with native C++ extension
4. Ran wimf_only_benchmark.py --input originals --threads 8
5. Observed extremely slow performance across all presets

Actual behavior:

• C++ native extension is active ("native": true)
• SIMD path is "scalar" (NEON not being used)
• Encoding is significantly slower than expected for Snapdragon 8 Gen 3
• Q10 Fast took several minutes

Expected behavior:

• NEON should be detected and used
• Performance should be at least 2-3× faster with NEON

Additional context:

• Android Bionic lacks qsort_r, which required patching Zstandard
• The patch works, but performance is still poor
• This may be due to NEON not being enabled in the build or runtime detection

Attachments:

• wimf.runtime_info() output:

{
  "architecture": "aarch64",
  "simd": "scalar",
  "native": true,
  "codec_version": "2.2",
  "effective_threads": 8
}

Related issues:

• Zstandard qsort_r compatibility on Android
• NEON detection/usage in WIMF

Priority: High (ARM/mobile is a key target platform)

Suggested fixes:

• Investigate NEON build flags for aarch64
• Check runtime NEON detection in WIMF
• Verify pybind11 build configuration for ARM SIMD

[merkalev]

Patch Instructions for Android/Termux

to get WIMF to build on Android, the bundled Zstandard source must be patched. android's Bionic C library does not provide qsort_r, so we replace it with qsort in the stableSort function.

---

method 1: manual edit (recommended)

Step 1: open the file:

nano third_party/zstd/zstd.c

Step 2: locate the stableSort function (around line 47400).

Step 3: add the following block before the #else /* C90 fallback.*/ section:

#elif defined(__ANDROID__) || defined(__BIONIC__)
    g_coverCtx = ctx;
    qsort(ctx->suffix, ctx->suffixSize, sizeof(U32),
          (ctx->d <= 8 ? &COVER_strict_cmp8 : &COVER_strict_cmp));

Step 4: save and exit (Ctrl+O, Enter, Ctrl+X).

Step 5: rebuild WIMF:

pip install -e . --no-build-isolation

---

method 2: sed one-liner (quick, what i used)

# Insert Android/Bionic support before the C90 fallback
sed -i '/#else \/\* C90 fallback\. \*\//i #elif defined(__ANDROID__) || defined(__BIONIC__)\n    g_coverCtx = ctx;\n    qsort(ctx->suffix, ctx->suffixSize, sizeof(U32),\n          (ctx->d <= 8 ? &COVER_strict_cmp8 : &COVER_strict_cmp));' third_party/zstd/zstd.c

Then rebuild:

pip install -e . --no-build-isolation