Skip to content

Docs: warn that pre-applying arabic_reshaper + python-bidi corrupts text on a Raqm build #9907

Description

@Syamjith-NK

Summary

The docs say libraqm "provides bidirectional text support (using FriBiDi), shaping (using HarfBuzz)". That is accurate, but nothing says what happens if the caller also does those two steps — which is what most Arabic-with-Pillow advice online recommends:

text = get_display(arabic_reshaper.reshape(text))   # the widely-copied recipe
draw.text((x, y), text, font=font)

On a Raqm build this is applied twice and the text comes out wrong. I would like to propose a short docs note, because the failure is silent to anyone who does not read Arabic: it renders cleanly, it just says something else.

Environment

Reproduced on Pillow 12.3.0 (and 12.2.0), Python 3.14, macOS.
raqm 0.10.5, harfbuzz 14.2.1, fribidi 1.0.16.

Reproducer

from PIL import Image, ImageDraw, ImageFont, features
import arabic_reshaper
from bidi.algorithm import get_display

print(features.check("raqm"), features.version("raqm"))   # True 0.10.5

FONT = "/System/Library/Fonts/SFArabic.ttf"   # any Arabic-capable font
text   = "الإمارات"                            # "the Emirates"
recipe = get_display(arabic_reshaper.reshape(text))

for name, s in (("plain", text), ("recipe", recipe)):
    img = Image.new("L", (700, 110), 255)
    ImageDraw.Draw(img).text((20, 15), s, font=ImageFont.truetype(FONT, 64), fill=0)
    img.save(f"out_{name}.png")

out_plain.png is correct. out_recipe.png is the same word rendered reversed — Raqm applies the bidirectional algorithm to a string that was already reordered into visual order.

Two things that are checkable without reading Arabic

1. The string length changes. The recipe merges the lam-alef pair into a single presentation codepoint:

input : 8 codepoints  0x627 0x644 0x625 0x645 0x627 0x631 0x627 0x62a
recipe: 7 codepoints  0xfe95 0xfe8d 0xfead 0xfe8e 0xfee3 0xfef9 0xfe8d

All 7 are in Arabic Presentation Forms-B (U+FE70–U+FEFF). Anything downstream doing an exact-match search, a length check, or an index into that string is now working on a different string than the user typed.

2. Diacritics are deleted. arabic_reshaper.reshape() drops tashkeel silently:

input    : 'مَرْحَبًا'   4 diacritics
reshaped : 'ﻣﺮﺣﺒﺎ'      0 diacritics

Pillow renders all four correctly on its own; after the recipe they are gone. For vocalised text — teaching material, Qur'anic text, TTS input — that is data loss rather than a display quirk.

Why this is worth a doc line

The recipe is not simply wrong. On a build without Raqm it is still necessary, because nothing else shapes or reorders the text. So the correct advice is conditional, and the condition is a runtime property of the environment doing the rendering — not of the Pillow version:

if not features.check("raqm"):
    text = get_display(arabic_reshaper.reshape(text))

Nothing in the docs connects those two facts, so people copy an unconditional recipe and it is a coin-flip whether it helps or corrupts. #3081 is still receiving comments eight years on, and the recipe is recommended in that thread alongside instructions for enabling Raqm — after which it becomes harmful.

Proposed change

A short note in the text-rendering docs, near the direction / features / language parameters that already carry "Requires libraqm":

Complex scripts. When libraqm is available, Pillow performs shaping and applies the bidirectional algorithm itself. Do not pre-process text with arabic_reshaper and python-bidi in that case — the reordering is applied twice and the result is incorrect. Check PIL.features.check("raqm") at runtime, in the environment doing the rendering, to decide whether that pre-processing is needed.

Happy to open the PR if that wording is welcome, or to adjust it.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions