Skip to content

Fix charset conversion for concatenated deliver_sm parts - #170

Open
cl77 wants to merge 1 commit into
kneodev:masterfrom
cl77:fix-concat-mo-gsm-encoding
Open

Fix charset conversion for concatenated deliver_sm parts#170
cl77 wants to merge 1 commit into
kneodev:masterfrom
cl77:fix-concat-mo-gsm-encoding

Conversation

@cl77

@cl77 cl77 commented Jul 8, 2026

Copy link
Copy Markdown

Problem

Parts of concatenated messages (with UDH) are written to the deliver_sm PDU using ksmppd's internal UTF-8 text representation, while single-part messages are correctly re-encoded to GSM 03.38 first.

In smpp_pdu_msg_to_pdu() (smpp/libsmpp/smpp_pdu_util.c), the loop over the parts produced by sms_split() has two branches:

  • no UDH (single-part): short_message is duplicated and then converted via charset_utf8_to_gsm() / alt-charset when data_coding is 0 - correct.
  • with UDH (concatenated): short_message = octstr_cat(udhdata, msgdata) with no charset conversion at all - the raw UTF-8 bytes leak into the PDU.

As a result, ESMEs receive concatenated text messages announced with data_coding = 0 (GSM default alphabet) whose payload is actually UTF-8. Example: the Euro sign arrives as e2 82 ac instead of the GSM7 escape sequence 1b 65. Single-part messages containing the same characters arrive correctly, which makes the behaviour inconsistent and impossible for the receiving ESME to interpret reliably.

Fix

Apply the same re-encoding logic already used for single-part messages to the UDH branch, before the UDH is prepended:

  • data_coding & 0xF0 or data_coding == 0 without alt_charset: convert UTF-8 to GSM 03.38
  • data_coding == 0 with alt_charset: convert to the configured alternative charset

Only DC_7BIT text parts are converted; binary (8-bit) and UCS-2 parts are unaffected (UCS-2 already worked correctly since the internal representation is UCS-2BE for those).

Verification

Test harness calling smpp_pdu_msg_to_pdu() with a 183-character DC_7BIT message containing a Euro sign (split into 2 parts, data_coding = 0):

Before (part 0, after 6-byte concat UDH):

data: 05 00 03 00 02 01 e2 82 ac 20 4c 6f 72 65 6d ...   (UTF-8 Euro)

After:

data: 05 00 03 00 02 01 1b 65 20 4c 6f 72 65 6d ...      (GSM7 Euro)

Single-part messages, UCS-2 and binary messages are unchanged.

Parts of concatenated messages (with UDH) were sent with the internal
UTF-8 representation of the message text, while single-part messages
were correctly re-encoded to GSM 03.38 before being written to the PDU.

As a result, ESMEs received concatenated text messages with
data_coding 0 whose payload was actually UTF-8, e.g. the Euro sign
arrived as e2 82 ac instead of the GSM7 escape sequence 1b 65.

Apply the same re-encoding logic used for single-part messages to the
UDH branch: convert DC_7BIT text parts from UTF-8 to GSM 03.38 (or to
alt-charset when configured) before prepending the UDH. Binary (8-bit)
and UCS-2 parts are unaffected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant