Skip to content

Tolerate additional entries in obsolete messages #79

Closed

Description

@mathieucaroff

I have a .po file whose translations are handled with Poedit and I get obsolete entries in my .po files. The parser fails when it encounters them.

Reproduction

en.po

#, fuzzy
#~| msgid "Latest version"
#~ msgid "Latest Version"
#~ msgstr "Latest version"

Parsing the content of en.po with gettext-parser (running gettextParser.po.parse(content)) produces the following error:

node_modules\gettext-parser\lib\poparser.js:224
            throw err;
            ^

SyntaxError: Error parsing PO data: Invalid key name "|" at line 2. This can be caused by an unescaped quote character in a msgid or msgstr value.

Expected behavior

The parser should not crash. It should either ignore the entry, or properly parse it to data.

Misc

I believe this issue was introduced by the following PR: #64 -- released in version 5.0.0. 4.2.0 is the last version before 5.0.0.

Have a good day !

Activity

  1. smhg commented on Jun 6, 2023

    @smhg
    Owner

    Thank you for reporting this! After some initial investigation any additional character after an obsolete marker (#~) seems to lead to this error. This is unrelated to the fuzzy flag.
    So currently only basic obsolete messages are supported. Obsolete messages currently cannot contain previous messages (|), references (:),...
    It would definitely be good to support this. A PR would be very much appreciated.

  2. changed the title [-]Tolerate fuzzy entries in obsolete messages[/-] [+]Tolerate additional entries in obsolete messages[/+] on Jun 6, 2023
  3. smhg commented on Jun 6, 2023

    @smhg
    Owner

    @KapitanOczywisty would you be able to have a look at this?

  4. KapitanOczywisty commented on Jun 6, 2023

    @KapitanOczywisty
    Contributor

    So currently only basic obsolete messages are supported. Obsolete messages currently cannot contain previous messages (|), references (:),...

    @smhg Yes, they can, by using #| and #:. I don't know where #~| came from, but It's not included anywhere in gettext documentation. All comments are connected to their msgid, so if that msgid is obsolete there is no need to mark comments as obsolete. Reversing that: why not #~, fuzzy? You never see #~,, likely because It's popular enough that people seen it correctly used and implemented.

  5. smhg commented on Jun 8, 2023

    @smhg
    Owner

    @KapitanOczywisty If I understand correctly #| means a previous translation and #~| is an obsolete previous translation?
    I can't tell whether that is actually usefull to programs, but moving from non-obsolete to obsolete just adds the ~ so it makes sense that's the result?
    And obsolete message parsing currently doesn't look at the 3rd character correctly I think?

  6. KapitanOczywisty commented on Jun 8, 2023

    @KapitanOczywisty
    Contributor

    If I understand correctly #| means a previous translation and #~| is an obsolete previous translation?

    I only found that: https://github.com/winlibs/gettext/blob/24b0fe071cff79acf4a3904591b481328732ddc1/source/gettext-tools/src/po-lex.c#L903-L906

    And obsolete message parsing currently doesn't look at the 3rd character correctly I think?

    I thinks parsing ignores whitespace and searches for msgid keyword. I can make this work wit #~| though this creates some questions:

    • should #~| be treated as #|?
    • if what should comment be named?
    • should this also be parsed for non-obsolete messages?
  7. mathieucaroff commented on Jun 16, 2023

    @mathieucaroff
    Author

    I believe the simplest, non-surprising behavior would be to treat as a regular comment, if that makes sense with regards to the way gettext-parser currently works.

  8. smhg commented on Jun 17, 2023

    @smhg
    Owner

    Basically #~| should be treated as if it were #~#|, right?
    It should be listed like this one I think. These are already parsed out.

    But looking a the code, this does not look straighforward.
    First this @KapitanOczywisty: do you remember why you grouped the obsolete state with the none state? Wouldn't it be better to separate those? Because different parts of the checks under those are applicable to different states, no?

  9. KapitanOczywisty commented on Jun 17, 2023

    @KapitanOczywisty
    Contributor

    do you remember why you grouped the obsolete state with the none state?

    At the time I was not aware of #~| so only correct use would be with keywords e.g. #~ msgid. I can fix that, just we need to decide how #~| should be handled. I'd prefer to treat it as #|, which means parse and compilation would yield different results than input, but all other solutions have major problems with compatibility anyway.

  10. smhg commented on Jun 17, 2023

    @smhg
    Owner

    As an example: if you prepend it to this line, it would appear on this line. Right?
    If not, can you give an example when it gets ambiguous?

  11. KapitanOczywisty commented on Jun 17, 2023

    @KapitanOczywisty
    Contributor

    In my suggestion, this file:

    #, fuzzy
    #~| msgid "Latest version"
    #~ msgid "Latest Version"
    #~ msgstr "Latest version"
    

    after parse would become:

    {
        // ...
        "obsolete": {
            "": {
                "Latest Version": {
                    "msgid": "Latest Version",
                    "comments": {
                        "flag": "fuzzy",
                        "previous": "msgid \"Latest version\""
                    },
                    "msgstr": [
                        "Latest version"
                    ]
                }
            }
        }
    }
    

    and then after compilation:

    #, fuzzy
    #| msgid "Latest version"
    #~ msgid "Latest Version"
    #~ msgstr "Latest version"
    
  12. kisaragi-hiu commented on Jul 6, 2026

    @kisaragi-hiu
    Contributor

    I think #112 / 8ebe726 also fixes this issue. (I just realized my #111 is a duplicate of this issue, as the problem being reported here is also the support for #~| lines.)

  13. smhg commented on Jul 6, 2026

    @smhg
    Owner

    I've added a small fix for the invalid format that got ignored by 8ebe726 (lines starting with |). This now throws again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions