pango, pygtk: fixing up newer pango and pygtk by d-goldin · Pull Request #74282 · NixOS/nixpkgs

d-goldin · 2019-11-26T17:30:50Z

Motivation for this change

We are currently on a rather old version of pango and are being kept back by pygtk, which in some
sort of abandoned-but-beloved limbo. This proposes a split into two pango versions to be able to move
the rest of the ecosystem along and just pin pygtk until we know what to do about it.

Edit: After the below discussion, we decided to not split and instead patch up pygtk.

This is related to some discussion in #71571 from a while ago.

Unfortunately, due to the size of the rebuild, I'm unable to rebuild this on my machine, so I'd be thankful if somebody could do a nix-review run.

Right now this requires a patch to cmake to correctly locate harfbuzz: https://gitlab.kitware.com/cmake/cmake/issues/19531 until we get 3.16 in (it was just released today, but has quite a lot of changes, so I'm leaving the patch in for now instead of bumping the version).

Things done

Notify maintainers

cc @7c6f434c as maintainer and @jtojnar because of context

worldofpeace · 2019-11-26T20:23:00Z

I don't think this will work because I tried it. You'd have to give pygtk it's own version of gtk overridden to use an older version of pango. And that's still a pretty poor solution, I didn't PR it.

Edit: though I'm not opossed to this anymore. I'd consider those applications that still use this abandoned library unmaintained. So tier of quality... it won't matter to much there's another version of gtk2 for this.
(though there's gimp...)

worldofpeace · 2019-11-26T20:25:44Z

I also think there's probably a lot of packages we could either remove/update to not use pygtk.
I believe bleachbit doesn't even need it anymore, so that's one popular application.

Edit: have wip work for this.

d-goldin · 2019-11-26T20:55:16Z

@worldofpeace: Thanks for the pointers, I'll look into it.

worldofpeace · 2019-11-26T21:31:40Z

Opened #74295 to remove some pygtk users that are also probably abandoned.

jtojnar · 2019-11-26T23:26:00Z

pygtk should build now, according to #65670 (comment)

worldofpeace · 2019-11-27T01:48:50Z

pygtk should build now, according to #65670 (comment)

Hmm forgot about. Looking through my branches when I did my testing, I didn't look like I succeeded in getting it to build even with that.

d-goldin · 2019-11-27T01:50:31Z

pygtk should build now, according to #65670 (comment)

I must admit, this issue has been kinda sitting in my stash for a while, so I might need to refresh my memory a bit. I think I saw this commit "back then" too. I tried this now again, and think to remember that they for some reason did not bring back a part of the API. Building pygtk with the most recent pango 1.44.7 release I'm getting the following (without any adjustments to pygtk), even though there was a remark about a fix;

[...]
pango.c: In function ‘_wrap_PangoFont__do_find_shaper’:
pango.c:3822:32: error: ‘PangoFontClass’ {aka ‘struct _PangoFontClass’} has no member named ‘find_shaper’
     if (PANGO_FONT_CLASS(klass)->find_shaper)
                                ^~
pango.c:3823:38: error: ‘PangoFontClass’ {aka ‘struct _PangoFontClass’} has no member named ‘find_shaper’
         ret = PANGO_FONT_CLASS(klass)->find_shaper(PANGO_FONT(self->obj), lang, ch);
[...]

Maybe I'm missing something.

@matthiasclasen: If possible, could you please advise? Were you able to compile pygtk 2.24.0 with 1.44.7?

d-goldin · 2019-11-27T01:56:21Z

Opened #74295 to remove some pygtk users that are also probably abandoned.

As far as we can, for sure! I was viewing this more as a defensive stance for the cases where we might not be able to, but maybe it's just pessimism. :)

jtojnar · 2019-11-27T12:29:51Z

Something in this vein might work:

diff --git a/pkgs/development/python-modules/pygtk/default.nix b/pkgs/development/python-modules/pygtk/default.nix
index 09ccb5c3d95..6741ac695b4 100644
--- a/pkgs/development/python-modules/pygtk/default.nix
+++ b/pkgs/development/python-modules/pygtk/default.nix
@@ -1,4 +1,4 @@
-{ stdenv, fetchurl, python, pkgconfig, gtk2, pygobject2, pycairo
+{ stdenv, fetchurl, fetchpatch, python, pkgconfig, gtk2, pygobject2, pycairo
 , buildPythonPackage, libglade ? null, isPy3k }:
 
 buildPythonPackage rec {
@@ -12,6 +12,19 @@ buildPythonPackage rec {
     sha256 = "04k942gn8vl95kwf0qskkv6npclfm31d78ljkrkgyqxxcni1w76d";
   };
 
+  patches = [
+    # not sure what this does but Fedora and Arch have it
+    (fetchpatch {
+      url = "https://src.fedoraproject.org/rpms/pygtk2/raw/93b36f49fdf19b3673c04f2f85310e88b1f0fcc6/f/0001-Fix-leaks-of-Pango-objects.patch";
+      sha256 = "031px4w5cshcx1sns430sdbr2i007b9zyb2carb3z65nzr77dpdd";
+    })
+  ];
+
+  postPatch = ''
+    # regenerate pango defs so it builds
+    python2 $(pkg-config pygobject-2.0 --variable codegendir)/h2def.py $(pkg-config pango --variable includedir)/pango-1.0/pango/*.h > pango.defs
+  '';
+
   nativeBuildInputs = [ pkgconfig ];
   buildInputs = stdenv.lib.optional (libglade != null) libglade;

d-goldin · 2019-11-27T12:38:47Z

I don't think this will work because I tried it. You'd have to give pygtk it's own version of gtk overridden to use an older version of pango. And that's still a pretty poor solution, I didn't PR it.

So, I'm currently trying this, specifically for gimp and the rabbit hole kinda opens a little bit deeper. Adwaita does not seem to build with the newer pango, at least "trivially". Gimp itself at least compiles though, but then lacks themes, gimp-with-plugins which pulls adwaita in, does not.

[...]
for file in `cd ../../Adwaita/scalable; find . -name "*.svg"`; do \
        context="`dirname $file`"; \
        /nix/store/g83k1kbjimimhbrk9nbl66f9nlwxqfyc-coreutils-8.31/bin/mkdir -p /nix/store/j36rdrd095gsg6lqlq05cym86mmlgiyn-adwaita-icon-theme-3.34.3/share/icons/Adwaita/scalable/$context; \
        /nix/store/pn3mrlw8l0pda4w5knfima4pq99glzvw-bash-4.4-p23/bin/bash /build/adwaita-icon-theme-3.34.3/install-sh -c -m 644 ../../Adwaita/scalable/$file /nix/store/j36rdrd095gsg6lqlq05cym86mmlgiyn-adwaita-icon-theme-3.34.3/share/icons/Adwaita/scalable/$file; \
        for size in 16x16 24x24 32x32 48x48 64x64 96x96; do \
                /nix/store/g83k1kbjimimhbrk9nbl66f9nlwxqfyc-coreutils-8.31/bin/mkdir -p /nix/store/j36rdrd095gsg6lqlq05cym86mmlgiyn-adwaita-icon-theme-3.34.3/share/icons/Adwaita/$size/$context; \
                /nix/store/35z14n5z8k31mb0qm2qqzg159a6557m3-gtk+3-3.24.12-dev/bin/gtk-encode-symbolic-svg ../../Adwaita/scalable/$file $size -o /nix/store/j36rdrd095gsg6lqlq05cym86mmlgiyn-adwaita-icon-theme-3.34.3/share/icons/Adwaita/$size/$context; \
        done \
done
Can't load file: Unrecognized image file format
Can't load file: Unrecognized image file format
Can't load file: Unrecognized image file format
Can't load file: Unrecognized image file format
Can't load file: Unrecognized image file format
Can't load file: Unrecognized image file format
[... ultimately fails due to that ...]

Also, maybe interesting to keep an eye on, even though there was no apparent activity in 10 months: https://gitlab.gnome.org/GNOME/gimp/issues/339

jtojnar · 2019-11-27T13:44:47Z

I believe https://gitlab.gnome.org/GNOME/gimp/issues/339 is already solved on master (#67576).

worldofpeace · 2019-11-27T14:19:18Z

@jtojnar Ahh right ✨ The postPatch there is probably what I was missing and the issue.

jtojnar · 2019-11-27T14:35:24Z

Does not seem to generate functions for me, though.

jtojnar · 2019-11-27T14:49:39Z

Or just drop the single virtual call https://github.com/flathub/org.glimpse_editor.Glimpse/blob/master/patches/pygtk-Drop-the-PangoFont-find_shaper-virtual-method.patch. Alatiera is planning an upstream MR.

d-goldin · 2019-11-30T12:48:21Z

I still need to look into Adwaita and whatever else comes up. If somebody has the resources to run a review build or maybe help test some applications, I'd be thankful.

jtojnar · 2019-12-01T04:00:44Z

adwaita is currently failing on staging, running bisect.

jtojnar · 2019-12-01T12:35:41Z

That is just weird:

Bisecting: 0 revisions left to test after this (roughly 0 steps)
[0a443f4c72595d0fc1b678ad4f511429c13c8c65] kodi: drop jasper dependency
running ./bisect.sh
/nix/store/5sjl4azzscizxvhv0m78v0ivw26q16y5-adwaita-icon-theme-3.34.0
9aa62321ea9c2f7a355d4c549cf1f803c0283ca8 is the first bad commit
commit 9aa62321ea9c2f7a355d4c549cf1f803c0283ca8
Author: c0bw3b <c0bw3b@users.noreply.github.com>
Date:   Sun Nov 17 19:44:10 2019 +0100

    gdk-pixbuf: disable JPEG2000 support
    
    jasper has unfixed CVE
    Upstream has no plan to switch to openjpeg AFAICT

 pkgs/development/libraries/gdk-pixbuf/default.nix | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
bisect run success

#73586

That is just weird as jasper did not propagate anything other than libjpeg, which is already a dependency of gdk-pixbuf.

That should have no bearing on MIME sniffing.

jtojnar · 2019-12-01T13:11:46Z

Continuing the discussion in #73586

d-goldin · 2019-12-01T16:30:34Z

Note for later: We probably want a release-notes entry and possibly doc adjustments to that one, regarding deprecated bitmap fonts.

jtojnar · 2019-12-01T23:42:05Z

Would wording like this work:

Pango was upgraded to 1.44, which no longer uses freetype for font loading. This means that type1 and bitmap fonts are no longer supported in applications relying on Pango for font rendering (notably, GTK application).
See <link xlink:href="https://gitlab.gnome.org/GNOME/pango/issues/386">upstream issue</link> for more information.

d-goldin · 2019-12-01T23:48:17Z

Would wording like this work:

Pango was upgraded to 1.44, which no longer uses freetype for font loading. This means that type1 and bitmap fonts are no longer supported in applications relying on Pango for font rendering (notably, GTK application).
See <link xlink:href="https://gitlab.gnome.org/GNOME/pango/issues/386">upstream issue</link> for more information.

I think the wording is good, but maybe we can find some other resource to link. I remember that I once had some release note PR and included the link to another PR in the description and the reviewers did not think linking to a nixpkgs PR was the best option. What do you think of maybe using https://blogs.gnome.org/mclasen/2019/05/25/pango-future-directions/ ?

jtojnar · 2019-12-02T00:13:12Z

IMHO the issue is better resource as it provides more details (including discussion about converting bitmap fonts). Only a small part of the blog post is dedicated to this and the link is in the issue OP for those interested.

d-goldin · 2019-12-02T00:14:54Z

Alright, personally I'm fine with the issue link, so I'll just add it and then we'll see.

* Removes an unused binding that prevents compilation with newer pango * Adds a patch to fix a memory leak

a patch to cmake to correctly locate harfbuzz: https://gitlab.kitware.com/cmake/cmake/issues/19531, needed for more recent pango.

jtojnar · 2019-12-03T14:29:01Z

Can we wrap this up? It is blocking Nautilus update.

Edit: did not see you already pushed the release notes.

d-goldin · 2019-12-03T14:31:47Z

@jtojnar: From my side, yes, we can give it a go and deal with any unexpected fallout then. I was just waiting on reviewers approval/further change requests.

d-goldin · 2019-12-03T16:20:57Z

Thanks for the support and help. I will try to watch for upcoming breakages, but otherwise, feel free to ping me.

worldofpeace · 2019-12-03T19:29:04Z

@d-goldin I believe I suggested not linking to our internal tracker or PRs, and to provide a better resource in the release notes. In this case it's an external issue so it's fine to link.

jtojnar · 2019-12-15T05:36:06Z

Another broken abandoned package is pangox-compat: https://hydra.nixos.org/build/108047288/nixlog/1

Not sure it is worth it trying to fix it.

d-goldin changed the title ~~Pango update split~~ pango: split into new and old version Nov 26, 2019

ofborg Bot added 6.topic: python Python is a high-level, general-purpose programming language. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 8.has: package (new) This PR adds a new package labels Nov 26, 2019

ofborg Bot requested review from 7c6f434c, LnL7, andir, edolstra, lovek323, np and ttuegel November 26, 2019 17:39

worldofpeace reviewed Nov 26, 2019

View reviewed changes

Comment thread pkgs/development/libraries/pango/1.43.nix Outdated

d-goldin requested review from FRidh and jonringer as code owners November 30, 2019 11:30

jtojnar closed this Dec 1, 2019

jtojnar reopened this Dec 1, 2019

d-goldin added 4 commits December 2, 2019 02:30

pango: 1.43.0 -> 1.44.7

e603d29

pygtk: compatibility with pango 1.44

1a33f39

* Removes an unused binding that prevents compilation with newer pango * Adds a patch to fix a memory leak

cmake: patching for harfbuzz location

fea6b5f

a patch to cmake to correctly locate harfbuzz: https://gitlab.kitware.com/cmake/cmake/issues/19531, needed for more recent pango.

release-notes: adding pango font loading deprecation

c25d4ab

d-goldin force-pushed the pango_update_split branch from 8aa178c to c25d4ab Compare December 2, 2019 01:33

ofborg Bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation labels Dec 2, 2019

jtojnar merged commit daa676f into NixOS:staging Dec 3, 2019

d-goldin deleted the pango_update_split branch December 3, 2019 14:31

tobim mentioned this pull request Dec 4, 2019

cmake: 3.15.4 -> 3.16.2 #74293

Merged

10 tasks

worldofpeace mentioned this pull request Dec 11, 2019

PyGTK deprecation tracking issue #75478

Closed

24 tasks

Uh oh!

Conversation

d-goldin commented Nov 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Things done

Notify maintainers

Uh oh!

Uh oh!

worldofpeace commented Nov 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

worldofpeace commented Nov 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

d-goldin commented Nov 26, 2019

Uh oh!

worldofpeace commented Nov 26, 2019

Uh oh!

jtojnar commented Nov 26, 2019

Uh oh!

worldofpeace commented Nov 27, 2019

Uh oh!

d-goldin commented Nov 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

d-goldin commented Nov 27, 2019

Uh oh!

jtojnar commented Nov 27, 2019

Uh oh!

d-goldin commented Nov 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtojnar commented Nov 27, 2019

Uh oh!

worldofpeace commented Nov 27, 2019

Uh oh!

jtojnar commented Nov 27, 2019

Uh oh!

jtojnar commented Nov 27, 2019

Uh oh!

d-goldin commented Nov 30, 2019

Uh oh!

jtojnar commented Dec 1, 2019

Uh oh!

jtojnar commented Dec 1, 2019

Uh oh!

jtojnar commented Dec 1, 2019

Uh oh!

d-goldin commented Dec 1, 2019

Uh oh!

jtojnar commented Dec 1, 2019

Uh oh!

d-goldin commented Dec 1, 2019

Uh oh!

jtojnar commented Dec 2, 2019

Uh oh!

d-goldin commented Dec 2, 2019

Uh oh!

jtojnar commented Dec 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

d-goldin commented Dec 3, 2019

Uh oh!

d-goldin commented Dec 3, 2019

Uh oh!

worldofpeace commented Dec 3, 2019

Uh oh!

jtojnar commented Dec 15, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

d-goldin commented Nov 26, 2019 •

edited

Loading

worldofpeace commented Nov 26, 2019 •

edited

Loading

worldofpeace commented Nov 26, 2019 •

edited

Loading

d-goldin commented Nov 27, 2019 •

edited

Loading

d-goldin commented Nov 27, 2019 •

edited

Loading

jtojnar commented Dec 3, 2019 •

edited

Loading