TMDbH freezes & crashes Matrix on Arch Linux

[cg00001]

I tried a Kodi 19 build that used TMDbH widggets on some menus, and every time I scrolled to those menus, or opened TMDbH it froze Kodi, and crashed it.
So I installed just TMDbH on an empty Matrix, same thing happens, pic (after showing that for a few seconds, Kodi is closed) and log below.
https://pastebin.com/KmTbUVri
Another log with with debug logging enabled:
https://pastebin.com/h1hhW9vi

[jurialmunkey]

You've got a PIL import error which indicates PIL/Pillow module is missing from your Python packages.

It shouldn't cause a crash though. Can you test latest matrix version and see if that helps? The error never writes out the traceback to the log, so hopefully disabling that will at least allow the script to progress.

[cg00001]

You've got a PIL import error which indicates PIL/Pillow module is missing from your Python packages.

It shouldn't cause a crash though. Can you test latest matrix version and see if that helps? The error never writes out the traceback to the log, so hopefully disabling that will at least allow the script to progress.

I'm using kodi-devel-x11 package from AUR in Arch, the latest version.
This is PIL?

[jurialmunkey]

Yeah that's PIL. However, since you're on Linux, your local dist packages are probably overriding the Kodi packages. Basically same reason you had issues with NumPy in Leia.

Did you try the change I pushed to Matrix?

[cg00001]

Yeah that's PIL. However, since you're on Linux, your local dist packages are probably overriding the Kodi packages. Basically same reason you had issues with NumPy in Leia.

Did you try the change I pushed to Matrix?

Yes, same problem (pic).
I suspected that might happen because Arch uses the newest Python version 3.9.2-1 while Kodi 19 uses an earlier one.
It's alright though, I use Kodi 19 just for testing, not much of the addons I use work in it anyway.
Maybe when a Kodi 19 binary comes out for Arch (I built mine from sources: kodi-devel), that issue would be resolved.
Thanks for the help.

[cg00001]

@jurialmunkey the first Kodi 19 binary (v19.0-6) arrived in Arch repos a few minutes ago, this issue persists.

[cg00001]

@jurialmunkey Tried latest TMDb Helper v2.4.36, addon opens (it didn't before), and works up until the point where it should pop up the players.

[drinfernoo]

@CgCgCgCgCg Latest TMDbHelper is 4.2.21.

[cg00001]

@CgCgCgCgCg Latest TMDbHelper is 4.2.21.

Tried it, freezes Kodi when I start TMDb Helper.

[JoeJazzy]

@CgCgCgCgCg Latest TMDbHelper is 4.2.21.

Tried it, freezes Kodi when I start TMDb Helper.

Noticed this happening when Kodi would crash trying to install the Arctic skins, it would try to add TMDbHelper and will crash to desktop until I remove the addon

[cg00001]

This is a Kodi crash log with a gdb Stack Trace, might help:
https://pastebin.com/91FGGZhe

I also tried downgrading numpy even lower than 1.15.4 (that was suggested and worked for Kodi Leia in the past), but no go.

[jurialmunkey]

Can I have a debug log with v4.2.22 please (latest github matrix version)?

[cg00001]

Can I have a debug log with v4.2.22 please (latest github matrix version)?

Of course:
https://pastebin.com/LCSfXiSg

[cg00001]

I won't uninstall my current Python (along with 30 + apps that are linked to it) to install v3.8.5 just to test Kodi Matrix, I'll just use Kodi Leia:
xbmc/xbmc#19324 (comment)

[jurialmunkey]

If you're going to go back to Leia, I will close this issue for now as I am unable to test myself and I imagine it will eventually get resolved when the Python version gets updated in Kodi as mentioned in that thread you linked.

[cg00001]

If you're going to go back to Leia, I will close this issue for now as I am unable to test myself and I imagine it will eventually get resolved when the Python version gets updated in Kodi as mentioned in that thread you linked.

As you wish, I have both Leia and Matrix -I meant I'll go back using Kodi Leia, not uninstall Kodi Matrix.
If Kodi devs reply to that issue I opened and it works, I'll comment here.

[jurialmunkey]

I meant I'll go back using Kodi Leia, not uninstall Kodi Matrix.

Ah I misunderstood! Then I might have something for you to test soon that will allow bypassing PIL. It will mean that image functions like blur won't be available but should at least mean TMDbH can be run.

[basilgello]

Following @CgCgCgCgCg report, I install the TMDbH addon on Debian bullseye (Kodi 19.0, python 3 upgraded to 3.9.2 from unstable) and got a crash on first run of addon. Subsequent runs are OK:

xbmc/xbmc#19324 (comment)

I guess it tries to access the database and directories not yet created @jurialmunkey

[basilgello]

Installing addon from scratch while python3-numpy is installed via apt crashes Kodi too with the following errors before thr actual crash:

2021-03-07 18:50:47.686 T:12781   ERROR <general>: /home/kodi/.kodi/addons/plugin.video.themoviedb.helper/resources/lib/api/mapping.py:54: SyntaxWarning: "is not" with a literal. Did you mean "!="?
                                                     if not pv and pv is not 0:
                                                   
2021-03-07 18:50:47.686 T:12781   ERROR <general>: /home/kodi/.kodi/addons/plugin.video.themoviedb.helper/resources/lib/api/mapping.py:88: SyntaxWarning: "is not" with a literal. Did you mean "!="?
                                                     if not v and v is not 0:

[jurialmunkey]

Syntax warning isn't an error and can be ignored.

[cg00001]

@jurialmunkey Sorry to interrupt but I think the basic problem with TMDb Helper and other addons and skins that freeze & crash Kodi 19 is Kodi devs should update Kodi to use Python v3.9.2 and not just keep using v3.8.5.

[basilgello]

Unfortunately, I have a strong suspect it is a core Python bug… I will do my best to debug the issue out but there are some candidates for buffer overflows already…

[basilgello]

This morning I finally managed how to force loading Python helper scripts into GDB session targeting Kodi:

cd /usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/
ln -sf ../../bin/python3.9m-gdb.py libpython3.9.so.1.0-gdb.py

Now GDB sees the auto-loaded script:

(gdb) info auto-load
gdb-scripts:  No auto-load scripts.
libthread-db:  No auto-loaded libthread-db.
local-gdbinit:  Local .gdbinit file was not found.
python-scripts:  
Loaded  Script                                                                 
Yes     /usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libpython3.9.so.1.0-gdb.py 
Yes     /usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28-gdb.py

and we can use py-bt etc.

See more on auto-loading GDB python script helpers for libraries:

• https://sourceware.org/gdb/onlinedocs/gdb/Auto_002dloading.html
• https://sourceware.org/gdb/onlinedocs/gdb/Auto_002dloading-safe-path.html
• https://stackoverflow.com/questions/40391404/loading-python-support-in-gdb

The Log

Running GDB:

gdb -batch -x /home/kodi/.kodi/numpy.gdb -p $(pidof kodi.bin)

with the following script /home/kodi/.kodi/numpy.gdb :

set pagination off

set disassembly-flavor intel

set print pretty on

set logging file /tmp/gdb.log
set logging on

c

echo \n=== bt full ===\n\n

bt full

echo \n=== py-bt-full ===\n\n

py-bt-full

echo \n=== py-locals ===\n\n

py-locals

echo \n=== end ===\n\n


set logging off

kill
quit

produces the following log: gdb.log.txt

Suspected Root Cause:

PyArray_FromScalar is a part of Numpy, but it is plugged into array (string, byte-string etc) processing.

The following code in TMDB Helper:

v = u'{}{}'.format(v, i) if v else u'{}'.format(i)

emits the proper PyUnicodeObject that plays nice with PyArray_FromScalar if v is an empty string:

=== py-locals ===

value = ['lib.monitor.images - PIL import error!\n', SystemError('../Objects/structseq.c:401: bad argument to internal function',)]
level = 1
v = ''
i = 'lib.monitor.images - PIL import error!\n'

=== end of breakpoint on PyArray_FromScalar ===

# We check that type is PyUnicodeType mapped to PyUnicodeObject in memory
$1 = (PyTypeObject *) 0x7f307e502d40 <PyUnicode_Type>

# Print PyUnicodeObject structure definition
type = struct {
/*    0      |    72 */    PyCompactUnicodeObject _base;
/*   72      |     8 */    union {
/*                 8 */        void *any;
/*                 8 */        Py_UCS1 *latin1;
/*                 8 */        Py_UCS2 *ucs2;
/*                 8 */        Py_UCS4 *ucs4;

                               /* total size (bytes):    8 */
                           } data;

                           /* total size (bytes):   80 */
                         }

# ... and its value
$2 = {
  _base = {
    _base = {
      ob_base = {
        ob_refcnt = 1805,
        ob_type = 0x7f307e502d40 <PyUnicode_Type>
      },
      length = 0,
      hash = 0,
      state = {
        interned = 1,
        kind = 1,
        compact = 1,
        ascii = 1,
        ready = 1
      },
      wstr = 0x0
    },
    utf8_length = 0,
    utf8 = 0x0,
    wstr_length = 139844877479568
  },
  data = {
    any = 0x7f302c3f15f0,
    latin1 = 0x7f302c3f15f0 "\260\026?,0\177",
    ucs2 = 0x7f302c3f15f0,
    ucs4 = 0x7f302c3f15f0
  }
}

but produces a smashed object if v is not empty;

=== py-locals ===

value = ['lib.monitor.images - PIL import error!\n', SystemError('../Objects/structseq.c:401: bad argument to internal function',)]
level = 1
v = 'lib.monitor.images - PIL import error!\n'
i = SystemError('../Objects/structseq.c:401: bad argument to internal function',)

$4 = {
  _base = {
    _base = {
      ob_base = {
        ob_refcnt = 4,
        ob_type = 0x7f307e502d40 <PyUnicode_Type>
      },
      length = 39,
      hash = -1,
      state = {
        interned = 0,
        kind = 1,
        compact = 1,
        ascii = 1,
        ready = 1
      },
      wstr = 0x0
    },
    utf8_length = 7597132136140925292,
    utf8 = 0x67616d692e726f74 <error: Cannot access memory at address 0x67616d692e726f74>,
    wstr_length = 5497012919308677989
  },
  data = {
    any = 0x2074726f706d6920,
    latin1 = 0x2074726f706d6920 <error: Cannot access memory at address 0x2074726f706d6920>,
    ucs2 = 0x2074726f706d6920,
    ucs4 = 0x2074726f706d6920
  }
}

[basilgello]

The following error message:

SystemError('../Objects/structseq.c:401: bad argument to internal function',)

is thrown by PyStructSequence_InitType2 detecting double-initialization inside numpy overrides:

https://github.com/python/cpython/blob/3.9/Objects/structseq.c#L401

=== py-bt-full ===
#48 Frame 0x7f3dba8e6a60, for file /usr/lib/python3/dist-packages/numpy/core/overrides.py, line 7, in <module> ()
    from numpy.core._multiarray_umath import (
#111 Frame 0x7f3dba8e6dc0, for file /usr/lib/python3/dist-packages/numpy/core/multiarray.py, line 12, in <module> ()
    from . import overrides
#174 Frame 0x7f3d9947cb20, for file /usr/lib/python3/dist-packages/numpy/core/__init__.py, line 22, in <module> ()
    from . import multiarray
#281 Frame 0x7f3dba9f4610, for file /usr/lib/python3/dist-packages/PIL/ImageFilter.py, line 20, in <module> ()
    import numpy
#344 Frame 0x7f3d9947c5b0, for file /home/gellmar/.kodi/addons/plugin.video.themoviedb.helper/resources/lib/monitor/images.py, line 11, in >
    from PIL import ImageFilter, Image

So it is definitely a NumPy bug starting a snowball of memory corruption.

[basilgello]

BTW commenting out kodi_log does not solve problem, so it is an incompletely initialized NumPy doing the crash :)

[cg00001]

@jurialmunkey BTW congratulations for the amazing Arctic Horizon skin that wasn't affected by the Python issues and allowed me to make a working Kodi Matrix build for personal use.

[jurialmunkey]

@CgCgCgCgCg @basilgello - Could you both test the latest matrix git version (v.4.2.25) and see if it is still crashing for you?

I'm now lazy importing the PIL image modules so they will only be imported if you use one of the image functions in a skin (e.g. blur or colour matching). Obviously will still crash if you turn blur on but at least you should be able to run TMDbH without a crash by keeping blurred disabled.

[basilgello]

@CgCgCgCgCg @basilgello - Could you both test the latest matrix git version (v.4.2.25) and see if it is still crashing for you?

I'm now lazy importing the PIL image modules so they will only be imported if you use one of the image functions in a skin (e.g. blur or colour matching). Obviously will still crash if you turn blur on but at least you should be able to run TMDbH without a crash by keeping blurred disabled.

LGTM 👍

[cg00001]

@CgCgCgCgCg @basilgello - Could you both test the latest matrix git version (v.4.2.25) and see if it is still crashing for you?

I'm now lazy importing the PIL image modules so they will only be imported if you use one of the image functions in a skin (e.g. blur or colour matching). Obviously will still crash if you turn blur on but at least you should be able to run TMDbH without a crash by keeping blurred disabled.

I downloaded v4.3.0 and confirm it works fine, no freeze & crash here.

[jurialmunkey]

excellent! Thanks for testing!

[basilgello]

@jurialmunkey python-pillow/Pillow#5323 thought s on this Pillow change?