[Bug]: matplotlib crashes if `_tkinter` doesn't have `file` #23074

konstin · 2022-05-19T19:34:48Z

Bug summary

In the python-build-standalone cpython distributions, _tkinter is doesn't have a separate shared library and therefore also doesn't have a __file__ attribute, causing a crash in matplotlib.

Code for reproduction

Tested on ubuntu 20.04


wget https://github.com/indygreg/python-build-standalone/releases/download/20220502/cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full.tar.zst
tar xf cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full.tar.zst
python/install/bin/python3 -m venv venv
venv/bin/pip install matplotlib
venv/bin/python -c "import matplotlib.pyplot as plt; plt.plot(list(range(10)), list(range(10)))"

Actual outcome

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2769, in plot
    return gca().plot(
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2274, in gca
    return gcf().gca(**kwargs)
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 867, in gcf
    return figure()
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 808, in figure
    manager = new_figure_manager(
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 326, in new_figure_manager
    _warn_if_gui_out_of_main_thread()
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 316, in _warn_if_gui_out_of_main_thread
    if (_get_required_interactive_framework(_get_backend_mod())
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 217, in _get_backend_mod
    switch_backend(dict.__getitem__(rcParams, "backend"))
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 268, in switch_backend
    switch_backend(candidate)
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 288, in switch_backend
    class backend_mod(matplotlib.backend_bases._Backend):
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/pyplot.py", line 289, in backend_mod
    locals().update(vars(importlib.import_module(backend_name)))
  File "/home/konsti/monotrail/pbsi/python/install/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/backends/backend_tkagg.py", line 1, in <module>
    from . import _backend_tk
  File "/home/konsti/monotrail/pbsi/venv/lib/python3.10/site-packages/matplotlib/backends/_backend_tk.py", line 24, in <module>
    from . import _tkagg
AttributeError: module '_tkinter' has no attribute '__file__'. Did you mean: '__name__'?

Expected outcome

matplotlib works even if _tkinter is not a shared library, maybe with reduced functionality but it shouldn't crash

Additional information

The code that causes this is

matplotlib/src/_tkagg.cpp

Lines 310 to 322 in f25c2d0

    
           // Handle PyPy first, as that import will correctly fail on CPython. 
        
           module = PyImport_ImportModule("_tkinter.tklib_cffi");   // PyPy 
        
           if (!module) { 
        
               PyErr_Clear(); 
        
               module = PyImport_ImportModule("_tkinter");  // CPython 
        
           } 
        
           if (!(module && 
        
                 (py_path = PyObject_GetAttrString(module, "__file__")) && 
        
                 (py_path_b = PyUnicode_EncodeFSDefault(py_path)) && 
        
                 (path = PyBytes_AsString(py_path_b)))) { 
        
               goto exit; 
        
           } 
        
           tkinter_lib = dlopen(path, RTLD_LAZY);

I've previously reported this at astral-sh/python-build-standalone#129 where @indygreg noted that "I'll likely need to converse with a matplotlib developer on how to best support this. I would like to have a compatibility story here, as matplotlib is a popular package and I'd like it to be supported.".

CC @indygreg

Operating system

ubuntu 20.04

Matplotlib Version

3.5.2

Matplotlib Backend

n/a

Python version

cpython-3.10.4+20220502-x86_64_v3-unknown-linux-gnu-pgo+lto-full

Jupyter version

n/a

Installation

pip

The text was updated successfully, but these errors were encountered:

QuLogic · 2022-05-19T20:12:58Z

cc @anntzer

anntzer · 2022-05-19T22:13:22Z

I don't really have a solution available? If the tkinter symbols are not public, I don't know what we can do...

tacaswell · 2022-05-19T22:16:42Z

This is part of our logic to find at runtime the tk libraries.

At a minimum we should not fail in this way (I think just protect it with PyObject_HasAttrString)

I'm not sure how much special casing we would take on to support @indygreg , but will definitely consider proposals !

anntzer · 2022-05-19T23:56:34Z

(fwiw I don't think we should try to catch this specific case with PyObject_HasAttrString -- next time it'll be the import that fails, or some other python distro that instead sets __file__ to None, or whatnot; if you really want to catch this, we should just raise a wrapper exception that wraps whatever failed in the whole load process.)

indygreg · 2022-05-20T01:18:16Z

Thanks for filing this @konstin!

What's the fundamental requirement here? From the source code it seems to be that matplotlib needs a function pointer / address of Tk_FindPhoto, Tk_PhotoPutBlock, and Tcl_SetVar (Windows only).

Is this functionality critical? If matplotlib could run with these symbols as optional, what's the user impact? Is that an option worth considering?

As it stands, teaching python-build-standalone to export these symbols is doable. But the reason the symbols were made private is so they don't collide with an external shared library that some random other Python extension could pull in. See astral-sh/python-build-standalone#114 for more.

A hacky compromise here could be to alias these special symbols under a non-standard name so they don't conflict with the official symbols. That requires any C code to be aware of the alternate symbol names. That's a tall ask. But it also might be a fair request since this is the only Python C extension I'm aware of trying to access C symbols from libraries that aren't explicitly in libpython. I think you've just gotten lucky so far that this has managed to largely just work.

tacaswell · 2022-05-20T01:58:42Z

If we do not find these symbols, the user does not get the tkagg backend.

we should just raise a wrapper exception that wraps whatever failed in the whole load process.

fair, and then catch that in the fallback logic and try the next one?

anntzer · 2022-05-20T06:51:04Z

Yes (so that basically means we need to raise an ImportError, as that's what the fallback machinery recognizes -- now you've conviced me we do need to do such a wrapping).

tacaswell · 2022-05-23T01:08:26Z

The point of this run-time lookup is so that we do not have to a build-time dependency on tk.

I think this is closed by #23089 (we will now gracefully skip the tkagg backend in python-build-standalone case).

If you want to support mpl + tkagg @indygreg I think this is something you should handle on your side. As I said above, we will definitely consider reasonable patches if needed.

This patch is the result of: $ bazel run //tools/python:requirements.update -- --upgrade $ bazel run //tools/python:mirror_pip_packages --config=k8_upstream_python -- --ssh_host software I added special logic to exclude `pygobject` from being fixed with `auditwheel`. For some reason `auditwheel` fixing causes issues at runtime. I suspect it has to do with assumptions that the code makes about files living relative to the path of the .so file, but I'm not sure. Either way, this pulls in the latest matplotlib. In particular, the fix of interest is matplotlib/matplotlib#23074. Unfortunately, this means that the `rules_python` toolchain does not allow for tkagg plots. We'll probably have to use the Gtk runtime as a work around. Signed-off-by: Philipp Schrader <philipp.schrader@gmail.com> Change-Id: I083dc0fcae8315a7a40a351abad3ae2df0031c5d

This patch adds support for using matplotlib plus its Gtk backend with `--config=k8_upstream_python`. Unfortunately, the Tk backend doesn't work because of how Python is packaged. See the following issue for more information: matplotlib/matplotlib#23074 Separately, we need to patch pygobject and matplotlib for various hermeticity reasons. To accomplish this, I've added patching support for wheels downloaded via rules_python. I should have set it up to use annotations (the same mechanism we use for injecting deps), but it was a little cumbersome. Annotations are not set up for using in arguments to repository rules. I instead opted for a separate JSON file. You can find it at `tools/python/patches.json`. You can try the two new example programs: $ bazel run --config=k8_upstream_python //build_tests:matplotlib_example $ bazel run --config=k8_upstream_python //build_tests:pygobject_example Signed-off-by: Philipp Schrader <philipp.schrader@gmail.com> Change-Id: I3c4e8648a8012aa23dedd53610e82d590d1c409d

driftregion · 2024-12-27T06:44:20Z

Adding some extra context for other lost souls. I encountered this issue after switching to a python interpreter provided by bazel's rules_python with a statically linked tkinter.

I have run the following code test.py under two interpreters

import sys
print(f"interpreter: {sys.executable}")
print("_tkinter" in sys.builtin_module_names)
import tkinter
print(tkinter.TkVersion)
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk

interpreter: /home/nick/uctrace/.venv/bin/python
False
8.6

interpreter: /home/nick/uctrace/bazel-bin/uctrace/gui.runfiles/rules_python++python+python_3_10_x86_64-unknown-linux-gnu/bin/python3
True
8.6
Traceback (most recent call last):
  File "/home/nick/uctrace/test.py", line 6, in <module>
    from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk
  File "/home/nick/.local/lib/python3.10/site-packages/matplotlib/backends/backend_tkagg.py", line 1, in <module>
    from . import _backend_tk
  File "/home/nick/.local/lib/python3.10/site-packages/matplotlib/backends/_backend_tk.py", line 25, in <module>
    from . import _tkagg
ImportError: module '_tkinter' has no attribute '__file__'

tacaswell · 2024-12-27T20:50:46Z

@driftregion what version of mpl do you have? Post 3.6 I think the failure should be graceful (as if tk is not available).

driftregion · 2024-12-28T04:41:13Z

@tacaswell mpl version 3.10.0. I'm doing some interactive plotting within a tkinter app, so TkAgg is the only choice of backend afaik. My workaround is to use a different interpreter that references an external _tkinter.cpython-310-x86_64-linux-gnu.so.

tacaswell · 2024-12-29T05:37:54Z

Ah, you are forcing the use of tk rather than us failing on auto-discovery. In that case we should be failing like this and using a differently built interpreter is your only option (as would be the case if you had a Python build that simply did not include _tkinter).

QuLogic added the GUI: tk label May 19, 2022

anntzer mentioned this issue May 21, 2022

Normalize tk load failures to ImportErrors. #23089

Merged

6 tasks

tacaswell closed this as completed May 23, 2022

tacaswell added this to the v3.6.0 milestone May 23, 2022

hrektts mentioned this issue May 22, 2023

[Bug]: matplotlib crashes if _tkinter doesn't have __file__ #25944

Closed

gdippolito mentioned this issue Jul 8, 2024

Support matplotlib >=3.6.0 nutonomy/nuscenes-devkit#1095

Open

bluthej mentioned this issue Aug 31, 2024

tkagg is not compatible with uv-managed Python installs astral-sh/uv#6893

Open

driftregion mentioned this issue Dec 27, 2024

Feature request: Hermetic toolchain w dynamic linkage bazel-contrib/rules_python#2540

Closed

driftregion mentioned this issue Dec 30, 2024

Can't import matplotlib on linux: module '_tkinter' has no attribute '__file__' astral-sh/python-build-standalone#129

Closed

Avasam mentioned this issue Feb 3, 2025

AttributeError: module '_tkinter' has no attribute '__file__' with indygreg standalone CPython distribution pyinstaller/pyinstaller#9012

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: matplotlib crashes if `_tkinter` doesn't have `file` #23074

[Bug]: matplotlib crashes if `_tkinter` doesn't have `file` #23074

konstin commented May 19, 2022

QuLogic commented May 19, 2022

anntzer commented May 19, 2022

tacaswell commented May 19, 2022

anntzer commented May 19, 2022

indygreg commented May 20, 2022

tacaswell commented May 20, 2022

anntzer commented May 20, 2022

tacaswell commented May 23, 2022

driftregion commented Dec 27, 2024

tacaswell commented Dec 27, 2024

driftregion commented Dec 28, 2024

tacaswell commented Dec 29, 2024

[Bug]: matplotlib crashes if _tkinter doesn't have __file__ #23074

[Bug]: matplotlib crashes if _tkinter doesn't have __file__ #23074

Comments

konstin commented May 19, 2022

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

QuLogic commented May 19, 2022

anntzer commented May 19, 2022

tacaswell commented May 19, 2022

anntzer commented May 19, 2022

indygreg commented May 20, 2022

tacaswell commented May 20, 2022

anntzer commented May 20, 2022

tacaswell commented May 23, 2022

driftregion commented Dec 27, 2024

tacaswell commented Dec 27, 2024

driftregion commented Dec 28, 2024

tacaswell commented Dec 29, 2024

[Bug]: matplotlib crashes if `_tkinter` doesn't have `file` #23074

[Bug]: matplotlib crashes if `_tkinter` doesn't have `file` #23074