Skip to content

Spawned-thread imports recurse in importlib _blocking_on bookkeeping #7586

@smarcd

Description

@smarcd

Summary

On current RustPython tip, imports executed on a spawned VM thread fail with:

  • RecursionError: in comparison

This reproduces in bare RustPython embedding, outside any PyO3 integration.

RustPython version

Local cargo git checkout pinned at d201c48e1.

Minimal repro

use rustpython::InterpreterBuilderExt;
use rustpython_vm::InterpreterBuilder;

fn main() {
    let interpreter = InterpreterBuilder::new().init_stdlib().interpreter();
    interpreter.enter(|vm| {
        vm.start_thread(|vm| {
            let result = vm.import("collections", 0);
            match result {
                Ok(_) => println!("collections ok"),
                Err(err) => {
                    let mut rendered = String::new();
                    let _ = vm.write_exception(&mut rendered, &err);
                    println!(
                        "import err class={} msg={}",
                        err.class().name(),
                        rendered.trim_end()
                    );
                }
            }
        })
        .join()
        .unwrap();
    });
}

Observed traceback

Traceback (most recent call last):
  File ".../Lib/collections/__init__.py", line 29, in <module>
    import _collections_abc
  File ".../Lib/_collections_abc.py", line 35, in <module>
    from abc import ABCMeta, abstractmethod
  File ".../Lib/abc.py", line 85, in <module>
    from _abc import ...
  File "_frozen_importlib", line 1368, in _find_and_load
  File "_frozen_importlib", line 421, in __enter__
  File "_frozen_importlib", line 311, in acquire
  File "_frozen_importlib", line 170, in __enter__
RecursionError: in comparison

Narrowing

These succeed on a spawned VM thread:

  • direct dict.setdefault(...)
  • raw _weakref.ref(...)
  • import abc
  • import _collections_abc

These fail on a spawned VM thread:

  • import weakref
  • import collections
  • import array because it pulls in collections

The failure consistently lands in:

  • Lib/importlib/_bootstrap.py
  • _BlockingOnManager.__enter__()
  • _ModuleLock.acquire()
  • _blocking_on bookkeeping

Additional notes

  • This still reproduces when using vm.start_thread(...), so it is not just an arbitrary non-owner-thread embedding misuse.
  • Monkeypatch experiments suggest the bug is not just _WeakValueDictionary itself; the broader import-lock bookkeeping path seems to be involved.

Why this matters

I am working on a PyO3 frontend/backend split with a RustPython backend. A correct backend design needs a reliable spawned-thread/runtime-thread execution model, and this currently blocks that work.

Related context:

If useful, I can provide the standalone repro crate as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions