Fix panic in select.select() when too many FDs specified#7948
Conversation
📝 WalkthroughWalkthroughThe PR imports ChangesSelect FD Limit Validation
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@extra_tests/snippets/stdlib_select.py`:
- Around line 48-65: The high-FD regression test that creates many sockets
(using TOO_MANY_SELECT_FDS, sockets = [ ... socket.socketpair()], and the
subsequent assert_raises(select.select, ...)) must be kept inside the
sys.platform != "win32" guard so Windows doesn't attempt to allocate thousands
of fds without the RLIMIT setup; move the block from the lines that create
sockets, run assert_raises(ValueError, select.select, sockets, [], [], 0),
delete sockets, and the subsequent socketpair/ValueError check (the lines
referencing sockets, a, b, socket.socketpair(), and assert_raises) into the same
non-Windows conditional that imports resource and sets RLIMIT_NOFILE.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: ca94b753-70ce-4193-966b-95c3ea32a8cf
📒 Files selected for processing (2)
crates/stdlib/src/select.rsextra_tests/snippets/stdlib_select.py
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
extra_tests/snippets/stdlib_select.py (1)
48-65:⚠️ Potential issue | 🟠 Major | ⚡ Quick winKeep the high-FD socket allocation inside the non-Windows guard.
Line 59-65 are dedented, so Windows still tries to allocate thousands of sockets before reaching the
ValueErrorassertion. That makes the regression test flaky or failing on Windows for the wrong reason.Suggested fix
# Too many descriptors for select.select() if sys.platform != "win32": import resource # 100 additional fds should be enough for interpreter needs need_fds = TOO_MANY_SELECT_FDS + 100 soft_max_fds, hard_max_fds = resource.getrlimit(resource.RLIMIT_NOFILE) soft_max_fds = max(soft_max_fds, need_fds) assert hard_max_fds >= soft_max_fds, "Not enough file descriptors for this test" resource.setrlimit(resource.RLIMIT_NOFILE, (soft_max_fds, hard_max_fds)) -sockets = [s for _ in range(TOO_MANY_SELECT_FDS // 2) for s in socket.socketpair()] -assert_raises(ValueError, select.select, sockets, [], [], 0) -del sockets -a, b = socket.socketpair() -# CPython disallows this on *nix systems too. -assert_raises(ValueError, select.select, [a] * TOO_MANY_SELECT_FDS, [], [], 0) -del a, b + sockets = [s for _ in range(TOO_MANY_SELECT_FDS // 2) for s in socket.socketpair()] + assert_raises(ValueError, select.select, sockets, [], [], 0) + del sockets + a, b = socket.socketpair() + # CPython disallows this on *nix systems too. + assert_raises(ValueError, select.select, [a] * TOO_MANY_SELECT_FDS, [], [], 0) + del a, b🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@extra_tests/snippets/stdlib_select.py` around lines 48 - 65, The high-FD socket creation and subsequent assertions are currently outside the Windows guard and must be moved inside the sys.platform != "win32" block: wrap or indent the block that creates sockets (the list comprehension using socket.socketpair() assigned to sockets), the assert_raises call that calls select.select with sockets, the del sockets, and the CPython-specific test using variables a, b (socket.socketpair() and the assert_raises on [a] * TOO_MANY_SELECT_FDS) so they only run when not on Windows; update the code around the symbols sockets, socket.socketpair, assert_raises(select.select, ...), and variables a,b accordingly to keep all high-FD allocation and CPython-only assertions inside the non-Windows guard.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@extra_tests/snippets/stdlib_select.py`:
- Around line 55-58: The current logic uses resource.getrlimit(RLIMIT_NOFILE)
and asserts hard_max_fds >= soft_max_fds which fails when hard_max_fds ==
resource.RLIM_INFINITY; modify the block around soft_max_fds, hard_max_fds and
the subsequent assert to explicitly handle RLIM_INFINITY: after calling
resource.getrlimit(RLIMIT_NOFILE) check if hard_max_fds ==
resource.RLIM_INFINITY and in that case skip or bypass the numeric comparison
(do not assert) and allow soft_max_fds to be increased to need_fds, then call
resource.setrlimit(RLIMIT_NOFILE, (soft_max_fds, hard_max_fds)) preserving the
RLIM_INFINITY sentinel if present; reference the variables soft_max_fds,
hard_max_fds, need_fds and the functions resource.getrlimit/resource.setrlimit
in your fix.
---
Duplicate comments:
In `@extra_tests/snippets/stdlib_select.py`:
- Around line 48-65: The high-FD socket creation and subsequent assertions are
currently outside the Windows guard and must be moved inside the sys.platform !=
"win32" block: wrap or indent the block that creates sockets (the list
comprehension using socket.socketpair() assigned to sockets), the assert_raises
call that calls select.select with sockets, the del sockets, and the
CPython-specific test using variables a, b (socket.socketpair() and the
assert_raises on [a] * TOO_MANY_SELECT_FDS) so they only run when not on
Windows; update the code around the symbols sockets, socket.socketpair,
assert_raises(select.select, ...), and variables a,b accordingly to keep all
high-FD allocation and CPython-only assertions inside the non-Windows guard.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 323e4791-56d8-4bb0-8ff1-c669f041bcbc
📒 Files selected for processing (2)
crates/stdlib/src/select.rsextra_tests/snippets/stdlib_select.py
Also:
* Add regression test into existing `extra_tests/snippets/stdlib_select.py`
* Stop calculating nfds on Windows as it is ignored there
Panic:
thread 'main' (189598) panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/libc-0.2.186/src/unix/linux_like/mod.rs:1777:9:
index out of bounds: the len is 16 but the index is 16
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' (189598) panicked at library/core/src/panicking.rs:225:5:
panic in a function that cannot unwind
stack backtrace:
0: 0xaaab763a0b88 - <<std[1934960bf7f41d0a]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[f1abae5f1257fe69]::fmt::Display>::fmt
1: 0xaaab75590ff0 - core[f1abae5f1257fe69]::fmt::write
2: 0xaaab763a94fc - <std[1934960bf7f41d0a]::sys::stdio::unix::Stderr as std[1934960bf7f41d0a]::io::Write>::write_fmt
3: 0xaaab7638b714 - std[1934960bf7f41d0a]::panicking::default_hook::{closure#0}
4: 0xaaab7639b288 - std[1934960bf7f41d0a]::panicking::default_hook
5: 0xaaab7639b478 - std[1934960bf7f41d0a]::panicking::panic_with_hook
6: 0xaaab7638b7ec - std[1934960bf7f41d0a]::panicking::panic_handler::{closure#0}
7: 0xaaab76382654 - std[1934960bf7f41d0a]::sys::backtrace::__rust_end_short_backtrace::<std[1934960bf7f41d0a]::panicking::panic_handler::{closure#0}, !>
8: 0xaaab7638c504 - __rustc[b7425922bef61dcf]::rust_begin_unwind
9: 0xaaab754f778c - core[f1abae5f1257fe69]::panicking::panic_nounwind_fmt
10: 0xaaab754f7714 - core[f1abae5f1257fe69]::panicking::panic_nounwind
11: 0xaaab754f786c - core[f1abae5f1257fe69]::panicking::panic_cannot_unwind
12: 0xaaab75a6283c - rustpython_vm::function::builtin::<impl rustpython_vm::function::builtin::sealed::PyNativeFnInternal<(rustpython_vm::function::builtin::OwnedParam<T1>,rustpython_vm::function::builtin::OwnedParam<T2>,rustpython_vm::function::builtin::OwnedParam<T3>,rustpython_vm::function::builtin::OwnedParam<T4>),R,rustpython_vm::vm::VirtualMachine> for F>::call_::h2471c8e242c9b51d
13: 0xaaab75db1e68 - rustpython_vm::types::slot::Callable::slot_call::hd1c1ad0ad14f306b
14: 0xaaab762c0a50 - rustpython_vm::protocol::callable::PyCallable::invoke::h9f6d571fca351ca6
15: 0xaaab75c550e8 - rustpython_vm::protocol::callable::<impl rustpython_vm::object::core::PyObject>::call_with_args::hed1f4a61aba2dced
16: 0xaaab762e7c24 - rustpython_vm::frame::ExecutingFrame::execute_call::h0ad3490dd74ed1e3
17: 0xaaab762fed40 - rustpython_vm::frame::ExecutingFrame::run::hcf90f0950fc26812
18: 0xaaab761e6768 - rustpython_vm::vm::VirtualMachine::with_frame::hd49ba6fcdf2422e2
19: 0xaaab75c45398 - rustpython_vm::builtins::function::<impl rustpython_vm::object::core::Py<rustpython_vm::builtins::function::PyFunction>>::invoke_with_locals::h42de3d2316941ce2
20: 0xaaab76132a80 - rustpython_vm::builtins::function::vectorcall_function::h7331cb67b334e867
21: 0xaaab763369d8 - rustpython_vm::protocol::callable::<impl rustpython_vm::object::core::PyObject>::vectorcall::h9019c5d16685c89a
22: 0xaaab762f4b54 - rustpython_vm::frame::ExecutingFrame::execute_call_vectorcall::h120134e11a58c946
23: 0xaaab76302a7c - rustpython_vm::frame::ExecutingFrame::run::hcf90f0950fc26812
24: 0xaaab761e6768 - rustpython_vm::vm::VirtualMachine::with_frame::hd49ba6fcdf2422e2
25: 0xaaab761e7f24 - rustpython_vm::vm::VirtualMachine::run_code_obj::h354618be6e5cc553
26: 0xaaab761e2d18 - rustpython_vm::vm::python_run::file_run::<impl rustpython_vm::vm::VirtualMachine>::run_any_file::h783d3127fbc0b523
27: 0xaaab757d700c - rustpython::run_rustpython::h354efb8d817cefbf
28: 0xaaab757c79e0 - std::thread::local::LocalKey<T>::with::hc9728e249843a926
29: 0xaaab757db860 - rustpython_vm::vm::interpreter::Interpreter::run::h42ac1fe9ed2287a2
30: 0xaaab757d7b30 - rustpython::run::hf14a209db5b4289c
31: 0xaaab757e2eb4 - rustpython::main::h1b59d8e13276ac48
32: 0xaaab757e2eec - std::sys::backtrace::__rust_begin_short_backtrace::h47e4b1f073f2155c
33: 0xaaab757e2ed4 - std::rt::lang_start::{{closure}}::h663a6c3dc7d80101
34: 0xaaab76399fd4 - std[1934960bf7f41d0a]::rt::lang_start_internal
35: 0xaaab757e2f44 - main
36: 0xfffed057655c - __libc_start_call_main
37: 0xfffed057663c - __libc_start_main@@GLIBC_2.34
38: 0xaaab755526f0 - _start
39: 0x0 - <unknown>
thread caused non-unwinding panic. aborting.
Aborted (core dumped) cargo run --release -- extra_tests/snippets/stdlib_select.py
There was a problem hiding this comment.
♻️ Duplicate comments (1)
extra_tests/snippets/stdlib_select.py (1)
49-69:⚠️ Potential issue | 🟠 Major | ⚡ Quick winKeep the high-FD regression block inside the non-Windows guard.
Lines 63-69 are still outside the
sys.platform != "win32"block, so Windows will try to allocate thousands of sockets before it ever reaches the Rust-side validation.🐛 Suggested fix
if sys.platform != "win32": import resource soft_max_fds, hard_max_fds = resource.getrlimit(resource.RLIMIT_NOFILE) if soft_max_fds != resource.RLIM_INFINITY: # 100 additional fds should be enough for interpreter needs need_fds = TOO_MANY_SELECT_FDS + 100 soft_max_fds = max(soft_max_fds, need_fds) if hard_max_fds != resource.RLIM_INFINITY: assert hard_max_fds >= soft_max_fds, ( "Not enough file descriptors for this test" ) resource.setrlimit(resource.RLIMIT_NOFILE, (soft_max_fds, hard_max_fds)) -sockets = [s for _ in range(TOO_MANY_SELECT_FDS // 2) for s in socket.socketpair()] -assert_raises(ValueError, select.select, sockets, [], [], 0) -del sockets -a, b = socket.socketpair() -# CPython disallows this on *nix systems too. -assert_raises(ValueError, select.select, [a] * TOO_MANY_SELECT_FDS, [], [], 0) -del a, b + sockets = [s for _ in range(TOO_MANY_SELECT_FDS // 2) for s in socket.socketpair()] + assert_raises(ValueError, select.select, sockets, [], [], 0) + del sockets + a, b = socket.socketpair() + # CPython disallows this on *nix systems too. + assert_raises(ValueError, select.select, [a] * TOO_MANY_SELECT_FDS, [], [], 0) + del a, b🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@extra_tests/snippets/stdlib_select.py` around lines 49 - 69, The high-FD regression test that allocates many sockets is not guarded by the sys.platform != "win32" check; move the block that creates sockets, asserts select.select raises, and the subsequent single-socket repeated-list test (the code using TOO_MANY_SELECT_FDS, sockets = [...], assert_raises(ValueError, select.select, sockets, [], [], 0), del sockets, and the a,b socketpair test) inside the non-Windows guard so Windows won't try to create thousands of socket pairs and will rely on the Rust-side validation instead.
🧹 Nitpick comments (1)
crates/stdlib/src/select.rs (1)
85-88: ⚡ Quick winExtract the platform-specific FD limit before the shared comparison.
Only the limit value differs here. Pulling that out first keeps the branch local and avoids duplicating the comparison logic.
♻️ Proposed refactor
- let too_many_fds = cfg_select! { - windows => v.len() > FD_SETSIZE as usize, - _ => v.len() > FD_SETSIZE, - }; + let fd_limit = cfg_select! { + windows => FD_SETSIZE as usize, + _ => FD_SETSIZE, + }; + let too_many_fds = v.len() > fd_limit;As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/stdlib/src/select.rs` around lines 85 - 88, The comparison duplicates only the platform-specific FD_SETSIZE value; extract that value first (e.g., define a fd_limit via the existing cfg_select! macro) and then perform the shared comparison once (compute too_many_fds = v.len() > fd_limit). Update the code that sets too_many_fds to use the new fd_limit variable and keep the cfg_select! usage limited to producing the platform-specific limit (FD_SETSIZE) so the comparison logic is not duplicated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@extra_tests/snippets/stdlib_select.py`:
- Around line 49-69: The high-FD regression test that allocates many sockets is
not guarded by the sys.platform != "win32" check; move the block that creates
sockets, asserts select.select raises, and the subsequent single-socket
repeated-list test (the code using TOO_MANY_SELECT_FDS, sockets = [...],
assert_raises(ValueError, select.select, sockets, [], [], 0), del sockets, and
the a,b socketpair test) inside the non-Windows guard so Windows won't try to
create thousands of socket pairs and will rely on the Rust-side validation
instead.
---
Nitpick comments:
In `@crates/stdlib/src/select.rs`:
- Around line 85-88: The comparison duplicates only the platform-specific
FD_SETSIZE value; extract that value first (e.g., define a fd_limit via the
existing cfg_select! macro) and then perform the shared comparison once (compute
too_many_fds = v.len() > fd_limit). Update the code that sets too_many_fds to
use the new fd_limit variable and keep the cfg_select! usage limited to
producing the platform-specific limit (FD_SETSIZE) so the comparison logic is
not duplicated.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 85cfe2f1-544e-4268-a3a3-5f85e459764f
📒 Files selected for processing (2)
crates/stdlib/src/select.rsextra_tests/snippets/stdlib_select.py
Also:
extra_tests/snippets/stdlib_select.pyPanic:
Summary by CodeRabbit
Bug Fixes
Tests