Not valid Cpython files cause crashes in crates/ruff_source_file/src/line_index.rs:115:28

[qarmin]

Ruff 0.1.11 - latest version from git

When trying to catch #9398 I found that ruff parses a lot of more files than cpython so it is possible that users will try to fix file with invalid syntax(not sure why, but currently I don't see similar crashes with valid files)

ruff --select ALL . --fix

error: Panicked while linting 130_IDX_0_RAND_6616489086532482962272.py: This indicates a bug in Ruff. If you could open an issue at:

    https://github.com/astral-sh/ruff/issues/new?title=%5BLinter%20panic%5D

...with the relevant file contents, the `pyproject.toml` settings, and the following stack trace, we'd be very appreciative!

panicked at crates/ruff_source_file/src/line_index.rs:115:28:
byte index 30 is out of bounds of `GUILLOTINE_PICS = [r"""
Ô|`
Backtrace:    0: ruff_cli::panic::catch_unwind::{{closure}}
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2021:9
   2: std::panicking::rust_panic_with_hook
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:783:13
   3: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:657:13
   4: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:170:18
   5: rust_begin_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645:5
   6: core::panicking::panic_fmt
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72:14
   7: core::str::slice_error_fail_rt
   8: core::str::slice_error_fail
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/str/mod.rs:88:9
   9: ruff_source_file::line_index::LineIndex::source_location
  10: ruff_linter::logging::DisplayParseError::from_source_code
  11: ruff_linter::logging::DisplayParseError::from_source_kind
  12: ruff_cli::diagnostics::lint_path
  13: ruff_cli::panic::catch_unwind
  14: ruff_cli::commands::check::lint_path
  15: rayon::iter::plumbing::Producer::fold_with
  16: rayon::iter::plumbing::bridge_producer_consumer::helper
  17: rayon_core::join::join_context::{{closure}}
  18: rayon::iter::plumbing::bridge_producer_consumer::helper
  19: rayon_core::job::StackJob<L,F,R>::run_inline
  20: rayon_core::join::join_context::{{closure}}
  21: rayon::iter::plumbing::bridge_producer_consumer::helper
  22: rayon_core::join::join_context::{{closure}}
  23: rayon::iter::plumbing::bridge_producer_consumer::helper
  24: rayon_core::job::StackJob<L,F,R>::run_inline
  25: rayon_core::join::join_context::{{closure}}
  26: rayon::iter::plumbing::bridge_producer_consumer::helper
  27: rayon_core::join::join_context::{{closure}}
  28: rayon::iter::plumbing::bridge_producer_consumer::helper
  29: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute
  30: rayon_core::registry::WorkerThread::wait_until_cold
  31: rayon_core::registry::ThreadBuilder::run
  32: std::sys_common::backtrace::__rust_begin_short_backtrace
  33: core::ops::function::FnOnce::call_once{{vtable.shim}}
  34: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2007:9
  35: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2007:9
  36: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/unix/thread.rs:108:17
  37: start_thread
             at ./nptl/pthread_create.c:444:8
  38: __GI___clone3
             at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

3800 files with problems -
NOT_PARSABLE.zip

[charliermarsh]

Awesome, thanks for reproducing!

[charliermarsh]

It looks like when we hit an EOF error, the location can be one-past-the-end (since that's the location of the cursor when lexing).

[charliermarsh]

Oohhh, I see the problem here. It's slightly different. We're using the "original" code when computing the display range for the "fixed" code (both of which contain syntax errors).