Pytest 8+ Slows Tests: Fix Blosc2 Multithreading Overhead

Hey there, fellow developers! Ever updated a tool and suddenly felt like your computer was running through molasses? Well, that's exactly the kind of head-scratcher we're diving into today. We're talking about a peculiar performance hit that many folks, especially those using python-blosc2, might encounter when upgrading their pytest version to 8.0 or newer. It's a situation where your test suite's execution time can more than double, turning a brisk minute-and-a-half run into a grueling four-minute marathon. Trust me, nobody wants to wait around for tests, especially in a fast-paced development environment or a busy CI/CD pipeline. This isn't just a minor annoyance; it can seriously impact your productivity and the efficiency of your automated builds. So, if your pytest suite suddenly feels sluggish after a recent update, particularly with blosc or blosc2 libraries involved, stick around because we're going to break down why this happens and, more importantly, how to fix it. We'll explore the underlying causes, the hidden global states, and the multithreading overhead that can silently creep into your testing setup, turning a simple version upgrade into a performance nightmare. Understanding these dynamics is crucial for maintaining a healthy, speedy, and reliable test suite, allowing you to focus on building awesome features rather than staring at a progress bar.

The Curious Case of Pytest 8 and Test Speed

Alright, guys, let's get right into the heart of the matter: the sudden and dramatic increase in test execution time with pytest version 8 and above. Imagine this scenario: your test suite is humming along nicely, finishing up in about 90-100 seconds. You decide to keep things fresh, update your dependencies, and boom! The same suite, with the exact same tests, now takes over 240 seconds to complete. That's a jump from roughly a minute and a half to over four minutes – a more than 150% increase in execution time! This isn't just a minor delay; it's a significant roadblock, especially for larger projects with extensive test suites. We're talking about a real productivity killer here. What changed? Why did an innocent pytest update unleash this performance monster? The initial reaction might be to blame pytest itself, but as we often find in the world of software, the truth is usually a bit more nuanced and involves interactions with other libraries, in this case, blosc and blosc2. The core of the issue, as we'll soon discover, isn't necessarily a bug in pytest but rather a change in its internal test discovery and execution order. This subtle shift exposes an underlying performance bottleneck related to how blosc2 manages its internal thread pool. When tests are executed in a different sequence, a crucial blosc2.set_nthreads(1) call, which previously ran early in the suite and inadvertently sped up subsequent tests, now happens much later. This leaves a large chunk of the test suite running with blosc2's default (often much higher) thread count, leading to severe multithreading overhead for many small, quick tests. The impact is particularly pronounced on systems with many logical cores, where the overhead of managing a large number of threads for trivial tasks becomes a huge drag. This scenario highlights the importance of not just updating dependencies but also monitoring performance regressions and understanding how changes in one library can ripple through your entire development stack. Our goal here is to unravel this mystery and ensure your test runs are always as snappy as they should be, maintaining that crucial fast feedback loop that developers cherish.

Digging Deeper: Unmasking the Threading Culprit

So, after the initial shock of seeing our test times skyrocket, it was time to put on our detective hats and figure out what in the world was going on. The first clue, a rather subtle one, emerged from examining the test execution logs. We noticed that pytest >= 8 started running tests from tests/ndarray before the ones in the main tests/ directory. Now, this might seem like a trivial detail, but in the intricate dance of a large test suite, execution order can be everything. This change in sequencing was the key piece of the puzzle. Why? Because it hinted that some test in tests/ was performing an action that affected the entire suite, and by running it later, that beneficial action was delayed. After some focused investigation and trying to isolate specific test files, we finally hit the jackpot! The culprit was found lurking in tests/test_storage.py::test_dparams_defaults. This particular test was making a call to blosc2.set_nthreads(1). For those unfamiliar, blosc2 is a high-performance compressor optimized for numerical arrays, and set_nthreads() allows you to control how many CPU threads it utilizes for compression and decompression operations. The test_dparams_defaults test, for its own validation purposes, was explicitly setting the number of Blosc2 threads to one. And here's the kicker, folks: this setting was persisting for all subsequent tests! In pytest < 8, this test would run relatively early, essentially