Fixing Ollama ROCm GPU Core Dumps In Docker & Podman

Alright, guys, if you're diving deep into the world of local large language models (LLMs) with Ollama and you've got an AMD GPU, especially something as potent as an AMD AI Max+ 395, you've probably encountered a few bumps in the road. One of the most frustrating issues, hands down, is when your ollama container, running in Docker or Podman, decides to throw a core dump while trying to detect your awesome ROCm GPU. It's like your super-powered hardware just… gives up, forcing ollama to fall back to the CPU. Trust me, nobody wants their cutting-edge AMD AI Max+ 395 GPU sitting idle when it should be accelerating your AI models! This isn't just about a minor hiccup; it's about unlocking the full potential of your hardware for tasks that demand serious computational muscle. We're talking about running those complex AI models faster, more efficiently, and without the agonizing wait times that come with CPU-only processing.

The problem usually manifests as a core dump during the GPU detection phase, leading to a complete bypass of your dedicated graphics card. This means your ollama instance is stuck using your CPU, which, while capable, just can't keep up with the demands of modern LLMs compared to a specialized GPU. The whole point of getting a powerful ROCm GPU like the AMD AI Max+ 395 is for its parallel processing capabilities, which are perfectly suited for AI workloads. When a core dump prevents ollama from seeing and utilizing that GPU, you're essentially leaving a massive amount of performance on the table. For users on systems like Fedora 43 Silverblue, integrating ollama with Podman using systemd quadlet can introduce unique challenges related to container isolation, device passthrough, and SELinux policies, making the troubleshooting process a bit more intricate. This issue is particularly critical because ollama relies heavily on efficient GPU utilization to deliver a responsive and practical experience for running AI models locally. Without proper ROCm GPU detection, your ollama setup, instead of being a powerhouse, becomes a bottleneck. So, let's roll up our sleeves and figure out why your ollama container is getting cold feet with your ROCm GPU and how to get it recognizing that AMD AI Max+ 395 like a champ. We'll dive into the nitty-gritty of the problem, decode those cryptic error messages, and arm you with the knowledge to get your AI acceleration back on track.

Understanding the Core Dump Conundrum: Ollama and ROCm GPUs

When ollama hits a core dump during ROCm GPU detection within a Docker or Podman container, it's essentially a fatal error where the program unexpectedly terminates and dumps its memory state for debugging. For us, this means ollama can't properly initialize or communicate with your AMD AI Max+ 395 GPU's ROCm drivers and associated libraries. Think of it like a translator suddenly forgetting how to speak a crucial language—the conversation between ollama and your GPU breaks down completely. This often points to deeper issues related to how the containerized environment is interacting with the host system's GPU drivers and hardware interfaces. The core dump is a critical symptom, indicating that ollama attempted an operation that caused a fundamental instability, likely within the libggml-hip.so or libhsa-runtime64.so libraries, which are central to ROCm acceleration. These libraries are the backbone of ROCm's ability to enable GPU acceleration for AI workloads, and when they fail, the entire system for ollama on the GPU crumbles.

Let's break down the scenario: you're trying to run ollama (which is fantastic for local LLMs) in a container, leveraging the ollama/ollama:rocm image, specifically designed for ROCm-compatible AMD GPUs. Your machine, a Fedora 43 Silverblue system, is equipped with an AMD AI Max+ 395, a beast of a card. You've set up your Podman container using a systemd quadlet—a pretty solid and modern way to manage containers on Linux. You've correctly added /dev/kfd and /dev/dri as devices, which are absolutely crucial for ROCm to access the GPU, and even mapped the necessary GroupAdd entries (like 39 and 105, often corresponding to render and video groups) for permissions. Despite all this diligent setup, the ollama process within the container crashes, specifically when it tries to discover available GPUs.... The key culprits often highlighted in the logs are modules like /usr/lib/ollama/rocm/libggml-hip.so and /usr/lib/ollama/rocm/libhsa-runtime64.so.1.14.60303. The libggml-hip.so is part of the GGML library, which ollama uses for its AI computations, specifically compiled to use ROCm's HIP (Heterogeneous-compute Interface for Portability). The libhsa-runtime64.so is the HSA (Heterogeneous System Architecture) runtime library, fundamental to ROCm operations. A core dump involving these libraries strongly suggests an incompatibility or corruption in the ROCm driver stack that ollama is attempting to use, either due to mismatches between the host kernel modules and the container's user-space libraries, or an issue with how the AMD AI Max+ 395 is being exposed and initialized within the container environment. The log snippet failure during GPU discovery and runner crashed clearly points to ollama being unable to establish a stable connection with the GPU, leading to its unfortunate fallback to the CPU. Understanding this basic failure point is the first, crucial step in our troubleshooting journey, laying the groundwork for us to investigate the specific components that are causing the breakdown in communication between ollama and your powerful ROCm GPU. We need to ensure that every part of the stack, from the host kernel drivers to the container's runtime libraries, is in perfect harmony for ollama to properly leverage your AMD AI Max+ 395.

Decoding the Error Logs: What Went Wrong?

Alright, let's get into the nitty-gritty of those logs, because they're telling us a pretty clear story about why your ollama container is having a meltdown during ROCm GPU detection. The most critical lines here confirm the core dump and point directly to the problematic components. We see Process 118677 (ollama) of user 0 dumped core. This is the unequivocal signal that the ollama process crashed hard. Following this, the logs mention Module /usr/lib/ollama/rocm/libggml-hip.so without build-id. and Module /usr/lib/ollama/rocm/libggml-hip.so. This is a huge red flag, guys. libggml-hip.so is the heart of ollama's ability to use ROCm GPUs. It's GGML (the library ollama uses for many computations) specifically compiled with HIP support for AMD GPUs. When this library is involved in a core dump, it usually means one of a few things: either there's a serious incompatibility between this library and your host's ROCm drivers, or the library itself is corrupted, or it's trying to access a GPU feature that isn't available or properly exposed.

Even more telling is the stack trace, which includes 0x00007fe874a75617 n/a (/usr/lib/ollama/rocm/libhsa-runtime64.so.1.14.60303 + 0x75617) and 0x00007fe874a73ec9 n/a (/usr/lib/ollama/rocm/libhsa-runtime64.so.1.14.60303 + 0x73ec9). This library, libhsa-runtime64.so, is the ROCm HSA runtime. It's the low-level interface that allows applications (like ollama via GGML) to talk directly to your AMD GPU. A crash here is absolutely critical because it means the fundamental communication layer between software and hardware has failed. This isn't just a minor application error; it's a deep-seated issue within the ROCm stack itself, or how that stack is being presented to the ollama container. It could indicate mismatched versions between your host ROCm driver and the ROCm libraries within the ollama:rocm container image, or perhaps a specific instruction or memory access from libggml-hip.so is leading to an illegal operation within the HSA runtime when interacting with your AMD AI Max+ 395.

Finally, we see time=2025-11-29T22:48:54.316Z level=INFO source=runner.go:449 msg="failure during GPU discovery" OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" extra_envs="map[GGML_CUDA_INIT:1 ROCR_VISIBLE_DEVICES:0]" error="runner crashed". This is ollama explicitly stating that its GPU discovery process failed because a