How to Force Ollama to Use Your AMD GPU—Even if It’s Not Officially Supported
Hello, Daniel here from Tiger Triangle Technologies! Today, I’m diving into a workaround to help you force Ollama to recognize and utilize an AMD GPU that’s technically unsupported. If you’re a fan of AMD like me and felt the sting of exclusion when Ollama's official list didn’t include your model, you’re in the right place.
Ollama announced support for AMD GPUs in March 2024, which was thrilling news for those of us who prefer Team Red. However, excitement quickly dwindled upon closer inspection: my AMD Radeon 6750 XT was missing from the compatibility list. If you’re in a similar boat, no worries—there’s a solution for us AMD users on Windows. Here’s a straightforward guide to get you up and running with your “unsupported” AMD card!
Step 1: Reviewing Ollama’s Official AMD Compatibility
Before you jump into any workaround, it’s best to understand the official limitations. Ollama’s documentation does list compatible GPUs, but if you’re not seeing your model there, as in my case, the software won’t natively tap into your card’s power. While Linux users have a bit more flexibility with override options, Windows users have to get creative.
Step 2: The Community Fix—An Ollama Fork for AMD GPUs
Enter a community-driven solution: LikeLoveWant’s fork of Ollama for AMD. This modified version of Ollama was created to extend support to more AMD models, including those not on the official list. The fork enables Ollama to leverage your AMD card, allowing you to experience the performance boost you’d expect from your GPU investment.
You can find this fork on GitHub, where you’ll notice a version specifically built for Windows users like us. You’ll also need to download and install the appropriate ROCm (Radeon Open Compute) libraries, which are essential for optimizing performance on AMD GPUs. If you’re unfamiliar, ROCm is AMD’s open-source stack for GPU programming. For those new to ROCm, don’t worry—you don’t need the entire development setup; just the library for your specific GPU.
Step 3: Precautions Before You Begin
As with any system modification, a few precautions are in order:
- Create a System Restore Point – This ensures that if anything goes wrong, you can revert to your system’s previous state.
- Backup Ollama Files – If you’ve been using Ollama, you’ve probably downloaded sizable model files. To avoid redownloading, back up your
.ollamafolder located atC:\Users\[YourUsername]\.ollama.
Step 4: Installing the Modified Ollama Version
With your system prepared:
- Visit the GitHub page for LikeLoveWant’s Ollama for AMD and download the latest release.
- During installation, Microsoft Defender may warn you about an “unknown publisher.” This is common with open-source software, so you can choose “Run Anyway.”
Step 5: Replacing ROCm Libraries
Once the modified Ollama is installed, the next step is integrating the proper ROCm libraries:
- Identify the LLVM target for your GPU model from the ROCm compatibility matrix. For instance, my 6750 XT is labeled
gfx1031. - Download the libraries and replace files in your Ollama program folder with the rocblas.dll and rocblas/library folder from the demo release that matches your GPU architecture. (eg. the file in （C:\Users\usrname\AppData\Local\Programs\Ollama\lib\ollama)
Pro Tip: Instead of directly overwriting the DLL files, consider renaming the original files to keep them as backups. This way, you have the option to revert if needed.
Final Thoughts
This workaround might seem technical, but it’s a manageable solution for those wanting to leverage their AMD GPU fully. It’s impressive how the tech community steps up to provide solutions where official support is limited. So, if Ollama isn’t making the most of your AMD hardware, give this community fix a try and watch your workflow improve.
Note: There are some generative AI models where the CPU will actually perform better than GPU. This is probably an indication to use a different model. To see this in action and learn how to improve performance check out my video on improving the performance of Ollama.
Stay tuned for more tutorials and insights from Tiger Triangle Technologies. Let’s continue optimizing together!