Tuesday, November 12, 2024

The Case of the Slow Chatbot - and how to make it lightning fast

 



It was a brisk afternoon at Tiger Triangle Technologies, and the hum of computations filled the air, more eager than a bustling London thoroughfare. I had scarcely finished my previous experiment—indeed, quite a robust affair with Blazor web applications—when a most peculiar challenge presented itself: the elusive matter of optimizing performance for the Ollama software suite.

Seated at my desk, surrounded by stacks of benchmark records, diagnostics, and freshly sharpened code, I felt a thrill much akin to that of a detective on the brink of discovery. “Ah, here we are,” I muttered. “Today, we’re diving deep into the mysteries of local model performance.”

The challenge was one fit for a connoisseur of computational intrigues, a true fundamental, if you will, in the realm of Ollama’s capabilities. For those as intent on maximizing Ollama’s prowess as I was, today’s expedition would surely reveal insights worth their weight in bytes.

Friday, November 8, 2024

Force Ollama to Use Your AMD GPU

 


How to Force Ollama to Use Your AMD GPU—Even if It’s Not Officially Supported

Hello, Daniel here from Tiger Triangle Technologies! Today, I’m diving into a workaround to help you force Ollama to recognize and utilize an AMD GPU that’s technically unsupported. If you’re a fan of AMD like me and felt the sting of exclusion when Ollama's official list didn’t include your model, you’re in the right place.

Ollama announced support for AMD GPUs in March 2024, which was thrilling news for those of us who prefer Team Red. However, excitement quickly dwindled upon closer inspection: my AMD Radeon 6750 XT was missing from the compatibility list. If you’re in a similar boat, no worries—there’s a solution for us AMD users on Windows. Here’s a straightforward guide to get you up and running with your “unsupported” AMD card!

Check if Ollama is Using Your GPU or CPU


How to Check if Ollama is Using Your GPU or CPU

If you're aiming to get the most out of Ollama, knowing whether it’s using your GPU or CPU can make a significant difference in performance. This blog post is part of a series where I dive into ways to boost your Ollama experience, and today, we're focusing on how to determine if Ollama is actually using your GPU.