How to Run Local LLM on Your Mac in 60 Seconds

Throughout the 2023 and 2024, the biggest trends in tech is the “rise” of AI systems. To be more specific, the rise of smart chatboxes that can mimic human conversations. Of course, later, AI-generated Art comes to the scene and the whole AI, or more specifically, content generation through Large Language Models or Transformation process exploded with a few leading edge companies like OpenAI coming to the scene.

The common theme of those AI companies is they require loads and loads of servers, GPUs, networking equipment, and energy to host and generate those texts and images. What if we can have our own smart chat boxes that runs right from your Mac? What if we can set it all up under 60 seconds?

There are a number of reasons why you want your own private LLM. The #1 reason would be privacy. Say you want to use an LLM to analyze your banking or patient data. Those are the kinds of data that you do not want to be in public or in the hands of another company that will pinky swear that it won’t use your data.

How to do it

The way to install an LLM on your Mac is just by following those 3 simple steps

• Go to Ollama website and download the app.
• Start the App, it will copy the app in your Applications folder (with your permission of course).
• Once the app is started, open Terminal and run these commands. The example command runs the DeepSeek model which made a splash by challenging OpenAI at a lower cost.

ollama run deepseek-r1

Once the prompt appears, you just ask a question and the model will try it best to answer.

So that’s it, you already have a smart chatbox running on your machine. To exit the prompt, use Ctrl + d to do it.

Next Steps

Now we got a working LLM running right from your Mac, what is the next steps? For one, you can put a nicer interface on top of the LLM. Ollama have a REST API, so 3rd party software can interface witht he LLM to give me more creature comfort like a nicer interface, some chat history and even create pictures for you, right on the UI.

Here are some of the examples of chat UI for your LLM:-

• webUI - The most popular UI, even have an UI that mocks OpenAI
• chatbox AI - One contender that have a desktop app.

Caveat and Pitfall

While it is nice to have an LLM working personally for you, there is some caveat that needs to be considered. There is a reason why companies like OpenClosedAI spent billions of dollars on millions of servers building their AI systems.

For starters, there is a vast gulf of capabilities and breadth of knowledge between the AI system that runs on your Mac and OpenAI. Your local LLM might know the height of the Statue of Liberty but might not know the straight line distance between the Statue and the Empire State Building. And that is just the surface and it goes even worse from there.

Plug

Support this free website by visiting my Amazon affiliate links. Any purchase you make will give me a cut without any extra cost to you

Other Ecosystem Items

• Apple TV (Amazon) / Apple TV 4K (Amazon)
• AirTags (Amazon)
• HomePod Mini (Amazon)