I spent this morning running models on a laptop. No cloud, no API costs, no latency. The hype says local AI replaces everything. The reality is narrower but more practical.
The useful stuff is small batch automation. I have a script that reads incoming emails, summarises them using a 7B model, and dumps the output into a local file. No data leaves the machine. No privacy risk. It handles about thirty emails in two minutes. That is genuinely useful.
Text classification works well locally. Sorting support tickets, flagging urgent messages, categorising notes. A 3B model does this faster than a human and costs nothing after setup. The key is keeping expectations low. These models hallucinate less when the task is simple and the context is short.
Code completion for local projects is the other win. Running Codestral or DeepSeek Coder on a GPU gives you autocomplete that rivals cloud services. No subscription. No network drops. It just works.
What fails is complex reasoning. Local models choke on multi-step logic, long documents, or anything requiring precise arithmetic. They are useful as tools not replacements. Treat them like a smart intern who needs clear instructions and constant checking.
The hardware barrier is real. A decent GPU helps but an M-series Mac runs 7B models comfortably. Start with Ollama, pick a model smaller than 10B parameters, and automate one thing you hate doing. That is where the value lives.
Paul