I run models locally on a Mac Mini with 64GB RAM. Not for privacy. For speed. Cloud APIs cost pennies but they add latency and rate limits. Local inference is instant.
First useful thing: text classification. I have a script that reads my email labels and sorts them into folders. No API call. No data leaving my machine. The model is a 7B parameter quantised version of Llama 3. It works. False positives are rare.
Second: structured data extraction. I feed it raw HTML from a few recurring invoices. It returns JSON with amounts, dates and vendor names. That goes straight into a spreadsheet. No regex maintenance. No fragile parsers.
Third: summarisation of personal notes. I write daily logs in plain text. A cron job runs a local model at midnight and produces a one paragraph summary. I review it the next morning. It helps me remember what I did.
What is not useful. Chatbots. Most people I know who set up local LLMs use them as chat interfaces. They talk to them for five minutes then forget. That is a toy. Real automation is headless. No GUI. No streaming. Just input, model, output.
If you run local models for automation, keep it simple. One script. One model. One task. Do not build a pipeline with five models chained together. That is maintenance debt waiting to happen.
Paul