The useful case for local LLMs is narrow. It is not about matching GPT-4. It is about running a small model on data that cannot leave the building.
The real driver is usually compliance or contract terms. Client data, patient records, internal strategy documents. Once the data cannot go to a cloud API, the local model becomes the only option. That is the entire argument. Everything else is a bonus.
What actually works in automation is structured extraction. Pulling fields out of invoices, routing support tickets, classifying documents, turning free text into JSON. Small models do this reliably when the task is fixed and the output schema is strict. They do not drift as much as people expect, because the task never changes.
What does not work is treating a local model like a general assistant. Running a 70B model on a workstation and hoping for GPT-4 quality is a waste of time and electricity. The useful models are the 7B to 13B range, quantised, running on a decent GPU. CPU-only inference is fine for a demo and painful for real workloads.
The pattern that survives contact with production is simple. One narrow task, one fixed prompt, one output schema. Test it against a batch of real examples before wiring it into anything. If it fails on the edge cases, do not add more prompt engineering. Change the schema or split the task.
Local LLMs are not a revolution. They are a way to keep certain jobs inside the firewall. That is enough.
Paul