Running a local model used to mean sacrificing quality for privacy. That trade has narrowed. For automation, the useful ones are small, fast and good at one thing. Extracting fields from invoices, classifying support tickets, rewriting text to a style guide. These are tasks where a mistake costs little and the data never leaves the box.
The real win is not the model itself, it is the pipeline around it. A local LLM behind a queue, with a fallback to a bigger model for hard cases, gives you most of the benefit without the data exposure. You can run that on a modest GPU or even CPU for small batches. Latency is predictable. No per-token billing. That changes what you are willing to automate.
What is not useful is trying to replace every API call with a local model. General knowledge, complex reasoning, long context summarisation. Those still struggle on commodity hardware. You end up with a slow, janky system that needs constant prompting tweaks. That is not automation, it is babysitting.
The other trap is overfitting to the demo. A local model that nails your test set but fails on a slightly different format is worse than no model. You need to test against real edge cases, and you need to log failures properly. That is boring but it is the actual work.
So the useful local LLM is the one you forget is there. It sits in the middle of a job, does the dull extraction, hands off to something deterministic. No chat window, no agent. Just a function that happens to be a model. That is where the value is.