I run a bunch of small models on my laptop. They do two things well. One is structuring messy text. I pipe in garbled meeting notes and get back clean summaries. The other is generating repetitive variants of templates. Think email responses with different tones or product descriptions with adjusted keywords. Neither requires a cloud connection or a GPU farm.
What I do not do is ask them to reason about business logic or make decisions. They hallucinate facts and miss context. You get plausible nonsense dressed in confident prose.
The real win comes from chaining models with simple code. I wrote a script that takes a support ticket, runs a small model to classify the sentiment, then passes the result to a rule engine for routing. That combination is reliable. The model does one fuzzy job while the code handles everything precise.
You want automation that works. Pick tasks where a wrong answer is harmless or easily caught. Test your model on real data before you build anything around it. Most people I see fail because they assume the model understands intent. It does not.
Run cheap, simple models for narrow jobs. Keep them offline. That is where the actual value lives.
Paul