I run local models for two things. One is automation that needs privacy. The other is automation that needs speed. Everything else stays on OpenAI.
The useful stuff is boring. I have a local model that reads my incoming invoices and categorises them. It never leaves my machine. No data uploaded. It takes two seconds per invoice. That used to take me five minutes.
I also use local LLMs to generate commit messages from git diffs. The model runs on my laptop. It finishes before I can switch windows. Cloud models would add latency and cost for no benefit.
But I do not use local models for writing. They are not good enough yet. The small ones hallucinate more. The large ones need expensive hardware. The middle ground is not worth the hassle.
The real win is combining both. Local for structured tasks with private data. Cloud for creative work where quality matters. Pick the tool by the job.
If you are trying to replace ChatGPT with a 7B model running on a Raspberry Pi, you will be disappointed. If you use local LLMs for the jobs they actually do well, they become indispensable.
Paul