I spent the last month running local models for small tasks. Not coding assistants or chatbots. Just automation glue.
The thing that works: categorising emails. I have a script that feeds each new message into a local 7B model. It tags the sender type, urgency, and topic. No data leaves my machine. No API costs. It's wrong about 5% of the time, but that's fine because I review the tags anyway. The 95% saves me clicking through 30 emails a morning.
What doesn't work: generating long content. Local models hallucinate more on long contexts and the quality drop from GPT-4 is real. Short classification tasks are where they shine.
Second useful thing: summarising my daily notes. I write stream-of-consciousness bullet points during the day. A local model turns them into a structured log each evening. Takes 10 seconds. I don't need it to be perfect. Just organised enough to search later.
Third: rewriting form responses. I have a set of standard replies for client queries. The model rephrases them based on the specific email tone. Saves me sounding robotic.
The pattern is clear. Local LLMs are not replacements for cloud models. They are cheap, private filters for structured data. Use them where accuracy can be 90% and speed matters. Everything else still goes to the cloud.
Paul