I run local LLMs. Not to replace ChatGPT, but to automate the small, repetitive tasks I do every day. The key is picking jobs where latency and privacy matter more than raw intelligence.
The most useful thing I found is text classification. Sorting emails, tagging notes, categorising support tickets. A tiny 7B model runs in under a second on my laptop and never sends data anywhere. It's reliable enough for a 90 percent accuracy filter. The remaining 10 percent I handle manually.
Summarisation works well too. Daily logs from my servers, long email threads, meeting transcripts. I feed them into a local model each morning and get a one paragraph summary. No API costs, no data leaks. It's not as creative as GPT-4 but it's consistent and fast.
Code generation is trickier. Simple one shot tasks like writing a bash script or a regex pattern work fine. Anything multi step still needs a cloud model. I treat local models as a first draft generator, then refine with a bigger model when needed.
The mistake I see is people trying to run massive 70B models locally. They buy expensive hardware and get disappointed. The real wins are with smaller models doing narrow tasks. A 7B or 13B model on a Mac Mini handles most automation needs without breaking a sweat.
Start with one boring task. Automate it. Expand from there.
Paul