Anyone use offline LLMs?
-
irlslut — 1 month ago(February 24, 2026 11:28 PM)
I've only ever tried Ministral 3 (which was fast) and Gemma 3 which was much slower, but the most human. I tried to them some film trivia but they just lie and completely fabricate entire storylines. I'm not sure if this is because I'm using a truncated model that fits my ram (8GB) or if that's the way they all are.
I've never really had a use for them but judging by the amount of models out there, I'm wondering what people find them useful for, if anything. -
元才 — 1 month ago(February 24, 2026 11:50 PM)
There are lots of uses for an offline LLM. Mostly homebrews to bypass filtering and guardrails put on by the models which can be restrictive (And sued).
8Gb is a fairly tight system. The reason why RAM has shot up in price is because these models cannibalise that stuff and producing all sorts of things from videos and gaming models to complex physics simulations which can produce fairly reliable results.
The LLMs which stick to language are only as good as the datasets which feed them. So the old garbage in, garbage out rules apply (As they've always done).
For movie trivia searches you might as well stick to the models which use someone else's RAM. You'll get a much quicker result and it will be more accurate -
Uncreative — 1 month ago(February 24, 2026 11:54 PM)
One person on here became a billionaire by inventing a new water purification system with ChatGPT. He even funded it by selling a ratapult he also invented on ChatGPT.
Plus there's about a dozen FBers who generated kiddie porn on Grok. -
irlslut — 1 month ago(February 26, 2026 01:55 AM)
Well that's interesting. IIRC, I saw a news clip years ago where the Japanese were using games on their Nintendo DS for this and considered them real girlfriends. One even had a wife IRL. This was way before AI. I wonder what they use now and if/how they dumped their DS girlfriends..
And hey, I found the clip, it's a webpage on the bbc site; "BBC click Japan geeks: Virtual girlfriends"