SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 15 hours agoWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?message-squaremessage-square14linkfedilinkarrow-up126arrow-down111file-text
arrow-up115arrow-down1message-squareWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · 15 hours agomessage-square14linkfedilinkfile-text
minus-squareinfinitevalence@discuss.onlinelinkfedilinkEnglisharrow-up4arrow-down1·14 hours agoThat’s fair, but nearly any modern CPU at least 32gb of RAM and a current GPU with 16gb is plenty. No need for a 4k system when a 1k-1.5k will do it. If you’re willing to Frankenstein things some of the used AI/ML/mining cards can be a decent value.
minus-squaremeowmeow@quokk.aulinkfedilinkEnglisharrow-up2arrow-down1·13 hours agoYes, but when you compare it to codex and Claude though, it’s significantly slower. Especially over time. Better crank that AC. I think in a few years we will have current cloud levels running pretty efficiently on current computers.
That’s fair, but nearly any modern CPU at least 32gb of RAM and a current GPU with 16gb is plenty. No need for a 4k system when a 1k-1.5k will do it.
If you’re willing to Frankenstein things some of the used AI/ML/mining cards can be a decent value.
Yes, but when you compare it to codex and Claude though, it’s significantly slower. Especially over time. Better crank that AC.
I think in a few years we will have current cloud levels running pretty efficiently on current computers.