• 1 Post
  • 9 Comments
Joined 3 years ago
cake
Cake day: June 25th, 2023

help-circle






  • I started working toward self hosting LLM for my small company using ollama and opencode as agent But I realized a good model like GLM 5 requures 250GB of RAM and 24GB vram with a 4080?? I dont know, this is what the LLM told me itself.

    I ended up using qwen-code2.7-7b-16k.

    Currently the best thing I have is my laptop, 16GB ram, i7 9750H gtx1650

    How do you guys selfhost? What models do you use that are actually good?