I’d like to keep the Local LLM Tools and Agents thread clean, so I’m opening this thread for any questions, discussions, or comments.
What kind of hardware do you need to run these? How do they compare to mainstream online models?
There are two things here: Tools that make use of LLMs, and tools for running LLMs locally.
The tools that work using LLMs, like Newelle, AutoGen, Dyad, don’t need you to be running LLMs locally, and can use API’s.
These should run perfectly on any hardware.
For running LLMs themselves locally, you should be able to do it on a computer without a dedicated GPU, but you’ll have to use small or distilled quantized models, like Google Gemma or Microsoft Phi.
Depends on which models you’re running, and how small they are.
If you have a dedicated GPU, you’ll be able to run larger models, but even if you aren’t, you’ll be surprised how intelligent some of the smaller models can be.
Try them.
Thanks! Going to try this
Do i need llm for this?
what does that mean? i am new to these things
Yes.
You’ll need an API key from Gemini, ChatGpt, etc.
What is a node.js ?
This is exactly what i was looking for, thanks for sharing
Which one?
Dyad
.