PieFedSelfhost an LLM
submitted by
ShimitarHi all,
i am quite an old fart, so i just recently got excited about self hosting an AI, some LLM…
What i want to do is:
- chat with it
- eventually integrate it into other services, where needed
I read about OLLAMA, but it’s all unclear to me.
Where do i start, preferably with containers (but “bare metal") is also fine?
(i already have a linux server rig with all the good stuff on it, from immich to forjeio to the arrs and more, reverse proxy, Wireguard and the works, i am looking for input on AI/LLM, what to self host and such, not general selfhosting hints)
Share on Mastodon
Share on Reddit
Share on WhatsApp
If you like videos, I’d highly recommend techno Tims video on how to do this. Its what I used when building mine. Link
100% agree. TechnoTim is quite good. Also take a look at NetworkChuck. But be aware, these two will send you down rabbit holes of self-hosting ideas. Awesome rabbit holes, but rabbit holes nonetheless. I’ve spent weeks playing with stuff they’ve suggested. N8n and MCP is my latest obsession.