Instead of using character ai, which will send all my private conversations to governments, I found this solution. Any thoughts on this? 😅
Instead of using character ai, which will send all my private conversations to governments, I found this solution. Any thoughts on this? 😅
Ollama.com is another method of self hosting. Figuring out which model type and size for what equipment you have is key, but it’s easy to swap out. That’s just an LLM, where you go from there depends on how deep you want to get into the code. An LLM by itself can work, it’s just limited. Most of the addons you see are extra things to give memory, speech, avatars, and other extras to improve the experience and abilities. Or you can program a lot of that yourself if you know Python. But as others have said, the more you try to get out, the more robust a system you’ll need, which is why you find the best ones online in cloud format. But if you’re okay with slower responses and lower features, self hosting is totally doable, and you can do what you want, especially if you get one of the “Jailbroke” models that has had some of the safety limits modified out of them to some degree.
Also as mentioned, be careful not to get sucked in. Even a local model can be convincing enough sometimes to fool someone wanting to see things. Lots of people recognize that danger, but then belittle people who are looking for help in that direction (while marketing realizes the potential profits and tries very hard to sell it to the same people).
Absolutely. TheBloke’s fine-tuned models with their guardrails removed are the only conversational models I will run. I get enraged looking at AI telling me to curb my speech.
I do use Python but I haven’t touched AI yet so it’s going to be a learning-curve if I go down that route. I am hoping to get finetuned models OOTB for this kind of stuff but I know it’s a hard ask.
I was going to buy 2-3 used GPUs/new budget GPUs like the B580 but with the tariffs the prices of these are INFLATED beyond what I can afford to pay for them. Once something changes (financially speaking) I’ll probably throw enough VRAM at it to at least get the 8B models (probably not FP16 but maybe quantised to 4K/8K) running smoothly.
Thanks for the reminder. I have wanted to use character AI for so long but couldn’t bear to give away my thought patterns to them (look at my hypocrisy: I’m giving it all away anyway when everyone is free to scrape Lemmy). I guess I’m an idiot.