Personally, I would like to use AI, but I don’t due to it being non local. I know there are local AI that could do things, but I don’t know which models are the good one for each task. If someone can give me pointers for it, I’d be grateful, for exemple a good model for local coding :)
depends on your hardware and your preferred language. i think wizardcoder is a pretty common choice but the smallest useful version is around 14GB so you need the vram to accommodate it.
you need space for the context and runtime parameters too, but i think it should work. worst case there are some offloading settings you can do depending on the server you use. only way to knew is to try, really.
Thanks for the pointers. For the hardware, I have a 9070 XT with 16 Gb of VRAM. It’s sure that it can be very expensive. As I only do this as a hobby, I don’t want to pay that amount of money. I’m okay with having a slow llm as it wouldn’t be a tool I’d use often. I prefer to try doing things on my own and use the ai to help for little tasks first, such as checking why this one line of code didn’t want to work correctly or things like that.
Personally, I would like to use AI, but I don’t due to it being non local. I know there are local AI that could do things, but I don’t know which models are the good one for each task. If someone can give me pointers for it, I’d be grateful, for exemple a good model for local coding :)
depends on your hardware and your preferred language. i think wizardcoder is a pretty common choice but the smallest useful version is around 14GB so you need the vram to accommodate it.
Thanks, I’ll dig into this. BTW, I have a 9070 XT, with 16 Gb VRAM, so it should do the job I guess.
you need space for the context and runtime parameters too, but i think it should work. worst case there are some offloading settings you can do depending on the server you use. only way to knew is to try, really.
[deleted by user]
Thanks for the pointers. For the hardware, I have a 9070 XT with 16 Gb of VRAM. It’s sure that it can be very expensive. As I only do this as a hobby, I don’t want to pay that amount of money. I’m okay with having a slow llm as it wouldn’t be a tool I’d use often. I prefer to try doing things on my own and use the ai to help for little tasks first, such as checking why this one line of code didn’t want to work correctly or things like that.