How to use it with GPT4all ?
#2
by eshaanagarwal - opened
Can anybody guide me to steps to use so that i can use it with gpt4all. I am a beginner and i dont know which file to download and how to initialise.
AFAIK this version is not compatible with GPT4ALL. It was trained to serve as base for a future quantized model that is not yet available as of today.
Then should i use it through HUgging face pipeline to make it work as a LLM for chatbot
That's right. It does work with huggingface tools. I published a Google Colab to demonstrate it. You'll probably need a paid colab subscription since it uses around 29GB of VRAM.
There is a PR for merging Falcon into GGML/llama.cpp so once that's finished, we will be able to use this within GPT4All: https://github.com/ggerganov/ggml/pull/231
zpn changed discussion status to closed