I am planing to use retrieval augmented generation (RAG) based chatbot to look up information from documents (Q&A).

I did try with GPT3.5 and It works pretty well. Now I want to try with Llama (or its variation) on local machine. The idea is to only need to use smaller model (7B or 13B), and provide good enough context information from documents to generate the answer for it.

Have anyone done it before, any comments?

Thanks!