We’ve been building LLM based tools for months, but I think that there should be efficient frameworks by now that actually add value. I tried langchain a while back but I felt like it was just an over complicated overhead where it was always simpler to make everything from scratch each time. Guidance has been the only real improvement for me as it does way more than basic prompt templating, but it is in no way a full framework.

Now there are LlamaIndex, TigerLab, Langchain… but I simply don’t have the time to test them all.

We need to run the models by ourselves, so no Open AI api, ideally run something compatible with TGI / VLLM. We need to connect to proper databases and vectorDB (currently using Milvus). And I’m looking for something that is actually useful and I don’t have to struggle and hack the library everytime I want to do something slightly different.

Does any of you have a good recommendation? Everything changes so quickly I feel like I can’t trust articles that are older than two months. So what are you currently using and what has been an overhyped crap?