Omg how can I run it on 3090?

For PDF , docx and like 50 more formats, use h2oGPT, great for this kind of stuff.

You should look at LoLLMs webui, it has those options

If you need any help with the local.env. File, tell me and il help out

I had some struggles with it, it works best for me in combination with llamacpp, and you need to run a docker command to start a mongo DB for you chats locally.

Even the search results can be queried on your device instead of API.

Im not great at troubleshooting errors but the install of chat-ui was pretty straightforward.

If you already have a llamacpp server it would be very easy to connect.

I enjoy the search functionality so much and I think its worth the hassle, if you need any help with it just comment here.

Cool! Can the list be added to the main repo ( GitHub - sindresorhus/awesome: 😎 Awesome lists about all kinds of interesting topics )

Or linked there under a small category?

People need to know about all of those great alternatives to ChatGPT :D

Wow looks very good indeed, how is the web extraction plugin? can you share some screenshots?

Agreed, will add that !

There are 3 options that I have found, they all work.

1. TextGenerationWebui - web_search extension (there is also a DuckDuckGO clone in github)
2. LolLLMs - There is an Internet persona which do the same, searches the web locally and gives it as context
3. Chat-UI by huggingface - It is also a great option as it is very fast (5-10 secs) and shows all of his sources, great UI (they added the ability to search locally and run LLMS models locally recently)

​

GitHub - simbake/web_search: web search extension for text-generation-webui

GitHub - ParisNeo/lollms-webui: Lord of Large Language Models Web User Interface

GitHub - huggingface/chat-ui: Open source codebase powering the HuggingChat app

If you ask me, try all 3 of them!

Thank you for the response!

Il try to adjust the temp too, how can I disable samplers in oobabooga? what is the setting?

Is there a way to set rep penalty lower than 1?

I found the fix for this issue (Tested by me only, thanks to u/FullOf_Bad_Ideas for the suggestion)

reduce the Repetition penalty to 1, the code will be much better, and closely resemble what is generated on the website. (tested multiple times with pong and snake)

Yep pretty good combo! I also use ooba+Silly and for internet query and pdf ingestion I use LolLLMs Great stuff!

I found out that for a simple task like “list 10 words that end with the letters en” i get only wrong answers with the dolphin 34B variant, while 13B tiegihter gets it right, am i doing something wrong with template?

Wow i gotta try it thanks for the hype! Does the GPTQ/AWQ versions differ from GGUF in terms of context? It listed that the context is only 4096

I also use Fooocus, its good for quick final results with one click.

But SillyTavern allows for much more than just image generation, I have a writing assistant that help me reply to costumers at work, a role play character, a general chatbot (that I wish I could connect to the internet for more up do date information), storywriter, and other cool characters that help throughout the day.

It also has an option to share it locally/cloudflare so i can use it on my phone and get images while not in my house in seconds, awesome tool that I couldn’t imagine a year ago :D

Why do you need 70b? for prompting SD?

I found that for good prompts even mistral 7b does the job good!

You dont need 3 GPU’s to run it all, I do it on 3090

I just installed TensorRT which improves the speeds by a big margin (automatic1111)

I generate 1024x1024 30step image in 3.5 secs instead of 9

There is LoLLMs which included stable diffusion inside it (no need to install it seperatly)

but I recommend installing all three of the listed programs, the UI is amazing in Silly totally worth it, and you can use automatic1111 manually and create with more control!

Any reason for Kobold instead of Ooba+SillyTavern?

What 23B model are you running?