- 4 Posts
- 11 Comments
Joined 1 year ago
Cake day: October 31st, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
1·1 year agoYes I understand all that
Auto regressive is like arima In time series forecasting
Then rnn came along
Then sequence to sequence
They all have the last prediction is used as input for the next prediction in common
Hence auto regressive
I had to read that a few times.
Auto-Regressive is like forecasting, it’s iterative.
LLM reliability is this vague concept of trying to get to the right answer.
Hence tree of thoughts as a way to ‘plan’ to that vague concept of the right answer.
Circumvents the univariate next token prediction limitation with parallel planning.
sounds like free will vs determinism.
The real question is, are minds deterministic.
btw
instead of starting from a premise and trying to justify it.
Evaluate the facts before forming your thesis.
Because this sounds like a religious argument.
‘Help me defend my belief.’
You shouldn’t be asking for facts to support your conclusion, because you shouldn’t have a conclusion without facts.
sounds like free will vs determinism.
The real question is, are minds deterministic.
How would a non compete work in this agreement
I was going to try to knowledge distill but they modified their tokenizer.
Either way neo has a 125M model, so a 248M model is x2 that. I imagine this could be useful for shorter context tasks. Idk, or to continue training for very tight uses cases
I came across it while looking for tiny mistral config jsons to replicate⁸
I imagine creating an app, putting it on everyone’s cell phone, and using a fraction of the power, you can build an llm easily that would surpass any single data center.
rm is the reward model… not the same as the lm model. I tried the lm, wasn’t impressed. Gpt-3.5 did better for summarizing quotes. It was good, but I honestly think open hermes and or synthia 1.3b do better