I loved Andrej’s talk about in his “Busy person’s intro to Large Language Models” video, so I decided to create a reading list to dive in deeper to a lot of the topics. I feel like he did a great job of describing the state of the art for anyone from an ML Researcher to any engineer who is interested in learning more.

The full talk can be found here: https://youtu.be/zjkBMFhNj_g?si=fPvPyOVmV-FCTFEx

Here’s the reading list: https://blog.oxen.ai/reading-list-for-andrej-karpathys-intro-to-large-language-models-video/

Let me know if you have any other papers you would add!

  • coumineolBEnglish
    1·
    1 year ago

    Thanks but here’s the problem with this list: most of the papers mentioned are on a very high technical level, and people who would be able to understand them are probably people who have already read them. Note that Andrej was careful to keep the material at a certain level because he addresses those who want to go one step further than talking to ChatGPT, without necessarily understanding all the underlying theory.

    • FallMindless3563OPBEnglish
      1·
      1 year ago

      The only book he explicitly mentions is “Thinking Fast and Slow” by Daniel Kahneman, but I think there are a ton of books that would be great resources along side the papers. I just happened to pull a lot of the papers from the footnotes and concepts he mentioned.