Karpathy's Recent Talk

Just finished watching the talk by Andrej Karpathy on the state of LLMs, it's worth the watch for sure and he is so good at clearly communicating intricate details. There is a lot of good stuff that he goes over recording the reward model and reinforcement learning that actually make GPTs or Fine-tuned LLM become chatbots or assistants. However, the one thing I really liked was how he discussed the process of a human generating text vs. an LLM. First, he presents an example of someone trying to respond to a question for a blog post, below is the slide with the steps the person might go through.

https://youtu.be/bZQun8Y4L2A?t=1240

You can see it's pretty complex in terms of all the intricate questions we ask ourselves. Additionally, we rely on resources to improve our knowledge in order to answer the question. This means we understand what we do not know but are able to utilize external resources such as talking with people or searching the internet to update our understanding. When it comes to provided quantitative answers we know that it's likely we will get the math operation wrong so we use other tools, however, we have a good sense of intuition about that qualitative answer, for example, the 53x population factor is plausible given 3 of the largest cities in the USA are in CA. What's more important though is that while we write a response to the question that prompted the blog, we actually introspectively reflect as we write which creates a constant feedback loop with on-the-fly updating.

Let's contrast that with what Andrej indicates that LLMs are doing. Remember this is the guy who help founded OpenAI and was an AI lead at Tesla. So I value his insight.

https://youtu.be/bZQun8Y4L2A?t=1360

It's clear from this slide that LLMs don't conduct or possess introspection. What I gathered from his talking over this slide is LLMs are really good at identifying and recalling the context of a prompt and predicting the sequence of tokens/words that would follow. However, they don't really know about their own knowledge or lack thereof, but they are really good at computing the most probably token/text to come next for natural language.

Reuse and Attribution