Search results
1 day ago · OpenAI also lost Daniel Kokotajlo, who worked on OpenAI's governance team, earlier this month, and in February co-founder Andrej Karpathy quit as well. While Sutskever and Karpathy have declined to delve too deeply into the reasoning behind their departures, Leike and Kokotajlo have made it clear that they resigned over disagreements on AI safety.
1 day ago · The decision by Andrej Karpathy to open-source the 700-lines llama.c inference interface demystified how developers can interact with LLMs. The public repository took off with thousands of stars, fork
1 day ago · Putting a lot of these concepts together, Andrej Karpathy’s at Tesla from a recent tweet speaks volumes: This is because good evals are very difficult to build - at Tesla I probably spent 1/3 of my time on data, 1/3 on evals, and 1/3 on everything else.
The parameter count and the dataset token count matches the original 124M GPT-2. With llm.c, which is quite efficient at up to ~60% model flops utilization, reproducing this model on one 8X A100 80GB SXM node takes ~90 minutes. For example, on Lambda this node goes for ~$14/hr, so the total cost of reproducing this model today is about $20.
3 days ago · “Who’s getting how many H100s and when is top gossip of the valley rn,” OpenAI’s Andrej Karpathy posted last week.
5 days ago · At the 2021 Conference on Computer Vision and Pattern Recognition on Monday, Tesla’s head of AI, Andrej Karpathy, revealed the company’s new supercomputer that allows the automaker to ditch radar and lidar sensors on self-driving cars in …
2 days ago · GPT-2 124M was the smallest model in the GPT-2 series released by OpenAI back in 2019. Andrej Karpathy's llm.c is an evolving 4,000 line C/CUDA implementation which can now train a GPT-2 model from scratch in 90 minutes against a 8X A100 80GB GPU server. This post walks through exactly how to run the training, using 10 billion tokens of FineWeb.
