DeepSeek-R1 Beats OpenAI's o1, Revealing All Its Training Secrets Out In The Open

A deep dive into how DeepSeek-R1 was trained from scratch and how this open-source research will accelerate AI progress like never before.

It’s incredible to see how far AI has progressed in the last decade.

Most of this progress came after Google released their groundbreaking paper called “Attention Is All You Need” in 2017.

It is then that other companies worked on the ideas discussed in this paper (the Transformer architecture) to build powerful LLMs.

One of these companies, OpenAI, before the release of transformers, was heavily focused on reinforcement learning (RL) and moved its trajectory towards LLMs, benefiting exponentially from Google’s open-sourced research.

Although OpenAI started as an organisation to democratise AI, it ended up making its research and products proprietary.

The last open-source model released by OpenAI was GPT-2, which was made publicly available in November 2019.

Since then, all their model advancements have been kept a secret.

With OpenAI releasing ‘o1’ a few months ago, they have really discovered something phenomenal and trained their newer LLMs to spend more time thinking (with long Chain-of-…