Open-R1: a Completely Open Reproduction Of DeepSeek-R1

Hey there! This post is an intro to the task, not a claim that we’ve replicated R1 yet. We’re integrating in the open, so as soon as we have evaluation numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.

True, but it appears like there’s absolutely nothing to be evaluated as of today. I assume the ultimate objective is to train a brand-new reasoning design and after that use the same assessment metrics as o1 and the DeepSeek-R1.

Well, there must be at least some sanity check and validation to guarantee the model was trained correctly.

Oh yes, if you are discussing the assessment variety of deepseek’s model it’s coming soon!

As pointed out in the post there is no design called Open-R1 to evaluate at all … not yet anyway. This is a blog site describing that Hugging face will take the R1 Deepseek model, work out how it was built as outlined in the paper and from what they launched, and then reproduce that process.

in truth this is practically how science works … A creates a plan, discovery or development and it is evaluated by B, C and D to see if it is reproduceable. Thats been the cornerstone of research now for a couple of centuries.

This blog is not stating they have actually already done so … Its a blog site outlining an intent to start training a design like R1 and calling it Open-R1.

Also DeepSeek-R1 was just launched last week, and even in their paper they described the calculate hours needed. While those are low calculate hours for a SOTA design this does not indicate you can train stated model in a week. I ‘d personally like to be able to train a transformer design in a week, but we may need to wait a while for that level of calculate innovation.

So there are no benchmarks for a design that has not been developed yet right? As described in the blog site, and again in reply to your concern.

However fear not, there is a GitHub Repo currently and factors (hell I might join myself), some prelim work done, and a strategy of attack. A great starting position.

n
@edbeeching
has examined the launched designs currently

( src: https://x.com/edwardbeeching/status/1884273209136275742)

R1 just trained on o1 outputs, so collectively …/ s. This is what the new AI czars are stating

Hi! This article is an intro to the project, not a claim that we have actually recreated R1 yet. We will totally share the missing piece when we have them, you can expect the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

That’s great and important to comprehend this remarkable buzz that lacks technical comprehension and explanation. Science has to do with reproduction, and if they claim to be open, let them fullfill the open part.

Please do publish the training expense.

We will!

Excalidraw Hi n
@bojan2501
thanks, we will undoubtedly be striving to ensure this training dish can work for small language models on customer hardware considering that not everyone has a cluster of H100s in your home:-RRB- The tool we used for the images was Excalidraw! https://excalidraw.com

anticipating it! WTF are your speaking about?

should be a joke

It’s actually cool to see how the entire open source neighborhood comes together!

Ops …

5.5 M is number reporter in the deepseekv3 tech report (just the training, not the experiment afaik), for R1 tough to estimate tbh however much less than 5.5 M imo

Historically, they have actually never launched code or datasets of their LLM training, so I wouldn’t anticipate this time to be different. If they would release it that would be remarkable naturally!

Yes of course!

So generally you’re asking to change existing censorship with another flavour of censorship?

The code for the designs are inside the model repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py

Hello Team, I’m Ray Bernard, the author and creator of EQUATOR. My research study team will be dealing with a paper focused on reproducing specific parts of DeepSeek R1. Our objective is to recreate the cold start and supply your group with a dataset that consists of COT and other strategies to support these efforts. We like to contribute our work to help. Please let me understand if you discover this beneficial. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/

Where is the examination numbers? without it you can’t call it recreation.

8 replies

True, however it appears like there’s nothing to be assessed since right now. I assume the is to train a brand-new reasoning model and after that utilize the same examination metrics as o1 and the DeepSeek-R1.

That’s rather intriguing, I was asking myself why the concerns the author exposed here are not being asked by others? I believe the work they have done is remarkable but at the exact same time I question why they would not put these missing out on pieces on if they are expected to be fully open.
Why even without reproduction and understanding of the development they could impact so much the marketplace in this method?

4 replies

Hi! This article is an introduction to the job, not a claim that we’ve reproduced R1 yet. We will completely share the missing out on piece when we have them, you can anticipate the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

Interesting read, and it is excellent that we see more effort into this direction: more optimization and less brute force.
Also question what tool did the author use for producing step diagram.

2 replies

Excalidraw I’m so happy that effort like this currently exist, I’m gon na try to contribute:-RRB- 1 reply

eagerly anticipating it! So racist articel

2 replies

WTF are your talking about?

Awesome to have this open reproduction began!

For Step # 1 check out https://github.com/open-thoughts/open-thoughts!

https://x.com/ryanmart3n/status/1884284101265612856

Let’s do this thing!

1 reply

It’s truly cool to see how the entire open source community comes together!

Does anyone understand the actual training cost of r1? I can’t discover it in the paper or the announcement post. Is the 6M expense reported by media simply the number drawn from v3’s training cost?

2 replies

Ops …

Has anybody asked the DeepSeek group to release their training data and code, or a minimum of share them privately with an independent replication job like this? Have they declined such a demand?

A faithful duplication depends upon utilizing the exact same dataset and hyperparameters. Otherwise, any major discrepancies with the released benchmarks would be hard to pin down-whether due to training data differences or the duplication approach itself.

1 reply

Historically, they have actually never released code or datasets of their LLM training, so I wouldn’t anticipate this time to be various. If they would launch it that would be incredible naturally!

In the meantime we need to make finest guess estimates and see if we can get there ourselves.

You provide excellent duplication process of Deepseek thinking training. I will attempt something comparable to it.

This is actually great info, can we tweak with specific usage case when code is launched?

1 reply

Yes of course!

Please think about eliminating prejudiced, polluted or unaligned training information and make an effort to remove copyrighted works from the crawl from consumption. This will make the design more usable. If you recycled anthropic curation checks, this might also help, remove obviouslybiased information will likely include a great deal of value. We don’t desire another polluted, unaligned open source design, right? And no business would ever use deepseek or a model that recycles it, right?
We value your work for the benefit of humanity, we hope.
Miike C from NJ

1 reply

So basically you’re asking to replace existing censorship with another flavour of censorship?

Can’t wait! Hopefully the design will be uncensored but whatever you can do is alright! Love seeing open source structure itself up. I’m not wise adequate to really help however I can contribute support lol

Hello guys, I am even simply searching for code for DeepSeek-V2, in order to totally comprehend multi-head hidden attention. You do not seem to have code in Hugging Face even for that. Or am I missing out on something? Don’t see anything in src/transformers/models. MLA is not appropriately explained in their paper, so it would be essential to have code for this.