Archive | News RSS for this section
December 1, 2014  

Simulation Overload

5460d0cab0c78227780074a9_1

We had a great launch last week and Higgs Hunters was off to a flying start. After a day or two though, we realised that something wasn’t right. Over the weekend, you all began seeing way too many simulations. That should start to fix itself now. Here’s what happened, and how we want to move forward:

On many Zooniverse projects, there are simulated data mixed in with the real data. We do this to help calibrate the project as a whole, and, in the case of Higgs Hunters, to see what sort of events can easily be seen in this data, and what can’t. It became apparent over the weekend that the balance of real and simulated data on Higgs Hunters was wrong. Several weeks ago when the ATLAS team delivered the Higgs Hunters data to the Zooniverse, a piece of metadata was missing and the developers at Zooniverse thought that a whole batch of simulated data was actual data. The entire dataset was uploaded and the project launched as planned.

We realised a day or two after launch that we weren’t telling volunteers when they had seen a simulation. This is important to know — especially if you think you found something really cool — and so we moved quickly to get the sims flagged as such after each classification. However, as many of you noticed, this fixed made it apparent that, in fact, 8/10 images were sims! Fixing the one mistake brought the other to the surface — and after a lot of searching we figured out what had happened.

Today we have done the following:

  1. We have paused 90% of the simulated data — so you won’t be asked to classify it now.
  2. We are aggressively retiring the simulations — so they’ll be removed 4x faster than the real data.
  3. ATLAS are preparing more data for Higgs Hunters — 10 times as much!
  4. Hopefully we can upload the new data before the end of 2014, and we’ll email participants to let them know.

After all of this, we now have ~8k subjects in Higgs Hunters and ~30% are sims — that means we’ll be done in ~100k classifications. So let’s call this Round One and see where it takes us. In the meantime, we thank everyone for their efforts on Higgs Hunters, and we apologise for all the confusion and frustration that this has caused some users.

We’re really sorry, and hope that this helps to explain some of the problems that have occurred and the ways in which we’re moving forward.

Advertisement