Common questions

Anyone — academic, industry, independent — who can submit a prediction file conforming to the required format. Each person joins one team; each team works on one track.

Yes. The three tasks share a data resource and a metric panel but are scored separately, so a team can enter one, two, or all three.

Yes — coding agents, recursive LLM systems, evolutionary search. The framework is not the point; the absence of a human in the loop is. A person may write the starting prompt, but nobody may read intermediate results and steer the agent on the strength of them. Prizes require the trajectory, the prompts and the harness code. At least one of them has to be attached for an entry to enter the scoring queue at all, and anything we cannot fully verify is not eligible for a prize even once it has scored.

They are scored on the same tasks, the same metrics and the same hidden test sets, and their leaderboards sit side by side — but prizes are awarded per track, so a human-designed method never competes with an agent-designed one for the same prize.

A predicted set of cells for the target condition: expression for Task 1, expression plus 3D coordinates for Tasks 2 and 3. You never submit cell-type labels — the organisers assign types with a frozen classifier applied identically to every submission, so hidden labels are never exposed.

No. Nothing in the metric panel requires matching cell counts, and where a comparison would otherwise be confounded by count — the shape group — both point clouds are subsampled to a shared size first. Cell number is scored separately and explicitly, as count_log_ratio.

No. Every spatial metric is agnostic to translation, rotation and — outside one acknowledged blind spot — reflection. That blind spot is laterality: a mirrored embryo currently scores identically to a correct one.

Baseline implementations for each task, data-loading utilities, the exact evaluation code, example submission files, a standalone scorer for any .h5ad, documentation, and executed notebook tutorials.

Training data is released for method development. Validation and test ground truth are withheld while they can still affect the board — until the final phase a validation submission returns a leaderboard score, not the answers. At the start of the final phase the validation answers are released for every task, since ranking has moved to the hidden test split by then, and the test leaderboard opens. Test ground truth is never distributed.

Competition datasets are provided for research and educational purposes. Any use, reuse, publication, presentation, or redistribution of the data outside the Challenge must appropriately acknowledge the Virtual Embryo Challenge and credit Dr. Neil Chi’s group at the University of California, San Diego (UCSD), which generated the dataset. As the dataset has not yet been published, any use of the data in a publication or public presentation must receive prior approval from Dr. Neil Chi and the data-generating team. Any publication using the dataset must also appropriately credit Dr. Qingquan Zhang and Dr. Neil Chi.

You may train on external data, public or your own, and for Task 1 this is expressly encouraged. What you may not use is measured data from a held-out stage or genotype, by any route — not directly, not through a model pre-trained on it, and not through a public dataset that contains it. Those are: E10.5 and E12.5 for Task 1; E7.5 and E7.75 in the embryo setting and E8.5, E10.5 and E12.5 in the heart setting for Task 2; and the Gata4 and β-catenin knockouts at E8.75 for Task 3. Nearness counts: external data is treated as a held-out stage when it sits closer to that stage than to the nearest stage we released for the same task, the midpoint being the boundary — holding out E9.5 and training on an external E9.45 embryo is using the answer under a different name, and so is a dataset labelled by somite count or Theiler stage that lands in the same window. Beyond the last released stage, as the extrapolation targets are, everything from the midpoint onwards counts. For the extrapolation targets the window is stated outright rather than derived: no external data from any time point between E9.5 and E13.5 may be used, and E10.5 and E12.5 — the validation and test targets themselves — are excluded absolutely; time points outside E9.5–E13.5 may be used provided the source and its stages are stated explicitly with the submission. For interpolation targets, external data too close to a held-out stage is also treated as held out: in Task 2 that means heart data from E8.25 to E8.75 and embryo data from E7.5 to E7.75 may not be used — the midpoint-derived protected windows around the held-out interpolation stages. The same applies to genotype: another allele of the same gene at a comparable stage, or a perturbation that phenocopies the held-out one, is the held-out condition. A general-purpose resource that spans one of those windows is not banned outright — remove the cells inside it before training and say so in the method summary. Using it with them left in is using the held-out data. Every external source must be disclosed with the submission; an undisclosed one is a violation whether or not it changed the result, because the disclosure is what makes this checkable.

Hidden splits are defined by developmental stage, perturbation condition, or combinations thereof — distinct from public splits. Models must learn generalisable developmental dynamics, not memorise.

Two anchors published with every score. The floor is copy_last — predict the preceding stage verbatim (wt_identity for Task 3). The ceiling is half the target scored against its other, disjoint half — deliberately not the target resubmitted verbatim, which is a number no honest generative model can reach.

A metric your output cannot produce, or that comes back NaN, counts as zero skill within its group. It is never treated as an excused absence.

Every attack found during the adversarial review is kept as a control row that is re-scored on every run. If a future change reopens one of those holes, the control row scores above its floor and the scoreboard flags it — the regression shows up in the table instead of needing another audit to find.

All ground-truth labels are hidden. Submission counts are limited in the final phase to reduce leaderboard probing. Duplicate registrations, unauthorised data use, code sharing across teams, or falsified results are grounds for disqualification.

Slack is the fastest, and the answer is visible to everyone else who has the same question — join the #virtual-embryo-challenge channel in the Pantheon workspace. Discord is the same conversation for people who prefer it. Email the organisers for anything that should not be public: eligibility, an entry you want withdrawn, a suspected scoring error, or anything involving your team’s data.

Email the organisers with your team name and the submission id — the id is on the submission’s own page, and it is what lets us find the exact file and the exact scoring run. Rescoring is possible while the phase is open; after a phase closes the board is fixed, so a problem is worth reporting the day you see it rather than at the end.

Both are on the site and both are accepted separately at registration: the Rules govern the competition — eligibility, teams, submission limits, prizes, what disqualifies an entry — and the Terms govern use of the site and the data, including the acknowledgement and prior-approval conditions on the datasets. The version you accepted is recorded against your registration, so a later revision does not silently change what you agreed to.

Yes — open an issue on the veckit repository. Problems with the metric implementations or the scorer belong there rather than in email, because the fix is a commit other participants need to see.

Announcements go to Slack and Discord first, and to the address you registered with if you opted into news at registration. The timeline page is the authoritative schedule and is updated when a date moves.