Evaluation

T2 — Spatial-temporal multiscale prediction. 3D MERFISH (500-gene panel) · predicts expression + 3D coordinates.

Task

Participants will predict, for an unobserved stage, both the gene expression and the 3D position of every cell. The two are scored as separate questions — tissue geometry carries a full quarter of the score on Task 2 — so matching every expression marginal while placing cells wrongly does not pass.

Each profiled embryo is expensive, so a developmental window is covered by a handful of time points rather than a dense series, and each embryo is imaged in its own arbitrary frame with no registration to any other. Every spatial metric here is therefore invariant to translation and rotation: a submission is never asked to register itself to an atlas.

The task runs on the heart and on the whole embryo, each with its own ladder, and each holds stages out from inside the observed range as well as beyond it. Recovering a stage that sits between two training stages is interpolation; predicting one past the last observation is extrapolation. They are different problems, and a model is asked for both. The heart setting follows heart-tube formation and looping; the embryo setting spans gastrulation and is the harder of the two.

Task 2 from the competition proposal: 3D MERFISH point clouds of whole embryos and hearts at successive stages, with dashed boxes marking the held-out validation and test stages between and beyond them. — A whole stage is held out. A submission carries both channels — what each cell expresses and where it sits — scored as separate questions.

A whole stage is held out. A submission carries both channels — what each cell expresses and where it sits — scored as separate questions.

Splits

Ground truth is withheld while it can still affect the board. Until the final phase a validation submission returns a leaderboard score, not the answers. At the start of the final phase the validation answers are released for every task — ranking has moved to the hidden test split by then, so they can no longer be used to probe it — and the test leaderboard opens. Test ground truth is never distributed.

Train (heart): E8.25 · E8.75 · E9.5; Heart-scope MERFISH. E8.75 ships as Task 3’s wild-type reference, so it is training input here rather than a stage that could be held back.
Validation (heart): E8.5 · E10.5; One interpolation target inside the observed range, one extrapolation target beyond it. Scored through the leaderboard until the final phase, when the answers are released.
Test (heart): E12.5; Extrapolation only. E8.5 is the one stage left inside the training range, so it is the validation target and interpolation is tested in the embryo setting instead. Held until the competition closes.
Train (embryo): E6.75 · E7.25 · E8.0; Whole-embryo scope, spanning gastrulation — the harder of the two settings.
Validation (embryo): E7.5; Sits between E7.25 and E8.0, so the embryo setting is interpolation throughout. Scored through the leaderboard.
Test (embryo): E7.75; Its hidden counterpart, also bracketed by training stages.

Three scales are scored at once, which is what multiscale means here: molecular — does each gene move the way it moves in the real embryo; cellular — is the mix of cell states right, in the right proportions; and tissue — is the thing the right size and shape, with cells beside the neighbours they should have. Each carries a quarter of the score, so getting the marginals right is not enough on its own. The held-out stages also sit both inside the observed range and beyond it, so a model has to interpolate and extrapolate. Full task page →