Community-wide Inter-observer Variability in Testate Amoeba Morphological Identification

How reproducible is species-level identification from microscopy images?

Morphological identification of testate amoebae from microscopy images remains a cornerstone of palaeoecological and ecological research, yet the reproducibility of this process across observers has never been systematically assessed. This project brings together more than 52 researchers and students from across the world, each independently identifying 1,000 microscopy images spanning 10 samples, and assigning a confidence score to every identification.
The resulting dataset — over 50k individual identifications — is being used to quantify inter-observer variability, assess the effect of observer experience on identification accuracy, and evaluate the consistency of confidence scoring across the community.

A custom Python pipeline handles data validation, taxonomic standardisation, and the harmonisation of 707 unique taxonomic entries into a curated reference taxonomy. All code is fully reproducible and will be openly available on GitHub.
Statistical analyses are now underway in R. Results will provide the first community-wide benchmark for morphological identification reliability in testate amoeba research, with direct implications for the interpretation of ecological and palaeoecological datasets.

A collaboration with the laboratory of Soil Biodiversity of the University of Neuchâtel. Manuscript in preparation.