Pre- and Post-publication Verification for Reproducible Data Mining in Macromolecular Crystallography

Raw diffraction data are our ground truth from which all subsequent workflows develop

Defining best practice in science is challenging. International consensus is facilitated by the International Science Council via its members such as the International Union of Crystallography (IUCr). The crystallographic community has many decades of tradition linking articles with the underpinning data, and is admired across all sciences accordingly. Crystallography has always been at the forefront of harnessing new technology in the service of consensus. Technology has provided new vast data-archiving opportunities, allowing the preservation of raw diffraction data, along with article and database depositions of a model's coordinates and associated structure factors. The raw diffraction data, which can now be preserved, are the ground truth from which all subsequent workflows develop. Journal editorial boards provide a practical forum for setting the criteria to decide if a study's files are truly the version of record. Within that, reality involves a variance of reasonable workflows. But what is a reasonable variance? Workflows must be detailed carefully by authors in explaining what they have done. There is a great, and increasing, diversity of macromolecular crystallography analyses, and yet an increased constraint on how much can be written in an article about the workflow used. Raw data provide the ultimate reproducibility evidence. A part of reproducibility and replicability is using an agreed vocabulary; the meaning of words such as precision and accuracy and, more recently, the confidence of a protein structure prediction should feature in approaching `truth'.