The reproducibility of study results is regarded as the foundation of science. Of course, representative studies should always provide the same results for the same procedure. But what if the individual research steps are no longer reproducible? In the framework of the research project VisOnFire, the Institute for Creative\Media/Technologies of the St. Poelten University of Applied Sciences is currently working with the Institute of Computer Graphics at the Johannes Kepler University of Linz on a solution to this problem: By means of a forensic visual tool it should be possible in the future for analysts to obtain a detailed description of the individual work steps which led to published study results.
"The primary goal of the project 'VisOnFire' lies in the realization of provenance at all levels, i.e. that the origin of the data is at all times clear and visible. This allows analysts a better understanding of workflows and thus also an insight into how changes in workflows affect the results. This is essential when it comes to reproducing study results", explained Marc Streit, project manager and assistant professor at the Institute of Computer Graphics of the Johannes Kepler University Linz. This should be made possible with a forensic visual tool for analyzing workflow provenance- graphs. For this purpose, innovative visual analytical methods for scalable workflow visualizations, possibilities for comparison of complex data structures as well as metrics for quantifying changes are currently being developed.
The key to reproducibility of data is in the collection of information about the processed data, the applied tools and algorithms as well as the parameters over time. "Until now, it was very complicated to find out what specific changes led to a variation in the results", said Wolfgang Aigner, scientific director of the Institute for Creative\Media/Technologies at the St. Poelten University of Applied Sciences. "Large-scale workflow analyses often take days or even weeks. Incorrect configurations or faulty scripts can lead to the need to repeat the entire analyses; this is very problematic and costs much time and money. The goal of 'VisOnFire' is to make reproducibility as simple and time-saving as possible."
The project VisOnFire (Visual Analysis of Large and Heterogeneous Scientific Workflows for Analytical Provenance) runs from January 2016 to December 2018 and is being carried out by the Fund for the Support of Scientific Research (FWF project P 27975-NBL) and financed by the province of Upper Austria and the Austrian National Foundation. The project consortium consists of the Johannes Kepler University Linz, the St. Poelten University of Applied Sciences as well as international cooperation partners Harvard University and the University of Rostock.