Manuel G. Marciani
First Stage Researcher
BSC
Abstract:
In this talk we will present the current implementation of workflow provenance using the community maintained open standard RO-Crate within Autosubmit, the BSC in-
house developed experiment and workflow manager. This tool is designed from the ground up to conduct climate, weather, and air quality experiments in different platforms types (local, HPC, cloud), and it is the backbone of multiple leading flagship European projects.
Workflow managers play a central role in receiving user input, processing it with local and remote jobs that run on different platforms and that generate output data. RO-Crate enables tracking of workflow prospective (what should happen, e.g. workflow configuration, Slurm job settings) and retrospective (what happened, e.g. log files, performance indicators) provenance. Hence, all the information already present within the workflow manager.
More information: hpckp.org/