Welcome to Taskblaster’s documentation!

Taskblaster is a Python tool for defining and running computational workflows.

A workflow is a specification for generating computational tasks. A task is a small unit of computational work with an input and an output. In Taskblaster it is a Python function along with an input specification. Workflows may specify tasks and subworkflows statically or via dynamic flow control structures. As tasks are generated, they are stored forming a directed acyclic graph and can be executed by worker processes.

Taskblaster makes it easy to parametrize a workflow on many inputs, and to execute the tasks whether on a supercomputer or on a laptop. Taskblaster also makes it easy to extend and modify a workflow along the way while keeping track of whether computations are up to date.

Some main design features of Taskblaster are:

  • Workflows and tasks are composed into a main workflow using a Python interface.

  • Running the main workflow generates tasks, which for conceptual and practical simplicity are organised into a directory structure that can be navigated with standard tools.

  • A command-line interface allows selecting which workflows and tasks to run and when. It also provides a range of reporting and inspection tools.

  • Taskblaster is a light-weight tool and does not require a database server or a persistent connection to a supercomputer. On a supercomputer, Taskblaster is best used together with myqueue and workload manager such as Slurm or Torque.

  • Taskblaster is free software released under GPLv3+.

Links:

Contents:

Indices and tables