.. _explanation_dynamical:

===================
Dynamical workflows
===================

Here we provide the language in which TaskBlaster thinks about dynamical workflows. For a hands on introduction see the  :ref:`tutorial_dynamical`.

Static workflows 
----------------

As static workflows are a building block of dynamical workflows, let's first
discuss the static workflows introduced in the previous section using a more general language.
For a static workflow, the tasks will be
generated immediately and the dependency graph will be a directed acyclic graph (DAG). When 
running the static workflow with multiple workers one obtains parallelism where available. 
However, besides slight parallelization advantage, for most parts, a static
workflow building block is to be considered as a sequential non-branching list
of instructions of what to calculate.

To give an example of the picture painted above, 
consider following computer program

::
   
    a = A()
    b = B(param_A=a)
    c = C(param_A=a)

A directed acyclic graph (DAG) can be made on the dependencies of these lines,
and we end up with a following relationship between the tasks.

.. tbinit:: basic_example
.. tbhiddenshellcommand:: tb init
.. tbfile:: basic_workflow.py
.. tbfile:: wfstyle.json
.. tbhiddenshellcommand:: tb view-workflow -s wfstyle.json -f basic_workflow.py -o basic_workflow.html StaticWorkflow
.. tbviewworkflow:: basic_workflow.html

.. raw:: html

   <div style="width:100%; background: #eee; clear: both; overflow: hidden">
   <iFrame scrolling="no" frameborder="0" width="100%" height="500px" src="../../_static/basic_workflow.html" title="Basic workflow"></iFrame>
   </div>

Note, that after completion of ``taskA``, one can parallely execute
``taskB`` and ``taskC``, so, to that end, TaskBlaster will supersede linear
execution of "workflow"-code. Even so, in the end, everything will appear as
it would have been executed linearly, since there is no branching, and no
other dynamics. Before we take a deeper look on how to make our
workflows dynamic, let's learn about a few more terms.

Branching
---------
In TaskBlaster, branching allows workflows to dynamically decide the next steps 
based on the results of previous tasks. This is essential for creating flexible 
and adaptive workflows that can handle various scenarios.


To define a branch in TaskBlaster, use the `@tb.branch` decorator. This decorator 
assigns a task to a specific branch. If the decorator is omitted, the task will 
be assigned to the default branch, `entry`.

.. note::

   ``@tb.branch`` (and all other decorator) must always be above the ``@tb.task`` decorator.

For example, one can define a workflow with two branches, `even` and `odd`,

::

   @tb.branch('even')
   @tb.task
   def taskA(self):
      ...

   @tb.branch('odd')
   @tb.task
   def taskB(self):
      ...

Conditional branching
---------------------
In some cases, the branching decision is based on the results of a task.
To define a conditional branch, use the `@tb._if` decorator with a condition
function. The condition function should return a boolean value based on the
results of the task.

::

   @tb._if(true='target', false='other_branch')
   @tb.task
   def taskC(self):
      ...


In this example, if `taskC` evaluates to True, the workflow will follow the 
target branch. If it evaluates to False, it will follow the other_branch.

Jumping to a branch
-------------------
You can also use the `@tb.jump` decorator to explicitly jump to a different 
branch.

::

   @tb.jump('target')
   @tb.task
   def taskD(self):
       ...
      
In this example, the workflow will jump to the branch ``target`` immediately
(even before it has executed taskD).

.. note::

   ``jump`` statements are often useful, after diverging the control from with
   if statement, one can converge from the diverged controlflow by jumping to a
   common branch.

Using `Phi` for Dynamic Input Parameter Selection
----------------------------------------------------

The `Phi` operator is used inside the ``tb.node`` definition to select the correct variable
to use, depending from which branch we have jumped from to the current branch.
In other words, consider the following example. We enter the branch ``C`` and
hence create the task ``mytask``. Depending whether we came to branch ``C``
from branch ``A`` or branch ``B`` we will create the task ``mytask``
with ``input_parameter`` set to either to ``self.valueA`` or ``self.valueB``
respectively.

::

    @tb.branch('C')
    @tb.task
    def mytask(self):
        return tb.node('mytask', input_parameter=self.Phi(A=self.valueA, B=self.valueB))

It provides a mechanism to reference data before it
exists and resolve these inputs once the task has been completed.

In the example below, the `gather_result` task will execute with the input argument 
number given by the output from the `even_task` if the ``even`` branch was executed
(more explicitly, if we came from ``even`` branch), 
or by the output from the `odd_task` if the odd branch was executed.

Note that this example is missing the control flow of how did we get to even or odd branch,
but that doesn't matter anymore in evaluating the Phi operator at the ``final`` branch.

::

   @tb.branch('even')
   @tb.jump('final')
   @tb.task
   def even_task(self):
         ...
   
   @tb.branch('odd')
   @tb.jump('final')
   @tb.task
   def odd_task(self):
         ...
  
   @tb.branch('final') 
   def gather_result(self, number):
       return tb.node('gather_result', number=self.Phi(even=self.even_task, odd=self.odd_task))


Dynamical workflows
-------------------

"Dynamical workflows" is a general term which is outside of the scenario
described above: all tasks can not be generated simultaneously, because
the dependency graph of tasks will depend on dynamical decisions along the way.
Put more simply, the next step in a workflow depends upon the results of a
task(s) that has yet to be executed.

When depicting workflows, one typically draws only one kind of dependency.
However, at this stage, there will be a variety of relationships.


Consider following piece of code:

::

   a = A()
   if B(a):
       c = C(a)
   else:
       d = D(a)


In Taskblaster's workflow syntax, you would represent this code in the as:

.. tbinit:: branch_example
.. tbhiddenshellcommand:: tb init
.. tbfile:: workflow.py
.. tbfile:: branch_style.json
.. tbhiddenshellcommand:: tb view-workflow -s branch_style.json -f workflow.py -o branch_workflow.html DynamicWorkflow
.. tbviewworkflow:: branch_workflow.html

.. raw:: html

   <div style="width:100%; background: #eee; clear: both; overflow: hidden">
   <iFrame scrolling="no" frameborder="0" width="100%" height="800px" src="../../_static/branch_workflow.html" title="Dynamic workflow"></iFrame>
   </div>


Dynamical workflows have the familiar parametric dependence represented by
the dashed arrows, but also a new kind of *shadowed* arrow, which represents
the control flow dependence. In this case, a red arrow means the if statement
evaluated to False, and the data flowed from A to D controlled by the
results of the else branch. A green arrow means that the if statement evaluated
to True.

Note that using single type of arrow (often used in workflow articles) is not
sufficient, because control flow might not follow parameter flow. For example, 
``taskB`` evaluates to either True or False, but this boolean result is not needed. 
Instead, the if and else branches use the result of ``taskA``. This detailed 
representation is often avoided in articles due to its complexity, even for 
simple workflows. However, TaskBlaster uses this approach to accurately 
represent workflows.



``taskA`` generates some information which is used by ``taskB``. ``taskB`` uses 
this information to decide whether the data from ``taskA`` should go to the C or D
branch (Control flow dependence). However, ``taskC`` in the C-branch takes an output 
from ``taskA``, as is presented by the blue arrow (Parametric dependence). This 
reflects the use of a variable in the pseudo code above.

In the tutorial :ref:`tutorial_dynamical` you will see how this workflow is represented in TaskBlaster.