Tutorial - Parallel Types

Tutorial describing the use of types for more advanced parallelism in Mesham Tutorial number six - prev :: next

Introduction
Up until this point we have been dealing with the default shared memory model of communication. Whilst this is a simple, safe and consistent model it can have a performance penalty associated with it. In this tutorial we shall look at overriding the default communication, via types, to a more message passing style.

A channel
function void main { var a:Int::channel[1,2]; var b:Int::allocated[single[on[2]]]; proc 1 { a:=23; };	   proc 2 { b:=a; print(itostring(b)+"\n"); }; };
 * 1) include 
 * 2) include

In this example we are using variable a as a channel, between processes 1 and 2. At line 8, process 1 writes the value 23 into this channel and at line 11, process 2 reads that value out of the channel. Note that channels are unidirectional (i.e. process 2 could not write to process 1 in this example.)

Pipes
function void main { var a:Int:: pipe[1,2]; var b:Int; var p;   par p from 0 to 2 { var i;      for i from 0 to 9 { var master:=i%2==0?1:2; var slave:=i%2==0?2:1; if (p==master) a:=i; if (p==slave) { b:=a; print(itostring(p)+": "+itostring(b)+"\n"); };      };    }; };
 * 1) include 
 * 2) include

This code demonstrates using the pipe type for bidirectional point to point communication. If you change the pipe to a channel then you will see that instead, only process 1 may send and only 2 may receive.

Extra parallel control
By default the channel type is a blocking call, we have a number of fine grained types which you can use to modify this behaviour.

function void main { var a:Int::channel[0,1]::nonblocking[]; var b:Int; proc 0 { a:=23; sync a; };   proc 1 { b:=a; sync a;      print(itostring(b)+"\n"); }; };
 * 1) include 
 * 2) include

In this code we are using the nonblocking type to override the default blocking behaviour of a channel. The type is connected to the sync keyword such that it will wait at that point for outstanding communication to complete. Try experimenting with the code to understand the differences these types make.

Collective communication
Mesham has a number of collective communication types, here we are just going to consider the reduce and broadcast here.

A broadcast
The broadcast type allows us to explicitly specify that a communication is to involve all processes (in current parallel scope.)

function void main { var a:Int; a::broadcast[2]:=23; print(itostring(a)+"\n"); };
 * 1) include 
 * 2) include

In this example we are declaring a to be a normal Int variable, then on line 6 we are coercing the broadcast type with the existing type chain of a just for that assignment and telling the type that process 2 is the root process. The root process is the one that drives the broadcast itself, i.e. here process 2 is sending the value 23 to all other processes. Then on line 7 we are just using a as a normal program variable to display its value. This use of types is actually quite a powerful one; we can append extra types for a specific expression and then after that expression has completed the behaviour is back to what it was before.

A reduction
Another very common parallel operation is to combine values from a number of processes and, applying some operation, reduce this to a resulting value.

function void main { var p;   par p from 0 to 19 { var a:Int; a::reduce[0,"sum"]:=p; if (p==0) print(itostring(a)+"\n"); }; };
 * 1) include 
 * 2) include

This code will combine all of the values of each process's p onto process 0 and sum them all up. Multiple operations are supported and are listed in the reduce type documentation

Eager one sided communication
Whilst normal one sided communications follow the Logic Of Global Synchrony (LOGS) model of shared memory communication and complete only when a synchronisation is issued, it is possible to override this default behaviour to complete communications at the point of issuing the assignment or access instead.

function void main { var i:Int::eageronesided::allocated[single[on[1]]]; proc 0 { i:=23; }; sync; proc 1 { print(itostring(i)+"\n"); }; };
 * 1) include 
 * 2) include

Compile and run this fragment, see that the value 23 has been set without any explicit synchronisation on variable i. Now remove the eager bit of the eager one sided type (or remove it altogether, remember onesided is the default communication) and see that, without a synchronisation the value is 0. You can add the sync keyword in after line 6 to complete the normal one sided call. We require a synchronisation between the proc calls here to ensure that process 1 does not complete before 0 which sets the value.