Hello and welcome to the fourth video of our GROOPS tutorial video series. This time we want
to focus on loops and conditions and I'm going to show you how you can use them to make your script
even more powerful and even more flexible. So we are using our script from the previous video,
where we visualized differences between kinematic and reduced-dynamic orbits for certain satellites.
If we go back to our data, you can see that we have data for three satellites: GRACE-1,
GRACE-2 and TerraSAR-X. And we have data for one week for each satellite. Of course
we could run our script for each satellite and each day,
but that's cumbersome, so it would be great if we could run our script once and it would plot
the differences for each day and each satellite. Thankfully, we can do this using
loop feature in GROOPS. First of all, we want to loop over our dates, so we'll add a new program
called "LoopPrograms", and here we can define a loop. There are different
options: You can use times, you can loop from a file, from a matrix and so on. In our case,
we will use "timeIntervals" and we will define our time period so it starts on 2015-01-01 and ends on
2015-01-08. Here, since we're using "timeIntervals",
the end point of the interval has to be one past the last day because intervals are always defined
between two time points. And for sampling we use "1", so we want a daily sampling. For convenience,
we will set both "timeStart" and "timeEnd" as global variables by right-clicking them,
so we can change them in the global section. We don't have to go to this location to change them.
What we do not need anymore is the "time" variable we used in the previous video
because this loop creates a new variable called "loopTime" – we can use a different name but
let's stick with "loopTime" – and in each iteration this variable contains the current
timestamp. For example, in the first iteration it will be the 2015-01-01, then 2015-01-02 and so on.
What we can also set here is if we want to continue after an error. If any of the programs
within this loop cause an exception, we can say, okay, just skip to the next iteration and continue
going on. So we'll set that to "yes" just to be sure. So this is our outer loop over the
days. We also want to add an inner loop over the satellites. In this case,
we're going to use a "manualList" so we can define a list of strings. This will be "GRACE-1",
"GRACE-2" and
"TerraSAR-X". And in this case, we will call our variable "satellite". We already have a
global variable "satellite", which we used in the previous video. We don't need that anymore because
now our loop will define the variable "satellite" and in the first iteration it would be "GRACE-1",
then "GRACE-2" and then "TerraSAR-X". Again, we'll set "continueAfterError" to "yes" because
if there is no data on one day, for example, we still want to visualize the other days. Okay,
so now we have our two loops: We have the outer loop over the days and the inner loop
over our satellites. For convenience's sake, we can set the whole loop – the inner loop – global
and let's call it "loopSatellite". Now in our global section I can easily change
the list of satellites. I can add some or disable some, so it's just for convenience.
Okay and now we have to move our programs into the loop. We can just copy or move them one by one.
Actually, for "FileCreateDirectories": We would keep creating the plot directory outside of the
loop because we don't really want to create this directory once, but the "tmp" directory we will
create in each loop. More on that in a second. Let's continue moving our programs.
Okay, so now I've moved all the programs we had on the outside before into the inner loop.
So all of these programs will be run for each satellite on each day.
As I said – or what we should now consider is that we we might use different "tmp" directories
for each loop. Just to be safe, so that we do not use any previous files from previous
iterations accidentally. So what we can do is we can say we want to replace –
it's Ctrl+F to go to Find & Replace – we want to replace all occurrences of "tmp" with
a global variable "{tmpDir}". Let's replace all. With that button you can
close all programs again. If we now add a global variable "tmpDir"
of type "filename" we can define a temporary directory we want to use for each iteration. Let's
use "tmp/{satellite}_{loopTime:%D}", so for each – if we now go to "FileCreateDirectories" – it will
create a temporary directory in the subdirectory "tmp" with the name of the satellite, underscore,
and the date parsed as an ISO date string. Which is convenient because then each day and satellite
will have their own temporary directory and there can be no confusion with the data.
Okay, so far so good. One thing we still have to change is – if you remember, we
used our variable "time" beforehand and, here for example, and now, of course, we don't want to use
"time" but instead our loop variable "loopTime". So we're going to replace that
and now we should be able to run this script.
As you can see it runs. If you go to our directory and go into the "plots" directory,
you can see, okay, there's a GRACE-1 plot, GRACE-2 plot, now a TerraSAR-X
plot and then for the second date a GRACE-1 plot and so on. The program will continue running.
It will take some time, of course, because we are creating quite a few plots.
Let's delete them for now because now I can show you another feature, which is we can parallelize
our script, so we can run our script in parallel. So far we ran it with a single process but if we
have tasks that can be run on multiple cores in parallel then this can be quite useful.
In this case, it's pretty easy. Let's just say we want to parallelize our time loop, so
we want to plot multiple dates at the same time. This is very simple because we can, here, adjust
in the outer loop the "processCountPerIteration" from "0" to "1". "0" means all processes are used
for each iteration, so all the programs use all processes. In this case, this doesn't make much
sense because none of these programs can benefit from more than one process. But if we change this
to "1" in this loop, only one process will be used for each iteration in this loop, so for each day.
If we now run our script, not with the single process executable but instead with the
MPI (Message Passing Interface) executable. If you have installed MPI, you can compile a multi-core
or multi-process executable and we can run this instead. And if we run this, we'll see,
okay, multiple processes are working at the same time and you can see multiple
plots – or all three plots – are generated at the same time. So it goes quite a bit faster.
What we can also do is – if you think this is useful for your script – we can write
log files. We could write a log file if we check this here. Then it writes
all the output into this log file. But another thing we could do is add a program called
"GroupPrograms" and if we move our inner loop into this program and define an "outputfileLog"
for this group – which we could say it should be in a directory "log" and should be...
... should use the date as a file name. We have to be careful: We have to create our log directory at
the beginning. But now what happens is that for each outer loop – for each day – it creates a
log file where the output from this inner loop is written to. If I run this again with MPI,
again, everything happens in parallel but now we have our log directory and we have
one file per day. And this file contains the log output of the inner loop, so
the conversion or the processing of the three satellites.
And one more thing I want to show you is we can also add a condition.
For example, if you remember, we said in our script that we want to
visualize outliers by black diamonds in the plot. And we could now say we only
want this to happen for GRACE satellites. So if I right-click this layer for the diamonds and
say "Set Condition", I can define a new condition – let's call it "conditionIsGrace" – and
for this condition there are multiple types you can use. You can check files, you can
run a command, and so on. But also you can match strings and patterns so I'll choose here
"stringContainsPattern". The string we check will be the content of
our "satellite" variable and the pattern will be "GRACE". So if "GRACE" is contained
in the string, the condition will be true. We have set this on this layer, so if we now run this
and check our plots, we should see in the GRACE plots there still are the
black diamonds – there are outliers – but if I go to a TerraSAR-X plot,
you can see there are no more outliers here. This is one way to use a condition.
I want to show you another way, or an alternative, for both conditions and loops because as you can
see, for loops I used the program "LoopPrograms" and for conditions I right-clicked and added
it. You can also do the reverse. Let's disable this program for now and add...
... add a new program called – again, let's use "LoopPrograms". This time we will loop
over our satellites and here we now are going to use the program "IfPrograms".
So it's just an alternative way of using conditions. We will use our "conditionIsGrace"
in this case and in this program – just to show you what I want to do –
I'm going to use the program "InstrumentConcatenate", which,
as the name says, concatenates instrument files. I can just quickly show you
that I can, for example, concatenate the orbit files – the original orbit files.
Let's use some output file. What I can do now is – here you can see I have one input file but
this element contains the ellipses, which means you can provide multiple files. You could now copy
and paste it and change the date but you can also set a loop on this element. Beforehand, we have to
go back to our previous loop, where we defined the loop over the intervals. We can set that
global – let's call it "loopIntervals". Now we can use this loop here. We can say
"Set Loop", "loopIntervals", and now this config element will be duplicated for each
entry in this loop, so in this case for the days. If I now run this program
– let's use the default one – you can see that it is run twice because we have
the condition that it's only run for GRACE satellites – GRACE-1 and GRACE-2 – and then
the program "InstrumentConcatenate" reads all the seven files and concatenates them.
Okay, so these are the basics of our loops and conditions.
Of course, this is a very powerful feature, so you can do quite a lot of things and add
complex behavior to your script. Yeah, so that's it for now, thanks for watching and bye bye!