An operator returns for a given operation an answer or an exception. When we compare two implementations, we say that an implementation is more robust than the other if it generates fewer exceptions. We also say that one is more efficient than the other if it returns a valid answer sooner than the other does. Finally, we consider that one implementation is more efficient than another if it raises an exception earlier, on condition that both implementations return an exception with the given operands (see 6_polybench_implementation for our implementation).
The robustness refers to the ability of dealing with exceptions, which can be quantified. Exceptions can be either magnitude overflows, i.e. when a number exceeding the maximum that can be stored in a memory word, or timeouts, i.e. no operation may last too long. The timeout exception means that the operator implementation cannot solve the given task in a pre-defined time.
In fact, memory managements implemented in available libraries are different: we have virtual memory space for some algorithms and limited for others. Due to this difference, we consider memory overflows, e.g. out-of-memory space, as magnitude overflows when it happens before the timeout exceptions.