We use head-to-head comparisons between two implementations. These comparisons are about execution times and numbers of exceptions. In cases where one of the implementations generates an exception (timeout or overflow), we do not compare their execution times, and exclude them from the total execution times. Thus, the total execution times mentioned later in this chapter can be understood as execution times for cases solvable by both implementations only. We also exclude the input/output execution time in our tests.
POLYBENCH's implementation for the automation of comparisons consists
of several UNIX shell scripts and C programs. It has been tested on
platforms Debian Linux and Sun Solaris. The database generation part
is strongly related to PIPS and 
infrastructure. Nonetheless,
once the polyhedral databases are created, the POLYBENCH can be used
with only a fragment of code.
6_fig:polybench_application represents some possible applications of POLYBENCH. Polyhedral libraries can use our POLYBENCH's set of tests so that their performance can be evaluated.
