summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorJohn Wickerson <j.wickerson@imperial.ac.uk>2020-09-15 10:20:04 +0000
committeroverleaf <overleaf@localhost>2020-09-15 10:23:49 +0000
commit440f9467f3d607338f1351cde725a4f86019bc6c (patch)
tree06812abf6c166965a559a10f67915219ecb5253e
parent669bc9dbe8f73b56ec34d68eecec9b7758d0890d (diff)
downloadfccm21_esrhls-440f9467f3d607338f1351cde725a4f86019bc6c.tar.gz
fccm21_esrhls-440f9467f3d607338f1351cde725a4f86019bc6c.zip
Update on Overleaf.
-rw-r--r--eval.tex5
1 files changed, 2 insertions, 3 deletions
diff --git a/eval.tex b/eval.tex
index 976a1ce..afacb6f 100644
--- a/eval.tex
+++ b/eval.tex
@@ -61,8 +61,7 @@ Figure~\ref{fig:existing_tools} shows a Venn diagram of our results.
We see that 167 (2.5\%), 83 (1.2\%) and 26 (0.4\%) test-cases fail in LegUp, Vivado HLS and Intel i++ respectively.
Despite i++ having the lowest failure rate, it has the highest time-out rate (540 test-cases), because of its remarkably long compilation time.
% We remark that although the Intel HLS Compiler had the smallest number of confirmed test-case failures, it had the most time-outs (which could be masking additional failures)
-Note that the absolute numbers here do not necessary correspond number of bugs found.
-Multiple programs could crash or fail due to the same bug.
+Note that the absolute numbers here do not necessary correspond to the number of bugs in the tools, because a single bug in a language feature that appears frequently in our test suite could cause many programs to crash or fail.
Hence, we reduce many of the failing test-cases to identify unique bugs, as summarised in Table~\ref{tab:unique_bugs}.
We write `$\ge$' in the table to indicate that all the bug counts are lower bounds -- we did not have time to go through the test-case reduction process for every failure.
@@ -183,7 +182,7 @@ Figure~\ref{fig:eval:vivado:mismatch} shows code that does not output the right
It returns \texttt{0x0} with Vivado HLS, instead of \texttt{0xF}. This test-case is much larger compared to the other test-cases that were reduced.
We could not reduce this program any further, as everything in the code was necessary to trigger the bug.
-The array \texttt{a} is initialised to all zeroes, as well as the other global variables \texttt{g} and \texttt{c}, so as to not introduce any undefined behaviour. However, \texttt{g} is also given the \texttt{volatile} keyword, which ensures that the variable is not optimised away. The function \texttt{d} then accumulates the values \texttt{b} that it is passed into a hash stored in \texttt{c}. Each \texttt{b} is eight bits wide, so function \texttt{e} calls the function 7 times for some of the bits in the 64-bit value of \texttt{f} that it is passed. Finally, in the main function, the array is initialised partially with a \code{for} loop, after which the \texttt{e} function is called twice, once on the volatile function and once on a constant. Interestingly, the second function call with the constant is also necessary to trigger the bug.
+The array \texttt{a} is initialised to all zeroes, as well as the other global variables \texttt{g} and \texttt{c}, so as to not introduce any undefined behaviour. However, \texttt{g} is also given the \texttt{volatile} keyword, which ensures that the variable is not optimised away. The function \texttt{d} then accumulates the values \texttt{b} that it is passed into a hash stored in \texttt{c}. Each \texttt{b} is eight bits wide, so function \texttt{e} calls the function seven times for some of the bits in the 64-bit value of \texttt{f} that it is passed. Finally, in the main function, the array is initialised partially with a \code{for} loop, after which the \texttt{e} function is called twice, once on the volatile function and once on a constant. Interestingly, the second function call with the constant is also necessary to trigger the bug.
\begin{figure}
\begin{minted}{c}