summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorYann Herklotz <git@yannherklotz.com>2020-09-15 11:23:53 +0100
committerYann Herklotz <git@yannherklotz.com>2020-09-15 11:23:53 +0100
commitc6b1a9b7db35440c41a8b645ff6b83c237793583 (patch)
tree9000e8c356dfa0ea0c3a668b4edb4ba9d3b8131e
parent3f6cc0f009f5d13dfd80c7fb6008062a44a94c66 (diff)
parent440f9467f3d607338f1351cde725a4f86019bc6c (diff)
downloadfccm21_esrhls-c6b1a9b7db35440c41a8b645ff6b83c237793583.tar.gz
fccm21_esrhls-c6b1a9b7db35440c41a8b645ff6b83c237793583.zip
Merge branch 'master' of https://git.overleaf.com/5f2d21b9b10d6c0001c164a4 into master
-rw-r--r--eval.tex5
1 files changed, 2 insertions, 3 deletions
diff --git a/eval.tex b/eval.tex
index 23abc5c..d2d7272 100644
--- a/eval.tex
+++ b/eval.tex
@@ -61,8 +61,7 @@ Figure~\ref{fig:existing_tools} shows a Venn diagram of our results.
We see that 167 (2.5\%), 83 (1.2\%) and 26 (0.4\%) test-cases fail in LegUp, Vivado HLS and Intel i++ respectively.
Despite i++ having the lowest failure rate, it has the highest time-out rate (540 test-cases), because of its remarkably long compilation time.
% We remark that although the Intel HLS Compiler had the smallest number of confirmed test-case failures, it had the most time-outs (which could be masking additional failures)
-Note that the absolute numbers here do not necessary correspond number of bugs found.
-Multiple programs could crash or fail due to the same bug.
+Note that the absolute numbers here do not necessary correspond to the number of bugs in the tools, because a single bug in a language feature that appears frequently in our test suite could cause many programs to crash or fail.
Hence, we reduce many of the failing test-cases to identify unique bugs, as summarised in Table~\ref{tab:unique_bugs}.
We write `$\ge$' in the table to indicate that all the bug counts are lower bounds -- we did not have time to go through the test-case reduction process for every failure.
@@ -183,7 +182,7 @@ Figure~\ref{fig:eval:vivado:mismatch} shows code that does not output the right
It returns \texttt{0x0} with Vivado HLS, instead of \texttt{0xF}. This test-case is much larger compared to the other test-cases that were reduced.
We could not reduce this program any further, as everything in the code was necessary to trigger the bug.
-The array \texttt{a} is initialised to all zeroes, as well as the other global variables \texttt{g} and \texttt{c}, so as to not introduce any undefined behaviour. However, \texttt{g} is also given the \texttt{volatile} keyword, which ensures that the variable is not optimised away. The function \texttt{d} then accumulates the values \texttt{b} that it is passed into a hash stored in \texttt{c}. Each \texttt{b} is eight bits wide, so function \texttt{e} calls the function 7 times for some of the bits in the 64-bit value of \texttt{f} that it is passed. Finally, in the main function, the array is initialised partially with a \code{for} loop, after which the \texttt{e} function is called twice, once on the volatile function and once on a constant. Interestingly, the second function call with the constant is also necessary to trigger the bug.
+The array \texttt{a} is initialised to all zeroes, as well as the other global variables \texttt{g} and \texttt{c}, so as to not introduce any undefined behaviour. However, \texttt{g} is also given the \texttt{volatile} keyword, which ensures that the variable is not optimised away. The function \texttt{d} then accumulates the values \texttt{b} that it is passed into a hash stored in \texttt{c}. Each \texttt{b} is eight bits wide, so function \texttt{e} calls the function seven times for some of the bits in the 64-bit value of \texttt{f} that it is passed. Finally, in the main function, the array is initialised partially with a \code{for} loop, after which the \texttt{e} function is called twice, once on the volatile function and once on a constant. Interestingly, the second function call with the constant is also necessary to trigger the bug.
\begin{figure}
\begin{minted}{c}