diff options
author | Yann Herklotz <git@yannherklotz.com> | 2020-09-15 11:23:53 +0100 |
---|---|---|
committer | Yann Herklotz <git@yannherklotz.com> | 2020-09-15 11:23:53 +0100 |
commit | c6b1a9b7db35440c41a8b645ff6b83c237793583 (patch) | |
tree | 9000e8c356dfa0ea0c3a668b4edb4ba9d3b8131e | |
parent | 3f6cc0f009f5d13dfd80c7fb6008062a44a94c66 (diff) | |
parent | 440f9467f3d607338f1351cde725a4f86019bc6c (diff) | |
download | fccm21_esrhls-c6b1a9b7db35440c41a8b645ff6b83c237793583.tar.gz fccm21_esrhls-c6b1a9b7db35440c41a8b645ff6b83c237793583.zip |
Merge branch 'master' of https://git.overleaf.com/5f2d21b9b10d6c0001c164a4 into master
-rw-r--r-- | eval.tex | 5 |
1 files changed, 2 insertions, 3 deletions
@@ -61,8 +61,7 @@ Figure~\ref{fig:existing_tools} shows a Venn diagram of our results. We see that 167 (2.5\%), 83 (1.2\%) and 26 (0.4\%) test-cases fail in LegUp, Vivado HLS and Intel i++ respectively. Despite i++ having the lowest failure rate, it has the highest time-out rate (540 test-cases), because of its remarkably long compilation time. % We remark that although the Intel HLS Compiler had the smallest number of confirmed test-case failures, it had the most time-outs (which could be masking additional failures) -Note that the absolute numbers here do not necessary correspond number of bugs found. -Multiple programs could crash or fail due to the same bug. +Note that the absolute numbers here do not necessary correspond to the number of bugs in the tools, because a single bug in a language feature that appears frequently in our test suite could cause many programs to crash or fail. Hence, we reduce many of the failing test-cases to identify unique bugs, as summarised in Table~\ref{tab:unique_bugs}. We write `$\ge$' in the table to indicate that all the bug counts are lower bounds -- we did not have time to go through the test-case reduction process for every failure. @@ -183,7 +182,7 @@ Figure~\ref{fig:eval:vivado:mismatch} shows code that does not output the right It returns \texttt{0x0} with Vivado HLS, instead of \texttt{0xF}. This test-case is much larger compared to the other test-cases that were reduced. We could not reduce this program any further, as everything in the code was necessary to trigger the bug. -The array \texttt{a} is initialised to all zeroes, as well as the other global variables \texttt{g} and \texttt{c}, so as to not introduce any undefined behaviour. However, \texttt{g} is also given the \texttt{volatile} keyword, which ensures that the variable is not optimised away. The function \texttt{d} then accumulates the values \texttt{b} that it is passed into a hash stored in \texttt{c}. Each \texttt{b} is eight bits wide, so function \texttt{e} calls the function 7 times for some of the bits in the 64-bit value of \texttt{f} that it is passed. Finally, in the main function, the array is initialised partially with a \code{for} loop, after which the \texttt{e} function is called twice, once on the volatile function and once on a constant. Interestingly, the second function call with the constant is also necessary to trigger the bug. +The array \texttt{a} is initialised to all zeroes, as well as the other global variables \texttt{g} and \texttt{c}, so as to not introduce any undefined behaviour. However, \texttt{g} is also given the \texttt{volatile} keyword, which ensures that the variable is not optimised away. The function \texttt{d} then accumulates the values \texttt{b} that it is passed into a hash stored in \texttt{c}. Each \texttt{b} is eight bits wide, so function \texttt{e} calls the function seven times for some of the bits in the 64-bit value of \texttt{f} that it is passed. Finally, in the main function, the array is initialised partially with a \code{for} loop, after which the \texttt{e} function is called twice, once on the volatile function and once on a constant. Interestingly, the second function call with the constant is also necessary to trigger the bug. \begin{figure} \begin{minted}{c} |