summaryrefslogtreecommitdiffstats
path: root/eval.tex
diff options
context:
space:
mode:
authorYann Herklotz <ymh15@ic.ac.uk>2021-04-04 21:03:01 +0000
committeroverleaf <overleaf@localhost>2021-04-05 12:44:36 +0000
commit6a7ebeb13cd708e5b9e2a09400255ccbd3f0242a (patch)
tree33565de568589a583c91e462c734da2e7e9d07bd /eval.tex
parent62a127dfb009b8ffe94ac348ecafb7f596406cbd (diff)
downloadfccm21_esrhls-6a7ebeb13cd708e5b9e2a09400255ccbd3f0242a.tar.gz
fccm21_esrhls-6a7ebeb13cd708e5b9e2a09400255ccbd3f0242a.zip
Update on Overleaf.
Diffstat (limited to 'eval.tex')
-rw-r--r--eval.tex4
1 files changed, 2 insertions, 2 deletions
diff --git a/eval.tex b/eval.tex
index 88ab6a1..a1b9b28 100644
--- a/eval.tex
+++ b/eval.tex
@@ -66,7 +66,7 @@ Figures~\ref{fig:eval:legup:crash}, \ref{fig:eval:intel:mismatch}, and~\ref{fig:
int a[2][2][1] = {{{0},{1}},{{0},{0}}};
int main() { a[0][1][0] = 1; }
\end{minted}
-\caption{This program leads to an internal compiler error (an unhandled assertion in this case) in LegUp 4.0. It initialises a 3D array with zeroes and then assigns to one element. The bug only appears when function inlining is disabled (\texttt{NO\_INLINE}).}
+\caption{This program leads to an internal compiler error (an unhandled assertion in this case) in LegUp 4.0. It initialises a 3D array with zeroes and then assigns one element. The bug only appears when function inlining is disabled (\texttt{NO\_INLINE}), thus confirming the effectiveness of generating random directives.}
\label{fig:eval:legup:crash}
\end{figure}
%An assertion error counts as a crash of the tool, as it means that an unexpected state was reached by this input.
@@ -224,7 +224,7 @@ For instance, the topmost ribbon represents the 31 test-cases that fail in all t
Interestingly, the blue ribbon shows that there are test-cases that fail in v2018.3, pass in v2019.1, and then fail again in v2019.2!
As in our Euler diagram, the numbers do not necessary correspond to the number of actual bugs, though we can observe that there must be at least six unique bugs in Vivado HLS, given that each ribbon corresponds to at least one unique bug.
-%\AD{This reminds me of the correcting commits metric from Junjie Chen et al.'s empirical study on compiler testing. Could be worth making the connection. }
+%\AD{This reminds me of the correcting commits metric from Junjie Chen et al.'s empirical study on compiler testing. https://xiongyingfei.github.io/papers/ICSE16.pdf. Could be worth making the connection. }
%\YH{Contradicts value of 3 in Table~\ref{tab:unique_bugs}, maybe I can change that to 6?} \JW{I'd leave it as-is personally; we have already put a `$\ge$' symbol in the table, so I think it's fine.}
%In addition to that, it can then be seen that Vivado HLS v2018.3 must have at least 4 individual bugs, of which two were fixed and two others stayed in Vivado HLS v2019.1. However, with the release of v2019.1, new bugs were introduced as well. % Finally, for version 2019.2 of Vivado HLS, there seems to be a bug that was reintroduced which was also present in Vivado 2018.3, in addition to a new bug. In general it seems like each release of Vivado HLS will have new bugs present, however, will also contain many previous bug fixes. However, it cannot be guaranteed that a bug that was previously fixed will remain fixed in future versions as well.