summaryrefslogtreecommitdiffstats
path: root/conclusion.tex
diff options
context:
space:
mode:
authorYann Herklotz <ymh15@ic.ac.uk>2020-12-17 14:04:42 +0000
committeroverleaf <overleaf@localhost>2020-12-31 14:48:38 +0000
commita5249eb597549437802d2ed852919e5b9a923840 (patch)
tree29a32aa1fba1dc0211be88497884d0c7a2db1690 /conclusion.tex
parentea9289245fbc493530e9435faf498cc4a824c70f (diff)
downloadfccm21_esrhls-a5249eb597549437802d2ed852919e5b9a923840.tar.gz
fccm21_esrhls-a5249eb597549437802d2ed852919e5b9a923840.zip
Update on Overleaf.
Diffstat (limited to 'conclusion.tex')
-rw-r--r--conclusion.tex8
1 files changed, 3 insertions, 5 deletions
diff --git a/conclusion.tex b/conclusion.tex
index a5a3557..fd28bcb 100644
--- a/conclusion.tex
+++ b/conclusion.tex
@@ -1,13 +1,11 @@
\section{Conclusion}
-We have shown how existing fuzzing tools can be modified so that their outputs are compatible with HLS tools. We have used this testing framework to run 6,700 test-cases through three different HLS tools, and 3,645 test-cases through three different version of Vivado HLS to show how bugs are fixed and introduced. In total, we found at least 6 unique bugs in all the tools. These bugs include crashes as well as instances of generated designs not behaving in the same way as the original code.
+We have shown how an existing fuzzing tool can be modified so that its output is suitable for HLS, and then used it in a campaign to test the reliability of three modern HLS tools. In total, we found at least \numuniquebugs{} unique bugs across all the tools, including crashes and miscompilation bugs.
-One can always question how much bugs found by fuzzers really \emph{matter}, given that they are usually found by combining language features in ways that are vanishingly unlikely to happen `in the wild'~\cite{marcozzi+19}. This question is especially pertinent for our particular context of HLS tools, which are well-known to have restrictions on the language features that they handle. \JW{I added the following two sentences, based on our rebuttal -- please check.} Nevertheless, we would argue that although the \emph{test-cases} we generated do not resemble the programs that humans write, the \emph{bugs} that we exposed using those test-cases are real, and could also be exposed by realistic programs. Moreover, it is worth noting that HLS tools not exclusively provided with human-written programs to compile: they are often fed programs that have been automatically generated by another compiler. Ultimately, we believe that any errors in an HLS tool are worth identifying because they have the potential to cause problems, either now or in the future. And when HLS tools \emph{do} go wrong (or indeed any sort of compiler for that matter), it is particularly infuriating for end-users because it is so difficult to identify whether the fault lies with the tool or with the program it has been given to compile.
-
-Further work could be done on supporting more HLS tools, especially ones that claim to prove that their output is correct before terminating. This could give an indication on how effective these proofs are, and how often they are actually able to complete their equivalence proofs during compilation in a feasible timescale.
+Further work could be done on supporting more HLS tools, especially ones that claim to prove that their output is correct before terminating, such as Catapult-C~\cite{mentor20_catap_high_level_synth}. % This could give an indication of how effective these proofs are, and how often they are actually able to complete their equivalence proofs during compilation in a feasible timescale.
Conventional compilers have become quite resilient to fuzzing over the last decade, so recent work on fuzzing compilers has had to employ increasingly imaginative techniques to keep finding new bugs~\cite{karine+20}. In comparison, we have found that HLS tools -- at least, as they currently stand -- can be made to exhibit bugs even using the relatively basic fuzzing techniques that we employed in this project.
-As HLS is becoming increasingly relied upon, it is important to make sure that HLS tools are also reliable. We hope that this work further motivates the need for rigorous engineering of HLS tools, either by validating that each output the tool produces is correct or by proving the HLS tool itself correct once and for all.
+As HLS is becoming increasingly relied upon, it is important to make sure that HLS tools are also reliable. We hope that this work further motivates the need for rigorous engineering of HLS tools, whether that is by validating that each output the tool produces is correct or by proving the HLS tool itself correct once and for all.
%%% Local Variables:
%%% mode: latex