summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorJohn Wickerson <j.wickerson@imperial.ac.uk>2020-09-15 09:59:34 +0000
committeroverleaf <overleaf@localhost>2020-09-15 10:00:13 +0000
commita05dfbf81ab3d206bed194dd653430b0ab155103 (patch)
treee86357bcd0a97e3ffd7fa6674c557b4726f3e04d
parent05e03565de7b4a5319c9d4c15f327317229f41b5 (diff)
downloadfccm21_esrhls-a05dfbf81ab3d206bed194dd653430b0ab155103.tar.gz
fccm21_esrhls-a05dfbf81ab3d206bed194dd653430b0ab155103.zip
Update on Overleaf.
-rw-r--r--conclusion.tex4
-rw-r--r--conference.bib15
-rw-r--r--eval.tex7
3 files changed, 18 insertions, 8 deletions
diff --git a/conclusion.tex b/conclusion.tex
index e027b27..4521155 100644
--- a/conclusion.tex
+++ b/conclusion.tex
@@ -3,7 +3,9 @@ We have shown how existing fuzzing tools can be modified so that their outputs a
One can always question how much bugs found by fuzzers really \emph{matter}, given that they are usually found by combining language features in ways that are vanishingly unlikely to happen `in the wild'~\cite{marcozzi+19}. This question is especially pertinent for our particular context of HLS tools, which are well-known to have restrictions on the language features that they handle. Nevertheless, we would argue that any errors in the HLS tool are worth identifying because they have the potential to cause problems, either now or in the future. And when HLS tools \emph{do} go wrong (or indeed any sort of compiler for that matter), it is particularly infuriating for end-users because it is so difficult to identify whether the fault lies with the tool or with the program it has been given to compile.
-Further work could be done on supporting more HLS tools, especially ones that claim to prove that their output is correct before terminating. This could give an indication on how effective these proofs are, and how often they are actually able to complete their equivalence proofs during compilation in a feasible time scale.
+Further work could be done on supporting more HLS tools, especially ones that claim to prove that their output is correct before terminating. This could give an indication on how effective these proofs are, and how often they are actually able to complete their equivalence proofs during compilation in a feasible timescale.
+
+Conventional compilers have become quite resilient to fuzzing over the last decade, so recent work on fuzzing compilers has had to employ increasingly imaginative techniques to keep finding new bugs~\cite{karine+20}. In comparison, we have found that the HLS tools we tested can be made to exhibit bugs even with relatively basic fuzzing techniques that we employed
As HLS is becoming increasingly relied upon, it is important to make sure that HLS tools are also reliable. We hope that this work further motivates the need for rigorous engineering of HLS tools, either by validating that each output the tool produces is correct or by proving the HLS tool itself correct once and for all.
diff --git a/conference.bib b/conference.bib
index 3e766f8..5164225 100644
--- a/conference.bib
+++ b/conference.bib
@@ -2,7 +2,7 @@
author = {Canis, Andrew and Choi, Jongsok and Aldham, Mark and Zhang, Victor and
Kammoona, Ahmed and Czajkowski, Tomasz and Brown, Stephen D. and Anderson, Jason
H.},
- title = {Legup: an Open-Source High-Level Synthesis Tool for Fpga-Based
+ title = {{LegUp}: an Open-Source High-Level Synthesis Tool for {FPGA}-Based
Processor/accelerator Systems},
journal = {ACM Trans. Embed. Comput. Syst.},
volume = {13},
@@ -82,7 +82,7 @@
year = {2020},
month = {June},
url = {https://bit.ly/hls-fintech},
- howpublished = {Press Release},
+ howpublished = {Press release},
}
@misc{hls_objdetect,
@@ -91,7 +91,7 @@
year = {2019},
month = {January},
url = {https://bit.ly/hls-objdetect},
- howpublished = {Press Release},
+ howpublished = {Press release},
}
@misc{hls_controller,
@@ -235,4 +235,13 @@ with {LegUp} High-Level Synthesis},
timestamp = {Tue, 06 Nov 2018 16:58:22 +0100},
biburl = {https://dblp.org/rec/conf/fpga/RamanathanFWC17.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
+}
+
+@InProceedings{karine+20,
+author = {Karine Even-Mendoza and Cristian Cadar and Alastair Donaldson},
+title = {Closer to the Edge: Testing Compilers More Thoroughly by Being Less Conservative About Undefined Behaviour},
+booktitle = {IEEE/ACM International Conference on Automated Software Engineering, New Ideas and Emerging Results Track (ASE-NIER 2020)},
+year = {2020},
+month = {09},
+location = {undefined},
} \ No newline at end of file
diff --git a/eval.tex b/eval.tex
index bf8dc5d..4b07961 100644
--- a/eval.tex
+++ b/eval.tex
@@ -40,17 +40,16 @@
Intel i++ & $\ge 1$\\
\bottomrule
\end{tabular}
- \caption{Unique bugs found in each tool.}
+ \caption{Unique bugs found in each tool. \JW{is `all versions' correct here? and should we add version numbers like in the Venn?}\YH{Yes it is actually correct here, I don't mind adding the specific version either though}}
\label{tab:unique_bugs}
\end{table}
We generate 6700 test-cases and provide them to three HLS tools: Vivado HLS, LegUp HLS and Intel i++.
We use the same test-cases across all tools for fair comparison.
We were able to test three different versions of Vivado HLS (v2018.3, v2019.1 and v2019.2).
-We were only able to test one version of LegUp: 4.0.
-At the point of writing, LegUp 7.5 is still GUI-based and therefore we could not script our tests.
+We tested one version of Intel i++ (version 18.1), and one version of LegUp (4.0).
+LegUp 7.5 is GUI-based and therefore we could not script our tests.
However, we were able to manually reproduce bugs found in LegUp 4.0 in LegUp 7.5.
-Finally, we tested one version of Intel i++ 18.1.
% Three different tools were tested, including three different versions of Vivado HLS. We were only able to test one version of LegUp HLS (version 4.0), because although LegUp 7.5 is available, it is GUI-based and not amenable to scripting. However, bugs we found in LegUp 4.0 were reproduced manually in LegUp 7.5.
% LegUp and Vivado HLS were run under Linux, while the Intel HLS Compiler was run under Windows.