summaryrefslogtreecommitdiffstats
path: root/intro.tex
diff options
context:
space:
mode:
authorJohn Wickerson <j.wickerson@imperial.ac.uk>2020-09-22 10:36:40 +0000
committeroverleaf <overleaf@localhost>2020-10-24 15:09:08 +0000
commit25fc95d19a586f774a99630ea34e58fb76e4e629 (patch)
tree5809da7bc117d8bfd2049b3bca90aa610589c5d5 /intro.tex
parentdfac4f477dfa32611be640c2fef65646e717a6f0 (diff)
downloadfccm21_esrhls-25fc95d19a586f774a99630ea34e58fb76e4e629.tar.gz
fccm21_esrhls-25fc95d19a586f774a99630ea34e58fb76e4e629.zip
Update on Overleaf.
Diffstat (limited to 'intro.tex')
-rw-r--r--intro.tex6
1 files changed, 3 insertions, 3 deletions
diff --git a/intro.tex b/intro.tex
index 990a241..896b185 100644
--- a/intro.tex
+++ b/intro.tex
@@ -15,7 +15,7 @@ int main() {
return b;
}
\end{minted}
- \caption{Miscompilation bug found in Xilinx Vivado HLS 2018.3 and 2019.2. The program returns \code{0x006535FF} but the correct result is \code{0x046535FF}.}
+ \caption{Miscompilation bug found in Xilinx Vivado HLS v2018.3, v2019.1 and v2019.2. The program returns \code{0x006535FF} but the correct result is \code{0x046535FF}.}
\label{fig:vivado_bug1}
\end{figure}
@@ -60,13 +60,13 @@ For this reason, we find it natural to adopt fuzzing for our HLS testing campaig
\paragraph{Our contribution}
This paper reports on our campaign to test HLS tools by fuzzing.
\begin{itemize}
- \item We use Csmith~\cite{yang11_findin_under_bugs_c_compil} to generate ten thousand valid C programs from within the subset of the C language that is supported by all the HLS tools we test. We augment each program with a random selection of HLS-specific directives.
+ \item We use Csmith~\cite{yang11_findin_under_bugs_c_compil} to generate thousands of valid C programs from within the subset of the C language that is supported by all the HLS tools we test. We also augment each program with a random selection of HLS-specific directives.
\item We give these programs to three widely used HLS tools: Xilinx Vivado HLS~\cite{xilinx20_vivad_high_synth}, LegUp HLS~\cite{canis13_legup} and the Intel HLS Compiler, which is also known as i++~\cite{intel20_sdk_openc_applic}. When we find a program that causes an HLS tool to crash, or to generate hardware that produces a different result from GCC, we reduce it to a minimal example with the help of the \creduce{} tool~\cite{creduce}.
\item Our testing campaign revealed that all three tools could be made to crash while compiling or to generate wrong RTL. In total, 6700 test cases were run through each tool out of which 272 test cases failed in at least one of the tools. Test case reduction was then performed on some of these failing test cases to obtain at least 6 unique failing test cases.
- \item To investigate whether HLS tools are getting more or less reliable over time, we also tested three different versions of Vivado HLS (2018.3, 2019.1, and 2019.2). We found that in general there about half as many failures in versions 2019.1 and 2019.2 compared to 2018.3. However, there were also test-cases that only failed in versions 2019.1 and 2019.2, meaning bugs were probably introduced due to the addition of new features.
+ \item To investigate whether HLS tools are getting more or less reliable over time, we also tested three different versions of Vivado HLS (v2018.3, v2019.1, and v2019.2). We found that in general there about half as many failures in versions v2019.1 and v2019.2 compared to v2018.3. However, there were also test-cases that only failed in versions v2019.1 and v2019.2, meaning bugs were probably introduced due to the addition of new features.
\end{itemize}
% we test, and then augment each program with randomly chosen HLS-specific directives. We synthesise each C program to RTL, and use a Verilog simulator to calculate its return value. If synthesis crashes, or if this return value differs from the return value obtained by executing a binary compiled from the C program by GCC, then we have found a candidate bug. We then use trial-and-error to reduce the C program to a minimal version that still triggers a bug.