summaryrefslogtreecommitdiffstats
path: root/intro.tex
diff options
context:
space:
mode:
authorYann Herklotz <git@yannherklotz.com>2021-03-30 18:48:23 +0100
committerYann Herklotz <git@yannherklotz.com>2021-03-30 18:48:23 +0100
commit0f40e13fab830957ac055e076055280cdb82efff (patch)
tree45853a4552e3535f5ae8993a12d16537f984d23e /intro.tex
parent4eb24224015629936727e37f60738ac412578f50 (diff)
downloadfccm21_esrhls-0f40e13fab830957ac055e076055280cdb82efff.tar.gz
fccm21_esrhls-0f40e13fab830957ac055e076055280cdb82efff.zip
Fix use of test-case
Diffstat (limited to 'intro.tex')
-rw-r--r--intro.tex4
1 files changed, 2 insertions, 2 deletions
diff --git a/intro.tex b/intro.tex
index 7634475..7fa4905 100644
--- a/intro.tex
+++ b/intro.tex
@@ -61,9 +61,9 @@ Ultimately, we believe that any errors in an HLS tool are worth identifying beca
Our approach to fuzzing HLS tools comprises three steps.
First, we use Csmith~\cite{yang11_findin_under_bugs_c_compil} to generate thousands of valid C programs within the subset of the C language that is supported by all the HLS tools we test. We also augment each program with a random selection of HLS-specific directives. Second, we give these programs to four widely used HLS tools: Xilinx Vivado HLS~\cite{xilinx20_vivad_high_synth}, LegUp HLS~\cite{canis13_legup}, the Intel HLS Compiler, also known as i++~\cite{intel20_sdk_openc_applic}, and finally Bambu~\cite{pilato13_bambu}. Third, if we find a program that causes an HLS tool to crash, or to generate hardware that produces a different result from GCC, we reduce it to a minimal example with the help of the \creduce{} tool~\cite{creduce}.
-Our testing campaign revealed that all four tools could be made to generate an incorrect design. In total, \totaltestcases{} test cases were run through each tool, of which \totaltestcasefailures{} failed in at least one of the tools. Test case reduction was then performed on some of these failing test cases to obtain at least \numuniquebugs{} unique failing test cases.
+Our testing campaign revealed that all four tools could be made to generate an incorrect design. In total, \totaltestcases{} test-cases were run through each tool, of which \totaltestcasefailures{} failed in at least one of the tools. Test-case reduction was then performed on some of these failing test-cases to obtain at least \numuniquebugs{} unique failing test-cases.
-To investigate whether HLS tools are getting more or less reliable over time, we also tested three different versions of Vivado HLS (v2018.3, v2019.1, and v2019.2). We found far fewer failures in versions v2019.1 and v2019.2 compared to v2018.3, but we also identified a few test cases that only failed in versions v2019.1 and v2019.2; this suggests that some new features may have introduced bugs.
+To investigate whether HLS tools are getting more or less reliable over time, we also tested three different versions of Vivado HLS (v2018.3, v2019.1, and v2019.2). We found far fewer failures in versions v2019.1 and v2019.2 compared to v2018.3, but we also identified a few test-cases that only failed in versions v2019.1 and v2019.2; this suggests that some new features may have introduced bugs.
In summary, the overall aim of our paper is to raise awareness about the reliability (or lack thereof) of current HLS tools, and to serve as a call-to-arms for investment in better-engineered tools. We hope that future work on developing more reliable HLS tools will find our empirical study a valuable source of motivation.