summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorYann Herklotz <git@yannherklotz.com>2020-09-14 22:39:56 +0100
committerYann Herklotz <git@yannherklotz.com>2020-09-14 22:39:56 +0100
commit544d9ed2d36726afeb96b4fa130ae36d95bed558 (patch)
tree55b50c436fd8235814a5a0b93a7e615156fe2b4f
parentfc5176c898d528e2ef9d44c462db7b57585b000a (diff)
parent4d3cd6723ec8ce7e156d8df5ce61ffd67e48977f (diff)
downloadfccm21_esrhls-544d9ed2d36726afeb96b4fa130ae36d95bed558.tar.gz
fccm21_esrhls-544d9ed2d36726afeb96b4fa130ae36d95bed558.zip
Merge branch 'master' of https://git.overleaf.com/5f2d21b9b10d6c0001c164a4 into master
-rw-r--r--eval_rewrite.tex2
-rw-r--r--method-new.tex24
2 files changed, 14 insertions, 12 deletions
diff --git a/eval_rewrite.tex b/eval_rewrite.tex
index 364117f..ab855b4 100644
--- a/eval_rewrite.tex
+++ b/eval_rewrite.tex
@@ -41,7 +41,7 @@
\label{tab:unique_bugs}
\end{table}
-To evaluate the different HLS tools 10,000 test cases were generated and fed into each tool, keeping the test cases constant so that the comparison between the tools was fair. Three different tools were tested, including three different versions of Vivado, which are shown in Table~\ref{tab:unique_bugs}. Bugs were found in all tools that were tested, and in total, ?? unique bugs were found and reported to the tool vendors.
+To evaluate the different HLS tools 10,000 \JW{check numbers} test cases were generated and fed into each tool, keeping the test cases constant so that the comparison between the tools was fair. Three different tools were tested, including three different versions of Vivado, which are shown in Table~\ref{tab:unique_bugs}. Bugs were found in all tools that were tested, and in total, ?? unique bugs were found and reported to the tool vendors.
\subsection{Bugs found}
diff --git a/method-new.tex b/method-new.tex
index d24e35a..e931f70 100644
--- a/method-new.tex
+++ b/method-new.tex
@@ -163,17 +163,19 @@ Having generated HLS-friendly programs and automatically augmented them with dir
%Figure~\ref{fig:method:toolflow} shows the three stages of testing, depicted as the testing environment in the dashed area.
For each HLS tool in turn, we compile the C program to RTL and then simulate the RTL.
Independently, we also compile the C program using GCC and execute it.
-Additionally, to ensure that our testing is scalable for a large number of large, random programs, we also enforce several time-outs: we set a 5-minute time-out for C execution and a 2-hour time-out for RTL simulation.
+To ensure that our testing is scalable for a large number of large, random programs, we also enforce several time-outs: we set a 5-minute time-out for C execution and a 2-hour time-out for RTL simulation.
We do not count time-outs as bugs, but we record them.
-\paragraph{Types of bugs}
-There two types of bugs that we can encounter in this testing setup: programs that cause the HLS tool to crash during compilation (e.g. an unhandled assertion violation or a segmentation fault), and programs where the software execution and the RTL simulation do not return the same value.
-Programs that cause either type of error are given to the reduction stage, which aims to minimise the programs and (hopefully) identify the root cause(s).
+%% JW: This paragraph is not really needed because we repeat the sentiment in the next subsection anyway.
+%There two types of bugs that we can encounter in this testing setup: programs that cause the HLS tool to crash during compilation (e.g. an unhandled assertion violation or a segmentation fault), and programs where the software execution and the RTL simulation do not return the same value. Programs that cause either type of error are given to the reduction stage, which aims to minimise the programs and (hopefully) identify the root cause(s).
-\paragraph{Tool-specific trivia}
-Vivado HLS implements C and RTL simulation by default.
-We were only able to test one version of LegUp HLS (version 4.0).
-LegUp 7.5 is GUI-based and not suitable for scripting; however, bugs we found in LegUp 4.0 were reproduced manually in LegUp 7.5.
+
+%\paragraph{Tool-specific trivia}
+%Vivado HLS implements C and RTL simulation by default.
+\JW{Maybe move this later on?
+``We were only able to test one version of LegUp HLS (version 4.0).
+LegUp 7.5 is GUI-based and not suitable for scripting; however, bugs we found in LegUp 4.0 were reproduced manually in LegUp 7.5.''
+}
%For all tools, we set a 5-minute and 2-hour for C simulation and RTL simulation respectively.
%In cases where simulation takes longer, we do not consider them as failures or crashes and we record them.
% When testing Intel HLS, we place three time-outs since its execution is generally slower than other HLS tools: a \ref{XX}-minute time-out for C compilation, a \ref{XX}-hour time out for HLS compilation and a \ref{XX}-hour time-out for co-simulation.
@@ -182,11 +184,11 @@ LegUp 7.5 is GUI-based and not suitable for scripting; however, bugs we found in
\subsection{Reducing buggy programs}
\label{sec:method:reduce}
-Once we discover any programs that crashes the HLS tool or whose C and RTL simulation do not match, we must further scrutinise these programs to identify the root cause(s) for these undesirable behaviours.
+Once we discover a program that crashes the HLS tool or whose C/RTL simulations do not match, we further scrutinise the program to identify the root cause(s) of the undesirable behaviour.
As the programs generated by Csmith can be fairly large, we must systematically reduce these programs to identify the source of a bug.
-Reduction is performed by removing some part of the original program and then provide this reduced program to the HLS tool for re-synthesis and co-simulation.
-The goal is to find a program that is small enough that still crashes or does not match the output of the C execution.
+Reduction is performed by iteratively removing some part of the original program and then providing the reduced program to the HLS tool for re-synthesis and co-simulation.
+The goal is to find the smallest program that still triggers the bug.
We apply two consecutive methods of reduction in this work.
We first perform a custom reduction in which we iteratively remove lines in strict order from the top-level function.
This method of reduction first reduces the pragmas and labels that were added before synthesis of the C program, after which is proceeds by iteratively commenting out lines in the C program one after another until any further reduction would eliminate the bug.