summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorJohn Wickerson <j.wickerson@imperial.ac.uk>2020-09-14 21:38:32 +0000
committeroverleaf <overleaf@localhost>2020-09-14 21:39:51 +0000
commit4d3cd6723ec8ce7e156d8df5ce61ffd67e48977f (patch)
tree10bec3975c0879eeff836617042b9f8ef6d79c42
parent5d1e6f9024d810b7c726f0b5e5ca18bf6ae0eb2c (diff)
downloadfccm21_esrhls-4d3cd6723ec8ce7e156d8df5ce61ffd67e48977f.tar.gz
fccm21_esrhls-4d3cd6723ec8ce7e156d8df5ce61ffd67e48977f.zip
Update on Overleaf.
-rw-r--r--eval_rewrite.tex2
-rw-r--r--method-new.tex24
2 files changed, 14 insertions, 12 deletions
diff --git a/eval_rewrite.tex b/eval_rewrite.tex
index a100ad7..4a980ad 100644
--- a/eval_rewrite.tex
+++ b/eval_rewrite.tex
@@ -41,7 +41,7 @@
\label{tab:unique_bugs}
\end{table}
-To evaluate the different HLS tools 10,000 test cases were generated and fed into each tool, keeping the test cases constant so that the comparison between the tools was fair. Three different tools were tested, including three different versions of Vivado, which are shown in Table~\ref{tab:unique_bugs}. Bugs were found in all tools that were tested, and in total, ?? unique bugs were found and reported to the tool vendors.
+To evaluate the different HLS tools 10,000 \JW{check numbers} test cases were generated and fed into each tool, keeping the test cases constant so that the comparison between the tools was fair. Three different tools were tested, including three different versions of Vivado, which are shown in Table~\ref{tab:unique_bugs}. Bugs were found in all tools that were tested, and in total, ?? unique bugs were found and reported to the tool vendors.
\subsection{Bugs found}
diff --git a/method-new.tex b/method-new.tex
index d24e35a..e931f70 100644
--- a/method-new.tex
+++ b/method-new.tex
@@ -163,17 +163,19 @@ Having generated HLS-friendly programs and automatically augmented them with dir
%Figure~\ref{fig:method:toolflow} shows the three stages of testing, depicted as the testing environment in the dashed area.
For each HLS tool in turn, we compile the C program to RTL and then simulate the RTL.
Independently, we also compile the C program using GCC and execute it.
-Additionally, to ensure that our testing is scalable for a large number of large, random programs, we also enforce several time-outs: we set a 5-minute time-out for C execution and a 2-hour time-out for RTL simulation.
+To ensure that our testing is scalable for a large number of large, random programs, we also enforce several time-outs: we set a 5-minute time-out for C execution and a 2-hour time-out for RTL simulation.
We do not count time-outs as bugs, but we record them.
-\paragraph{Types of bugs}
-There two types of bugs that we can encounter in this testing setup: programs that cause the HLS tool to crash during compilation (e.g. an unhandled assertion violation or a segmentation fault), and programs where the software execution and the RTL simulation do not return the same value.
-Programs that cause either type of error are given to the reduction stage, which aims to minimise the programs and (hopefully) identify the root cause(s).
+%% JW: This paragraph is not really needed because we repeat the sentiment in the next subsection anyway.
+%There two types of bugs that we can encounter in this testing setup: programs that cause the HLS tool to crash during compilation (e.g. an unhandled assertion violation or a segmentation fault), and programs where the software execution and the RTL simulation do not return the same value. Programs that cause either type of error are given to the reduction stage, which aims to minimise the programs and (hopefully) identify the root cause(s).
-\paragraph{Tool-specific trivia}
-Vivado HLS implements C and RTL simulation by default.
-We were only able to test one version of LegUp HLS (version 4.0).
-LegUp 7.5 is GUI-based and not suitable for scripting; however, bugs we found in LegUp 4.0 were reproduced manually in LegUp 7.5.
+
+%\paragraph{Tool-specific trivia}
+%Vivado HLS implements C and RTL simulation by default.
+\JW{Maybe move this later on?
+``We were only able to test one version of LegUp HLS (version 4.0).
+LegUp 7.5 is GUI-based and not suitable for scripting; however, bugs we found in LegUp 4.0 were reproduced manually in LegUp 7.5.''
+}
%For all tools, we set a 5-minute and 2-hour for C simulation and RTL simulation respectively.
%In cases where simulation takes longer, we do not consider them as failures or crashes and we record them.
% When testing Intel HLS, we place three time-outs since its execution is generally slower than other HLS tools: a \ref{XX}-minute time-out for C compilation, a \ref{XX}-hour time out for HLS compilation and a \ref{XX}-hour time-out for co-simulation.
@@ -182,11 +184,11 @@ LegUp 7.5 is GUI-based and not suitable for scripting; however, bugs we found in
\subsection{Reducing buggy programs}
\label{sec:method:reduce}
-Once we discover any programs that crashes the HLS tool or whose C and RTL simulation do not match, we must further scrutinise these programs to identify the root cause(s) for these undesirable behaviours.
+Once we discover a program that crashes the HLS tool or whose C/RTL simulations do not match, we further scrutinise the program to identify the root cause(s) of the undesirable behaviour.
As the programs generated by Csmith can be fairly large, we must systematically reduce these programs to identify the source of a bug.
-Reduction is performed by removing some part of the original program and then provide this reduced program to the HLS tool for re-synthesis and co-simulation.
-The goal is to find a program that is small enough that still crashes or does not match the output of the C execution.
+Reduction is performed by iteratively removing some part of the original program and then providing the reduced program to the HLS tool for re-synthesis and co-simulation.
+The goal is to find the smallest program that still triggers the bug.
We apply two consecutive methods of reduction in this work.
We first perform a custom reduction in which we iteratively remove lines in strict order from the top-level function.
This method of reduction first reduces the pragmas and labels that were added before synthesis of the C program, after which is proceeds by iteratively commenting out lines in the C program one after another until any further reduction would eliminate the bug.