diff options
Diffstat (limited to 'evaluation.tex')
| -rw-r--r-- | evaluation.tex | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/evaluation.tex b/evaluation.tex index 8f20b29..5d2034e 100644 --- a/evaluation.tex +++ b/evaluation.tex @@ -15,21 +15,21 @@ Our evaluation is designed to answer the following three research questions. \JW \begin{description} \item[RQ1] How fast is the hardware generated by \vericert{}, and how does this compare to existing HLS tools? \item[RQ2] How area-efficient is the hardware generated by \vericert{}, and how does this compare to existing HLS tools? -%\item[RQ3] How long does \vericert{} take to produce hardware? +\item[RQ3] How long does \vericert{} take to produce hardware? \end{description} \subsection{Experimental Setup} \paragraph{Choice of HLS tool for comparison.} We compare \vericert{} against LegUp 4.0 because it is open-source and hence easily accessible, but still produces hardware ``of comparable quality to a commercial high-level synthesis tool''~\cite{canis11_legup}. -\paragraph{Choice of benchmarks.} We evaluate \vericert{} using the PolyBench/C benchmark suite\footnote{\url{http://web.cs.ucla.edu/~pouchet/software/polybench/}} and some CHStone benchmarks\footnote{\url{http://www.ertl.jp/chstone/}}. \JW{I would make these footnotes into proper cites, because there is no limit on how long the bibliography can be.} PolyBench/C is a modern benchmark suite that has been previously used to evaluate HLS tools~\cite{choi+18}. For completeness, we use the full set of 24 benchmarks. We set the benchmark parameters so that all datatypes are integers (since \vericert{} only supports integers) and all datasets are `small' (to fit into the small on-chip memories). A current limitation of \vericert{}, as discussed in Section~\ref{?}, is that it does not support addition and subtraction operations involving integer literals not divisible by 4. To work around this, we lightly modified each benchmark program so that literals other than multiples of 4 are stored into variables before being added or subtracted. \JW{Any other notable changes to the benchmarks?} +\paragraph{Choice of benchmarks.} We evaluate \vericert{} using the PolyBench/C benchmark suite~\cite{polybench}. PolyBench/C is a modern benchmark suite that has been previously used to evaluate HLS tools~\cite{choi+18}. For completeness, we use the full set of 24 benchmarks. We set the benchmark parameters so that all datatypes are integers (since \vericert{} only supports integers) and all datasets are `small' (to fit into the small on-chip memories). A current limitation of \vericert{}, as discussed in Section~\ref{?}, is that it does not support addition and subtraction operations involving integer literals not divisible by 4. To work around this, we lightly modified each benchmark program so that literals other than multiples of 4 are stored into variables before being added or subtracted. \JW{Any other notable changes to the benchmarks?} \paragraph{Experimental setup.} In order to generate a hardware implementation, the Verilog produced by the HLS tool-under-test must be synthesised to a netlist; the resultant netlist can then be placed-and-routed for a particular FPGA device. We use Intel Quartus~\cite{quartus} for both of these tasks, and we target an Arria 10 FPGA. \begin{table*} \begin{tabular}{lcccccccccccc} \toprule - Bench & \multicolumn{2}{c}{Cycles} & \multicolumn{2}{c}{Frequency / MHz} & \multicolumn{2}{c}{LUTs} & \multicolumn{2}{c}{Reg} & \multicolumn{2}{c}{BRAMs} & \multicolumn{2}{c}{DSPs}\\ + \textbf{Benchmark} & \multicolumn{2}{c}{\bf Cycles} & \multicolumn{2}{c}{\bf Frequency / MHz} & \multicolumn{2}{c}{\bf LUTs} & \multicolumn{2}{c}{\bf Registers} & \multicolumn{2}{c}{\bf Block RAMs} & \multicolumn{2}{c}{\bf DSPs}\\ & L & V & L & V & L & V & L & V & L & V & L & V\\ \midrule adpcm & 30241 & 121386 & 90.05 & 66.3 & 7719 & 51626 & 12034 & 42688 & 7 & 0 & 0 & 48\\ @@ -38,7 +38,7 @@ Our evaluation is designed to answer the following three research questions. \JW mips & 7754 & 18482 & 98.95 & 78.43 & 5049 & 10617 & 4185 & 7690 & 0 & 0 & 0 & 0\\ \bottomrule \end{tabular} - \caption{CHStone programs synthesised in \legup{} 5.1 and with \vericert{}}\label{tab:chstone} + \caption{CHStone programs synthesised with \legup{} 5.1 (L) and with \vericert{} (V) \JW{I guess this table is for the chop?}}\label{tab:chstone} \end{table*} \subsection{RQ1: How fast is \vericert{}-generated hardware?} |