summaryrefslogtreecommitdiffstats
path: root/eval.tex
diff options
context:
space:
mode:
authorYann Herklotz <git@yannherklotz.com>2020-09-08 16:56:33 +0100
committerYann Herklotz <git@yannherklotz.com>2020-09-08 16:56:33 +0100
commit3f121b82024a84d8ffdb2be9f1cb48118b03df0d (patch)
tree1cbb4c5c735624680e79c70f4038fb215b4c9374 /eval.tex
parent4a84076f7aaf131d8d6c767dffb9b1af919ab834 (diff)
downloadfccm21_esrhls-3f121b82024a84d8ffdb2be9f1cb48118b03df0d.tar.gz
fccm21_esrhls-3f121b82024a84d8ffdb2be9f1cb48118b03df0d.zip
Add evaluation to the top of the section
Diffstat (limited to 'eval.tex')
-rw-r--r--eval.tex6
1 files changed, 2 insertions, 4 deletions
diff --git a/eval.tex b/eval.tex
index db52a81..a18998c 100644
--- a/eval.tex
+++ b/eval.tex
@@ -1,4 +1,4 @@
-\section{Result}
+\section{Evaluation}
During the implementation stage, the testing system successfully detected bugs among all three tools. The test cases used were not the same for each tool, because, as being mentioned, each tool has its own supported syntax. Thus, to ensure the validness of test cases during the implementation stage, test cases were all generated based on the supported syntax of the targeted tools.
@@ -10,9 +10,7 @@ Moving on to the LegUp HLS version 4.0, the C/RTL unmatched result section is co
Finally, as for Intel HLS, three points need to be taken into account when analyzing the result for Intel HLS. Firstly, the hashing function used for Intel HLS is much simpler comparing to which is used for both Vivado HLS and LegUp HLS. One possible result of using simple hashing function is that a considerable amount of bug can go undetected, which can lower the total number of cases that have C and RTL results unmatched. Secondly, as Intel HLS is running under the Windows environment, it runs relatively slower comparing to other tools that run under the Linux system. Based on the result, a considerable amount of test runs was timed out, which ultimately decreases the total number of valid test runs. Therefore, similar to what has been mentioned about the reduced overall valid test cases for Vivado HLS, when analyzing the Intel HLS, this should also be taken into consideration. Lastly, as Intel HLS uses the I++ compiler instead of GCC, differences can exist. Although theoretically, the C++ compiler does have stricter requirements, it should be compatible with compiling C programs. And as the test programs were generated through Csmith in C language, words like “bool” or “class”, which are only supported in C++ but not in C, do not exist. Also, Csimth was forbidden to generate operations like malloc, which might trigger incompatibility between C and C++. Thus, although compatibility can pose a potential problem, it should not have a huge impact. Besides those three points, similar to Vivado HLS, Intel HLS will alert about using logical AND operation with constants, but it does not error out immediately. So the “invalid test cases” section is set to not applicable.
-\section{Evaluation}
-
-We generate $10$ thousand test programs to test on three different HLS tools, including three versions of Vivado HLS, one version of LegUp HLS and one version of Intel HLS.
+We generate $10$ thousand test programs to test on three different HLS tools, including three versions of Vivado HLS, one version of LegUp HLS and one version of Intel HLS.
We provide the same set of programs to all HLS tools and show that different tools give rise to unique bugs.