When we have a stable design it is time to start the regression testing. We have to put together a test suite containing all the testcases we would like to use for our functional simulation. When we run all these testcases we will produce a lot of test result log files. We need a system to handle the whole regression testing and the Mongoose Simulation Environment can help you.
Regression Testing in Mongoose
Mongoose uses test sequences written in the Mongoose Script Language (MSL) to perform regression testing. A test sequence is a collection of testcases that are executed in sequence. You can control when to start a new testcase and where to send it. It can run as a background job on your own host or it can be sent to a batch queue manager (LSF) and run on a dedicated simulation host. This flow diagram shows the process of generating a test sequence, running a test sequence, analyzing the result and display the result in a web browser and/or send an email or SMS with a summary of all test results.
Here is an example of a test sequence file. You can mix unix commands and Mongoose script commands. Lines starting with UC: are a unix commands and lines starting with MC: are Mongoose commands. You can write this file by hand or you can use the Test Sequence Generator to automatically generate one from the testcases you selected.
//$$HEADER /*************************************************************************/ // Module: ETC_TEST // Design: ETC // Written by: Sven-Ake Andersson ZooCad Consulting // Description: Test sequence file used for regression runs /*************************************************************************/
UC:echo 'Start test sequence' MC:DisplayTime MC:ClearAllCounters MC:ClearFileNameAddOns MC:AddTestCountToFileName MC:AddTagNameToFileName MC:SaveTickerInfo MC:StartTicker MC:SetTestCounter 1 MC:SetTagName ETC MC:SetReportFormatShort MC:SelectLogFile On MC:SetReleaseName today // Start of test sequence UC:echo 'Testcase running : AllInstructionsExternalExcl.tc' 1:AllInstructionsExternalExcl.tc MC:WaitSeconds 20 MC:WaitForSimulationToFinish 100 UC:echo 'Testcase running : BypassExternalExcl.tc' 1:BypassExternalExcl.tc MC:WaitSeconds 20 MC:WaitForSimulationToFinish 100 // Add more testcases here // .................. MC:GenerateErrorReport MC:DisplayHtmlFile // Stop routine UC:echo 'End of test sequence' MC:SendErrorCount MC:SendSMS 0706420380 MC:SendEmail zoodesign@comhem.se MC:DisplayTime MC:EndOfTestSequence
This is the email and SMS message sender window.
After running our testcases we have all simulation log files saved in the result/printout directory:
Let's use the Log File Analyzer/Test Report Generator to generate a condensed test report file.
The Test Report Generator will search all log files for important information and put it into the report file. The two errors reported are from testcases testing a function not implemented and can be ignored. We are finally ready for implementing the design into the FPGA.
