1.10

General

• Rename the project to Igor Pro Universal Testing Framework
  (IUTF). This also renamed all procedure files. Although the old main
  procedure file unit-testing.ipf still exists, users are advised to switch to
  igortest.ipf. This change is fully backward compatible as all, now deprecated,
  names are kept until the next major release.
• Add a flag to allow shuffling the test suite and test case order,
  see here
• Fix the order of test case execution: IUTF now always executes the
  test cases from the same test suite together
• Add support for rerunning "flaky" tests on failure, see
  here
• Document behaviour of runtime errors and aborts in data generators
  and hooks
• Support test case and data generator names longer than 255 bytes
• Document how the user can abort a test run
• Add IUTF_RestoreTracing() for restoring the procedure files from before tracing
• Adopt reuse specification
• Add data generators as virtual test cases. This allows to use all
  test assertions inside data generators and failing test assertions
  are then attributed to the corresponding data generator. Before they
  were attributed to the very first test case being run.
• Also treat user hooks as virtual test cases.
• Support tracing with tests which save the Igor Pro experiment in a
  different folder
• Be stricter when checking for compatible reentry functions (Igor Pro
  6 only)
• Skip test case end user hook as well if the test case was skipped
• Skip test suite end hook when we did not call the test suite begin
  hook at all
• Require that test cases marked as UTF_EXPECTED_FAILURE have failing test assertions
• Fix UTF_FTAG_TAP_DIRECTIVE/UTF_FTAG_TAP_DESCRIPTION in Igor Pro 9 due to now illegal characters in
  dimension labels
• Completely revise Readme.md
• Copy all history output also to stdin/stderr (Igor Pro 8 and higher)
• Enhance the output on failed assertions to include the full test
  case name
• Add the test case name on failure for TAP output
• Allow creating TAP and JUNIT output simultaneously
• Introduce the test result wave from which TAP and JUNIT output is
  generated from
• Fixed code coverage instrumentation to track non-instrumented lines
  correctly
• Add support for outputting code coverage results into the standard
  Cobertura format
• Fix testcase discovery for multi-multi data test cases (Igor Pro 6
  only)
• Add testing on MacOSX Catalina for Igor Pro 8/9
• Ensure that the abort flag is always cleared on test begin
• Add test to showcase how tracing works in an independent module
• Fix call stack traversing in an independent module
• Add complexity calculation for cobertura output, see
  wikipedia and the original article page
  36
• Fixed a bug where when running IUTF in an independent module, the
  run function was searched in ProcGlobal instead of the
  independent module.
• Fixed a bug where generating the detailed message for failed wave
  equality assertions did not ignore the case for comparing dimension
  labels.

Test assertions

• Added assertions for checking runtime errors, see *_RTE, *_ANY_RTE and *_NO_RTE
• Changed parameter types of *_EQUAL_STR and *_NEQ_STR to be bass-by-value. This might break cases where a null
  string was passed. For null strings the test assertions
  *_NULL_STR and *_PROPER_STR should be used.
• Output messages from INFO() also on stderr and in JUNIT
• Added assertions for compilation testing, see *_COMPILATION and *_NO_COMPILATION

1.09

General

• Add Code Coverage determination, see here (IP9 Build 38812 or higher)
• Allow analytics of code coverage tracing data, see here
• Add support for multi data and multi-multi data test cases
• Add INFO() function to output more information on failed test assertions, see here
• Add support for test code running in background functions, see also here.
• Add support for checking for free/local wave leaks (IP9 Build 39622 or higher)
• Add UTF_SKIP tag
• Mark test cases with zero sized data generator waves as skipped
• Call data generator only once for MD/MMD test cases
• Enforce that we have at least one assertion in each test case
• Fix hitting the sprintf limit (IP 8 or lower)
• Abort flag does no longer cover runtime errors
• Execute the test cases from top to bottom in each test suite
• Test the basic parts of our testing framework using the very tiny test environment VTTE
• Add generic function to report wrapper results
• Reorganize code and split it into more files
• TestCaseEnd: Silently ignore non-killable working folder
• Output state messages to stdout (IP 8 or higher)
• Enhance output on failed test assertion in test cases
• Tighten the check for test case signatures
• AfterFileOpenHook: Make it more robust
• Execute the builtin hooks also for failing user test hooks
• Always clear runtime errors before AbortOnRTE
• .gitlab.ci.yml: Add CI
• Moved NULL_WAVE flag to major flags for wave comparison in documentation
• Output failure summary at the end
• Allow unsaved experiments in some cases
• Allow fixed log file naming
• New option debugMode for more fine-grained debug control

Test assertions

• *_CLOSE_VAR: Prevent singularity
• *_SMALL_VAR: Change tolerance so that 0 is considered small with zero tolerance
• Add assertions <, <=, >, >= for double arguments, see *_LT_VAR,
  *_LE_VAR, *_GT_VAR, and *_GE_VAR
• *_EQUAL_STR: Make case sensitive comparison the default
• Added assertions for Int64 and UInt64 variables, see *_EQUAL_INT64 and
  *_EQUAL_UINT64

*_WAVE

• Require valid types as arguments
• Make the output more human readable

*_EQUAL_WAVES and *_EQUAL_TEXTWAVES

• Allow to check matching sizes of waves of different types
• Allow identical wave references
• Compare zero sized waves properly with set mode
• Make error reporting for custom mode patterns better
• Output descriptive error messages in WAVE_DATA mode
• Handle invalid mode correctly
• Work around EqualWaves dimension labels bug with certain IP versions
• Complain for unknown modes
• Make modes wave signed thus allowing -1 to be passed in to check all modes

JUNIT output

• Improve accuracy of test case/suite durations
• Remove optional TestSuite attribute disabled
• Add a <failure> tag for each failed assertion
• Handle expected failure test cases as skipped
• Add JUNIT reference and updated JUNIT section in documentation
• Drop timezone information as required by the "standard"
• Nicify properties output

TAP output

• Handle TODO gracefully
• Now also holds skipped testcases

See here for the full changelog.

1.08

• EvaluateRTE: Avoid Igor crash due to wrong printf usage
• Avoid passing CHECK_WAVE(..., NULL_WAVE) assertion for existing wave. This required to change the value of NULL_WAVE.
• Documentation/Readme.md: minor style fixes

1.07

• Convert documentation to sphinx and extend it a lot!
• Allow selecting test cases and test suites using a regular expression as paramter to RunTest.
• Define a fixed order of the builtin hooks and the user hooks in which they are called. The begin user hooks are called after the builtin ones, the end user hooks before the builtin ones.
• Catch aborts in user hooks
• Lots of code cleanup
• Output more info on assertion failure
• Extend the wave checking assertions
• Handle manual aborts during execution better
• Add new run mode for automated execution
• JUNIT support: Make testcase classname unique for successive runs
• Add support for running the unit testing framework in an Independent Module
• JUNIT output: Avoid hitting sprintf string limit