New coverage and profiling library

I’ve released a new library called instrument (markdown formatting is broken on haxelib, see GitHub - AlexHaxe/haxe-instrument: a profiling and coverage library for Haxe for a proper version).

with instrument you can:

  • print all function entries and exits with time spent as they happen
  • print a hierarchical overview of what function called which other function with counter and total time spent
  • print a summary (or export to a csv file) of all function calls with counter and total time spent
  • export call data to build a flame graph (experimental, requires d3-flame-graph and some manual setup)
  • print a detailed list of all types, fields, branches or expressions not covered
  • print a file by file summary of types, fields, branches and expressions covered
  • print a package summary of files, types, fields, branches and expressions covered
  • print an overall coverage summary
  • export coverage data as lcov.info to use in IDE extensions (see [Solved] Green tests are nice, but what about coverage? - #6 by cedx) or CI
  • export coverage data as codecov xml to submit it to codecov.io (results untested / unverified)
  • export coverage data as EMMA xml or JaCoCo xml (results untested / unverified)

sample use:

-lib instrument
--macro instrument.Instrumentation.profiling([include packages], [include folders], [exclude packages])
--macro instrument.Instrumentation.coverage([include packages], [include folders], [exclude packages])
-D coverage-console-summary-reporter
-D coverage-console-package-summary-reporter

replace “include packages”, “include folders” and “exclude packages” with actual values.
make sure you include your source folder if you want to measure coverage, otherwise you will only see coverage stats of types touched during run!

you might have to experiment on what packages to include / exclude to make it work on your code, e.g. having js and php targets in one code base will give you “You cannot access the js package while targeting X” messages, if you don’t exclude mixed target code (conditionals help too).

you can even measure coverage outside of a test framework, just instrument your code and run it.
confirmed to work with munit and utest testing libraries.

instrument library is inspired by mcover and coverme

instrument is slower than mcover for most targets (except larger PHP projects) and it detects more branches (maybe even too many in some situations - so branch coverage WIP)

19 Likes

I’ve started to replace mcover by instrument on my projects. They use utest as testing library, and generate a LCOV report for code coverage.

I see that instrument works better than mcover: methods without braces are correctly covered :partying_face: (more info: Methods using return expression (i.e. without braces) are not covered · Issue #53 · massiveinteractive/mcover · GitHub).

Thanks @ablum for this great library!

2 Likes