The pyEDAA.Reports Documentation

A collection of various (EDA tool-specific) report data formats.

Main Goals

This package provides abstract data models and specific implementations for report formats. The supported report formats are commonly used for any programming language or have a specifc context with Electronic Design Automation (EDA) tools. Examples are unit test summaries (like Ant JUnit XML), code coverage (like Cobertura) and documentation coverage reports.

While the data models and file format implementations can be used as a library, a CLI program pyedaa-report will be provided too. It allows reading, converting, concatenating, merging, transforming and writing report files.

Roadmap

It’s also planned to support console outputs from simulators and synthesis/implementation tools to create structured logs and reports for filtering and data extraction.

Report Formats

🚧 Code Coverage

Code coverage measures used and unused code lines, statements, branches, etc. Depending on the programming language this is measured by instrumenting the code/binary and running the program, it’s test cases or simulating the code. In generate code coverage is a measure of test coverage. Unused code is not (yet) covered by tests.

The code coverage metric in percent is a ratio of used code versus all possibly usable code. A coverage of <100% indicates unused code. This can be dead code (unreachable) or untested code (⇒ needs more test cases).

Supported tools

• Coverage.py / pytest-cov

• Aldec Riviera-PRO

• others tbd. (GHDL with enabled coverage)

Supported file formats

• Cobertura

• others tbd. (gcov)

🚧 Documentation Coverage

Documentation coverage measures the presence of code documentation. It primarily counts for public language entities like publicly visible constants and variables, parameters, types, functions, methods, classes, modules, packages, etc. The documentation goal depends on the used coverage collection tool’s settings. E.g. usually, private language entities are not required to be documented.

The documentation coverage metric in percent is a ratio of documented language entity versus all documentation worthy langauge entities. A coverage of <100% indicates undocumented code.

Supported tools

• docstr_coverage

• others tbd. (GHDL)

Supported file formats

• tbd.

Unit Test Summaries

Results of (unit) tests (also regression tests) are collected in machine readable summary files. Test cases are usually grouped by one or more test suites. Besides the test’s result (passed, failed, skipped, …) also the test’s outputs and durations are collected. Results can be visualized as a expandable tree structure.

The total number of testcases indicates the spend effort in testing and applying many test vectors. In combination with code coverage, it can be judged if the code has untested sections.

Supported features

• Read Ant JUnit XML files (and various dialects)

• Merge Ant JUnit reports

• Concatenate Ant JUnit reports

• Transform the hierarchy of reports

• Write Ant JUnit reports (also to other dialects)

Supported file formats

• Ant JUnit4 XML format and various dialects

• JUnit5 XML format (Open Test Reporting)

• OSVVM YAML format

Supported tools

• CTest

• GoogleTest

• Ant + JUnit4

• JUnit5

• OSVVM

• pyTest

Consumers

This layer is used by:

• pyTooling/Actions → PublishTestResults

• 🚧 pyTooling/Sphinx-Reports

Contributors

• Patrick Lehmann (Maintainer)

• Unai Martinez-Corral

• and more…

Todo

add license texts here