Since v0.19.1, the package grew from 74 to 372 classes in 15 to 59 modules, and the testsuite from 161 to 1256
tests. v1.0.0 is the first stable release (development status Production/Stable). pyEDAA.Reports requires
pyTooling 10.1.
pyEDAA.Reports.CodeCoverage is a new code coverage data model in two layers: a model per report
format, converted to a common model.
The common model has a physical hierarchy - CoverageSummary (the
report’s root) → Directory →
File → Line →
Branch - and a logical one of language units -
Package, Module,
SourceFile, Class,
Function and Method -, each
spanning its file from StartLine to EndLine. Every element knows its Parent and the report’s
Root.
A line, a branch and a unit carry a LineCoverageStatus and, if the
report says, a coverage count; a branch names its target line.
A file keeps its lines in a list indexed by line number: GetLine(),
IterateLines(), LastLineNumber.
Directory.IterateFiles(),
CoverageSummary.IterateUnits() and
Unit.IterateUnits() take an optional predicate and
yield the elements in the order they were added.
CoverageSummary.Aggregate() computes the
counters of every file, directory and unit - lines, branches, partial lines, and the line, branch and combined
coverage.
Document is the abstract base of a report file, read in two steps
(Analyze(), Convert()) and converted by ToCoverageSummary(). A report that can’t be read, decoded
or parsed raises CodeCoverageError, chained to the cause.
A report format’s model is one module; a package holds a format’s dialects, a module each.
Readers for ten code coverage formats, each with a format model converted to the common model:
pyEDAA.Reports.CodeCoverage.Cobertura: the Cobertura XML format, after Cobertura’s DTD. A report is
validated against the new Any-Cobertura.xsd; Cobertura-04.xsd is the strict translation of
the DTD. CoveragePyCobertura reads coverage.py’s dialect
strictly, NVCCobertura NVC’s dialect
(nvc --cover-export --format=cobertura), each with a schema of its own.
pyEDAA.Reports.CodeCoverage.LCOV: lcov’s tracefile format, also written by llvm-cov, coverage.py and
GHDL - a strict line parser. Branches (BRDA) and MC/DC conditions (MCDC) become branches of their
line.
pyEDAA.Reports.CodeCoverage.CoveragePy: coverage.py’s JSON format (coverage json, formats 2 and
3). The stated format version chooses the JSON Schema, reverse-engineered from coverage.py.
pyEDAA.Reports.CodeCoverage.GHDL: GHDL’s own coverage JSON (ghdl -r --coverage, statement
coverage). MergedReport merges several simulation runs: a line ran
if it ran in any run.
pyEDAA.Reports.CodeCoverage.Gcov: the JSON report GCC’s gcov writes (gcov --json-format),
gzip-compressed or plain, with branches, calls and MC/DC conditions (gcc -fcondition-coverage). Each JSON
object is validated against the schema of its format version (GCC 9 to 13, GCC 14 and later).
pyEDAA.Reports.CodeCoverage.JaCoCo: JaCoCo’s XML report (jacococli.jar report --xml, Maven, Ant,
Gradle), validated against a translation of JaCoCo’s DTD.
pyEDAA.Reports.CodeCoverage.LLVM: LLVM’s JSON code coverage export (llvm-cov export -format=text,
from Clang, rustc or Swift), format versions 2.0.0 to 3.1.0. Line counts are derived from the segments as
llvm-cov show derives them.
pyEDAA.Reports.CodeCoverage.AldecUCDB: the XML Aldec’s acdb2xml exports from a coverage database
of Riviera-PRO or Active-HDL, with a schema reverse-engineered from Riviera-PRO reports. Only statement
coverage is converted. Transferred from pyEDAA.UCIS, contributed by Aldec.
pyEDAA.Reports.CodeCoverage.UCIS: the XML interchange format of the Accellera UCIS 1.0 standard.
Statements and branches are converted; toggle, condition, FSM, assertion and covergroup coverage are read into
the format’s model only. PyUCIS reads pyucis’ dialect.
pyEDAA.Reports.CodeCoverage.QuestaSim: the XML report Questa SIM writes from its coverage database
with vcover report -xml -details. A report without details or a functional coverage report raises
CodeCoverageError with a hint.
pyEDAA.Reports.CodeCoverage.Cobertura writes the common model - read from any format - as Cobertura XML:
Document.FromCoverageSummary(),
Generate(), Write(), valid according to Cobertura-04.xsd.
pyedaa-reports coverage is a new command: it reads a code coverage report in one of thirteen
formats (--input=[<format>:]<file>), shows its line and branch figures and writes it as Cobertura XML
(--output=[Cobertura:]<file>).
pyEDAA.Reports.DocumentationCoverage is a language-neutral documentation coverage model, filled by
a reader per language - Python for now.
ItemKind names an item’s language construct, e.g. Class,
Method, Entity, Port; each kind belongs to an
ItemCategory.
CoverageCounters counts total, expected, covered, uncovered,
ignored and excluded items and computes the coverage;
CoverageState has the new flag Excused.
The physical hierarchy - DocumentationReport →
Directory →
File - and the logical one of
Items, whose counters count the item and every item below
it.
SourceFileWarning is a critical warning for a source file that
can’t be read or parsed; it carries the report’s file
(File).
pyEDAA.Reports.DocumentationCoverage.Python.Analyzer reads a package’s Python source code and returns
its aggregated documentation report. It counts what docstr_coverage counts, plus class fields and module
variables documented in the forms Sphinx’ autodoc reads, and honours docstr_coverage’s inherited and
excused comments. CountedItems chooses fields,
variables, protected and private items; regular expressions exclude items. Files are read in the same order on
every platform.
pyEDAA.Reports.Requirements is a new data model for requirements management: a
RequirementSpecification with registries of persons, personas, labels,
documents and layers; Requirements with a
RequirementKind (after ISO/IEC/IEEE 29148), an
Obligation and an ItemState;
specification items in an N:M relation to the requirements; Links
between items; an item’s history, rationale, verification methods and acceptance criteria; and traces along the
links, transitive and safe in cycles.
pyEDAA.Reports.RiskManagement is a new data model of a risk register: a
Risk with probability, impact and state; implications and mitigations,
shared by risks; links between risks of a kind; a risk’s score and level, owner and next review.
pyEDAA.Reports.FunctionalCoverage is a new tool-neutral data model of functional coverage, after
SystemVerilog covergroups and the Accellera UCIS standard: a
FunctionalCoverageReport with
Covergroups and their
Coverpoints and
Crosses, each holding bins of a
BinKind with a count and an at-least count over
ValueRanges; a weight and a goal per element, as
SystemVerilog’s option.weight and option.goal or OSVVM’s CovWeight and Goal.
FunctionalCoverageReport.Aggregate()
computes the bin and hit counters and two ratios:
BinCoverage (SystemVerilog, UCIS) and
HitCoverage (OSVVM).
Document is the abstract base of a functional coverage format
reader, as in the code coverage model.
pyEDAA.Reports holds what the data models share: Person is a real person - user
ID, name, e-mail address.
The data models’ errors derive from pyTooling’s common bases DuplicateElementError,
AlreadyInHierarchyError and NotInSameHierarchyError. Each
model raises one error per duplicate element kind, e.g.
DuplicateFileError, and an error named after its root, e.g.
AlreadyInRiskRegisterError.
pyEDAA.Reports.Unittesting
A testcase keeps the message and details explaining its status and its captured standard output and standard
error: the new TestcaseOutputMixin with the read-only properties
Message, Details, StandardOutput and StandardError.
Every test entity carries a Title, a Summary and a Description, written for a reader; copies,
conversions and merges keep them.
pyEDAA.Reports.Unittesting.JUnit: the reader keeps the message attribute and the text of
<failure>, <error> and <skipped>, and the text of <system-out> and <system-err>, for every
dialect; the writers emit them again.
pyEDAA.Reports.Unittesting.JUnit and its dialects: a test case without a class (empty classname)
belongs to its Testsuite directly - new parameter testcases,
property Testcases, methods
AddTestcase() and
AddTestcases(). Every dialect reads, converts and writes it
back with classname="".
New JUnit dialects, each read, validated by a new strict schema, converted and written, and available in
pyedaa-reports unittest:
Catch2JUnit: Catch2’s JUnit reporter (v3), as Catch2-JUnit.
NextestJUnit: cargo-nextest’s report, as nextest-JUnit. A flaky
test is passed with a rerun count above zero.
GoJUnitReport: go-junit-report (v2), as GoJUnitReport-JUnit.
TestLoggerJUnit: the .NET test logger JunitXml.TestLogger
(dotnet test --logger junit), as TestLogger-JUnit.
pyEDAA.Reports.Unittesting.pyTooling reads the XML test report pyTooling’s pytest plugin writes
(--pytooling-xml) - nested test suites, the texts, the message and the runner’s node ID. The file is
validated against the schema pyTooling 10 ships.
pyEDAA.Reports.Unittesting.OpenTestReporting reads
Open Test Reporting 0.2.0, the XML format the JUnit
Platform (JUnit 5 and 6) writes: Events the event-based
format, Hierarchy the hierarchical one. Open Test
Reporting’s XSDs are bundled unmodified.
pyedaa-reports unittest: the format is optional. A file without format is read as Any-JUnit
(--input=[<format>:]<file>, --merge=[<format>:]<pattern>) and written as pyTest-JUnit
(--output=[<format>:]<file>). An unsupported format notes the supported ones and the default. A value may
hold a Windows drive letter.
⚠️ AnalysisDuration and ModelConversionDuration of a unit test document
(Document) are None before
Analyze() / Convert() ran,
instead of timedelta(seconds=-1). Test for None instead of a negative duration.
⚠️ pyEDAA.Reports requires pyTooling 10.1: pyTooling[terminal] >= 10.1.0, < 11.0. The JUnit dialects
merge their doc-strings with DocStringMergeStrategy.BaseLast, as pyTooling 10 replaced
InheritDocString’s merge.
⚠️ ReportException derives from pyTooling.Exceptions.ExceptionBase instead of the
builtin Exception. ex.args of any exception of this package is now empty; str(ex) is unchanged.
Read ex.message or str(ex) instead of ex.args[0].
⚠️ The exception classes carry the ***Error suffix. ***Exception is reserved for
ReportException, the package-wide base. No aliases are kept - rename at the use site.
The four dialect-specific variants in JUnit are renamed with them.
⚠️ parent is a keyword-only parameter: a positional parent raises TypeError. Pass parent=...
to the constructors of pyEDAA.Reports.Unittesting, pyEDAA.Reports.Unittesting.JUnit and
Base.
⚠️ pyEDAA.Reports.Unittesting: title, summary and description follow name in the
constructors of Testcase, Testsuite,
TestsuiteSummary and their base classes, so every positional argument
after name is bound three parameters off. Pass the parameters after name as keyword arguments.
⚠️ pyedaa-reports unittest matches format names case-sensitively: pytest-junit is rejected.
Write the names as listed (pyTest-JUnit, Any-JUnit, …).
⚠️ pyEDAA.Reports.DocumentationCoverage: Class, Module, Package, _Type and _Unit are
removed; Base is the base of the new model. Use
Item with an
ItemKind.
⚠️ pyEDAA.Reports.DocumentationCoverage.Python: Coverage, AggregatedCoverage,
ClassCoverage, ModuleCoverage and PackageCoverage are removed. Use
Item and its counters; a package’s own __init__.py
figures are its file’s counters.
⚠️ pyEDAA.Reports.DocumentationCoverage.Python: DocStrCoverage and DocStrCoverageError, the
wrapper of docstr_coverage, are removed. Use Analyzer,
which reads the source code itself.
⚠️ pyEDAA.Reports.CLI: Application.HeadLine is renamed to
Application.HEADLINE, as pyTooling 10 spells class constants
in upper case. Rename the attribute at every use; there is no alias.
⚠️ pyEDAA.Reports.CLI: Application.ISSUE_TRACKER_URL is removed, as pyTooling 10 removed
TerminalApplication.ISSUE_TRACKER_URL. Read pyEDAA.Reports.__issue_tracker_url__ instead.
⚠️ pyEDAA.Reports.DocumentationCoverage: the class-per-construct model is replaced by the
language-neutral model of Item,
DocumentationReport,
Directory and
File.
⚠️ pyEDAA.Reports.DocumentationCoverage.Python: DocStrCoverage and DocStrCoverageError are
removed, and with them the use of docstr_coverage: the documentation coverage is read by
Analyzer.
⚠️ ReportException - and with it every exception of this package - joins the pyTooling
exception hierarchy, so except ExceptionBase covers this package like it covers pyEDAA.CLITool,
pyEDAA.OSVVM, pyEDAA.OutputFilter and pyEDAA.ToolSetup.
⚠️ The exception names follow PEP 8 - “you should use the suffix Error on your exception names” - instead
of the package’s older ***Exception convention.
⚠️ pyedaa-reports unittest: the format names are matched case-sensitively, as coverage
matches them.
⚠️ pyEDAA.Reports.Unittesting: title, summary and description follow name in the
constructors of Testcase, Testsuite
and TestsuiteSummary; message, details, standardOutput and
standardError follow keyValuePairs in Testcase’s.
⚠️ parent is a keyword-only parameter in every constructor taking one. A positional parent raises
TypeError instead of landing in another parameter.
⚠️ Requirements: pyTooling[terminal] >= 10.1.0, < 11.0 instead of >= 9.0, < 10.0. The build
requires setuptools >= 84.2.1 and pyTooling >= 10.1.0, < 11.0.
⚠️ pyEDAA.Reports.CLI: Application.HeadLine is renamed to
Application.HEADLINE, as TerminalApplication.HEADLINE in
pyTooling 10; main() formats its error messages with the palette
Application.FOREGROUND.
⚠️ pyEDAA.Reports.CLI: Application.DUNDER_MODULE
is the package pyEDAA.Reports, whose __issue_tracker_url__ the exception printers read;
ISSUE_TRACKER_URL is gone, as pyTooling 10’s TerminalApplication expects.
pyEDAA.Reports.CLI: main() catches ReportException
and reports it. Before, every error of this package but UnittestException was announced as “an unknown or
unhandled exception” with an invitation to file a bug.
pyEDAA.Reports.Unittesting.JUnit: the reader reads the file, then parses its bytes, each step reporting
its own error, like the other readers. Before, every exception became Couldn’t open ‘…’.
pyEDAA.Reports.Unittesting.JUnit.Document takes name, startTime, duration, status and
testsuites as keyword parameters; FromTestsuiteSummary()
constructs the document from them and computes the counters; the dialects inherit it.
The generic JUnit reader ignores <flakyFailure>, <flakyError>, <rerunFailure> and <rerunError>
in a <testcase>, like <properties>; it raised Unknown element … in junit file. A dialect’s schema
decides whether they are allowed.
Testsuite.Aggregate() counts a testcase with status
ExpectedFailed as passed and UnexpectedPassed as failed instead of raising
UnittestError.
Enumerations, whose values are neither read from nor written to text, are plain Enums with
integer values instead of StringEnums; messages and representations name a member
by its name:
pyEDAA.Reports.CodeCoverage.CoveragePy: RegionKind.
pyEDAA.Reports.DocumentationCoverage: ItemCategory,
ItemKind.
pyEDAA.Reports.Requirements: ItemState,
RequirementKind, Obligation,
LinkKind, EventKind.
pyEDAA.Reports.RiskManagement: Probability,
Impact, RiskLevel,
RiskState, LinkKind,
MitigationKind,
MitigationState.
Every module with code carries from __future__ import annotations and writes forward references unquoted.
New requirement jsonschema ~= 4.26 for the JSON Schemas of JSON report formats. A JSON reader imports it in
Analyze(), before measuring the analysis duration: importing a reader or pyEDAA.Reports.CLI doesn’t
load it.
pyedaa-reports imports the readers and writers of a command when the command runs: help and
version load none of them, unittest no code coverage reader, coverage no JUnit dialect.
The test extra describes every test level: tests/requirements.txt includes
tests/app/requirements.txt.
Python 3.15 is supported and listed in the classifiers.
The report readers store a measured duration as a timedelta, as pyTooling 10’s
Stopwatch hands it out; AnalysisDuration and ModelConversionDuration
return it unchanged instead of converting seconds. ⚠️ A duration not measured yet is None instead of a
negative timedelta.
The data models - Requirements, RiskManagement,
FunctionalCoverage and Person - attach key-value pairs with
pyTooling’s KeyValuePairsMixin, accessed with the dictionary syntax; a key is any
hashable value.
Dependencies: wheel ~= 0.48.0, lxml >= 6.1.3, Coverage ~= 7.17, mypy ~= 2.4.
ruamel.yaml isn’t a requirement any more: no module imports it since the OSVVM YAML reader moved to
pyEDAA.OSVVM. The documentation requires it, for the overrides file of the dependency tables.
The documentation is rendered with the extension pyTooling.Sphinx instead of sphinx_reports. The
dependency tables are generated from the requirements files. New page Abbreviations.
Code Coverage describes the two layers, both hierarchies of the common model, the states and the counters, and
per format its model and its conversion. Report Formats is a table of the formats 19 coverage tools
write. A new page explains why pyEDAA.Reports reads UCIS XML itself.
New section Schemas: an overview and 26 schema pages, each with a download link, a validation command and
the source.
New toctree Data Models with the pages Functional Coverage, Requirements and Risk Management. Functional Coverage describes
weights and goals and both coverage ratios, with an example.
Documentation Coverage describes the language-neutral data model and the Python reader.
The JUnit data model has sections on the cargo-nextest, go-junit-report and JunitXml.TestLogger dialects, and on
the test case without a class, as pytest writes it for a skipped test module. New page on the Open Test
Reporting data model.
The CLI reference renders the commands and options of pyedaa-reports.
An optional parameter’s doc-string description opens with Optional,.
A :raises: field names one parameter, and the fields follow the parameter order of the signature.
The section labels of the dependency page follow the DEP/ hierarchy.