pyEDAA.OSVVM.TestsuiteSummary

Reader for OSVVM test report summary files in YAML format.

Exceptions

• UnittestException: Base-exception for all unit test related exceptions.

Classes

• OsvvmException: Undocumented.

• Testcase: A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.

• Testsuite: A testsuite is a mid-level element in the test entity hierarchy representing a group of tests.

• BuildInformation: Undocumented.

• Settings: Undocumented.

• TestsuiteSummary: A testsuite summary is the root element in the test entity hierarchy representing a summary of all test suites and cases.

• BuildSummaryDocument: A testsuite summary is the root element in the test entity hierarchy representing a summary of all test suites and cases.

---

Exceptions

exception pyEDAA.OSVVM.TestsuiteSummary.UnittestException

Base-exception for all unit test related exceptions.

Inheritance

---

Classes

class pyEDAA.OSVVM.TestsuiteSummary.OsvvmException

Inheritance

class pyEDAA.OSVVM.TestsuiteSummary.Testcase(name, startTime=None, setupDuration=None, testDuration=None, teardownDuration=None, totalDuration=None, status=<TestcaseStatus.Unknown: 0>, assertionCount=None, failedAssertionCount=None, passedAssertionCount=None, warningCount=0, errorCount=0, fatalCount=0, expectedWarningCount=0, expectedErrorCount=0, expectedFatalCount=0, keyValuePairs=None, parent=None)

A testcase is the leaf-entity in the test entity hierarchy representing an individual test run.

Test cases are grouped by test suites in the test entity hierarchy. The root of the hierarchy is a test summary.

Every test case has an overall status like unknown, skipped, failed or passed.

In addition to all features from its base-class, test cases provide additional statistics for passed and failed assertions (checks) as well as a sum thereof.

Inheritance

Parameters:

• name (str)

• startTime (datetime | None)

• setupDuration (timedelta | None)

• testDuration (timedelta | None)

• teardownDuration (timedelta | None)

• totalDuration (timedelta | None)

• status (TestcaseStatus)

• assertionCount (int | None)

• failedAssertionCount (int | None)

• passedAssertionCount (int | None)

• warningCount (int)

• errorCount (int)

• fatalCount (int)

• expectedWarningCount (int)

• expectedErrorCount (int)

• expectedFatalCount (int)

• keyValuePairs (Mapping[str, Any] | None)

• parent (Testsuite | None)

Aggregate(strict=True)

Aggregate all test entities in the hierarchy.

Return type:

Tuple[int, int, int, int, int, int, timedelta]

Returns:

Parameters:

strict (bool)

property AssertionCount: int

Read-only property returning the number of assertions (checks) in a test case.

Returns:

Number of assertions.

property ErrorCount: int

Read-only property returning the number of encountered errors.

Returns:

Count of encountered errors.

property ExpectedErrorCount: int

Read-only property returning the number of expected errors.

Returns:

Count of expected errors.

property ExpectedFatalCount: int

Read-only property returning the number of expected fatal errors.

Returns:

Count of expected fatal errors.

property ExpectedWarningCount: int

Read-only property returning the number of expected warnings.

Returns:

Count of expected warnings.

property FailedAssertionCount: int

Read-only property returning the number of failed assertions (failed checks) in a test case.

Returns:

Number of assertions.

property FatalCount: int

Read-only property returning the number of encountered fatal errors.

Returns:

Count of encountered fatal errors.

property Name: str

Read-only property returning the test entity’s name.

Returns:

property Parent: TestsuiteBase | None

Read-only property returning the reference to the parent test entity.

Returns:

Reference to the parent entity.

property PassedAssertionCount: int

Read-only property returning the number of passed assertions (successful checks) in a test case.

Returns:

Number of passed assertions.

property SetupDuration: timedelta | None

Read-only property returning the duration of the test entity’s setup.

Returns:

Duration it took to set up the entity.

property StartTime: datetime | None

Read-only property returning the time when the test entity was started.

Returns:

Time when the test entity was started.

property Status: TestcaseStatus

Read-only property returning the status of the test case.

Returns:

The test case’s status.

property TeardownDuration: timedelta | None

Read-only property returning the duration of the test entity’s teardown.

Returns:

Duration it took to tear down the entity.

property TestDuration: timedelta | None

Read-only property returning the duration of a test entities run.

This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not distinguishable, assign setup and teardown durations with zero.

Returns:

Duration of the entity’s test run.

property TotalDuration: timedelta | None

Read-only property returning the total duration of a test entity run.

this duration includes setup and teardown durations.

Returns:

Total duration of the entity’s execution (setup + test + teardown)

property WarningCount: int

Read-only property returning the number of encountered warnings.

Returns:

Count of encountered warnings.

__contains__(key)

Returns True, if a key-value pairs was annotated by this key.

Parameters:

key (str) – Name of the key-value pair.

Return type:

bool

Returns:

True, if the pair was annotated.

__delitem__(key)

Delete a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

None

__getitem__(key)

Access a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

Any

Returns:

Value of the accessed key.

__init__(name, startTime=None, setupDuration=None, testDuration=None, teardownDuration=None, totalDuration=None, status=<TestcaseStatus.Unknown: 0>, assertionCount=None, failedAssertionCount=None, passedAssertionCount=None, warningCount=0, errorCount=0, fatalCount=0, expectedWarningCount=0, expectedErrorCount=0, expectedFatalCount=0, keyValuePairs=None, parent=None)

Initializes the fields of a test case.

Parameters:

• name (str) – Name of the test entity.

• startTime (Optional[datetime]) – Time when the test entity was started.

• setupDuration (Optional[timedelta]) – Duration it took to set up the entity.

• testDuration (Optional[timedelta]) – Duration of the entity’s test run.

• teardownDuration (Optional[timedelta]) – Duration it took to tear down the entity.

• totalDuration (Optional[timedelta]) – Total duration of the entity’s execution (setup + test + teardown)

• status (TestcaseStatus) – Status of the test case.

• assertionCount (Optional[int]) – Number of assertions within the test.

• failedAssertionCount (Optional[int]) – Number of failed assertions within the test.

• passedAssertionCount (Optional[int]) – Number of passed assertions within the test.

• warningCount (int) – Count of encountered warnings.

• errorCount (int) – Count of encountered errors.

• fatalCount (int) – Count of encountered fatal errors.

• keyValuePairs (Optional[Mapping[str, Any]]) – Mapping of key-value pairs to initialize the test case.

• parent (Optional[Testsuite]) – Reference to the parent test suite.

• expectedWarningCount (int)

• expectedErrorCount (int)

• expectedFatalCount (int)

Raises:

• TypeError – If parameter ‘parent’ is not a Testsuite.

• ValueError – If parameter ‘assertionCount’ is not consistent.

__iter__()

Iterate all annotated key-value pairs.

Return type:

Generator[Tuple[str, Any], None, None]

Returns:

A generator of key-value pair tuples (key, value).

__len__()

Returns the number of annotated key-value pairs.

Return type:

int

Returns:

Number of annotated key-value pairs.

__new__(**kwargs)

__setitem__(key, value)

Set the value of a key-value pair by key.

If the pair doesn’t exist yet, it’s created.

Parameters:

• key (str) – Key of the key-value pair.

• value (Any) – Value of the key-value pair.

Return type:

None

__str__()

Formats the test case as human-readable incl. statistics.

f"<Testcase {}: {} - assert/pass/fail:{}/{}/{} - warn/error/fatal:{}/{}/{} - setup/test/teardown:{}/{}/{}>"

Return type:

str

Returns:

Human-readable summary of a test case object.

class pyEDAA.OSVVM.TestsuiteSummary.Testsuite(name, kind=TestsuiteKind.Logical, startTime=None, setupDuration=None, testDuration=None, teardownDuration=None, totalDuration=None, status=<TestsuiteStatus.Unknown: 0>, warningCount=0, errorCount=0, fatalCount=0, testsuites=None, testcases=None, keyValuePairs=None, parent=None)

A testsuite is a mid-level element in the test entity hierarchy representing a group of tests.

Test suites contain test cases and optionally other test suites. Test suites can be grouped by test suites to form a hierarchy of test entities. The root of the hierarchy is a test summary.

Inheritance

Parameters:

• name (str)

• kind (TestsuiteKind)

• startTime (datetime | None)

• setupDuration (timedelta | None)

• testDuration (timedelta | None)

• teardownDuration (timedelta | None)

• totalDuration (timedelta | None)

• status (TestsuiteStatus)

• warningCount (int)

• errorCount (int)

• fatalCount (int)

• testsuites (Iterable[TestsuiteType] | None)

• testcases (Iterable[Testcase] | None)

• keyValuePairs (Mapping[str, Any] | None)

• parent (TestsuiteType | None)

AddTestcase(testcase)

Add a test case to the list of test cases.

Parameters:

testcase (Testcase) – The test case to add.

Raises:

• ValueError – If parameter ‘testcase’ is None.

• TypeError – If parameter ‘testcase’ is not a Testcase.

• AlreadyInHierarchyException – If parameter ‘testcase’ is already part of a test entity hierarchy.

• DuplicateTestcaseException – If parameter ‘testcase’ is already listed (by name) in the list of test cases.

Return type:

None

AddTestcases(testcases)

Add a list of test cases to the list of test cases.

Parameters:

testcases (Iterable[Testcase]) – List of test cases to add.

Raises:

• ValueError – If parameter ‘testcases’ is None.

• TypeError – If parameter ‘testcases’ is not iterable.

Return type:

None

AddTestsuite(testsuite)

Add a test suite to the list of test suites.

Parameters:

testsuite (TypeVar(TestsuiteType, bound= Testsuite)) – The test suite to add.

Raises:

• ValueError – If parameter ‘testsuite’ is None.

• TypeError – If parameter ‘testsuite’ is not a Testsuite.

• AlreadyInHierarchyException – If parameter ‘testsuite’ is already part of a test entity hierarchy.

• DuplicateTestcaseException – If parameter ‘testsuite’ is already listed (by name) in the list of test suites.

Return type:

None

AddTestsuites(testsuites)

Add a list of test suites to the list of test suites.

Parameters:

testsuites (Iterable[TypeVar(TestsuiteType, bound= Testsuite)]) – List of test suites to add.

Raises:

• ValueError – If parameter ‘testsuites’ is None.

• TypeError – If parameter ‘testsuites’ is not iterable.

Return type:

None

Aggregate(strict=True)

Aggregate all test entities in the hierarchy.

Return type:

Tuple[int, int, int, int, int, int, int, int, int, int, int, int, int, int, timedelta]

Returns:

Parameters:

strict (bool)

property AssertionCount: int

Read-only property returning the number of all assertions in all test cases in the test entity hierarchy.

Returns:

Number of assertions in all test cases.

property ErrorCount: int

Read-only property returning the number of encountered errors.

Returns:

Count of encountered errors.

property Errored: int

Read-only property returning the number of tests with errors in the test suite hierarchy.

Returns:

Number of errored tests.

property Excluded: int

Read-only property returning the number of excluded tests in the test suite hierarchy.

Returns:

Number of excluded tests.

property ExpectedErrorCount: int

Read-only property returning the number of expected errors.

Returns:

Count of expected errors.

property ExpectedFatalCount: int

Read-only property returning the number of expected fatal errors.

Returns:

Count of expected fatal errors.

property ExpectedWarningCount: int

Read-only property returning the number of expected warnings.

Returns:

Count of expected warnings.

property Failed: int

Read-only property returning the number of failed tests in the test suite hierarchy.

Returns:

Number of failed tests.

property FailedAssertionCount: int

Read-only property returning the number of all failed assertions in all test cases in the test entity hierarchy.

Returns:

Number of failed assertions in all test cases.

property FatalCount: int

Read-only property returning the number of encountered fatal errors.

Returns:

Count of encountered fatal errors.

property Inconsistent: int

Read-only property returning the number of inconsistent tests in the test suite hierarchy.

Returns:

Number of inconsistent tests.

property Kind: TestsuiteKind

Read-only property returning the kind of the test suite.

Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming language construct.

Test summaries always return kind Root.

Returns:

Kind of the test suite.

property Name: str

Read-only property returning the test entity’s name.

Returns:

property Parent: TestsuiteBase | None

Read-only property returning the reference to the parent test entity.

Returns:

Reference to the parent entity.

property Passed: int

Read-only property returning the number of passed tests in the test suite hierarchy.

Returns:

Number of passed tests.

property PassedAssertionCount: int

Read-only property returning the number of all passed assertions in all test cases in the test entity hierarchy.

Returns:

Number of passed assertions in all test cases.

property SetupDuration: timedelta | None

Read-only property returning the duration of the test entity’s setup.

Returns:

Duration it took to set up the entity.

property Skipped: int

Read-only property returning the number of skipped tests in the test suite hierarchy.

Returns:

Number of skipped tests.

property StartTime: datetime | None

Read-only property returning the time when the test entity was started.

Returns:

Time when the test entity was started.

property Status: TestsuiteStatus

Read-only property returning the aggregated overall status of the test suite.

Returns:

Overall status of the test suite.

property TeardownDuration: timedelta | None

Read-only property returning the duration of the test entity’s teardown.

Returns:

Duration it took to tear down the entity.

property TestDuration: timedelta | None

Read-only property returning the duration of a test entities run.

This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not distinguishable, assign setup and teardown durations with zero.

Returns:

Duration of the entity’s test run.

property TestcaseCount: int

Read-only property returning the number of all test cases in the test entity hierarchy.

Returns:

Number of test cases.

property Testcases: Dict[str, Testcase]

Read-only property returning a reference to the internal dictionary of test cases.

Returns:

Reference to the dictionary of test cases.

property TestsuiteCount: int

Read-only property returning the number of all test suites in the test suite hierarchy.

Returns:

Number of test suites.

property Testsuites: Dict[str, TestsuiteType]

Read-only property returning a reference to the internal dictionary of test suites.

Returns:

Reference to the dictionary of test suite.

property TotalDuration: timedelta | None

Read-only property returning the total duration of a test entity run.

this duration includes setup and teardown durations.

Returns:

Total duration of the entity’s execution (setup + test + teardown)

property WarningCount: int

Read-only property returning the number of encountered warnings.

Returns:

Count of encountered warnings.

property Weak: int

Read-only property returning the number of weak tests in the test suite hierarchy.

Returns:

Number of weak tests.

classmethod __class_getitem__()

Parameterizes a generic class.

At least, parameterizing a generic class is the main thing this method does. For example, for some generic class Foo, this is called when we do Foo[int] - there, with cls=Foo and params=int.

However, note that this method is also called when defining generic classes in the first place with class Foo[T]: ….

__contains__(key)

Returns True, if a key-value pairs was annotated by this key.

Parameters:

key (str) – Name of the key-value pair.

Return type:

bool

Returns:

True, if the pair was annotated.

__delitem__(key)

Delete a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

None

__getitem__(key)

Access a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

Any

Returns:

Value of the accessed key.

__init__(name, kind=TestsuiteKind.Logical, startTime=None, setupDuration=None, testDuration=None, teardownDuration=None, totalDuration=None, status=<TestsuiteStatus.Unknown: 0>, warningCount=0, errorCount=0, fatalCount=0, testsuites=None, testcases=None, keyValuePairs=None, parent=None)

Initializes the fields of a test suite.

Parameters:

• name (str) – Name of the test suite.

• kind (TestsuiteKind) – Kind of the test suite.

• startTime (Optional[datetime]) – Time when the test suite was started.

• setupDuration (Optional[timedelta]) – Duration it took to set up the test suite.

• testDuration (Optional[timedelta]) – Duration of all tests listed in the test suite.

• teardownDuration (Optional[timedelta]) – Duration it took to tear down the test suite.

• totalDuration (Optional[timedelta]) – Total duration of the entity’s execution (setup + test + teardown)

• status (TestsuiteStatus) – Overall status of the test suite.

• warningCount (int) – Count of encountered warnings incl. warnings from sub-elements.

• errorCount (int) – Count of encountered errors incl. errors from sub-elements.

• fatalCount (int) – Count of encountered fatal errors incl. fatal errors from sub-elements.

• testsuites (Optional[Iterable[TypeVar(TestsuiteType, bound= Testsuite)]]) – List of test suites to initialize the test suite with.

• testcases (Optional[Iterable[Testcase]]) – List of test cases to initialize the test suite with.

• keyValuePairs (Optional[Mapping[str, Any]]) – Mapping of key-value pairs to initialize the test suite with.

• parent (Optional[TypeVar(TestsuiteType, bound= Testsuite)]) – Reference to the parent test entity.

Raises:

• TypeError – If parameter ‘testcases’ is not iterable.

• TypeError – If element in parameter ‘testcases’ is not a Testcase.

• AlreadyInHierarchyException – If a test case in parameter ‘testcases’ is already part of a test entity hierarchy.

• DuplicateTestcaseException – If a test case in parameter ‘testcases’ is already listed (by name) in the list of test cases.

classmethod __init_subclass__()

Function to initialize subclasses.

__iter__()

Iterate all annotated key-value pairs.

Return type:

Generator[Tuple[str, Any], None, None]

Returns:

A generator of key-value pair tuples (key, value).

__len__()

Returns the number of annotated key-value pairs.

Return type:

int

Returns:

Number of annotated key-value pairs.

__new__(**kwargs)

__setitem__(key, value)

Set the value of a key-value pair by key.

If the pair doesn’t exist yet, it’s created.

Parameters:

• key (str) – Key of the key-value pair.

• value (Any) – Value of the key-value pair.

Return type:

None

__str__()

Formats the test entity as human-readable incl. some statistics.

Return type:

str

class pyEDAA.OSVVM.TestsuiteSummary.BuildInformation

Inheritance

__init__()

Return type:

None

class pyEDAA.OSVVM.TestsuiteSummary.Settings

Inheritance

__init__()

Return type:

None

class pyEDAA.OSVVM.TestsuiteSummary.TestsuiteSummary(name, startTime=None, setupDuration=None, testDuration=None, teardownDuration=None, totalDuration=None, status=<TestsuiteStatus.Unknown: 0>, warningCount=0, errorCount=0, fatalCount=0, testsuites=None, keyValuePairs=None, parent=None)

A testsuite summary is the root element in the test entity hierarchy representing a summary of all test suites and cases.

The testsuite summary contains test suites, which in turn can contain test suites and test cases.

Inheritance

Parameters:

• name (str)

• startTime (datetime | None)

• setupDuration (timedelta | None)

• testDuration (timedelta | None)

• teardownDuration (timedelta | None)

• totalDuration (timedelta | None)

• status (TestsuiteStatus)

• warningCount (int)

• errorCount (int)

• fatalCount (int)

• testsuites (Iterable[TestsuiteType] | None)

• keyValuePairs (Mapping[str, Any] | None)

• parent (TestsuiteType | None)

__init__(name, startTime=None, setupDuration=None, testDuration=None, teardownDuration=None, totalDuration=None, status=<TestsuiteStatus.Unknown: 0>, warningCount=0, errorCount=0, fatalCount=0, testsuites=None, keyValuePairs=None, parent=None)

Initializes the fields of a test summary.

Parameters:

• name (str) – Name of the test summary.

• startTime (Optional[datetime]) – Time when the test summary was started.

• setupDuration (Optional[timedelta]) – Duration it took to set up the test summary.

• testDuration (Optional[timedelta]) – Duration of all tests listed in the test summary.

• teardownDuration (Optional[timedelta]) – Duration it took to tear down the test summary.

• totalDuration (Optional[timedelta]) – Total duration of the entity’s execution (setup + test + teardown)

• status (TestsuiteStatus) – Overall status of the test summary.

• warningCount (int) – Count of encountered warnings incl. warnings from sub-elements.

• errorCount (int) – Count of encountered errors incl. errors from sub-elements.

• fatalCount (int) – Count of encountered fatal errors incl. fatal errors from sub-elements.

• testsuites (Optional[Iterable[TypeVar(TestsuiteType, bound= Testsuite)]]) – List of test suites to initialize the test summary with.

• keyValuePairs (Optional[Mapping[str, Any]]) – Mapping of key-value pairs to initialize the test summary with.

• parent (Optional[TypeVar(TestsuiteType, bound= Testsuite)]) – Reference to the parent test summary.

Return type:

None

AddTestsuite(testsuite)

Add a test suite to the list of test suites.

Parameters:

testsuite (TypeVar(TestsuiteType, bound= Testsuite)) – The test suite to add.

Raises:

• ValueError – If parameter ‘testsuite’ is None.

• TypeError – If parameter ‘testsuite’ is not a Testsuite.

• AlreadyInHierarchyException – If parameter ‘testsuite’ is already part of a test entity hierarchy.

• DuplicateTestcaseException – If parameter ‘testsuite’ is already listed (by name) in the list of test suites.

Return type:

None

AddTestsuites(testsuites)

Add a list of test suites to the list of test suites.

Parameters:

testsuites (Iterable[TypeVar(TestsuiteType, bound= Testsuite)]) – List of test suites to add.

Raises:

• ValueError – If parameter ‘testsuites’ is None.

• TypeError – If parameter ‘testsuites’ is not iterable.

Return type:

None

Aggregate(strict=True)

Aggregate all test entities in the hierarchy.

Return type:

Tuple[int, int, int, int, int, int, int, int, int, int, int, int, int, int, timedelta]

Returns:

Parameters:

strict (bool)

property AssertionCount: int

Read-only property returning the number of all assertions in all test cases in the test entity hierarchy.

Returns:

Number of assertions in all test cases.

property ErrorCount: int

Read-only property returning the number of encountered errors.

Returns:

Count of encountered errors.

property Errored: int

Read-only property returning the number of tests with errors in the test suite hierarchy.

Returns:

Number of errored tests.

property Excluded: int

Read-only property returning the number of excluded tests in the test suite hierarchy.

Returns:

Number of excluded tests.

property ExpectedErrorCount: int

Read-only property returning the number of expected errors.

Returns:

Count of expected errors.

property ExpectedFatalCount: int

Read-only property returning the number of expected fatal errors.

Returns:

Count of expected fatal errors.

property ExpectedWarningCount: int

Read-only property returning the number of expected warnings.

Returns:

Count of expected warnings.

property Failed: int

Read-only property returning the number of failed tests in the test suite hierarchy.

Returns:

Number of failed tests.

property FailedAssertionCount: int

Read-only property returning the number of all failed assertions in all test cases in the test entity hierarchy.

Returns:

Number of failed assertions in all test cases.

property FatalCount: int

Read-only property returning the number of encountered fatal errors.

Returns:

Count of encountered fatal errors.

property Inconsistent: int

Read-only property returning the number of inconsistent tests in the test suite hierarchy.

Returns:

Number of inconsistent tests.

property Kind: TestsuiteKind

Read-only property returning the kind of the test suite.

Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming language construct.

Test summaries always return kind Root.

Returns:

Kind of the test suite.

property Name: str

Read-only property returning the test entity’s name.

Returns:

property Parent: TestsuiteBase | None

Read-only property returning the reference to the parent test entity.

Returns:

Reference to the parent entity.

property Passed: int

Read-only property returning the number of passed tests in the test suite hierarchy.

Returns:

Number of passed tests.

property PassedAssertionCount: int

Read-only property returning the number of all passed assertions in all test cases in the test entity hierarchy.

Returns:

Number of passed assertions in all test cases.

property SetupDuration: timedelta | None

Read-only property returning the duration of the test entity’s setup.

Returns:

Duration it took to set up the entity.

property Skipped: int

Read-only property returning the number of skipped tests in the test suite hierarchy.

Returns:

Number of skipped tests.

property StartTime: datetime | None

Read-only property returning the time when the test entity was started.

Returns:

Time when the test entity was started.

property Status: TestsuiteStatus

Read-only property returning the aggregated overall status of the test suite.

Returns:

Overall status of the test suite.

property TeardownDuration: timedelta | None

Read-only property returning the duration of the test entity’s teardown.

Returns:

Duration it took to tear down the entity.

property TestDuration: timedelta | None

Read-only property returning the duration of a test entities run.

This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not distinguishable, assign setup and teardown durations with zero.

Returns:

Duration of the entity’s test run.

property TestcaseCount: int

Read-only property returning the number of all test cases in the test entity hierarchy.

Returns:

Number of test cases.

property TestsuiteCount: int

Read-only property returning the number of all test suites in the test suite hierarchy.

Returns:

Number of test suites.

property Testsuites: Dict[str, TestsuiteType]

Read-only property returning a reference to the internal dictionary of test suites.

Returns:

Reference to the dictionary of test suite.

property TotalDuration: timedelta | None

Read-only property returning the total duration of a test entity run.

this duration includes setup and teardown durations.

Returns:

Total duration of the entity’s execution (setup + test + teardown)

property WarningCount: int

Read-only property returning the number of encountered warnings.

Returns:

Count of encountered warnings.

property Weak: int

Read-only property returning the number of weak tests in the test suite hierarchy.

Returns:

Number of weak tests.

classmethod __class_getitem__()

Parameterizes a generic class.

At least, parameterizing a generic class is the main thing this method does. For example, for some generic class Foo, this is called when we do Foo[int] - there, with cls=Foo and params=int.

However, note that this method is also called when defining generic classes in the first place with class Foo[T]: ….

__contains__(key)

Returns True, if a key-value pairs was annotated by this key.

Parameters:

key (str) – Name of the key-value pair.

Return type:

bool

Returns:

True, if the pair was annotated.

__delitem__(key)

Delete a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

None

__getitem__(key)

Access a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

Any

Returns:

Value of the accessed key.

classmethod __init_subclass__()

Function to initialize subclasses.

__iter__()

Iterate all annotated key-value pairs.

Return type:

Generator[Tuple[str, Any], None, None]

Returns:

A generator of key-value pair tuples (key, value).

__len__()

Returns the number of annotated key-value pairs.

Return type:

int

Returns:

Number of annotated key-value pairs.

__new__(**kwargs)

__setitem__(key, value)

Set the value of a key-value pair by key.

If the pair doesn’t exist yet, it’s created.

Parameters:

• key (str) – Key of the key-value pair.

• value (Any) – Value of the key-value pair.

Return type:

None

__str__()

Formats the test entity as human-readable incl. some statistics.

Return type:

str

class pyEDAA.OSVVM.TestsuiteSummary.BuildSummaryDocument(yamlReportFile, analyzeAndConvert=False)

Inheritance

Parameters:

• yamlReportFile (Path)

• analyzeAndConvert (bool)

__init__(yamlReportFile, analyzeAndConvert=False)

Initializes the fields of a test summary.

Parameters:

• name – Name of the test summary.

• startTime – Time when the test summary was started.

• setupDuration – Duration it took to set up the test summary.

• testDuration – Duration of all tests listed in the test summary.

• teardownDuration – Duration it took to tear down the test summary.

• totalDuration – Total duration of the entity’s execution (setup + test + teardown)

• status – Overall status of the test summary.

• warningCount – Count of encountered warnings incl. warnings from sub-elements.

• errorCount – Count of encountered errors incl. errors from sub-elements.

• fatalCount – Count of encountered fatal errors incl. fatal errors from sub-elements.

• testsuites – List of test suites to initialize the test summary with.

• keyValuePairs – Mapping of key-value pairs to initialize the test summary with.

• parent – Reference to the parent test summary.

• yamlReportFile (Path)

• analyzeAndConvert (bool)

Return type:

None

Analyze()

Analyze the YAML file, parse the content into an YAML data structure.

Hint

The time spend for analysis will be made available via property AnalysisDuration..

Return type:

None

Convert()

Convert the parsed YAML data structure into a test entity hierarchy.

This method converts the root element.

Hint

The time spend for model conversion will be made available via property ModelConversionDuration.

Raises:

UnittestException – If XML was not read and parsed before.

Return type:

None

__contains__(key)

Returns True, if a key-value pairs was annotated by this key.

Parameters:

key (str) – Name of the key-value pair.

Return type:

bool

Returns:

True, if the pair was annotated.

__iter__()

Iterate all annotated key-value pairs.

Return type:

Iterator[Testsuite]

Returns:

A generator of key-value pair tuples (key, value).

_analysisDuration: float

TODO: replace by Timer; should be timedelta?

_modelConversion: float

TODO: replace by Timer; should be timedelta?

AddTestsuite(testsuite)

Add a test suite to the list of test suites.

Parameters:

testsuite (TypeVar(TestsuiteType, bound= Testsuite)) – The test suite to add.

Raises:

• ValueError – If parameter ‘testsuite’ is None.

• TypeError – If parameter ‘testsuite’ is not a Testsuite.

• AlreadyInHierarchyException – If parameter ‘testsuite’ is already part of a test entity hierarchy.

• DuplicateTestcaseException – If parameter ‘testsuite’ is already listed (by name) in the list of test suites.

Return type:

None

AddTestsuites(testsuites)

Add a list of test suites to the list of test suites.

Parameters:

testsuites (Iterable[TypeVar(TestsuiteType, bound= Testsuite)]) – List of test suites to add.

Raises:

• ValueError – If parameter ‘testsuites’ is None.

• TypeError – If parameter ‘testsuites’ is not iterable.

Return type:

None

Aggregate(strict=True)

Aggregate all test entities in the hierarchy.

Return type:

Tuple[int, int, int, int, int, int, int, int, int, int, int, int, int, int, timedelta]

Returns:

Parameters:

strict (bool)

property AnalysisDuration: timedelta

Read-only property returning analysis duration.

Note

This includes usually the duration to validate and parse the file format, but it excludes the time to convert the content to the test entity hierarchy.

Returns:

Duration to analyze the document.

property AssertionCount: int

Read-only property returning the number of all assertions in all test cases in the test entity hierarchy.

Returns:

Number of assertions in all test cases.

property ErrorCount: int

Read-only property returning the number of encountered errors.

Returns:

Count of encountered errors.

property Errored: int

Read-only property returning the number of tests with errors in the test suite hierarchy.

Returns:

Number of errored tests.

property Excluded: int

Read-only property returning the number of excluded tests in the test suite hierarchy.

Returns:

Number of excluded tests.

property ExpectedErrorCount: int

Read-only property returning the number of expected errors.

Returns:

Count of expected errors.

property ExpectedFatalCount: int

Read-only property returning the number of expected fatal errors.

Returns:

Count of expected fatal errors.

property ExpectedWarningCount: int

Read-only property returning the number of expected warnings.

Returns:

Count of expected warnings.

property Failed: int

Read-only property returning the number of failed tests in the test suite hierarchy.

Returns:

Number of failed tests.

property FailedAssertionCount: int

Read-only property returning the number of all failed assertions in all test cases in the test entity hierarchy.

Returns:

Number of failed assertions in all test cases.

property FatalCount: int

Read-only property returning the number of encountered fatal errors.

Returns:

Count of encountered fatal errors.

property Inconsistent: int

Read-only property returning the number of inconsistent tests in the test suite hierarchy.

Returns:

Number of inconsistent tests.

property Kind: TestsuiteKind

Read-only property returning the kind of the test suite.

Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming language construct.

Test summaries always return kind Root.

Returns:

Kind of the test suite.

property ModelConversionDuration: timedelta

Read-only property returning conversion duration.

Note

This includes usually the duration to convert the document’s content to the test entity hierarchy. It might also include the duration to (re-)aggregate all states and statistics in the hierarchy.

Returns:

Duration to convert the document.

property Name: str

Read-only property returning the test entity’s name.

Returns:

property Parent: TestsuiteBase | None

Read-only property returning the reference to the parent test entity.

Returns:

Reference to the parent entity.

property Passed: int

Read-only property returning the number of passed tests in the test suite hierarchy.

Returns:

Number of passed tests.

property PassedAssertionCount: int

Read-only property returning the number of all passed assertions in all test cases in the test entity hierarchy.

Returns:

Number of passed assertions in all test cases.

property Path: Path

Read-only property to access the path to the file of this document.

Returns:

The document’s path to the file.

property SetupDuration: timedelta | None

Read-only property returning the duration of the test entity’s setup.

Returns:

Duration it took to set up the entity.

property Skipped: int

Read-only property returning the number of skipped tests in the test suite hierarchy.

Returns:

Number of skipped tests.

property StartTime: datetime | None

Read-only property returning the time when the test entity was started.

Returns:

Time when the test entity was started.

property Status: TestsuiteStatus

Read-only property returning the aggregated overall status of the test suite.

Returns:

Overall status of the test suite.

property TeardownDuration: timedelta | None

Read-only property returning the duration of the test entity’s teardown.

Returns:

Duration it took to tear down the entity.

property TestDuration: timedelta | None

Read-only property returning the duration of a test entities run.

This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not distinguishable, assign setup and teardown durations with zero.

Returns:

Duration of the entity’s test run.

property TestcaseCount: int

Read-only property returning the number of all test cases in the test entity hierarchy.

Returns:

Number of test cases.

property TestsuiteCount: int

Read-only property returning the number of all test suites in the test suite hierarchy.

Returns:

Number of test suites.

property Testsuites: Dict[str, TestsuiteType]

Read-only property returning a reference to the internal dictionary of test suites.

Returns:

Reference to the dictionary of test suite.

property TotalDuration: timedelta | None

Read-only property returning the total duration of a test entity run.

this duration includes setup and teardown durations.

Returns:

Total duration of the entity’s execution (setup + test + teardown)

property WarningCount: int

Read-only property returning the number of encountered warnings.

Returns:

Count of encountered warnings.

property Weak: int

Read-only property returning the number of weak tests in the test suite hierarchy.

Returns:

Number of weak tests.

classmethod __class_getitem__()

Parameterizes a generic class.

At least, parameterizing a generic class is the main thing this method does. For example, for some generic class Foo, this is called when we do Foo[int] - there, with cls=Foo and params=int.

However, note that this method is also called when defining generic classes in the first place with class Foo[T]: ….

__delitem__(key)

Delete a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

None

__getitem__(key)

Access a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

Testsuite

Returns:

Value of the accessed key.

classmethod __init_subclass__()

Function to initialize subclasses.

__new__(**kwargs)

__setitem__(key, value)

Set the value of a key-value pair by key.

If the pair doesn’t exist yet, it’s created.

Parameters:

• key (str) – Key of the key-value pair.

• value (Any) – Value of the key-value pair.

Return type:

None

__str__()

Formats the test entity as human-readable incl. some statistics.

Return type:

str

__len__()

Returns the number of annotated key-value pairs.

Return type:

int

Returns:

Number of annotated key-value pairs.