pyEDAA.Reports.Unittesting.pyTooling

Reader for pyTooling’s XML test report format.

pyTooling can write a test report (pytest option --pytooling-xml) in a format of its own, which carries two things JUnit XML can’t express: test suites nest, and every test suite and test case can carry a title, a summary and a description besides its name.

The format’s version - a FormatVersion - is the version of its XML schema, which a report names in its xsi:noNamespaceSchemaLocation attribute (e.g. TestReport-v0.1.xsd). A report is validated against that schema and converted into the unified test entity hierarchy of pyEDAA.Reports.Unittesting.

Example

from pathlib import Path
from pyEDAA.Reports.Unittesting.pyTooling import Document

document = Document(Path("report/unit/TestReport.xml"), analyzeAndConvert=True)
for testcase in document.IterateTestcases():
  print(f"{testcase.Name}: {testcase.Status.name} - {testcase.Title}")

Variables

Classes

  • FormatVersion: Version of pyTooling’s test report format, as the XML schema a report names states it.

  • Document: A document reader for pyTooling’s XML test report format.

Variables

pyEDAA.Reports.Unittesting.pyTooling.XML_SCHEMA_INSTANCE_NAMESPACE

Namespace of the xsi:* attributes.

'http://www.w3.org/2001/XMLSchema-instance'
pyEDAA.Reports.Unittesting.pyTooling.SCHEMA_FILES

Supported schema files (as named by xsi:noNamespaceSchemaLocation) and the format version they define.

{'TestReport-v0.1.xsd': <FormatVersion.Version0_1: '0.1'>}
pyEDAA.Reports.Unittesting.pyTooling.STATUS_MAP

Mapping of a <Testcase>’s status attribute to a test case status.

{'errored': <TestcaseStatus.Errored: 2048>,
 'expectedToFail': <TestcaseStatus.ExpectedFailed: 136>,
 'failed': <TestcaseStatus.Failed: 16>,
 'passed': <TestcaseStatus.Passed: 8>,
 'skipped': <TestcaseStatus.Skipped: 2>,
 'unexpectedlyPassed': <TestcaseStatus.UnexpectedPassed: 144>}

Classes

class pyEDAA.Reports.Unittesting.pyTooling.FormatVersion(*values)[source]

Version of pyTooling’s test report format, as the XML schema a report names states it.

Inheritance

Inheritance diagram of FormatVersion

Version0_1 = '0.1'

Format version 0.1, defined by TestReport-v0.1.xsd.

static _generate_next_value_(name, start, count, last_values)

Return the lower-cased version of the member name.

classmethod Parse(value)

Convert a string to the member carrying that value.

Parameters:

value (str | None) – The string to convert. None and the empty string mean no value was given.

Return type:

Self | None

Returns:

The member carrying that value. If no value was given: DEFAULT, if the enumeration declares one, otherwise None.

Raises:

  • TypeError – If parameter ‘value’ is not of type str.

  • ValueError – If no member of this enumeration carries that value.
    The note lists the values it accepts.

__str__()

Return str(self).

__format__(format_spec, /)

Return a formatted version of the string as described by format_spec.

__new__(value)
__repr__()

Return repr(self).

__new_member__(*values)

values must already be of type str

__hash__()

Return hash(self).

__lt__(value, /)

Return self<value.

__le__(value, /)

Return self<=value.

__eq__(value, /)

Return self==value.

__ne__(value, /)

Return self!=value.

__gt__(value, /)

Return self>value.

__ge__(value, /)

Return self>=value.

__iter__()

Implement iter(self).

__mod__(value, /)

Return self%value.

__rmod__(value, /)

Return value%self.

__len__()

Return len(self).

__getitem__(key, /)

Return self[key].

__add__(value, /)

Return self+value.

__mul__(value, /)

Return self*value.

__rmul__(value, /)

Return value*self.

__contains__(key, /)

Return bool(key in self).

encode(encoding='utf-8', errors='strict')

Encode the string using the codec registered for encoding.

encoding

The encoding in which to encode the string.

errors

The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

replace(old, new, /, count=-1)

Return a copy with all occurrences of substring old replaced by new.

count

Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If count is given, only the first count occurrences are replaced. If count is not specified or -1, then all occurrences are replaced.

split(sep=None, maxsplit=-1)

Return a list of the substrings in the string, using sep as the separator string.

sep

The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit

Maximum number of splits. -1 (the default value) means no limit.

Splitting starts at the front of the string and works to the end.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

rsplit(sep=None, maxsplit=-1)

Return a list of the substrings in the string, using sep as the separator string.

sep

The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit

Maximum number of splits. -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

join(iterable, /)

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

capitalize()

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold()

Return a version of the string suitable for caseless comparisons.

title()

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

center(width, fillchar=' ', /)

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count()

Return the number of non-overlapping occurrences of substring sub in string S[start:end].

Optional arguments start and end are interpreted as in slice notation.

expandtabs(tabsize=8)

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find()

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].

Optional arguments start and end are interpreted as in slice notation. Return -1 on failure.

partition(sep, /)

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

index()

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].

Optional arguments start and end are interpreted as in slice notation. Raises ValueError when the substring is not found.

ljust(width, fillchar=' ', /)

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower()

Return a copy of the string converted to lowercase.

lstrip(chars=None, /)

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

rfind()

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].

Optional arguments start and end are interpreted as in slice notation. Return -1 on failure.

rindex()

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].

Optional arguments start and end are interpreted as in slice notation. Raises ValueError when the substring is not found.

rjust(width, fillchar=' ', /)

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rstrip(chars=None, /)

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

rpartition(sep, /)

Partition the string into three parts using the given separator.

This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

splitlines(keepends=False)

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

strip(chars=None, /)

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase()

Convert uppercase characters to lowercase and lowercase characters to uppercase.

translate(table, /)

Replace each character in the string using the given translation table.

table

Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.

upper()

Return a copy of the string converted to uppercase.

startswith()

Return True if the string starts with the specified prefix, False otherwise.

prefix

A string or a tuple of strings to try.

start

Optional start position. Default: start of the string.

end

Optional stop position. Default: end of the string.

endswith()

Return True if the string ends with the specified suffix, False otherwise.

suffix

A string or a tuple of strings to try.

start

Optional start position. Default: start of the string.

end

Optional stop position. Default: end of the string.

removeprefix(prefix, /)

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix(suffix, /)

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

isascii()

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

islower()

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isupper()

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

istitle()

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isspace()

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

isdecimal()

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit()

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isnumeric()

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isalpha()

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isalnum()

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isidentifier()

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

isprintable()

Return True if all characters in the string are printable, False otherwise.

A character is printable if repr() may use it in its output.

zfill(width, /)

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.

format(*args, **kwargs)

Return a formatted version of the string, using substitutions from args and kwargs. The substitutions are identified by braces (‘{’ and ‘}’).

format_map(mapping, /)

Return a formatted version of the string, using substitutions from mapping. The substitutions are identified by braces (‘{’ and ‘}’).

static maketrans()

Return a translation table usable for str.translate().

If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.

__sizeof__()

Return the size of the string in memory, in bytes.

__dir__()

Returns public methods and other interesting attributes.

__reduce_ex__(proto)

Helper for pickle.

class pyEDAA.Reports.Unittesting.pyTooling.Document(xmlReportFile, analyzeAndConvert=False)[source]

A document reader for pyTooling’s XML test report format.

The root element <TestReport> becomes the test suite summary, every nested <Testsuite> a test suite and every <Testcase> a test case. The optional <Title>, <Summary> and <Description> elements become the entities’ Title, Summary and Description. A test case’s <Message> (why it didn’t pass) becomes its Message, its nodeID attribute (the test runner’s identifier) the key-value pair "nodeID".

Inheritance

Inheritance diagram of Document

Parameters:

  • xmlReportFile (Path)

  • analyzeAndConvert (bool)

__init__(xmlReportFile, analyzeAndConvert=False)[source]

Initializes the document reader.

Parameters:

  • xmlReportFile (Path) – Path to the XML report file.

  • analyzeAndConvert (bool) – Optional, if true, analyze (parse and validate) the file and convert its content.

Return type:

None

_xmlDocument: _ElementTree | None

Parsed and validated XML document.

_schemaVersion: FormatVersion | None

Format version named by the report’s schema location.

property SchemaVersion: FormatVersion

Read-only property to access the report format’s version (_schemaVersion).

The version is named by the schema the report points at - e.g. 0.1 by TestReport-v0.1.xsd -, so it is known once the file was analyzed.

Returns:

The format version.

Raises:

UnittestError – If the file wasn’t analyzed yet.

Analyze()[source]

Analyze the XML file: parse it, determine the format version and validate it against the version’s XML schema.

Hint

The time spend for analysis will be made available via property AnalysisDuration.

Raises:
Return type:

None

Convert()[source]

Convert the parsed and validated XML data structure into a test entity hierarchy.

The test suite summary is named after the report file (without extension). Afterwards, the hierarchy is aggregated, so all counters and states are computed.

Hint

The time spend for model conversion will be made available via property ModelConversionDuration.

Raises:

UnittestError – If the XML file was not analyzed before.
Call ‘Document.Analyze()’ or create the document using ‘Document(path, analyzeAndConvert=True)’.

Return type:

None

_ConvertTestsuite(testsuiteElement, parent)[source]

Convert a <Testsuite> element and its children to a test suite.

Parameters:

  • testsuiteElement (_Element) – The XML element node representing a test suite.

  • parent (TestsuiteSummary | Testsuite) – The parent test entity.

Return type:

None

_ConvertTestcase(testcaseElement, parent)[source]

Convert a <Testcase> element and its children to a test case.

Parameters:

  • testcaseElement (_Element) – The XML element node representing a test case.

  • parent (Testsuite) – The parent test suite.

Return type:

None

_path: Path

Path to the YAML file.

_analysisDuration: timedelta | None

Duration of Analyze(), or None before it ran.

_modelConversion: timedelta | None

Duration of Convert(), or None before it ran.

AddTestsuite(testsuite)

Add a test suite to the list of test suites.

Parameters:

testsuite (TestsuiteType) – The test suite to add.

Raises:

  • ValueError – If parameter ‘testsuite’ is None.

  • TypeError – If parameter ‘testsuite’ is not a Testsuite.

  • AlreadyInTestsuiteError – If parameter ‘testsuite’ is already part of a test entity hierarchy.

  • DuplicateTestcaseError – If parameter ‘testsuite’ is already listed (by name) in the list of test suites.

Return type:

None

AddTestsuites(testsuites)

Add a list of test suites to the list of test suites.

Parameters:

testsuites (Iterable[TestsuiteType]) – List of test suites to add.

Raises:

  • ValueError – If parameter ‘testsuites’ is None.

  • TypeError – If parameter ‘testsuites’ is not iterable.

Return type:

None

Aggregate(strict=True)

Aggregate all test entities in the hierarchy.

Return type:

Tuple[int, int, int, int, int, int, int, int, int, int, int, int, int, int, timedelta]

Parameters:

strict (bool)

property AnalysisDuration: timedelta | None

Read-only property to access the analysis duration (_analysisDuration).

Note

This includes usually the duration to validate and parse the file format, but it excludes the time to convert the content to the test entity hierarchy.

Returns:

Duration to analyze the document, or None if it wasn’t analyzed yet.

property AssertionCount: int

Read-only property to return the number of all assertions in all test cases in the test entity hierarchy.

Returns:

Number of assertions in all test cases.

property Description: str | None

Read-only property to access the test entity’s description (_description).

Returns:

The test entity’s description, or None if it wasn’t recorded.

property ErrorCount: int

Read-only property to access the number of errors in this entity.

Returns:

Number of errors.

property Errored: int

Read-only property to access the number of tests with errors in the test suite hierarchy.

Returns:

Number of errored tests.

property Excluded: int

Read-only property to access the number of excluded tests in the test suite hierarchy.

Returns:

Number of excluded tests.

property ExpectedErrorCount: int

Read-only property to access the number of expected errors.

Returns:

Count of expected errors.

property ExpectedFatalCount: int

Read-only property to access the number of expected fatal errors.

Returns:

Count of expected fatal errors.

property ExpectedWarningCount: int

Read-only property to access the number of expected warnings.

Returns:

Count of expected warnings.

property Failed: int

Read-only property to access the number of failed tests in the test suite hierarchy.

Returns:

Number of failed tests.

property FailedAssertionCount: int

Read-only property to access the number of all failed assertions in all test cases in the test entity hierarchy.

Returns:

Number of failed assertions in all test cases.

property FatalCount: int

Read-only property to access the number of fatal errors in this entity.

Returns:

Number of fatal errors.

classmethod GetMethodsWithAttributes(predicate=None)

Return the class’ methods that carry at least one matching attribute.

Parameters:

predicate (type[TAttr] | Iterable[type[TAttr]] | None) – Optional, an attribute class, an iterable of attribute classes, or None to accept every attribute.

Return type:

dict[Callable[..., Any], tuple[Attribute, ...]]

Returns:

Dictionary of methods and the matching attributes attached to them.

Raises:

  • ValueError – If an element of parameter ‘predicate’ is not a sub-class of Attribute.

  • ValueError – If parameter ‘predicate’ is neither an attribute class nor an iterable of those.

property Inconsistent: int

Read-only property to access the number of inconsistent tests in the test suite hierarchy.

Returns:

Number of inconsistent tests.

property Kind: TestsuiteKind

Read-only property to access the kind of the test suite.

Test suites are used to group test cases. This grouping can be due to language/framework specifics like tests grouped by a module file or namespace. Others might be just logically grouped without any relation to a programming language construct.

Test summaries always return kind Root.

Returns:

Kind of the test suite.

property ModelConversionDuration: timedelta | None

Read-only property to access the conversion duration (_modelConversion).

Note

This includes usually the duration to convert the document’s content to the test entity hierarchy. It might also include the duration to (re-)aggregate all states and statistics in the hierarchy.

Returns:

Duration to convert the document, or None if it wasn’t converted yet.

property Name: str

Read-only property to access the test entity’s name.

Returns:

The test entities name.

property Parent: TestsuiteBase | None

Read-only property to access the reference to the parent test entity.

Returns:

Reference to the parent entity.

property Passed: int

Read-only property to access the number of passed tests in the test suite hierarchy.

Returns:

Number of passed tests.

property PassedAssertionCount: int

Read-only property to access the number of all passed assertions in all test cases in the test entity hierarchy.

Returns:

Number of passed assertions in all test cases.

property Path: Path

Read-only property to access the path to the file of this document.

Returns:

The document’s path to the file.

property SetupDuration: timedelta | None

Read-only property to access the duration of the test entity’s setup.

Returns:

Duration it took to set up the entity.

property Skipped: int

Read-only property to access the number of skipped tests in the test suite hierarchy.

Returns:

Number of skipped tests.

property StartTime: datetime | None

Read-only property to access the time when the test entity was started.

Returns:

Time when the test entity was started.

property Status: TestsuiteStatus

Read-only property to access the aggregated overall status of the test suite.

Returns:

Overall status of the test suite.

property Summary: str | None

Read-only property to access the test entity’s summary (_summary).

Returns:

The test entity’s summary, or None if it wasn’t recorded.

property TeardownDuration: timedelta | None

Read-only property to access the duration of the test entity’s teardown.

Returns:

Duration it took to tear down the entity.

property TestDuration: timedelta | None

Read-only property to access the duration of a test entities run.

This duration is excluding setup and teardown durations. In case setup and/or teardown durations are unknown or not distinguishable, assign setup and teardown durations with zero.

Returns:

Duration of the entity’s test run.

property TestcaseCount: int

Read-only property to return the number of all test cases in the test entity hierarchy.

Returns:

Number of test cases.

property Tests: int

Read-only property to access the number of tests in this entity.

Returns:

Number of tests.

property TestsuiteCount: int

Read-only property to return the number of all test suites in the test suite hierarchy.

Returns:

Number of test suites.

property Testsuites: Dict[str, TestsuiteType]

Read-only property to access a reference to the internal dictionary of test suites.

Returns:

Reference to the dictionary of test suite.

property Title: str | None

Read-only property to access the test entity’s title (_title).

The title is a short label written for a reader, while the name identifies the test entity.

Returns:

The test entity’s title, or None if it wasn’t recorded.

property TotalDuration: timedelta | None

Read-only property to access the total duration of a test entity run.

this duration includes setup and teardown durations.

Returns:

Total duration of the entity’s execution (setup + test + teardown)

property WarningCount: int

Read-only property to access the number of warnings in this entity.

Returns:

Number of warnings.

property Weak: int

Read-only property to access the number of weak tests in the test suite hierarchy.

Returns:

Number of weak tests.

_ConvertDuration(element)[source]

Convert the duration attribute (in seconds) of an XML element node.

Parameters:

element (_Element) – The XML element node with an optional duration attribute.

Return type:

timedelta | None

Returns:

The duration, or None if the attribute is absent.

classmethod __class_getitem__()

Parameterizes a generic class.

At least, parameterizing a generic class is the main thing this method does. For example, for some generic class Foo, this is called when we do Foo[int] - there, with cls=Foo and params=int.

However, note that this method is also called when defining generic classes in the first place with class Foo[T]: ….

__contains__(key)

Returns True, if a key-value pairs was annotated by this key.

Parameters:

key (str) – Name of the key-value pair.

Return type:

bool

Returns:

True, if the pair was annotated.

__delitem__(key)

Delete a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

None

__getitem__(key)

Access a key-value pair by key.

Parameters:

key (str) – Name if the key-value pair.

Return type:

Any

Returns:

Value of the accessed key.

__getstate__()

Return the object’s state for pickling, collecting every slot of the class hierarchy.

Return type:

dict[str, Any]

Returns:

Dictionary of slot names and their values.

Raises:

ExtendedTypeError – If a slot was never assigned, so it has no value to serialize.

classmethod __init_subclass__()

Function to initialize subclasses.

__iter__()

Iterate all annotated key-value pairs.

Return type:

Generator[Tuple[str, Any]]

Returns:

A generator of key-value pair tuples (key, value).

__len__()

Returns the number of annotated key-value pairs.

Return type:

int

Returns:

Number of annotated key-value pairs.

__new__(**kwargs)
__setitem__(key, value)

Set the value of a key-value pair by key.

If the pair doesn’t exist yet, it’s created.

Parameters:

  • key (str) – Key of the key-value pair.

  • value (Any) – Value of the key-value pair.

Return type:

None

__setstate__(state)

Restore the object’s state from unpickling, requiring exactly the slots of the class hierarchy.

Parameters:

state (dict[str, Any]) – Dictionary of slot names and their values.

Raises:

ExtendedTypeError – If the given state misses a slot or carries an unexpected one.

Return type:

None

__str__()

Formats the test entity as human-readable incl. some statistics.

Return type:

str

_description: str | None

Description of the test entity, e.g. its doc-string.

_summary: str | None

Summary of the test entity, e.g. the first paragraph of its doc-string.

_title: str | None

Short label of the test entity, written for a reader.

_ConvertTexts(element, entity)[source]

Convert the <Title>, <Summary> and <Description> child elements of an XML element node.

Parameters:

  • element (_Element) – The XML element node with optional <Title>, <Summary> and <Description> children.

  • entity (TestsuiteSummary | Testsuite | Testcase) – The test entity to update.

Return type:

None