Test runner
test_runner
Agent-optimized test runner with structured output.
Runs the target's pytest with a reporting plugin the runner ships itself
(:mod:axm_audit._report_plugin, standard library + pytest hooks only), loaded
through -p with its directory on the subprocess PYTHONPATH. Any
interpreter that has pytest therefore works, and nothing is installed in it.
Results are compact and token-efficient for AI coding agents.
FailureDetail
dataclass
Structured detail for a single test failure.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
error_type
instance-attribute
Exception class name, e.g. AssertionError.
file
instance-attribute
Relative file path.
line
instance-attribute
Line number of the assertion / raising statement.
message
instance-attribute
One-line error message.
test
instance-attribute
Full node ID, e.g. tests/unit/test_x.py::TestFoo::test_bar.
traceback
instance-attribute
Bounded failure excerpt: head and tail of the failure text.
The first _FULL_EXCERPT_FAILURES failures of a run carry the full
excerpt (see :func:_failure_excerpt); later ones only their last
_MAX_TB_LINES lines, so a run with hundreds of reds stays compact.
NonTestCauseDetail
dataclass
Serializable projection of a classified non-test cause.
TestReport is a dataclass whose payload is produced by
dataclasses.asdict: a pydantic model attached to it would survive that
traversal as an object and break every JSON consumer of the report. The
classifier remains the single source of the classification itself; this
only mirrors its verdict in the report's own payload shape.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
from_cause(cause)
classmethod
Return the projection of cause, or None when there is none.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
TestCase
dataclass
Lossless verdict for one collected pytest item.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
TestReport
dataclass
Compact test execution report.
All fields use None rather than empty containers when no data
exists so that dataclasses.asdict produces a minimal payload.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
coverage_unavailable = None
class-attribute
instance-attribute
Why coverage was requested but could not be measured, else None.
__post_init__()
Populate additive execution metadata for directly-built reports.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
model_dump()
record_execution(*, return_code, collected, target_statuses)
Attach pytest evidence and compute the report's canonical verdict.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
build_pytest_cmd(*, report_path, coverage_path, files, markers, stop_on_first, collect_only=False, project_path=None, coverage_available=True)
Build the pytest command line.
The structured result is written by the runner's own plugin
(:data:PLUGIN_MODULE, loaded with -p; :func:run_pytest puts it on
the PYTHONPATH). coverage_available=False (the target has no
pytest-cov) drops every --cov* / --no-cov option, all of which
pytest would reject with exit 4.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
build_test_report(*, report_data, total_cov, per_file_cov, mode=None, last_coverage=None, include_cases=False, return_code=0, stdout='', stderr='', allow_collection_errors=False)
Build a TestReport from the plugin's JSON and coverage data.
Always parses failures and populates coverage — no mode branching.
The captured stdout/stderr and return_code are optional. When
they are supplied, the cause of a red that no test failure accounts for is
classified and attached to non_test_cause — explanatory only, it never
influences the verdict, and it stays None for a green run or a run red
by test failures.
Returns None for failures and coverage_by_file when no
data exists.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
parse_collector_errors(collectors)
Extract FailureDetail items from the report's collectors list.
Collector errors occur before test discovery completes (e.g.
SyntaxError in a test file, broken imports).
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
parse_coverage(coverage_path)
Parse coverage JSON into total % and per-file dict.
Files whose basename equals __main__.py are excluded from the
per-file map (they typically contain only a python -m entry
point and are not meaningfully unit-testable). The aggregate
total_pct from pytest-cov is left untouched, in line with
coverage.py's exclude_also convention of filtering reports
rather than rewriting the underlying totals.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
parse_failures(tests)
Extract FailureDetail items from the report's tests list.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
parse_json_report(report_path)
Read and parse the JSON file written by the runner's pytest plugin.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
probe_runner_environment(project_path)
Probe what the interpreter chosen for project_path gives pytest.
Raises:
| Type | Description |
|---|---|
PytestEnvironmentError
|
no usable pytest in that interpreter. |
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
| Python | |
|---|---|
read_test_report(project_path, *, report_path, coverage_path, return_code, files=None, include_cases=False, stdout='', stderr='', allow_collection_errors=False)
Read runner-owned pytest artifacts through the canonical report builders.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
run_pytest(cmd, project_path, *, timeout=_COVERAGE_RUN_TIMEOUT, capture_output=True, text=True, check=False)
Run tests or collection with the runner's own reporting plugin.
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
run_tests(project_path, *, mode='failures', files=None, markers=None, stop_on_first=True, include_cases=False)
Run tests with agent-optimized structured output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
project_path
|
Path
|
Root of the project to test. |
required |
mode
|
str
|
Accepted for backward compatibility but ignored — all modes now produce the same unified output (failures + coverage). |
'failures'
|
files
|
list[str] | None
|
Specific test files or paths to run. |
None
|
markers
|
list[str] | None
|
Pytest markers to filter ( |
None
|
stop_on_first
|
bool
|
Stop on first failure ( |
True
|
Returns:
| Type | Description |
|---|---|
TestReport
|
Structured |
Source code in packages/axm-audit/src/axm_audit/core/test_runner.py
| Python | |
|---|---|
615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 | |