Skip to content

Testing

testing

Node testing rule — the Vitest pendant of :class:TestCoverageRule.

Emits QUALITY_TESTS (the shared cross-framework test-suite-health rule) via Vitest's JSON reporter.

False-green guard (from the research): a green vitest run proves nothing if zero tests ran (passWithNoTests). We assert numTotalTests > 0 AND all passed; an empty suite fails this rule.

NodeTestRule

Bases: NodeToolRule

Run Vitest and require a non-empty, fully-passing suite.

Mirrors the Python testing invariant: a green run with zero tests is NOT a pass (guards against passWithNoTests). Score is the pass ratio times 100, forced to 0 when no tests ran.

Source code in packages/axm-audit/src/axm_audit/core/rules/node/testing.py
Python
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
@register_rule("testing", framework=Framework.NODE)
class NodeTestRule(NodeToolRule):
    """Run Vitest and require a non-empty, fully-passing suite.

    Mirrors the Python testing invariant: a green run with zero tests is NOT a
    pass (guards against ``passWithNoTests``). Score is the pass ratio times
    100, forced to 0 when no tests ran.
    """

    binary = "vitest"
    install_hint = "Install Vitest: npm install -D vitest"

    @property
    def rule_id(self) -> str:
        """Unique identifier (shared cross-framework: test-suite health)."""
        return "QUALITY_TESTS"

    @property
    def args(self) -> list[str]:
        """Run the suite once with the JSON reporter (no watch)."""
        return ["run", "--reporter=json"]

    @property
    def findings_returncodes(self) -> frozenset[int]:
        """Vitest exits 1 when tests fail — a finding we score, not a crash."""
        return frozenset({1})

    def score_output(self, parsed: object, project_path: Path) -> CheckResult:
        """Score by pass ratio; an empty suite is a hard fail (false-green guard)."""
        total, passed_count, failed = _test_counts(parsed)
        if total == 0:
            return CheckResult(
                rule_id=self.rule_id,
                passed=False,
                message="No tests ran (numTotalTests == 0)",
                severity=Severity.ERROR,
                score=0,
                details={"total": 0},
                fix_hint="Add tests — an empty suite never passes this gate.",
            )
        score = round(passed_count / total * 100)
        passed = failed == 0
        return CheckResult(
            rule_id=self.rule_id,
            passed=passed,
            message=f"Tests: {passed_count}/{total} passed",
            severity=Severity.ERROR if not passed else Severity.INFO,
            score=score,
            details={"total": total, "passed": passed_count, "failed": failed},
            fix_hint="Fix the failing tests above" if failed else None,
        )
args property

Run the suite once with the JSON reporter (no watch).

findings_returncodes property

Vitest exits 1 when tests fail — a finding we score, not a crash.

rule_id property

Unique identifier (shared cross-framework: test-suite health).

score_output(parsed, project_path)

Score by pass ratio; an empty suite is a hard fail (false-green guard).

Source code in packages/axm-audit/src/axm_audit/core/rules/node/testing.py
Python
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
def score_output(self, parsed: object, project_path: Path) -> CheckResult:
    """Score by pass ratio; an empty suite is a hard fail (false-green guard)."""
    total, passed_count, failed = _test_counts(parsed)
    if total == 0:
        return CheckResult(
            rule_id=self.rule_id,
            passed=False,
            message="No tests ran (numTotalTests == 0)",
            severity=Severity.ERROR,
            score=0,
            details={"total": 0},
            fix_hint="Add tests — an empty suite never passes this gate.",
        )
    score = round(passed_count / total * 100)
    passed = failed == 0
    return CheckResult(
        rule_id=self.rule_id,
        passed=passed,
        message=f"Tests: {passed_count}/{total} passed",
        severity=Severity.ERROR if not passed else Severity.INFO,
        score=score,
        details={"total": total, "passed": passed_count, "failed": failed},
        fix_hint="Fix the failing tests above" if failed else None,
    )