Skip to content

Experiment

experiment

Experiment checks — the FORM of an experiment folder, never its substance.

axm-init grades the SHAPE a scaffolded experiment must carry: the five working directories and the two root files. The substance (manifest validity, input hashing, DAG coherence, freeze anteriority, metrics) is the job of a dedicated tool in another package — duplicating it here would be an architecture regression, so no check in this module ever reads the CONTENT of manifest.yaml.

check_experiment_files(project)

Check: manifest.yaml and README.md exist at the experiment root.

Existence only — the manifest CONTENT is never parsed here, so a freshly scaffolded experiment whose manifest still holds TODO placeholders passes this form check.

Parameters:

Name Type Description Default
project Path

Experiment root directory.

 required

Returns:

Type Description
CheckResult

A failed CheckResult naming exactly the missing file(s), or a
CheckResult

passed one.
Source code in packages/axm-init/src/axm_init/checks/experiment.py
Python
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
def check_experiment_files(project: Path) -> CheckResult:
    """Check: ``manifest.yaml`` and ``README.md`` exist at the experiment root.

    Existence only — the manifest CONTENT is never parsed here, so a freshly
    scaffolded experiment whose manifest still holds TODO placeholders
    passes this form check.

    Args:
        project: Experiment root directory.

    Returns:
        A failed ``CheckResult`` naming exactly the missing file(s), or a
        passed one.
    """
    present = {name for name in _REQUIRED_FILES if (project / name).is_file()}
    missing = _missing_entries(_REQUIRED_FILES, present)
    if missing:
        listed = ", ".join(missing)
        return CheckResult(
            name="experiment.experiment_files",
            category="experiment",
            passed=False,
            weight=5,
            message=f"Experiment root missing {len(missing)} file(s): {listed}",
            details=[f"Missing: {listed}"],
            fix="Create manifest.yaml and README.md at the experiment root.",
        )
    return CheckResult(
        name="experiment.experiment_files",
        category="experiment",
        passed=True,
        weight=5,
        message="Experiment root carries manifest.yaml and README.md",
        details=[],
        fix="",
    )

check_experiment_structure(project)

Check: the experiment folder carries the five working directories.

Parameters:

Name Type Description Default
project Path

Experiment root directory.

 required

Returns:

Type Description
CheckResult

A failed CheckResult naming exactly the missing directories, or
CheckResult

a passed one.
Source code in packages/axm-init/src/axm_init/checks/experiment.py
Python
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def check_experiment_structure(project: Path) -> CheckResult:
    """Check: the experiment folder carries the five working directories.

    Args:
        project: Experiment root directory.

    Returns:
        A failed ``CheckResult`` naming exactly the missing directories, or
        a passed one.
    """
    present = {name for name in _REQUIRED_DIRS if (project / name).is_dir()}
    missing = _missing_entries(_REQUIRED_DIRS, present)
    if missing:
        listed = ", ".join(f"{name}/" for name in missing)
        return CheckResult(
            name="experiment.experiment_structure",
            category="experiment",
            passed=False,
            weight=5,
            message=(
                f"Experiment layout missing {len(missing)} directory(ies): {listed}"
            ),
            details=[f"Missing: {listed}"],
            fix=(
                "Create inputs/, scripts/, outputs/, analysis/ and figures/ "
                "at the experiment root."
            ),
        )
    return CheckResult(
        name="experiment.experiment_structure",
        category="experiment",
        passed=True,
        weight=5,
        message=(
            "Experiment layout complete "
            "(inputs/, scripts/, outputs/, analysis/, figures/)"
        ),
        details=[],
        fix="",
    )