Several runs in one state
A fact often rests on more than one command. Run each through memprove run without changing anything in between;
then one Checkpoint cites them all (--evidence r1 --evidence r2). run prints, on the line before its summary,
every run of this repository state you can cite (r1 (PASS), r2 (PASS)); labels belong to this context and start at
r1 with each begin; --evidence also takes an Evidence ID or last. The daemon seals into each new run’s
Snapshot the earlier runs of the same Task and Workspace when the repository state is the same (same commit, branch,
index, tracked file contents and status); nothing is carried when either state has an untracked file. The latest run
of a command in a state answers for that state: when the latest run of an argv in the same Task, Workspace and state
did not pass, every earlier passed run of it is overruled (unconfirmed, evidence_not_passed; run lists it as r1 (PASS, overruled by r2)), and a --fact citing it is refused; a test run is overruled by a later failed test run of
the same argv even when a command run of it passed in between. A commit, an edit of a tracked file, git add or a
branch switch starts a new state: the runs before it can no longer be cited, and --evidence r1 is refused with the
list of the runs you can cite now (exit 2, nothing sent).
A pytest suite is run as memprove run --test-format pytest -- <python> -m pytest -q. -q and addopts = "-q" in
pyproject.toml or pytest.ini add up to -qq, which prints no N passed summary, only a line of progress marks;
the count is then made from that line, mark by mark (., F and E are cases, F and E failures; s, x and
X are not cases), and the lines run prints say (counted from the progress line). If you can, take the -q out
of addopts, so that pytest prints its summary; --test-cases <N> still wins over either count, with a warning when
it differs. A last line that is neither is recorded without a count and does not pass, and the message says that
pytest printed no summary. A command run (run without --test or --test-format) cannot support a fact; when its
argv is the Task’s last passed test run, run says which arguments record it as a test run, and checkpoint create --fact refuses to cite it before any request.
From docs/guide.md at 2625064cbf0a · Markdown