Scientific validation¶

These pages track the tests that pin PROTEUS physics against published benchmarks, analytical limits, and cross-implementation checks. Each page covers one source file and records the reference cited, the test ids that carry the reference_pinned marker, and the scope of the comparison. reference_pinned is a pytest marker for a test that compares the code with a published value, an analytical limit or a second implementation. A physics module's behavior is considered validated when every source file with a public physical API has at least one pinned test inventoried here.

A case with a plot shows the tested values against their reference in a light and a dark version. The test writes the plot with the same numbers it asserts, through validation_plot, a pytest fixture of the proteus-mpl package that draws and checks the plot of a test. This command writes all plots to docs/assets/validation/:

pytest -m reference_pinned tests --validation-plots

Without --validation-plots the tests write nothing, but each plot test compares the data of its plot with the committed pair and fails when that pair is missing or stale. The command makes every plot again; commit only the pair of the case you changed, because the bytes match only for the same matplotlib and Pillow versions. The physics inside a coupled module, such as Zalmoxis, is validated in the documentation of that module.

Pages by process¶

Adding a page¶

Create docs/Validation/<module>/<file>.md when the first reference_pinned test for that source file lands, add it to the table above and to the Validation tab in the nav of mkdocs.yml, and cite the benchmark (paper, figure, table), the analytical limit, or the cross-implementation setup the test compares against.

Adding a plot¶

Give the test the validation_plot fixture and call it after the asserts with a function that draws a new figure from the tested values, and a case name: validation_plot(figure, '<case>'). The fixture writes <case>.svg and <case>_dark.svg (kind='map' writes AVIF for maps), at most 60 kB per file. Commit both files and show them on the page with the #only-light and #only-dark suffixes, followed by the reference, the tolerance, the test id and the command.