Parameter grids (ensembles)¶

It is often useful to run grids of forward models, where each point in a grid represents a different set of parameters. This is done with the proteus grid command:

proteus grid -c input/example.grid.toml

Configure a grid by creating a TOML file that specifies the grid's axes and how it should be run. The example at input/example.grid.toml uses the dummy configuration file as a reference and modifies it for every combination of the parameters declared in the .grid.toml file. See the configuration guide for how to edit configuration files, and the parameter grid tutorial for a worked end-to-end example.

Defining grid axes¶

A grid configuration file (*.grid.toml) contains two kinds of entries: top-level settings that control the whole ensemble, and one TOML table per parameter axis.

The top-level settings are:

use_slurm and jax_cache take only a TOML true or false; a quoted "false", a 0 or a 1 stops the grid at load with an error that names the setting and the grid file.

Each parameter axis is a TOML table whose name is the dotted path of the config field to vary. For example, ["planet.mass_tot"] sweeps config.planet.mass_tot, and ["outgas.fO2_shift_IW"] sweeps the mantle redox offset. Any field documented in input/all_options.toml (or the configuration reference) can serve as an axis. Every top-level key must be accounted for: it is either one of the settings above or the path of one configuration field, given as a table with one of the methods below and the keys that method requires. Any other key, a path that names a whole section, an unquoted dotted header such as [planet.mass_tot] (TOML reads it as the section planet), an axis table with an unknown method or a missing required key, or a direct axis whose values is not a non-empty list of numbers and booleans or of strings (an empty list, a NaN, a single value without brackets, a nested array or table) stops the grid at load, before an earlier output folder is removed, with an error that names the key and the grid file. The grid manager then builds every case config and reads it back with the config loader before it removes an earlier output folder of the same name; a value the configuration refuses (an unknown module, a value out of its range, a quoted boolean) stops the grid there with the case index, the key and the value. This check runs in a hidden folder output/.grid-check-* that the grid removes when the check ends; a check that is killed (for example at a time limit) can leave that folder behind, and it is safe to remove.

Each axis declares a method that controls how its values are generated:

The following example sweeps planet mass and hydrogen inventory:

ref_config = "input/dummy.toml"
output     = "auto"
symlink    = ""
use_slurm  = false
max_jobs   = 10
max_days   = 1
max_mem    = 12
jax_cache  = false

# Planet mass [M_earth]: four explicit values
["planet.mass_tot"]
    method = "direct"
    values = [0.7, 1.0, 2.0, 3.0]

# Hydrogen inventory [ppmw]: 5000, 10000, 15000, 20000
["planet.elements.H_budget"]
    method = "arange"
    start  = 5000
    stop   = 20000
    step   = 5000

PROTEUS runs the Cartesian product of all axes, so this example produces 16 cases (the four masses combined with the four hydrogen budgets). Each additional axis multiplies the case count by its own length, so adding a third axis with three values would yield 48 cases. The total number of cases must stay below 1,000,000, which is the limit imposed by the case_NNNNNN folder-naming scheme.

Some parameters are only meaningful when a matching mode is set in the reference config, because the mode selects how the value is interpreted. For example, sweeping planet.elements.H_budget requires an H_mode such as "ppmw" in the base config, and planet.elements.S_budget requires the corresponding S_mode (for example "S/H"). Set the mode once in ref_config, then vary the budget along a grid axis. The planet and volatiles reference lists which fields depend on which modes.

Dispatching the grid¶

Grids can be dispatched with or without using a workload manager. In PROTEUS, we use the Slurm workload manager, which can allow running large ensembles of models on high-performance compute clusters. The subsections below detail cases with/without Slurm.

Before committing compute to a large grid, validate it with a dry run, which generates the grid and writes every per-case config file without launching any simulations:

proteus grid -c input/example.grid.toml --dry-run

Without Slurm¶

Firstly, set use_slurm = false. In this case, the GridPROTEUS routine will manage the individual subprocesses which compose the grid. The variable max_jobs specifies the maximum number of CPU cores which should be utilised by the grid at any one time. This is limited by the number of CPU cores available on your machine. This method works without Slurm, and can be applied on servers or on multicore personal computers.

In this case, you will need to make sure that PROTEUS stays open in order to manage its subprocesses.

With Slurm¶

Alternatively, you can access high performance compute nodes through the Slurm workload manager (e.g. on Habrok and Snellius). This is a two-step process. To do this, set use_slurm = true in your grid's configuration file. Then set max_mem and max_days to specify how much memory should be allocated to each job (each simulation). The shipped example uses 12 GB and 1 day. Ensure that these values are within the limits of the server you are working on.

With these options enabled, running PROTEUS will produce a script called slurm_dispatch.sh in the specified output folder, as well as write the required configuration files to a subfolder called cfgs/.

To dispatch your grid via Slurm, you must then run the command sbatch <path> where <path> is the path to the dispatch script created by the proteus grid command. You will be prompted to do this in the terminal.

Monitor your running jobs with squeue -u $USER. To cancel all of your running jobs, use scancel -u $USER. The original PROTEUS process does not need to stay open when using Slurm to manage the subprocesses.

Set jax_cache = true to add a shared JAX compilation cache to the dispatch script. Each job then exports JAX_COMPILATION_CACHE_DIR pointing at a jax_cache/ subdirectory of the grid output, so array tasks reuse each other's compiled kernels instead of recompiling the same interior solver per task. The cache is bounded at 80 GiB (JAX_COMPILATION_CACHE_MAX_SIZE), with a 1 s minimum compile time and a 4 KiB minimum entry size so only worthwhile kernels are stored. These exports are written into the Slurm dispatch script only; a local grid run (use_slurm = false) spawns subprocesses without them.

The cluster guides give site-specific Slurm settings: Habrok, Snellius, and Kapteyn.

Grid output layout¶

Running a grid creates one folder per case inside the output directory, alongside copies of the inputs needed to reproduce the ensemble:

output/<grid name>/
├── ref_config.toml      # copy of the reference config
├── copy.grid.toml       # copy of the grid definition
├── manager.log          # grid manager log
├── cfgs/                # generated per-case config files (case_000000.toml, ...)
├── logs/                # per-job logs (Slurm dispatch)
├── case_000000/         # full PROTEUS run directory for the first case
├── case_000001/
└── ...

Each case_NNNNNN/ folder is a complete PROTEUS run directory, numbered in the same order as the grid points listed in manager.log. Because the reference config and grid definition are copied into the output folder, an ensemble can be regenerated or extended from its output directory alone. To run a grid again from these copies, copy them out of the output folder first: a grid whose output folder or symlink folder holds its grid file or base config, also through a link, stops before anything is removed. The per-case output files are described on the Running and output page.

Viewing grid status¶

Use the CLI to view the status of a grid, such as to check cases which are finished. For example, the command below will summarise the top-level statuses of the demo grid.

proteus grid-summarise -o output/grid_demo/

Add -s to find out which cases have a particular status. For example, the command below will list all completed cases.

proteus grid-summarise -o output/grid_demo/ -s completed

Packaging grid results¶

Use the CLI to package the results of a grid into a zip file; for example to share or back it up. The command below creates pack.zip in the grid_demo/ folder. This does not store all the data for each case, only the most important files: the 1D resolved interior and atmospheric data are dropped, while files containing global parameters and auto-generated plots are preserved.

proteus grid-pack -o output/grid_demo/

Post-processing a grid¶

Once a grid has run, proteus grid-analyse extracts each case's final-row results into summary CSV files and makes overview plots. It is configured by an optional [post_processing] table in the same .grid.toml used to run the grid (so a grid needs only one config file). Point it at the grid's output folder:

proteus grid-analyse -o output/<grid folder>

With -o alone, the options come from copy.grid.toml in that folder (the copy of the grid file saved when the grid was launched). Add -c <grid file> to analyse the folder with the [post_processing] options of that file instead, for example after changing the plotted outputs. -c alone also works when the grid file's output names a fixed folder; with output = "auto" the folder name is generated at launch, so pass it with -o.

The swept axes are always read from copy.grid.toml when the folder has one, so editing the grid file afterwards does not change them. If that copy no longer passes the current grid-file checks (for example a grid launched with an older PROTEUS), the analysis stops with an error naming the rejected keys; update them in the copy, or remove it to use the axes of the file passed with -c.

The [post_processing] table is checked when the grid is launched with proteus grid and again by proteus grid-analyse: an unknown key, a value of the wrong type, an unknown colour map, style or plot format stops the command with an error before any case runs.

The [post_processing] table accepts:

solidification_time is computed by the analysis as the first time a case's Phi_global is at or below its params.stop.solid.phi_crit (the solidification stop criterion). It is computed the same way when that stop criterion (params.stop.solid.enabled) is disabled, for example in a grid that runs every case to a fixed end time, and is empty (NaN) only for a case that never reached phi_crit.

The outputs are written under the grid's output folder:

output/<grid name>/post_processing/
├── extracted_data/
│   ├── final_extracted_data_all.csv            # every case
│   ├── final_extracted_data_completed.csv      # completed cases
│   └── final_extracted_data_running_error.csv  # running, errored or unknown-status cases
└── grid_plots/
    ├── summary_grid_statuses.<fmt>             # case count per final status
    └── ecdf_grid_plot.<fmt>                    # ECDF per (parameter, output) panel

The CSVs are tab-separated (like other PROTEUS output), one row per started case, holding the case number, status, status category (Completed, Running, Error, or Unknown when the status file is missing, unreadable or holds an unrecognised code), config_source, the swept parameter values, every helpfile column at the final row, and solidification_time. The swept values come from the case's init_coupler.toml; a case that stopped before writing it (for example on a configuration error) takes them from cfgs/case_<N>.toml, the config the grid wrote for it, and config_source names the file used.

The plots take their colours from the plot style: the series colours from its colour cycle, the greys from its text and face colours, and the ticks, spines and grid from the style itself.

The status-summary plot has one horizontal bar per final status, plus a Not started bar for grid cases that have no case folder yet (for example still queued on SLURM). Completed, running and errored statuses take the colour of the style's cycle nearest to green, blue and red; unknown and not-started cases are grey.

The ECDF grid plot uses the completed cases. It has one row per swept parameter and one column per requested output; curves are coloured by the parameter value. A parameter with no more values than the colour cycle of the style takes the cycle colours and a legend; one with more takes the colour map, trimmed at each end until its colours stand out against the face colour of the style, and a colourbar with a tick at each swept value (string values get a legend). A numeric parameter is coloured on a log scale when its default plot setting is logarithmic and its values span at least a factor of 10. The layout follows Figure 5 of Nicholls et al. (2024). Each curve is a percentage of all completed cases with that parameter value, so it levels off below 100 % when some cases have no value for the output (for example cases that never solidified). Outputs whose default plot setting is logarithmic (for example P_surf) are drawn on a log axis when their values are all positive and span at least a factor of 10. When some cases have a zero value (for example a surface pressure of 0 bar after the atmosphere escaped), the axis is a 0 + log axis: a short linear segment from 0 to the decade below the smallest positive value, then logarithmic, so those cases appear at 0. Over a narrower range the axis is linear.

See also: Running and output | Configuration file | Execution and output reference | Parameter grid tutorial | Bayesian inference