Benchmarks and comparisons

This page keeps accuracy, convergence, and cost together while reporting them as separate measurements. Every driver writes CSV as well as plots.

Install the plotting dependency before running the suite:

python -m pip install '.[visual]'

Record the software and hardware configuration with published results.

Driver	Study	Varies	Records
diffusive_representations.py	Time-memory representation accuracy and coefficient cost	spectrum, order, time, mode count	analytic relative error, spectrum construction, recurrence-coefficient preparation
time_fractional_scaling.py	Application performance	mesh, timestep, modes, representation	one setup/first-step sample, median steady cost, iterations, estimated mode storage
spatial_operator_accuracy.py	Spatial numerical-control accuracy	sinc target, Riesz quadrature degree	error against an analytic spectral power or over-resolved Riesz action
spatial_operator_scaling.py	Firedrake spatial-operator performance	mesh and the operator-specific numerical control	one setup/first-application sample and median repeated application cost
riesz_backend_comparison.py	Serial Riesz backend comparison	problem size	dense, matrix-free and H-matrix cost, compression and error against dense

Time-derivative representations

diffusive_representations.py evaluates the same two analytic functions with every diffusive representation at equal mode count and no timestep error. Run it with:

python -m benchmarks.diffusive_representations

Equal-mode error against exact auxiliary-mode evolution for polynomial and exponential inputs.

Warm spectrum construction and recurrence-coefficient preparation are timed separately. End-to-end stepping is measured by the application driver below.

BirkSong and Diethelm are the supported choices for production use. Diethelm2022, its experimental quadrature choices, and YuanAgrawal remain literature-comparison options. Appearing in a plot does not make them recommendations.

Convergence with mode count

Fixing the time treatment and refining only the Gauss-Jacobi node count shows the representation error separately from timestep error.

Diethelm (2008) node refinement for the same analytic inputs and fractional orders. Each curve changes one representation control.

Fractional time stepping

time_fractional_scaling.py uses:

• the moving heat source on a torus from the thermal-scanner demo

• the layered Caputo-Wismer pulse from the skullball demo, including its absorbing outer boundary.

The default study varies:

• mesh level \(h\), holding \(\Delta t\) and \(L\) fixed

• timestep \(\Delta t\), holding \(h\) and \(L\) fixed

• diffusive Gauss-Jacobi nodes \(L\), holding \(h\) and \(\Delta t\) fixed.

It runs both representations, using each list’s middle value as the baseline:

python -m benchmarks.time_fractional_scaling

Run the same study with MPI:

mpiexec -n 4 python -m benchmarks.time_fractional_scaling

A full Cartesian sweep is available when interactions between \(h\), \(\Delta t\), and \(L\) are important:

python -m benchmarks.time_fractional_scaling \
  --mesh-levels 0,1,2 \
  --timesteps 0.04,0.02,0.01 \
  --nodes 8,16,32 \
  --sweep cartesian

CSV output records problem size, parameters, storage, timings, iterations, and MPI ranks. These are performance sweeps. They do not measure solution error. The driver warms each application before recording cases so the first item in the table does not absorb all one-time compilation. Each steady sample covers the requested physical duration and continues long enough to avoid reporting submillisecond samples.

Spatial fractional operators

spatial_operator_accuracy.py first isolates the representation controls from mesh and solver cost. Spectral sinc quadrature is compared with analytic scalar eigenvalue powers. Riesz target quadrature is compared with an over-resolved dense Galerkin action on a fixed mesh:

python -m benchmarks.spatial_operator_accuracy

Each panel changes one representation control while holding the operator, fractional order, and reference fixed.

spatial_operator_scaling.py varies mesh size, spectral sinc target, and Riesz quadrature degree on an irregular-island mesh:

python -m benchmarks.spatial_operator_scaling
mpiexec -n 4 python -m benchmarks.spatial_operator_scaling

Use --riesz-quadrature-rule ordinary to compare Duffy tensor Gauss. The selected rule is fixed within one run.

The default mesh levels are 0 and 1 because a level-2 matrix-free Riesz application takes several minutes on a laptop. Request larger levels explicitly when that cost is appropriate. The driver warms each operator before recording setup, and repeated applications run long enough to avoid reporting submillisecond samples. Setup includes construction and first application. Steady spectral timing uses the populated cache.

riesz_backend_comparison.py compares all Riesz backends on separated patches:

python -m benchmarks.riesz_backend_comparison

This pure-backend benchmark is serial and uses moderate default sizes. It is a backend comparison rather than a claim of asymptotic complexity. Exercise distributed CG1 or CG2 H-matrices through the Firedrake driver:

mpiexec -n 4 python -m benchmarks.spatial_operator_scaling \
  --operators riesz --riesz-assembly hmatrix --field-degree 2

Quick validation

Every driver provides a small smoke run:

python -m benchmarks.diffusive_representations --smoke
python -m benchmarks.time_fractional_scaling --smoke
python -m benchmarks.spatial_operator_accuracy --smoke
python -m benchmarks.spatial_operator_scaling --smoke
python -m benchmarks.riesz_backend_comparison --smoke

Results default to benchmarks/benchmarks-output/. Use --output on the single-study drivers or --output-directory on the diffusive suite.

Plots are generated automatically:

• time-fractional-scaling-steady.png compares steady timestep cost across mesh size, timestep, and diffusive node count

• time-fractional-scaling-setup.png shows the corresponding construction and first-step cost

• spatial-operator-accuracy.png reports error under spectral sinc and Riesz quadrature refinement

• spatial-operator-scaling.png compares mesh scaling, spectral sinc target, and Riesz quadrature degree

• riesz-backend-comparison.png compares all three Riesz apply paths and shows hierarchical storage and construction sampling.

Construction and first-step cost.

Steady per-timestep cost for the same sweeps.

Mesh, spectral-target, and Riesz-quadrature scaling.

Riesz backend construction, application, accuracy, and storage.

Regenerate the two Firedrake performance plots from their CSV files without rerunning Firedrake:

python -m benchmarks.plot_results

Pass --no-plots for CSV only.