Benchmarks and code comparisons

OpenBNCT checks transport calculations against analytic solutions, published numerical problems, independent Monte Carlo and measured/digitized phantom data. Each comparison supports a particular claim under its stated inputs and gates.

What the comparison measures

Convergence and artifact verification are separate from these physical comparisons. A frozen historical result does not promise every later configuration will pass.

Deterministic transport versus OpenMC

The current-source project workflow has a documented synthetic layered-head comparison with continuous-energy OpenMC, matched thermal scattering and transported capture photons. The README records S8 and 5 million histories. Each component column below is the ratio S_N/MC:

A ratio of 1 means equal mean dose. A ratio of 0.86 is 14% below the Monte Carlo result; 1.06 is 6% above. These values describe structure means, not every voxel or a gamma pass rate. That fixture has no nitrogen component to grade.

These observations are stated in the current-source README and project report implementation. The earlier layered-head note describes older S4/free-gas and local-photon configurations; its ratios answer a different comparison. It must not be combined with the later matched-physics result.

Source-bin weighting, thermal scattering, component response and photon policy matter. With local capture-photon deposition, the documented photon mean was roughly 3–4 times high. The project photon-transport path changes that model; one component improvement does not qualify all transport cases.

Use project verify on your project rather than borrowing this fixture’s ratios. Its default gates are total-dose ratio within 5% and gamma pass rate at least 95% for evaluated structures, using global 3%/3 mm gamma with a 10% low-dose cutoff. Monte Carlo uncertainty can make the verdict inconclusive. A component within 15% does not automatically pass these stricter, differently defined gates.

Public CT geometry: a separate comparison

The head-and-neck CT benchmark uses a public scan cropped to 206,226 voxels at 4 mm, with an explicitly synthetic research target and illustrative boron assumptions. Its final recorded source build is b46ffb2, with 28 neutron groups, S4/P1, transported photons and 6 million OpenMC histories.

Structure-mean total-dose S_N/MC ratios are 0.982 for brain, 1.003 for BODY and 0.952 for the synthetic target. These are close regional means; they do not establish voxelwise agreement. The recorded overall verdict is INCONCLUSIVE, with eleven structures unresolved under the configured gates. Monte Carlo photon/total voxels were particularly noisy, and BODY’s reported gamma result came from a single evaluated voxel.

Keep the photon policy and build identity with these results. The earlier local-deposition run used a different build and reported photon ratios of 6.03 in brain and 8.10 in the target. Comparing those runs does not isolate one code change. The final run’s single-host timing was also observed with other jobs active, so it is not a controlled speed comparison.

This extends the tested geometry beyond a synthetic phantom. The scan, synthetic target, coarse small structures, illustrative uptake and unresolved statistics retain their stated research scope.

Analytic and canonical evidence

NF-BNCT-003 tests a one-group near-pure B-10 absorber. The recorded boron-dose attenuation slope matches the analytic −0.2308 cm⁻¹ within the declared tolerance. It tests that simple model rather than heterogeneous anatomical accuracy.

Reed and Azmy test source deposition, heterogeneity and discretization against published references. Reed records near-percent regional agreement; Azmy’s quadrant errors are 0.17%, 0.67% and 3.9% under its declared setup.

Kobayashi passes predeclared near-field probes at 5%, while documenting severe deep-field ray effects. The OpenMC multigroup comparison reports off-lobe underfill and large diagonal overshoot. Increasing angular order can sharpen those lobes. Its near-field pass does not establish full-field agreement.

Measured phantom comparisons

The historical three-group FiR 1 cylindrical-water comparison records peak-normalized depth-profile χ² = 6.7 over 12 bins. The PMMA comparison records χ² = 6.0 over 15 bins. Their sigma-weighted passes concern those normalized profiles, digitized from TECDOC-1223 figures.

Peak normalization removes overall amplitude, so a shape pass does not establish absolute dose. Later 28-group variants retain documented differences; the old three-group verdict cannot be transferred to them. Consult the water and PMMA records for the exact configurations.

Other research evidence

NF-BNCT-001’s 600-million-history OpenMC candidate passes its statistical gates; independent reproduction remains required for reference promotion. NF-BNCT-002’s frozen deep-penetration case remains unexecuted in the evidence catalogue.

Scenario optimization, PK timing, microdosimetry, joint uncertainty and prompt-gamma studies have their own models and gates. A known-answer optimizer fixture tests that optimization problem. Prompt-gamma examples also record localization failures. See the validation dossier and benchmark catalogue.

What a competitive comparison can establish

OpenMC comparisons support independent transport assessment on matched research cases. MCNP/PHITS parser fixtures support interchange. The repository does not establish clinical equivalence or broad superiority over commercial treatment-planning systems. A meaningful code comparison needs matching source physics, data, geometry, dose definitions, resolution and acceptance criteria, with statistical uncertainty and runtime conditions reported.