Maria Grazia Pia, INFN Genova Geant4 Physics Validation Geant4 Space User Workshop Pasadena, 6-10 November 2006 M.G. Pia On behalf of the LowE EM and Advanced Examples Working Groups Pablo Cirrone Giacomo Cuttone Francesco Di Rosa Susanna Guatelli Alfonso Mantero Barbara Mascialino Luciano Pandola Andreas Pfeiffer MG Pia Pedro Rodrigues Giorgio Russo Andreia Trindade Valentina Zampichelli

Maria Grazia Pia, INFN Genova Geant4 Toolkit objective criteria Provide objective criteria to evaluate Geant4 physics models precision –Document their precision against experimental data allsystematically –Test all Geant4 physics models systematically –Quantitative statistical methods –Quantitative tests with rigorous statistical methods Wide set of physics processes and models Versatility of configuration according to use cases How accurate is Geant4 physics modelling? most appropriate model Which is the most appropriate model for my simulation?

Maria Grazia Pia, INFN Genova Verification and Validation of Geant4 physics Verification compliance of the software results with the specifications (the underlying physics model) = compliance of the software results with the specifications (the underlying physics model) –Unit tests (at the level of individual Geant4 classes)Validation comparison against experimental data = comparison against experimental data (Goodness-of-Fit) –Quantitative estimate of the agreement between Geant4 simulation and reference data through statistical methods (Goodness-of-Fit) systematic quantitative A systematic, quantitative validation of Geant4 physics models against reference experimental data is essential to establish the reliability of Geant4-based applications

Maria Grazia Pia, INFN Genova Strategy –Rigorous methods –Systematic, quantitative comparisons –Address all modeling options –Statistical analysis of compatibility with experimental data Adopt the same method also for hadronic physics validation –Start from the bottom (low energy) –Progress towards higher energy based on previous sound assessments Guidance to users based on objective ground –not only “educated-guess” PhysicsLists K. Amako et al., Comparison of Geant4 electromagnetic physics models against the NIST reference data IEEE Trans. Nucl. Sci., Vol. 52, Issue 4, Aug. 2005, pp Statistical Toolkit Goodness-of-Fit test Quantitatitative comparison of experimental - simulated distributions

Maria Grazia Pia, INFN Genova Statistical Toolkit Launched as an ESA project 2 nd development cycle –Released April 2006 Goodness-of-fit tests –Binned distributions –Unbinned distributions –Performance analysis –Power analysis The most complete software tool for 2-sample GoF tests IEEE Trans. Nucl. Sci., December July 2006.

Maria Grazia Pia, INFN Genova Recent validation activities Atomic relaxation –Fluorescence and Auger transition energies Bremsstrahlung –Angular distributions Proton Bragg peak –Electromagnetic interactions –Elastic scattering –Pre-equilibrium –Nuclear de-excitation + other validation activities in Advanced Examples More details: see talks at IEEE NSS 2006

Maria Grazia Pia, INFN Genova Geant4 Atomic Relaxation Geant4 Atomic Relaxation modelsFluorescence Auger electron emission It is used by Geant4 packages: Low Energy Electromagnetic –Photoelectric effect –Low Energy electron ionisation –Low Energy proton ionisation (PIXE) –Penelope Compton scattering Hadronic Physics –Nuclear de-excitation –Radioactive decay Geant4 Low Energy Electromagnetic package takes into account the detailed atomic structure of matter and the related physics processes It includes a package for Atomic Relaxation –Simulation of atomic de-excitation resulting from the creation of a vacancy in an atom by a primary process These physics models are relevant to many diverse experimental applications

Maria Grazia Pia, INFN Genova Courtesy ESA Space Environment & Effects Analysis Section X-Ray Surveys of Asteroids and Moons Induced X-ray line emission: indicator of target composition (~100  m surface layer) Cosmic rays, jovian electrons Geant3.21 ITS3.0, EGS4 Geant4 Solar X-rays, e, p Courtesy SOHO EIT C, N, O line emissions included Geant4 fluorescence Geant4 fluorescence Original motivation from astrophysics requirements Wide field of applications beyond astrophysics 250 keV

Maria Grazia Pia, INFN Genova Atomic Relaxation in Geant4 Two steps: 1.Identification of the atomic shell where a vacancy is created by a primary process (photoelectric, Compton, ionisation) cross sections The creation of the vacancy is based on the calculation of the primary process cross sections relative to the shells of the target atom Cross section modeling and calculation specific to each process products 2.Generation of the de-excitation chain and its products Common package, used by all vacancy-creating processes Geant4 Atomic Relaxation Generation of fluorescence photons and Auger electrons Determination of the energy of the secondary particles produced

Maria Grazia Pia, INFN Genova Modelling foundation in Geant4 Low Energy Electromagnetic Package Calculation of shell cross sections EPDL97 –Based on the EPDL97 Livermore Library for photoelectric effect EEDL –Based on the EEDL Livermore Library for electron ionisation –Based on Penelope model for Compton scattering Detailed atom description and calculation of the energy of generated photons/electrons EADL –Based on the EADL Livermore Library

Maria Grazia Pia, INFN Genova Validation of Geant4 Atomic Relaxation Previous partial validation studies (collaboration with ESA Advanced Concepts Division) –Pure materials: limited number of elements examined –Complex materials: complex experimental set-up, large uncertainties on the target material composition NIST database Systematic validation project: NIST database as reference Authoritative, systematic collection of experimental data

Maria Grazia Pia, INFN Genova Method and tools Geant4 test code to generate fluorescence and Auger transitions from all elements –Geant4 Atomic Relaxation handles 6 ≤ Z ≤ 100 Selection of experimental data subsets from NIST database –The NIST database also contains data from theoretical calculations Comparison of simulated/NIST data with Goodness-of-Fit test –Data grouped for the comparison as a function of Z according to the initial vacancy and transition type –Statistical Toolkit ( –Kolmogorov-Smirnov test p-value –The result of the agreement is expressed through the p-value of the test

Maria Grazia Pia, INFN Genova Fluorescence – Shell vacancy K Shell-end Kolmogorov- Smirnov D p-value Geant4 ○ NIST Z E (keV)

Maria Grazia Pia, INFN Genova Fluorescence – Shell vacancy L1 Shell-end Kolmogorov- Smirnov D p-value Geant4 ○ NIST

Maria Grazia Pia, INFN Genova Fluorescence – Shell vacancy L2 Shell-end Kolmogorov- Smirnov D p-value Geant4 ○ NIST

Maria Grazia Pia, INFN Genova Fluorescence – Shell vacancy L3 Shell-end Kolmogorov- Smirnov D p-value Geant4 ○ NIST

Maria Grazia Pia, INFN Genova Auger electron emission Scarce experimental data in the NIST database –Often multiple data for the same Auger transition: ambiguous reference Analysis in progress: comparison of Geant4 simulation data against the NIST subset of experimental data Preliminary results: good qualitative agreement as in the case of X-ray fluorescence –Rigorous statistical analysis to be completed, will be included in publication

Maria Grazia Pia, INFN Genova Geant4 electron Bremsstrahlung 2 electromagnetic physics packages Standard Low Energy 3 Bremsstrahlung processes G4eBremsstrahlung G4PenelopeBremsstrahlung G4eLowEnergyBremsstrahlung Tsai 2BN2BS angular distributions angular distribution

Maria Grazia Pia, INFN Genova Validation of Geant4 EM physics K. Amako et al., IEEE Trans. Nucl. Sci. 52 (2005) 910 Ongoing large-scale project Photon mass attenuation coefficient Range, Stopping power (e, p,  ) NISTNIST NSS 2006 Atomic relaxation (fluorescence, Auger effect) Proton Bragg peak Electron Bremsstrahlung Bremsstrahlung Difficult to find reference data Thin/thick target experiments Difficult to disentangle effects (because of the continuous part) 1 st validation cycle: focus on low energy

Maria Grazia Pia, INFN Genova Angular distributions Angular distribution of photons is strongly model-dependent Angle (deg) Penelope Standard Low Energy (TSAI) 70 keV Angle (deg) Penelope TSAI 2BS 2BN Low Energy Package

Maria Grazia Pia, INFN Genova The experimental set-up e - beam(70 keV-10 MeV) incident on a slab of material Z axis electrons Photon (energy, θ) θ YieldEnergyPolar Angle Yield, Energy and Polar Angle of the emitted photons Electrons and  -rays are absorbed Bremsstrahlung photons can be transmitted Secondary production threshold = 0.5  m Quantitatitative comparison of experimental - simulated distributions Statistical Toolkit Goodness-of-Fit test in progress

Maria Grazia Pia, INFN Genova Data sets N. Starfelt et al., Phys. Rev. 102 (1956) Thin target: Be, Al, Au - 2.7, 4.5, 9.7 MeV Double differential cross sections W.E. Dance et al., Journal of Appl. Phys. 39 (1968) Thick target: Al, Fe – 0.5, 1 MeV Double differential cross sections Integrated  yield R. Ambrose et al., NIM B 56/57 (1991) 327 Absolute and relative yield Preliminary results Work in progress! Simulation production: still running Statistical analysis: still preliminary, to be completed

Maria Grazia Pia, INFN Genova Double differential  at 2.7 MeV on thin (2.63 mg/cm 2 ) Be target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 4.5 MeV on thin (2.63 mg/cm 2 ) Be target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation Preliminary Kolmogorov-Smirnov p-value = 0.17 Preliminary Kolmogorov-Smirnov p-value = 0.13

Maria Grazia Pia, INFN Genova Double differential  at 9.7 MeV on thin (2.63 mg/cm 2 ) Be target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 2.7 MeV on thin (0.878 mg/cm 2 ) Al target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 2.7 MeV on thin (0.878 mg/cm 2 ) Al target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 4.5 MeV on thin (0.878 mg/cm 2 ) Al target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 9.7 MeV on thin (0.878 mg/cm 2 ) Al target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 2.7 MeV on thin (0.209 mg/cm 2 ) Au target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 4.5 MeV on thin (0.209 mg/cm 2 ) Au target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Double differential  at 9.7 MeV on thin (0.209 mg/cm 2 ) Au target Energy (MeV) N. Starfelt et al., Phys. Rev. 102 (1956) 1598 data + simulation data + simulation

Maria Grazia Pia, INFN Genova Angular distribution Red = data Black = simulation o  Al  Fe Standard package Absolute comparison W.E. Dance et al., Journal of Applied Physics 39 (1968) keV electrons on Al (0.548 g/cm 2 ) and Fe (0.257 g/cm 2 ) Thick target experiment 500 keV Preliminary  2 test p-value = 0.10

Maria Grazia Pia, INFN Genova Angular distribution precise agreement! W.E. Dance et al., Journal of Applied Physics 39 (1968) keV Preliminary  2 test p-value = 0.68 Preliminary  2 test p-value = 0.03

Maria Grazia Pia, INFN Genova Angular distribution W.E. Dance et al., Journal of Applied Physics 39 (1968) keV Preliminary  2 test p-value not meaningful Preliminary  2 test p-value = 0.33

Maria Grazia Pia, INFN Genova Angular distribution Red = data Black = simulation o  Al  Fe Same test for 1 MeV primary electrons (threshold: 50 keV) Absolute comparison W.E. Dance et al., Journal of Applied Physics 39 (1968) 2881 Targets: Al (0.548 g/cm 2 ) and Fe (0.613 g/cm 2 ) 1 MeV Preliminary Fe  2 test p-value not meaningful

Maria Grazia Pia, INFN Genova Angular distribution Good agreement for Al - Reasonable also for Fe (2BN) precise agreement! W.E. Dance et al., Journal of Applied Physics 39 (1968) MeV Preliminary Fe  2 test p-value = 0.06 Preliminary Fe  2 test p-value = 0.68

Maria Grazia Pia, INFN Genova Angular distribution 2BS: good for Al and Fe (except in the backward direction) W.E. Dance et al., Journal of Applied Physics 39 (1968) MeV Preliminary Fe  2 test p-value = 0.36 Preliminary Fe  2 test p-value not meaningful

Maria Grazia Pia, INFN Genova Integral  yield Total  yield on Al integrated on (0   ) and on energy (E th  E max ) Also available for other flavours of Geant4 Bremsstrahlung models o  data  simul. W.E. Dance et al., Journal of Applied Physics 39 (1968) 2881 Further investigation in progress Preliminary Preliminary

Maria Grazia Pia, INFN Genova Energy distribution at 70 keV 70 keV electrons impinging on Al (25.4 mg/cm 2 ) Penelope Low Energy - TSAI Photon energy (keV) Intensity/Z (eV/sr keV) 70 keV e - photon direction 45 deg R. Ambrose et al., Nucl. Instr. Meth. B 56/57 (1991) 327

Maria Grazia Pia, INFN Genova Relative comparison at 70 keV Relative comparison (45° direction) Shapes of the spectra are in good agreement Intensity/Z (eV/sr keV) Photon energy (keV) Penelope Low Energy - TSAI Intensity/Z (eV/sr keV)

Maria Grazia Pia, INFN Genova Proton Bragg peak Compare various Geant4 electromagnetic models Assess lowest energy range of hadronic interactions – elastic scattering –pre-equilibrium + nuclear deexcitation to build further validation tests on solid ground Results directly relevant to various experimental use cases Oncological radiotherapy Medical Physics LHC Radiation Monitors High Energy Physics Space Science Astronauts’ radiation protection

Maria Grazia Pia, INFN Genova Relevant Geant4 physics models Standard Low Energy – ICRU 49 Low Energy – Ziegler 1977 Low Energy – Ziegler 1985 Low Energy – Ziegler 2000 New “very low energy” models Parameterized (à la GHEISHA) Nuclear Deexcitation –Default evaporation –GEM evaporation –Fermi break-up Pre-equilibrium –Precompound model –Bertini model Elastic scattering –Parameterized models –Bertini Intra-nuclear cascade –Bertini cascade –Binary cascade Hadronic Electromagnetic Subset of results shown here Full set of results in publication coming shortly

Maria Grazia Pia, INFN Genova Experimental data CATANA hadrontherapy facility in Catania, Italy –high precision experimental data satisfying rigorous medical physics protocols –Geant4 Collaboration members Validation measurements Markus Ionization chamber 2 mm Sensitive Volume = 0.05 cm 3 Resolution 100  m Markus Chamber

Maria Grazia Pia, INFN Genova Geant4 simulation Accurate reproduction of the experimental set-up quantitative This is the most difficult part to achieve a quantitative Geant4 physics validation Geometrybeam Geometry and beam characteristics must be known in detail and with high precision Ad hoc beam line set-up for Geant4 validation to enhance peculiar effects of physics processes E proton = 63.5 MeV  E = 300 keV

Maria Grazia Pia, INFN Genova Electromagnetic processes Electromagnetic options  Standard EM  Low Energy EM – ICRU 49  Low Energy EM – Ziegler 1977  Low Energy EM – Ziegler 1985  Low Energy EM – Ziegler 2000

Maria Grazia Pia, INFN Genova Electromagnetic processes Standard EM: p, ions, , e- e+p-value CvMKSAD Left branch Right branch Whole curve CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 1 M events mm Geant4 Experimental data Standard EM

Maria Grazia Pia, INFN Genova Electromagnetic processes Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+p-value CvMKSAD Left branch Right branch Whole curve CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 1 M events mm Geant4 Experimental data LowE EM – ICRU49

Maria Grazia Pia, INFN Genova Electromagnetic processes Low Energy EM – Ziegler 1977: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ 1 M events mm Geant4 Experimental data LowE EM – Ziegler 1977 CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test

Maria Grazia Pia, INFN Genova Electromagnetic processes LowE EM – Ziegler 1985 Low Energy EM – Ziegler 1985: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ Subject to further investigation 1 M events mm Geant4 Experimental data

Maria Grazia Pia, INFN Genova Electromagnetic processes Low Energy EM – Ziegler 2000: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ Subject to further investigation 1 M events mm Geant4 Experimental data LowE EM – Ziegler 2000

Maria Grazia Pia, INFN Genova Electromagnetic processes Summary p-value LowE ICRU49 LowE Ziegler 1977 Standard Left branch (CvM) Right branch (KS) Whole curve (AD) CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test LowE – ICRU49 Best EM option: LowE – ICRU49 Selected for further EM + Hadronic tests

Maria Grazia Pia, INFN Genova Electromagnetic processes + Elastic scattering Elastic scattering options  HadronElastic process with LElastic model  HadronElastic process with BertiniElastic model  UHadronElastic process with HadronElastic model

Maria Grazia Pia, INFN Genova EM + Elastic scattering Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ LElastic HadronElastic with LElasticp-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 LElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 1 M events mm Geant4 Experimental data

Maria Grazia Pia, INFN Genova EM + Elastic scattering Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ HadronElastic UHadronElastic with HadronElasticp-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 HadronElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 0.5 M events mm Geant4 Experimental data

Maria Grazia Pia, INFN Genova Electromagnetic processes + Elastic scattering + Hadronic inelastic scattering Hadronic Inelastic options  Precompound with Default Evaporation  Precompound with GEM Evaporation  Precompound with Default Evaporation + Fermi Break-up  Bertini

Maria Grazia Pia, INFN Genova EM + hadronic physics Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ LElastic HadronElastic with LElastic Precompound Default Evaporation Precompound with Default Evaporationp-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 LElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 1 M events mm Geant4 Experimental data Precompound default

Maria Grazia Pia, INFN Genova EM + hadronic physics Standard EM: p, ions, , e- e+ LElastic HadronElastic with LElastic Precompound Default Evaporation Precompound with Default Evaporationp-value CvMKSAD Left branch Right branch Whole curve CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 1 M events mm Geant4 Experimental data Standard EM LElastic Precompound default

Maria Grazia Pia, INFN Genova EM + hadronic physics Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ HadronElastic Precompound Default Evaporation UHadronElastic with HadronElastic Precompound with Default Evaporationp-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 HadronElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 0.5 M events mm Geant4 Experimental data Precompound default

Maria Grazia Pia, INFN Genova EM + hadronic physics Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ LElastic HadronElastic with LElastic Precompound GEM Evaporation Precompound with GEM Evaporationp-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 LElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 0.5 M events mm Geant4 Experimental data Precompound with GEM Evaporation

Maria Grazia Pia, INFN Genova EM + hadronic physics Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ LElastic HadronElastic with LElastic Precompound Fermi Break-up Precompound with Fermi Break-upp-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 LElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 0.5 M events mm Geant4 Experimental data Precompound with Fermi Break-up

Maria Grazia Pia, INFN Genova EM + hadronic physics Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ LElastic HadronElastic with LElastic Bertini Inelastic p-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 LElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 1 M events mm Geant4 Experimental data Bertini Inelastic

Maria Grazia Pia, INFN Genova EM + hadronic physics Low Energy EM – ICRU49: p, ions Low Energy EM – Livermore: , e- Standard EM:e+ HadronElastic with BertiniElastic Bertini Inelastic p-value CvMKSAD Left branch Right branch Whole curve LowE EM – ICRU49 BertiniElastic CvM Cramer-von Mises test KS Kolmogorov-Smirnov test AD Anderson-Darling test 0.5 M events mm Geant4 Experimental data Bertini Inelastic

Maria Grazia Pia, INFN Genova Electromagnetic + Hadronic Summary p-value Standard LElastic Precompound LowE ICRU49 LElastic Precompound GEM LowE ICRU49 LElastic Bertini Inelastic LowE ICRU49 LElastic Precompound Fermi Break-up LowE ICRU49 LElastic Precompound LowE ICRU49 HadronElastic Precompound LowE ICRU49 Bertini Elastic Bertini Inelastic Left branch (CvM) Right branch (KS) Whole curve (AD) Key ingredients electromagnetic  Precise electromagnetic physics elastic scattering  Good elastic scattering model pre-equilibrium  Good pre-equilibrium model

Maria Grazia Pia, INFN Genova …and behind everything Unified Process A rigorous software process Incremental and iterative lifecycle RUP  as process framework, tailored to the specific project Mapped onto ISO 15504

Maria Grazia Pia, INFN Genova Conclusion Geant4 physics validation carried on by a small, young team with rigorous methods –Underlying vision –Systematic approach –Rigorous quantitative analysis Current projects –Atomic relaxation: final results –Bremsstrahlung: preliminary results –Proton Bragg peak: mature stage, refinements by end 2006 Publications coming soon