New Spectral Classification Technique for Faint X-ray Sources: Quantile Analysis JaeSub Hong Spring, 2006 J. Hong, E. Schlegel & J.E. Grindlay, ApJ 614, 508, 2004 The quantile software (perl and IDL) is available at

Extracting Spectral Properties or Variations from Faint X-ray sources Hardness Ratio HR 1 =(H-S)/(H+S) or HR 2 = log 10 (H/S) e.g. S: keV, H: keV X-ray colors C 21 = log 10 (C 2 /C 1 ) : soft color C 32 = log 10 (C 3 /C 2 ) : hard color e.g. C 1 : keV, C 2 : keV, C 3 : keV

Hardness Ratio Pros Easy to calculate Require relatively low statistics (> 2 counts) Direct relation to Physics (count  flux) Cons Different sub-binning among different analysis Many cases result in upper or lower limits Spectral bias built in sub-band selection Pros Easy to calculate Require relatively low statistics (> 2 counts) Direct relation to Physics (count  flux) Cons Different sub-binning among different analysis Many cases result in upper or lower limits Spectral bias built in sub-band selection

Hardness Ratio Pros Easy to calculate Require relatively low statistics (> 2 counts) Direct relation to Physics (count  flux) Cons Different sub-binning among different analysis Many cases result in upper or lower limits Spectral bias built in sub-band selection Pros Easy to calculate Require relatively low statistics (> 2 counts) Direct relation to Physics (count  flux) Cons Different sub-binning among different analysis Many cases result in upper or lower limits Spectral bias built in sub-band selection e.g. simple power law spectra (PLI =  ) on an ideal (flat) response S band : H band  ~ 0  ~ 1  ~ – 4.2 : 4.2 – 8.0 keV = 1:1 4:1 27:1 0.3 – 1.5 : 1.5 – 8.0 keV = 1:5 1:15:1 0.3 – 0.6 : 0.6 – 8.0 keV = 1:24 1:41:1

Hardness Ratio Pros Easy to calculate Require relatively low statistics (> 2 counts) Direct relation to Physics (count  flux) Cons Many cases result in upper or lower limits Spectral bias built in sub-band selection Pros Easy to calculate Require relatively low statistics (> 2 counts) Direct relation to Physics (count  flux) Cons Many cases result in upper or lower limits Spectral bias built in sub-band selection e.g. simple power law spectra (PLI =  ) on an ideal (flat) response S band : H bandSensitive to (HR~0) 0.3 – 4.2 : 4.2 – 8.0 keV  ~ – 1.5 : 1.5 – 8.0 keV  ~ – 0.6 : 0.6 – 8.0 keV  ~ 2

X-ray Color-Color Diagram C 21 = log 10 (C 2 /C 1 ) C 32 = log 10 (C 3 /C 2 ) C 1 : keV C 2 : keV C 3 : keV Power-Law :  & N H Intrinsically Hard More Absorption

X-ray Color-Color Diagram Simulate 1000 count sources with spectrum at the grid nods. Show the distribution (68%) of color estimates for each simulation set. Very hard and very soft spectra result in wide distributions of estimates at wrong places.

X-ray Color-Color Diagram Total counts required in the broad band ( keV) to have at least one count in each of three sub-energy bands Sensitive to C 21 ~0 and C 32 ~0

Use counts in predefined sub-energy bins. Count dependent selection effect Misleading spacing in the diagram Use counts in predefined sub-energy bins. Count dependent selection effect Misleading spacing in the diagram Hardness ratio & X-ray colors

Use counts in predefined sub-energy bins. Count dependent selection effect Misleading spacing in the diagram Use counts in predefined sub-energy bins. Count dependent selection effect Misleading spacing in the diagram Hardness ratio & X-ray colors e.g. simple power law spectra (PLI =  ) on an ideal (flat) response S band, H bandSensitive to Median 0.3 – 4.2, 4.2 – 8.0 keV  ~ 04.2 keV 0.3 – 1.5, 1.5 – 8.0 keV  ~ 11.5 keV 0.3 – 0.6, 0.6 – 8.0 keV  ~ 20.6 keV

Search energies that divide photons into predefined fractions. : median, terciles, quartiles, etc Search energies that divide photons into predefined fractions. : median, terciles, quartiles, etc How about Quantiles? e.g. simple power law spectra (PLI =  ) on an ideal (flat) response S band, H bandSensitive to Median 0.3 – 4.2, 4.2 – 8.0 keV  ~ 04.2 keV 0.3 – 1.5, 1.5 – 8.0 keV  ~ 11.5 keV 0.3 – 0.6, 0.6 – 8.0 keV  ~ 20.6 keV

Quantiles Quantile Energy (E x% ) and Normalized Quantile (Q x ) x% of total counts at E < E x% Q x = (E x% -E lo ) / (E lo -E up ), 0<Q x <1 e.g. E lo = 0.3 keV, E up =8.0 keV in 0.3 – 8.0 keV Median (m=Q 50 ) Terciles (Q 33, Q 67 ) Quartiles (Q 25, Q 75 )

Quantiles Low count requirements for quantiles: spectral-independent 2 counts for median 3 counts for terciles and quartiles No energy binning required Take advantage of energy resolution Optimal use of information

Hardness Ratio HR 1 = (H-S)/(H+S) -1 < HR 1 < 1 HR 1 = (H-S)/(H+S) -1 < HR 1 < 1 HR 2 = log 10 [ (1+HR 1 )/(1-HR 1 ) ] m=Q 50 = (E 50% -E lo )/(E up -E lo ) 0 < m < 1 m=Q 50 = (E 50% -E lo )/(E up -E lo ) 0 < m < 1 Median HR 2 = log 10 (H/S) -  < HR 2 <  HR 2 = log 10 (H/S) -  < HR 2 <   qDx= log 10 [ m/(1-m) ] -  < qDx <  qDx= log 10 [ m/(1-m) ] -  < qDx <  

Hardness ratio simulations (no background) S: keV H: keV Fractional cases with upper or lower limits

Hardness Ratio vs Median (no background) Hardness Ratio keV Median keV

Hardness Ratio vs Median (source:background = 1:1) Hardness Ratio keV Median keV

Quantile-based Color-Color Diagram (QCCD) Quantiles are not independent m=Q 50 vs Q 25 /Q 75 Power-Law :  & N H Proper spacing in the diagram Poor man’s Kolmogorov -Smirnov (KS) test An ideal detector keV Intrinsically Hard More Absorption E 50% =

Overview of the QCCD phase space

Color estimate distributions (68%) by simulations for 1000 count sources Quantile Diagram keV Conventional Diagram keV E 50% =

Realistic simulations ACIS-S effective area & energy resolution An ideal detector E 50% =

100 count source with no background Quantile Diagram keV Conventional Diagram keV

100 source count/ 50 background count Quantile Diagram keV Conventional Diagram keV

50 count source without background Quantile Diagram keV Conventional Diagram keV

50 source count/ 25 background count Quantile Diagram keV Conventional Diagram keV

Energy resolution and Quantile Diagram E lo = 0.3 keV E hi = 8.0 keV  E/E = 10% at 1.5 keV E 50% : from E lo + f  E lo to E hi – f  E hi from ~ 0.4 keV to ~ 7.8 keV

Energy resolution and Quantile Diagram E lo = 0.3 keV E hi = 8.0 keV  E/E = 20% at 1.5 keV E 50% : from E lo + f  E lo to E hi – f  E hi from ~ 0.4 keV to ~ 7.6 keV

Energy resolution and Quantile Diagram E lo = 0.3 keV E hi = 8.0 keV  E/E = 50% at 1.5 keV E 50% : from E lo + f  E lo to E hi – f  E hi from ~ 0.5 keV to ~ 7.0 keV

Energy resolution and Quantile Diagram E lo = 0.3 keV E hi = 8.0 keV  E/E = 100% at 1.5 keV E 50% : from E lo + f  E lo to E hi – f  E hi from ~ 0.7 keV to ~ 6.5 keV

Energy resolution and Quantile Diagram E lo = 0.3 keV E hi = 8.0 keV  E/E = 200% at 1.5 keV E 50% : from E lo + f  E lo to E hi – f  E hi from ~ 1.0 keV to ~ 6.0 keV

Energy resolution and Quantile Diagram E lo = 0.3 keV E hi = 8.0 keV  E/E = 500% at 1.5 keV E 50% : from E lo + f  E lo to E hi – f  E hi from ~ 1.2 keV to ~ 5.0 keV

 E/E = 10% at 1.5 keV  E/E = 100% at 1.5 keV Energy resolution and Quantile Diagram

Sgr A* (750 ks Chandra )

Sgr A* (750 ks Chandra )

Sgr A* (750 ks Chandra )

Sgr A* (750 ks Chandra )

Sgr A* (750 ks Chandra )

Swift XRT Observation of GRB Afterglow GRB : Spectral softening with ~ constant N H GRB050509b : Short burst afterglow, softer than the host Quasar

Spectral Bias Stability Sub-binning Phase Space Sensitivity Energy Resolution Physics Quantile Analysis None Good No Need Meaningful Evenly Good Sensitive Indirect X-ray Hardness Ratio or Colors Yes Upper/Lower Limits Required Misleading? Selectively Good Insensitive Direct Score Board

Future Work Find better phase spaces. Handle background subtraction better. Find better error estimates: half sampling, etc. Implement Bayesian statistics?

Conclusion: Quantile Analysis Stable spectral classification with limited statistics No energy binning required Take advantage of energy resolution Quantile-based phase space is a good indicator of spectral sensitivity of the detector. The basic software (perl and IDL) is available at

In principle, by simulations: slow and redundant Maritz-Jarrett Method : bootstrapping Q 25 & Q 75 : not independent MJ overestimates by ~10% 100 count source: consistent within ~5% Quantile Error Estimates

by Maritz-Jarrett Method PL:  =2, N H =5x10 21 cm -2 >~30 count : within ~ 10% <~30 count : overestimate up to ~50% MJ requires 3 counts for Q 50 5 counts for Q 33, Q 67 6 counts for Q 25, Q 75  mj /  sim