SDSS photo-z with model templates. Photo-z Estimate redshift (+ physical parameters) –Colors are special „projection” of spectra, like PCA.

Slides:



Advertisements
Similar presentations
Realistic photometric redshifts Filipe Batoni Abdalla.
Advertisements

S.Towers TerraFerMA TerraFerMA A Suite of Multivariate Analysis tools Sherry Towers SUNY-SB Version 1.0 has been released! useable by anyone with access.
Eötvös University Budapest in the Network.  Seniors: István Csabai (node coordinator): »Photometric redshift estimation, virtual observatories, science.
Applications of UDFs in Astronomical Databases and Research Manuchehr Taghizadeh-Popp Johns Hopkins University.
An Experience of Aladin Usage for the RC Catalog Radio Source Investigation Zhelenkova Olga, SAO RAS.
Aggregating local image descriptors into compact codes
A new result on space-time variation of alpha – Part C John Webb, School of Physics, University of New South Wales, Australia Group members as per Michael.
CLASH Photometric redshifts for galaxies in cluster field Stephanie Jouvel & CLASH photo-z team London sept 20131Stephanie Jouvel Jouvel et al 2013 submitted.
ECE 8527 Homework Final: Common Evaluations By Andrew Powell.
Galaxy and Mass Power Spectra Shaun Cole ICC, University of Durham Main Contributors: Ariel Sanchez (Cordoba) Steve Wilkins (Cambridge) Imperial College.
Instance Based Learning
Fast High-Dimensional Feature Matching for Object Recognition David Lowe Computer Science Department University of British Columbia.
RESULTS AND ANALYSIS Mass determination Kauffmann et al. determined masses using SDSS spectra (Hdelta & D4000) Comparison with our determination: Relative.
A High-Level Comparison of Photo-z Codes on Luminous Red Galaxies Manda Banerji (UCL) Filipe Abdalla (UCL), Ofer Lahav (UCL), Valery Rashkov (Princeton)
Bayesian Photometric Redshifts (BPZ) Narciso Benítez 1,2 (2000) Narciso Benítez 1,2 et al. (2004) Dan Coe 1,2,3 et al. (2006) Johns Hopkins University.
The SDSS is Two Surveys The Fuzzy Blob Survey The Squiggly Line Survey.
Statistical Learning Theory: Classification Using Support Vector Machines John DiMona Some slides based on Prof Andrew Moore at CMU:
Computer Vision James Hays, Brown
Spatial Indexing and Visualizing Large Multi-dimensional Databases I. Csabai, M. Trencséni, L. Dobos, G. Herczegh, P. Józsa, N. Purger Eötvös University,
Playing in High Dimensions Bob Nichol ICG, Portsmouth Thanks to all my colleagues in SDSS, GRIST & PiCA Special thanks to Chris Miller, Alex Gray, Gordon.
New Insight Into the Dust Content of Galaxies Based on the Analysis of the Optical Attenuation Curve.
● DES Galaxy Cluster Mock Catalogs – Local cluster luminosity function (LF), luminosity-mass, and number-mass relations (within R 200 virial region) from.
Spatial Indexing of large astronomical databases László Dobos, István Csabai, Márton Trencséni ELTE, Hungary.
What can we learn from the luminosity function and color studies? THE SDSS GALAXIES AT REDSHIFT 0.1.
COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.
SNLS: Overview and High-z Spectroscopy D. Andrew Howell (Toronto) for the SNLS Collaboration (see: for full list)
RGB Color Balance with ExCalibrator – “Take 2” SIG Presentation B. Waddington 5/21/2013.
1 Pattern Recognition Concepts How should objects be represented? Algorithms for recognition/matching * nearest neighbors * decision tree * decision functions.
EMIS 8381 – Spring Netflix and Your Next Movie Night Nonlinear Programming Ron Andrews EMIS 8381.
Peter Capak Associate Research Scientist IPAC/Caltech.
Full Spectral Analysis of Galaxies - Are we there yet? Ben Panter, Edinburgh
EÖTVÖS UNIVERSITY BUDAPEST Department of Physics of Complex Systems VO Spectroscopy Workshop, ESAC Spectrum Services 2007 László Dobos (ELTE)
Overview of Supervised Learning Overview of Supervised Learning2 Outline Linear Regression and Nearest Neighbors method Statistical Decision.
Indexing and Visualizing Multidimensional Data I. Csabai, M. Trencséni, L. Dobos, G. Herczegh, P. Józsa, N. Purger Eötvös University,Budapest.
Gyöngyi Kerekes Eötvös Lóránd University, Budapest MAGPOP 2008, Paris István Csabai László Dobos Márton Trencséni.
Peter-Christian Zinn | Bringing order to chaos | SKANZ 2012 | Auckland, New Zealand
EÖTVÖS UNIVERSITY BUDAPEST Department of Physics of Complex Systems Photometric parallax estimation using the MILES catalog and BaSeL models István Csabai.
Advanced Stellar Populations Advanced Stellar Populations Raul Jimenez
Large surveys and estimation of interstellar extinction Oleg Malkov Institute of Astronomy, Moscow Moscow, Apr 10-11, 2006.
Photometric Redshifts in Astro-Wise PhotRedCatalog Astro-Wise Workshop Leiden 2008 Jan Snigula, MPE.
Baryonic signature in the large-scale clustering of SDSS quasars Kazuhiro Yahata Dept. of Phys., University of Tokyo. Issha Kayo, Yasushi Suto, Matsubara.
The Research on Algorithms of Estimating Photometric Redshifts Using SDSS Galaxy Data Wang Dan China-VO group Chinese Virtual Observatory.
Chapter1: Introduction Chapter2: Overview of Supervised Learning
1 Galaxy Evolution in the SDSS Low-z Survey Huan Lin Experimental Astrophysics Group Fermilab.
Emission Line Galaxy Targeting for BigBOSS Nick Mostek Lawrence Berkeley National Lab BigBOSS Science Meeting Novemenber 19, 2009.
Web based spectrum databases and utilities László Dobos Tamás Budavári István Csabai MAGPOP kick-off meeting, January Cassis.
Vince Oliver PhD student of ELTE University Budapest  Assignments within MAGPOP project.
Probing Cosmology with Weak Lensing Effects Zuhui Fan Dept. of Astronomy, Peking University.
Fast Query-Optimized Kernel Machine Classification Via Incremental Approximate Nearest Support Vectors by Dennis DeCoste and Dominic Mazzoni International.
KNN Classifier.  Handed an instance you wish to classify  Look around the nearby region to see what other classes are around  Whichever is most common—make.
Stellar Population Mass Estimates Roelof de Jong (STScI AIP) Eric Bell (MPIA Univ. of Michigan)
A relation to estimate the redshift from the X-ray afterglow light curve Bruce Gendre (IASF-Roma/INAF) & Michel Boër (OHP/CNRS)
The Stellar Assembly History of Massive Galaxies Decoding the fossil record Raul Jimenez Licia Verde UPenn Alan Heavens Ben.
Miguel CerviñoESAC 23 March 2007 How to use the SEDs produced by synthesis models? Miguel Cerviño (IAA-CSIC/SVO) Valentina Luridiana (IAA-CSIC/SVO)
LOGO Recognition and Measuremeant for LAMOST Galaxy Spectra 张健楠 天水 2015.
Mahdi Khadem Ferdowsi University of Mashhad Supervisors: Dr S. Arbabi (INO) Dr J. Ghanbari (FUM)
Budapest Group Eötvös University MAGPOP kick-off meeting Cassis 2005 January
RiskTeam/ Zürich, 6 July 1998 Andreas S. Weigend, Data Mining Group, Information Systems Department, Stern School of Business, NYU 2: 1 Nonlinear Models.
Bayesian Template-Based Approach to Classifying SDSS-II Supernovae from 3-Year Survey Brian Connolly Photometric Supernova ID Workshop 3/16/12.
MicroArray Data Analysis Candice Quadros & Amol Kothari.
Photometric redshift for clusters through A1689 and simulation studies Stephanie Jouvel, Ofer Lahav, Ole Host.
1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.
Spectral classification of galaxies of LAMOST DR3
Nonparametric Density Estimation – k-nearest neighbor (kNN) 02/20/17
Understanding the near infrared spectrum of quasars
Photometric redshift estimation.
Behavior of Reddenings in the Color-Magnitude & Color-Color Diagrams of SDSS Filters 김성수, 이명균.
Classification of GAIA data
Photometric Redshift Training Sets
WFIRST Photo-z Study Peter L. Capak, Andreas Faisst,
Presentation transcript:

SDSS photo-z with model templates

Photo-z Estimate redshift (+ physical parameters) –Colors are special „projection” of spectra, like PCA

LIGHT Spectrum 1M objects BROADBAND FILTERS MAGNITUDE SPACE 270M objects REDSHIFT PHYSICAL PARAMETRS age, dust, SFH, etc. GALAXY early, late 3000 DIMENSIONAL POINT DATA 5 DIMENSIONAL POINT DATA 5-10? DIMENSION 3-10 DIMENSION PCA

Photo-z techniques Empirical –Polyfit –Neural net –Nearest neighbor Tempate fitting –Empirical templates Repair –Model templates All the same: –generate a reference set (from observed photometry, synthetic photometry of observed or model spectra) –Linear (weighted sum) or nonlinear function of neighbors’ redshift The key issue: get a good reference set –Easy to get good results for a good reference set –Extrapolation: only hope is better models

Catalogs Test set: DR6 spectro set, galaxies (few outliers removed) Charlot et al.: 100k stochastic SFH model library –u,g,r,i,z synthetic magnitudes, 200 redshift bins in z=0-1 Using colors only for redshift estimation

Fast kd-tree based NN in SQL server Index the color space with a search tree –Find k-nearest neighbors quickly Implemented in SQL server (SQL+CLR) –Local polyfit, average, weighted (photo errors) sum Time to calculate photoz for DR5 (200M object) –Tempate fitting: 150 processor- day –Kd-fit: 10 processor-day

Spectro training set Local linear fit 150 NN Δz= Average 150 NN Δz=0.0306

100k Stochastic library Average 150 NN Δz= Local linear fit 150 NN Δz=0.2275

Best subset Conclusion: too rich reference set with probably un-physical templates Subset 100 of 100k –Iteration Closest templates in color+redshift space Removing templates those cause systematic errors

Best subset: 1st step Average of 150 NN Δz =

Best subset: final iteration Average of 150 NN Dz = Local linear fit of 150 NN Dz =

Open questions Cannot reach as good estimation as with the empirical reference set –Are the templates in the best set „physical”? Systematic calibration mismatch –SDSS filter curves? –Model: dust ? Different sampling –How to compare the sets and find the „offset” in colors –Correcting for the average difference vector Δ(ug,gr,ri,iz)= ( ,0.0036, ,0.0212) does not improve the results Find optimal subset with matching models and observations in spectral space (Laszlo)