1. Astro-Wise prompt ● awe is a shell with uses such as: – Python – data reduction (parallel, “dpu”) – interactive analysis (eclipse) – calculator – plotting (matplotlib) – array computations (numarray) Astro-Wise workshop, november 2005 Ewout Helmich

2. Some more features ● uses the “readline” library: – command line history (“up”, “down”) – command line history search (“up, “down”) – command completion (“tab”) – ctrl-a: go to beginning of line – ctrl-e: go to end of line – ctrl-k: delete until end of line – ctrl-u: delete until beginning of line – etc.

3. What does it look like? Python 2.3.5 (#4, Aug 15 2005, 11:45:46) [GCC 3.4.3 20050227 (Red Hat 3.4.3-22.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Welcome to the Astro-Wise Environment |31-OCT-05 Migration to new database and cluster hardware completed. |17-OCT-05 Secondary standards for SA98 have been added. Importing Astro-Wise packages. Please wait... Initializing Distributed Processing Unit... Current profile: - username : awehelmich - database : aw01.omegarac.nova.aw - project : awe>

4. Imported classes and modules awe> dir() ['AssociateList', 'AssociateListTask', 'Astrom', 'AstrometricCatalog', 'AstrometricCorrection', 'AstrometricParameters', 'AtmosphericExtinctionCoefficient', 'AtmosphericExtinctionCurve', 'AtmosphericExtinctionFrames', 'AtmosphericExtinctionZeropoint', 'BaseFrame', 'BiasFrame', 'BiasTask', 'Catalog', 'Chip', 'CoaddTask', 'CoaddedRegriddedFrame', 'ColdPixelMap', 'ColdPixelsTask', 'Config', 'CosmicMap', 'DarkCurrent', 'DarkCurrentTask', 'DataObject', 'DateTime', 'DomeFlatFrame', 'DomeFlatTask', 'Env', 'FITS', 'Filter', 'FringeFlatTask', 'FringeFrame', 'GAstromTask','GAstrometric', 'GainLinearity', 'GainTask', 'HotPixelMap', 'HotPixelsTask', 'IlluminationCorrection', 'IlluminationCorrectionFrame', 'IlluminationTask', 'Image', 'Imstat', 'Instrument', 'Lamp', 'LongAstrom', 'MasterFlatFrame', 'MasterFlatTask', 'NightSkyFlatFrame', 'NightSkyFlatTask', 'ObservingBlock', 'PhotExtinctionCurve', 'PhotRefCatalog', 'PhotSkyBrightness', 'PhotSrcCatalog', 'PhotTransformation', 'PhotcatTask', 'PhotomTask', 'PhotometricExtinctionReport', 'PhotometricParameters', 'PhotometricReport', 'PixelMap', 'ProcessTarget', 'Processor', 'QuickCheckFrame', 'QuickCheckTask', 'RawBiasFrame', 'RawDarkFrame', 'RawDomeFlatFrame', 'RawFitsData', 'RawFrame', 'RawScienceFrame', 'RawTwilightFlatFrame', 'ReadNoise', 'ReadNoiseTask', 'ReduceTask', 'ReducedScienceFrame', 'RegridTask', 'RegriddedFrame', 'SatelliteMap', 'SaturatedPixelMap', 'ScienceFrame', 'ScienceTask', 'ShutterCheckFrame', 'SourceList', 'SourceListTask', 'SubWinStat', 'TwilightFlatFrame', 'TwilightFlatTask', 'WeightFrame', '__builtins__', '__doc__', '__file__', '__name__', 'doc', 'dpu', 'eclipse', 'glob', 'initpy', 'line', 'newsfile', 'numarray', 'os', 'p', 'profiles', 'pylab', 'sys', 'tag'] awe>

5. Help! ● Use dir() and help() to get going awe> help(ReadNoise) Help on class ReadNoise in module astro.main.ReadNoise: class ReadNoise(astro.database.DBMain.DBObject, astro.main.ProcessTarget.ProcessTarget) | Class for read noise objects. | | The read-noise in ADU is rms/sqrt(2) of the difference between two | bias exposures. The read noise object can be created using: | | >>> rn = ReadNoise() | >>> rn.raw_bias_frames = [RawBiasFrame(pathname='bias1.fits'), | RawBiasFrame(pathname='bias2.fits')] | >>> rn.make()

6. Object model (1) ● Classes for conventional (calibration) files: – (Raw)BiasFrame – (Raw)DomeFlatFrame – (Raw)TwilightFlatFrame – RawScienceFrame/ReducedScienceFrame – etc. ● Classes have methods: – RawBiasFrame.statistics – RawBiasFrame.load_image – RawBiasFrame.read_header

7. Object model (2) ●... and properties: – RawBiasFrame.DATE_OBS – RawBiasFrame.OBSERVER – RawBiasFrame.filename – RawBiasFrame.imstat.median – RawBiasFrame.instrument.name – etc. ● get a list of all methods and properties: awe> dir(RawBiasFrame)

8. What information is stored? ● This can be asked of the class in question: awe> RawDomeFlatFrame.get_persistent_properties() ['DATE', 'DATE_OBS', 'EXPTIME', 'LST', 'MJD_OBS', 'OBJECT', 'OBSERVER', 'UTC', 'chip', 'extension', 'filename', 'filter', 'globalname', 'imstat', 'instrument', 'is_valid', 'lamp', 'object_id', 'observing_block', 'overscan_x_stat', 'overscan_y_stat', 'prescan_x_stat', 'prescan_y_stat', 'process_status', 'quality_flags', 'raw_fits_data', 'template'] awe> Chip.get_persistent_properties() ['NAXIS1', 'NAXIS2', 'OVSCX', 'OVSCY', 'PRSCX', 'PRSCY', 'name', 'object_id', 'orientation'] awe> Filter.get_persistent_properties() ['central_wavelength', 'has_fringes', 'mag_id', 'name', 'object_id']

9. History tracking ● if red is an instance of a ReducedScienceFrame, then: awe> red.bias.raw_bias_frames[0].filename awe> bias.process_params.OVERSCAN_CORRECTION awe> sourcelist.frame.reduced.raw.EXPTIME ● and: ● similarly:

10. Database queries ● Queries in the form:. [....] == “==” can be ==, !=, >, >=, <, <= “value” can be int, float, string, DateTime type, or an object ● Example: awe> q = RawTwilightFlatFrame.imstat.median < 1000.0

11. FITS files ● FITS files are stored on dataserver ● DataObjects have store() and retrieve() methods ● All images are DataObjects ● After selecting a DataObject, it can be retrieved ● Example: awe> red = ReducedScienceFrame.max('creation_date') awe> red.retrieve()

12. Object model (3) ● “Tasks”/”Recipes” are used to create the objects. ● Example: DBRecipes/Bias.py – perform queries to the database for input – retrieve the input files – instantiate the target class – assign input to dependencies of target class – set any configuration parameters – call make method

13. Running tasks (1) ● On your own machine: awe> task = ReadNoiseTask(date='2000-04-28', chip='ccd50', instrument='WFI') awe> task.execute() 16:37:25 - Querying database for instances of class RawBiasFrame 16:37:25 - Retrieving WFI.2000-04-28T16:23:07.621_1.fits 16:37:26 - Running : imcopy 'WFI.2000-04-28T16:23:07.621_1.fits[1]' 'tmpVGjBc9.fits' 16:37:26 - Retrieving WFI.2000-04-28T16:24:12.078_1.fits 16:37:27 - Running : imcopy 'WFI.2000-04-28T16:24:12.078_1.fits[1]' 'tmphhBTCt.fits' 16:37:27 - Making ReadNoise object etc.

14. Running tasks (2) ● Using the Distributed Processing Unit interface awe> dpu.run('ReadNoise', d='2000-04-28', c='ccd50', i='WFI') 16:40:19 - Sending job with identifier b391d3c21131638002 to DPU awe> dpu.get_logs() 16:40:47 - ++++++++++++++++++++++++++++++++++++++++++++ 16:40:47 - job_status = 0 16:40:47 - ++++++++++++++++++++++++++++++++++++++++++++ 16:40:47 - 16:40:30 - Querying database for instances of class RawBiasFrame 16:40:30 - Retrieving WFI.2000-04-28T16:23:07.621_1.fits 16:40:30 - Running : imcopy 'WFI.2000-04-28T16:23:07.621_1.fits[1]' 'tmpWdCpvJ.fits' 16:40:31 - Retrieving WFI.2000-04-28T16:24:12.078_1.fits 16:40:31 - Running : imcopy 'WFI.2000-04-28T16:24:12.078_1.fits[1]' 'tmpLSCP1J.fits' 16:40:31 - Making ReadNoise object