1. Astrometric Calibration
OmegaCen
Astrometric Calibration
Parallel Processing
Dataserver
Erik Deul

2. Astrometric Calibration
Gnomonic projection
Forward
(a,b) -> (x,h)
Backward
(x,h) -> (a,b)
04/12/2003
OmegaCen - UK/Vista meeting

3. Astrometric Calibration
Single field
Polynomial description
x(x,y) = a + S bij xi yj
h(x,y) = c + S dij xi yj
04/12/2003
OmegaCen - UK/Vista meeting

4. Astrometric Calibration
Overlap fields
Independent set of params/field
Dependence of b and d with field
x(x,y) = a + SS bijk xi yj f k
h(x,y) = c + SS eij xi yj f k
Spatial terms Chebychev polynomes
Pn+1 – f Rn(f) + Pn-1(f) = 0
04/12/2003
OmegaCen - UK/Vista meeting

5. Astrometric Calibration
Errors
Centering errors
dx = c1 +b1h + x (c1 x + f1h)
dh = f1 +b1 x + h (c1 x + f1h)
Aberration
dx = c2 +a1 x + ½ c2 ( x2 + h2)
dh = f2 +a1 h + ½ f2 (x2 + h2)
Refraction
dx = k0 (1 +x2 + h2)/(1 +xo x + ho h) ( x0 - x)
dh = k0 (1 +x2 + h2)/(1 +xo x + ho h) ( h0 - h)
Const: independent of object position
First order: rotation 1” -> 0”02
Second order: 1”-> 0”02 displacement 1o off-axis
Const: displacement direction dependent
First order: scaling parameter
Second order: 1”-> 0”006 displacement 1o off-axis
Const: displacement
First order: smaller in scale
Second order: slightly larger than aberration
04/12/2003
OmegaCen - UK/Vista meeting

6. Astrometric Calibration
Least Squares
Pairing
Extracted objects with reference catalog
USNO-A2 100 sources (RMS 0”3)
GSC 8 source (RMS 0”3)
Tycho 1 source (RMS 0”03)
Hipparcos 0.1 source (RMS 0”003)
PPM 0.3 source (RMS 0”1)
Extracted objects in overlap
Internal extraction precision RMS 0.1 pixel (< 0”03)
Minimization
Sum of squared differences weighted with positional precision knowledge
Iterate
Kappa-Sigma clipping > 4s excursions
Remove erroneaus pairrings
Minimize cenering errors
04/12/2003
OmegaCen - UK/Vista meeting

7. Astrometric Calibration
Example
DENIS strip
30o long
10’ wide
180 frames
Specifics
1” pixel
RMS 0”3
04/12/2003
OmegaCen - UK/Vista meeting

8. Astrometric Calibration
Example
WFI dither
5 pointings
8’ x 15’ wide
8 frames pp
Specifics
0”26 pixel
RMS 0”1
04/12/2003
OmegaCen - UK/Vista meeting

9. Astrometric Calibration
Example
WFI dither
Tracing pointing
Tracing scaling
04/12/2003
OmegaCen - UK/Vista meeting

10. Astrometric Calibration
Example
WFI Dither
PSF anisotropy
04/12/2003
OmegaCen - UK/Vista meeting

11. OmegaCen - UK/Vista meeting
Parallel Processing
Embarrassingly parallel
32 CCD’s -> 32 CPU’s
N frames -> N CPU’s
Few places in pipeline need synchronization
Astrometric calibration overlap solution
Parts of swarping
Processing I/O bound
Few floating point operations per pixel (few GHz)
Many pixels, limited bandwidth (1 Gb/s)
Administrative burden
Bind data to machine (disk)
Weight processor (net distance server, cpu speed, ..)
Status WEB page
04/12/2003
OmegaCen - UK/Vista meeting

12. OmegaCen - UK/Vista meeting
Parallel Processing
Inter process communication
WEB technologies
Python methods can be called
Python objects can be passed
Exceptions are thrown and caught
XMLRPC
Similar to WEB services like SOAP
Implicit interface (no WSDL)
Authentication user/passwd mechanism
No site limitations
Could run as GRID machine
AstroWise specific
04/12/2003
OmegaCen - UK/Vista meeting

13. OmegaCen - UK/Vista meeting
Parallel Processing
Scheduler – Servers - Clients
Client
Scheduler
Server
Server
Server
Server
Server
04/12/2003
OmegaCen - UK/Vista meeting

14. OmegaCen - UK/Vista meeting
Parallel Processing
Scheduler – Servers - Clients
Client
OmegaCen
Scheduler
Server
Any Place
Server
Server
Server
Server
HPC-Groningen
04/12/2003
OmegaCen - UK/Vista meeting

15. OmegaCen - UK/Vista meeting
Parallel Processing
Example server class
class TaskRequestHandler(xmlrpcserver.RequestHandler):
def call(self, method, params):
dispatch = {'kill': self.kill,
'weightTask': self.estimate_cost,
'runTask': self.run_task,
'run_task': self.run_task,
'estimate_cost': self.estimate_cost}
if method not in dispatch:
print 'Server does not contain XML-RPC procedure', method
return 0
print "Dispatching:", method, "at", time.ctime(time.time()), "on", self.server.port
return dispatch[method](*params)
04/12/2003
OmegaCen - UK/Vista meeting

16. OmegaCen - UK/Vista meeting
Parallel Processing
def run_task(self, task):
"""Execute the task
Arguments:
task -- A pickled task
This method takes a pickled task as argument and performs the following operations:
1. unpickle the task
2. do task.execute()
3. pickle the task
4. return the pickled task
The task's execute method is expected to store the results in
the task's state. It is also expected to handle it's own errors.
"""
try:
task = pickle.loads(task)
task.execute()
task = pickle.dumps(task)
return task
except Exception, e:
print e
return 0
Example server tasks
def estimate_cost(self, task):
'''Try to determine the cost of a task
'''
try:
task = pickle.loads(task)
return task.estimate_cost()
except Exception, e:
print e
return 0
04/12/2003
OmegaCen - UK/Vista meeting

17. OmegaCen - UK/Vista meeting
Parallel Processing
WSDL Client
class A:
def do_nothing(self):
pass
def do_good(self, *args, **kwargs):
return args, kwargs
def do_bad(*args, **kwargs):
raise RuntimeError('Things went really bad')
a = A()
t = Task(a, 'do_nothing')
t.execute()
print t.result, t.exception
t = Task(a, 'do_good', 1, 2, b=3)
t = Task(a, 'do_bad', 1, 2, b=3)
ts = TaskSequence([Task(A(), 'do_good', 1),
Task(A(), 'do_good', 2),
Task(A(), 'do_good', 3)])
ts.execute()
for t in ts.tasks:
04/12/2003
OmegaCen - UK/Vista meeting

18. OmegaCen - UK/Vista meeting
Data Server
Peer-to-peer server mechanism
Like napster/kaaza
Local data owner
Caching service
Ensemble of data servers
Serve data for pipeline processing
WEB based interface
wget enabled
Python class Storage, methods
retrieve
store
04/12/2003
OmegaCen - UK/Vista meeting

19. OmegaCen - UK/Vista meeting
Data Server
OmegaCen
USM
Napoli
04/12/2003
OmegaCen - UK/Vista meeting

20. OmegaCen - UK/Vista meeting
WEB VO
Distributed processing
Parallel processing, WEB services
Data Server, WEB repositories
Current VO
Cone search
Simple Image Access
Remapping Montage, Swarp?
Future VO
Are there commonalities
Web Services
04/12/2003
OmegaCen - UK/Vista meeting

21. OmegaCen - UK/Vista meeting
Data Server
Filenaming convention
Context Type | Possible values (not exhaustive)
Project | Calibration, Science, Survey, Personal, Other
Owner: | Name (pipeline, develop, user, ...)
Instrument | OmegaCam, MegaCam, INT, WFI, ...
Strategy: | Standard, Deep, Freq, Other
Mode: | Stare, Jitter, Dither, Other
Time: | Date and Time
Filter: | Number, Name
Chip: | Number
Type: | Raw, (Master) Bias, (Master) Dome/Twilight Flat,
| (Master) Flag, Col/Hot pix,
| (Master) Weight, (Reduced) Science
Proc | MJD
Extension | fits, cat, …
The Filename name structure could then be setup as follows:
[Project]-[Owner]-[Instrument]-[Startegy]-[Mode]-[Date]-[Time]-[Filter]-[Chip]-[Process Level]-[Process Type]-[File Type]-[MJD].[extension]
Filling in most known possible values to a maximum of three distinctive characters:
[Cal|Sci|Sur|Per|Oth]-[Pip|Dev|EV]-[Ome|Meg|INT|WFI]-[Sta|Dee|Fre|Oth]-[Sta|Jit|Dit|Oth]-[YYYY:MM:DD]-[HH:MM:SS]-[#]-[#|Nam]-[Mas|Oth]-[Dom|Twi|Red|Cold|Hot]-[Raw|Bias|Flat|Flag|Wei|Sci|Pix]
Unique identifier
GID from Oracle DB
numeric
04/12/2003
OmegaCen - UK/Vista meeting