1. 2018/11/23
An Application for the Data Grid : A Prototype of the Japanese Virtual Observatory System
Masatoshi Ohishi
National Astronomical Observatory of Japan
　　　　　　and Sokendai
2018/11/23
GGF7 in Tokyo

2. Collaborators NAOJ Y. Mizumoto N. Yasuda Y. Shirasaki M. Tanaka
Ochanomizu U.
Y. Masunaga
Fujitsu Ltd.
K. Monzen
K. Kawarai
Y. Ishihara
H. Yanaka
Y. Yamaguchi
M. Ishida
N. Saito
2018/11/23
GGF7 in Tokyo

3. Supported by Research Grant from the MEXT （文部科学省 科研費 特定領域研究 「情報学」
　（文部科学省　科研費
　　特定領域研究
　　「情報学」
　　松岡・下條組）
Research Grant from the Sokendai
（総合研究大学院大学　企画研究）
2018/11/23
GGF7 in Tokyo

4. Orion nebula
In the IR
2018/11/23
GGF7 in Tokyo

5. More than
5,000 Galaxies
2018/11/23
GGF7 in Tokyo

6. SUBARU telescope in Hawaii (optical & IR)
2018/11/23
GGF7 in Tokyo

7. 45m Radio Telescope at Nobeyama
2018/11/23
GGF7 in Tokyo

8. Time and Spectral Dimensions The Multiwavelength Crab Nebulae
Crab star
1053 AD
X-ray,
optical,
infrared, and
radio
views of the nearby Crab Nebula, which is now in a state of chaotic expansion after a supernova explosion first sighted in 1054 A.D. by Chinese Astronomers.
2018/11/23
GGF7 in Tokyo
Slide courtesy of Robert CalTech.

9. Unknowns of Astronomy Physics just after the Big Bang ?
How Galaxies were formed ?
Averaged density of the Universe
etc.
→　statistical analyses for homogeneous survey data are crucial
2018/11/23
GGF7 in Tokyo

10. Data Productivity Nobeyama Radio ：~1TB/yr Subaru@Hawaii ：~20TB/yr
ALMA（planned）　 ：~PB/yr
Flood of excellent data (survey data)
Digitized & Archived
2018/11/23
GGF7 in Tokyo

11. HPC + High Speed NETWORK
+GRID Technology
→ it is possible to connect and seamlessly use remote DBs and computing facilities for data analyses
→ Virtual Observatory
2018/11/23
GGF7 in Tokyo

12. Telescope is
behind me !
2018/11/23
GGF7 in Tokyo

13. アイデア 観 測 検 索
データ 解 析
研究発表
実望遠鏡での 観測
仮想望遠鏡での
2018/11/23
GGF7 in Tokyo

14. JVO (will) consists of Seamless Access to Federated DBs
Various Data Analysis tools
Advanced Analysis such as Data Mining
Federation with Real Telescopes
→　crucial to control data quality in VO
2018/11/23
GGF7 in Tokyo

15. JVO prototype : Ver. 1
Establish seamless access to federated DBs by using the Grid technology
Define JVO Query Language, and implement its parser
Test environment is in a closed subnet
2018/11/23
GGF7 in Tokyo

16. Parse the JVO QL into each query sequence
Query request
Web
Broser
JVO Query Language
利用者独自
サービス
Catalog Query Service
ASTRONOMER
JVO
Browser
Viewer
Commands callable from GT2
JVO
Portal
Parse the JVO QL into each query sequence
Catalog DB
Data Archive Service
JVO
Contoroller
skycat
Globus
Toolokit V2
Data Management DB
Data
GridFTP each query
UDDI Server
（FreeSoft)
Resolve servers to request queries
　　Globus Toolkit
Service
Registry
Data Analysis Services
Security Service
Resource Management
Data Management
2018/11/23
GGF7 in Tokyo

17. JVO Query Language(1) Catalog Query
2018/11/23
GGF7 in Tokyo

18. JVO Query Language(2) Image Query
2018/11/23
GGF7 in Tokyo

19. UDDI Registory / Servlet Engine
JVO Prototype System Architecture
Redhat Linux
2Mass
DataBase
Redhat Linux
JVO Service
JVO
Controller
Solaris8 (Possible at Linux)
Gb Ether
GSDL
UDDI Registory / Servlet Engine
UDDI Maintenance Tool
Java2SE 1.4
Globus Toolkit (Server)
Java2SE 1.4
Hypertext
contents
Apache Tomcat
Maintenance Tool
JVO Service (X match)
soapuddi
UDDI4J
Netscape Communicator
Java2 Plugin
JVO Client
(applet)
Apache
ＳＯＡＰ
① Applet
DownLoad
JDBC
register
update
delete
JVO Service
(select)
SQL
library
③ Find Service
② Request
Observation
JVO Controller
DB Service
Image
Data
(copy)
DB Service
Java2SE 1.4
PostgreSQL (DBMS)
PostgreSQL
JVO Server
Monitoring
UDDI4J
UDDI
Data
SUBARU
SupCam
z-band
Image
Data
2Mass DB
get & parse GSDL
pre condition check
execute Service
post condition check
Redhat Linux
SUBARU
DataBase
⑦ issue SQL
JVO Service
⑤ execute Service
③ get GSDL
GSDL
(copy)
④’ Get GSDL
by GridFTP
GSDL
⑤’ execute Service
Check status
Globus Toolkit (Client)
Internet
Globus Toolkit (Server)
⑥’ spawn Service
④ Get GSDL
by GridFTP
JVO Service (image)
AVO
JVO Service
JVO Service (X match)
SUBARU
DataBase
Globus Toolkit
Globus Toolkit (Server)
GridFTP
GSDL
JVO Service
(select)
SQL
library
NVO
⑥ spawn Service
Globus Toolkit
JVO Service
(count)
SQL
library
JVO Service
(select)
SQL
library
JVO Service
(image)
JVO Service
(X match)
JVO Service
(count)
SQL
library
⑦’ issue SQL
Image
Data
Image
Data
(copy)
Copy imate data
by GridFTP
Image
Data
(copy)
DB Service
ORACLE
2018/11/23
GGF7 in Tokyo
The following free software is included by globus toolkit.
OpenSSL (Secure Socket Library)
OpenLDAP (LDAP Server)
wu-ftpd (ftp server)
Copy imate data
by GridFTP
SUBARU
SupCam
i-band
Free Software
In House Software
Image
Data
Commercial Software
Function

20. How it works ? Read JVO QL Create “Work Flow”, and execute each step
When necessary, access to the registry to expand the step of the work flow
Thus the steps increases dynamically
2018/11/23
GGF7 in Tokyo

21. 2018/11/23
GGF7 in Tokyo

22. 2018/11/23
GGF7 in Tokyo

23. Query Files can be stored in a local PC (WS)
2018/11/23
GGF7 in Tokyo

24. Refreshed automatically in every 15 seconds
Monitor Window:
Refreshed automatically in every 15 seconds
2018/11/23
GGF7 in Tokyo

25. Sample Outputs
2018/11/23
GGF7 in Tokyo

26. 2018/11/23
GGF7 in Tokyo

27. FAQs on our Prototype
Q) Contents of environment variables are not passed to the executing machines.
A) Prepare a shell script to get environment variables and another script to execute the command. Then GridFTP these shell scripts , and execute the shells in the remote machine.
Q) It is impossible to prepare all users’ account to all relevant machines.
A) Map all accounts on a portal to a single representative one, such as globus.
2018/11/23
GGF7 in Tokyo

28. FAQs
Q) GT2 has too large overhead before starting actual jobs !! They are several seconds.
A) For batch jobs, such as database accesses, the overhead could be negligible. However users can never wait for several second for interactive data analyses.
Request) Overheads must be < 0.1 seconds
excluding the network delays.
2018/11/23
GGF7 in Tokyo

29. Future Prototype version 2 (in progress) implement data analysis tools
Interoperation with VOs in the world
We have joined the International Virtual Observatory Alliance, and are contributing standardizing protocols for interoperability.
2018/11/23
GGF7 in Tokyo

30. VOs in the World NVO (USA), AVO (EU),AstroGrid (UK)
Germany, Canada, Russia, Australia
India, China, JAPAN
Protocols to interconnect VOs
→　International Virtual Observatory Alliance (IVOA)
2018/11/23
GGF7 in Tokyo

31.
2018/11/23
GGF7 in Tokyo

32. 2018/11/23 GGF7 in Tokyo MVC GRID環境 A GRID環境 C GRID環境 B MVC MVC サービス
天文台殿
MVC
サービス
天体カタログ
天体カタログDB　
MVC
天体カタログ
Web
サーバ
JVO
ポータル
VO用
ブラウザ
セキュリティ管理
資源管理
データ管理
仮想観測 指示
仮想観測実行
サービス
研究者
VO用
Viewer
ワーク
フロー
天文台殿
サービス
利用者独自
サービス
DB/DA
DA/DB　 DB
Data
セキュリティ管理
資源管理
データ管理
サービス
レジストリA
セキュリティ管理
資源管理
データ管理
GRID環境 A
サービス
レジストリC
サービス
レジストリB
サービス
他システム
pipeline 処理
データ管理
資源管理
セキュリティ管理
Web
サーバ VO
ポータル
サービス
他システム 処理
データ管理
資源管理
セキュリティ管理
VO用
ブラウザ
サービス
他システム 処理
データ管理
資源管理
セキュリティ管理
サービス
セキュリティ管理
研究者
VO用
Viewer
資源管理
他システム
データ管理 処理
2018/11/23
GGF7 in Tokyo
GRID環境 C
GRID環境 B

33. ALMA in Chile (sub-mm wave)
2018/11/23
GGF7 in Tokyo

34. JVO Home Page
2018/11/23
GGF7 in Tokyo