1. Fifty Years of Japanese HPC ─From transistor to Exascale─
Yoshio Oyanagi
Education Center for Computational Sciences
Kobe University
2017/4/14
Fifty years of Japanese HPC

2. Fifty years of Japanese HPC
Outline of my talk
Historical overview of HPC in Japan, as contrasted to US, Europe, China, …..
What is the difference between Japan and US trends?
In Japan, big electronic companies manufactured vector and parallel supercomputers besides main frames.
In US, venture companies were active in vector (Cray, Convex etc.) as well as parallel (TMC, FPS, Meiko, nCUBE, KSR, ….) machines.
2017/4/14
Fifty years of Japanese HPC

3. Fifty years of Japanese HPC
“50 Years of HPC” A Column in HPCwire Japan (Since July, Updated every Monday.)
(Sorry in Japanese)
2017/4/14
Fifty years of Japanese HPC

4. Fifty years of Japanese HPC
1960’s in Japan
OKITAC-5090 (1961)
Transistors + Diodes + core memory
Floating point (decimal 10+2)
Add/sub 0.4 ms
Mult ms (peak kFlops)
Not very high performance !!
2017/4/14
Fifty years of Japanese HPC

5. Fifty years of Japanese HPC
1960’s in US
CDC IBM System/360
4 Mflops (peak)
Contract with DARPA for the ILLIAC IV
Pictures from Wikipedia
2017/4/14
Fifty years of Japanese HPC

6. Fifty years of Japanese HPC
1960’s Japan USA
FACOM 270
HITAC 5020, 8000
TOSBAC 3400
NEAC 2200
MELCOM 3100, 9100
OUK 9400
FACOM
IBM 360 model 91 (1967)
IBM 360 model 85 (1968)
Intel founded (1968)
IBM 2938 array proc. (1969)
CDC7600 (1969)
AMD founded (1969)
2017/4/14
Fifty years of Japanese HPC

7. Primordial Ages(1970’s) Japan USA/Europe
1970 FACOM
1977 FACOM APU
1978 HITAC M-180 IAP
1978 PAX project started
1979 HITAC M-200H IAP
1979 MELCOM COSMOIII IAP
1981 MITI Supercomputer Project started(～89)
(green: parallel architecture)
1970 IBM System/370
1971 CDC STAR-100 announced
1972 Goodyear STARAN
1974 DAP, BSP and HEP started
1975 ILLIAC IV operational
1976 Cray-1 shipped to LLNL
1976 FPS AP-120B
1977 Siemens SMS-201
1979 HEP single processor operational
(red: vector architecture)
2017/4/14
Fifty years of Japanese HPC

8. Characteristics of Japanese vectors
1. Japanese vector processors only a couple of years after Cray-1.
2. Manufactured by main-frame vendors with semiconductor facilities (not ventures)
Vector processors are attached to mainframes
3. FACOM APU (first Japanese vector)
a) CPU and APU share the main memory (1 MW×36bits) Heterogeneous architecture
b) Vector register of 1792 words
c) list vector and DO-loop with IF supported
d) 22 Mflops (single), 11 Mflops (double)
e) installed in NAL in August 1977
2017/4/14
Fifty years of Japanese HPC

9. Characteristics of Japanese vectors
4. HITAC IAP (1978-)
a) memory-to-memory (no vector registers)
b) summation, inner product and 1st order recurrence could be vectorized in FORTRAN
c) vectorization of loops with IF’s (M280)
d) M-200H IAP (1979): 48 Mflops,
M-280H IAP (1982): 67 MFlops
5. NEC ACOS 1000 IAP (1982)
a) 36 bit machine
b) peak 28 MFlops
2017/4/14
Fifty years of Japanese HPC

10. 1st Generation (1H of 1980’s) Japan US/others
1980 PAX-32
1981 MITI Supercomputer
Project (～89)
1982 Fifth Generation Project (～92)
1982 NEC ACOS-1000 IAP
1982 HITAC M280H IAP
1983 HITAC S-810/20 (630 MF)
1983 FACOM VP-200
1984 PAX-64J
1985 FACOM VP-400 (1.142 GF)
1985 NEC SX-2 (1.3 GF)
1981 CDC Cyber 205
1982 Cosmic Cube Project
1982 Cray X-MP/2 (420 MF)
1982 Alliant FX/8 delivered
1982 HEP installed
1983 Encore, Sequent and TMC founded,
1983 ETA span off from CDC
1984 Cray X-MP/4 (820 MF)
1984 Legend group in China started
1985 Convex C1
1985 Intel iPSC/1, T414, nCUBE/1, Stellar, Ardent
1985 Cray-2 (1.952 GF)
1986 CM-1 shipped
1986 FPS T-series
2017/4/14
Fifty years of Japanese HPC

11. Characteristics of Japanese SC in the 1st Generation
1. Compatibility with the mainframes
2. Single processor with multiple pipes
3. Large main memory (256MB vs. X-MP 32MB)
4. Large vector registers
5. List-directed vector instructions (gather/scat)
6. Different control of vector units
7. No commercial parallel machines
2017/4/14
Fifty years of Japanese HPC

12. Characteristics of Japanese SC’s in the 1st Generation
8. Aggressive installation of SC’s in universities and laboratories
7 SC centers open to all university people
Laboratory SC’s for physics, chemistry, …
In US, most SC’s are in DOE, DOD, NASA, not available for university researchers
Lax Report (1982)
5 NSF centers in
2017/4/14
Fifty years of Japanese HPC

13. Characteristics of Japanese SC’s in the 1st Generation
9. MITI Supercomputer Project ( fy)
Started before the Japanese supers (S810, VP, SX)
Targets:
10 GFlops parallel vector machine (PHI),
Dataflow machine (sigma-1)
Josephson device, MPP etc.
In alliance with six companies
2017/4/14
Fifty years of Japanese HPC

14. 2nd Generation (2H of 1980’s) Japan US and Europe etc.
1986 “863” Plan in China
1986 Manheim Supercomputer seminar (→ISC)
ETA-10 (10 GF)
1987 CM-2
1988 Cray Y-MP (2.66 GF)
1988 Intel ipsc/2
1988 First Supercomputing Conference in Orlando
1989 ETA shut down, JvN SC shut down
1989 BBN TC2000, Myrias SPS-2, Meiko CS, NCube2
1990 Intel ipsc/860, MasPar MP-1
1987 HITAC S-820 (3 GF)
1989 FACOM VP2600 (5 GF)
1990 MITI Supercomputer Project ended
1990 NEC SX-3 (22 GF)
1990 QCDPAX completed
2017/4/14
Fifty years of Japanese HPC

15. Characteristics of Japanese SC in the 2nd Generation
1. Vector multiprocessor appeared in Japan
with modest multiplicity 2-4 vs. 8 (Y-MP, ETA)
2. The semiconductor technology developed for the vector is transferred to mainframes
(Used to be: mainframe→vector)
3. Still no commercial MPP’s in Japan, although parallel research was active in academia
2017/4/14
Fifty years of Japanese HPC

16. Installation of SC’s in Japan (1987-8)
Cray Research (Cray-1, X-MP):10
ETA (ETA-10): 1
Hitachi (S810):15
NEC (SX-1/2):7
Fujitsu (VP-50, 100, 200, 400):36
2017/4/14
Fifty years of Japanese HPC

17. US-Japan Trade Conflicts
1985/9: Plaza Accord (G5)
1985: SX-2 was cancelled by NCAR after bidding
1986/9: US-Japan Semiconductor Agreement
1987: Japanese SC was cancelled by MIT after bidding
2017/4/14
Fifty years of Japanese HPC

18. Fifty years of Japanese HPC
Super 301 Act
In 1989, US government decided to apply “Super 301 Act” (Omnibus Trade and Competitiveness Act of 1988) to Japan and identified three items including supercomputers.
Washington threatened Japanese governmental institutions to purchase US supercomputers. (Titech ETA10, ETL X-MP)
On the other hand, US governmental institutions had no Japanese supercomputers at all.
2017/4/14
Fifty years of Japanese HPC

19. 3rd Generation (1H of 1990’s) Japan USA/Europe
1992 RWCP started
1992 CP-PACS started
1993 Fujitsu NWT
1993 Fujitsu VPP500
1993 HITAC S-3800 (32 GF)
1993 NEC Cenju-3
1994 Fujitsu AP1000
1995 NEC SX-4
1990 MasPar MP-1
1990 Intel iPSC/860
1991 HPCC started (-96)
1991 Cray Y-MP C90 (16 GF)
1991 Intel Paragon, TMC CM-5
1992 FPS bankrupt
1992 MasPar MP-2
1993 Top500 started
1993 HPCN started (～2001)
1993 Cray T3D, CS6400, nCUBE2, KSR1
1993 SSI shut down
1993 IBM SP-1
1993 Cray-3
1994 SP-2
1994 TMC, KSR Chapter 11
2017/4/14
Fifty years of Japanese HPC

20. Characteristics of Japanese SC in the 3rd Generation
Drastic changes in Japanese vectors
1. Hitachi (S3800)
Shared memory vector parallel processor
using ECL technology
2. Fujitsu (VPP500)
Distributed memory vector parallel proc.
using GaAs as well as silicon tech.
3. NEC (SX-3)
Cluster of shared memory vector parallel
Unix as host OS
4. No action to “Attack of killer micros.”(SC90)
◆ Commercial MPPs, sold as a parallel testbed
2017/4/14
Fifty years of Japanese HPC

21. Trends in US (1st half of 1990’s)
Active vector machines: Cray Y-MP, C90 and Convex C2
MPP:
Performance of COTS CPU’s was increasing drastically
custom CPU → commodity CPU chips
ventures company → big companies (IBM, Cray, Intel)
Strong national initiative: HPCC, NII, HPC Act 1991
2017/4/14
Fifty years of Japanese HPC

22. Fifty years of Japanese HPC
HPCC in US
Blue Book (1991/2, G.W.Bush)
“Grand Challenges: High Performance Computing and Communications”
The High Performance Computing Act of 1991
Trade conflicts:
“Japan should buy more US supers” (M. Kantor, 1993/4) Otherwise, possible retaliation.
6 out of 11 in public sector bought US machines in the 1994 supplementary budget.
In Europe, Rubbia Report (1991/1)
European Teraflop Initiative
2017/4/14
Fifty years of Japanese HPC

23. 4th Generation (2H of 1990’s) Japan USA/Europe/China
1995 DOE: ASCI started
1995 Cray T90 (57.6 GF)
1995 Cray Computer bankrupt
1996 Dawning in China started
1996 Cray T3E
1996 Cray Research merged into SGI
1996/10 Serymour died
1996 nCUBE merged into Oracle
1996 MasPar went out of HPC
1997 ASCI Red (Intel)
1997 NSF: NPACI started
1998 Cray SV1
1998 ASCI Blue Pacific (IBM)
1998 ASCI Blue Mountain (SGI)
1995 Fujitsu VPP300
1996 cp-pacs completed
1996 Hitachi SR2201
1996 Fujitsu VPP700
1996 Fujitsu AP3000
1997 NEC Cenju-4
1998 Hitachi SR8000
1998 NEC SX-5
2017/4/14
Fifty years of Japanese HPC

24. Characteristics of Japanese SC in the 4th G.
1. Fujitsu and NEC seem to follow the line of conventional vector supercomputers in CMOS
Fujitsu: distributed memory
NEC: cluster of shared memory nodes
2. Hitachi adopted RISC (pseudo)vector architecture.
SR2201: sliding window (proposed by Tsukuba) SR8000: 8 tightly coupled CPUs in one node
(self-developed CPU with Power architecture)
2017/4/14
Fifty years of Japanese HPC

25. Fifty years of Japanese HPC
US Trends
ASCI Project (originally )
SC’s in LANL, LLNL, SNL
Red (1+), Blue (3+), White (10+), Q(30+), Purple (100+) --- every 2 years
PITAC (I: , II: )
IT2 project
NPACI ( )
Petaflops I in Pasadena (1994) Petaflops II in Santa Barbara (1999)
2017/4/14
Fifty years of Japanese HPC

26. Fifty years of Japanese HPC
NCAR Procurement
1996/5/20 UCAR selected SX-4 for NCAR
finalists were C90, SX-4, VPP700
Criticism by Cray and Obey
Tax from US people should be used for US competitiveness
NEC (and Fujitsu) was dumping
1996/7/29 Cray filed an apeal to DoC and ITC for dumping
1997
454% anti-dumping customs to NEC supercomputers
388.7% to Fujitsu
2017/4/14
Fifty years of Japanese HPC

27. Fifty years of Japanese HPC
Japanese machines in Top 20
2017/4/14
Fifty years of Japanese HPC

28. Fifth Generation (1H of 2000) Japan US/Europe
2000: Tera became Cray
2000: SGI: Origin 3000
2000: ASCI White (LLNL, IBM) 12 TF
2001: NSF: TeraGrid stared
2002: ASCI Q (LANL, HP) 20TF
2002: DOD: HPCS started
2002: Cray (NEC) SX-6 to ARSC
2004: NASA Columbia (SGI) 64TF
2004: BlueGene/L at IBM 90TF
2004: “The Path to Extreme Computing” conf. (to Exa)
2005: BlueGene/L (360TF)
2005: ASCI Red Storm (SNL) Cray XT3/XT4
2001: NEC: anti-dumping customs cleared
2001: Fujitsu PRIMEPOWER 2000
2001: NEC SX-6
2002: Earth Simulator 40TF
2002: Fujitsu PRIMEPOWER HPC2500
2003: NEC SX-7
2004: NEC SX-8
2004: Hitachi SR11000
2017/4/14
Fifty years of Japanese HPC

29. Fifty years of Japanese HPC
Japan Trends
NEC kept making vector machines after the Earth Simulator
Fujitsu: PRIMEPOWER (Sparc) and PRIMERGFY (x86)
Hitachi: OEM of IBM servers. SR11000 (Power4+, 6.8 GF/CPU)
NAREGI (2003-8): Grid project
2017/4/14
Fifty years of Japanese HPC

30. Fifty years of Japanese HPC
Japan Trends
IT Strategic Headquarter (2001): e-Japan, but only network was emphasized.
At this stage, level up of supercomputers was to be promoted according to the needs of each field (not a national project).
Earth Simulator attained 36 Tflops (2002)
Information Science and Technology committee in Mext has been discussing the measures to promote computational science and technology since August 2004.
2017/4/14
Fifty years of Japanese HPC

31. Japanese machines in top 20
2017/4/14
Fifty years of Japanese HPC

32. Fifty years of Japanese HPC
US Trends
DOE: ASCI continues
BlueGene and Red Storm added
NSF: Teragrid ( )
DOD: HPCS (High Productivity Computing Systems)
2002: Phase I (IBM, Cray, Sun, HP, SGI)
2003: Phase II (Cray, IBM, Sun)
2006: Phase III (Cray: XC, IBM: PERCS)
HEC Revitalization Act of 2004 and 2005
Exa started in “The Path to Extreme Computing” in Santa Fe (2004/10)
2017/4/14
Fifty years of Japanese HPC

33. Sixth Generation (2H of 2000 and later) Japan US/Europe/China
: K Computer P.
2011: 10 PF attained
2006: T2K open SC
2008: installed in 3 univ.
2006: TSUBAME 1.0
2010: TSUBAME 2.0
2009: Fujitsu fx-1 to JAXA
Cray XT/XE/XC
2005: ASCI Purple (LLNL, IBM)
2006: Rangers (TACC)
2007: NCSA, BlueWaters started (IBM→2011 Cray)
2007: Kraken (Tennessee)
2008: Roadrunner (LANL)
2010: Tianhe-1A (NUDT)
2010: PRACE started
2011: NSF, SXEDE
2013 America COMPETES Act
2017/4/14
Fifty years of Japanese HPC

34. History of the K Computer
Recommendation to a Mext committee (2005):
Promote a national project to construct a leading edge supercomputer
Government decision (July 25, 2005)
Riken started the project (October 2005)
Mext funded four projects to promote “Element Technologies for Future Supercomputers” in $40M per year (in total)
Four groups were accepted
System Interconnect (Kyushu U and Fujitsu)
Interconnect by IP (U of Tokyo, Keio U etc)
Low Power Device and Circuits (Hitachi, U of Tokyo, U of Tsukuba)
Optical Connection of CPU and Memory (NEC and Titech)
2017/4/14
Fifty years of Japanese HPC

35. History of the K Computer
2005: Proposed architectures and killer applications
2007: Hybrid machine with vector and scalar
2009: Five strategic application fields defined
2009: Withdrawal of vector machine
2009: the Government Revitalization Unit proposed to shutdown the Next Generation Supercomputer Project (Nov. 13, 2009)
2009: HPCI started (High Performance Computing Infrastructure, alliance of supercomputer centers and users)
2017/4/14
Fifty years of Japanese HPC

36. Fifty years of Japanese HPC
Original Proposal
Scalar computer
Large scale processing
Special-purpose computer
2017/4/14
Fifty years of Japanese HPC

37. Fifty years of Japanese HPC
Friday, November 13, 2009
The 3rd WG of the Government Revitalization Unit
“Why should it be No.1 in the world?”
“Is No.2 not enough?”
Vote：
Abolish １
Postpone ６
Budget shrink ５
Conclusion
Freeze the project!
←村田(謝) 蓮舫(Hsieh Lien Fang)
Picture from
2017/4/14
Fifty years of Japanese HPC

38. Fifty years of Japanese HPC
No.1 in the world
8.162 Pflops attained using 80% of the systemーNo.1 in the Top500 (ISC2011) June 20, 2011
10.51 Pflops in SC2011 (Seattle)
Open to users: September 2012
1st period: 2012/9 to 2014/3
2nd period: 2014/4 to 2015/3
RIST is to manage K Computer users (Research Organization for Information Science & Technology)
2017/4/14
Fifty years of Japanese HPC

39. What’s next? Toward EXA FLOPS!!
Preliminary consideration among scientists (In US, since 2004)
Mext started a WG for future HPC (April 2011)
Hardware-System Software-Application co-design is important.
Should be science-driven.
We identified possible break throughts in science and technology. Social needs are also considered.
Linpack Exaflops is not our target.
Limitation by budget, power, ….
Several different architectures are considered.
2017/4/14
Fifty years of Japanese HPC

40. Fifty years of Japanese HPC
Architecture
Application
Algorithm
2017/4/14
Fifty years of Japanese HPC

41. Fifty years of Japanese HPC
Two subgroups worked
Application Subgroup
Application
Numerical library
Algorithm
Automatic tuning
System Subgroup
CPU and architecture
Compiler
System software
Final Report in March 2012
2017/4/14
Fifty years of Japanese HPC

42. Memory and memory bandwidth Big technical issueES
SoC
Large B/Flop
Small system K
standard
0.1
Small B/Flops
Large B/Flops
0.1
GPU
Small B/Flop
2017/4/14
Fifty years of Japanese HPC

43. WG for Future HPCI Policy
2012/4～2014/3 (25 meetings)
26 members, Chair: Oyanagi
Recommendations
Hierarchical deployment of supercomputers
Leading machines: Flagship system and its complementary systems
Flagship: “exascale” machine toward 2020
Support to users, especially industry users
2017/4/14
Fifty years of Japanese HPC

44. Complementary systems to the flagship
Flagship system
Leading systems
Allocated thru HPCI
Complementary systems to the flagship
Universities and laboratories
2017/4/14
Fifty years of Japanese HPC

45. Development plans of exascale machines in the world (preliminary)
　　基本設計　　　　　　　試作・詳細設計　　　　　製造（量産）　設置・調整　　　　運用
エクサ向けアプリケーション開発・利用
システム・ノード設計　　　　実験機　　　　　　　　　　　システム製造・設置
Ｐ環境設計　P環境構築　　　　　　↓アプリ開発開始　　　　　　　　　　　　↓P環境更新　　　
8th Framework Program (FP8) Horizon 2020
第12次5カ年計画　　　　　↓100 PF　　　　　　　　　　第13次5カ年計画 EF ↓
2017/4/14
Fifty years of Japanese HPC

46. 9 Key topics for the post-K (Komiyama Committee)
Healthy longevity
Innovative drug design
Integrated computational bioscience
Disaster mitigation and environment
Earthquake and tsunami
Advanced prediction of climate and environment
2017/4/14
Fifty years of Japanese HPC

47. 9 Key topics for the post-K
Energy problem
5. New technology for energy production, storage and usage
6. Innovative clean energy system
Industry competitiveness
7. New functional devices and materials
8. Innovative design and manufacturing
Basic science
9. Fundamental law and evolution of the universe
2017/4/14
Fifty years of Japanese HPC

48. Observation of Japanese HPC (1/3)
Japan has manufactured vector supercomputers since 1977
Japanese vector computers were designed as an extension of main frames.
In the early stage, development cost was amortized among mainframes (US-Japan conflicts).
Stress of easiness of use (compilers and tools)
Very good vectorizing compilers.
Users exploited the power of vectorization.
Users were spoiled by them.
Application software development not enough.
2017/4/14
Fifty years of Japanese HPC

49. Observation of Japanese HPC (2/3)
Until late 1990’s, Japanese vendors focused on vector machines. Late entry to parallel architectures.
Vendors thought parallel machines were for specialized purposes (eg. image processing).
Most users dared not try to harness parallel machines in the 80’s.
Users found difficulties in using message passing.
Some computer scientists were interested in building parallel machines, but they were not used for practical scientific computing.
2017/4/14
Fifty years of Japanese HPC

50. Observation of Japanese HPC (3/3)
Practical parallel processing for scientific computing was started by application users: QCDPAX, NWT, CP-PACS, GRAPE’s, ES.
After the K Computer, parallel users increased not only in academia but also in industry
We must prepare for the post-K exascale.
Hierarchical deployment
Home development
Co-design, Science-driven
2017/4/14
Fifty years of Japanese HPC

51. Fifty years of Japanese HPC
Thank you
for your
attention
2017/4/14
Fifty years of Japanese HPC