1. Dr. Huseyin COTUK ULAKBIM, Turkish NREN 21st May 2014, Dublin Operating Cloud: NREN’s CASE TNC 2014

2. Content Turkish NREN ULAKBIM and TUBITAK NREN Responsibilities Cloud Computing Governmental Services on Cloud OpenStack ULAKBIM Cloud –FATIH Project –PARDUS Project –Other Projects Conclusion

3. TUBITAK roof organization of ULAKBIM is the leading agency for management, funding and conduct of research in Turkey was established in 1963 with a mission of advancing science and technology, conducting research and supporting Turkish researchers is responsible for promoting, developing, organizing, conducting and coordinating research and development in line with national targets and priorities

4. Turkish NREN - ULAKBIM is the one of the 34 members of the NREN consortium was founded as research and development support unit of TUBITAK, in 1996 is responsible for operating the National Academic Network (ULAKNET) and National Science e-Infrastructure (TRUBA) Departments –Network Technologies Unit –Educational Technologies Unit –Cahit Arf Information Center

5. ULAKNET provides network services to more than 200 units including –universities (100K researchers and 2.5M students), –public institutions, –military and police academies –national R&D centers IPv6 enabled backbone TR eduGAIN TR eduROAM Ulak-CSIRT 20 Gbps GÈANT connection (5 Gbps backup) 25 Gbps commercial internet connection via local ISP

6. TRUBA High performance computing Grid services Data intensive computing –Scientific data warehouses Cloud computing services Map/reduce services via Hadoop clusters

7. ULAKBIM EU PROJECT PARTICIPATION Network Projects –GEANT3Plus, GEANT3, BSI (coordinator) and GEANT2 Transition to IPv6 Project –GEN6 Pan-European e-Infrastructure Projects –EGI-InSpire, PRACE2IP, EGEEIII, EGEEII Regional e-Infrastructure Projects –HP-SEE, SEE-GRID-SCI, SEE-GRID2, SEE-GRID, EUMedGrid-Support and EUMED-Grid. Programme and Policy Projects –e-IRGSP3 and SEERA-EI

8. NRENs Responsibilities supporting research and education communities like universities and research institutes have important roles on shaping national ICT strategies important players of innovation platform that encourages the development of new technologies also support academic communities via providing Grid, High Performance Computing (HPC) resources

9. What is Cloud? Cloud computing started with a dream Offering IT services as public utility with a subscription like water and electricity “Computing may someday be organized as a public utility” - John McCarthy, MIT Centennial in 1961.

10. Cloud Statistics $131 billion Estimated global cloud market at 2013 50 million Number of physical servers in the world 84% CIOs who reduced operational costs by moving to cloud 80% CIOs who are delivering at least one of their infrastructure through private cloud 60% CIOs who think that their first priority is cloud 60% Server loads that are virtualized in 2013 27% Governmental institutions that implemented cloud infrastructure in 2012 1/3 Ratio of IT budgets spent for cloud in 2013 48% IT companies that spent for cloud advertisement in 2012 21% Amount of annual saving with the help of apps moved to cloud

11. Why Cloud ? Scalability Cost effective Pay per use Flexibility Self service usage Easy metering High availability Distributed Service oriented Advanced security Green IT (energy saving) Quality of service Continuity

12. Cloud - Risks Data Security and Privacy Service Provider Dependency Management Interface and Remote Access Bandwidth and Data Transfer Software Licences

13. Cloud Models Public Cloud (39% – 32% in 5 years) –No management, maintenance –Many highly-available service providers (Amazon, Google, Rackspace, Microsoft) –Privacy problems Private Cloud (43% – 25% in 5 years) –Security, privacy –Control, cost, and reliability –Management, maintenance, continuity Hybrid Cloud (17% – 43% in 5 years) –Trend towards this model

14. Cloud Service Models Source: http://cloudblueprint.wordpress.com/cloud-taxonomy/ Infrastructure as a Service (IaaS) Hipervisor, virtual machine, storage, network, load balancer, IP pools, OS images,… Platform as a Service (PaaS) Database, web server, development environments, … Software as a Service (SaaS) Email, CRM, ERP, mobile applications, …

15. Government Services on Cloud Governmental institutions are also looking forward to deploying their services on cloud as well Governmental data is taught as valuable enough to conceal from stakeholders most of the governmental organizations prefer to implement these kind of private services on their own data centers

16. Government Services on Cloud (Cont.) they are lacking of required knowledge about innovative technologies they need professional and trustable support for either implementing or managing of these kind of services Rather than using commercial or community driven platforms, getting these services from reliable governmental organizations is a good alternative NRENs could become valuable partners for operating reputable cloud services

17. What is OpenStack? Infrastructure that provides cloud computing technologies all together Open source (Apache License) Widespread hipervisor support (KVM, Xen, ESXi, Hyper-V, LXC, Docker) Written in Python and Django All services provides RESTful web services JSON and XML data format support Public, private, hybrid working models Widespread plugin and driver support

18. Why OpenStack ? 132 country More than 150 vendor support More than 13.000 developers Completely free Different user profiles (service providers, governmental and educational institutions, research institutes, and private companies) Flexible usage, and easy management Stable (9th version) A BCDEFGHI AustinBexarCactusDiabloEssexFolsomGrizzlyHavanaIcehouse

19. OpenStack Developers Rackspace NASA Canonical (Ubuntu) Wells Fargo Bank Dell, HP, IBM, Fujitsu, Hitachi Redhat, Suse eBay Cisco, Juniper Intel NSA Mirantis NetApp, EMC Yahoo Enovance Huawei Comcast Sony PSP4 AT&T Walt Disney Comp.

20. OpenStack in the World Rackspace Cloud ( All services and customers ) NASA ( Nebula ) CERN ( 3500 Hosts, 3 Clouds, 60K Cores, 300K Estimated cores in 2015 ) NSA ( Secure custom version ) Cisco ( Webex ) Intel ( Windows applications ) HP Cloud ( All services and customers ) DreamHost ( IaaS ) RedHat ( Fully Supported, OpenShift) DELL ( OpenStack based projects ) PayPal ( Most of 117 million customers are running on OpenStack ) AT&T ( Large scale projects )

21. OpenStack Components Dashboard: Horizon Identity: Keystone Compute: Nova Networking: Neutron Block Storage: Cinder Image: Glance Object Storage: Swift Orchestration: Heat Telemetry: Ceilometer DBaaS: Trove Common Libraries: Oslo

22. Version History VersionPublishment DateComponents Austin21st October 2010Nova, Swift Bexar3rd February 2011Nova, Glance, Swift Cactus15th April 2011Nova, Glance, Swift Diablo22nd September 2011Nova, Glance, Swift Essex5th April 2012Nova, Glance, Swift, Horizon, Keystone Folsom27th September 2012 Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder Grizzly4th April 2013 Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder Havana17th October 2013 Nova, Glance, Swift, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer Icehouse17th April 2014 Nova, Glance, Swift, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove

23. Horizon customized web based graphical interface for users and administrators virtual machines, OS images, storage, security, network management suitable for adding 3rd party applications developed with Django service interaction via API calls usage statistics

24. Keystone defines endpoints, users, roles, groups, and authorizations on OpenStack. provides role-based access control (RBAC) between users and services. User-password, token, and AWS style authentication support replies requests for services coming from API. Key based access to virtual servers MySQL backend (LDAP alternative)

25. Nova management of computing resources virtualization layer (hypervisor) horizontal scaling transfers incoming new server requests to hypervisor resource templates (flavors) supports live migration supports snapshot power management console access

26. KVM (hipervisor)

27. Virt-manager (GUI for KVM)

28. Neutron nova-network quantum neutron DHCP service, Public IP allocation L3 service (intervlan routing, subnet) Openvswitch, Linux bridge, direct attach network management Bandwidth management Security policies and access rules Widespread plugin support for different vendors Load Balancer as a service (LBaaS) Firewall as a service (FWaaS) VPN as a service (VPNaaS)

29. Cinder manages block storage units provides native API for several known unified storage brand and model allows creating RAW disk drives for services and attaching these disks hot-plug(PnP) Under VM, disk can be formatted by any file system Supports snapshots and recovery

30. Glance manages image services Supports several image formats (qcow, qcow2, vdi, vmdk, raw, vhd, etc.) Object Storage (Swift) service can be used at the backend for storing operating system images images can be modified in accordance with the demands

31. Swift (Object Storage) provides scalable redundant storage does not have traditional file-system structure Objects and files are placed in several servers in a distributed manner Replication and data integrity are provided by OpenStack can be scaled horizontally by adding new servers ideal for images, static content, multimedia and archive files

32. Heat provides template-based orchestration for various cloud services works as template-driven engine for developers and system administrators ensures auto resize and scale Uses OpenStack-native ReST API or CloudFormation-compatible Query API.

33. Ceilometer works as telemetry service contains severals counters related to usage possibility of parametric invoice services allows project-based monitoring with the help of some monitoring tools

34. Trove Target: Scalable and trustable cloud database service Relational and non-relational database support CRUD, access control DB diagnostics Production level usage –Rackspace, HP, eBay Single instance MySQL and Percona support HA and cluster in under development

35. Oslo (Library) contains all the shared libraries and codes that are used by whole OpenStack projects provides consistency and stability between projects

36. Sample Architecture

37. Third Party Applications Savanna (Hadoop as a service) (now Sahara) Murano (Application catalog) Orchestration Automation (puppet, chef, juju, salt) Bare metal provisioning (maas, cobbler, kick start)

38. Amazon WS Compatibility All APIs are fully compatible with –Amazon EC2 –Amazon S3 it is possible to migrate the applications written for Amazon web services with minimal effort

39. ULAKBIM Cloud Test studies were started at the end of 2011 Old generation HPC nodes are used Virtual clusters for grid services on Open Nebula (migration from physical to virtual) IaaS has been offered for 1.5 years OpenStack infrastructure will be extended as part of EGI Federated Cloud (certification process completed)

40. ULAKBIM Data Center

41. ULAKBIM Computing Resources - Servers YearPcsCPU / GPUMemory 2003128 1 core CPU 1 GB 2006256 4 core CPU 4 GB 2007 30 8 core CPU 16 GB 2008 16 8 core CPU 16 GB 2009154 8 core CPU 24 GB 2010120 48 core CPU128 GB 2011192 24 core CPU128 GB 2012 32512 core GPU 24 GB 2013128 16 core CPU256 GB

42. ULAKBIM Computing Resources - Storage YearCapacity 2006 48 TB 2008 560 TB 2009 72 TB (Lustre) 2010 768 TB (Lustre) 2011 480 TB (Lustre) 20121152 TB (ZFS)

43. ULAKBIM Computing Resources - Interconnect YearConnectionPorts 20074x DDR144 20084x QDR648 Infiniband Management- Network PcsConn.SpeedPorts 1 1 Gbps 576 1 1 Gbps 432 20 1 Gbps 48 110 Gbps1152 OR 140 Gbps 288

44. FATIH Project FATIH (Movement of Enhancing Opportunities and Improving Technology) Ministry of National Education one of the biggest education investment projects of Turkey (8 billion TL budget) aims to create opportunities in education and improve technology usage in school –42.000 schools –570.000 classes –10.6 million tablets –470.000 smart boards –705.000 teachers –Distance education centers –Online contents

45. PARDUS Project National operating system of Turkey based on Debian architecture Developed by ULAKBIM KDE and Gnome desktop alternatives Call center support is used in several governmental institutions including Ministry of National Defense Embedded in smart boards used in FATIH Project Embedded Libre Office

46. Other Projects internal projects other TUBITAK institutes universities several research institutions various governmental institutions including several ministries

47. ULAKBIM Cloud Statistics Total ResourcesAmount Total number of cores~ 15.000+ Total amount of memory 100 TB Total amount of storage 2.5 PB OpenStack FatihAmount Total number of VMs170 Total number of cores765 Total amount of memory2 TB Total amount of storage80 TB OpenStack MasterAmount Total number of VMs150 Total number of cores1100 Total amount of memory3 TB Total amount of storage60 TB

48. Challenges Version upgrades (release notes) Maintenance, management, sustainability Security (multi-tenancy) L3 agent high availability issues Bugs and patches Optimization (kernel, user space, network) Problem detection and elimination

49. Troubleshooting https://ask.openstack.org/‎ https://launchpad.net/openstack https://wiki.openstack.org/‎ https://www.openstack.org/blog/ docs.openstack.org https://wiki.openstack.org/wiki/Mailing_Li sts‎ Logs under /var/log directory

50. Results ULAKBIM is working on pioneer studies on cloud computing like as in several services and technologies Server and storage services belong to several projects are offered via this cloud infrastructure Besides, FATIH and PARDUS projects, there are many other projects running on cloud. After implementing production level cloud environment, ULAKBIM has been following the newest technologies and innovations closely.

51. Any Questions? Dr. Huseyin COTUK ULAKBIM, Turkish NREN Chief Researcher huseyin.cotuk@tubitak.gov.tr 00-90-312-298-93-26