Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduce yourself Presented by

Similar presentations


Presentation on theme: "Introduce yourself Presented by"— Presentation transcript:

1 Transition in Campus CyberInfrastructure: Community Clusters, Storage and Co-Loc
Introduce yourself Presented by Dwight McKay, Director of Systems Engineering ITaP Rosen Center for Advanced Computing Purdue University

2 Introduction Community Clusters Summarize model and issues
Outline Introduction Community Clusters Summarize model and issues Perspective is that of an infrastructure builder / operator Focus on computational and storage allocation, growth strategies, funding Start with our structure and the sea change we see as we move from a centrally funded world to being a resource provider on-campus and beyond

3 Introduction Community Clusters Summary
This is the overall structure of ITaP Introduction Community Clusters Summary

4 Introduction Community Clusters Summary
The Rosen Center is one of five business units. We focus on the cyberinfrastructure needs of the campus and beyond Introduction Community Clusters Summary

5 Rosen Center for Advanced Computing
This is the structure of RCAC Talk about each sub-unit Introduction Community Clusters Summary

6 Rosen Center for Advanced Computing
This is the structure of RCAC Talk about each sub-unit Introduction Community Clusters Summary

7 Transition in Research Computing Support
Move from the center to a service, a collaborator Our history has been to share systems, first with job scheduling, then as we transitioned out of the centrally funded realm we built clusters from old lab systems and then moved into the condo model that we call community clusters. Introduction Community Clusters Summary

8 Transition in Research Computing Support
Change in Direction From central purchase to researcher / project purchase From central shared facility to resource or service provider From service desk to partner Move from the center to a service, a collaborator Our history has been to share systems, first with job scheduling, then as we transitioned out of the centrally funded realm we built clusters from old lab systems and then moved into the condo model that we call community clusters. Introduction Community Clusters Summary

9 Transition Implications
Paying Customers Higher Expectations Service & Support Formal Agreements Cultural Change This transition is the driver for moving into new models of systems acquisition, resource allocation and collaboration The biggest change is that we now have customers explicitly paying for services Introduction Community Clusters Summary

10 Rosen Center for Advanced Computing
While we are structured into specific areas, the boundaries between these areas are more fluid than this diagram suggests. The structure is more of a matrix with projects and people spanning across the reporting boundaries as needed to support our customers. Also note that we incorporate a research group as well as user support. Research User Support Infrastructure Introduction Community Clusters Summary

11 Rosen Center for Advanced Computing
Project A typical project has connections into both computational infrastructure, accounts, queues, etc. AND high level support, consultation, code optimization, application support, etc. Larger, more complex projects often pull in larger sets of resources, such as project specific WAN links, project management, software development and custom infrastructure design and deployment. Project Introduction Community Clusters Summary

12 Rosen Center for Advanced Computing
Our teams have people who span groups. A person reporting to Seb to manage a system in a grid project would also participate in the system team, come to our meetings, work and act like a member of our team to provide a close connection and better achieve the support needs the grid project needs. We also embed people into research teams to provide IT expertise needed to move a project along. Project Introduction Community Clusters Summary

13 Transition Implications
New Business Models Needed HW/SW, infrastructure, support? Unpredictable Demand & Funding Planning for power/space/cooling? Non-paying “Users” How do we pay for hardware, software, services, people in this new environment? If we are not centrally funded, how do we predict the demand we will see from our customers? What about our “general” users; those who did not buy into our services? Introduction Community Clusters Summary

14 Custom Arrangements for Specific Projects
Community Clusters Condominium Model Purchase Computation “by the node” Nodes come with bundle of services Purchase Storage “by the TeraByte” Storage comes with bundle of services Custom Arrangements for Specific Projects Introduction Community Clusters Summary

15 Services Community Clusters HW Installation HW Maintenance
Facilities Support Network Connection Infiniband Connnection OS installation and management Disk Storage Archival Storage On-Call Support Disaster Recovery Security Introduction Community Clusters Summary

16 Services Node Bundle Community Clusters HW Installation HW Maintenance
Facilities Support Network Connection Infiniband Connnection OS installation and management Disk Storage Archival Storage On-Call Support Disaster Recovery Security Node Bundle Introduction Community Clusters Summary

17 Tiered Cycle Allocation
Community Clusters Tiered Cycle Allocation Owners guaranteed specific share Owners given first pick of idle cycles Owners agree to harvesting remaining idle cycles “Use the whole buffalo.” -- Brad Bird Brad Bird is the director of “The Incredibles”. Important for space/power/cooling, non-paying customers, cycles for Grid users Introduction Community Clusters Summary

18 Tiered Cycle Allocation
Community Clusters Tiered Cycle Allocation Owner Introduction Community Clusters Summary

19 Tiered Cycle Allocation
Community Clusters Tiered Cycle Allocation Owner Pre-empt Introduction Community Clusters Summary

20 Tiered Cycle Allocation
Community Clusters Tiered Cycle Allocation Owner Mention Condor here. Cycle Harvesting Pre-empt Introduction Community Clusters Summary

21 Community Clusters Challenges
Business model that spans system generations & recovers shared infrastructure costs Cluster Heterogeneity Multiple communities sharing one cluster (TeraGrid, OSG, NWICG) Other architectures & special needs? We are heading towards the end of a three year cycle on cluster building. How do we do retirement? How do we pay for interconnection infrastructure and upgrade it over time? What do we do when the particular node we use is nolonger available? What about those folks who need a shared memory system? Introduction Community Clusters Summary

22 Storage Community Clusters Three Primary Tiers
Fast for scratch files Commodity for home directories Archival for “longer term” storage Custom Storage for Specific Projects Introduction Community Clusters Summary

23 Storage Challenges Community Clusters
Business model that spans lifetime of data / researcher? Media Purchase vs. Space Rental? Data Retention Policy Initially we had researchers buy disk trays. But storage technology is progressing and there’s a potential danger in being stuck maintaining old storage. How long to keep something and how do we help a research take his data with him when he might have 10s to 100s of TB? Introduction Community Clusters Summary

24 Connectivity Challenges
Community Clusters Connectivity Challenges Multiple classes of network are needed Direct Routes Data Center to Key Research WANs Data Center to Key Research Labs Introduction Community Clusters Summary

25 Introduction Community Clusters Summary

26 Four Fold Network Community Clusters Commodity Secure Research
High Performance / Large Data Network Research Low Latency Introduction Community Clusters Summary

27 Summary Transition to resource provider / partner / service
Architecture Computation -> Community Clusters Storage -> 3 Tiers Connectivity -> Direct Data Center connections to Research WAN and Lab Access Challenges Customer expectations and implications Serving up storage and other architectures “by the slice” Recovering ALL the costs, especially support Introduction Community Clusters Summary


Download ppt "Introduce yourself Presented by"

Similar presentations


Ads by Google