Presentation is loading. Please wait.

Presentation is loading. Please wait.

Requirements and Services: The Application Point of View Jennifer M. Schopf Argonne National Lab (Steven Newhouse, OMII)

Similar presentations


Presentation on theme: "Requirements and Services: The Application Point of View Jennifer M. Schopf Argonne National Lab (Steven Newhouse, OMII)"— Presentation transcript:

1 Requirements and Services: The Application Point of View Jennifer M. Schopf Argonne National Lab (Steven Newhouse, OMII)

2 The Never Ending Road Trip Jennifer M. Schopf Argonne National Lab (Steven Newhouse, OMII)

3 What If… l We had a Grid based around scientists, not scientists using what they had to from the Grid l Tools fit into a users normal environment with little or no adaptation in work style l There were simple, reliable, persistent services that were easy to use So why aren’t we here already?

4 The Need: l Several projects developing or delivering middleware without clear guidance l Users and uses have grown and changed l Middleware has grown and changed l (warning needs to be spread of Steven’s move and that fact that I’m spending a year in the UK ….) l Vital need for information exchange at this juncture

5 And Our Guest Today Is… l Many current projects, we’ve focused on those in the UK due to OMII mission (and lack of US understanding into these) l Current application developers with some Grid or Web Services experience l Those with software that might be of broader use or interest l Those who have expressed dissatisfaction with current tools

6 So Far l Oxford Security Workshop l Networking for Non-Networkers Workshop (NNFN) l T. Cooper-Chadwich, Southampton, gYacht l S. Cox, Southampton, GeoDise l M. Daw, Manchester, AG l M. Giles, Oxford, gViz l C. Goble, Manchester, MyGrid & Integrative Biology Project l S. Lloyd, Oxford, eDiamond l J. MacLaren, Manchester, UoM Broker l A. Martin, Oxford, ClimatePreciction.NET l M. McKeown, Manchester, OGSI:lite and WSRF:lite l S. Pickles, Manchester, TeraGyroid & GRENADE l A. Porter, Manchester, Reality Grid l A. Rector, Manchester, CLEF l M. Rider, Manchester, eViz

7 Still To Come: l Edinburgh l Glasgow l University College London l Imperial College l Southampton (again) l … and others by request

8 The Results So Far l Need for Training l Security l Functionality u Jobs u Data u What isn’t mentioned l What tools should look like l Infrastructure/Operations

9 Training l Grid vision still needs to be sold – u “What do these tool give me over SSH, scp?” u “What if I don’t want to stop using my magnifying glass to read x-rays?” l Still need basic common practices to be written: for user, developers *and* admins u Web service basics u Firewalls u Builds and packaging l No surprise: Communication is still a large unsolved problem in Grid computing

10 Security: What’s Old News l If it isn’t easy users aren’t interested l All users hate firewalls, all system administrators love them l Anonymizing data is hard l Still need a lot of information sharing: u How fire walls interact with Grid/Web services u Security audits

11 Security: What’s Surprising Us l User focus on need for data integrity not authentication/authorization u Time and again this was mentioned l Delegation seems to be the next big question u GT2-style delegation needed in a services world u No one has an agreed upon solution yet

12 Questions? l Need for Training l Security l Functionality u Jobs u Data u What isn’t mentioned l What tools should look like l Infrastructure/Operations

13 Functionality l Prediction of Pete Beckman (TeraGrid): u All users want is SSH, scp, and top

14 Functionality l All users talk about is job submission and file transfer capabilities l When asked about trouble spots they also want tools to tell how tools are progressing l Occasionally would like an application specific service, like a viz tool l Many other tools are on the horizon, but they simply don’t seem to be on the 6 month horizon for the users we’ve spoken with to date

15 Job Submission: No Surprises l Want simple, dependable “run my application” interfaces u This was identified at GF1! l Only resource discovery is “small” u How many nodes have a matlab license? u NOT: which cluster should I use?

16 Job Submission: Urgent Needs l Tools to understand errors while a job is running- something stopped, where and why? u TeraGyroids’ use of SSH for debugging u Need for Global Job Unique ID l What to do when a job fails- u Resubmit or ignore? u Workflow issues u Steering

17 File Transfer l People seem pretty happy with GridFTP l Some reliability (RFT) would fill out rest of use u This needs 3 rd party transfer (delegation) l Some projects starting to work with provenance issues or access to databases l Still issues with performance and making sure background infrastructure is all as it should be – more later

18 What (These) Users Aren’t Talking About… l Notification – except for job progress tracking l Registries or resource discovery l QoS, reservations, brokering, advanced scheduling techniques l Job migration, checkpointing l Accounting and pricing (but we’re talking with users, not admins so far) l Data replication, migration l Instruments

19 …And Why We Think This Is l A gap still exists between the computer science research and tool building community and the average user l Large difference between short term needs and long term planning- u Most users are still trying for basic functionality and dealing with today’s hurdles u Most researchers are looking at the greener pastures a few years out

20 Questions So Far? l Need for Training l Security l Functionality u Jobs u Data u What isn’t mentioned l What tools should look like l Infrastructure/Operations

21 User’s View of Tools l Users have strong opinions on tools – what a surprise! l Mostly known problems, but the prioritization of certain aspects wasn’t known

22 Tools Should Be l Vertical solutions u End to end use cases, not horizontal pieces that don’t work together l Simple u One job, one tool – think unix! u Work easily for the 80% case, and rest is possible if needed l Ease of use/installs u Bundle all together so you have entire use together u Don’t reinstall things I already have

23 Sometimes There Is No “Best” Tool l Same workspace may have many solutions, each with slightly different attributes, all valuable u BPEL vs SCUFL u Cluster monitors l Users should not be forced to use a tool from another environment if there is no underlying scientific need for it l However, we still need to try to avoid replication of tools

24 Composable Functionality l Lego blocks of basic functions l “Shims” to fit between where needed u API mismatches u Data translation u Interfaces to legacy code u SOAP Lab (wraps command line to look like SOAP)

25 Interfaces l Need simple APIs at the user level u eg. SAGA-RG (but then we knew this) l User API might sit a layer above standard tool APIs to mitigate upgrade effects u eg. HiCog l The API a user sees and the API the infrastructure not only can but should be different – different goals

26 Environments l Tools need to fit in with existing “user comfort zone” u Biologists like Perl u CFD folks like MatLab u HEP (EGEE) are used to Python l This is a sys admin’s and tool developer’s nightmare – but for usability it’s a must

27 Questions? l Need for Training l Security l Functionality u Jobs u Data u What isn’t mentioned l What tools should look like l Infrastructure/Operations

28 Builds and Upgrades l Need for a reproducible build u Hands off process, works every time u Verification tools l Need better understanding of effects of upgrades, or else users don’t want it u What will change, what will be affected l Tools are being used “off label” u Tool for usecase A in common use for usecase B u Scalability becomes an issue u New/different functions needed

29 Understanding System Stability l Need for basic tools to verify functionality and performance l “I can’t transfer my files today” u What’s broken? u What changed? u How do I fix it? u Why couldn’t someone find this before me? l Strong needs for quality assurance tests on all platforms – clusters, networks, AG, etc.

30 System Tests l Often system tests don’t look like current applications u Tests for firewall functionality don’t include checks for all ports in current use u System benchmarks don’t look like “my” application u Ping tests aren’t enough to assure that a GridFTP transfer will work l Need for better testing, verification- for the user, and even by the user!

31 WebMD for Applications l Basic diagnostics needed for users l Q - “I’m trying to transfer a 1 gig file between A and B and can’t” l A- “Is your cert ok? Here’s how to check” Y/N, if Y… l A – “Is the route between you’re hosts up? Here’s how to run traceroute for your system…”

32 Topics of Conversation l Need for Training l Security l Functionality u Jobs u Data u What isn’t mentioned l What tools should look like l Infrastructure/Operations

33 Our Conclusions… So Far l Delegation l Job tracking l Easing tool use u Wrappers u Composability l Verification and instability analysis l Diagnosis

34 The Point l We can’t say it any more simply l Grid tool developers must continue to talk and interact with application scientists – without them, we are nothing

35 How To Find Us: l Jennifer M. Schopf u u Will be in Edinburgh l Steven Newhouse u


Download ppt "Requirements and Services: The Application Point of View Jennifer M. Schopf Argonne National Lab (Steven Newhouse, OMII)"

Similar presentations


Ads by Google