1. 1 G2 and ActiveSheets Paul Roe QUT p.roe@qut.edu.au Yes Australia!

2. 2 Overview: Two Projects 1.G2:.NET framework which makes cycle stealing easy 2.Active sheets: Spreadsheets for cluster / grid computing Underlying theme: Databases are cool!

3. 3 G2 Cycle Stealing Framework

4. 4 Cycle Stealing Requirements Simplicity No new libraries or prog models No complex deployment No volatile workstations to consider Maximise volunteers (minimise volunteer requirements) System must be transparent Application programmer/ user Volunteer (user / sys admin) Security No extras to install No extra sys admin No interference Eye candy! System must be transparent.NET CLR and Frameworks G2 framework

5. 5 Client SOAP HTTP G2 Virtual Server Internet Code G2 Virtual Web Service Server Functional (dataflow) model of computation based on WS Virtual high performance web service server G2 programming model identical to.NET web services Code and data automatically and transparently distributed volunteers G2 Job DB Data

6. 6 G2 leverages.NET.NET CLR provides sandbox security –Jobs run within IE Use Windows scheduler to detect idle PC Standard web service communications, extended to support: –G2 intermediary server –Web service caching / persistence –Reliable messaging Code deployment (download) –Extended to support automatic code upload to server Isolated storage, extended to support: –Automatic and transparent file caching –Job checkpointing Visual studio plug-in to support writing G2 components “add G2 reference” vs “add web reference” SQL server database to store jobs (persisted web services) on server –DB provides reliability, assume volunteers and clients may fail

7. 7 G2 Family G2 Classic G2 P2P –Peer to peer –Distributed objects –Uses remoting rather than web services –Custom remoting channel G2X –XML/Yukon version of G2 –Supports querying of database G3 –Adaptive G2 –Combines P2P, G2X and dedicated cluster

8. 8 Status ‘Local’ applications –Data mining –Code cracking – brute force cipher analysis –GA: VLSI layout Lot of performance tuning, scheduling etc. G2 classic: general release soon, seeking early users Formal model of performance g2.fit.qut.edu.au

9. 9 ActiveSheets Vision Spreadsheet driven HPC = accessible HPC Spreadsheets powerful and popular end user tools No parallel programming or grid expertise required Good pre and post processing of data, charting etc.

10. 10 Parallel Computing for Users Spreadsheets: dataflow and data parallelism Parallelism from data and independent functions –Natural parallelism a user can understand ABCDE 11234=sum(A1:D1) 2=A1+B1=C1+D1 3=A2*C2

11. 11 Excel Complex and very sophisticated Extensible –Can define custom functions and plug-ins –Custom functions (stubs) represent backend functionality Sequential evaluation mechanism –Extend Excel to support parallel evaluation (plug-in) –Peacefully coexist with Excel don’t re-implement evaluation or break existing spreadsheets

12. 12 Architecture Recently rebuilt using.NET Support for WS & NetSolve Auto generate Excel stub funs from metadata e.g. WSDL Local database cache Pluggable backend adapters Job Manager (Result Cache) AS Engine (.NET) … Adapters NetSolve Web Services … Excel Add-in Windows service

13. 13 Status and Plans Currently testing, hope to make a distribution available soon (July) Looking for beta testers Future plans –Remote data –Greater job control –Workflow –More support for WS (e.g. WS-security) and OGSA

14. 14 Questions? If you are interested in either project please contact me: p.roe@qut.edu.au

15. 15 G2 System Architecture VolunteersClientServer Client Program G2 proxy G2 Framework Programmer supplied Key: Web Server G2 Middle- ware Job DB Web Browser Parallel App. Web Browser Parallel App. Web Browser Parallel App. Web services Off the shelf.NET

16. 16 G2 Code Distribution Code Upload Refers to HTML/ HTTP Client Server G2 Proxy Client Component Volunteer Component Web Server G2 Middleware Manages SOAP/ HTTP Manages Web Browser Volunteer.NET Runtime Volunteer Host Loads Embeds Code Download Server Volunteer assembly cache Download assembly cache SOAP/ HTTP

17. 17 G2 File Staging File caches G2 Server Files Isolated storage Volunteer Files Isolated storage Web service file request 123 Files on the LAN/Internet read via HTTP or NTFS 2 Files