Presentation is loading. Please wait.

Presentation is loading. Please wait.

Christopher Salembier CSC-8530. Review Cycle Stealing Network of Workstations (NOW) as a parallel computer Making use of unused resources Workstation.

Similar presentations


Presentation on theme: "Christopher Salembier CSC-8530. Review Cycle Stealing Network of Workstations (NOW) as a parallel computer Making use of unused resources Workstation."— Presentation transcript:

1 Christopher Salembier CSC-8530

2 Review Cycle Stealing Network of Workstations (NOW) as a parallel computer Making use of unused resources Workstation may be active or inactive Used in many volunteer computing projects Berkeley Open Infrastructure for Network Computing (BOINC) Distributed.net GridGain middleware used Java based cloud enabling platform Configured through Spring framework IoC container

3 (Intended)Project Focus The cycles / resources themselves Research findings Fine-grained kernel-level stealing Complex algorithms for scheduling and allocation Existing platforms Did none of it Needed C or lower level language Wouldn’t have any distributed element “Application” level stealing instead

4 What Got Missed Client-Side Issues GridGain only active within JVM No recovery on shutdown / offline Cannot save task Grid registers as a failure “Pull” jobs Bandwidth monitoring WAN usage

5 The Distributed Task Parsing the contents of Wikipedia 24 GB+ of text Requires 3 passes Unused processing power on work servers Ultimate goal is general text parsing system General purposes grid for processing large files Use method injection to configure algorithm

6 Implementation - Worker public final void register() { pauseLock = new ReentrantLock() waiting = pauseLock.newCondition(); StealFactory.registerNewThread( this ); if ( StealFactory.isBlockThreadActive() pauseProcess(); } public final void pauseProcess() { if ( !isBlocked ) { isBlocked = true; while ( isBlocked ) waiting.await(); } public final void wakeProcess() { pauseLock.lock(); isBlocked = false; waiting.signal(); pauseLock.unlock(); } public final void unregister() { StealFactory.unregisterThread( this ); } public abstract Serializable process() throws GridException; public final Serializable execute() throws GridException { Serializable result = null; register(); result = process(); unregister(); return result; }

7 Implementation - Worker public void addTask( CycleSteal job ) { synchronized ( mutex ) registeredJobs.add( job ); } public void removeTask( CycleSteal job ) { synchronized ( mutex ) { registeredJobs.remove( job ); } public void blockTasks() { synchronized ( mutex ) { for ( final CycleSteal job : registeredJobs ) { new Thread( new Runnable() { public void run() job.pauseProcess(); } ).start(); } public void notifyTasks() { synchronized( mutex ) { for ( CycleSteal job : registeredJobs ) { job.wakeProcess(); } public void setMonitoredValue( Long value ) { currentSystemState = value; if ( isWaiting ) { lock.lock(); isWaiting = false; waiting.signal(); lock.unlock(); } public void run() { while ( running ) { checkSystemState(); lock.lock(); isWaiting = true; while ( isWaiting ) waiting.await(); lock.unlock(); } private void checkSystemState() { if ( currentSystemState >= systemStateThreshold && isBlocking ) { isBlocking = false; StealFactory.notifyRegisteredThreads(); else if ( currentSystemState < systemStateThreshold && !isBlocking ) { isBlocking = true; StealFactory.blockRegisteredThreads(); }

8 Implementation - Master public void onFinished( GridTaskFuture taskFuture ) { GridNode node = null; HashMap map = null; UUID sesID = taskFuture.getTaskSession().getId(); node = assignThread.removeFuture( sesID ); map = (HashMap )taskFuture.get(); processJobResult( map ); if ( !assignThread.isRunning() ) { if ( !assignThread.hasWorkOutstanding() ) shutdownMasterNode(); else assignJobsToNode( node ); }

9 Configuration All components configured as Spring beans Application Context XML file Inversion of Control pattern Allows run-time configuration Grid Itself Node type Master / Worker User / CPU monitor Stealing properties

10 Configuration

11 Testing No UI to verify Reliant on log files Massive amount of data generated Log entries not always sequential 1 st Round of Testing failed Though code was working Block Thread & Manager needed significant rework Though difficult, log ultimately helped solve problem

12 Demo I hope this works!!! Run nodes of each types View logs View output

13 Conclusion Original Goal not really met Project a Success Learned a lot about… Cycle stealing Complex multi-thread Java systems General distributed programming issues


Download ppt "Christopher Salembier CSC-8530. Review Cycle Stealing Network of Workstations (NOW) as a parallel computer Making use of unused resources Workstation."

Similar presentations


Ads by Google