1 Network Access to Charm Programs: CCS Orion Sky Lawlor 2003/10/20

2 Introduction Let’s say you’re running a week- long parallel simulation Say something goes wrong on day two Wouldn’t it be nice to be able to look at (diagnose), and interact with (fix) the job as it’s running? Hurdles: network protocol, firewalls, authentication,...

3 Our Solution Basic network protocol CCS- converse client/server Data to send NetFEM: current FEM mesh liveViz: 2D image of computation liveViz3d: 3D object view pieces Debugger: application state Appspecter: performance data

4 CCS Network Protocol

5 CCS: Converse Client-Server TCP-based protocol (like HTTP) Client sends request to server: String request name Binary request data Server sends binary response back Works even if client behind firewall Allows any sort of interaction Includes authentication component Cryptographic, based on SHA-1

6 CCS Network Protocol CCS Server Code 1.) Send request to server 3.) Send Response to Client Client/CCS Client Library Parallel Application 2.) Call appropriate request handler Request handler name and request data Response data Request data

7 CCS: Advantages Built into Charm++ Just run using “++server” option High performance 500us end-to-end over real networks Saturates fast ethernet Easy to use Register server routine from Charm program Sample client library available in both C and Java

8 NetFEM

9 NetFEM Client: pretty pictures Wave dispersion off a crack

10 NetFEM: Protocol Request/response very simple Give me the latest FEM mesh Here is the latest FEM mesh PUP based network-byte-order binary mesh format PUP_toNetwork4 NetFEM Server Library FEM Application NetFEM Client Request New mesh Response: New Mesh

11 NetFEM: Client Tcl/Tk user interface C++ VTK Visualization (OpenGL)

12 NetFEM: Application Side Can add to any program that deals with FEM meshes Easy to “publish” attributes: NetFEM n=NetFEM_Begin(2,t,NetFEM_POINTAT); NetFEM_Nodes(n,nnodes,(double *)g.coord,"Position (m)"); NetFEM_Vector(n,(double *)g.d,"Displacement (m)"); NetFEM_Vector(n,(double *)g.v,"Velocity (m/s)"); NetFEM_Elements(n,nelems,3,(int *)g.conn,"Triangles"); NetFEM_Scalar(n,g.S11,1,"X Stress (pure)"); NetFEM_Scalar(n,g.S22,1,"Y Stress (pure)"); NetFEM_Scalar(n,g.S12,1,"Shear Stress (pure)"); NetFEM_End(n);

13 LiveViz

14 LiveViz – What is it? Visualization tool Inspect your program’s current state Client runs on any machine (java) You code the image generation 2D and 3D modes

15 LiveViz – Monitoring Your Application LiveViz allows you to watch your application’s progress Can use it from work or home Doesn’t slow down computation when there is no client

16 Running LiveViz Build and run a server cd pgms/charm++/ccs/liveViz/pollserver Make./charmrun server ++server-port n

17 Running LiveViz Run the client cd pgms/charm++/ccs/liveViz/client./run_client [ [ ]] Should get a result window:

18 LiveViz Request Model LiveViz Server Library Client Parallel Application 1.) Get Image 3.) Poll for Request 4.) Poll Request Returns Work 5.) Image Chunk Passed to Library 6.) Send Assembled Image to Client 2.) Buffer Request

19 Main: Setup worker array, pass data to them Workers: Start looping Send messages to all neighbors with ghost rows Wait for all neighbors to send ghost rows to me Once they arrive, do the regular Jacobi relaxation Calculate maximum error, do a reduction to compute global maximum error If timestep is a multiple of 64, load balance the computation. Then restart the loop. Jacobi 2D Example Structure Main: Setup worker array, pass data to them Workers: Start looping Send messages to all neighbors with ghost rows Wait for all neighbors to send ghost rows to me Once they arrive, do the regular Jacobi relaxation Calculate maximum error, do a reduction to compute global maximum error If timestep is a multiple of 64, load balance the computation. Then restart the loop. Main: Setup worker array, pass data to them Workers: Start looping Send messages to all neighbors with ghost rows Wait for all neighbors to send ghost rows to me Once they arrive, do the regular Jacobi relaxation Calculate maximum error, do a reduction to compute global maximum error If timestep is a multiple of 64, load balance the computation. Then restart the loop.

20 #include void main::main(...) { // Do misc initilization stuff // Now create the (empty) jacobi 2D array work = CProxy_matrix::ckNew(0); // Distribute work to the array, filling it as you do } #include void main::main(...) { // Do misc initilization stuff // Create the workers and register with liveviz CkArrayOptions opts(0);// By default allocate 0 // array elements. liveVizConfig cfg(true, true);// color image = true and // animate image = true liveVizPollInit(cfg, opts);// Initialize the library // Now create the jacobi 2D array work = CProxy_matrix::ckNew(opts); // Distribute work to the array, filling it as you do } LiveViz Setup

21 Adding LiveViz To Your Code void matrix::serviceLiveViz() { liveVizPollRequestMsg *m; while ( (m = liveVizPoll((ArrayElement *)this, timestep)) != NULL ) { requestNextFrame(m); } void matrix::startTimeSlice() { // Send ghost row north, south, east, west,... sendMsg(dims.x-2, NORTH, dims.x+1, 1, +0, -1); }

22 Adding LiveViz To Your Code void matrix::serviceLiveViz() { liveVizPollRequestMsg *m; while ( (m = liveVizPoll((ArrayElement *)this, timestep)) != NULL ) { requestNextFrame(m); } void matrix::startTimeSlice() { // Send ghost row north, south, east, west,... sendMsg(dims.x-2, NORTH, dims.x+1, 1, +0, -1); } void matrix::startTimeSlice() { // Send ghost row north, south, east, west,... sendMsg(dims.x-2, NORTH, dims.x+1, 1, +0, -1); // Now having sent all our ghosts, service liveViz // while waiting for neighbor’s ghosts to arrive. serviceLiveViz(); }

23 Generate an Image For a Request void matrix::requestNextFrame(liveVizPollRequestMsg *m) { // Compute the dimensions of the image bit we’ll send // Compute the image data of the chunk we’ll send – // image data is just a linear array of bytes in row-major // order. For greyscale it’s 1 byte, for color it’s 3 // bytes (rgb). // The liveViz library routine colorScale(value, min, max, // *array) will rainbow-color your data automatically. // Finally, return the image data to the library liveVizPollDeposit((ArrayElement *)this, timestep, m, loc_x, loc_y, width, height, imageBits); }

24 OPTS=-g CHARMC=charmc $(OPTS) LB=-module RefineLB OBJS = jacobi2d.o all: jacobi2d jacobi2d: $(OBJS) $(CHARMC) -language charm++ \ -o jacobi2d $(OBJS) $(LB) –lm jacobi2d.o: jacobi2d.C jacobi2d.decl.h $(CHARMC) -c jacobi2d.C OPTS=-g CHARMC=charmc $(OPTS) LB=-module RefineLB OBJS = jacobi2d.o all: jacobi2d jacobi2d: $(OBJS) $(CHARMC) -language charm++ \ -o jacobi2d $(OBJS) $(LB) -lm \ -module liveViz jacobi2d.o: jacobi2d.C jacobi2d.decl.h $(CHARMC) -c jacobi2d.C Link With The LiveViz Library

25 LiveViz Summary Easy to use visualization library Simple code handles any number of clients Doesn’t slow computation when there are no clients connected Works in parallel, with load balancing, etc.

26 Conclusions CCS provides useful low-level network protocol High-level Charm++ libraries for use with CCS NetFEM, for FEM meshes LiveViz, for arbitrary images many others