PanDA Monitor Development ATLAS S&C Workshop by V.Fine (BNL)

ATLAS User driven development Maintenance: – Implementing the new “Feature Requests” – Fixing the “known” bugs Development: – high priority, short term developments to address high priority needs of operations and analysis users – plotting system: “flot”-based plots (after investigating options) to add requested plotting features – ”enhanced” job info and statistics pages – jQuery integrated and used as standard basis for implementing 'client intelligence' and high quality UI – ”json” introduced as standard basis for server->client data export – modular, OO design used in new developments for greater maintainability, reliability, flexibility, and easier reuse 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

ADC Monitoring Feb 16, 2011: “User request driven” approach DONE

Monitor: Before and After DB Page Generators Panda Monitor Web Server URL QUERY SQL QUERY HTML DB Page Generators Panda Monitor Web Server URL QUERY SQL QUERY histogramsjson flot html/json JQuery “generic” code Where: 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

Jaroslava’s feedback Slide 5 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

Related Savannah tickets 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

There were 72 Savannah tickets and a bunch of the communications Trend: the number of “Bug Report” declined, the number of the “Feature request” grew and right now it is on the level 2.5 request per week. There was no complain about quality /performance of the “plotting” approach and its implementation. The requests are coming from users, managers, shifters, PanDA developers Among them: Reprocessing jobs’ time distribution Analysis “merging” jobs monitoring Flexible statistic against of any imaginable filtering per user / date / site / cloud region/ processingType … in the other words against of any job parameter the Panda DB records. There was no new “plotting”-related request recently. There is the growing number of requests for the new “Monitoring” tables (requests (# “High priority tasks”, # “Page with list of multicloud sites”). We are taking in account this trend to set our development priorities. Related Savannah tickets 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, DONE

Site performance issue analysis 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

reco performance: Intel vs. AMD From: Alexandre Vaniachine Date: March 18, :21:03 PM GMT+03:00 To: david rousseau Cc: Wensheng Deng Subject: reco performance: Intel vs. AMD …. we observed a difference in HI reconstruction performance on Intel vs. AMD that is larger than the difference in the GHz. Do you think it would be interesting to investigate if there is such a difference on the level of particular algorithms? 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

DOE/NSF Bi-weekly Operations Meeting 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

ATLAS User Analysis in the US Cloud 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

Panda Monitor / Dashboard Panda Monitor / Dashboard cross references 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

Output Merging for Analysis Jobs 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

Summary After a month of the development followed by one month of use Monitor is free of the “known” bug and has been enhanced with about 2 dozens of the new features requested by ATLAS community via Savannah system. It includes the graphical representation of the user selected information. Supporting large Monitor user communities in operations and analysis Quick response to up to 4-6 requests/week from ops and analysis users Incrementally upgrading monitor design/function in the context of the production monitor Near term developments to come: more work to identify common effort and common code with other monitoring activities improve flexibility of data summary histograms client side intelligence, ajax to improve response time and minimize DB load server side modular python framework for easier support and integration with other monitoring 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,

Questions? 5/4/2011 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9,