Presentation is loading. Please wait.

Presentation is loading. Please wait.

Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Red Hat on Jenkins: By the numbers Nam Duong Red Hat, Inc. https://openshift.redhat.com.

Similar presentations


Presentation on theme: "Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Red Hat on Jenkins: By the numbers Nam Duong Red Hat, Inc. https://openshift.redhat.com."— Presentation transcript:

1 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Red Hat on Jenkins: By the numbers Nam Duong Red Hat, Inc. https://openshift.redhat.com

2 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Agenda Products currently leveraging Jenkins Deployment size: # of Masters/Slaves/Jobs in use across Jboss Middleware QE Top 10 list of interesting Plugins Code contributions Experiences/Pitfalls/Lessons learned Best Practices Demo: Production/Staging server (sheer numbers) OpenShift on Jenkins EC2 and GIT plugin demo

3 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Red Hat products Products currently leveraging Jenkins JBoss offerings: Enterprise Application Platform JBoss Developer Studio SOA JBoss Portal JBoss OSGI Many, many others OpenShift – OpenShift is our Platform as a Service (PaaS) offering. Our Jenkins instances are running in the cloud, hosted on OpenShift Flex and used to build and test OpenShift

4 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Deployment Size JBoss Middleware Team: – 1 Master IBM System x3550 M3, 2x E5640 CPU, 24GB RAM, 2x SAS 73GB 15K-RPM in RAID1 – Slaves connected 113 (various OS: linux, Win, Solaris, HP, AIX) – ~4700 Jobs configured (~1000 running/month) – ~95% of Middleware test automation is called/executed via Jenkins OpenShift Flex Team: – 1 Master 7.5 GB, 4CPU, 10GB EBS (RHEL 5.6) – 10 Slaves Building EC2 Images and 5 slaves running tests – 20 jobs (RHEL 6.1) – ~90% of our automation is executed via Jenkins

5 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Interesting plug-ins used Management (Groovy*, Disk Usage*, Build Publisher*, Workspace Cleanup*) Cloud Control (EC2 plugin, DeltaCloud*) Enterprise focused plugins (SmartFrog) Code Coverage (cobertura, emma) Source control (GIT/SVN/CVS) *Represents code contributions

6 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Code Contributions Memory leaks: Involved in diagnosis, resolution In the Hudson timeframe, we contributed to stability through reporting of bugs/instability (large deployment) First ones to run into these types of problems (pure volume) LTS (long term support - Jenkins stable branch) Development and maintenance of plugins (see previous) reporting bug + bug fixes

7 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Experiences/Pitfalls/Lessons learned 1 master may not handle all jobs at once - potentially provide failover functionality on Jenkins Jobs getting stuck due to polling console output through ssh. Need to limit jobs from writing to console. Disk Utilization issues - Overcome through quotas set on jobs Logging was too verbose, Jenkins instance stalled due to low/no disk space OOM issues when trying to view the giant logs - fix in progress

8 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Best Practices Set limits on memory usage (quota) of slave Set limits (quota) on Disk usage for jobs Tools on shared NFS drive – for maintenance and replication/recovery Memory – lots of memory to buffer incremental growth of Master Jenkins master and all slaves under CFEngine control (config management/recovery) Like all reliable services, use Dev/Stg/Prod increments. Can scale vertically, but has anyone tried scaling horizontally?

9 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Demo Production instance within corporate firewall: Production: Staging our plugins Staging: Jenkins running on OpenShift Flex; testing OpenShift Flex EC2 and GIT plugin demo Used to create a client testbed Creates an EC2 instance, prints the IP address, pulls down all code bits

10 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Links/Contact https://openshift.redhat.com We appreciate your feedback of your user experience

11 Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Thank You To Our Sponsors Platinum Sponsor Gold Sponsor Silver Sponsor Bronze Sponsors Coming Soon: The CloudBees Newsletter for Jenkins Please complete the Jenkins survey to help us better serve the community (bonus: a chance to win an Apple TV!)


Download ppt "Jenkins User Conference Jenkins User Conference San Francisco, Oct 2 nd 2011 #jenkinsconf Red Hat on Jenkins: By the numbers Nam Duong Red Hat, Inc. https://openshift.redhat.com."

Similar presentations


Ads by Google