1. June 2011 David Front Weizmann Institute
Frontier rpms
June 2011
David Front
Weizmann Institute
8 June 2011
frontier rpms

2. 8 June 2011
frontier rpms

3. Why rpms? Once a site (takes the effort and) moves to use rpms
upgrading SW is faster, and easier compared to using tarballs
It should also be more safe … assuming that proper testing of the new rpm release has been done
Yum package manager is supported by frontier rpms
With yum, installing, upgrading and removing is easier that via rpm:
- yum is aware of rpm dependencies
- No need to specify rpm version, only package name
Automatic upgrade possibility:
- Periodically do ‘yum upgrade package’ (via crontab)
Note, however, that relocation is not supported by yum
8 June 2011
frontier rpms

4. Frontier rpms – history and status
Frontier has been written by CMS and distributed as tarballs
Favia Donno did create initial frontier rpms for ATLAS:
frontier-squid
frontier-tomcat
frontier-servlet
frontier-awstats
A new release of frontier rpms is available, is in use at CERN ATLAS product servers since today, and is planned to be defined as ‘production ready’ soon:
In particular, there is a need for rpms to be tested out of CERN first
Current use of frontier rpms:
ATLAS (and CMS?) has squids installed as rpms
The other frontier rpms are installed only at ATLAS servers at CERN:
CMS, and non CERN ATLAS frontier servers are installed via tarballs.
8 June 2011
frontier rpms

5. Installing frontier rpms: Prerequisites
The (configurable) user to own rpm files should exist prior to rpm installation: Default:
‘squid’ for squid
‘dbfrontier’ for the other rpms
If you are installing frontier rpms, instead of existing non rpm (tarball) package/s:
First uninstall the existing non rpm SW:
Remove the existing directory in which the software was installed
Take care to remove/undo changes at other files that have been edited manually or by scripts. In particular, undo crontab (manual) changes.
Verify that squid/tomcat service actually stopped:
Be aware that: Stopping squid/tomcat may take a long time, or: ‘service frontier-{squid or tomcat} status’ may say by mistake ‘stopped’,
while ‘ps –efw|grep {squid or tomcat}’ is not empty.
8 June 2011
frontier rpms

6. Installing frontier rpms: configuration
Manually take care of per rpm and per site and per site/machine files, if any.
For details, consult later slides, and:
Configuration changes:
As a rule of thumb, configuration changes are applied to files as follows:
The directory is /etc/{squid|awstats(|tomcat)}
Configuration changes of customize.sh are applied when the appropriate server restarts.
Hence, there should be no need to reinstall an rpm in order to apply (such) configuration changes.
In addition, for frontier-squid, squidconf is applied at install time
8 June 2011
frontier rpms

7. Installing frontier rpms (cont.)
In order to install frontier rpms, as user root, import the gpg public key once (per machine):
rpm --import
Using the command 'rpm': as user root:
rpm -Uvh [--prefix <RPM_INSTALL_PREFIX>]
Relocating an rpm:
The defaults of RPM_INSTALL_PREFIX are:
frontier-squid: /
frontier-awstats: /
frontier-tomcat: /data/dbfrontier
frontier-servlet: /data/dbfrontier/tomcat/webapps/atlr (for servlet atlr)
The frontier rpms are relocatable
Exception: frontier-tomcat and frontier-servlet rpms can not be relocated to /
The plan is that / will be the default for them as well.
8 June 2011
frontier rpms

8. The current rpm versions
At:
frontier-awstats noarch.rpm
frontier-servlet noarch.rpm
frontier-tomcat noarch.rpm
frontier-squid-2.7.STABLE9-5.3.sl5.x86_64.rpm
8 June 2011
frontier rpms

9. Installing via yum get cernFrontier repository file once per machine:
wget -O /etc/yum.repos.d/cern-frontier.repo
Now you may use yum as usual. for example:
yum install package1 [package2] [...]
yum upgrade package1 [package2] [...]
yum remove package1 [package2] [...]
8 June 2011
frontier rpms

10. Testing the installation: tomcat, squid
For a simple test of tomcat and squid servers, using fnget.py:
you may adjust and run commands like the following, from bash:
wget -O /tmp/fnget.py
chmod +x /tmp/fnget.py
server=atlasfrontier1.cern.ch # Adjust this to your server
servlet=atlr # Adjust this to you servlet
port= # To test tomcat. Adjust if needed
/tmp/fnget.py \ --url=' \ --sql="select 'TOMCAT_IS_HAPPY' from dual" \ --refresh-cache
port=8080 # To test squid. Adjust if needed
/tmp/fnget.py \ --url=' \ --sql="select 'SQUID_IS_HAPPY' from dual“ \ --refresh-cache
If all went well, you should get records like the following:
'TOMCAT_IS_HAPPY' CHAR
'SQUID_IS_HAPPY' CHAR
8 June 2011
frontier rpms

11. Testing the installation: awstats, logrotate
To see that awstats works,
one may wait a while and see that the numbers or the upgraded squid do grow
Logrotates:
Logs should not grow too much:
/data/tomcat_logs/catalina.out
/data/tomcat_logs/modifiedqueries.out
/data/squid_logs/access.log
To see the allowed sizes:
sudo –u <squid/tomcat user> crontab –l
Find there when logrotate should work and sizes are allowed
8 June 2011
frontier rpms

12. Testing the installation: load
(After a while,) monitor the upgraded server/s host load, verifying that it did not rise or drop considerably, as a result of the upgrade:
CERN: use lemon
Else where: ganglia or whatever host monitoring tool is in use
8 June 2011
frontier rpms

13. Building a frontier rpm
rpms should be built at voatlas137
The related sources should be checked out from svn, under: /data/$USER/rpms
Build an rpm:
cd /data/$USER/rpms/<rpm Name>
pushd ./scripts; ../../scripts/rpmbuild_.sh;popd
2) Copy rpm to the web:
pushd ./scripts; ../../scripts/rpm2web.sh ; popd
8 June 2011
frontier rpms

14. Plan
Once the new rpms will be defined as production ready, it will be suggested that all ATLAS frontier sites will use these frontier rpm/s instead of existing rpm or tarball/s
Enhance frontier rpms to support CMS
Support more than one servlet at a site. ATLAS does not require this because it has only one servlet per site
Add a /etc/tomcat/servlets.conf configuration file
Add a /etc/tomcat/servlets.passwd configuration file (will reside at sindes at CERN)
3) Avoid file duplication at underlying (CMS) tarball and at the rpms code (mainly at frontier-squid). This will make the maintenance of frontier code simpler
4) TBD: Add more monitoring tools into the rpms
8 June 2011
frontier rpms

15. 8 June 2011
frontier rpms

16. Configuration: frontier-squid
Optional configuration files:
1) /etc/squid/squidconf:
Used at rpm installation
Consists of export commands
Main usage: to specify user and group for frontier-squid, other than default user and group: squid, squid
Example (as of CERN):
export FRONTIER_USER=dbfrontier
export FRONTIER_GROUP=users
2) /etc/squid/customize.sh
Used per restart of the squid server.
Consists of specific commands that control the content of squid.conf file.
Examples: Appear at /etc/squid/customize.sh.example_*
and at
-See also: Flavia Donno's documentation for previous version of frontier-squid rpms:
8 June 2011
frontier rpms

17. Configuration: frontier-servlet
Per servlet configuration file should be edited manually. May be done after installation:
<install dir>/tomcat/conf/Catalina/localhost/<servlet name>.xml
Example of such a file name: /data/dbfrontier/tomcat/conf/Catalina/localhost/atlr.xml
8 June 2011
frontier rpms

18. Configuration: frontier-awstats
Optional file to control environment variables at install time
/etc/awstats/awstats.conf
Syntax: export <variable>=<value>
Example: export FRONTIER_USER=<something other than dbfrontier>
Editing of the following files may be done (before or) after rpm installation:
1) A file for rsync with the password of user $SITE$PROJ
(Example: cernatlas) /etc/awstats/password-file
2) An optional per awstats client site/machine config file.
For CERN clients, this file is not needed, but for other sites it may be needed.
File: /etc/awstats/customize.sh
## WARNING
Care should be taken when changing variable in this file,
( and in particular PATH_LOG).
Testing of frontier rpms has not been done with changing of these variables
The rpm installs an example file: /etc/awstats/customize.sh.example_CERN:
8 June 2011
frontier rpms

19. urls Documentation of new versions of frontier rpms:
The main url of frontier:
Current frontier rpms:
To browse svn files of frontier rpms:
To checkout a frontier rpms svn directory (example: frontier-tomcat):
svn co svn+ssh://svn.cern.ch/reps/frontier/rpms/frontier-tomcat/trunc frontier-tomcat
Atlas frontier documentation by John Destefano:
Documentation of (previous) frontier server rpms, by Flavia Donno:
Documentation of (previous) versions of frontier-squid rpm, by Flavia Donno:
Previous versions of frontier rpms, created by Flavia Donno, are at:
CMS has a lot of frontier documentation
8 June 2011
frontier rpms