1. AFS at CERN and other High Energy Physics Sites
Rainer Többicke / CERN

2. AFS Software Servers: OpenAFS 1.2.6 Clients: OpenAFS 1.2.7
Focus: stability
Additional patches
Local debugging
Clients: OpenAFS 1.2.7
Linux RPMs
Historical accumulation: wide spread of partly ancient versions
… and system architectures
March 2003
Rainer Többicke / CERN

3. Service Total disk space: 15 Sun Servers, 10 PC servers
~ 10 TB installed
~ 6 TB usable
15 Sun Servers, 10 PC servers
Sun A1000 RAID (-5) Systems
Clients
2000 off-site
March 2003
Rainer Többicke / CERN

4. Usage Private user files - home directories Scratch
14000 users, 5000 active, 2TB disk space
Scratch
“low quality” space at descretion of major user groups, ~ 1 TB disk space
Software development
Production environment
Software
Controlled by major user groups
March 2003
Rainer Többicke / CERN

5. Evolution: # of users
March 2003
Rainer Többicke / CERN

6. Evolution: User space
March 2003
Rainer Többicke / CERN

7. Evolution: Project space
March 2003
Rainer Többicke / CERN

8. The Future of AFS History Characteristics What do we require?
Distributed file systems
March 2003
Rainer Többicke / CERN

9. History IBM sponsored development at CMU in the late 80s
Transarc Corporation, later owned by IBM
Strong academic (CMU, MIT, Umich) user community
AFS 3.2 CERN in 1993
Now almost ubiquitous in HEP
OpenAFS - open source as of 2000
Active development – 4-5 releases/year
Alternative implementations: Arla, Red Hat
Prime time quality software
March 2003
Rainer Többicke / CERN

10. AFS characteristics Global file system NAS approach
Security, flexibilty
NAS approach
No special infrastructure
Moderate performance – single path to data
Performance (in 2002) for big files
Slightly below NFS with caching: MB/s
approaching RFIO without caching: ~30 MB/s
March 2003
Rainer Többicke / CERN

11. AFS characteristics – cont’d
Scaling
Number of files in a directory ~ 62000
2 GB file size limit
No single directory bigger than a “disk”
AFS cache size?
Scaling – non-issues
Number of servers
Performance limits
No file striping – file access limited by single server
Client disk caching
Flat client/server topology, no proxies or cascades
March 2003
Rainer Többicke / CERN

12. Requirements ? Global access Security / access control Manageability
Cascading / proxies?
Security / access control
Manageability
Performance
infrastructure files
data files
Scalability
Cost
Standard in AFS
Kerberos, ACLs
AFS volumes … OK
problematic
Looks OK
Free
March 2003
Rainer Többicke / CERN

13. Distributed File Systems
Production, IP (NAS)
NFS V3, V4
Microsoft Dfs
SAN based
GFS, CXFS
GPFS
Recent Developments
Lustre, Intermezzo , StorageTank
 PASTA III (2002) “Distributed File Systems”
March 2003
Rainer Többicke / CERN

14. Summary
From a shaky “interesting idea” in 1993 AFS has grown to a stable system
There is a need for a distributed file system for LCG
Characteristics?
March 2003
Rainer Többicke / CERN