Presentation is loading. Please wait.

Presentation is loading. Please wait.

Viglen Rocks Cluster Training University of Liverpool

Similar presentations


Presentation on theme: "Viglen Rocks Cluster Training University of Liverpool"— Presentation transcript:

1 Viglen Rocks Cluster Training University of Liverpool
David Power HPC Systems Engineer

2 Viglen Rocks Cluster Training
Rocks Overview Managing Rocks Under the hood with Torque/Maui IPMI Management Implementing Disk Quotas Using Rocks 2

3 Rocks Overview Rocks Overview Managing Rocks
Under the hood with Torque/Maui IPMI Management Implementing Disk Quotas Using Rocks 3

4 Rocks Overview Rocks is an open-source Linux cluster distribution that enables end users to easily build computational clusters, grid endpoints and visualization tiled-display walls It was started by National Partnership for Advanced Computational Infrastructure and the SDSC (San Diego Supercomputer Centre) in 2000 It is based on CentOS with a modified anaconda installer that simplifies mass installation onto many computers. Installations are customised with additional software packages called rolls

5 Adding Software to Compute Nodes
Rocks Overview Managing Rocks Under the hood with Torque/Maui IPMI Management Implementing Disk Quotas Using Rocks 5

6 Building RPMs in Rocks Use rpmbuild if a .spec file is provided
Use the Rocks built in mechanism to create RPMs from directories. Build your software from source and install on the frontend: configure make make install Or, just untar a binary bundle

7 Building RPMs in Rocks Creating RPM packages in Rocks:
rocks create package <path> <package-name> rocks create package /opt/myapp myapp

8 Contribute the RPM Your distribution looks for packages from Rolls and in the contrib area. Copy your RPMs in to the contrib directory. cp myapp x86_64.rpm /export/rocks/install/contrib/5.X/x86_64/RPMS

9 Extend XML Compute Node installation profile can be extended with additional applications by creating an extend-compute.xml file and editing this: cd /export/rocks/install/site-profiles/5.1/nodes/ cp skeleton.xml extend-compute.xml vi extend-compute.xml

10 Add a package to the Compute Profile
original modified (with myapp) <kickstart> <description> Skeleton XML Node </description> <changelog> </changelog> <!— <package>myapp</package> <package></package> --> <post> </post> </kickstart>

11 Rebuild Distribution RPM package is present in the contrib directory
XML node file is extended (and updated) Now we need to rebuild the distribution (apply changes above) This must be done in /export/rocks/install $ cd /export/rocks/install/ $ rocks create distro

12 Reinstall the nodes PXE Boot Network boot is first in BIOS boot order
Set Rocks Boot action to install Reboot the host $ rocks set host boot <host> action=<boot-action> compute-0-0 action=install (or action=os to cancel reinstall)

13 Reinstall the nodes Files updated in /tftpboot
/tftpboot/pxelinux/pxelinux.cfg/XXYYZZAABBCC default rocks prompt 0 label rocks localboot 0

14 Reinstall the nodes Initiate a reboot on the nodes to re-install.
$ rocks run host <host> <command> compute-0-0 /sbin/init 6

15 Check the PXE boot action
List the current PXE boot action on all the nodes. $ rocks list host boot HOST ACTION frontend: os compute-0-0: os compute-0-1: os compute-1-0: os compute-1-1: install compute-2-0: os compute-2-1: os

16 <post> section The <post> section of the extend-compute.xml can be used to do any post installation configuration The file is XML based and certain characters are not permitted: Special Character XML syntax > > < < “ " & & <post> ... /path/to/daemon > /path/to/outputfile </post>

17 <post> section You can put in standard shell commands in to the <post> section <post> ... chkconfig --level 2345 rocks-grub off </post> You can also put in python commands <eval shell=“python”> python shell code </eval>

18 Append files in the <post> section
... <file name=“/etc/motd” mode=“append”> Updated MOTD message </file> </post>

19 Create files in the <post> section
... <file name=“/etc/rc.d/rocksconfig.d/post-99- dofinalconf” mode=“create” perms=“0755”> #!/bin/bash cd /share/apps/myapp cp file /opt/myapp rm -f /etc/rc.d/rocksconfig.d/post-99-dofinalconf </file> </post>

20 Problems with adding new packages
Verify the syntax of the xml extend-compute file $ xmllint –noout /path/to/extend-compute.xml Are kickstart files being generated correctly? $ /export/rocks/sbin/kickstart.cfg --client=compute-0-0

21 Sync Files across Cluster (411)
411 is used to keep files consistent across the cluster Used for passwd/shadow/group etc Can be used for other files also /var/411/Files.mk FILES_NOCOMMENT = /etc/passwd \ /etc/group \ /etc/shadow $ rocks sync users

22 Torque/Maui Rocks Overview Managing Rocks
Under the hood with Torque/Maui IPMI Management Implementing Disk Quotas Using Rocks 22

23 Software Included Torque Maui mpiexec pbstools pbspython ftp://ftp.sara.nl/pub/outgoing/

24 Installed daemons Frontend Compute maui pbs_server
pbs_mom (not running) mpiexec (mostly for the man-page) Compute pbs_mom mpiexec

25 Scheduling Features Maui provides a rich set of scheduling features
Maui can schedule on CPUs Walltime Memory Disk size Network Topology

26 Needed Job Info For scheduling to be useful one needs info about the jobs At least number of cpus and walltime Memory requirement also useful #PBS –lwalltime=HH:MM:SS #PBS –lnodes=10:ppn=8 #PBS –lpmem=1gb

27 Memory handling on Linux
Torque/Maui supports two memory specification types, (p)mem and (p)vmem on linux. pmem is not enforced Used only as information to the sceduler pvmem is enforced Terminating procs that cross the limit Limits vmem size setting ulimit –v on the processes

28 Torque Hacking Torque is installed in /opt/torque
qmgr is the torque mgt. command Note: backup your working config $ qmgr –c “print server” > /tmp/pbsconfig.txt

29 Torque Hacking Roll back to escape from a messed up system:
$ qterm $ pbs_server –t create $ qmgr < /tmp/pbsconfig.txt This will bring you back to where you started Note: this will wipe the whole queue setup and all current queued and running jobs will be lost!

30 Maui Hacking Most things can be achieved by modifying:
/etc/maui/maui.cfg Maui needs a restart after changing the config file: $ service maui restart

31 Should I edit Torque or Maui?
If you can achieve the same thing by changing either torque or maui, use maui. Restarting maui is rather lightweight operation, and seldom causes problems for live systems. Restarting pbs_server can make the system oscillatory for a few minutes pbs_server needs to contact all pbs_moms to get back in state.

32 Prioritising Short Jobs
Often it is useful to give shorter jobs higher priority (maui configuration). Use the XFACTOR feature in maui rather than torque queues with different priorities XFACTORWEIGHT 1000

33 Prioritising Short Jobs
XFACTOR is defined as XFACTOR=(walltime+queuetime)/walltime XFACTOR will increase faster for shorter walltimes thus giving higher priorities for short jobs. Depends on users giving reasonable walltime limits.

34 Prioritising large jobs (maui)
In a cluster with a diverse mix of jobs it is useful to prioritize the large jobs and make the smaller ones fill the gaps. CPUWEIGHT 1000 MEMWEIGHT 100 This should be combined with fairshare to avoid starving users falling outside this prioritisation.

35 Fairshare (maui) Also known as Can be done on several levels
“keeping all users equally unhappy” Can be done on several levels users, groups … Set a threshold USERCFG[DEFAULT] FSTARGET=10 FSWEIGHT 100 Users having more than 10% will get reduced priority and vice versa.

36 Adjusting your policy You can play with the weights to fine tune your scheduling policies XFACTORWEIGHT 100 FSWEIGHT 1000 RESWEIGHT 10 CPUWEIGHT 1000 MEMWEIGHT 100 Analyse the prioritisation with $ diagnose -p

37 Job Node Distribution Default is MINRESOURCE Spread or pack?
Run on the nodes which gives at least unused resources. Spread or pack? NODEALLOCATIONPOLICY PRIORITY Select the most busy nodes NODECFG[DEFAULT] PRIORITYF=JOBCOUNT Select the least busy nodes NODECFG[DEFAULT] PRIORITYF=-1.0*JOBCOUNT

38 Node Access Policy Default access policy is SHARED
Can choose to limit this to SINGLEJOB or SINGLEUSER, i.e. NODEACCESSPOLICY SINGLEUSER Single user access prevents users from stepping on each others toes while allowing good utilisation for serial jobs.

39 Throttling Policies Sometimes one needs to limit the user from taking over the system… MAXPROC MAXNODE MAXPS MAXJOB MAXIJOB (I indicating idle, scheduled but not running) All can be set for all individuals users and groups USERCFG[DEFAULT] USERCFG[USERA] MAXPROC=16

40 Debugging and Analysing
Lots of tools: pbsnodes node status qstat –f all details of a job diagnose –n -- node status from maui diagnose –p -- job priority calculation showres –n -- job reservation per node showstart estimated job start time checkjob check job status checknode check node status

41 Example: Express Queue
Goal: Supporting development and job script testing, but prevents misuse Basic philosophy: Create a separate queue Give it the highest priority Throttle is so it is barely usable

42 Example: Express Queue
Create the queue with qmgr create queue express set queue express queue_type = Execution set queue express resources_max.walltime = 08:00:00 set queue express resources_default.nodes = 1:ppn=8 set queue express resources_default.walltime = 08:00:00 set queue express enabled = True set queue express started = True

43 Example: Express Queue
Increase the priority and limit the usage (maui.cfg) CLASSWEIGHT 1000 CLASSCFG[express] PRIORITY=1000 MAXIJOB=1 \ MAXJOBPERUSER=1 QLIST=express QDEF=express \ QOSCFG[express] FLAGS=IGNUSER This will allow users to test job scripts and run interactive jobs with a good turnaround.

44 Check jobs running on the nodes
Check for jobs running on the cluster (showq): nodes]# showq ACTIVE JOBS JOBNAME USERNAME STATE PROC REMAINING STARTTIME asmith Running :13:35:10 klangfeld Running :22:01:12 2 Active Jobs of 948 Processors Active (19.41%) 16 of 79 Nodes Active (20.25%)

45 Check host jobs are running on
Whats nodes are the jobs running on: (checkjob JOBID) : nodes]# checkjob 1473 checking job 1473 ... Req[0] TaskCount: 120 Partition: DEFAULT Network: [NONE] Memory >= 0 Disk >= 0 Swap >= 0 Opsys: [NONE] Arch: [NONE] Features: [NONE] Allocated Nodes: [compute-2-31:8][compute-2-30:8][compute-2-29:8][compute- 2-28:8] [compute-2-27:8][compute-2-26:8][compute-2-25:8][compute- 2-24:8] [compute-2-23:8][compute-2-35:8][compute-2-34:8][compute- 2-33:8] [compute-2-32:8][compute-2-21:8][compute-2-20:8]

46 Take nodes OFFLINE Take nodes offline (no jobs will be allocated) :
nodes]# pbsnodes -o compute-2-0 nodes]# pbsnodes compute-2-0 compute-2-0 state = offline np = 12 ntype = cluster status = opsys=linux,uname=Linux compute-2-0.local el5 #1 SMP Wed Jan 21 10:41:14 EST x86_64,sessions=? 0,nsessions=? 0,nusers=0,idletime=248891,totmem= kb,availmem= kb,physmem= kb,ncpus=12,loadave=0.00,ne tload= ,state=free,jobs=,varattr=,rectime=

47 Bring nodes ONLINE Take nodes offline (no jobs will be allocated) :
nodes]# pbsnodes -c compute-2-0 nodes]# pbsnodes compute-2-0 compute-2-0 state = free np = 12 ntype = cluster status = opsys=linux,uname=Linux compute-2-0.local el5 #1 SMP Wed Jan 21 10:41:14 EST x86_64,sessions=? 0,nsessions=? 0,nusers=0,idletime=248981,totmem= kb,availmem= kb,physmem= kb,ncpus=12,loadave=0.00,ne tload= ,state=free,jobs=,varattr=,rectime=

48 Create Queues Create additional queues using qmgr
qmgr -c "create queue infiniband“ qmgr -c "set queue infiniband queue_type = Execution“ qmgr -c "set queue infiniband enabled = True“ qmgr -c "set queue infiniband started = True“ qmgr -c "set queue infiniband resources_default.neednodes = ib“ qmgr -c "create queue primare“ qmgr -c "set queue primare queue_type = Execution“ qmgr -c "set queue primare enabled = True“ qmgr -c "set queue primare started = True“ qmgr –c “set queue default resources_default.neednodes = ethernet”

49 Create Queues Assign resources to nodes:
In file: /opt/torque/server_priv/nodes ... compute-1-7 np=8 ethernet compute-1-8 np=8 ethernet compute-1-9 np=8 ethernet compute-2-0 np=12 ib compute-2-10 np=12 ib compute-2-11 np=12 ib compute-2-12 np=12 ib

50 Create Queues Verify nodes report resources correctly:
~]# pbsnodes compute-1-0 compute-1-0 state = free np = 8 properties = ethernet ~]# pbsnodes compute-2-0 compute-2-0 np = 12 properties = ib

51 Queue Priority Set the priority levels for the queues:
In File: /opt/maui/maui.cfg ... CLASSWEIGHT CLASSCFG[default] PRIORITY=1 CLASSCFG[infiniband] PRIORITY=1 CLASSCFG[primare] PRIORITY=10000 Save and restart maui service maui restart

52 Check Job Priorities (As root) use the diagnose command:
~]# diagnose -p diagnosing job priority information (partition: ALL) Job PRIORITY* Cred(Class) Serv(QTime) Weights ( 1) 1( 1) (10000) 0.0( 0.0) ( 1.0) 15.5( 0.2) ( 1.0) 7.7( 0.1) Percent Contribution (100.0) 0.0( 0.0) *indicates system priority set on job Red text indicates a job on high priority queue (classcgf)

53 Submitting to the queues
Submit to Infiniband Queue #PBS -q infiniband Submit to Primare Queue #PBS -q primare Submit to Primare Queue (on IB nodes) #PBS -l nodes=NN:ppn=PP:ib Submit to Primare Queue (on Ethernet nodes) #PBS -l nodes=NN:ppn=PP:ethernet

54 Restricting User Access to Queue
Allow only users from the primare group use primare Q qmgr -c "set queue primare acl_group_enable=true" qmgr -c "set queue primare acl_groups=primare"

55 Summary Limit the number of queues.
You need good information about walltime.

56 IPMI Management Rocks Overview Managing Rocks
Under the hood with Torque/Maui IPMI Management Implementing Disk Quotas Using Rocks 56

57 IPMI Overview IPMI is a standard which defines a set of common interfaces to monitor system health and manage the system. IPMI operates independent of the OS IPMI operates even if the monitored system is powered off, but connected to a power source IPMI BMCs can be communicated to using IP addressing or queried locally.

58 Configuring the IPMI Module
RPMs installed: OpenIPMI OpenIPMI-tools (inc ipmitool) OpenIPMI-libs Kernel Modules required: ipmi_msghandler ipmi_devinf ipmi_si

59 Configuring the IPMI Module
Set the IP address (static): ipmitool lan set 1 ipsrc static ipmitool lan set 1 ipaddr ipmitool lan set 1 netmask ipmitool lan set 1 defgw ipaddr Set the IP address (dhcp): ipmitool lan set 1 ipsrc dhcp

60 Configuring the IPMI Network
IPMI modules part of the Rocks Network Edit the /etc/hosts.local file compute-ipmi-0-0 compute-ipmi-0-1 Run dbreport hosts or rocks report hosts Sync the /etc/hosts file using 411 (/var/411/Files.mk) $ rocks sync users

61 Configuring the IPMI Network
IPMI Modules on a separate subnet Step 1: Add a network $ rocks add network <network-name> <network> <subnet> $ rocks add network ipmi

62 Configuring the IPMI Network
List the networks on your Rocks cluster $ rocks list network NETWORK SUBNET NETMASK private: public: ipmi:

63 Configuring the IPMI Network
Step 2: Add the interfaces through Rocks Host first must be installed Then secondary NICs can be added After all hosts are configured just re-install

64 Configuring the IPMI Network
Add host interface: $ rocks add host interface <host> <iface> ip=<address> subnet=<name> gateway=<address> name=<hostname> $ rocks add host interface compute-0-0 ipmi ip= subnet=ipmi gateway=ipmi gateway=1 name=compute-ipmi-0-0

65 Configuring the IPMI Network
List host interface: $ rocks list host interface compute-0-0 SUBNET IFACE MAC IP NETMASK GATEWAY MODULE NAME private eth0 00:15:17:79:d3:c e1000e compute-0-0 Ipmi ipmi ipmi-0-0

66 (Alternative) Configuring the IPMI Network
Add hosts to /etc/hosts.local: compute-1-0-ipmi compute-1-1-ipmi compute-1-2-ipmi compute-1-3-ipmi Regenerate the /etc/hosts file (which will append all entries from /etc/hosts.local $ dbreport hosts > /etc/hosts ... # import from /etc/hosts.local

67 Check the IPMI serivice is running
Use the tentakel command: tentakel -g rack2 chkconfig --list '| grep ipmi' ### compute-2-35(stat: 0, dur(s): 0.16): ipmi :off 1:off 2:on 3:on 4:on 5:on 6:off ### compute-2-23(stat: 0, dur(s): 0.17): If any nodes have the service set to off, turn on using: tentakel -g rack2 chkconfig --level 2345 ipmi on

68 Using IPMItool Chassis Status ipmitool chassis status
System Power : on Power Overload : false Power Interlock : inactive Main Power Fault : false Power Control Fault : false Power Restore Policy : always-off Last Power Event : ac-failed Chassis Intrusion : active Front-Panel Lockout : inactive Drive Fault : false Cooling/Fan Fault : false

69 Using IPMItool Chassis Status over the network
ipmitool –U ADMIN –P ADMIN –h compute-ipmi-0-0 chassis status System Power : on Power Overload : false Power Interlock : inactive Main Power Fault : false Power Control Fault : false Power Restore Policy : always-off Last Power Event : ac-failed Chassis Intrusion : active Front-Panel Lockout : inactive Drive Fault : false Cooling/Fan Fault : false

70 Using IPMItool Reset the IPMI Module (sometimes causes network issues)
ipmitool mc reset cold ipmitool –U ADMIN –P ADMIN –h compute-ipmi-0-0 mc reset cold Then reload network modules on nodes: rmmod e1000e ; modprobe e1000e

71 Using IPMItool Controlling the Power Initiate a clean shutdown
ipmitool –U ADMIN –P ADMIN –h compute-ipmi-0-0 power soft Power Off ipmitool –U ADMIN –P ADMIN –h compute-ipmi-0-0 power off Power On ipmitool –U ADMIN –P ADMIN –h compute-ipmi-0-0 power on Power Cycle ipmitool –U ADMIN –P ADMIN –h compute-ipmi-0-0 power cycle

72 Implementing Disk/User Quotas
Rocks Overview Managing Rocks Under the hood with Torque/Maui IPMI Management Implementing Disk/User Quotas Using Rocks 72

73 Disk Quotas (general) In order to use disk quotas, they must first be enabled: Modify /etc/fstab Remount the filesystem(s) Run quotacheck Assign Quotas

74 Disk Quotas (general) Modify /etc/fstab
Edit the line with your home directory LABEL=/state/partition On this line change the mount options in include grpquota and usrquota defaults -> grpquota,usrquota,defaults Reboot the machine

75 Disk Quotas (ext3) When the system has come back up, run quotacheck
This examines quota-enabled file systems, building a table of the current disk usage for each one. quotacheck –guvma -g -- group quotas -u -- user quotas -v -- report the operation as it progresses -m -- do not remount filesystems as RO -a -- check all mounted non-NFS filesystems

76 Disk Quotas (ext3) Turn File system quotas on Setup a user quota
quotaon –guva Setup a user quota edquota –u <username> Modify the number of days for the hard and soft limits edquota –t Propagate these settings to all other users edquota –p <user_with_updated_quota> -u users..

77 Disk Quotas (ext3) Editing the quota limits (edquota)
Disk quotas for user viglen (uid 501): Filesystem blocks soft hard inodes soft hard /dev/sda Blocks: The amount of space in 1K blocks the user is currently using. Inodes: The number of files the user is currently using. Soft Limit: The maximum blocks/inodes a quota user may have on a partition. The role of a soft limit changes if grace periods are used. When this occurs, the user is only warned that their soft limit has been exceeded. When the grace period expires, the user is barred from using additional disk space or files. When set to zero, limits are disabled. Hard Limit: The maximum blocks/inodes a quota user may have on a partition when a grace period is set. Users may exceed a soft limit, but they can never exceed their hard limit.

78 Disk Quotas (ext3) Reporting on quota usage:
repquota /state/partition1 Useful TIP: Display user quotas at their login (/etc/profile) quota –u $USER

79 Disk Quotas (XFS) Confirm quotas are enabled xfs_quota –x –c state
*[Steps below assume /etc/fstab has been updated and the system has been rebooted]* Confirm quotas are enabled xfs_quota –x –c state Check the current space usage xfs_quota –x –c quot

80 Disk Quotas (XFS) Set user limits (where username is the user account name) Limits below implement a 6GB hard limit (cant write more than 6GB) and a 5GB soft limit (warnings issued after 5GB) xfs_quota –x /state/partition1 xfs_quota> limit bsoft=5g bhard=6g username xfs_quota> quota -h username Disk quotas for User username(id) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/sdb G 5G 6G 00 [------] /state/partition1

81 Disk Quotas (XFS) Disable user quota, set the limits back to 0
xfs_quota –x /state/partition1 xfs_quota> limit bsoft=0 bhard=0 username xfs_quota> quota -h username Disk quotas for User username(id) Filesystem Blocks Quota Limit Warn/Time Mounted on /dev/sdb G [------] /state/partition1

82 Disk Quotas (XFS) Report on XFS Quota Usage
scripts]# xfs_quota -x xfs_quota> report User quota on /state/partition1 (/dev/sdb1) Blocks User ID Used Soft Hard Warn/Grace root [0 days] maui [ ] icr [ ] biouser [ ] viglen [ ] klangfeld [ ] pmills [ ] troc [ ] ngardiner [ ]

83 Implementing Disk/User Quotas
Rocks Overview Managing Rocks Under the hood with Torque/Maui IPMI Management Implementing Disk/User Quotas Using Rocks 83

84 System Access Access the Rocks Cluster via ssh (port 22)
From Linux systems this is already built in (ssh from the CLI) For graphical forwarding use ssh with X-forwarding ssh –X headnode To copy file use scp scp file scp –r directory From Windows systems use putty For graphical forwarding in Windows use Xming

85 System Access From Windows systems use putty
For graphical forwarding in Windows use Xming To copy files use WinSCP

86 System Environment To setup you environment to compile/run applications, u se the modules environment ~]$ which pgcc /usr/bin/which: no pgcc in $PATH ~]$ pgcc -V -bash: pgcc: command not found ~]$ module available /usr/share/Modules/modulefiles pgi/32/ pgi/64/ rocks-mpich2/gnu/1.1.1-p1 pgi/32/10.9-mpich pgi/64/10.9-mpich ~]$ module load pgi/64/10.9 /share/apps/pgi/10.9/linux86-64/10.9/bin/pgcc pgcc bit target on x86-64 Linux -tp istanbul-64

87 Job Control Check Jobs running on the queue [viglen@ulgqcd ~]$ showq
ACTIVE JOBS JOBNAME USERNAME STATE PROC REMAINING STARTTIME 0 Active Jobs of 496 Processors Active (0.00%) 0 of 31 Nodes Active (0.00%) IDLE JOBS JOBNAME USERNAME STATE PROC WCLIMIT QUEUETIME 0 Idle Jobs BLOCKED JOBS

88 Sample Submission Script
#!/bin/bash #PBS -l nodes=4 #PBS –l walltime=2:00 #PBS -o mytest.out #PBS -e mytest.err cd $PBS_O_WORKDIR NPROCS=`wc -l $PBS_NODEFILE | awk '{ print $1 }'` /opt/mpich/bin/mpirun -np $NPROCS -machinefile $PBS_NODEFILE hello

89 Job Control Submit a job to the queue
qsub]$ qsub q_sleep.sh 110.ulgqcd.ph.liv.ac.uk qsub]$ showq ACTIVE JOBS JOBNAME USERNAME STATE PROC REMAINING STARTTIME viglen Running :00:00 Wed Nov 3 23:20:45 1 Active Job of 496 Processors Active (0.20%) 1 of 31 Nodes Active (3.23%)

90 Job Control Delete a job from the queue
qsub]$ qsub q_sleep.sh 110.ulgqcd.ph.liv.ac.uk qsub]$ qdel 110

91 Job Control Take a look at the output before jobs finishes
qsub]$ qpeek 190 memtester version (64-bit) Copyright (C) 2007 Charles Cazabon. Licensed under the GNU General Public License version 2 (only). pagesize is 4096 pagesizemask is 0xfffffffffffff000 want 90000MB ( bytes) got MB ( bytes), trying mlock ...locked. Loop 1/1: Stuck Address : ok Random Value : ok Compare XOR : Compare SUB : Compare MUL : Compare DIV : Compare OR : Compare AND : Sequential Increment: ok Solid Bits : testing 49 Block Sequential : setting 11

92 Torque/PBS Submit Options
#PBS –I # Submit an interactive job #PBS –q queue_name # Submit to a certain queue #PBS –l nodes=compute-0-0 # Submit to a specific host #PBS –l nodes=4:ppn=4 # 4 Host with 4 Processes per node #PBS –j oe # Combine stdout and stderr #PBS –o stdout_file # Specify the Output file #PBS –e stderr_file # Specify the Error file #PBS –V # Import you environment vars

93 Torque/PBS Variables $PBS_O_HOST # Hostname where job launched $PBS_O_QUEUE # Originating Queue $PBS_O_WORKDIR # Execution Queue $PBS_JOBID # Job Identifier $PBS_JOBNAME # Job name $PBS_NODEFILE # Nodefile (for parallel jobs) $PBS_O_HOME # Current Home Directory $PBS_PATH # PATH

94 Thank You! Web: http://www.viglen.co.uk/hpc


Download ppt "Viglen Rocks Cluster Training University of Liverpool"

Similar presentations


Ads by Google