ORNL is managed by UT-Battelle for the US Department of Energy Process Management Adam Simpson OLCF...
-
Upload
leon-merritt -
Category
Documents
-
view
216 -
download
0
Transcript of ORNL is managed by UT-Battelle for the US Department of Energy Process Management Adam Simpson OLCF...
ORNL is managed by UT-Battelle for the US Department of Energy
Process Management
Adam SimpsonOLCF User Support
2 Intellectual Property Training August 2014
Process Management
• Daemon Process– Perpetually running program– Run even when user is not logged in
• Scheduled process– Process that runs at scheduled time intervals
• Run process every x hours
• Run process at xx:xx ever day
3 Intellectual Property Training August 2014
Cron
• Don’t use Cron at the OLCF– Jobs subject to be deleted without notice– Cron untility subject to be disabled– Crontab not persistent between OS upgrades– Not robust w.r.t system downtimes– Only available on a small set of resources
4 Intellectual Property Training August 2014
Daemons: Chained Jobs
• Recursively submitted PBS jobs– nth+1 job depends on completion of nth job
• Recommended option for OLCF resources
• size 0 Job– Does not count against allocation– Runs on service node
• Create dependency with qsub flag
-W depend=afterok:{jobid}
=afternotok:{jobid}
=afterany:{jobid}
5 Intellectual Property Training August 2014
Daemons: daemon.py
import time
from datetime import datetime
while True:
print str(datetime.now())
time.sleep(30)
6 Intellectual Property Training August 2014
Daemons: launcher.pbs
#PBS -l walltime=24:00:00
#PBS -l nodes=0
#PBS -A PRJ123
qsub -W depend=afterok:$PBS_JOBID launcher.pbs
python -u $HOME/daemon.py >> $HOME/daemon.out 2>&1
7 Intellectual Property Training August 2014
Daemons: running
• Start
$ qsub launcher.pbs
• Monitor $ showq –u $USER active jobs------------------------
{jobid} Running
blocked jobs-----------------------
{jobid#2} Hold
• Stop
$ qdel {jobid} {jobid#2}
8 Intellectual Property Training August 2014
Scheduled Jobs
• Instead of Cron use qsub
qsub -a [[[[CC]YY]MM]DD]hhmm[.SS]
• [CC] – first digits of year
• [YY] – second digits of year
• [MM] – month
• [DD] – day
• hhmm – hour minute
• [.SS] - seconds
9 Intellectual Property Training August 2014
Scheduled Jobs: sched.py
from datetime import datetime
print str(datetime.now())
10 Intellectual Property Training August 2014
Scheduled jobs: launcher.pbs
#PBS -l walltime=00:05:00
#PBS -l nodes=0
#PBS -A PRJ123
qsub -a 1030 launcher.pbs
python $HOME/sched.py >> $HOME/sched.out 2>&1