qsub.md

Listing all the nodes on the cluster

qhost

Checking current jobs per node

qstat -f

Submitting jobs with qsub

# submit a job to a particular queue
qsub -q 256G-batch

# submit a job to a particular node
qsub -q 256G-batch@perun20

# submit a job to multiple queues
qsub -q 256G-batch,768G-batch,2T-batch

# use the directory from which the script is called as working directory
qsub -cwd

# merge stderr and stdout stream into a single stream
qsub -j y[es]

# set a 'hard' run time (i.e. wall-clock time) limit
# how long are you allowing your job to run?
qsub -l h_rt 00:30:00

# submit a command without making a script
qsub -N <job_name> -b y "iqtree -s alignment.aln -nt 10 -m LG+G"

# it seems that to get -b y[es] "" to work, -N must be given as well

# submit an array-job
qsub -t 1-1000 job.sh

# submit an array job, but only run 20 jobs at any given time (concurrently)
qsub -t 1-1000 -tc 20 job.sh
# each 'sub-job', or task, gets a task ID, which you can retrieve with special variable $SGE_TASK_ID

Cancelling jobs

# cancel a particular job
qdel [job_id]
# this also works for array jobs
# the array job gets a single job id, while all the subjobs get task ids
# so this qdel will kill the job, along with all tasks

# cancel a particular subjob
qdel [jobid].[taskid]
# this will kill this particular task/subjob, and the next subjob/task in the queue will start to be processed

# cancel all jobs submitted by a user
qdel -u [user]

Monitoring current jobs

qstat -j <job_id>

Monitoring finished jobs

# check memory or other stuff from certain finished jobs
qacct -j [jobid]

# list all stats of all previous jobs of a certain user
qacct -o [userid] -j

# get list ids
qacct -o [userid] -j | grep jobnumber

Checking the cluster, queue or node configuration

qconf -sq 768G-batch

Special environment variables

# job id
$JOB_ID

# task id, if using a array job
$SGE_TASK_ID

# number of threads
# if specified with -pe smp 10,
# $NSLOTS returns 10
$NSLOTS