Advanced PBS Information

Introduction

This document is intended to give ideas for ways in which PBS can be used in an advanced way. This is not likely to be useful to the majority of users. Please be sure you are very familiar with the information from the PBS Batch Information and PBS Tutorial before attempting the techniques listed here.

Job Dependencies

Table copied from Moab documentation

Specifying a job dependency occurs when you submit the job that depends on another. For example, if you have two job scripts, named jobA and jobB, and jobB needs to wait until jobA has completed running, first submit jobA as normal. If your jobA submission generated the job id "801008.m4bi", then you can then submit the second with the "-W depend=" syntax, as shown in the example below. While the example uses the "-W" parameter on the command line, it is also acceptable to put it inside the script, using "#PBS -W" and so on.

[user@m4b-1-4 job_dependencies]$ qsub jobA
814569.m4bi
[user@m4b-1-4 job_dependencies]$ qsub -W depend=afterok:814569.m4bi jobB
814570.m4bi
[user@m4b-1-4 job_dependencies]$ showq -u user

active jobs------------------------
JOBID              USERNAME      STATE  PROC   REMAINING            STARTTIME

814569               user    Running     1    00:01:32  Fri Apr  6 09:45:45

1 active job             1 of 2440 processors in use by local jobs (0.04%)
                        602 of 610 nodes active      (98.69%)

eligible jobs----------------------
JOBID              USERNAME      STATE  PROC     WCLIMIT            QUEUETIME


0 eligible jobs

blocked jobs-----------------------
JOBID              USERNAME      STATE  PROC     WCLIMIT            QUEUETIME

814570               user       Hold     1    00:02:10  Fri Apr  6 09:46:01

1 blocked job

Total jobs:  2

[user@m4b-1-4 job_dependencies]$ 

Notice that the second job is in the "Hold" state. This is because the dependency specified has not been met yet, and is perfectly normal. In addition, although the above example uses the full job id syntax (like "jobnumber.m4bi"), you can also use just the simple job number without the hostname suffix. Note also that if you remove a job using qdel, and other jobs depend on it running successfully, the scheduler will automatically also remove those jobs that depend on it.

It is also possible to retrieve the job identifier from within a job using the $PBS_O_JOBID variable, which contains the number and name of the current job. This way, it should be possible to create analysis/evaluation jobs that wait for processing jobs to finish, and then have the analysis job submit other jobs that must wait for itself to finish, and so on.

Floating Software Licenses

Some kinds of software require that a license be checked out from a license manager server, and it is possible for the scheduler to monitor these licenses. This means that if you request the appropriate licenses, the scheduler will keep your job from executing until it is certain it can provide the licenses you requested.

The syntax to request a license or licenses is similar to the job dependency syntax shown above. For example, to request one fluent license and two fluent-par licenses, you would use the following "-W" syntax:

$ qsub -W x=GRES:fluent+1,fluent-par+2 jobA

Note that, as before, you can specify the license requirements on the command line, or inside the script with a line something like this:

#PBS -W x=GRES:fluent+1,fluent-par+2

To get information about which licenses are monitored, or to request that an additional license be monitored, please contact the support staff.

Account Links

• Log In
• Account Management
• Support Ticket Tracker
• Mail Archive
• FSL Notes

Main Menu

• Main
• Resources
• Principles of Use
• Account Policies
• Getting an Account
• FSL Staff/Support
• Utilization

Quick Links

• marylou4 FAQ
• PBS Batch Information
• PBS Batch Tutorial
• PBS Script Generator
• Advanced PBS Information
• Samba (SMB/CIFS) Usage Information
• Current Top 500 Supercomputers list (external)