RUSH RENDER QUEUE - COMMAND LINE OPTIONS
(C) Copyright 1995,2000 Greg Ercolano. All rights reserved.
V 102.41a 05/27/04
Strikeout text indicates phased out or obsolete operators.

Rush Command Line

  rush -ac  
rush -ac <cpuspec..> [jobid..]

Add Cpus.

Adds cpus to the running job. Examples:

    Add Cpu Examples
    rush -ac tahoe@300 Add one cpu on tahoe at 300 priority
    rush -ac tahoe=2@300 +farm=10@100k Add two cpus on tahoe, 10 cpus for +farm hostgroup
    rush -ac +any=10@30 Add any 10 available cpus at priority 30

See the Cpus submit command for more info. To remove cpus, see 'rush -rc'.

  rush -af  
rush -af <framerange..> [jobid..]

Add Frames.

Adds frames to the current framelist. Adding frames that already exist yields an error. Example:

    Add Frames Examples
    rush -af 10-15 Add 10 thru 15 to the current job
    rush -af 20-25=Hold Add 20 thru 25 in Hold state
    rush -af 30-35:"To be done" Add 30 thru 35, Notes "To be done"
    rush -af 40-45=Done:"To be done" Add 40 thru 45 as Done with Notes

See Frames submit command for more info.

  rush -an  
rush -an <hostname|+hostgroup..> [jobid..]

Add Nevercpus.

Prevents the job from rendering on the specified hosts. Example:

    Add Never Cpus Examples
    rush -an tahoe +farm Add host tahoe and all hosts in hostgroup
    +farm as machines to never use.

See Nevercpus submit command for more info.

  rush -autodump  
rush -autodump <off|done|fail|donefail> [jobid..]

Automatic Dump.

Configures job to automatically dump when there are no more frames to render.

    Autodump Examples
    autodump off Don't autodump; job remains when frames are DONE (default)
    autodump done Job dumps itself when all frames are DONE
    autodump fail Job dumps itself if job finishes with any FAIL frames (New in 102.40)
    autodump donefail Job dumps itself if all frames are DONE or FAIL

See AutoDump submit command for more info.

  rush -checkconf  
rush -checkconf <filename>

Check for errors in rush.conf file..

(Administrative) Checks a rush.conf file for errors. Returns an exit code indicating success or failure; 0=ok, 1=error(s) found. Example: 

#!/bin/csh -f

###
### Script to edit the rush.conf file, and rdist it out
###

set TMPFILE=/usr/tmp/rush.conf.$$
cp /usr/local/rush/etc/rush.conf $TMPFILE
vi $TMPFILE

rush -checkconf $TMPFILE
if ( $? ) then
    echo Config file has errors
    set err=1
else
    foreach i ( tahoe superior erie )
        echo --- $i
	rdist -c $TMPFILE ${i}:/usr/local/rush/etc/rush.conf
    end
    set err=0
endif
rm -f $TMPFILE
exit $err

  rush -checkhosts  
rush -checkhosts <filename>

Checks a hosts file for errors.

(Administrative) Returns an exit code indicating success or failure; 0=ok, 1=error(s) found.

  rush -command  
rush -command 'cmd [args..]' [jobid..]
rush -command [jobid..] < file

Set Render Command.

Changes the command that is executed every frame, normally set in your Submit Script by the Command. Example:

If 'cmd [args]' are not supplied, the command will be read from stdin. This is useful to avoid shell quoting issues. If multiple lines are specified, they will be joined into one single line.

    Command Examples
    rush -command '/bin/sleep 30' Set command to sleep 30 seconds per frame
    rush -command < file Set command to contents of 'file' on stdin
    rush -command '/job/run/myrender' tahoe.21 tahoe.22 Change command for two jobs

Note that a full path to the command is normally required. Spaces are used to separate arguments, but the entire command and arguments must be quoted.

See also the submit script docs on Command for more info.

The command is usually a simple Render Script. Here are some example 3rd Party Render Scripts.

  rush -cont  
rush -cont [jobid..]

Continue a Paused Job.

Continues a job that was paused with rush -pause.

(New in rush 102.40h and up)
No longer cancels 'Waitfor'; it was decided the old behavior (102.40g and back) of having 'rush -cont' cancel 'WaitFor' was a bug, not a feature. To properly cancel WaitFor (to get the job running) is to use 'rush -waitfor -'.

  rush -cp  
rush -cp [jobid..] <.JobTid [..]> <@pri>
rush -cp [jobid..] <cpuspec [..]> <=#cpus>

Change Priority.

Changes priority for the cpus specified by 'JobTid' to the new priority '<@pri>'. JobTid values (Job Task IDs) can be found in the 'rush -lc' report.

Priorities can be changed while cpus are busy. If a busy cpu's priority is changed, the busy task(s) inherit the new priority, which is used for all subsequent scheduling and priority battles.

Example: 


% rush -lc
CPUSPEC[HOST]        STATE       FRM  PID     JOBTID  PRI   ELAPSED 
+any=10@100[vaio]    Busy        0007 1030    131     100   00:01:45 
+any=10@100[vaio]    Busy        0008 1031    132     100   00:01:45 
+any=10@100[tahoe]   Busy        0004 19829   133     100   00:00:05 
+any=10@100[tahoe]   Busy        0005 19831   134     100   00:00:05 

% rush -cp .131 .132 @350k
vaio.21 +any=10@100: priority was 100, now 350k
vaio.21 +any=10@100: priority was 100, now 350k

% rush -lc
CPUSPEC[HOST]        STATE       FRM  PID     JOBTID  PRI   ELAPSED
+any=10@100[vaio]    Busy        0007 1030    131     350k  00:02:03 
+any=10@100[vaio]    Busy        0008 1031    132     350k  00:02:03 
+any=10@100[tahoe]   Busy        0004 19829   133     100   00:00:28 
+any=10@100[tahoe]   Busy        0005 19831   134     100   00:00:28

Caveats:

  • Currently the CPUSPEC column does not reflect the change, though the effective priority (shown in the PRI column) does.

  • GUIs (like IRUSH) make best use of this option, since they hide the mundane task of visually indexing JOBTIDs from the -lc report, and inserting dot prefixes on the rush command line.

'rush -cp' can be used to change the number of cpus in a cpuspec. For instance, 'rush -cp +any=50@100 =40' will change '+any=50@100' to '+any=40@100' without killing any running frames.

Example: 


% setenv RUSH_JOBID vaio.36
% rush -lc
CPUSPEC[HOST]          STATE       FRM  PID     JOBTID  PRI   ELAPSED  NOTES
+any=10@500[tahoe]     Busy        0012 11443   343     500   00:00:10 
+any=10@500[tahoe]     Busy        0013 11445   344     500   00:00:10 
+any=10@500[nt-1]      Busy        0008 293     345     500   00:00:11 
+any=10@500[nt-1]      Busy        0009 246     346     500   00:00:11 
+any=10@500[vaio]      Busy        0015 11489   347     500   00:00:07 
+any=10@500[vaio]      Busy        0014 11484   348     500   00:00:09 
+any=10@500[imac]      Busy        0011 22530   351     500   00:00:11 
+any=10@500[imac]      Busy        0010 22528   352     500   00:00:11 

% rush -cp +any=10@500 =3@500
vaio.36 +any=3@500: priority was +any=10@500, now +any=3@500

	 NOW SET TO 3 CPUS, BUT RENDERS STILL RUNNING, NOTHING IS KILLED..

% rush -lc
CPUSPEC[HOST]          STATE       FRM  PID     JOBTID  PRI   ELAPSED  NOTES
+any=3@500[tahoe]      Busy        0021 11453   343     500   00:00:41 
+any=3@500[tahoe]      Busy        0020 11451   344     500   00:00:41 
+any=3@500[nt-1]       Busy        0016 280     345     500   00:00:43 
+any=3@500[nt-1]       Busy        0017 71      346     500   00:00:43 
+any=3@500[vaio]       Busy        0022 11499   347     500   00:00:39 
+any=3@500[vaio]       Busy        0023 11501   348     500   00:00:39 
+any=3@500[imac]       Busy        0018 22534   351     500   00:00:42 
+any=3@500[imac]       Busy        0019 22536   352     500   00:00:42 

	 TIME PASSES, RENDERS START FINISHING, 3 CPU LIMIT TAKES EFFECT
% rush -lc
CPUSPEC[HOST]          STATE       FRM  PID     JOBTID  PRI   ELAPSED  NOTES
+any=3@500[tahoe]      Busy        0026 11459   343     500   00:00:04 
+any=3@500[tahoe]      JobPass     -    -       344     500   00:01:01 Cpu max (3/3 busy)
+any=3@500[nt-1]       Idle        -    -       345     500   00:01:01 Cpu max (4/3 busy)
+any=3@500[nt-1]       Idle        -    -       346     500   00:01:01 Cpu max (4/3 busy)
+any=3@500[vaio]       JobPass     -    -       347     500   00:01:01 Cpu max (3/3 busy)
+any=3@500[vaio]       JobPass     -    -       348     500   00:01:01 Cpu max (3/3 busy)
+any=3@500[imac]       Busy        0024 22540   351     500   00:00:07 
+any=3@500[imac]       Busy        0025 22542   352     500   00:00:05

  rush -criteria  
rush -criteria 'criteria strings' [jobid..]

Job Criteria.

Sets the criteria string for the job. Use '-' to specify no criteria.

See Criteria submit command for more info on how criteria is specified.

  rush -dcatlog  
rush -dcatlog [host..]

Daemon cat(1) Log File.

View the rushd.log daemon on the local [remote] host.

  • rush -dcatlog prints the current rushd.log (usually today's)
  • rush -dcatoldlog prints the last rotated log (usually yesterday's).

Caveat: for reasons strange and unusual, all daemon 'catlog' output has a '>' prepended to each line.

  rush -deltaskfu  
rush -deltaskfu [..]

Delete Task, Force Unconditionally

(Administrative) Users should not use this command. Use 'rush -rc' to remove cpus from your job.

Deletes individual 'tasks' from a job server or cpu server. Only root and the rush adminuser can use this command.

This command has two forms; in one case it takes two arguments where a task (or cpu) is forcibly removed from the job's cpu report, and in the second case a task (or cpu reservation) is forcibly removed from the cpu server's tasklist (i.e. 'rush -tasklist'). Examples:

    Delete Task Examples
    rush -deltaskfu vaio.55 117 Delete JOBTID #117 from job vaio.55
    rush -deltaskfu vaio.55 183 tahoe    Delete CPUTID #183 (from 'rush -tasklist') on host tahoe

JOBTID values can be found in -lc, -lcf and -tasklistfull reports.
CPUTID values can be found in -lcf and -tasklistfull reports.

Caveat: This command is not intended for casual use. It is to be used by administrators in unusual situations where cpus are stuck due to communications problems between machines. Use this command carefully.

  rush -dependon  
rush -dependon [-|depjobid[,..]]

Set Dependency On Other Jobs.

Sets new 'dependon' jobids for the current [specific] job. See the submit script docs on DependOn for more info.

Read the examples carefully, to avoid confusing jobid(s) to affect with new jobids for the dependon command; remember to separate all 'dependon' jobids with commas instead of spaces.

Caveats:

Setting 'dependon' to a new value disables all previous values (i.e., clears them first).

Setting dependon to '-' will disable it, but you will have to un-Hold any frames manually.

All 'dependon' jobids must have the same hostname as the job being modified.

See also, rush -waitfor.

To differentiate between the new value for 'dependon', and the jobid(s) to be affected, the new value for 'dependon' must be a comma-separated list of jobids with no spaces. Examples:

rush -dependon - # Disable all dependencies
 
rush -dependon tahoe.37,tahoe.38, # Job now depends on 'tahoe.37' and 'tahoe.38'.
# Trailing comma required
 
rush tahoe.40 -dependon tahoe.37,tahoe.38,     # tahoe.40 now depends on 'tahoe.37' and 'tahoe.38'
 
rush tahoe.40 -dependon tahoe.37, # tahoe.40 now depends on job 'tahoe.37'
# Trailing comma on tahoe.37 is required

  rush -dexit  
rush -dexit [remotehost..]

Daemon Exit.

(Administrative) Tells a daemon to exit immediately via TCP. Acknowledges success. Only root and the rush adminuser can use this command.

(New in rush 102.41 and up) Automatically saves job checkpoint before exiting.

  rush -dexitnow  
rush -dexitnow [remotehost..]

Daemon Exit Now.

(Administrative) Tells daemon to exit immediately, via UDP. There is no acknowledge. Only root and the rush adminuser can use this command. Use only in case of emergency.

(New in rush 102.41 and up) Automatically saves job checkpoint before exiting.

  rush -dlog  
rush -dlog <flags> [remotehost..]

Daemon Debug Logging

(Administrative) Enables certain daemon debug logging flags. Only root and the rush adminuser can use this command. Use 'rush -dlog -' to disable all debugging.

Logging Flags


These are flags that can be used with:

  • 'rush -dlog' to enable daemon debugging messages in the rushd.log, eg: 'rush -dlog utE'
  • 'rush -d' to enable client side debugging messages to the user's screen, eg: 'rush -d t -lfi'
  • In the rush.conf file's LogFlags

These flags can be combined to accumulate logging verbosity.

  • All flags can be enabled by specifying 'a', e.g. 'rush -dlog a'.
  • All flags can be disabled by specifying '-', e.g. 'rush -dlog -' 

    - - A single dash is used to disables all debugging flags
    a - all
    b - bump mechanism logging
    c - select() info
    d - log duplicate/redundant receipt of packet drops
    e - events (time oriented, async, lic unpause)
    f - fork/wait
    h - hostname lookups
    j - Log job submissions and job state changes
    k - Log bumped/killed/usurped tasks
    l - Logical string evaluations
    m - memory calculations (RAM) during priority battles, etc
    n - network commands (udp/tcp)
    o - connect()/open()/close()/bind()/socket() (low level)
    p - parse command line arguments, submit scripts
    q - sequence id errors are printed (default:off)
    r - reboot management/transactions
    s - signals
    t - tcp
    u - udp
    w - 'waitfor' checks
    y - yp lookups
    C - class ToWords/FromWords (PROBABLY OBSOLETE)
    D - dumpsig debugging (Bluesky: 1.21)
    F - File loading line-by-line debugging
    E - Errors not normally displayed (benign, but suspect)
    J - Job Objects (win2k job objects kill mechanism msgs)
    K - Actual kill commands executed
    L - Locks on files
    M - Microsoft specific debugging
    P - server: Priority mechansim, client: log 'rush -push' mechanism
    R - Reaper msgs
    S - Server/Client context switches
    T - task/taskack transactions
    U - update (scheduling, priority mechanism, idle cpu management)
    W - 'Wakeup' pass states; pass->unpass state changes
    X - Random UDP message dropping -- TESTING ONLY!!
        ('a' does not affect this option, it must be specified explicitly)
    Z - reserved for on the fly debugging problems

    @ - PROTOCOL TEST FOR TASK_TIMEOUT. 
        Induces 60 second sleep before sending TASK_START.
        ('a' does not affect this option, it must be specified explicitly)

Examples:

rush -dlog ut tahoe      Enable UDP (u) and TCP (t) debugging for daemon on host tahoe
rush -dlog - tahoe Disable all debugging flags for the daemon on host tahoe

  rush -dlogstats  
rush -dlogstats [host|+hostgroup..]

(New in rush 102.40f and up)

Daemon Log Statistics

(Administrative) Useful for easily determining problems with machines on large networks.

Checks the local [remote] daemon log for particular kinds of errors, and logs the number of times each error appears in a table summary. 

% rush -dlogstats +any
#                        ********* Problems ********** **Security* ************* Normal **************
# SID   Hostname         Alert  Fail Error  Exit  Fork  Bump Secur Start   Rot  Info JobSt JobEn JobKi
l 0     geneva               2     0     0     0     0     0     0     0     1     0     0     0     0
l 0     rotwang              0     0     0     1     0     0     1     1     1     2     0     0     0
l 0     ontario              0     0     0     0     0     0     0     1     1     2     1     0     0

This report shows that geneva has two 'ALERT' errors in its daemon log, which might indicate a problem worth investigating with 'rush -dcatlog geneva | grep ALERT'.

  rush -done  
rush -done <framerange|framestate..> [jobid..]

Done Frames.

Changes frame(s) into the Done state for current [specific] job. If frames are running, they are killed and enter the 'Done' state on completion.

The frames to affect can either be specified as a frame range (i.e. 1-100) or as a frame state, for which all frames matching that state will be changed. Examples:

% rush -done 1-100 # 'Done' frames 1 through 100
% rush -done fail # 'Done' all frames currently Fail
% rush -done fail que    # 'Done' all frames currently Fail or Que

  rush -donecommand  
rush -donecommand <command|-> [jobid..]
rush -donecommand [jobid..] < file

PHASE OUT.

Synonym for -jobdumpcommand, which is what future programs should use in its place.

  rush -donemail  
rush -donemail [email[,email..]] [jobid..]

Set Job Done Mail.

Changes who to send mail to when job is done. In version 102.40h+, this means when the job renders its last frame, or when the job is dumped. Which means if you requeue frames after a job is done, you will receive an email each time the last frame is rendered.

See DoneMail submit command for more info and examples.

See also DumpMail submit command if you want mail sent only once, when the job is actually dumped.

  rush -down  
rush -down downhost[,downhost..] jobhost

A Machine Is Down.

Tells rush jobs a machine is down.

If a machine hangs or goes down, rush has no way to know it.. for all rush knows, the remote machine is just under heavy load, and is being unresponsive.

Use '-down' to requeue frames in jobs that are stuck in the Run or Start state, because the remote machine is either hung or has been turned off unexpectedly.

Normally frames will requeue when the remote is rebooted, but in cases where the machine needs to stay off for a while, the user can use '-down' to manually tell rush the remote is down, and to requeue the hung frames elsewhere.

This also helps jobs that won't dump because some frames are 'stuck' on a host that is down.

Warning: This command should only be used in extreme situations where job(s) show the frames running on a remote machine that is known to be down, or not running the frame.

rush -down huron,erie tahoe     # tell all jobs on tahoe
# that machines 'huron' and 'erie' are down
rush -down huron,erie +any     # tell all hosts that 'huron' and 'erie' are down

  rush -dump  
rush -dump [jobid|user|user@host..]

Dump Job.

Dumps the job. Busy frames are killed right away.

If no arguments are specified, the RUSH_JOBID variable is used to determine which job to dump.

If jobid(s) are specified, those jobs will be dumped.

If a user's name is specified, all jobs owned by that user on the local machine will be dumped (e.g., 'rush -dump fred').

In the case of user@host, all jobs owned by 'user' served by 'host' will be dumped.
i.e., 'rush -dump fred@tahoe' will dump all jobs owned by fred that were submitted to host tahoe.

% rush -dump tahoe.37 tahoe.38    # Dumps jobs tahoe.37 and tahoe.38
% rush -dump fred # Dumps all jobs owned by 'fred' served by the local machine
% rush -dump fred@superior # Dumps all jobs owned by 'fred' served by host 'superior'

  rush -end  
rush -end [jobid..]

End Job.

Ends a job. Similar to rush -dump, but waits for running frames to finish (i.e., running frames are not killed).

  rush -fail  
rush -fail <framerange|framestate..> [jobid..]

Fail Frames.

Changes frame(s) into the Fail state for current [specific] job. If frames are running, they are killed and enter the 'Fail' state on completion.

The frames to affect can either be specified as a frame range (i.e., 1-100) or as a frame state, for which all matching frames will be affected. Examples:

% rush -fail 1-100 # Fail frames 1 through 100
% rush -fail done # Fail all frames currently Done
% rush -fail done hold    # Fail all frames currently Done or Hold

  rush -frameflags  
rush -frameflags <-|keepnotes..> [jobid..]

Change Frame Flags.

    Frame Flags
    rush -frameflags keepnotes Disables automatic clearing of 'NOTES" field in framelist
    when a frame starts rendering.
    rush -frameflags - No flags. (Default)
    Default behavior is for NOTES field in framelist to be cleared
    whenever a frame starts rendering.

See also the submit script docs on FrameFlags for more info.

  rush -fu  
rush [args..] -fu

Force User1

The -fu flag augments other operations. It overrides warnings pertaining to lack of permission to do an operation because the user invoking it is not the owner of the job.

For instance, if you want to control another person's job, you might get an error, e.g.:

% rush -an vaio va-229 # Attempt to add 'vaio' as a nevercpu to someone's job
rush: va-229: you're not owner!   # Fails because you're not the job's owner
 
% rush -an vaio va.229 -fu         # Same command with -fu to force it...
Add nevercpu 'vaio' # ...now it works

See also the $RUSH_FU environment variable.

Note: This feature may be disabled by the system administrator via the DisableFu flag in the Rush Configuration File.

1 This acronym is rumored to have alternate, pejorative expansions.

  rush -getoff  
rush -getoff [-msg 'text'] [remotehost|+hostgroup..]

Get Off A Machine.

Similar to 'rush -offline', but kills and requeues any currently running frames before taking the localhost [remotehost] offline.

If no arguments are specified, assumes the localhost.

(New in rush 102.40 and up) An optional remark can be specified via "-msg 'text'" which will show up in 'rush -lac' reports, eg: 

% rush -getoff -msg 'Client Demo' ontario
ontario: Offline (killing 1 procs)

% rush -lac
HOST       OWNER    JOBID          TITLE                     FRM  PRI   PID        ELAPSED REMARKS
---------- -------- -------------- ------------------------- ---- ----- ------- ---------- --------
ontario    -        -              -                         -    -     -          Offline Client Demo
:

Examples of 'rush -getoff':

% rush -getoff # Kills frames running on localhost (requeing them elsewhere)
# leaving the localhost offline (won't accept any more renders)
% rush -getoff tahoe erie      # Kills frames running on tahoe and erie (requeing the frames elsewhere)
# leaving both hosts offline (won't accept any more renders)
% rush -getoff -msg 'Bad Memory' huron # offline 'huron' with the remark 'Bad Memory' (New in 102.40)

  rush -hold  
rush -hold <framerange|framestate..> [jobid..]

Hold Frames.

Changes frame(s) into the Hold state for current [specific] job. If frames are running, they are killed and enter the 'Hold' state on completion.

When a frame is in the Hold state the frame will not be rendered, and a job will not autodump if there are any Hold frames in the frame list.

The frames to affect can either be specified as a frame range (i.e., 1-100) or as a frame state, for which all matching frames will be affected. Examples:

% rush -hold 1-100 # Hold frames 1 through 100
% rush -hold fail # Hold all frames currently Fail
% rush -hold fail done    # Hold all frames currently Fail or Done

  rush -imgcommand  
rush -imgcommand 'command [args..]' [jobid..]
rush -imgcommand [jobid..] < file

Set Image Display Command

Used by IRUSH to determine the command to execute whenever the user double clicks on a frame in the IRUSH 'Frames' report. Normally, the command invoked should display the rendered image for that frame.

This should be a complete command to view an image in your favorite image viewer, with %d or %04d in place of the frame number.

If 'cmd [args]' are not supplied, the command will be read from stdin. This is useful to avoid shell quoting issues. If multiple lines are specified, they will be joined into one single line.

The image viewing command must background itself, or you must force it by using '&' (Unix), or 'start /b' (Windows). Examples:

% rush -imgcommand 'xv /job/images/foo.%04d.sgi &' # (Unix) Invokes 'xv' to view images
C:\> rush -imgcommand "start /b flip //nt/images/s1.%04d.png" # (Windows) Invokes 'flip' in background to view images