From: "Anton Blomberg" <anton.blomberg@swiss.se>
Subject: Jobdonecommand not executed
   Date: Thu, 21 Feb 2008 12:00:25 -0500

Hi
I'm trying to use the -jobdonecommand to execute a python script when a rush 
job is done.
There is no problem with the submit or rendering, but the jobdonecommand is 
never executed.
I set it to:
jobdonecommand python //server01/jobdone.py
but that script never runs. And no errors about it is logged.
Are there any parameters that can interfere with the jobdonecommand that I 
need to set to enable/disable it?

/Anton
--
Swiss International | www.swiss.se

Anton Blomberg wrote:
> I'm trying to use the -jobdonecommand to execute a python script when a rush 
> job is done.
> There is no problem with the submit or rendering, but the jobdonecommand is 
> never executed.
> I set it to:
> jobdonecommand python //server01/jobdone.py

Hi Anton,

	If you're using one of the submit scripts, and are using
	the "Job Done Command:" prompt, then you would just type at
	the prompt:

		python /path/to/your/jobdone.py

	(ie. without typing 'jobdonecommand')

	When the last frame finishes rendering, that command
	will be executed, and the stdout/stderr of the command
	will appear in a file called 'jobdonecommand.log' in the
	log directory for your job.

	You would only specify the 'jobdonecommand' as part of the
	command *if* you are supplying this in the 'Submit Options:'
	prompt of the submit form (where you can insert any submit
	commands), or if you are writing your own submit script,
	and are doing all the submit commands yourself, eg:

rush -submit << EOF
    title          YOUR_JOB
    frames         1-10
    command        python /path/to/your/command.py -arg1 -arg2
    jobdonecommand python /path/to/your/jobdone.py
EOF

	If you are still having trouble, supply more details
	about which prompt you're filling out, and include
	the contents of 'rush -ljf' and 'rush -lf' for your job,
	the directory contents of your log directory, and if
	there is one, the contents of the 'jobdonecommand.log'
	file.

> but that script never runs. And no errors about it is logged.
> Are there any parameters that can interfere with the jobdonecommand that I 
> need to set to enable/disable it?

	There shouldn't be anything else you should need to set.
	Let me know how it goes by following up on the group.

-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

Hi Greg,
thank you for your fast answer.

> If you are still having trouble, supply more details
> about which prompt you're filling out, and include
> the contents of 'rush -ljf' and 'rush -lf' for your job,
> the directory contents of your log directory, and if
> there is one, the contents of the 'jobdonecommand.log'
> file.

I am using a custom made python GUI-script to do the submit.
It seems not to be the problem due to all the info showing up properly for 
the job in irush.
There must be something wrong with my syntax/some setting that prevents the 
script to run.
Here is my 'rush -ljf' for the test job (i shortened down things like long 
paths and command, because there's no problems with it and it takes a lot of 
long args):

C:\Rush\bin>rush.exe -ljf jobserver.709
          Jobid: jobserver.709
          State: Done
       ActState: Done
       RebootId: 13
          Title: MAYA_TEST.MB
    DefPriority: 1
         LogDir: //logserver/logs/rush/jobserver.709
         LogExt:
       LogFlags: -
     ImgCommand: //tools/imgc.pyw
            Ram: 2048
           Nice: 10
        MaxCpus: 0
        MaxTime: 02:00:00
   MaxTimeState: fail
   LicPauseSecs: 00:01:00
  LicPauseTimer: 00:00:00
     FrameFlags: off
        Command: python //tools/maya-render.py -arg -arg -arg
ChkCommand:
 JobDoneCommand: python //tools/jobdone.py
JobStartCommand:
 JobDumpCommand:
   AutoDump: off
   WaitForState: dump
    WaitForTime: -
           User: anton.blomberg (11002/11001)
       DoneMail: -
       DumpMail: -
      StartDate: Thu Feb 21 17:10:36 2008
        EndDate: -
        Elapsed: 17:40:49
         Frames: 1
      FramesQue: 0
      FramesRun: 0
     FramesDone: 1
     FramesFail: 0
     FramesHold: 0
           Cpus: doris4=1@1
       Notes[0]: -
       Criteria:

and here is 'rush -lf':
STAT FRAME   TRY HOSTNAME        PID     JOBID            START 
ELAPSED  NOTES
Done 0001    7   doris4          716     jobserver.709      02/21,17:40:53 
00:00:35


there is no jobdonecommand.log.


/Anton
--
Swiss International | www.swiss.se


"Greg Ercolano" <erco@(email surpressed)> wrote in message 
news:1691-rush decimal general at seriss decimal com...
> Anton Blomberg wrote:
>> I'm trying to use the -jobdonecommand to execute a python script when a 
>> rush
>> job is done.
>> There is no problem with the submit or rendering, but the jobdonecommand 
>> is
>> never executed.
>> I set it to:
>> jobdonecommand python //server01/jobdone.py
>
> Hi Anton,
>
> If you're using one of the submit scripts, and are using
> the "Job Done Command:" prompt, then you would just type at
> the prompt:
>
> python /path/to/your/jobdone.py
>
> (ie. without typing 'jobdonecommand')
>
> When the last frame finishes rendering, that command
> will be executed, and the stdout/stderr of the command
> will appear in a file called 'jobdonecommand.log' in the
> log directory for your job.
>
> You would only specify the 'jobdonecommand' as part of the
> command *if* you are supplying this in the 'Submit Options:'
> prompt of the submit form (where you can insert any submit
> commands), or if you are writing your own submit script,
> and are doing all the submit commands yourself, eg:
>
> rush -submit << EOF
>    title          YOUR_JOB
>    frames         1-10
>    command        python /path/to/your/command.py -arg1 -arg2
>    jobdonecommand python /path/to/your/jobdone.py
> EOF
>
> If you are still having trouble, supply more details
> about which prompt you're filling out, and include
> the contents of 'rush -ljf' and 'rush -lf' for your job,
> the directory contents of your log directory, and if
> there is one, the contents of the 'jobdonecommand.log'
> file.
>
>> but that script never runs. And no errors about it is logged.
>> Are there any parameters that can interfere with the jobdonecommand that 
>> I
>> need to set to enable/disable it?
>
> There shouldn't be anything else you should need to set.
> Let me know how it goes by following up on the group.
>
> -- 
> Greg Ercolano, erco@(email surpressed)
> Seriss Corporation
> Rush Render Queue, http://seriss.com/rush/
> Tel: 626-795-5922
> Fax: 626-795-5947
> Cel: 310-266-8906 

Anton Blomberg wrote:
> C:\Rush\bin>rush.exe -ljf jobserver.709
>           Jobid: jobserver.709
> [..]
>          LogDir: //logserver/logs/rush/jobserver.709
> [..]
>  JobDoneCommand: python //tools/jobdone.py

	Hmm, that all looks OK, provided the directory
	//logserver/logs/rush/jobserver.709/
	..and the path
	//tools/jobdone.py
	..are both accessible from host "jobserver"
	(which is where the jobdonecommand is invoked).

	eg. login to machine 'jobserver' as anton.blomberg,
	open a DOS window and invoke these two commands:

dir \\logserver\logs\rush\jobserver.709
dir \\tools/jobdone.py

	..if they both work, then I'm stumped (see below).

	If they fail, then that's the problem; the paths
	have to resolve on the jobserver as well as the
	render nodes.

	BTW, the fact I'm showing backslashes (\) in that dir command
	instead of frontslashes (/) is just to serve the purposes of
	DOS; the frontslashes you have submitted with are fine.

	I'm sure it's /trying/ to run the command, but my guess
	offhand is it can't because maybe it can't reach either
	of those two directories from the machine "jobserver".

	Take a look in jobserver's c:\rush\var\Orushd.log
	to see if there are any errors about the command or the
	log directory. If rush couldn't run the command, and couldn't
	create the log, it would advertise the problem in the rushd.log
	on the job server machine (in this case, "jobserver").

	Look in the Orushd.log (as opposed to the rushd.log) because
	the command tried to run yesterday (02/21), and the rushd.log
	rotates out at midnight ever night.

	If need be, contact me directly via private email, and
	we can arrange an interactive chat so we can try a few
	things in realtime.

-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

Greg Ercolano wrote:
> 	eg. login to machine 'jobserver' as anton.blomberg,

    Oops, forgot this is a windows machine.

    Correction to the above: login to machine 'jobserver'
    as *what ever user the rushd service is configured to run as*,
    since that is the user rush will be running your command as,
    and is the permission context that will be used to access the
    file server.

    BTW, I'm a little surprised that the UNC paths you're using
    don't include a volume name. Normally UNC paths are of the format:

\\someserver\somevolume\xxx

    but your path to the jobdone.py script seems to be missing a volume
    name, which I think is not possible under windows.

    Also, a typo in my last response:

> dir \\logserver\logs\rush\jobserver.709
> dir \\tools/jobdone.py

    The frontslash in that last line should be a backslash, eg:

	dir \\tools\jobdone.py

    My guess is that's not a valid path, due to the missing volume name,
    but since the log directory looks correct (\\logserver\logs\rush\..)
    I would think the jobdonecommand.log would be created, and show an
    error message from python about the script not being found.

    However, if \\logserver\logs is not a valid UNC path on jobserver,
    that would explain the absence of the jobdonecommand.log, and surely
    there will be an error in the rushd.log about it.


-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

> 	Take a look in jobserver's c:\rush\var\Orushd.log
>	to see if there are any errors about the command or the
>	log directory. If rush couldn't run the command, and couldn't
>	create the log, it would advertise the problem in the rushd.log
>	on the job server machine (in this case, "jobserver").

    Turns out this was the problem being advertised in the rushd.log:

02/29,15:50:58 INFO       jobdonecommand 'python //agnes/resources/tech/admin/jobdone.py' failed for jobid=sierra2.890: unknown uid 11002

    ..which means the submitting user's uid/gid on the job server
    was different from the uid/gid for his account on the machine
    he was submitting from, the discrepancy causing this error.

    Solution is for the sysadmin to reconfigure his account on
    the job server to have matching uid/gid values to his account
    on the submit host.