From: Greg Ercolano <erco@(email surpressed)>
Subject: [Q+A]  Is there a simple non-GUI python script to submit Maya 2011
   Date: Thu, 20 Jan 2011 15:33:27 -0500
Msg# 1996
View Complete Thread (8 articles) | All Threads
Last Next
We're messing around with writing our own submit script in python..
is there a good place to start?
    Yes; have a look at:
    http://www.seriss.com/rush-current/rush/rush-submit.html#Python-Irush

    This shows a simple python script that submits a job as well as
    manages its rendering.

    This example is actually written in several languages on that same
    page, so it's kind of a 'Rosetta Stone' for the different languages.

    In the case of the python script shown, it handles batching.
    You would just need to tweak in some Maya specifics (eg. setting
    Maya 2011 environment variables for the different platforms, and
    invoking the maya "Render" command)

    Here is a slightly modified version of that script that does just that.
    I've highlighted in red the Maya specific stuff added, to help you out.
    The text in green is the text to change for the job you want to submit.

#!/usr/bin/python
import os,sys,re

# Parse jobid from rush submit
def ParseJobid(rushoutfile):
    rushout = open(rushoutfile, 'r')
    rushout_lines = ""
    while 1:
        line = rushout.readline()
        if ( line == "" ):
            break
        rushout_lines += line
    print rushout_lines
    rushout.close()
    return(re.search("RUSH_JOBID.(\S+)", rushout_lines).groups()[0])

# SUBMIT THE JOB IF NO ARGS SPECIFIED
if len(sys.argv) <= 1:
    if ( os.environ.has_key("RUSH_ISDAEMON") ):		# Prevent 'network worm' style recursion
        sys.exit(1)

    # User should change these as needed
    title  = "MY_TEST"                        # Title
    ram    = 10                               # Ram this job needs
    sfrm   = 1                                # Start frame
    efrm   = 23                               # End frame
    batch  = 5                                # Batch frames (1 for none)
    scene  = "//meade/net/tmp/scenes/foo.ma"  # Scene file to render
    logdir = "//meade/net/tmp/logs"           # Log directory based on scene pathname
    cpus   = "+any=5.1@1"                     # Cpus the job should use

    # Create log directory if none
    if ( not os.path.isdir(logdir) ):
        os.mkdir(logdir, 0777)

    # SUBMIT THE JOB
    #    Save output to a temp file so we can parse back jobid..
    #
    tmpfile = "/var/tmp"
    if ( os.path.isdir("c:/temp") ):
        tmpfile = "c:/temp"
    tmpfile = "%s/submit-out.%d" % ( tmpfile, os.getpid() )

    # Submit the job
    submit = os.popen("rush -submit > " + tmpfile, 'w')
    submit.write("title    %s\n" % title +
                 "ram      %d\n" % ram +
                 "frames   %d-%d,%d\n" % ( sfrm, efrm, batch) +
                 "logdir   %s\n" % logdir +
                 "command  python %s -render %s %s %s\n" % ( sys.argv[0], scene, batch, efrm ) +
                 "cpus     %s\n" % cpus)
    err = submit.close()

    # Read submit output, parse jobid, remove tmp file
    os.environ["RUSH_JOBID"] = ParseJobid(tmpfile);
    os.remove(tmpfile)

    # Submit Failed
    if ( err or os.environ["RUSH_JOBID"] == "" ):
        sys.exit(1)

    # Submit OK
    print "--- Starting irush.."		# optional
    os.system("irush -button Frames")		# optional
    sys.exit(0)

# RENDERING ON REMOTE MACHINE?
if ( sys.argv[1] == "-render" ):

    # Set maya environment variables
    if ( os.path.isdir("c:/") ):
	### WIN
	os.environ["PATH"] = "c:/Program Files/Autodesk/Maya2011/bin;" + os.environ["PATH"]
    elif ( os.path.isdir("/Applications") ):
	### MAC
	os.environ["PATH"] = "/Applications/Autodesk/maya2011/Maya.app/Contents/bin:" + os.environ["PATH"]
    elif ( os.path.isdir("/usr/autodesk") ):
	### LINUX
	os.environ["PATH"] = "/usr/autodesk/maya2011/bin:" + os.environ["PATH"]

    scene   = sys.argv[2]
    batch   = int(sys.argv[3])
    lastfrm = int(sys.argv[4])
    sfrm    = int(os.environ["RUSH_FRAME"])
    efrm    = int(os.environ["RUSH_FRAME"]) + batch - 1
    if ( efrm > lastfrm ):
        efrm = lastfrm

    # PRINT FRAMES BEING RENDERED
    if ( sfrm == efrm ):
        print "--- Working on frame %d" % sfrm
    else:
        print "--- Working on frames %d - %d" % ( sfrm, efrm )

    # START RENDER, CHECK FOR ERRORS
    cmd = "Render -s %d -e %d %s" % ( sfrm, efrm, sys.argv[2] )
    print "--- Executing: %s" % cmd
    sys.stdout.flush()
    err = os.system(cmd)
    if err:
        print "--- Render failed (EXIT CODE=%s)" % (err >> 8)  # Failed? show error code
        sys.exit(1)                                            # 'sys.exit(1)' tells rush frame "Fail"

    print "--- Render OK"                                      # Worked?
    sys.exit(0)                                                # 'sys.exit(0)' tells rush frame "Done"

# BAD ARGUMENT
stderr.write("%s: unknown argument %s\n" % ( sys.argv[0], sys.argv[1] ) )
sys.exit(1)


-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)


   From: Daniel Browne <dbrowne@(email surpressed)>
Subject: Priorities in CPU menu
   Date: Thu, 27 Jan 2011 13:50:19 -0500
Msg# 2002
View Complete Thread (8 articles) | All Threads
Last Next 
Hi Greg & Co.,

I'm implementing render priority levels, and the default selections I'm adding to my submit scripts don't display properly in the CPU drop down lists. The @ symbol and priority level aren't visible until you select one and the field updates. Backslash didn't seem to escape it.

-Dan

   From: Greg Ercolano <erco@(email surpressed)>
Subject: Re: Priorities in CPU menu
   Date: Thu, 27 Jan 2011 14:43:30 -0500
Msg# 2003
View Complete Thread (8 articles) | All Threads
Last Next 
Daniel Browne wrote:
> I'm implementing render priority levels, and the default selections I'm add=
> ing to my submit scripts don't display properly in the CPU drop down lists.=
>  The @ symbol and priority level aren't visible until you select one and th=
> e field updates. Backslash didn't seem to escape it.

Hi Daniel,

	Hmm, can you show some perl code snippets to give context
	as to where you're adding this?

	'@' is a special character in perl, and in some contexts
	it's a special character in the GUI as well.

	For instance, if specifying:

$somevar = <<"EOF";
..
+any=2\@100
..
EOF

	..and 'somevar' ends up being shown in the GUI, and the
	GUI isn't showing it properly, try in your perl script:

$somevar = <<"EOF";
..
+any=2\@\@100
..
EOF

	which may help the GUI look right.

	Definitely give some context on what you're doing
	and how it's being used to show up in the GUI.
	It kinda sounds like you might be trying to use
	a pulldown list, or some such.

-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)


   From: Daniel Browne <dbrowne@(email surpressed)>
Subject: Re: Priorities in CPU menu
   Date: Thu, 27 Jan 2011 15:12:39 -0500
Msg# 2004
View Complete Thread (8 articles) | All Threads
Last Next 
The double + escaped @ symbols did the trick. This was the GUI definition section where you establish all the presets in your drop-down menus. Thanks Greg.


On Jan 27, 2011, at 11:43 AM, Greg Ercolano wrote:

[posted to rush.general]

Daniel Browne wrote:
> I'm implementing render priority levels, and the default selections I'm add=
> ing to my submit scripts don't display properly in the CPU drop down lists.=
> The @ symbol and priority level aren't visible until you select one and th=
> e field updates. Backslash didn't seem to escape it.

Hi Daniel,

	Hmm, can you show some perl code snippets to give context
	as to where you're adding this?

	'@' is a special character in perl, and in some contexts
	it's a special character in the GUI as well.

	For instance, if specifying:

$somevar = <<"EOF";
..
+any=2\@100
..
EOF

	..and 'somevar' ends up being shown in the GUI, and the
	GUI isn't showing it properly, try in your perl script:

$somevar = <<"EOF";
..
+any=2\@\@100
..
EOF

	which may help the GUI look right.

	Definitely give some context on what you're doing
	and how it's being used to show up in the GUI.
	It kinda sounds like you might be trying to use
	a pulldown list, or some such.

-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

----------
Dan "Doc" Browne
System Administrator
Evil Eye Pictures

dbrowne@(email surpressed)
Office: (415) 777-0666 x105



   From: Greg Ercolano <erco@(email surpressed)>
Subject: Re: Priorities in CPU menu
   Date: Thu, 27 Jan 2011 16:09:58 -0500
Msg# 2005
View Complete Thread (8 articles) | All Threads
Last Next 
Daniel Browne wrote:
> The double + escaped @ symbols did the trick. This was the GUI definition s=
> ection where you establish all the presets in your drop-down menus. Thanks =
> Greg.

	If that's the case, keep in mind that what you send into the interface
	is what will come out. So if "foo@@bar" goes into the GUI pulldown menu,
	foo@@bar is what will come out the other end if the user picks it.
	(data in will be the same as data out).

	So be sure to handle converting that '@@' back into '@' on the
	post-submit end of things.


-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)


   From: Daniel Browne <dbrowne@(email surpressed)>
Subject: Adding Linux boxes
   Date: Mon, 07 Feb 2011 14:33:37 -0500
Msg# 2007
View Complete Thread (8 articles) | All Threads
Last Next 
Hi Greg,

I don't know if I mentioned this in my last correspondence but I'm running tests to integrate linux into our render farm. My implementation and initial rush render tests have been successful. I thought I'd ask if based on your experiences you thought it better to have discreetly separate settings directories on the network (i.e. shaders, plugins, library elements). I'm just trying to think of how best to balance the needs of each platform vs ease of maintenance.

Thanks,

-Dan

----------
Dan "Doc" Browne
System Administrator
Evil Eye Pictures

dbrowne@(email surpressed)
Office: (415) 777-0666 x105



   From: Greg Ercolano <erco@(email surpressed)>
Subject: Re: Adding Linux boxes
   Date: Mon, 07 Feb 2011 14:47:18 -0500
Msg# 2008
View Complete Thread (8 articles) | All Threads
Last Next 
Daniel Browne wrote:
> Hi Greg,
> 
> I don't know if I mentioned this in my last correspondence but I'm running =
> tests to integrate linux into our render farm. My implementation and initia=
> l rush render tests have been successful. I thought I'd ask if based on you=
> r experiences you thought it better to have discreetly separate settings di=
> rectories on the network (i.e. shaders, plugins, library elements). I'm jus=
> t trying to think of how best to balance the needs of each platform vs ease=
>  of maintenance.

Hi Dan!

	Hmm, do you mean installing the renderer/plugins and such on an NFS server
	vs. locally on each machine, and having PATH settings point to the NFS
	server? Or perhaps by "settings" you mean initialization files that get
	loaded by these programs from the users home directory, or..?
	(I think I didn't quite fully catch your question)

	Some renderers let you install the software on NFS, some don't.
	For instance, some installers like to put stuff on the local machine
	(eg. .so's, mods to the user's login paths, etc). Usually under unix
	these can be coerced to be on NFS servers, but under Windows they often
	like/need to dive into the registry to install properly, and drop stuff
	in c:\windows\system32.

	The thing about installing software on an NFS server is that the
	executables have to load and 'page' over the network, and some
	applications (maya) need to load a LOT of stuff from its install
	directories to come up properly (like .mel and python initialization files)
	which are usually much quicker to load from local disk than over NFS,
	thus impacting per-frame render times. So in other words it's often
	worth it to go to the extra effort to install the software locally
	on each machine to keep render times smaller. But in some cases it
	doesn't matter; depends on how much you find it affects your render
	times, and if your renderer/shaders/whatever behave properly when
	installed over the network.


-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)


   From: Daniel Browne <dbrowne@(email surpressed)>
Subject: Re: Adding Linux boxes
   Date: Mon, 07 Feb 2011 14:57:50 -0500
Msg# 2009
View Complete Thread (8 articles) | All Threads
Last Next 
I was thinking primarily in terms of shaders or  dylibs; the maya binary itself is always local to the render node. Speaking of which, do you happen to know of a good guide to the syntax of mentalray rayrc files?


On Feb 7, 2011, at 11:47 AM, Greg Ercolano wrote:

[posted to rush.general]

Daniel Browne wrote:
> Hi Greg,
> 
> I don't know if I mentioned this in my last correspondence but I'm running =
> tests to integrate linux into our render farm. My implementation and initia=
> l rush render tests have been successful. I thought I'd ask if based on you=
> r experiences you thought it better to have discreetly separate settings di=
> rectories on the network (i.e. shaders, plugins, library elements). I'm jus=
> t trying to think of how best to balance the needs of each platform vs ease=
> of maintenance.

Hi Dan!

	Hmm, do you mean installing the renderer/plugins and such on an NFS server
	vs. locally on each machine, and having PATH settings point to the NFS
	server? Or perhaps by "settings" you mean initialization files that get
	loaded by these programs from the users home directory, or..?
	(I think I didn't quite fully catch your question)

	Some renderers let you install the software on NFS, some don't.
	For instance, some installers like to put stuff on the local machine
	(eg. .so's, mods to the user's login paths, etc). Usually under unix
	these can be coerced to be on NFS servers, but under Windows they often
	like/need to dive into the registry to install properly, and drop stuff
	in c:\windows\system32.

	The thing about installing software on an NFS server is that the
	executables have to load and 'page' over the network, and some
	applications (maya) need to load a LOT of stuff from its install
	directories to come up properly (like .mel and python initialization files)
	which are usually much quicker to load from local disk than over NFS,
	thus impacting per-frame render times. So in other words it's often
	worth it to go to the extra effort to install the software locally
	on each machine to keep render times smaller. But in some cases it
	doesn't matter; depends on how much you find it affects your render
	times, and if your renderer/shaders/whatever behave properly when
	installed over the network.


-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

----------
Dan "Doc" Browne
System Administrator
Evil Eye Pictures

dbrowne@(email surpressed)
Office: (415) 777-0666 x105