From: "Mark Smith" <Mark.Smith@capital-fx.co.uk>
Subject: RE: Using Rush in a slightly different way [for Framecycler]
   Date: Fri, 09 Sep 2005 06:29:55 -0700

Thanks Greg...that works great.

I've been playing around trying to customise the GUI all morning.
Managed to get it to look pretty much how I want with all the required
fields. Unfortunately I'm a Python man and I'm finding Perl to hard to
hack at in order to get my customised GUI to feed in the required info.

Is there a Python version of the submit-generic script?

A GUI for this sort of thing would be really cool. There definitely a
huge need for it where I am (artists just won't use it if it doesn't
have a GUI). We'd love it if you could knock one up for us. It would be
great if it was general enough so that it could be used for similar
tasks like quicktime generation etc.

Thanks,

Mark
 

Mark Smith
IT Engineer
Capital FX
20 Dering Street
London W1S 1AJ
United Kingdom
 
T: +44 (0)20 7493 9998
F: + 44 (0)20 7493 9997
-----Original Message-----
From: Greg Ercolano [mailto:erco@(email surpressed)] 
Sent: 08 September 2005 19:15
To: void@(email surpressed)
Subject: Re: Using Rush in a slightly different way [for Framecycler]

[posted to rush.general]

Mark Smith wrote:
> I'd like to use Rush in a way that it might not have been designed to 
> and thought I'd ask if it was possible first.

	Actually, Rush is designed with customization in mind.
	You can shove just about anything command line oriented
	through it, including backup tape operations, filmout
operations,
	it's even been used for MP3 generation.. what else are you going
	to do with all those CD drives on your render farm? ;)

> We use Framecycler for playing back sequences. To allow real time 
> playback of really long sequences Framecyler has its own proprietary 
> file format called .ihss. Now you can make these ihss files in the 
> Framecycler GUI or use Framecycler's command line utility (no GUI at 
> all) called Sequence Publisher that takes a sequence of frames and 
> creates one .ihss file.
> 
> So I want to make these .ihss files on the farm and do it through
Rush. 

	Sure, should be no problem, me thinks.
	It's just like any other renderer or compositor..

> So instead of doing an operation on lots of frames spread over a farm,
I 
> need to do one huge operation on a whole sequence just on one render 
> node, and output one file.
> 
> Is this possible?

	Yes, you can submit it as a 'single frame' rush job.
	This is done when generating Quicktimes as well.

> Sequence publisher has the following syntax:
> publisher /path/to/frames.#.cin /path/to/outputfile.ihss -r 50% 50%

	You can submit it to Rush using 'submit-generic'.

	Or, you can make your own submit script using this tutorial as a
guide:
	http://www.seriss.com/rush-current/rush/rush-tutorial.html

	..or, just hack up a copy of one of the simpler example scripts
	in the rush/examples directory.

	To submit it through submit-generic, you might specify:



        Shell: perl
    Job Title: FRAME_CYCLER
       Frames: 1
Batch Frames: 1
Log Directory: //path/to/some/empty/dir
      Commands: system('publisher /path/to/frames.#.cin
/path/to/outputfile.ihss -r 50% 50%'); exit(0);
          Cpus: +any=1@50



	I would think that would do it.. just change the pathnames as
needed.

         Set the "Cpus:" to whatever +hostgroup is appropriate for
framecycler,
	ie. whichever group of machines has the publisher software
installed.
	(I assume if you have a mix of Windows and Linux machines, you
probably
	have +win and +linux hostgroups, so to submit only to windows,
you'd
	set "Cpus: +win=1@50" or similar.)

	I didn't put in any error checks, because I don't know if
publisher uses
	exit codes or not. So with the above, Rush will always say
"Done", even
	if it failed.

	To add error checking, you might use for "Commands":

      Commands: my $err = system('publisher /path/to/frames.#.cin
/path/to/outputfile.ihss -r 50% 50%') >> 8;
                if ( $err ) { print "--- FAILED: PUBLISHER RETURNED
$err\n"; exit(1); }
                else        { print "--- SUCCEEDED\n";
exit(0); }


	If there's a general desire for an 'official' GUI oriented rush
submit
	script for this kind of thing, let me know and I'll post one.

-- 
Greg Ercolano, erco@(email surpressed)
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Cel: (Tel# suppressed)
Fax: (Tel# suppressed)

Mark Smith wrote:

Thanks Greg...that works great.

I've been playing around trying to customise the GUI all morning.

	The code for submit-generic is probably not as good a starting point
	for making a new script, since it's highly specialized to embed commands
	into various shells and stuff.

	A better starting point might be a script like submit-shake,
	or submit-nuke.

Managed to get it to look pretty much how I want with all the required
fields. Unfortunately I'm a Python man and I'm finding Perl to hard to
hack at in order to get my customised GUI to feed in the required info.

	I've not written anything in Python.. I'm kind of waiting to see
	if either Ruby or Python jumps out at me.

	The submit scripts don't have any need for OOP really,
	so they can be written in just about any scripting language.

Is there a Python version of the submit-generic script?

	No, and I'm not sure a python version of submit-generic
	would be as helpful as a python version of one of the other
	scripts.

	What I might do is make a simple Python example script
	that shows how to submit a job through Rush.

A GUI for this sort of thing would be really cool. There definitely a
huge need for it where I am (artists just won't use it if it doesn't
have a GUI).

	What's the command line help look like for the 'publisher'?
	(ie. does it show anything when you type 'publisher -help')

	I can write it, but it'll be in Perl in order to be releasable
	with the rest of rush.

We'd love it if you could knock one up for us. It would be
great if it was general enough so that it could be used for similar
tasks like quicktime generation etc.

	Does 'publisher' support making quicktimes?

--
Greg Ercolano, erco@(email surpressed)
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Cel: (Tel# suppressed)
Fax: (Tel# suppressed)

Is there a Python version of the submit-generic script?

    No, and I'm not sure a python version of submit-generic
    would be as helpful as a python version of one of the other scripts.

    What I might do is make a simple Python example script
    that shows how to submit a job through Rush.

A simple Python submit script would be brilliant...I'm afraid Perl sends
shivers up my spine.

    I converted the "rush/examples/input-example.pl" program into python,
    and attached it here.

    There's already a .csh and .pl version of the script, so a .py file
    rounds it out nicely ;)

    I'm glad I did this experiment -- it made me look closely at Python.

    I'm still new to it, so I know my knowledge is limited so far,
    but I'm pretty sure I /won't/ be thinking 'gee, I should port
    all the submit scripts to Python..!" ..it's not a good fit IMHO.

    Though Python's capable, Python seems to be more of a programming language
    than a scripting language.. ie. it's more like C++ than CSH or Perl;
    it sticks to strict programming guidelines (which is good, if writing programs),
    but lacks syntax that makes /scripting/ pleasant.

    For instance, I miss the ability to 'easily' test for the existence of a file,
    something scripts often do:

        PERL                           PYTHON
	if ( -e "/some/file" )         if ( os.path.exists("/some/file") ):
	if ( ! -e "/some/file" )       if ( not os.path.exists("/some/file") ):

    Also, the language is so OOP-y, it's like having to specify absolute paths
    to everything.. makes for redundancy that clutters up code quickly:

	if ( os.path.isdir(foo) ): ..
	if ( os.path.isdir(bar) ): ..
	if ( os.path.isdir(bla) ): ..

    ..all that "os.path" stuff seems redundant readability wise,
    harder to read than the csh/perl equivalent:

	if ( -d $foo ) ..
	if ( -d $bar ) ..
	if ( -d $bla ) ..

    I also miss being able to easily embed variables in strings,
    making commands with lots of arguments harder to read in Python:
	
	$render -f $somefile $sfrm $efrm				# Csh
	system("$render -f $somefile $sfrm $efrm");			# Perl
	os.system(render+" -f "+somefile+" "+str(sfrm)+" "+str(efrm))	# Python

    ..all those (")s and (+)s and str()s make scripting way more difficult
    than it needs to be.

    Also, I'd really miss being able to do assignments within conditionals
    something I do often in Perl and C:

        PERL/C                               PYTHON
	while ( $foo = ReadStuff() ) ..      while (1): foo = ReadStuff(); if ( not foo ): break; ..
	if ( $foo = IsOkay() ) ..            foo = IsOkay(); if ( foo ): ..

    And all those icky ':'s on the end of if/else/while
    seem unnecessary if line breaks are present. In a language where
    white space is significant, I think it would help here.

    I don't mind the whitespace indenting thing as much as I thought I would,
    and not having to specify $ in front of variables is nice, though not nice enough
    to loose the ability to embed variables in strings. Would be neat if Python
    would allow '$'s to be used to expand variables in strings, instead of that
    icky "%(var)s" syntax.

#!/usr/bin/env python
import sys
import os

#
# PYTHON EXAMPLE OF HOW TO USE THE 'input' PROGRAM
#
#     The 'input' program comes with rush. The executable
#     can be found, under UNIX: /usr/local/rush/examples/bin/input
#     and under WINDOWS: c:/rush/examples/bin/input.exe
#
#     Documentation for the input program's definition file
#     can be found at http://seriss.com/rush-current/input/
#     or refer to the submit-*.pl examples for actual working examples.
#

# FIND THE RUSH 'INPUT' PROGRAM
#     Depends on the local OS.
#
input = "/usr/local/rush/examples/bin/input"
if ( not os.path.exists(input) ): input = "c:/rush/examples/bin/input"

# WHERE'S THE TMP DIR
tmp = "/var/tmp"				# modern unix
if ( not os.path.isdir(tmp) ): tmp = "/usr/tmp"	# old unix
if ( not os.path.isdir(tmp) ): tmp = "c:/temp"	# windows

# WHERE'S THE TEXT EDITOR TO SHOW RESULTS (WHEN USER HITS 'VIEW')
#     Depends on the OS.
#
editor = "/usr/bin/gedit"					# linux
if ( not os.path.exists(editor) ): editor = "/usr/sbin/jot"	# sgi
if ( not os.path.exists(editor) ): editor = "notepad"		# windows
if ( os.path.isdir("/Applications") ): editor = input+" -phF "	# darwin

# CREATE DEFINITION FILE NEEDED BY INPUT
#     This file defines all the input fields for the GUI,
#     and is passed to the input program with the -d argument.
#
inputdata = tmp + "/input-example.in"
f=open(inputdata, "w")
f.write("""

# CREATE A WINDOW
window
{
    name      "Test Input"	# The title of the window
    xysize    560 180 		# The x/y size of the window
    resizable 1			# Allow window to be resizable (optional)
    menubar   yes		# Include the menubar at the top (optional)
    scroll    yes		# Make a scroll bar if needed
}

# SET STARTING POSITION FOR FIRST FIELD
#     The first input field will be started at this x/y position
#     on the screen. Each subsequent field will be created below the last,
#     unless you change the xy value.
#
xy  150 50

# CREATE AN INPUT FIELD FOR A PATHNAME
input
{
    name        "Enter Some Pathname:"	# Prompt shown on screen
    dbname      "SomePathname"		# Name of this field for output file
    default     "//tahoe/net/foo"	# Default value (optional)
    filebrowser yes			# Include a 'Browse' button
    xysize      280 24			# x/y size of the input box
    help
    {
        This text shows up when the user hits the "?" button.
	If this 'help' section is not specified, no "?" button will appear.
    }
}

# CREATE ANOTHER INPUT FIELD
#     This will be created below the above field automatically.
#     We *could* specify a new 'xy' or 'xyinc' command to change
#     the position, but in this case we want the fields stacked
#     one above the other.
#
input
{
    name        "Enter Other Stuff:"	# Prompt shown on screen
    dbname      "Stuff"			# Name of this field for output file
    default     "stuff"			# Default value (optional)
    xysize      280 24			# x/y size of the input box
    # No help specified; no "?" will appear
}

# CREATE A CHOOSER
#    Again, this chooser will be placed directly below the last field.
#
choice
{
    name        "Pick Something:"	# Prompt shown on screen
    dbname      "Toppings"		# Name of this field for output file
    option	"pepperoni"		# First option in chooser
    option	"sausage"		# Second option in chooser
    option	"olives"		# Third option in chooser
    default     "sausage"		# Default value (must be one of above)
    xysize      100 24			# x/y size of the chooser
    help
    {
        pick something.
    }
}

# CREATE A SUBMIT AND CANCEL BUTTON
#    The 'Submit' button will save out what the user
#    entered, passing its filename to the submitcmd,
#    which is executed after the file is saved.
#
#    The 'submitcmd' can be any executable, but it
#    must return an exit code of 0 to indicate success,
#    or an exit code of 1 to indicate failure.
#
#    In the following, we execute 'notepad', or whatever
#    text editor we determined (above), which will
#    simply bring up the input program's output file
#    in the text editor, so you can see what the user entered.
#    In real life, you would make the 'submitcmd' your own
#    script, so that it parses the user's entry, and does
#    something with it.
#
#    In this case, we want to offset the X position
#    of these two buttons off to the right, so we
#    use this 'xyinc' command, which adds the x/y values
#    to the "current" x/y position
#
xyinc 200 0

submit
{
    submitname   "View"
    cancelname   "Cancel"
    submitcolor  51		# submit button color (see color table in docs)
    cancelcolor  51		# color of cancel button
    submitcmd    "%(editor)s"	# command to run when the user hits 'Submit'
    showfail     1		# if submitcmd fails, pop dialog to show error
}
""" % locals() )		# finished write() command
f.close()

# Create empty hist file if none exists
histfile = tmp+"/input-example.hist"
if ( not os.path.exists(histfile) ): open(histfile,"w").close()

# Invoke input
os.system(input + " -P -d " + inputdata + " -H " + histfile)