From: Greg Ercolano <erco@(email surpressed)>
Subject: [Q+A] How do I make the submit scripts load show-specific environment
   Date: Mon, 29 Jan 2007 16:49:02 -0800

> I am wondering if you have a preferred method for reading in a text file
> or .cshrc rather than putting the env variables in submit scripts by
> hand. We are starting to get different environments for different artists
> based on what show or shot that they are working on. There are a number
> of ways that I could think of to do this, but thought that I would ask.

    Here's what I'd recommend to automate it.

    If your pathnames all follow a naming convention where the SHOW name
    always appears in the same part of all scene pathnames, then the
    easiest thing to do would be to add your own logic to the submit
    scripts that parses the scene file pathname for the showname, and
    loads the appropriate environment settings for that show. eg:


# RENDER THE FRAME -- HANDLES "-render"
#   Invoked on each machine to render the frame.
#   Arguments passed via rush from the MAIN_Submit() section.
#
sub MAIN_Render()
{
    [..]

    # LOAD SPECIAL ENVIRONMENT FOR THIS SHOW/SHOT, IF ANY
    LoadShowEnvironment($ENV{ScenePath});

    # DO RENDER HERE
    system("Render -r sw $ENV{ScenePath} ...");

    [..]
}


    LoadShowEnvironment() could either just set environment variables
    based on the show name, or better yet, have it try to load a ".env"
    file from the show's top level directory, and load that. This way
    the environment settings get backed up to tape, and can be sourced
    by users as well as the submit scripts.

    LoadShowEnvironment() might be implemented as these three functions to
    make it easy to automate determining the show name from a scene file,
    and if a .env file exists in that show's top level directory, load it:


# LOAD A CSH SCRIPT'S ENVIRONMENT SETTINGS INTO PERL ENVIRONMENT
#    $1 = csh script to be 'sourced'
#    Returns: $ENV{} modified as per csh script's settings.
#
sub LoadEnvFromCsh($)
{
     my ($rcfile) = $_[0];
     my $vars = `csh -fc 'source $rcfile; printenv'`;
     foreach ( split(/\n/, $vars) )
         { if ( /(^[^=]*)=(.*)/ ) { $ENV{$1} = $2; } }
}

# PARSE THE SHOWNAME FROM A SCENE FILE PATHNAME
#     Assumes all pathnames follow a local naming convention of some kind
#     where the show name is (in this case) the 4th directory name in the path:
#
#     //ourserver/share/jobs/SHOWNAME/....
#       |_______| |___| |__| |______|
#           1       2    3      4
#
sub ParseShowName($)
{
    my $pathname = $_[0];
    $pathname =~ s%\\%/%g; $pathname =~ s%//*%/%g;      # //foo/bar\bla/SHOW/SHOT -> /foo/bar/bla/SHOW/SHOT
    print "PATH=$pathname\n";
    my @dirs = split(/\//, $pathname);                  # $1=(foo) $2=(bar) $3=(bla) ..
    return ( defined($dirs[4]) ? $dirs[4] : "");        # return 4th pathname element as show name
}

# PARSE SHOW ENVIRONMENT FOR GIVEN PATH
#    It is assumed pathname follows a file system standard
#    where the show name always appears in the same part of the filename.
#
sub LoadShowEnv($)
{
    my $pathname = $_[0];
    my $show = ParseShowName($pathname);

    # CHECK IF SHOW HAS A .env FILE -- IF SO, LOAD IT
    my $envfile = "//server/share/jobs/$show/.env";
    if ( $show ne "" && -r $envfile )
        { LoadEnvFromCsh($envfile); }
}


    The ParseShowName() assumes the 4th dir in the pathname is the
    showname, ie:

        //server/share/jobs/SHOWNAME/
          ------ ----- ---- --------
            1      2    3     4

    ..so modify that as needed if your setup is different.

    In LoadShowEnv(), modify the pathname where it searches for your
    show's environment file.

    The trick to 'source' CSH style environment variable settings into
    perl in LoadEnvFromCsh() is a good one; it spawns a csh to a) source
    the settings, then b) prints out the environment with printenv(1),
    then the perl parses that output to load the variable settings back
    into perl so it can pass them on to the renderer.

    Make sure the csh commands in the .env file do NOT assume there's
    a tty..  ie. no stty(1) or tputs(1) commands. If there are any,
    they should be if'ed out with eg. if ( `tty` == 'not a tty' ) ..

    Since your company name is 'evileye', you should probably add these
    functions to your own global 'evileye.pl' script that you can load
    at the top of all your submit scripts with e.g.:

        require "//server/share/jobs/rushscripts/evileye.pl"

    ..then just glue in the appropriate call to LoadShowEnv() in the
    render section of the scripts, just before the renderer is called.

Here is what I do to capture the users environment at submission time.  The will only work on *nix type machines.

/bin/env | /bin/egrep -v '^(TERM|DISPLAY|_|SHELLOPTS|BASH_VERSINFO|EUID|GROUPS|PPID|UID)=' | /bin/sed -e "s/^\([^=]*\)=\(.*\)/\1='\2'; export \1/"

Robert Minsk wrote:

Here is what I do to capture the users environment at submission time.
The will only work on *nix type machines.

/bin/env | /bin/egrep -v '^(TERM|DISPLAY|_|SHELLOPTS|BASH_VERSINFO|EUID|GROUPS|PPID|UID)=' | /bin/sed -e "s/^\([^=]*\)=\(.*\)/\1='\2'; export \1/"

    Under Windows you can do similar things with e.g. the 'set' command from DOS,
    and using perl to parse out the variables.

    But in general I advise against 'snapshotting' the user's workstation environment
    in favor of having a 'clean' environment be derived at render time, so weird
    platform specific variables (like PATH and LD_LIBRARY_PATH) aren't propagated
    to other machines/platforms during rendering.

    Also, snapshotting environments can cause trouble when doing handoffs between
    TDs or when render wranglers have to resubmit jobs.

    'Snapshotting' only works in very special circumstances where either there's
    a tight reign on the user's environments, or a very homogenous rendering
    environment where workstations and render farm machines are all precisely
    configured the same way, or when the person writing the scripts and doing admin
    really knows what they're doing. (Robert qualifies for the latter, so I won't
    question his reasons for doing this.. ;)

    Usually the best situation is to have the submit scripts establish a 'pristine'
    environment related to the show/shot. Sometimes unix-y companies have shell
    level aliases that set eg. SHOW and SHOT environment variables to keep track of
    what show you're on, and it's desirable to pass such variables on to the
    submit scripts so they can determine the show/shot that way for environment
    settings.

--
Greg Ercolano, erco@(email surpressed)
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

On Monday 29 January 2007 17:27, Greg Ercolano wrote:
> [posted to rush.general]
>
> Robert Minsk wrote:
> > Here is what I do to capture the users environment at submission time.
> > The will only work on *nix type machines.
> >
> > /bin/env | /bin/egrep -v
> > '^(TERM|DISPLAY|_|SHELLOPTS|BASH_VERSINFO|EUID|GROUPS|PPID|UID)=' |
> > /bin/sed -e "s/^\([^=]*\)=\(.*\)/\1='\2'; export \1/"
>
>      Under Windows you can do similar things with e.g. the 'set' command
> from DOS, and using perl to parse out the variables.
>
>      But in general I advise against 'snapshotting' the user's workstation
> environment in favor of having a 'clean' environment be derived at render
> time, so weird platform specific variables (like PATH and LD_LIBRARY_PATH)
> aren't propagated to other machines/platforms during rendering.
>

I am of the opposite camp in that very important information is in the users 
environment and that the environment should be logged with the render job.  
But we will not argue the point on here.