From: Dado Feigenblatt <dado@(email surpressed)>
Subject: user environment
   Date: Fri, 09 Mar 2007 15:50:15 -0500

Hi, I've just started on rush

I couldn't find anything related to that in the docs.
We use tcsh on Fedora Core 4.

Could someone please point me in the right direction?

Thanks
Dado Feigenblatt

Dado Feigenblatt wrote:

Hi, I've just started on rush

I'm trying to send my first simple sphere Maya render to rush but the user environment the job runs in is crippled.

I couldn't find anything related to that in the docs.

	There's a few newsgroup articles that cover this, and the reasons
	snapshots of user environments are not passed through Rush by default:
	http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+761+761

	..mainly it's an issue of cross platform and cross-processor
	(eg. 32 bit library and PATH variables vs 64 bit) compatibility.

	Also, a rather long history in production administration has taught me
	that unless there's VERY careful manipulation of user environments
	(small percentage of the time) this won't work. And even when there
	*is* a tight reign, if users can change their ~/.cshrc files, they will,
	and ultimately break them, affecting rendering if these files are used.

       [EDIT: Also, rush does not tie itself to any shells; Rush doesn't want to
       know about shells or login specifics; it just wants to run commands,
       and assumes those commands handle their own environment settings,
       to ensure the command will run the same no matter which user runs it.]

	This is why rush does not source or take into account user environments.
	It's almost always not a good idea, and is a source of random render
       behavior that's hard to debug.

	Usually what is needed are variables associated with a particular
	production, which is often based on the show/shot/element, which one
	can add logic to the scripts to parse from the scene filenames.
	See: http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1459+1460+1461+1462

	If you're in that small percentage where you have tight reign
	on user environments and you've somehow got everything working consistently,
	then you can use the LoadEnvFromCsh() perl function below to source
	.cshrc files into the perl environment at render time.

We use tcsh on Fedora Core 4.

	Right, and because perl is not a csh based language, it doesn't
	source csh oriented rc files. To do this, you can use this technique:

# LOAD A CSH SCRIPT'S ENVIRONMENT SETTINGS INTO PERL ENVIRONMENT
#    $1 = csh script to be 'sourced'
#    Returns: $ENV{} modified as per csh script's settings.
#
sub LoadEnvFromCsh($)
{
  my ($rcfile) = $_[0];
  my $vars = `csh -fc 'source $rcfile; printenv'`;
  foreach ( split(/\n/, $vars) )
      { if ( /(^[^=]*)=(.*)/ ) { $ENV{$1} = $2; } }
}

	..and just call that in the submit script just before the renderer
	is invoked (typically in the MAIN_Render() function)

	Here's a similar function for Bourne Shell (sh, bash) settings
	e.g. from /etc/profile or /etc/bashrc files:

# LOAD A BASH SCRIPT'S ENVIRONMENT SETTINGS INTO PERL ENVIRONMENT
#    $1 = bash profile to be 'sourced'
#    Returns: $ENV{} modified as per bash profile's settings.
#
sub LoadEnvFromBash($)
{
  my ($rcfile) = $_[0];
  my $vars = `bash -c '. $rcfile; printenv'`;
  foreach ( split(/\n/, $vars) )
      { if ( /(^[^=]*)=(.*)/ ) { $ENV{$1} = $2; } }
}

	More info on the above here:
	http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-viewthread+1459+1460+1461+1462
	http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+811+811

	But beware; sourcing /user/ environments (~/.cshrc) tends to vary
	widely.. users often have typos in aliases, or invoke programs that
	makes assumptions about stdout/err being a tty, or play around with
	the window manager, and that can cause havoc with rendering leading
	to hard-to-trace problems.

	If you're sourcing centrally managed files, that's fine
	(eg. 'source /usr/YOURCOMPANY/adm/.everyone_rc; source /usr/YOURCOMPANY/adm/.maya_8.5')

We have a series of config scripts (some in python) that should be sourced for proper user environment setup.

	Beware of common pitfalls when eg. user handoffs are done.

	For instance, one user might have maya 8.5 environment variables
	set up in their ~/.cshrc, and their render works fine when submitted
	as themselves, but then handing off to a render watcher who resubmits
	their job doesn't have those settings, the render might fail for them.

	This kind of problem is avoided if user environments are not allowed
	into the mix.

	See the first link above for software specific environment settings,
	as opposed to sourcing 'user' environments.

	If you have project-specific settings you're concerned about,
	often the best way is to add some logic to the submit script
	that determines the show/shot from the scene file's pathname, eg:
	http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+761+761

--
Greg Ercolano, erco@(email surpressed)
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

On Friday 09 March 2007 13:37, Greg Ercolano wrote:

>
> # LOAD A CSH SCRIPT'S ENVIRONMENT SETTINGS INTO PERL ENVIRONMENT
> #    $1 = csh script to be 'sourced'
> #    Returns: $ENV{} modified as per csh script's settings.
> #
> sub LoadEnvFromCsh($)
> {
>        my ($rcfile) = $_[0];
>        my $vars = `csh -fc 'source $rcfile; printenv'`;
>        foreach ( split(/\n/, $vars) )
>            { if ( /(^[^=]*)=(.*)/ ) { $ENV{$1} = $2; } }
> }
>

There are certain environment variables you may want to avoid setting.  These 
are the ones I have gleamed over the years.
TERM, DISPLAY, _, SHELLOPTS, BASH_VERSINFO, EUID, GROUPS, PPID, UID

There are several ways you could modify Gregs routine above to handle this.  
I'll leave that as an exercise to the reader.  As Greg said, you really must 
know what you are doing and you must have a really tight reign on the users 
environment.  If you do this I would recommend printing the users environment 
as the last step before running your render command so you will have a copy 
in the rush log file.  This is very useful for looking at what could be 
causing a failure for a user.

Robert Minsk wrote:
> [..]  If you do this I would recommend printing the users environment
> as the last step before running your render command so you will have a copy
> in the rush log file.  This is very useful for looking at what could be
> causing a failure for a user.

	Right; I believe most of the submit scripts have an option for this:

		Print Environment? on/off

	..which is by default 'off'. You'll find this option if you
	scroll down to the very bottom of the 'Rush' tab in the submit
	script's input form.

	Turning this 'on' executes a 'printenv' (unix) or 'set' (windows)
	just before the render runs, so you can see what the environment
	settings are within the frame logs for debugging.

	Or, you can jam a system("printenv") command into the perl script
	where ever you want to print the environment, eg:

		# SHOW ENVIRONMENT, WHETHER WINDOWS OR UNIX
		system($G::iswindows ? "set" : "printenv|sort");

	..or even:

                # SHOW ENVIRONMENT
                foreach (sort(keys(%ENV))) { print "$_=$ENV{$_}\n"; }

	..which is pretty much the equivalent 'pure perl' way to print
	the environment.

Thanks for the replies. Interesting reading.
I'm still trying to wrap my mind around rush's architecture.

I'll be using LoadEnvFromCsh() for the time being.

Thanks.
Dado Feigenblatt

Thanks for the replies. Interesting reading.

I'll be using LoadEnvFromCsh() for the time being.

Thanks.
Dado Feigenblatt