From: Greg Ercolano <erco@(email surpressed)>
Subject: Re: Staggering render start times.
   Date: Sat, 14 May 2004 01:53:45 -0700

Dylan Penhale wrote:

How can I stagger the start time of each frame of a render job?

Because of the size of some of our renders, setting them all off a once is

putting quite a load on our file server and slowing workstations. Is it
possible to set an advanced submit option to delay each nodes frame start
time?

	Trouble is, even if you stagger, eventually once all the
	machines are rendering, they'll all be hitting your server.

	If you've got a network large enough to where it slows your
	server down, you may want to consider some kind of network
	flow control at the switches, so that the workstations have
	higher priority to the network than the farm.

	To stagger start, you could any one of several things..
	just to give you an idea:

		o Submit with the 'maxcpus' set to '1', then slowly advance
		  the maxcpus value with an external script:

			#!/bin/csh -f
			set jobid = $1
			# Every 10 seconds, add two cpus. Eventually, removes maxcpus limit
			foreach i ( 2 4 8 10 0 )
			    rush -maxcpus $i $jobid -fu
			    sleep 10
			end

		  ..or build this behavior into the submit script.

		o Ramp up sleep times in the render script for first 20 frames:

			if ( $frame <= 20 ) { sleep($frame*5); }

		o Enter a loop that polls the server's uptime before starting
		  a frame to render, eg. in perl:

			while ( 1 )
			{
			    # Parse output of 'uptime' to get load average
			    # If load average is high (>8), sleep until it gets lower
			    my $load=`rsh SERVER uptime | sed 's/.*load averages: //;s/ .*//';
			    if ( $load > 8.0 ) { sleep(10); next; }
			    last;
			}