From: "Abraham Schneider" <aschneider@(email surpressed)>
Subject: dynamically balance renderfarm between 3D and comp renderings
   Date: Fri, 15 Jun 2012 12:37:36 -0400

Hi!

At the moment, we use different render machines for 3D and comp renderings (3DS Max/Vray and Nuke), which is not a perfect solution for the size of our company. So I'm thinking about using one renderfarm for all, managed by Rush. Now there is one question:

How would it be possible with Rush to dynamically balance the amount of machines that are rendering 3D and comp setups based on current needs and/or time of day? During normal working hours, it would be helpful to designate more machines to Comp renderings, at night all the machines should render 3D. This could be done with normal priority and grouping functions of Rush. But sometimes it would be helpful to temporarily use more machines for 3D during normal working hours.

Are there any concepts for re-prioritize already running jobs in a global way? Something like a big percentage slider where I can manually change the proportion and all the running and queued jobs will use this new setting?

Hope this makes any sense. But I'm absolutely open for any better solution/way to deal with the limited amount of renderers we have, using them effectively for 3D AND comp.


Thanks, Abraham


Abraham Schneider
Senior VFX Compositor
 

ARRI Film & TV Services GmbH
Tuerkenstr. 89
D-80799 Muenchen / Germany

Phone (Tel# suppressed) 

EMail aschneider@(email surpressed)
www.arri.de/filmtv
________________________________


ARRI Film & TV Services GmbH
Sitz: München Registergericht: Amtsgericht München
Handelsregisternummer: HRB 69396
Geschäftsführer: Franz Kraus, Dr. Martin Prillmann, Josef Reidinger

On 6/15/12 12:37 PM, Abraham Schneider wrote:
> Hi!
> 
> At the moment, we use different render machines for 3D and comp =
> renderings (3DS Max/Vray and Nuke), which is not a perfect solution for =
> the size of our company. So I'm thinking about using one renderfarm for =
> all, managed by Rush.

	Right.

	To do this, you can make separate +3d and +comp hostgroups, so that
	3dsmax/vray renders have Cpus: requesting the +3d hostgroup, and
	Nuke renders have Cpus: requesting the +comp hostgroup.

	Both hostgroups could have overlapping host memberships,
	or completely separate host memberships.

> Now there is one question:
> 
> How would it be possible with Rush to dynamically balance the amount of =
> machines that are rendering 3D and comp setups based on current needs =
> and/or time of day? During normal working hours, it would be helpful to =
> designate more machines to Comp renderings, at night all the machines =
> should render 3D. This could be done with normal priority and grouping =
> functions of Rush. But sometimes it would be helpful to temporarily use =
> more machines for 3D during normal working hours.

	Yes, a small script could handle this I think.

	The script would be run twice a day by crontab:
	once at night, and once during the day to enforce the dynamic changes.

	The script would also have a user interface to let you
	control the sliders for the behavior changes.

	I'd be happy to help write such a script in the language
	of your choice (python/perl).

> Are there any concepts for re-prioritize already running jobs in a =
> global way? Something like a big percentage slider where I can manually =
> change the proportion and all the running and queued jobs will use this =
> new setting?

	Post what specifics you'd like to see.

	I can envision a few ways to do this, all would involve
	a crontab configured to run twice a day to enforce the changes;
	once in the morning, once at night.

	One example:

		During the day, comp jobs request the +comp hostgroup,
		and 3d jobs request the +3d group.

		At night, the dynamic scheduler script would run,
		adding special +3d_night and +comp_night hostgroups
		to the jobs, enhancing their rendering abilities.

		Sliders could set either the priority or the cpus.

		Perhaps too, this could include workstation rendering
		where a hostgroup +work_night is added to the jobs.

		In the morning, these +xx_night groups would be removed
		from the jobs, either by setting their cpus to =0 to gradually
		bring the renders to a stop, or by actually removing them
		to immediately kill/requeue the renders.

	Another way would be to play with the priorities; you could
	for instance use priorities to control things so that both kinds
	of jobs request +comp and +3d cpus, but the priorities are different.

	During the day, COMP jobs would request cpus:

		+comp=30@100		-- normal priority on comp machines
		+3d=5@1			-- low priority on 3d machines

	..so at night:

		+3d=5@1 --> +3d=10@100	-- raised priority

	..and during the day, these would be changed back.

	I can see where a slider would control how much to 'add'
	to the priority, so that if jobs were submitted at different
	priorities during the day, their relationship to each other
	would be maintained at night.

	Similarly for 3d jobs..



-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)