   From: Antoine Durr <antoine@(email surpressed)>
Subject: Re: cpu balancing script
   Date: Tue, 05 Jun 2007 20:50:51 -0400

Msg# 1576
View Complete Thread (16 articles) | All Threads
Last Next

On 2007-06-01 12:17:44 -0700, Greg Ercolano <erco@(email surpressed)> said:

Antoine Durr wrote:

I'm under the impression that to balance cpus per user requires an
external script that dynamically resets priorities based on who should
have more or fewer cpus than they already have.  Does anyone have an
example of such a script?


	I'm happy to see any responses.

	Just please remember not to post proprietary or confidential code.
	(ie. code with restrictive rights banners)

	For more info, please refer to this newsgroup's FAQ:
	http://seriss.com/cgi-bin/rush/newsgroup-threaded.cgi?-view+1

So I whipped up something this afternoon. It's pretty simplistic, andI'm curious as to where it might fall flat on its nose.



#!/usr/bin/perl -w

#
# rush_rebalance.pl
#
# A cpu priority rebalancing script for the Rush render queue
# (c) 2007 Antoine Durr, Floq FX Inc, use at your own risk, YMMV.
#
# For each iteration, get a list of all the running cpus (rush -laj).  Place
# these in a job:cpu_count hash, and sort them descending by how many cpus they
# have running.  Thus, the highest-running job gets priority 1, the next
# highest gets priority 2, and so on.  Now for each job, get a list of the
# jobtaskids (rush -lc), and reset them to the priority.
#
# This is a pretty simplistic setup, and has no provisions for user-desired
# prioritization.  I have no idea how well or poorly this will fare in a large
# environment, or whom it will piss off in the process.
#

while (1)
{
   # generate the list of running and paused jobs

$runlist = `rush -laj | grep -v Done | grep -v RESERVE | tail +3 |awk '{print \$2" " \$7}'`;

   chomp $runlist;

   # set up host=>cpu_count hash, e.g. host.123, 12, host.125, 4
   %runlist = split(/\s+/, $runlist);

   $priority = 1; # reset priorities
   # sort numerically based on hash value
   foreach $job (sort { $runlist{$b} <=> $runlist{$a} } keys %runlist) {
       # print "job: $job, cpus: $runlist{$job} newpriority: $priority\n";

       # grab the list of jobtid's
       $jobtidlist = `rush -lc $job | awk '{print \$5}' | tail +2`;
       chomp $jobtidlist;

       @jobtidlist = split(/\s+/, $jobtidlist);

$jobtidlist = join(" .", @jobtidlist); # prepend period infront of each


       print "rush -fu -cp $job .$jobtidlist \@$priority\n";

       # comment this next line in to have this script actually do something!
       # `rush -fu -cp $job .$jobtidlist \@$priority\n`;

       $priority++;
   }

   sleep 5;
}

--
Floq FX Inc.
10839 Washington Blvd.
Culver City, CA 90232
310/430-2473

Last Next