From: Gary Jaeger <gary@(email surpressed)>
Subject: logs
   Date: Mon, 14 Mar 2011 12:47:25 -0400

I'm curious how people manage the log files. For instance say we have a maya scene with lots of render layers. And if we want to submit different jobs for each render layer, then specifying different directories for logs gets to be a pain. I wish we could specify some sort of variable like <JobID> so that rush made a new directory every time we did a submit with a unique ID. Then we could have a /logs/<JobID> for each job. Would that make sense?

. . . . . . . . . . . .
Gary Jaeger // Core Studio
86 Graham Street, Suite 120
San Francisco, CA 94129
(Tel# suppressed)
http://corestudio.com	

Gary Jaeger wrote:
> [posted to rush.general]
> 
> I'm curious how people manage the log files. For instance say we have a =
> maya scene with lots of render layers. And if we want to submit =
> different jobs for each render layer, then specifying different =
> directories for logs gets to be a pain. 

	BTW, I have an offline version of submit-maya that lets you
	specify multiple layers and frame ranges for each, so that
	you can submit a multi-layer, multi-frame-range job in a
	single submission/single job.

	It will be part of the new submit-maya script in the next release.
	
	I'll email it to you offline.
	Anyone else interested in testing that, feel free to contact me
	directly, and I'll provide it to you.

> I wish we could specify some =
> sort of variable like <JobID> so that rush made a new directory every =
> time we did a submit with a unique ID. Then we could have a =
> /logs/<JobID> for each job. Would that make sense?

	There is such a thing; if you specify a '%s' on the end
	of the log directory pathname, rush will automatically stick
	the jobid on the end, and will automatically make the directory. eg:
	http://www.seriss.com/rush-current/rush/rush-submit-cmds.html#LogDir

	So for instance, in the submit-maya form, under the Rush tab
	if you specify:

		Log Directory: /some/path/myscene/logs/%s

	..that will create a subdir in the 'logs' directory named after the
	job's jobid, and will use that as the log directory.

	You might want to have a cron job clean that dir out of dirs older
	than a week or so. Or you can configure the job's 'jobdumpcommand'
	to remove it for you; I leave it to my customers to construct
	the correct 'rm -rf' command to clean out the production dirs..
	too scary for me ;)

-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)

Hi Gary

We do the scary rm -rf on all *.log files and folders within the log
folders that are older than a set number of days.

#Rush - log files and folders
"find /server/resources/logs/rush -name "*.log" -ctime +14 -ls -exec
rm -rf {} \; | awk '{print "Size: "$1, "\tOwner: "$5,"\tDate: "$8,$9,
"\tDeleting: "$11}' > /var/log/delrushRushlogs.log
# Send confirmation email
cat /var/log/delrushRushlogs.log | mail -s "Old rush logs Flushed from
Server" admin@(email surpressed)"

We do something similar to remove old done jobs as well.



On 15 March 2011 04:46, Greg Ercolano <erco@(email surpressed)> wrote:
> [posted to rush.general]
>
> Gary Jaeger wrote:
>> [posted to rush.general]
>>
>> I'm curious how people manage the log files. For instance say we have a =
>> maya scene with lots of render layers. And if we want to submit =
>> different jobs for each render layer, then specifying different =
>> directories for logs gets to be a pain.
>
>        BTW, I have an offline version of submit-maya that lets you
>        specify multiple layers and frame ranges for each, so that
>        you can submit a multi-layer, multi-frame-range job in a
>        single submission/single job.
>
>        It will be part of the new submit-maya script in the next release.
>
>        I'll email it to you offline.
>        Anyone else interested in testing that, feel free to contact me
>        directly, and I'll provide it to you.
>
>> I wish we could specify some =
>> sort of variable like <JobID> so that rush made a new directory every =
>> time we did a submit with a unique ID. Then we could have a =
>> /logs/<JobID> for each job. Would that make sense?
>
>        There is such a thing; if you specify a '%s' on the end
>        of the log directory pathname, rush will automatically stick
>        the jobid on the end, and will automatically make the directory. eg:
>        http://www.seriss.com/rush-current/rush/rush-submit-cmds.html#LogDir
>
>        So for instance, in the submit-maya form, under the Rush tab
>        if you specify:
>
>                Log Directory: /some/path/myscene/logs/%s
>
>        ..that will create a subdir in the 'logs' directory named after the
>        job's jobid, and will use that as the log directory.
>
>        You might want to have a cron job clean that dir out of dirs older
>        than a week or so. Or you can configure the job's 'jobdumpcommand'
>        to remove it for you; I leave it to my customers to construct
>        the correct 'rm -rf' command to clean out the production dirs..
>        too scary for me ;)
>
> --
> Greg Ercolano, erco@(email surpressed)
> Seriss Corporation
> Rush Render Queue, http://seriss.com/rush/
> Tel: +1 626-576-0010 ext.23
> Fax: +1 626-576-0020
> Cel: +1 310-266-8906
>



-- 
Dylan Penhale

Dylan Penhale wrote:
> #Rush - log files and folders
> "find /server/resources/logs/rush -name "*.log" -ctime +14 -ls -exec rm -rf {} \; 

	The thing that scares me about commands like this is
	what if someone happens to create a filename with spaces, eg:

mkdir "/server/resources/logs/rush/somedir "; touch "/server/resources/logs/rush/somedir /root"

	..with a command like that, in 14 days will it construct
	an 'rm' command that looks like:

		rm -rf /server/resources/logs/rush/somedir /root

	..and blow away the /root directory? ;D

	It shouldn't; find(1) is /supposed/ to guarantee the command
	will be executed such that {} will be passed to rm as a single argument,
	but it makes you think twice.. especially across platforms and OS updates.

	Certainly the OS does global core removals that same way. *Still* it makes
	it hard to sleep at night ;) I've seen bugs creep into vendor specific
	rm(1) and find(1) commands that are.. scary.

	It's that kind of thing that worries me, someone coming up with some
	nutty platform or situation where the rm -rf somehow sneaks out into
	unexpected parts of the file system, and a whole prod dir disappears.

-- 
Greg Ercolano, erco@(email surpressed)
Seriss Corporation
Rush Render Queue, http://seriss.com/rush/
Tel: (Tel# suppressed)ext.23
Fax: (Tel# suppressed)
Cel: (Tel# suppressed)