[Issues] Memory Leak in reduction pipeline ....

[Oliver-Mark Cordes]

Hi Guys!

I found out that the complete reduction process has a big big memory leak, 
which breaks the reduction process ...

I started the following process:

dpu.run( 'Reduce', i='WFI', d='2001-06-21', f='#844', oc=6, o='DEEP1a-0?', 
C=0 )


which means to reduce 20 RawScienceFrames one after another and 
unfortunately during the reduction of the 20th image the pipeline broke up 
with

[node1] 18:52:28 - ERROR - Error loading pixelmap Cal-OCORDES-WFI---------ccd50---Hot-Pix-53929.5226041.fits

which worked fine for all other images (I attached the complete log!)

During a second attempt I followed the memory consumption of the awe 
process which exceeds 20?? MB at the end. I think this is a python limit, 
Each node in the cluster has 2GB of memory which is OK so far.

But the real problem is that some of the recipies do not free their memory 
when one step is done. I think this may be a general problem of eclipse 
and python? I'm not sure but this is a strong suspicion ;-)

If you want to test the command above this should be possible from 
Groningen but it will use some default ever calib. files which shouldn't 
be a problem anyway.

Hope you find the bug, because it is strange to split the reduction in 
small steps ...


Cheers,
 	Oliver
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 20060713_175240971.log.gz
Type: application/octet-stream
Size: 159870 bytes
Desc: 
Url : http://listman.astro-wise.org/pipermail/issues/attachments/20060713/7d329860/20060713_175240971.log-0001.obj