[Comm2011] Fwd: Re: Evalso-light

[John P. McFarland]

Hi Ewout,

I have a directory structure where there is the test data:

OMEGACAM/2005-09-08/OMEGACAM.<timestamp>.fits

and a new entry that showed up today with one file so far:

/OMEGA/1998-12-31/raw/OMEGA.1999-01-01T00:00:00.fits.fz

I downloaded this one at ~4.5MBps.  I was told (I think by Stefano) that
they use f(un)pack to work with the compression.  As this software uses
CFITSIO, I believe our FITSIO module will work properly.  In preliminary
tests, this appears to be the case.

I'm going to set up an automated script that uses wget to download the data
in mirror fashion.  I just need to work out how to prevent filling up our
disks in the long run.

It seems that EVALSO will work just fine in this mode for us.

Keep me up to date on your activities.  I will be available pretty much
anytime over the next two weeks via e-mail and via Skype a bit less often.
Given your connection for Skype will likely be limited, text-based should
work just as well as audio (even better for some things) whenever we need
that real-time back and forth.  Good luck with everything!

Cheers,


-=John


On Fri, 25 Mar 2011, helmich wrote:

> Hi,
>
> We did a test transfer via Evalso(-light) which according to the log went
> at 7.3MByte/s (!). The transfer scripts only searches for FITS files (we
> tried a dummy file with .fits extension, idea was to put md5sums in there,
> - no luck, my guess is the fpack compression fails badly on fake .fits
> files).
>
> So, I will probably have to send md5sums separately by email. Further issue
> is that I couldn't see the particular file on the FTP server in Garching.
> This is because the transfer scripts assume the file is in format
> OMEGA.<date-obs>.fits, which apparently the data handling software should
> create (see Stefano's mail below).
>
> One part (the first) in the chain is still missing: the services that
> upload data from the IWS to the DHS are not running. Andrea is looking into
> this. Problem is that here too, the knowledge and responsibility appears to
> be fragmented. From the DHS the scripts are now set to transfer every
> single file to Garching, so once we start we'll see soon enough how the
> whole thing manages.
>
> Ewout
>
>
>
> -------- Original Message --------
> Subject: Re: Evalso-light
> Date: Fri, 25 Mar 2011 13:54:52 +0100
> From: Stefano Zampieri <szampier at eso.org>
> To: helmich <helmich at astro.rug.nl>
>
> Hi Ewout,
>  the file ended up in the wrong directory because of its
>  name. The expected name is <OLAS_ID>.<timestamp>.fits
>  which is the ESO standard archive name generated by OLAS.
>  As far as I know the OLAS_ID for OMEGACAM is OMEGA therefore
>  the filename should be OMEGA.YYYY-MM-DDThh:mm:ss.fits
>  and it will be saved in directory /OMEGA
>  (I will create a link on the FTP server for OMEGA)
>
> Stefano
>
> On 25/03/2011 13:41, helmich wrote:
>> Hi Stefano,
>>
>> We've done a test transfer of a file ( OCAM_IMG_FLAT279_0001.fits.fz ).
>> It seems to have gone nicely, at 7MB/s, but I cannot find it in the
>> /OMEGACAM directory on the FTP server. The log file of the transfer is
>> attached. Any ideas?
>>
>> Regards,
>>
>> Ewout
>>
>
>