[Comm2011] Fwd: Re: Evalso-light

[E. Helmich]

Yup, that OMEGA directory is new, Stefano said he'd make it.

Great, if we can keep ourselves automatically up-to-date this way.

Ewout


On Fri, 25 Mar 2011 15:05:36 +0100 (CET), "John P. McFarland"
<mcfarland at astro.rug.nl> wrote:
> Hi Ewout,
> 
> I have a directory structure where there is the test data:
> 
> OMEGACAM/2005-09-08/OMEGACAM.<timestamp>.fits
> 
> and a new entry that showed up today with one file so far:
> 
> /OMEGA/1998-12-31/raw/OMEGA.1999-01-01T00:00:00.fits.fz
> 
> I downloaded this one at ~4.5MBps.  I was told (I think by Stefano) that
> they use f(un)pack to work with the compression.  As this software uses
> CFITSIO, I believe our FITSIO module will work properly.  In preliminary
> tests, this appears to be the case.
> 
> I'm going to set up an automated script that uses wget to download the
> data
> in mirror fashion.  I just need to work out how to prevent filling up our
> disks in the long run.
> 
> It seems that EVALSO will work just fine in this mode for us.
> 
> Keep me up to date on your activities.  I will be available pretty much
> anytime over the next two weeks via e-mail and via Skype a bit less
often.
> Given your connection for Skype will likely be limited, text-based should
> work just as well as audio (even better for some things) whenever we need
> that real-time back and forth.  Good luck with everything!
> 
> Cheers,
> 
> 
> -=John
> 
> 
> On Fri, 25 Mar 2011, helmich wrote:
> 
>> Hi,
>>
>> We did a test transfer via Evalso(-light) which according to the log
> went
>> at 7.3MByte/s (!). The transfer scripts only searches for FITS files (we
>> tried a dummy file with .fits extension, idea was to put md5sums in
> there,
>> - no luck, my guess is the fpack compression fails badly on fake .fits
>> files).
>>
>> So, I will probably have to send md5sums separately by email. Further
> issue
>> is that I couldn't see the particular file on the FTP server in
> Garching.
>> This is because the transfer scripts assume the file is in format
>> OMEGA.<date-obs>.fits, which apparently the data handling software
> should
>> create (see Stefano's mail below).
>>
>> One part (the first) in the chain is still missing: the services that
>> upload data from the IWS to the DHS are not running. Andrea is looking
> into
>> this. Problem is that here too, the knowledge and responsibility appears
> to
>> be fragmented. From the DHS the scripts are now set to transfer every
>> single file to Garching, so once we start we'll see soon enough how the
>> whole thing manages.
>>
>> Ewout
>>
>>
>>
>> -------- Original Message --------
>> Subject: Re: Evalso-light
>> Date: Fri, 25 Mar 2011 13:54:52 +0100
>> From: Stefano Zampieri <szampier at eso.org>
>> To: helmich <helmich at astro.rug.nl>
>>
>> Hi Ewout,
>>  the file ended up in the wrong directory because of its
>>  name. The expected name is <OLAS_ID>.<timestamp>.fits
>>  which is the ESO standard archive name generated by OLAS.
>>  As far as I know the OLAS_ID for OMEGACAM is OMEGA therefore
>>  the filename should be OMEGA.YYYY-MM-DDThh:mm:ss.fits
>>  and it will be saved in directory /OMEGA
>>  (I will create a link on the FTP server for OMEGA)
>>
>> Stefano
>>
>> On 25/03/2011 13:41, helmich wrote:
>>> Hi Stefano,
>>>
>>> We've done a test transfer of a file ( OCAM_IMG_FLAT279_0001.fits.fz ).
>>> It seems to have gone nicely, at 7MB/s, but I cannot find it in the
>>> /OMEGACAM directory on the FTP server. The log file of the transfer is
>>> attached. Any ideas?
>>>
>>> Regards,
>>>
>>> Ewout
>>>
>>
>>