[KIDS] INTDR2 catalog ingestion

[Hugo Buddelmeijer]

Hi Nicola et al.,

This night I've done a first ingestion of all the 52 catalogs made with 
KIDSCAT beta v1.0. They can be found by querying for SourceLists with 
the name "INTDR2test_*", like:
   awe> slsintdr2 = SourceList.name.like("INTDR2test_*")
   awe> len(slsintdr2)
   52
Or through the dbviewer:
   http://socuteurl.com/funnytumble

They have 'test' in the name because they are simply a conversion from 
the .cat files to AW SourceLists. No thorough assessment of the catalogs 
are done yet. However, they seem okay, and we will start using them here.

What has not yet been done:
- Ingesting the sexconf/sexparams (first thing to do)
- Ingestion of the masks (the weights used are from AW right?)
- Ingestion of the intermediate catalogs
- Ingestion(/recreation) of QC plots

What happened to the columns:
- FLUX_APER and FLUXERR_APER are renamed to FLUX_APER_<aperturesize>
   and FLUXERR_APER_<aperturesize>. So FLUX_APER_10 is the flux in a
   10 arcsecond aperture.
- The VIGNET columns have been removed.
- Some columns are renamed according to the FITSCOLUMN_NAMES dictionary
   in SourceList.py:
 
http://www.astro-wise.org/viewvc/awe/astro/main/SourceList.py?annotate=1.228#l2788
- The HTM column is added.

I will improve the catalog ingestion, starting with the sexconf, since 
this will improve the data lineage and reproducebility. If there is 
anything wrong, unclear or missing, please notify me!

Greetings,
Hugo

P.S. I moved this discussion to kids at astro-wise.org, that way everyone 
gets them. Contact Gijs if you're not on this list yet.



On 21/11/12 10:48, Nicola R. Napolitano wrote:
>
> I forgot to include Francesco in the mailing list....reply to this
> message if needed.
>
> Cheers,
>
> Nicola
>> Hi everyone,
>>
>> the catalogs for the 52 coadds selected for the INT-DR2 have been
>> produced using KIDSCAT beta v1.0 and current products can be
>> downloaded at the following link
>>
>> http://vstportal.oacn.inaf.it/KIDSCAT/KIDSCAT_INTDR2_52FIELDS/
>>
>> We have started to have a visual check to the catalogs in order to
>> have a firs catalog QC before these are passed/ingested to AW for the
>> QC of the coadds. In particular we are concentrating to the following
>> checks:
>>
>> 1) look at the plot FWHM vs SNR in the PLOTS directory (Check that
>> there is only one stellar sequence and that the spread is reasonably
>> within a couple of pixels. Check that there are no weird features like
>> clouds of points at low FWHM - e.g. satellite tracks?)
>> 2) in the same directory there are now also the mag limit plots right
>> implemented from Francesco (mag lim estimates are in
>> OUTPUT_RES/MLIM.dat, see also Francesco's message);
>> 3) look at a 2D X-Y plot of all sources versus the ones with col7
>> (IMAFLAGS_ISO) <1 "bona fide" sources to see whether the distribution
>> is uniform and cross-check with the fits image to see whether the
>> major stars are correctly masked;
>> 4) plot flux_radius (col9) vs mag_auto (col15) for an independent
>> check of the stellar sequences;
>> 5) report comments on the test above in a table with the following
>> columns:
>> #FIELD; BAND; FWHM vSNR; 2Dplot; 2Dflag0 plot; visual vs ds9;
>> fluxRvsmag_A; extraComm
>>
>> I'll gather all you comments and I'll put in a table which will posted
>> on the wiki
>>
>> http://wiki.astro-wise.org/projects:kids:data_deliveries:internaldelivery2.0_scratch#quality_check_of_the_52_catalog_for_int-dr2
>>
>> Cheers,
>>
>> Nicola
>>
>> --
>> Nicola R. Napolitano
>> Research Astronomer
>> INAF - Observatory of Capodimonte
>> Salita Moiariello, 16, 80131, Napoli (Italy)
>> phone: +39 081 5575509; fax: +39 081 456710
>> email:napolita at na.astro.it  <mailto:napolita at na.astro.it>
>> web page:http://www.na.astro.it/~napolita  <http://www.na.astro.it/%7Enapolita>
>
>
> --
> Nicola R. Napolitano
> Research Astronomer
> INAF - Observatory of Capodimonte
> Salita Moiariello, 16, 80131, Napoli (Italy)
> phone: +39 081 5575509; fax: +39 081 456710
> email:napolita at na.astro.it  <mailto:napolita at na.astro.it>
> web page:http://www.na.astro.it/~napolita
>