[KIDS] INTDR2 catalog ingestion

Fri Nov 23 12:33:40 CET 2012

On 11/23/12 12:20 PM, Hugo Buddelmeijer wrote:
> Hi Nicola,
>
> As I understood it, there is a first Sextractor run to measure the 
> SEEING_FWHM of the final Sextractor run. It would be useful to ingest 
> these as well. However this does not have priority.
>

these are .cat.1. These are the ones obtained with the default .sex 
configuration. They might be useful only if one wants to reproduce the 
catalog with the assumed first guess FWHM, I don't think this might be 
any particular useful.

> Furthermore, I'm not sure which catalog this would be. For every final 
> catalog there are two earlier runs, with the extensions ".cat" and 
> ".cat.1" in the KIDSCAT_INTDR2_52FIELDS directory. From the KIDSCAT 
> readme I figured that the ".cat.1" catalog is the FWHM catalog, correct? 

it is the first sex run catalog to be used for the FWHM optimization.

> What is the difference with the "<ObservingBlockName>.cat" and 
> "<CoaddFilename>.cat"?

in the second the first column (2DPHOT) contains a flag for the "sure 
star" (=1) and the name has been changed back to the coadd file name to 
make the association to the coadd easier.

Hope this clarifies. Of course Francesco can add details if needed.

Cheers,

  Nicola

>
> Greetings,
> Hugo
>
>
> On 23/11/12 12:10, Nicola R. Napolitano wrote:
>>
>> Hi Hugo,
>>
>> I'm navigating within the catalogs. It works nice! Well done.
>>
>> I'm looking at the todo list: what do you mean by intermediate catalogs?
>>
>> Cheers,
>>
>> Nicola
>>
>>
>> On 11/23/12 11:58 AM, Hugo Buddelmeijer wrote:
>>> Hi Nicola et al.,
>>>
>>> This night I've done a first ingestion of all the 52 catalogs made
>>> with KIDSCAT beta v1.0. They can be found by querying for SourceLists
>>> with the name "INTDR2test_*", like:
>>> awe> slsintdr2 = SourceList.name.like("INTDR2test_*")
>>> awe> len(slsintdr2)
>>> 52
>>> Or through the dbviewer:
>>> http://socuteurl.com/funnytumble
>>>
>>> They have 'test' in the name because they are simply a conversion from
>>> the .cat files to AW SourceLists. No thorough assessment of the
>>> catalogs are done yet. However, they seem okay, and we will start
>>> using them here.
>>>
>>> What has not yet been done:
>>> - Ingesting the sexconf/sexparams (first thing to do)
>>> - Ingestion of the masks (the weights used are from AW right?)
>>> - Ingestion of the intermediate catalogs
>>> - Ingestion(/recreation) of QC plots
>>>
>>> What happened to the columns:
>>> - FLUX_APER and FLUXERR_APER are renamed to FLUX_APER_<aperturesize>
>>> and FLUXERR_APER_<aperturesize>. So FLUX_APER_10 is the flux in a
>>> 10 arcsecond aperture.
>>> - The VIGNET columns have been removed.
>>> - Some columns are renamed according to the FITSCOLUMN_NAMES dictionary
>>> in SourceList.py:
>>>
>>> http://www.astro-wise.org/viewvc/awe/astro/main/SourceList.py?annotate=1.228#l2788 
>>>
>>>
>>> - The HTM column is added.
>>>
>>> I will improve the catalog ingestion, starting with the sexconf, since
>>> this will improve the data lineage and reproducebility. If there is
>>> anything wrong, unclear or missing, please notify me!
>>>
>>> Greetings,
>>> Hugo
>>>
>>> P.S. I moved this discussion to kids at astro-wise.org, that way everyone
>>> gets them. Contact Gijs if you're not on this list yet.
>>>
>>>
>>>
>>> On 21/11/12 10:48, Nicola R. Napolitano wrote:
>>>>
>>>> I forgot to include Francesco in the mailing list....reply to this
>>>> message if needed.
>>>>
>>>> Cheers,
>>>>
>>>> Nicola
>>>>> Hi everyone,
>>>>>
>>>>> the catalogs for the 52 coadds selected for the INT-DR2 have been
>>>>> produced using KIDSCAT beta v1.0 and current products can be
>>>>> downloaded at the following link
>>>>>
>>>>> http://vstportal.oacn.inaf.it/KIDSCAT/KIDSCAT_INTDR2_52FIELDS/
>>>>>
>>>>> We have started to have a visual check to the catalogs in order to
>>>>> have a firs catalog QC before these are passed/ingested to AW for the
>>>>> QC of the coadds. In particular we are concentrating to the following
>>>>> checks:
>>>>>
>>>>> 1) look at the plot FWHM vs SNR in the PLOTS directory (Check that
>>>>> there is only one stellar sequence and that the spread is reasonably
>>>>> within a couple of pixels. Check that there are no weird features 
>>>>> like
>>>>> clouds of points at low FWHM - e.g. satellite tracks?)
>>>>> 2) in the same directory there are now also the mag limit plots right
>>>>> implemented from Francesco (mag lim estimates are in
>>>>> OUTPUT_RES/MLIM.dat, see also Francesco's message);
>>>>> 3) look at a 2D X-Y plot of all sources versus the ones with col7
>>>>> (IMAFLAGS_ISO) <1 "bona fide" sources to see whether the distribution
>>>>> is uniform and cross-check with the fits image to see whether the
>>>>> major stars are correctly masked;
>>>>> 4) plot flux_radius (col9) vs mag_auto (col15) for an independent
>>>>> check of the stellar sequences;
>>>>> 5) report comments on the test above in a table with the following
>>>>> columns:
>>>>> #FIELD; BAND; FWHM vSNR; 2Dplot; 2Dflag0 plot; visual vs ds9;
>>>>> fluxRvsmag_A; extraComm
>>>>>
>>>>> I'll gather all you comments and I'll put in a table which will 
>>>>> posted
>>>>> on the wiki
>>>>>
>>>>> http://wiki.astro-wise.org/projects:kids:data_deliveries:internaldelivery2.0_scratch#quality_check_of_the_52_catalog_for_int-dr2 
>>>>>
>>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Nicola
>>>>>
>>>>> -- 
>>>>> Nicola R. Napolitano
>>>>> Research Astronomer
>>>>> INAF - Observatory of Capodimonte
>>>>> Salita Moiariello, 16, 80131, Napoli (Italy)
>>>>> phone: +39 081 5575509; fax: +39 081 456710
>>>>> email:napolita at na.astro.it <mailto:napolita at na.astro.it>
>>>>> web page:http://www.na.astro.it/~napolita
>>>>> <http://www.na.astro.it/%7Enapolita>
>>>>
>>>>
>>>> -- 
>>>> Nicola R. Napolitano
>>>> Research Astronomer
>>>> INAF - Observatory of Capodimonte
>>>> Salita Moiariello, 16, 80131, Napoli (Italy)
>>>> phone: +39 081 5575509; fax: +39 081 456710
>>>> email:napolita at na.astro.it <mailto:napolita at na.astro.it>
>>>> web page:http://www.na.astro.it/~napolita
>>>>
>>>
>>
>>
>

-- 
Nicola R. Napolitano
Research Astronomer
INAF - Observatory of Capodimonte
Salita Moiariello, 16, 80131, Napoli (Italy)
phone: +39 081 5575509; fax: +39 081 456710
email: napolita at na.astro.it
web page: http://www.na.astro.it/~napolita