[KIDS] QC catalogs for KiDS-INT-DR2, single-band sourcelists
Nicola R. Napolitano
napolita at na.astro.it
Fri Nov 16 16:56:54 CET 2012
Hi Gijs and all,
thanks for summing up things, here below some feedback which I prepared
after having heard from Francesco who seems didn't receive this message
(please add him to this mailing list). Let me announce that the 52
catalogs are going to be produced tomorrow (we got a window on the
beowulf to run KIDSCAT in parallel). Francesco has indeed adapted
KIDSCAT to make use of the 60 beowulf CPUs and we are in the condition
that we can produce 400 catalogs in 1h. So we are ready for massive
extraction runs.
On 11/15/12 3:33 PM, Gijs Verdoes Kleijn wrote:
> Dear CATALOG-QC-TEAM,
>
> The QC for KiDS catalogs falls in 2 parts: single-band and 4-band QC.
>
> * SINGLE-BAND SOURCELISTS
>
> We have the following QC checks in place in KiDS-CAT (thanks!), is my
> understanding:
>
> *
> FWHM vs SNR. DONE in KiDS-CAT. NN+HB can send algorithm/code to
> HB so Hugo can check implementation in AWE.
> *
> star/galaxy separation. DONE in KiDS-CAT. NN+HB can send
> algorithm/code to HB so Hugo can check implementation in AWE.
>
here I guess you mean NRN+FLB. Algorithm/code can be send straight away,
if this is needed to start implementing the codes in AWE and possibly
solve all the system compatibility problems. Be aware that the S/G
separation also is not yet implemented in KIDSCAT but still as matlab
code. You can either wait both tasks (S/G and limiting mag belo) are
translated in some pgplot script or you can implement in AWE as they are.
I don't know whether this might be really high priority as we don't
think that the current version of the catalog produced will be already
ingested as integrated sourcelist rather than external sourcelist (see
below).
> *
>
>
>
> *
> limiting magnitude. FLB can send a matlab (actually octave
> script) performing the analysis of the S/N vs. 2''mag diagram to
> HB Hugo Buddelmeijer who adds algorithm to qcSourceLists_INTDR2.py
> <http://www.astro-wise.org/viewvc/awe/astro/experimental/kids/qcSourceLists_INTDR2.py?view=markup&sortby=date>
>
>
as below, matlab codes will be sent
> *
>
>
> The NN+HB / FLB+HB indicate that I think it will be useful to
> implement those in Astro-WISE acting upon the ingested KiDS-CAT
> sourcelists. Nicola and Francesco can send the relevant info/code to
> Hugo Buddelmeijer who coordinates the implementation in Astro-WISE.
Currently catalogs contain 136 columns if the VIGNET ones are thrown
away. We don't know yet how many of these columns will be kept and how
many can be possibly added. I've been told that AWE can handle 100
columns as integrated sourcelist. How about using the current version of
the catalogs as external sourcelist? Linked to the original Coadd
together with the other plots (FWHM vs SNR, S/G, limiting,
completeness, etc.) in the DBviewer as below?
>
> The following QC checks are planned for implementation:
>
> *
> HB/DONE?: The qualityWISE link for KiDS-CAT sourcelists in
> Astro-WISE (example link in DBVIEWER
> <http://dbview.test.astro-wise.org/DbView?project=KIDS&set_cookie=&QSort=creation_date&QDesc=descending&2ndQSort=creation_date&2ndQDesc=descending&numrows=100&fcontext_project_only=on&show_expanded=no&Exportselect=NoDownload&class_str=SourceList&mode=table_res&depth=1&python_dict=%2528dp0%250AS%2527query%2527%250Ap1%250A%2528dp2%250AsS%2527expanded%2527%250Ap3%250A%2528dp4%250AS%2527SourceList%2527%250Ap5%250A%2528lp6%250AS%2527open%2527%250Ap7%250AaS%2527root%2527%250Ap8%250Aag5%250AasS%2527SourceList.chip%2527%250Ap9%250A%2528lp10%250AS%2527prefilled%2527%250Ap11%250AaS%2527link_property%2527%250Ap12%250AaS%2527Chip%2527%250Ap13%250AasS%2527SourceList.filter%2527%250Ap14%250A%2528lp15%250Ag11%250AaS%2527link_property%2527%250Ap16%250AaS%2527Filter%2527%250Ap17%250AasS%2527SourceList.instrument%2527%250Ap18%250A%2528lp19%250Ag11%250AaS%2527link_property%2527%250Ap20%250AaS%2527Instrument%2527%250Ap21%250AassS%2527deriv%0A%20%0A%20ed%2527%2%0A50Ap22%250A%2528dp23%250As.&expanded_leaf=&expanded_leaf_type=&expanded_leaf_class=&derived_leaf=&node_state=&SourceList.expanded=open&SourceList.chip.expanded=prefilled&SourceList.filter.expanded=prefilled&SourceList.instrument.expanded=prefilled&SourceList.SLID.op=%3D&SourceList.SLID=5967351&SourceList.detection_frame.base=BaseFrame&SourceList.frame.base=CoaddedRegriddedFrame&SourceList.instrument.name=OMEGACAM&SourceList.instrument.name.op=%3D>)
> directs to the QualityWISE pages of their coadds. These show
> catalog based information: see example
> <http://quality.test.astro-wise.org/QualityWISE?object_str=SourceList&object_id=ce10420f2fbdeb45e0407d81e60e7648&project=KIDS>.
> These plots are NOT based on the KiDS-INT-DR2 catalogs. They use a
> different sextractor run. Should we do something about this?
>
yes Francesco and I agree that this should be very straightforward and
we can provide all plots as above that you need to link to a page like this.
> *
> *
> FLB: completeness. FLB: you probably have best insight now how to
> do this.
>
this is work in progress. Francesco will send an email next week about
completeness and its implementation in KIDSCAT.
If you like, you can contact him directly and discuss this issue.
> *
>
>
>
> *
> GVK: spatial distribution of sources and flag values. GVK's
> proposal: a KiDS images with source/flag information overlayed as
> step1?
>
>
that would be very useful, although this might depend on the way we
ingest the catalogs, right?
> Summing these up: the CATALOG-QC-TEAM should verify we are not missing
> crucial basic ones:
>
> *
> CATALOG-QC-TEAM: CFHTLS at Terapix had its final release (T0007).
> One product is synoptic table of CFHTLS
> <http://terapix.iap.fr/cplt/T0007/table_syn_T0007.html>, giving
> access to QC plots/numbers. Which ones do you think are important
> at this stage for KiDS QC?
>
From what I can see we are can provide all the plot/numbers as in that
page (either already in progress or to be derived).
In particular
Stacking method: we are still stacked on only one right?
u g r i : done
chi2 : ?
cats-{u-g-r-i} : done
Mask- {u-g-r-i} : done
Mask [binary] : done
cats-{ugriz} +e(b-v): TBD
Galaxy Counts : can be done
Star: Color-Color tracks: can be done
Seeing (FWHM in arc-second): done
Completeness limit point-like[80%/50%]: in progress
ompleteness limit extended (disk) [80%/50%]: in progress
unmasked MegaCam field (deg2): done
Median stellar Mag-Diff [KiDS-SDSS]: TBD (FLB, MR)
> *
>
>
>
> What does CATALOG-QC-TEAM deliver for KiDS-INT-DR2 single-band
> sourcelists?:
> My proposal:
> 1.the QualityWISE page: those we have in hand, they come for free with
> Astro-WISE coadd production. Input on improving the plots presented
> there is welcome.
OK
> 2. A set of additional plots and/or numbers per catalog that
> complements the QualityWISE page with KiDS-INT-DR2 specific stuff.
> This can be either presented as one page per coadd or gallery of a
> certain plot for the 52 single-band sourcelists. Danny is looking into
> how we can implement such a presentation. I see this set as an
> experiment from which we can choose how to develop QualityWISE for
> single-band sourcelists further after KiDS team meeting in December.
see previous comment on the DBviewer.
Hope this can help.
Cheers,
Nicola
>
> Discussion has opened: all input welcomed (keep emailing
> kids at astro-wise.org!)
>
> Above information and more is contained at our wiki spot for
> CATALOG-QC-KIDS-INT-DR2 for single-bands:
> http://wiki.astro-wise.org/projects:kids:data_deliveries:internaldelivery2.0_scratch#qc_based_on_single-band_catalogs
>
> Regarding 4-BAND CATALOGS: I propose to leave that effort as a
> separate task. Details will follow in another email.
>
> Best wishes, Gijs
>
> --
> --------------------------------------------------------------------
> |dr Gijs Verdoes Kleijn | astronomer |
> |e-mail:verdoes at astro.rug.nl | OmegaCEN / Kapteyn Institute / |
> |www:www.astro.rug.nl/~verdoes | Target |
> |tel: +31-50-3638326 | University of Groningen |
> |mobile: +31-654658050 | postal address: |
> | | Kapteyn Astronomical Institute |
> | | Postbus 800, 9700 AV, Groningen|
> | | The Netherlands |
> --------------------------------------------------------------------
>
>
> _______________________________________________
> KiDS mailing list
> KiDS at astro-wise.org
> http://listman.astro-wise.org/mailman/listinfo/kids
--
Nicola R. Napolitano
Research Astronomer
INAF - Observatory of Capodimonte
Salita Moiariello, 16, 80131, Napoli (Italy)
phone: +39 081 5575509; fax: +39 081 456710
email: napolita at na.astro.it
web page: http://www.na.astro.it/~napolita
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listman.astro-wise.org/pipermail/kids/attachments/20121116/7b86c402/attachment.html
More information about the KiDS
mailing list