[KIDS] Improved background subtraction to remove CCD edge vignetting

Fri Jan 18 12:09:25 CET 2013

Hi Hugo and everybody,

Francesco and I finally got the chance to check the photometry on the 
post improved background subtraction algorithm (IBSA) from Hugo. The 
IBSA is very well improving the object detection as the more regular 
background structure prevents spurious source extraction. It remained to 
figure out whether the new background affects object photometry.

To assess this we have compared KIDSCAT magnitudes within 5'' diameter 
apertures against SDSS petrosian magnitudes.

We have considered the KIDS_175.0_-0.5i field (see Fig. 1 before and 
after IBSA on the left and right panel respectively) since all others 
coadds from Hugo were mostly showing less severe horizontal features.

We have first concentrated on the area affected by the CCD82 vignetting 
where the background correction has to have been stronger (green area in 
Fig. 1).
Here we have considered all flag=0 sources from KIDSCAT matched to SDSS  
and computed  the delta_mag  for images  before-IBSA and after-IBSA. We 
have selected about 900 sources in the three catalogs.

The delta mags are plotted in Fig. 2 (delta_mag= mag_SDSS-mag_KCAT). We 
have selected only sources brighter than 21 (SDSS i-band). Objects have 
generally consistent magnitudes (mean~-0.06 both before and after IBSA 
respectively, mostly due to an aperture effect) while the scatter of the 
histogram grows after IBSA (sig=0.20, 0.26 before and after IBSA, 
respectively), in particular because of an excess of sources after IBSA 
in the negative tail. We have checked that this tails is not mag 
dependent. Thus there seems that the IBSA possibly subtracts some of the 
source flux. This likely happens for ~20% of the sources in the given 
CCD82-selected area.

We have made the same plot for the whole image in order to see whether 
this effect is present also in the area with less problematic 
background. The delta_mag plot is shown in Fig. 3. Here we see that the 
two distributions, although presenting yet a negative tail, are much 
more overlapping with each other, which means that the overall 
photometry on the whole field is almost unchanged. Thus the IBSA does 
introduces a difference in the photometry, but apparently this is mainly 
affecting sources in regions affected by strong horizontal features.

Conclusions: IBSA improves  source detection  without basically 
affecting the source photometry in the regions with moderate BKG 
variation (e.g.  normal CCD gaps), while it can introduce some 
photometry variation in the area of strong CCD vignetting. We propose to 
add a flag to the catalog for these latter cases in order to warn about 
photometric reliability due to CCD failures.

Cheers,

Nicola and Francesco

On 1/10/13 11:30 AM, Hugo Buddelmeijer wrote:
> Hi Nicola,
>
> Thanks for testing that the bar removal indeed improved KIDS_175.0_-0.5
> i as well, indeed a troublesome observation.
>
> We decided in the telecon to check the effect on the photometry on a
> full tile. KIDS_184.0_-1.5 is a good test case, since it shows bar
> problems in at least g, r and i. Could you compare the photometry for
> these coadds:
>
> band:  old object_id                new object_id
> u: ce4fc4615d66ebf2e0407d81e60e07f4 D2DE8960273B568DE0407D81E60E7CAD
> g: ce4cd66f33e9ad69e0407d81e60e6525 D2DE4D015103EF62E0407D81E60E7A67
> r: ce4cfe555662aa8ee0407d81e60e6745 D2ED2945BFE943D9E0407D81E60E02BC
> i: ce4ebb600dd0e946e0407d81e60e7a9d D2DE896026D9568DE0407D81E60E7CAD
>
> These are all three improved by the new algorithm, although i still
> shows residual bars. The u band image is visually improved as well,
> although this is a side-effect. Links to the dbviewer are on the wiki:
>     http://wiki.astro-wise.org/projects:kids:tbarvignetting
>
> Greetings,
> Hugo
>
>
> On 09/01/13 13:52, Nicola R. Napolitano wrote:
>> Hi Hugo and all,
>>
>> the corrected images look very good, nice job.
>> We had a quick check to the image that looked worse than the others with
>> clear severe features for CCD vignetting, i.e. KIDS_175.0_-0.5i from the
>> wiki page.
>> In attachment the catalogs before and after the correction. Red are all
>> sources, green the "good" ones.
>> The situation is indeed improved and also if there remains some slight
>> horizontal features in the coadd in the area of CCD82, the catalogs are
>> note significantly affected. Of course we want to look in more details
>> at the extracted sources in that area.
>>
>> All other images you sent in your examples with residual CCD gaps did
>> not actually present spurious detection in catalogs, thus we expect the
>> improvement here is on the photometry measurement uncertainty.
>>
>> Cheers,
>>
>> Nicola
>>
>>
>>
>> On 1/9/13 9:56 AM, Hugo Buddelmeijer wrote:
>>> Dear KiDS members,
>>>
>>> An improved background subtraction algorithm for RegriddedFrames was
>>> created to remove the vignetting effect at the edges of the CCDs. We
>>> (Groningen) plan to use the method for KIDS ESO DR1.
>>>
>>> * Example *
>>> See this example CoaddedRegriddedFrame of KIDS_184.0_-0.5 in r:
>>> before: http://tinyurl.com/arce9oj
>>> after: http://tinyurl.com/aso2q5k
>>> And in the i-band, KIDS_131.0_-1.5:
>>> before: http://tinyurl.com/b33pu47
>>> after: http://tinyurl.com/a3muamz
>>>
>>> * Procedure *
>>> The method is selected by choosing BACKGROUND_SUBTRACTION_TYPE 4 for a
>>> RegriddedFrame. (Only in 'current', not yet in 'AWBASE'.) This performs
>>> the following:
>>> 1) Create a mask to flag all sources and bad pixels.
>>> 2) Calculate the median value of the background pixels of each row.
>>> 3) Subtract this median value from the row.
>>> 4) Let Swarp remove the rest of the background (as usual).
>>>
>>> * KIDS ESO DR1 *
>>> Based on our experiments the Groningen team suggests to use the new
>>> method for the KIDS ESO DR1 release. We plan to start processing next
>>> week so there will be time to reprocess bad cases, if they arise. Do you
>>> agree?
>>>
>>> * Caveats *
>>> There are currently no known cases where the quality of the
>>> CoaddedRegriddedFrames decreases. Chip-filling galaxies will probably be
>>> a problem, but that was already the case with the original method. The
>>> vertical bar pattern seen occasionally at the corner CCDs is not treated.
>>>
>>> * Future *
>>> The 'bar pattern' arises because the vignetting of the background is
>>> slightly different in the science images than in the calibration flats.
>>> It might therefore be possible to improve the flatfielding to remove the
>>> bar already during the creation of ReducedScienceFrames. This requires
>>> more investigation and will not be achieved in time for ESO DR1.
>>>
>>> * Sources *
>>> The flux of the sources is affected differently by the vignetting than
>>> the background. Correcting the source flux requires a separate solution
>>> and is therefore beyond the scope of this discussion. (There can be a
>>> flux error of about 1-2% in the affected regions of the individual
>>> exposures, leading to a potential flux error of 0.2-0.4% in the coadds.)
>>>
>>> * More Information *
>>> The wiki contains more information about the bars in general and
>>> examples of the improved background subtraction:
>>> http://wiki.astro-wise.org/projects:kids:tbarvignetting
>>> In particular in the section "Improved background subtraction".
>>>
>>> Greetings,
>>> Hugo Buddelmeijer
>>> _______________________________________________
>>> KiDS mailing list
>>> KiDS at astro-wise.org
>>> http://listman.astro-wise.org/mailman/listinfo/kids
>>
>>
>>
>> _______________________________________________
>> KiDS mailing list
>> KiDS at astro-wise.org
>> http://listman.astro-wise.org/mailman/listinfo/kids
> _______________________________________________
> KiDS mailing list
> KiDS at astro-wise.org
> http://listman.astro-wise.org/mailman/listinfo/kids

-- 
Nicola R. Napolitano
Research Astronomer
INAF - Observatory of Capodimonte
Salita Moiariello, 16, 80131, Napoli (Italy)
phone: +39 081 5575509; fax: +39 081 456710
email: napolita at na.astro.it
web page: http://www.na.astro.it/~napolita

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Fig1.ps
Type: application/postscript
Size: 6702666 bytes
Desc: not available
Url : http://listman.astro-wise.org/pipermail/kids/attachments/20130118/5cb3eaae/attachment-0003.ps 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Fig2.ps
Type: application/postscript
Size: 15096 bytes
Desc: not available
Url : http://listman.astro-wise.org/pipermail/kids/attachments/20130118/5cb3eaae/attachment-0004.ps 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Fig3.ps
Type: application/postscript
Size: 14850 bytes
Desc: not available
Url : http://listman.astro-wise.org/pipermail/kids/attachments/20130118/5cb3eaae/attachment-0005.ps 