[KIDS] Improved background subtraction to remove CCD edge vignetting

Fri Jan 18 12:25:18 CET 2013

Hi Nicola,

thanks for performing this analysis, and good that the photometry seems 
nearly not affected.

Do you also have plots of the delta-mag between the KiDS-CAT magnitudes 
before and after IBSA?
Comparing the difference on the KiDS photometry directly might give us 
more detailed information than comparing both separately to SDSS.
That would also allow a comparison at fainter magnitudes.

Cheers,
Jelte

On 18/01/13 12:09, Nicola R. Napolitano wrote:
>
> Hi Hugo and everybody,
>
> Francesco and I finally got the chance to check the photometry on the 
> post improved background subtraction algorithm (IBSA) from Hugo. The 
> IBSA is very well improving the object detection as the more regular 
> background structure prevents spurious source extraction. It remained 
> to figure out whether the new background affects object photometry.
>
> To assess this we have compared KIDSCAT magnitudes within 5'' diameter 
> apertures against SDSS petrosian magnitudes.
>
> We have considered the KIDS_175.0_-0.5i field (see Fig. 1 before and 
> after IBSA on the left and right panel respectively) since all others 
> coadds from Hugo were mostly showing less severe horizontal features.
>
> We have first concentrated on the area affected by the CCD82 
> vignetting where the background correction has to have been stronger 
> (green area in Fig. 1).
> Here we have considered all flag=0 sources from KIDSCAT matched to 
> SDSS  and computed  the delta_mag  for images  before-IBSA and 
> after-IBSA. We have selected about 900 sources in the three catalogs.
>
> The delta mags are plotted in Fig. 2 (delta_mag= mag_SDSS-mag_KCAT). 
> We have selected only sources brighter than 21 (SDSS i-band). Objects 
> have generally consistent magnitudes (mean~-0.06 both before and after 
> IBSA respectively, mostly due to an aperture effect) while the scatter 
> of the histogram grows after IBSA (sig=0.20, 0.26 before and after 
> IBSA, respectively), in particular because of an excess of sources 
> after IBSA in the negative tail. We have checked that this tails is 
> not mag dependent. Thus there seems that the IBSA possibly subtracts 
> some of the source flux. This likely happens for ~20% of the sources 
> in the given CCD82-selected area.
>
> We have made the same plot for the whole image in order to see whether 
> this effect is present also in the area with less problematic 
> background. The delta_mag plot is shown in Fig. 3. Here we see that 
> the two distributions, although presenting yet a negative tail, are 
> much more overlapping with each other, which means that the overall 
> photometry on the whole field is almost unchanged. Thus the IBSA does 
> introduces a difference in the photometry, but apparently this is 
> mainly affecting sources in regions affected by strong horizontal 
> features.
>
> Conclusions: IBSA improves  source detection  without basically 
> affecting the source photometry in the regions with moderate BKG 
> variation (e.g.  normal CCD gaps), while it can introduce some 
> photometry variation in the area of strong CCD vignetting. We propose 
> to add a flag to the catalog for these latter cases in order to warn 
> about photometric reliability due to CCD failures.
>
> Cheers,
>
> Nicola and Francesco
>
> On 1/10/13 11:30 AM, Hugo Buddelmeijer wrote:
>> Hi Nicola,
>>
>> Thanks for testing that the bar removal indeed improved KIDS_175.0_-0.5
>> i as well, indeed a troublesome observation.
>>
>> We decided in the telecon to check the effect on the photometry on a
>> full tile. KIDS_184.0_-1.5 is a good test case, since it shows bar
>> problems in at least g, r and i. Could you compare the photometry for
>> these coadds:
>>
>> band:  old object_id                new object_id
>> u: ce4fc4615d66ebf2e0407d81e60e07f4 D2DE8960273B568DE0407D81E60E7CAD
>> g: ce4cd66f33e9ad69e0407d81e60e6525 D2DE4D015103EF62E0407D81E60E7A67
>> r: ce4cfe555662aa8ee0407d81e60e6745 D2ED2945BFE943D9E0407D81E60E02BC
>> i: ce4ebb600dd0e946e0407d81e60e7a9d D2DE896026D9568DE0407D81E60E7CAD
>>
>> These are all three improved by the new algorithm, although i still
>> shows residual bars. The u band image is visually improved as well,
>> although this is a side-effect. Links to the dbviewer are on the wiki:
>>     http://wiki.astro-wise.org/projects:kids:tbarvignetting
>>
>> Greetings,
>> Hugo
>>
>>
>> On 09/01/13 13:52, Nicola R. Napolitano wrote:
>>> Hi Hugo and all,
>>>
>>> the corrected images look very good, nice job.
>>> We had a quick check to the image that looked worse than the others 
>>> with
>>> clear severe features for CCD vignetting, i.e. KIDS_175.0_-0.5i from 
>>> the
>>> wiki page.
>>> In attachment the catalogs before and after the correction. Red are all
>>> sources, green the "good" ones.
>>> The situation is indeed improved and also if there remains some slight
>>> horizontal features in the coadd in the area of CCD82, the catalogs are
>>> note significantly affected. Of course we want to look in more details
>>> at the extracted sources in that area.
>>>
>>> All other images you sent in your examples with residual CCD gaps did
>>> not actually present spurious detection in catalogs, thus we expect the
>>> improvement here is on the photometry measurement uncertainty.
>>>
>>> Cheers,
>>>
>>> Nicola
>>>
>>>
>>>
>>> On 1/9/13 9:56 AM, Hugo Buddelmeijer wrote:
>>>> Dear KiDS members,
>>>>
>>>> An improved background subtraction algorithm for RegriddedFrames was
>>>> created to remove the vignetting effect at the edges of the CCDs. We
>>>> (Groningen) plan to use the method for KIDS ESO DR1.
>>>>
>>>> * Example *
>>>> See this example CoaddedRegriddedFrame of KIDS_184.0_-0.5 in r:
>>>> before: http://tinyurl.com/arce9oj
>>>> after: http://tinyurl.com/aso2q5k
>>>> And in the i-band, KIDS_131.0_-1.5:
>>>> before: http://tinyurl.com/b33pu47
>>>> after: http://tinyurl.com/a3muamz
>>>>
>>>> * Procedure *
>>>> The method is selected by choosing BACKGROUND_SUBTRACTION_TYPE 4 for a
>>>> RegriddedFrame. (Only in 'current', not yet in 'AWBASE'.) This 
>>>> performs
>>>> the following:
>>>> 1) Create a mask to flag all sources and bad pixels.
>>>> 2) Calculate the median value of the background pixels of each row.
>>>> 3) Subtract this median value from the row.
>>>> 4) Let Swarp remove the rest of the background (as usual).
>>>>
>>>> * KIDS ESO DR1 *
>>>> Based on our experiments the Groningen team suggests to use the new
>>>> method for the KIDS ESO DR1 release. We plan to start processing next
>>>> week so there will be time to reprocess bad cases, if they arise. 
>>>> Do you
>>>> agree?
>>>>
>>>> * Caveats *
>>>> There are currently no known cases where the quality of the
>>>> CoaddedRegriddedFrames decreases. Chip-filling galaxies will 
>>>> probably be
>>>> a problem, but that was already the case with the original method. The
>>>> vertical bar pattern seen occasionally at the corner CCDs is not 
>>>> treated.
>>>>
>>>> * Future *
>>>> The 'bar pattern' arises because the vignetting of the background is
>>>> slightly different in the science images than in the calibration 
>>>> flats.
>>>> It might therefore be possible to improve the flatfielding to 
>>>> remove the
>>>> bar already during the creation of ReducedScienceFrames. This requires
>>>> more investigation and will not be achieved in time for ESO DR1.
>>>>
>>>> * Sources *
>>>> The flux of the sources is affected differently by the vignetting than
>>>> the background. Correcting the source flux requires a separate 
>>>> solution
>>>> and is therefore beyond the scope of this discussion. (There can be a
>>>> flux error of about 1-2% in the affected regions of the individual
>>>> exposures, leading to a potential flux error of 0.2-0.4% in the 
>>>> coadds.)
>>>>
>>>> * More Information *
>>>> The wiki contains more information about the bars in general and
>>>> examples of the improved background subtraction:
>>>> http://wiki.astro-wise.org/projects:kids:tbarvignetting
>>>> In particular in the section "Improved background subtraction".
>>>>
>>>> Greetings,
>>>> Hugo Buddelmeijer
>>>> _______________________________________________
>>>> KiDS mailing list
>>>> KiDS at astro-wise.org
>>>> http://listman.astro-wise.org/mailman/listinfo/kids
>>>
>>>
>>>
>>> _______________________________________________
>>> KiDS mailing list
>>> KiDS at astro-wise.org
>>> http://listman.astro-wise.org/mailman/listinfo/kids
>> _______________________________________________
>> KiDS mailing list
>> KiDS at astro-wise.org
>> http://listman.astro-wise.org/mailman/listinfo/kids
>
>
>
>
> _______________________________________________
> KiDS mailing list
> KiDS at astro-wise.org
> http://listman.astro-wise.org/mailman/listinfo/kids

-- 
Dr. Jelte T. A. de Jong
KiDS project manager
Sterrewacht Leiden
Leiden University, Leiden, the Netherlands
E: jelte at strw.leidenuniv.nl
T: +31-(0)715275818
W: jelte.jdejong.net

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listman.astro-wise.org/pipermail/kids/attachments/20130118/045a80b5/attachment.html 