[KIDS] Improved background subtraction to remove CCD edge vignetting

[Hugo Buddelmeijer]

Hi Nicola,

Thanks for testing that the bar removal indeed improved KIDS_175.0_-0.5 
i as well, indeed a troublesome observation.

We decided in the telecon to check the effect on the photometry on a 
full tile. KIDS_184.0_-1.5 is a good test case, since it shows bar 
problems in at least g, r and i. Could you compare the photometry for 
these coadds:

band:  old object_id                new object_id
u: ce4fc4615d66ebf2e0407d81e60e07f4 D2DE8960273B568DE0407D81E60E7CAD
g: ce4cd66f33e9ad69e0407d81e60e6525 D2DE4D015103EF62E0407D81E60E7A67
r: ce4cfe555662aa8ee0407d81e60e6745 D2ED2945BFE943D9E0407D81E60E02BC
i: ce4ebb600dd0e946e0407d81e60e7a9d D2DE896026D9568DE0407D81E60E7CAD

These are all three improved by the new algorithm, although i still 
shows residual bars. The u band image is visually improved as well, 
although this is a side-effect. Links to the dbviewer are on the wiki:
   http://wiki.astro-wise.org/projects:kids:tbarvignetting

Greetings,
Hugo


On 09/01/13 13:52, Nicola R. Napolitano wrote:
>
> Hi Hugo and all,
>
> the corrected images look very good, nice job.
> We had a quick check to the image that looked worse than the others with
> clear severe features for CCD vignetting, i.e. KIDS_175.0_-0.5i from the
> wiki page.
> In attachment the catalogs before and after the correction. Red are all
> sources, green the "good" ones.
> The situation is indeed improved and also if there remains some slight
> horizontal features in the coadd in the area of CCD82, the catalogs are
> note significantly affected. Of course we want to look in more details
> at the extracted sources in that area.
>
> All other images you sent in your examples with residual CCD gaps did
> not actually present spurious detection in catalogs, thus we expect the
> improvement here is on the photometry measurement uncertainty.
>
> Cheers,
>
> Nicola
>
>
>
> On 1/9/13 9:56 AM, Hugo Buddelmeijer wrote:
>> Dear KiDS members,
>>
>> An improved background subtraction algorithm for RegriddedFrames was
>> created to remove the vignetting effect at the edges of the CCDs. We
>> (Groningen) plan to use the method for KIDS ESO DR1.
>>
>> * Example *
>> See this example CoaddedRegriddedFrame of KIDS_184.0_-0.5 in r:
>> before: http://tinyurl.com/arce9oj
>> after: http://tinyurl.com/aso2q5k
>> And in the i-band, KIDS_131.0_-1.5:
>> before: http://tinyurl.com/b33pu47
>> after: http://tinyurl.com/a3muamz
>>
>> * Procedure *
>> The method is selected by choosing BACKGROUND_SUBTRACTION_TYPE 4 for a
>> RegriddedFrame. (Only in 'current', not yet in 'AWBASE'.) This performs
>> the following:
>> 1) Create a mask to flag all sources and bad pixels.
>> 2) Calculate the median value of the background pixels of each row.
>> 3) Subtract this median value from the row.
>> 4) Let Swarp remove the rest of the background (as usual).
>>
>> * KIDS ESO DR1 *
>> Based on our experiments the Groningen team suggests to use the new
>> method for the KIDS ESO DR1 release. We plan to start processing next
>> week so there will be time to reprocess bad cases, if they arise. Do you
>> agree?
>>
>> * Caveats *
>> There are currently no known cases where the quality of the
>> CoaddedRegriddedFrames decreases. Chip-filling galaxies will probably be
>> a problem, but that was already the case with the original method. The
>> vertical bar pattern seen occasionally at the corner CCDs is not treated.
>>
>> * Future *
>> The 'bar pattern' arises because the vignetting of the background is
>> slightly different in the science images than in the calibration flats.
>> It might therefore be possible to improve the flatfielding to remove the
>> bar already during the creation of ReducedScienceFrames. This requires
>> more investigation and will not be achieved in time for ESO DR1.
>>
>> * Sources *
>> The flux of the sources is affected differently by the vignetting than
>> the background. Correcting the source flux requires a separate solution
>> and is therefore beyond the scope of this discussion. (There can be a
>> flux error of about 1-2% in the affected regions of the individual
>> exposures, leading to a potential flux error of 0.2-0.4% in the coadds.)
>>
>> * More Information *
>> The wiki contains more information about the bars in general and
>> examples of the improved background subtraction:
>> http://wiki.astro-wise.org/projects:kids:tbarvignetting
>> In particular in the section "Improved background subtraction".
>>
>> Greetings,
>> Hugo Buddelmeijer
>> _______________________________________________
>> KiDS mailing list
>> KiDS at astro-wise.org
>> http://listman.astro-wise.org/mailman/listinfo/kids
>
>
>
>
> _______________________________________________
> KiDS mailing list
> KiDS at astro-wise.org
> http://listman.astro-wise.org/mailman/listinfo/kids