How We Reduce Our Mosaic Data
Photometry from Mosaic data offers several unique challenges. Typically,
Mosaic data is taken by dithering the telescope to 5 positions offset by
several hundred pixels (e.g., an arcminute or two) in order to assure that
each object is observed a minimum of 3 times despite the inter-chip gaps
and cosmetic defects. After the initial processing and flat-fielding,
these 5 ditherings are geometrically corrected and then stacked to form
a single image.
However, the photometric accuracy of the stacked images is compromised
by the following:
-
Significant PSF variations exist: a stellar image may be the average of 3,
4, or 5
exposures, each with differing PSFs. In addition, chip-to-chip PSF variations
are seen.
- The 8 chips do not have the same color responses. This has two effects:
- The color transformations are dependent upon which chip(s) an object was
observed.
- Photometric zero-points are different chip-to-chip even after the best
flat-fielding,
as the amount of light seen by each chip is color-dependent.
While earlier work by Massey & Slesnick showed that
aperture
photometry was still possible at the 2% level, our Survey project needs
PSF-fitting on these croweded
fields. We have therefore decided to side-step
these problems by
treating
the 8 chips and 5 ditherings separately, doing the photometry
right and then averaging
the photometry.
A critical step in this process is external calibration of all of
our northern fields using
the Lowell 1.1-m Hall telescope. The time required to obtain
adequate calibration
with Mosaic is prohibitive: to move a set of standards from chip to chip for all
the CCDs in
each of the 5 colors requires 2 hrs just in read-out time;
a single standard star field observation requires 15 minutes. In addition,
using
external
photometric calibration allows us to utilize good, but not perfectly photometric
nights at the 4-m.
Chip-to-Chip dependent Color Terms and Zero-points
Our calibration efforts are on-going, but here are some tenative findings.
The photometry was done by Zhenye Mei, an excellent MIT student who
participated in the 2001 MIT Field Camp at Lowell Observatory.
The equations that we solved were as follows:
u=U+u1+u2*X+u3*(U-B)
b=B+b1+b2*X+b3*(B-V)
v=V+v1+v2*X+v3*(B-V)
r=R+r1+r2*X+r3*(V-R)
i=I+i1+i2*X+i3*(R-I)
u, b, v, r, and i refer to the instrumental magnitudes,
while U, B, V, R, and I refer to standard magnitudes..
Thus, u1, b1, v1, r1, and i1 are the zero-points, u2, b2, v2, r2, and i2 are the
extinction terms, and u3, b3, v3, r3, and i3 are the color terms.
We insisted of course that the color terms and zero points were constant
from night to night, but allowed them to be different from chip-to-chip;
the extinction we insisted was constant for all the chips but varied from
night-to-night. "Typical" values for the extinction for two photometric
nights were:
u2=0.53-0.55
b2=0.15-0.19
v2=0.09-0.12
r2=0.02-0.05
i2=0.01-0.02
|
Color Terms |
Zero Points |
| Chip |
u3 |
b3 |
v3 |
r3 |
i3 |
u1 |
b1 |
v1 |
r1 |
i1 |
| imt1 |
-.043 |
-.122 |
+.013 |
-.031 |
-.080 |
1.536 |
-.065 |
-.185 |
-.316 |
+.274 |
| imt2 |
-.001: |
-.128 |
+.004 |
-.045 |
-.038 |
1.523 |
-.070: |
-.172 |
-.324 |
+.286 |
| imt3 |
-.090 |
-.115 |
+.030 |
-.024 |
-.006 |
1.545 |
-.075 |
-.187 |
-.340 |
+.278 |
| imt4 |
-.043 |
-.113 |
+.013 |
-.042 |
-.041 |
1.526: |
-.078 |
-.188 |
-.359 |
+.268 |
| imt5 |
-.031 |
-.109 |
+.017 |
-.045 |
-.040 |
1.525: |
-.080 |
-.185 |
-.357 |
+.255 |
| imt6 |
-.074 |
-.140 |
-.001 |
-.068 |
-.034 |
1.551 |
-.060 |
-.176 |
-.335 |
+.253 |
| imt7 |
-.033 |
-.113 |
-.013 |
-.068 |
-.016 |
1.560: |
-.075: |
-.165 |
-.310 |
+.276 |
| imt8 |
+.002 |
-.140 |
+.005 |
-.071 |
-.045 |
1.516: |
-.076 |
-.190 |
-.332 |
+.247 |
The instrumental magnitudes were defined so that a star with with 1 count/sec/image was assigned a value of 25.00. Thus for a very blue star
(U-B=-1.0) with U=20.00 observed at the zenith (X=1.00) we would expect
u=20.00+1.54 +.55x1.00 -0.04 x -1.00 = 22.13.
This is 2.87 mag brighter than 25.00, so we would expect about 14 ADU/sec, or
39 e/sec; this can be compared to the "35 e/sec"
estimated in the Mosaic Manual .
The coefficients in the above table are good to approximately .005 mag;
less certain values are indicated by ":", where the errors are approximately
.010 mag. Thus the differences
of 0.09 in the color terms for "U" would translate to errors of .2 mag
from a U-B=-1 to +1, if ignored, as might be the case if the images were
stacked.
The values for "U" are actually for stars with U-B>0; for stars bluer
than this, a correction is needed, as star deviate by 0.1mag from this
relationship by the time that U-B=-1.0 is reached. (This just says that the
color-term is not linear over a large color range.)
How We're Doing It
Excellent information can be gleamed from:
Frank Valdes' IRAF
guide to the MSCRED package
Buell
Jannuzi's and Chris Greer's Deep Wide-Field Reduction Guide
Here are our variant on these, where we have attempted to high-light
what is still unclear to us.
- Modify the existing cross-talk coefficients using sparsely
populated fields
(Good to 0.01%)
- xtcoeff obj110 xtalkobj110.dat @vnoao8 @snoao8
- Use xtalkcor to see if you like it.
- Here's some history of the cross-talk coefficients:
| Victim |
Source |
Old |
Aug2000(BJ) |
Sept2000(Us) |
| im1 |
im2 |
.00116 |
.00133 |
.00152 |
| im2 |
--- |
--- |
--- |
--- |
| im3 |
im4 |
.00289 |
.00267 |
.00310 |
| im4 |
--- |
--- |
--- |
--- |
| im5 |
--- |
--- |
--- |
--- |
| im6 |
im5 |
.00153 |
.00159 |
.00168 |
| im7 |
--- |
--- |
--- |
--- |
| im8 |
im7 |
.00062 |
.00085 |
.00085 |
Note that a change of .0001 is just barely noticable on a saturated star:
(0.0001: x 30,000ADUs = 3ADUs)
- Correct the headers for the error in the shutter timing.
Our measurements in October 2000
showed a correct t(real)=t(requested)-0.027 seconds. (In other words, failure
to take this into account in a 3 sec standard exposure would lead to a
1% error in the photometry.)
- Do initial processing:
- setinstrument
- Construct master bias frame (stable to 1 ADU during a week run).
- zerocombine zero*.fits rejection=minmax process+ Do NOT do bad pixel stuff.
- ccdproc the data to correct for cross-talk, remove overscan,
trim,
and remove bias. Do not do bad pixel stuff.
- If there were bias-jumps, use a high order (3900 spline3); otherwise,
a constant will do. Does not matter to 1ADU if the bias has been treated the
same way.
- Modify the existing bad pixel maps using long and short dome flat exposures.
- copy the default bad pixel maps into something else, and modify them
by eye:
- imreplace bphil2_0009.pl[431:431,3631:3631] 1.
- etc.
- hedit *.fits[1] bpm bphil1_0009 add+ ver- show+ etc.
- Construct master dome flats and sky flats. These appear to be stable
during a run, other than 0.3% dust donuts that come and go.
- Iteratively construct a master flat-field as follows:
- flatcombine dflat*.fits process+ (turn bad pixel fixing on)
- Remove ghost from combined dome flats
(10% effect at U; smaller at other color)
- mscpupil type=ratio--->domenopup
- flatcombine sflat*.fits process+
- Divide sky flat by dome flat.
- Won't happen naturally as sky flats have type "flat"
- Remove ghost from flattened sky flat...somehow--->skynopup
- Save ghost from flattened sky flat for future use.
- mscpupil sflat* type=data
- masterflat=skynopup * domenopup
- Process all the data to correct for cross-talk, remove overscan, trim,
remove
bias,
interpolate over bad pixels (but keep track of these!), and flat-field the data.
- De-ghost the object frames using the templates determined above and rmpup
- Measure the saturation in the flattened chips, and revise headers:
- hedit *.fits[1] SATURATE 39000 add+ ver- show+
- hedit *.fits[2] SATURATE 23000 add+ ver- show+
- hedit *.fits[3] SATURATE 37000 add+ ver- show+
- hedit *.fits[4] SATURATE 24000 add+ ver- show+
- hedit *.fits[5] SATURATE 27000 add+ ver- show+
- hedit *.fits[6] SATURATE 27000 add+ ver- show+
- hedit *.fits[7] SATURATE 29000 add+ ver- show+
- hedit *.fits[8] SATURATE 24000 add+ ver- show+
- Generate new bad pixel maps that include the saturated stars
- ccdproc
- Examine the images and see if anything needs to be added to these
bad pixel maps.
- Determine the geometrical correction for all chips.
- Geometrically correct the images (without running fixpix!)
- mscimage @f2 mos//@f2 inter=sinc minter=linear
- Break the images apart for photometry:
- procedure phccd (input)
string input {prompt="Input list of images to be broken into 8 pieces"}
begin
string inlist, img
int i, j, ix, iy, im, y1,y2,x1,x2
#get parameters
inlist=input
#loop through the image list treating the first image as the reference:
list=inlist
while(fscan(list,img)!=EOF) {
printf("trying image %s\n", img)
imgets(img,"i_naxis1")
ix=(int(imgets.value))/4.
print(ix)
imgets(img,"i_naxis2")
iy=(int(imgets.value))/2.
print(iy)
for(j=1; j<=2; j=j+1) {
y1=(j-1)*iy+1
y2=y1+iy-1
for (i=1; i<=4; i=i+1) {
x1=(i-1)*ix+1
x2=x1+ix-1
im=(j-1)*4+i
print(im,x1,x2,y1,y2)
s1=img//'['//x1//':'//x2//','//y1//':'//y2//']'
s2='im'//im//img
imcopy(s1,s2)
}
}
}
end
- For the cosmetically beautiful images:
- Load proto and use fixpix to interpolate over saturated stars
- msccmd "imcopy $input $output" input=@f2all//pup.fits output=f//@f2all//pup.fits
- msccmd "fixpix $inlist mask=BPM"
- mscimage the new images inter=sinc minterp=linear. New bad pixel maps.
- Add anything we need to do the output bad pixel maps
- mscstack the mosaic'd images.
- Preserve on tape: everything
- Preserve on disk:
- 1) The mosaic'd images without fixpix (mosobjXXXpup.fits) and their
associated bad pixel maps (linmosobjXXXpup_bpm.pl)
- 2) The mosaic'd images with fixpix (mosfobjXXXpup.fits) and their
associated bad pixel maps (mosfobjXXXpup_bpm.pl)
- 3) The final stacked images: m31f3[UBVRI].fits
Naming conventions: what do we save, and when do we save it?
- objXXXpup.fits: depupiled, wcs-corrected, fully reduced individual
frames (assumes that we've made the final pass through ccdproc).
- bad pixel maps: satobsXXX directory (contains saturated stars as bad
pixels).
- mosobjXXXpup.fits: mosaiced images suitable for putting through phccd.cl;
bad pixel maps linobj*.pl
- imNmosobjXXXpup.fits: output from phccd.cl, individual images suitable
for photometry.
- fobjXXXpup.fits: same, but with saturated stars now interpoluated across;
(bad pixel maps in individual subdirectories "satobjxxx)
- mosfobsXXXpup.fits: mosaiced images; bad pixel maps mosfobjXXXpup_bpm_pl
- stacked images.
A Detailed Example
In the following example, I assume that we already have the following files;
- Revised bad pixel map bphil*.pl
- Revised cross-term file: xtalkphil.dat
- Master flats: master[UBVRI].fits
- Master bias frame: Zerona.fits
- Getting the data onto disk
- Read in an entire night onto disk using mscrfits
- Divide the data into appropriate subdirectories; "m31f2ubvri" for example
- Copy the calibration files locally.
- cp /data4/massey/survey/bphil*.pl .
- cp /data4/massey/survey/xtalkphil.dat .
- cp /data4/massey/survey/master*.fits .
- cp /data4/massey/survey/Zerona.fits .
- ccdproc obj*.fits function=spline3 order=3900
- Prepare master flats if you haven't already:
- cdproc dflat*.fits and combine (nightly)
- ccdproc sflat*.fits and combine (for the run, removing the bum ones)
- mscpupil n6FlatU.fits n6FlatUpupalt type="ratio"
- mscarith naSflatU.fits / n6FlatUpupalt skynU
- mscpupil skynU skynUalt type="ratio" <---this is how different the
sky and domes are!
- mscpupil skynU Upup type="data" [to be used on object frames]
- mscarith skynUalt * n6FlatUpupalt masterU
- test the new flat:
- ccdproc objxxx.fits [cross, over, trim, bias, bad pixel but not flat]
- mscarith objxxx.fits / masteru testmaster
- mscarith objxxx.fits / n6flatUpupalt testdome
- rmpup testmaster testmastpup Upup
- rmpup testdome testdomepup Upup
- See which is better!
Pay particular attention to the cresent on chip 8.
- Make sure that the flats are going to function as FLATS!
- msccmd "hselect $input ccdmean yes" input=master*.fits
- msccmd "hedit $input ccdmeant delete+ verify- show+" input=master*.fits
- ccdproc everything with flat on.
- Remove the ghost pupil:
- rmpupil @f3U @f3U//pupalt type=difference
- But for I:
mscpupil @f3I @f3I//pupalt type="difference"
actually rmpupil with Ipup2.fits is better!
>>>> actually rmpupil with Ipupfinal.fits is best!!!!
- Now comes the tedious part: making sure that the bad pixel maps are OK:
- mscdisplay each image and use imreplace
or if there's a particular chip....
such as streaks in 5:
- imreplace bpobj474/bpm_im5.pl[1:2048,1926:1949] 1.0
- imreplace bpobj474/bpm_im5.pl[1:2048,4080:4096] 1.0
- imreplace bpobj475/bpm_im5.pl[1:2048,4080:4096] 1.0
- imreplace bpobj475/bpm_im5.pl[1:2048,4080:4096] 1.0
(maybe should do this on the original images...)
- msccmatch @f3all//pupalt
- mscimage @f3all//pupalt n//@f3all//pupalt but wasn't very satisified with
the results
- imreplace the bad pixel maps if there's some problem on ccd3 (use 1e4, say)
- mscstack
- Revise the wcs:
- Incidently: you can always do a:
msccmd "wcscopy $input $in2 ver+
Input files (obj092): obj092a<---where it's GOING TO!!!!!!!!!
Second list of input files (obj093): obj092<---where it's COMING FROM!!!!!
- mscgetcatalog to get a list of ra and decs ---> m31f3
- LOAD immatchx
- msccmatch @all m31f3 to redo the wcs. Produces objXXXpup.fits,
de-pupiled, wcs-corrected fully reduced individual images.
- Correct the saturation parameters in the headers:
- hedit *.fits[1] SATURATE 39000 add+ ver- show+
- hedit *.fits[2] SATURATE 23000 add+ ver- show+
- hedit *.fits[3] SATURATE 37000 add+ ver- show+
- hedit *.fits[4] SATURATE 24000 add+ ver- show+
- hedit *.fits[5] SATURATE 27000 add+ ver- show+
- hedit *.fits[6] SATURATE 27000 add+ ver- show+
- hedit *.fits[7] SATURATE 29000 add+ ver- show+
- hedit *.fits[8] SATURATE 24000 add+ ver- show+
- Obtain mosaiced versions suitable for photometry:
- mscimage @all//pup.fits mos//@all//pup.fits interp=sinc minterp=linear
- files mos* > allmos
- phccd allmos [makes separate images]
- For the picture book:
- Make the fixed versions:
- msccmd "impcopy $input $output" input=@f2all//pup.fits output=f//@f2all//pup.fits
- Examine the fixed versions: and make sure there are no other bad pixels.
- msccmd "fixpix $input BPM" input=@f2all//pup.ifts
mscimage f//@f2all//pup.fits ---> mosf...inter=sinc minter=linear
- msccstack after much playing around
FOR SHAY:
- Revise the above example by going through it carefully.
- Stack some images from a non-photometric night.
- Do all the current UBVRI data.
- Do all the WR data.
- Do all the IF data.
- Figure out how to do photometry on the images.
Some remaining questions
How good is the PSF on the geometrically corrected images? Can we actually
DO PSF-fitting on these? Does it matter if we use "sinc" or "linear"?
How do we really keep track of the bad-pixels? None of the photometry
routines currently usees these.
Ghosts: some thoughts.
We found that the fine structure in the ghost image is
color-sensitive; a narrow-band 4700 Angstrom
image yields a high-contrast template ghost,
but fails to reproduce the details
of the ghost at U.
The broad-band U ghost is very large: 10%.
Failure to remove this from the flat-field would
introduce a 10% photometric error in the reduced data.
The first image shows the residual
ghost in a U-band M31 exposure after processing
(including
flat-fielding). The second image shows the image after the U-band template
ghost has been
subtracted. The red lines indicate the bad pixel maps.
E-mail:
massey@lowell.edu
Last updated 30 May 2001