Articles in this series:
Binning (this page)
Having measured the main parameters of the QHY183M sensor at full resolution, I want to now look at the impact of binning. The purpose of this investigation is to decide which specific binning strategy to use. For example, should I bin all raw frames (i.e. flats, darks, lights) followed by all subsequent reduction and photometry? Or should all reduction be done on unbinned frames, with the calibrated lights binned just before photometry?
Why bin at all? The practical reasons are to reduce storage space, and processing time. At full resolution each frame is 40MB, and each step in the process will involve dealing with 20 megapixel images. Binning 2×2 reduces file size to 10MB, and other things being equal reduces the number of computations needed by this factor of 4. Also on my system, full resolution gives a pixel scale of 0.3″/pix. My seeing is usually >4″ so this is way oversampled. Even at 2×2 binning the pixel scale is only 0.6″/pix.
So if I can bin without losing precision I will do so.
The binning tool
Binning during acquisition (i.e., setting the camera driver to 2×2 bin) is not a good idea. To see why not, consider a pixel in the middle of its range, at 2,000 ADU as output by the 12-bit ADCs (analog-digital converter) on the chip. The camera driver multiplies this by 16 before it is output (I have checked that this happens both with the native QHY driver as well as the ASCOM driver), giving 32,000.
Now consider 4 adjacent pixels at the same 2,000 ADU. If we set 2×2 binning the driver will add all 4 ADUs together, then multiply by 16. This should give 128,000 but of course the output FITS file is restricted to 16 bit integers. So instead of 128,000 what is output is 65,535. We have ‘digital saturation’, whereas the physical pixels are still well within their usable range, and lost dynamic range.
So I will bin in software post acquisition, by taking the output ADU, dividing by 16, then adding the binned pixels together. Using the same example, the 32,000 output is divided by 16 to get 2,000 ADU. The 2×2 binned pixel then has 8,000 ADU. All nicely within range.
I implemented this in a little Python program called FITSbin. It’s based on the algorithm described here. It’s pretty quick, taking just 4mins to bin 200 frames of 20MP each on my 2014 vintage Pentium G3258. I haven’t checked it, but my experience with other programs that use a for loop through each pixel is that this is substantially quicker.
Binning strategies
There are several alternative ways (‘strategies’) to do binning in the photometry pipeline. I illustrate these below. The ‘B’ in a circle is the point where binning is done.
- ‘Original’: The procedure without binning. A bit simplified because I’ve left out Flat Darks.
- ‘Bin Early’: The simplest to implement. All acquired raw frames are binned before any processing. Processing will take the least time.
- ‘Bin Late’: Light frames will only be binned just prior to photometry.
- ‘Bin The Masters’: Light frames binned before any processing, and master flat and dark binned after being combined.
- ‘Bin After Dark’: Light frames are binned after dark subtraction, but before flat framing.
Several of these are motivated by my expectation that it would be beneficial to do all dark processing before binning because there will be RTS affected pixels on the dark frames. Binning too early might allow these to affect (contaminate) the other 3 pixels that may not be RTS affected in the 2×2 bin. At the moment this is just a vague notion. I will try to test this.
Binning Darks – before or after?
Is a master dark made from binned raw darks the same as one that is binned after combining unbinned raw darks? This is one of the differences between ‘Bin The Masters’ and ‘Bin Early’. Here are the histograms:
I cannot see much difference. But the real test is how well the hot pixels in light frames subtract out with either of these master darks.
In both cases most of the hot pixels have subtracted out cleanly. Just 2 remain. Out of 20 million, that’s very good. How about doing binning after dark subtraction i.e., ‘Bin Late’, or ‘Bin After Dark’?
Binning after dark subtraction
Again, very similar. The mean, min and max are all very slightly higher, which may mean that in the unbinned calibration some hot pixels have been suppressed. But the effect is small.
Conclusions
I have not yet tested for any differences due to when binning of flat frames is done. But we know that the sensor’s response to light is very linear. So binning flats at any point in the process should be fine.
Given that the different strategies I have tested have similar results, I will adopt the one that is easiest to implement. That is ‘Bin Early’. It has the added advantage of reducing the number of pixels to be processed in making a master flat. Because I will be taking many raw flats (probably ~100) this will save processing time.
This strategy also simplifies how I think about and communicate the parameters of the sensor.
Unbinned, it is:
- 20MP with 2.4 micron pixels
- Read noise = 2.67 e-
- Full well = 12,500 e-
- Limit of linearity = 3,300 ADU
But binned 2×2:
- 5MP with 4.8 micron pixels
- Read noise = 2 x 2.67 = 5.34 e-
- Full well = 50,000 e-
- Limit of linearity = 13,200 ADU
These are the theoretical numbers. In the next post I will present actual measurements.