Introduction
While trying to process some images taken with my SBig STF-8300M in PixInsight I was encountering problems; PI would not
correctly read images from my SBig STF-8300M, either the original image collected by Maxim or the calibrated image. This seems to have to do with whether the images are saved in 16-bit signed or unsigned format; there is a lot of discussion in the PI forum about how this should be done, Maxim using non-standard FITS files, etc. In the end it appears the “fix” is to manually reformat all of the calibrated light subs in Maxim into 32 bit floating point format.
During the forum discussion people made statements like
It’s a really bad idea to let maxim touch your files at all… the less maxim does, the better.
Stop letting maxim calibrate images and do it all in PI. the results are superior anyway.
Avoid MaximDL like the plague once it has captured the image.
Interesting. Until now, I have been happy with Maxim calibrating the images. It is easy, happens automatically under ACP/Maxim, and the only problems encountered were caused by Some Idiot using bad flat or dark subs. Googling around, I do not find any studies showing the relative performance of Maxim versus PI in the calibration of images. I don’t know the source of these statements indicating the superior performance of PI in calibration.
On the other hand, my earlier tests on image alignment showed that PI was superior to Maxim and Registar. Maybe PI has better performance in calibration? Would this offset the extra manual work required to prepare and calibrate the images for PI
calibration (compared to almost no effort in Maxim calibration)?
So, here goes my test. Does PixInsight calibrate light subs better than Maxim?
Test data
I assembled a test data set of bias, dark, flat, and light frames. I use the same frames for calibrating under both PI and Maxim.
- 1 light frame (an OIII light sub, 30 minute exposure). This is the image to be calibrated. The camera is trying to get to -10C, but the ambient is high enough that the actual temperature is -4C. I just grabbed a sub from last night’s imaging run. Maxim calibrates using the original file. Since PI cannot open this file correctly, it uses a copy of the same file saved as 32 bit floating point.
- 20 Dark frames at -10C, 30 minute subs each. Collected in Maxim.
- 25 dusk sky flats with OIII filter collected in Maxim under ACP’s control. These flats are between -5 and -10 degrees C; at dusk the ambient temperature is still around 100 Fahrenheit so the camera can’t quite cool to -10C.
- 40 Bias frames at -10C collected in Maxim.
In PI, I set the probe to display values in 16 bit integer mode. Hopefully this allows comparison of “ADU” units with the
corresponding Maxim values.
The Maxim calibration involved using “Set Calibration” to load the various calibration individual subs into Maxim and clicking
the button that creates Masters for the groups of files (Bias, Dark, and Flat). I then opened the raw file and clicked “Calibrate”.
In the fight to get PI to calibrate the images, I tried several different cases as listed below. The various cases are variations on using floating point versions of the various calibration files for PI.
Note that only Case 4 adequately calibrated the images.
Case 1 – All files float
+ all files (bias, dark, flat, light) are exported in floating format from Maxim
+ FITS configured to rescale with 0-65535
Result: the PI conversion runs, but is not correct.
- Calibrated image is too dark, zeros in background, no nebulosity.
- flat is wrong; it looks a lot like the bias.
- values in the PI created calibration master files are different from Maxim.
- Bias ranges 40000..15000..40000 across frame.
Maxim => 19500..1900..1950 - Flat ranges 0..6000..0 across frame
Maxim => 15500..18000..15500 - Dark values around 1000 across frame
Maxim => around 300 across frame
Case 2 – Fits configuration at 32767
This was the same as Case 1, except that the FITS configuration is set to 0..32767 instead of 65535. This was tried in an attempt to address image values being off by a factor of 2.
Result: PI Conversion fails to run.
After PI calibrates the individual flats the integration process tries to stack the flats to make the master. This process fails because one of the calibrated flats is “open in another process” and cannot be opened. On different runs, it will be a different file that is open in another process.
Case 3 – Use Maxim calibration masters
It turns out the Maxim calibration masters are already in float format. This case uses those masters directly PI, rather than converting the individual subs to float for PI to create its own masters.
FITS is configured as 65535 again (for the light frame).
- Result: Calibration runs, but it looks like the flats are not correctly applied.
- Dust motes are visible.
- Darker center of image. Looks like flats are too strong?
- Values (flat, dark, bias) of the masters when opened in PI match those reported by Maxim.
Case 4 – Use original individual 16 bit Maxim calibration files
Another weirdness – it turns out the individual calibration subs created by Maxim are readable by PI, even though they are in the same 16-bit format which doesn’t work for the light frames. For unknown reasons the calibration files can be read.
So, in this case allow PI to use the original Maxim individual subs to build it’s masters. The only thing different between PI and Maxim processing in this case is PI starts with the float version of the light sub.
+ Calibration files (bias, dark, flat) are left in “16 bit” state as created by Maxim. No conversion performed.
+ Light file is exported in 32 bit float format. Can tell this is necessary because PI does not open this 16 bit file correctly.
+ FITS configured to rescale with 0..65536
Result: As seen below, the calibration looks pretty good, very similar to the Maxim result.
- Dark master displays around 2000 “ADU”, versus Maxim’s 300.
- Bias master is slightly lower than Maxim: 1870..1800..1870
- Flat master is similar to Maxim, 15800..18000..15800
- Bias has much larger values than dark. I don’t understand this. However, the same thing is true in Maxim, so OK.
Comparison of PI versus Maxim calibrated light sub
So, how does the case 4 calibration of the light frame differ between the Maxim process and the PI process? The image below shows the two images, where the left image is the PI result and the right image is the Maxim result (converted to float).
Visual results:
- The Maxim file is smaller, 2Mb versus 8Mb for Float form. The PI result is also an 8 Mb xisf file. I don’t know how Maxim got the file down to 2Mb, maybe that is a clue why PI can’t open it?
- PI result has more hot pixels.
- PI result has more noise (much grainier to my eye).
- Stars look the same (roundness, brightness) even when zoomed in significantly.
- Maxim result may have more dark pixels; a little hard to tell because of the increased noise in the PI image.
Tool Measurements
I used several PI tools to measure the noise and SNR for the converted images. Measurements for the Maxim result used the Maxim calibrated image converted to 32 bit float.
Statistics tool results:
Maxim K PI K
count (%) 99.75290 99.95980
count (px) 2116573 2120963
mean 0.0072616 0.0075407
median 0.0071210 0.0072508
stdDev 0.0068079 0.0073814
avgDev 0.0011528 0.0013096
MAD 0.0009753 0.0009751
minimum 0.0000006 0.0000476
maximum 0.9739980 0.9766412
- PI has more pixels?
- PI has slightly higher background (median)
- PI has larger deviations
Noise comparison script:
The NoiseEvaluation tool was used after performing a linear fit on the PI image to match it to the Maxim result.
PI = 8.6e-4
Maxim = 9.34e-4
- Maxim has slightly more noise? This does not jive with what I see visually.
SNR Calculation:
Calculated the SNR ratio as SNR ratio = MAD / Noise
PI = 1.13 Maxim = 1.044
- Again, the PI result seems to have a slightly better SNR ratio by 0.09, although visually I wouldn’t have thought so.
ContrastBackgroundNoiseRatio tool:
Parameters: Background percentile 100th cycle-spins 8
Maxim PI
Median 466.48 475.15
Contrast 61.31 68.67
Background Mean 466.41 481.45
Background Noise 66.63 77.95
CBNR 0.92 0.88
- This tool indicates the Maxim result has a better contrast (0.92 vs 0.88) and less noise (66.63 vs 77.95).
Conclusions:
Visually it seems clear the Maxim result is better. The various PI tools indicate that the two calibrations are fairly close; some tools indicate the Maxim image is better, some indicate the PI image is better. One might suspect I am not running the tools in the correct way:)
Case 5 – Case 4 plus Superbias and optimized dark frames
I turned on the optimize dark frames parameter (0.004) in the Darks frame screen. I also created 2 Superbias frames, one based on the Case 1 Bias master (from float format bias frames) and one based on the Case 4 bias master (based on master created from original Maxim bias subs).
This did not work. The background is over-reduced (lots of zero values). Somehow the superbias is not working correctly. Both superbias files had the same result.
I also tried only setting the Optimize Darks parameter = 0.004. This is the only difference between this frame and the Case4 result. It is fairly indistinguishable from the Case4 result; optimizing the darks didn’t seem to help any.
Thus, it seems that
- the superbias makes things worse, and
- optimizing dark frames does not help appreciably.
Final Conclusions
- The Maxim calibration appears to be at least as good as the PI calibration; I think it is better.
- There may be further fine tuning possible in PI to improve the results.
- Using PI for calibration requires a lot of manual operations compared to the almost automatic operation of Maxim. I see no reason to change to PI for calibration.