Here are some comments out of order from your questions and comments.
The extra test data you generated has a lot of value. It is always good to have others duplicate an experiment to make sure the first person (me) wasn't just generating garbage. Thanks for the time you put into this.
I think your conclusions after running the experiment match my own. Recall, that you don't have to shoot Bias Frames at all unless you may want to scale your Dark Frames for some other reason. So long as you shoot for Flats longer than about 3 seconds, your Flat-Darks should be all you need for calibration. Just staying out of the exposure zone of less than 3 seconds should suffice.
Regarding getting the data out of PixInsight more easily, it is drop dead easy. Try this procedure:
- Open the Blink Process in PixInsight
- Load up all of your experiment frames into Blink. You can load Bias, Dark, Flat, and even Light frames all at once if you wish. It makes everything available for analysis.
- Select the first Frame in the Blink Process file list.
- Use Ctrl-A to select all the rest of the files.
- Click the next to last icon at the bottom of the Blink window. (Series Analysis Report) This will output the basics but you should also output some additional data while you are at it. It can allow you to analyse many other aspects of the data.
- In the "Statistics" window that opens from the Blink Process, select the following additional data (using a check mark in the box) to output in the report:
OBJECT (This is the target name from SGP's sequence)
DATE-OBS (UTC time of frame capture)
IMAGETYP (Image Type -- Dark, Flat, etc)
EXPOSURE (Exposure time -- this will be repeated in the report. some PI processes set the other to 0)
CCD-TEMP (The actual temperature reported for the frame at time of capture)
SET-TEMP (The set-point temperature you requested. This can and often is different than above)
GAIN (The gain setting from the driver)
EGAIN (The electron gain assumed by the driver)
- After selecting the above extra data, check the Write Text File Box
- Select the output folder for the report. It will always be called "Statistics.txt"
- Press OK to generate the report.
At that point, blink will grab all the data from every file and put in the Statistics.txt report file. Now you can load all that into a spreadsheet. I use LibreOffice Calc but you can use Excel or OpenOffice or most any other spreadsheet. To look at and plot the data, I use this method:
- Open the spreadsheet.
- Use File | Open to open the statistics.txt file.
- You will get the import dialog. I just take the defaults and press OK.
- You will then get a listing of all the statistics and extra information with each frame on a separate line and all the data lined up in proper columns.
- I do the analysis using a pivot table. It is the easiest, quickest method to get a summary of all the data.
- Select all the data on the sheet -- just click in the upper left corner for all lines and columns
- Select Data | Pivot Table | Create from the menus
- Use "Current Selection" (the default) in the Select Source dialog that pops up.
- You now get the Pivot Table Layout dialog
- Fill out the dialog as described below.
- Click twice on the highlighted "Data" designation in the Column Fields area and press delete to remove it.
- In the Available Fields area, drag the IMAGETYP data component from the list into the Page Fields area at the top and drop it there.
- In the Available Fields area, drag the SET-TEMP data component from the list into the Columns Fields area and drop it there.
- Drag the EXPOSURE data component from Available Fields to the Row Fields area and drop it there.
- Drag the Mean data component from the Available Fields area to the Data Fields area and drop it there.
- The Mean that you just dropped will now read "Sum - Mean". You will need to change that next.
- Double click on the Sum - Mean data component and the Field Data dialog will pop up.
- Select "Average" and press OK. The Data Fields area will now read "Average - Mean"
- Expand the "Options" area of the Pivot Table Layout dialog.
- Un-check the Total Rows and Total Columns check-boxes.
- Press OK and the pivot table will be created in a new sheet. You can create it in a specific area but for this quick intro having it on a new sheet is just as good.
The table will be created on a new sheet. It will contain a selector called IMAGETYP at the top where you can select just the types you want to see. For this exercise, use just the BIAS and DARK types. Next there will be a table which contains the table tile Average - Mean and a selector for SET-TEMP. The SET-TEMP will only contain a single entry if all the frames has the same Set-Point. Next down, there will be a selector for EXPOSURE. Make sure all are selected. The first data column has all the different exposure times for all frames of the selected type. The second column has the Average of the Mean ADU for all frames of the selected types at the corresponding exposure time.
The two data columns (Exposure and Mean ADU) can be plotted to give a result just like what you posted.
I find it interesting that your second set of exposures with delay looks as it does. I am not totally surprised, though. If I might suggest an experiment for you try next time you have spare time, it would be to repeat that second part of the test but put dummy Bias Frames between the Dark Frame exposures and then have SGP rotate through the events. I think you may find that the lowered value of the mean ADU will rise back and begin to look like the first plot of frames without a delay in between.
I suggest this because that data is reminiscent of what I was seeing as inconsistent results in my own runs. Those results cleaned up for me when I placed Bias Frame captures between the Dark Frames.
Another experiment to try is the run the the low exposure times in reverse order. In other words, instead of running the frames as 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0, try them as 0.0, 1.0, 0.8, 0.6, 0.4, and 0.2. I expect you will get the same plot as your first one (in red). If so, it says that the steep slope is not due to cumulative self warming of the chip but is a result of the active timing circuitry on the chip.
[EDIT] I say this because my original plot which looks like your No_Delay plot was actually run with a 10 second delay between all exposures. Nearly all of my data had the 10 second delay built into the SGP sequence.
Anyway, thanks so much for going to the trouble of running this data through your own camera. It really does help to see what others are getting and bolsters my opinion that what I see from my camera is not unique.
Edited by jdupton, 19 October 2018 - 07:18 AM.