Here is a verbose end-to-end description of how I reduce a night of acm data. As with a lot of my webdocs, the lines that are in RED are the actual command line strings you issue to perform the jobs. The rest is just Faulkner-esque prose to make things clear.
CODE VERSION USED: /home/sco/Installs/install_sco_20180522/codes
Reduction notes for 20180403
Did this work on scohome in: /home/sco/ACM_Red1/20180403 (May30,2018)
Here I start with a copy of my 20180402 reduction notes. I will refine the end2end description.
% wheredata Y
/home/sco/HET_work/acm_nights
20180403
% acm_list none none
Using local BaseDir and Date files.
Survey images for: 20180403
15 likely bias images listed in the file list.BIAS
15 likely dark images listed in the file list.DARK
103 likely SHUTTER-OPEN images listed in the file list.OPEN
*** All sky images are g, but we use exposure times from 3 to 10sec.
I delete the firts 13 0.1sec images. So, 90 OPEN images.
I also delete some repates, tec... Ended up with 64 images.
In ./Bias
% acm_basic_list list.BIAS 4
Number of images = 15
Mean signal = 1382.10667
Median signal = 1382.200000
mean error = 0.037118
*** All images look okay in bigds9 review
% cp list.BIAS list.BIASF
% process_acm_bias list.BIASF 5 MEAN N
% cp 20180403T062502.6_acm_sci_BIAS_MEAN.fits BM20180403.fits
In ./Dark
% list_split_exptime list.DARK DDD N
Line count results:
15 DDD.5sec
*** All darks were 5sec.
Make darkrate from 5sec images:
% process_acm_darkrate DDD.5sec 5 5sec ../Bias/BM20180403.fits N
ID string (usually UT date) = 5sec
Number of acm dark images = 15
Mean dark level in stacked image = 0.27961 adu/sec
sigma (error for single dark) = 0.03435 adu/sec
*** This was a good, low darkrate.
% cp 20180403T062908.4_acm_sci_DARKRATE_5sec.fits DR20180403.fits
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Process the list.OPEN images in SkyImages
% cd ./SkyImages
% acm_ccd_correct list.OPEN ../Bias/BM20180403.fits ../Dark/DR20180403.fits
% ls -1 /home/sco/ACM_Red1/20180403/SkyImages/*proc.fits > list.1
*** This is 64 processed images. As this is a fairly large number to process manually, I
am going to go through the set in bigds9 and just pick out a working set of the best
images. I'll choose only the ones with lots of stars and no obvious bright stars or
sources of light that will complicate the sky surface brightness derivations.
I make a working list and run it with bigds9:
% ls -1 /home/sco/ACM_Red1/20180403/SkyImages/201*proc.fits > list.1
% bigds9 list.1 9 9
% mv big.MARK list.FINAL
% cp list.FINAL ../S
*** These are all g images ======> 24 images
% gethead @list.FINAL FILTER EXPTIME
20180403T025832.4_acm_sci_proc.fits g` 8
20180403T032508.4_acm_sci_proc.fits g` 5
20180403T040056.0_acm_sci_proc.fits g` 5
20180403T042947.5_acm_sci_proc.fits g` 6
20180403T052406.0_acm_sci_proc.fits g` 6
20180403T053306.8_acm_sci_proc.fits g` 6
20180403T055938.1_acm_sci_proc.fits g` 6
20180403T060639.2_acm_sci_proc.fits g` 6
20180403T061909.5_acm_sci_proc.fits g` 6
20180403T070737.8_acm_sci_proc.fits g` 6
20180403T072125.5_acm_sci_proc.fits g` 6
20180403T075403.4_acm_sci_proc.fits g` 10
20180403T080928.0_acm_sci_proc.fits g` 10
20180403T081806.8_acm_sci_proc.fits g` 15
20180403T100319.0_acm_sci_proc.fits g` 3
20180403T101951.8_acm_sci_proc.fits g` 3
20180403T102448.4_acm_sci_proc.fits g` 3
20180403T103653.9_acm_sci_proc.fits g` 3
20180403T104226.6_acm_sci_proc.fits g` 3
20180403T105534.9_acm_sci_proc.fits g` 3
20180403T110129.4_acm_sci_proc.fits g` 3
20180403T110601.3_acm_sci_proc.fits g` 3
20180403T111010.1_acm_sci_proc.fits g` 3
20180403T111018.2_acm_sci_proc.fits g` 3
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
For astrometry I run all these steps in /home/sco/ACM_Red1/20180403
% pwd
/home/sco/ACM_Red1/20180403
% mkdir local_red
***** Step 1: Run the wcs_rough code ******
% wcsf_rough ./S/list.FINAL N
Would you also like to run in verbose mode? (Y/N): N
Perform visual inspection with usno_identify? (Y/N): Y
*** About 3-4 of the inintial images had nonsense initila wcs. I could not
ID the fields and hence could not derive a valide wcs.
***** Step 2: Run the wcs_viscat code ******
% wcsf_viscat ./S/list.FINAL N
Would you also like to run in verbose mode? (Y/N): N
Skip the use of wcs_refine? (Y/N): Y
**** Basically, I just hit a return for all the images that look good (i.e.
have stars that line up with the catlog sources).
***** Step 3: Make cdfp catalogs with rough astrometry ******
# Now I make a list of the local images (full path) and run the wcsf_imgcat code
# We have to make the new list becasue we need to use the images with rough WCS in their headers.
% pwd
/home/sco/ACM_Red1/20180403
% ls -1 /home/sco/ACM_Red1/20180403/20*proc*fits > list.2 # 31 images
% cp list.2 ./S
% wcsf_imgcat ./S/list.2 N
Would you also like to run in verbose mode? (Y/N): N
**** This takes a few seconds per image, then all support files, including
the roughly wcs-calibrated FITS images, are in ./local_red/IMGCAT0/
***** Step 4: Run the wcs_ code ******
# Compute the final WCS headers
% wcsf_final ./S/list.2 N
Would you also like to run in verbose mode? (Y/N): N
**** I get a review image, but the bad point flagging is done
automatically. The image is just a confirmation that the
starting wcs is approximately correct. Just enter returns.
For the final fits, I select 2lin or trs fits. With a trs
fit, use "X" flip.
*** When finished:
% gethead ./local_red/WCS/20*fits RMSRA RMSDEC NUMWCS FILTER EXPTIME
20180403T042947.5_acm_sci_proc.fits 0.3930 0.4500 17 g` 6
20180403T052406.0_acm_sci_proc.fits 0.3840 0.4030 10 g` 6
20180403T053306.8_acm_sci_proc.fits 0.2590 0.1650 6 g` 6
20180403T055938.1_acm_sci_proc.fits 0.1510 0.2330 9 g` 6
20180403T060639.2_acm_sci_proc.fits 0.3120 0.1580 5 g` 6
20180403T061909.5_acm_sci_proc.fits 0.2870 0.0880 8 g` 6
20180403T070737.8_acm_sci_proc.fits 0.4860 0.4940 4 g` 6
20180403T072125.5_acm_sci_proc.fits 0.3000 0.2050 6 g` 6
20180403T075403.4_acm_sci_proc.fits 0.5300 0.2440 4 g` 10
20180403T080928.0_acm_sci_proc.fits 0.1630 0.2430 8 g` 10
20180403T081806.8_acm_sci_proc.fits 0.2340 0.4100 5 g` 15
20180403T100319.0_acm_sci_proc.fits 0.8050 1.3640 11 g` 3
20180403T101951.8_acm_sci_proc.fits 0.3620 0.4930 30 g` 3
20180403T102448.4_acm_sci_proc.fits 0.1880 0.4210 21 g` 3
20180403T103653.9_acm_sci_proc.fits 1.1900 1.3190 118 g` 3
20180403T104226.6_acm_sci_proc.fits 0.3870 0.3060 76 g` 3
20180403T105534.9_acm_sci_proc.fits 0.3120 0.3040 40 g` 3
20180403T110129.4_acm_sci_proc.fits 0.8940 1.5820 72 g` 3
20180403T110601.3_acm_sci_proc.fits 0.2770 0.3840 32 g` 3
20180403T111010.1_acm_sci_proc.fits 0.3300 0.3930 75 g` 3
20180403T111018.2_acm_sci_proc.fits 0.4630 0.3930 54 g` 3
*** 21 images
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Photometric calibration
Step 1: Make a list of wcs-calibrated images
% pwd
/home/sco/ACM_Red1/20180403
% ls -1 /home/sco/ACM_Red1/20180403/local_red/WCS/20*fits > list.zp1 # 21 images
% cp list.zp1 ./S
Step 2: Make first pass for PS1 data - identify the stars
% photcal_fitslist ./S/list.zp1 ps1 N
*** Folow instructions to get PS1 data
% wc -l ./local_red/ZPTAB/griBVR.cdfp
368 ./local_red/ZPTAB/griBVR.cdfp
Step 3: Make a final pass to derive the ZP values
% photcal_fitslist ./S/list.zp1 ps1 N
*** After this, the ZP calibrated images are in:
% ls local_red/ZP
Make a tbale of ZP
gethead ./local_red/ZP/*.fits FILTER PSYSNAME ZPSEC ZPERR NUMZP > Phot.Table
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Measure sky boxes
% pwd
/home/sco/ACM_Red1/20180403
% mkdir SkyBoxes
% cd SkyBoxes
% ls -1 /home/sco/ACM_Red1/20180403/local_red/ZP/20*fits > list.0
% mkdir S
% cp list.0 S
% mkdir ./local_red
# measure the boxes interactively
% ds9_fitslist ./S/list.0 N # Measure boxes
When the boxes have been measured.:
% cd ./local_red/CAT
% ls -1 20*fits > RUN_TAB1
% cat RUN_TAB1
#
acm_skysb_stats 20180403T042947.5_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T052406.0_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T053306.8_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T055938.1_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T060639.2_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T061909.5_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T070737.8_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T072125.5_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T075403.4_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T080928.0_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T081806.8_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T100319.0_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T101951.8_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T102448.4_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T103653.9_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T104226.6_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T105534.9_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T110129.4_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T110601.3_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T111010.1_acm_sci_proc.fits Y N
acm_skysb_stats 20180403T111018.2_acm_sci_proc.fits Y N
% chmod 777 RUN_TAB1
% RUN_TAB1
Assemble the finale results file: Readme.Matt_20180403
Note: I edit the skysb values into singlevalue files, then
% calstats.py -v list.g
Simple stats for numbers in: list.g
Mean = 19.01719
Median = 19.26800
Standard deviation = 0.53085
Minimum = 17.63200
Maximum = 19.57400
Number of values = 21
Mean error of then mean = 0.11870
% cat Readme.Matt_20180403
Matt:
Here are my results for 20180403. Format is the same as other summaries.
For this nearly full-moon night we get:
= 19.07 -+0.12 mag per sq.arcsec 21 images
Night Comments:
A valuable night: 20180403 Moon rise at 3:40UT
Pointing test nights where I took 30-60 stars each night almost all in g. I
also took bias/dark data sets (which appear to be at normal levels). Both
nights had a very bright moon a day or two after full moon. Clear in early
part of the night with some clouds around 6:00UT. Heavy clouds and high
wind after 8:23UT.
Table 1: Sky surface brightness measurements
Image = basename of the acm image
band = photometric system transformed to
exp = image exposure time in seconds
skySB = calibrated sky surface brightness in mags per sq.arcsecond
m.e. = mean error of skySB
naps = number of apertures (usually BOX regions)
Image band exp skySB m.e. naps
------------------------------ ---- --- ----- ----- ----
20180403T042947.5_acm_sci_proc g 6 19.574 0.002 4
20180403T052406.0_acm_sci_proc g 6 19.529 0.003 5
20180403T053306.8_acm_sci_proc g 6 19.490 0.002 5
20180403T055938.1_acm_sci_proc g 6 19.135 0.003 4
20180403T060639.2_acm_sci_proc g 6 19.312 0.003 6
20180403T061909.5_acm_sci_proc g 6 19.141 0.004 5
20180403T070737.8_acm_sci_proc g 6 18.186 0.005 5
20180403T072125.5_acm_sci_proc g 6 18.042 0.004 4
20180403T075403.4_acm_sci_proc g 10 18.678 0.004 6
20180403T080928.0_acm_sci_proc g 10 18.401 0.005 4
20180403T081806.8_acm_sci_proc g 15 17.632 0.002 4
20180403T100319.0_acm_sci_proc g 3 19.352 0.005 5
20180403T101951.8_acm_sci_proc g 3 18.820 0.005 5
20180403T102448.4_acm_sci_proc g 3 19.451 0.003 5
20180403T103653.9_acm_sci_proc g 3 18.790 0.003 5
20180403T104226.6_acm_sci_proc g 3 19.268 0.003 6
20180403T105534.9_acm_sci_proc g 3 19.492 0.002 5
20180403T110129.4_acm_sci_proc g 3 19.139 0.002 7
20180403T110601.3_acm_sci_proc g 3 19.319 0.003 5
20180403T111010.1_acm_sci_proc g 3 19.314 0.005 8
20180403T111018.2_acm_sci_proc g 3 19.296 0.006 5
Table 2: Photometric ZP data
F = FILTER name in the image header
S = standard photometric syystem transformed to
ZPSEC = ZP for a 1-sec exposure
err = mean error of ZPSEC
N = Number of points used to derive the calibration
Image Name F S ZPSEC err N
20180403T042947.5_acm_sci_proc.fits g` g -2.33073 0.023559 11
20180403T052406.0_acm_sci_proc.fits g` g -2.15475 0.052093 4
20180403T053306.8_acm_sci_proc.fits g` g -2.82125 0.007728 4
20180403T055938.1_acm_sci_proc.fits g` g -2.50720 0.010017 5
20180403T060639.2_acm_sci_proc.fits g` g -2.83325 0.059889 4
20180403T061909.5_acm_sci_proc.fits g` g -2.98433 0.026541 6
20180403T070737.8_acm_sci_proc.fits g` g -2.80633 0.007688 3
20180403T072125.5_acm_sci_proc.fits g` g -2.20200 0.062003 3
20180403T075403.4_acm_sci_proc.fits g` g -2.50467 0.007172 3
20180403T080928.0_acm_sci_proc.fits g` g -2.78640 0.048599 5
20180403T081806.8_acm_sci_proc.fits g` g -3.59125 0.029061 4
20180403T100319.0_acm_sci_proc.fits g` g -2.25160 0.007903 5
20180403T101951.8_acm_sci_proc.fits g` g -2.97700 0.020522 22
20180403T102448.4_acm_sci_proc.fits g` g -2.19206 0.029535 17
20180403T103653.9_acm_sci_proc.fits g` g -3.21756 0.024320 34
20180403T104226.6_acm_sci_proc.fits g` g -2.38932 0.028266 19
20180403T105534.9_acm_sci_proc.fits g` g -2.73883 0.009231 18
20180403T110129.4_acm_sci_proc.fits g` g -2.66081 0.032489 16
20180403T110601.3_acm_sci_proc.fits g` g -2.22053 0.019789 15
20180403T111010.1_acm_sci_proc.fits g` g -2.72588 0.026171 26
20180403T111018.2_acm_sci_proc.fits g` g -2.73713 0.017703 23