SBSKY: 174 reduced acm images
Updated: July 01, 2019

In May-June of 2019 I reduced 174 acm images. By this I mean the images were WCS- and ZP-calibrated. I made estimates of mean sky surface brightness for each umage and installed these in the hear. These data were collected with the routine ACM_ANALYSIS_TABLE. In retropsect, this script is rather klugy. In my early pipeline versions I was not retaining the HET PUPILLUM card. I had a special script to derive RSTRT (the combined tracker offset from X_STRT,Y_STRT). Finally, I had special script calls to derive the acm image type using various recipes unvolving header card values, yet a few of my final images were tageed as OPEN SKY images, yet that had EXPTIME values of just a few tenths of a second.

Maybe a smoother approach is to produce a list of the original image paths on mcs. This could be a standard output by survey_headers.sh.


I do this work in:   $scohtm/scocodes/finding_images/three_sets/acm/work1

I start with the file listing the galaxy images: 
% fitsfind /home/sco/acm_SBSKY > list.acm     
**** This make a list of the fullpath image names. 

I make a table file of images based on general FITS header information: 
% survey_headers.sh list.acm EX1 N 
*** This takes < 1 second for the 174 pfc images on sco2019 

Here is a sample of what I get: 
% cat EX1.parlab
im         image table number 
uthrs      UT time (floating point hours)  
fil        filter name  
source     image source  
isoobs     observation time in ISO8601 format  
object     object name    
imbase     FITS image basename  
% head -5 EX1.table
# data
    1   3.400306 B     acm      2018-02-06T03:24:01.18 none                 20180206T032412.5_acm_sci     
    2   5.557278 g`    acm      2019-02-04T05:33:26.26 none                 20190204T053337.2_acm_sci     
    3   6.356166 g`    acm      2019-02-04T06:21:22.29 none                 20190204T062133.3_acm_sci     
    4  10.662222 r`    acm      2018-02-06T10:39:44.03 none                 20180206T103955.3_acm_sci     

A useful file this routine makes is:    acm_mcs_paths.out
I can modify the paths in thsi file to process the original acm files 
on my home computer using path_replace: 

% path_replace acm_mcs_paths.out /hetdata/data/ /media/sco/DataDisk1/sco/AD/HET_work/acm_raw_subsets/ file.2 N 
% head -5 file.2 
/media/sco/DataDisk1/sco/AD/HET_work/acm_raw_subsets/20180206/acm/20180206T032412.5_acm_sci.fits
/media/sco/DataDisk1/sco/AD/HET_work/acm_raw_subsets/20190204/acm/20190204T053337.2_acm_sci.fits
/media/sco/DataDisk1/sco/AD/HET_work/acm_raw_subsets/20190204/acm/20190204T062133.3_acm_sci.fits
/media/sco/DataDisk1/sco/AD/HET_work/acm_raw_subsets/20180206/acm/20180206T103955.3_acm_sci.fits
/media/sco/DataDisk1/sco/AD/HET_work/acm_raw_subsets/20190427/acm/20190427T065924.2_acm_sci.fits

I can also make a second useful table if acm images are in the image list: 
% 1d2table ACMSUP.dat ACMSUP.params none EX2 N  
This makes the EX2 table:
% cat EX2.parlab
PUPUILLUM     PUPUILLUM 
EXPTIME     EXPTIME 
RSTRT     RSTRT 
imgtype     imgtype 
% head -5 EX2.table
# data 
 -99.00  5.00  1031.8  open 
 0.587689  7.00  1528.5  open 
 0.804511  7.00  791.5  open 
 -99.00  5.00  1055.3  open 


NOTE: The "none" indicates I do not wish to copy an image list file 
      to be the images file for our new (EX2) table file. 
 

I can now summarize the image types: 
% acm_table_qc EX2 N N
  0   0   174   0   (Nbias,Ndark,Nopen,Nnone)  
 
I can join EX1 and EX2 to make a single table:
% table_merge EX1 EX2 FULL N

Now I can make a lot more plots: 
  point_selector FULL im PUPUILLUM N

    
Hence, the fairly general survey_headers.sh recognizes acm image. If it can construct the original mcs image locations, then I can build a tool that gathers acm information in an eefficient (fast!) dedicated OTW code.


A comparative test

The new version of acm_table, named acm_table_markII, seems to run a lot faster. The acm_table table file has more data, but I rarely use. The markII varison uses a single, effecient OTW code to access only header information. There is one difference: the olde acm_table does prepare the ./local_red/BASIC files that are used to fill the revided FITS headers. I'll deal with this later. For now, I want a fats way to review the raw acm images and establish the viable sky images (and bias frames).


Both codes require (BaseDir,Date) files to identify images to be reduced. 
For example: 
% cat BaseDir
/media/sco/DataDisk1/sco/AD/HET_work/acm_nights
% cat Date
20190612

   To run:     
     % acm_table 10.0 N     
     % acm_table_markII N   

   Nims = total number of images 
   Ptime = processing time in seconds (to nearest second) 

   Code                 Machine   Disk    Nims  Ptime  Table_name    Nbias Ndark Nopen Nnone  Rate(images/sec)  
  acm_table             sco2019   scodat   523   21.0    ACM          5     10    508    0       24.90 
  acm_table_markII      sco2019   scodat   523    1.0   ACMDAT        5     10    283   225     523.00 

  acm_table             sco2019   ssd      523   21.0    ACM          5     10    508    0       24.90  
  acm_table_markII      sco2019   ssd      523    0.0   ACMDAT        5     10    283   225    >523.00

  acm_table             mcs       hetdata  523  112.0      ACM        5     10    508    0       46.69     
  acm_table_markII      mcs       hetdata  523   33.0   ACMDAT        5     10    283   225      15.85    
   

Note that the number of open and unidentified (none) image classes does change between the two codes. This is because I dicovered in the course of this upgrade that many images with very short (less that 1 second) expousre times were being classifed as open images. For practical purposes, the new markII code classifieis these image types as "none". I was surpised that my tests using the ssd (ssd = solid state drive) on sco2019 did not have a faster time compared to my scodat run (scodat is an HDD = hard disk drive). I ran these tests a number of times to check for mistakes, but no change was evident.




Back to SCO CODES page