Gathering and using image calibration material.
Last updated: Feb 25,2020
We often need a general way to collect material for calibrating the astrometric and phototometric
properties of images. Simply put, we want to know the sky positions (Ra,Dec) and magnitudes of
of some sources that are visible on our image of interest (our "target image"). When thing flow
smoothly, this will really be all we need. In the real world we'd like some backup, and in this
context I mean a standard picture of some part of the sky that overlaps with our target image. For
many applications, the online DSS (Digital Sky Survey) images are ideal.
This originally was part of a document discussing the automated reduction of HET acm images.
By the way, acm images are also known acquition camera, acam, acqcam, and other things
that different folks use. I use "acm", because like the images obtained with other HET
devices (gc1,gc2,bib,etcs...) this is the acronym that appears in the name of every acm
image file (e.g. 20191018T024545.3_acm_sci.fits). In the couse assembling an acm pipeline,
it became clear that the assembly of sitable calibration material often presented a
stumbling point. This is really a very general topic and hence I have moved it here to
a stand-alone document. Most of the tools dicussed here deal with the batch retrieval for
of data over the internet. These processes can be slow, especially if retieving data
many different fields (say, 20 or 30 acm fields from a single night). Hence, these tools are
designed to require no user interaction beyond invoking the initial request. In this current
suite of tools, the stored data sets (DSS imagse and cdfp files from the USNO-B1.0 and PS1
catalogs) are stored in a local subdirectory (./local_red/Store_DSS/). Hence, we must have
the routines that allow us to review and acces these data sets during any pipleine process.
Here is a link to a set of early Jan2020 working notes.
- Test images and basic tools.
- Batch data gathering.
- Review the collected data sets.
- Gathering simle catalogs for the input images.
- Appendix A: The basic command list.
Test images and basic tools.
I have collected test data in the "AUTO_match_for_wcs" directory. The "SAVE" subdirectory there
has files for making test and demo runs on different machines that I regularly use.
Machines I use a lot:
Name Properties linux kernel (uname -srm)
--------- -------------------------------------------- -------------------------------------------
sco2019 my home machine running Ubuntu 18.04 Linux 2.6.32-754.24.2.el6.x86_64 x86_64
mcs RA machine (HET control room) running RedHat6 Linux 4.15.0-72-generic x86_64
steveo my laptop running Ubuntu 16.04 Linux 4.4.0-171-generic x86_64
Directory locations:
sco2019 /home/sco/AUTO_match_for_wcs
mcs /home/mcs/sco/AUTO_match_for_wcs
steveo /home/sco/AUTO_match_for_wcs
Hence , if I am running on sco2019 in ./tmp:
% mkdir tmp
% cp /home/sco/AUTO_match_for_wcs/S/list.0_sco2019 list.0
% cat list.0
/home/sco/AUTO_match_for_wcs/S/Rsco2043.fits
/home/sco/AUTO_match_for_wcs/S/Rsco3226.fits
/home/sco/AUTO_match_for_wcs/S/Rsco5261.fits
/home/sco/AUTO_match_for_wcs/S/20191018T024545.3_acm_sci.fits
I can now view these images using a call to bigds9:
% bigds9 list.0 1 10
If I wanted local copies of these images, I could do this:
% impuller list.0
I used the bigds9 call in the last step above to make the figure shown below.
 |
|
The 4 test images provide in ./AUTO_match_for_wcs.
In the upper-left is the NGC 3379 field (Rsco2043.fits) which has 3 large galaxies,
a moderate density of stars and no WCS present in the image FITS header.
In the upper-right we have the NGC 7479 field (Rsco3226.fits) which does
have WCS insalled in the header with moderate quality.
In the lower-left is the NGC 6946 field (Rsco5261.fits) which has a very large galaxy,
a very high density of stars and no WCS present in the image FITS header.
In the lower-right is a slightly processed (fbp-corrected) acm image (20191018T024545.3_acm_sci.fits)
that has the usual poorly calibrated WCS information in the image FITS header.
|
These images represent a number of extreme cases and hence are
useful for testing various matching and calibration routines. We have
wide area images with lots of star and few stars. We have the case of a
smaller are on the sky with few stars.
Return to top of page.
Batch data gathering.
To calibrate the photometric zeropoint and install a viable WCS for an image we need
catalogs, and it is often useful to have a standard image of the cataloged area.
In this section we are primarily concerned with gathering material for each FITS image
listed in an input file (list.0 in the case shown above). This process will be conducted
in an (unsupervised) batch mode. This means slow internet-dependent processes can be be
run prior to reduction steps that require human interaction. If we have file listing
our images of interest (list.0), we can gather calibration data using the following steps:
% fitslists list.0
% dssbase_run_markII list.for_mode2 none none none none list N
The first routine (fitslists) uses header information
to determine the size and depth for gathering calibaration material for each image in the list.
The last routine above, dssbase_run_markII has
two modes of operation. We have used the more comprehensive mode (mode2) here that uses a file
(list.for_mode2) that speciifes image size and photometry limits for each image individually. The
file (list.for_mode2) was built in the fitslists run.
Return to top of page.
Review the collected data sets.
Once our caliration gathering routines have been run, then we need tools that
enable access to the stored data sets.
% dssbase_list Y
% dssbase_locate 10:47:49.90 +12:34:57.0 5.0
% cat Best.DSS
dss_RA10c47c49p989DEC+12c34c55p92_1426
To view this image and confirm that we have indeed located the appropriate dataset:
% ds9 ./local_red/Store_DSS/dss_RA10c47c49p989DEC+12c34c55p92_1426.fits
*** As an aside, if you want to get the Ra,Dec of NGC 3379:
% rc3gal n3379
Results of the RC3 search:
Name_RC3 RA_2000 DEC_2000 Type Input_name
----------- ----------- ---------- ------ ----------
NGC 3379 10:47:49.90 +12:34:57.0 .E.1... n3379
dssget 161.957916 12.582500 10.0 dss poss1_red
RAdeg,DECdeg,l,b: 161.9579163 12.5824995 233.490 57.630
Btot,DIA(armin),b/a,V: 10.24 5.37 0.891 793.0
T, B-Vt, U-Bt : -5.00 0.96 0.53
Other names:
NGC 3379
UGC 5902
PGC 32256
In our ds9 run above we should see an image of NGC3379. The basename of that image (the
part of the name exclding the full path and the ".fits" suffix) is "dss_RA10c47c49p989DEC+12c34c55p92_1426"
and this encodes the Ra,Dec of the image center as well as the size of the image. This same basename
is also used to names the files in Store_DSS that contain the USNO-B1.0 and PS1 data catalogs. Of course.
using the ds9 run above is not extremely practical, but the point here is with the proper basename
identified, downstream scripts can use that string to properly acces the image and catalog data
for this field.
A more practical way to review the dss image sets is to use:
% dssbase_review Y
Below are two of the dss sets gathered for two of our test images. These
are the types of images we can view in a call to
dssbase_review.
 |
|
The DSS image collected for the NGC7479 image is shown above
with the USNO sourec overplotted in RED and the PS1 sources
overplotted in GREEN. The default sky areas pulled for each image
can be changed in the fitslists code. In this case, beacuse the
PFC pointing information can be quite bad, we have gathered USNO
targets over a wider sky area. This is to insure that enough USNO
targets, our main astrometric sources, will overlap with the PFC
image of NGC7479. Note that this dss set was collected for the image in the
upper-right panel of the top figure on this page.
|
 |
|
The DSS image collected for the acm image is shown above
with the USNO sourec overplotted in RED and the PS1 sources
overplotted in GREEN. We have zoomed in here to show the
that the aperatures are sized approximately to scale with magnitude.
Note that this dss set was collected for the image in the
lower-right panel of the top figure on this page.
|
Return to top of page.
Gathering simple catalogs for the input images.
We've essentially gathered all of the calibration material in a batch process.
Another thing that is useful for automating, or at least streamlining, the
calibration process is to derive source catalogs for the input images using a batch
process. We can do this using the routine image_catalogs.
% image_catalogs list.0 Set0 Y N
arg1 - Name of file with list of input FITS images
arg2 - output directory name appendix (can be "none")
arg3 - Clean all but the table files (Y/N)
arg4 - run in debug mode (Y/N)
The image catalogs will be stored in ./locat_red/IMAGE_CATS_Set0
Note that even on a fast machine like sco2019 the time to process our
four test images is fairly long (42 seconds). This is becasue the first
three test images are PFC images with large numbers of stars. However, this
is precisely why we run this step as a batch job.
Return to top of page.
Appendix A: The basic command list.
Here we list the sequence of commands used to gather and review calibration data
for our four test images. The test images are listed in the "list.0" input file.
% fitslists list.0
% dssbase_run_markII list.for_mode2 none none none none list N
% dssbase_list Y
% dssbase_review Y # Optional
% image_catalogs list.0 none N N
% image_catalogs_review list.0 # Optional
Return to top of page.
Back to SCO code page