ZP runs using PanSTARRS
  • How to build a list of Ra,Dec for the PS1 webtool.
  • How to query PS1.
  • Build a cdfp file with the gri data.
  • Cross-match catalogs and compute ZP values.
  • Appendix 1: An early manual query for PS1 data.


    • How to build a list of Ra,Dec for the PS1 webtool.

    Here I use the 14 wcs-calibrated images described in previous sections to calibrate photometric zeropoints using PanSTARRS gri photometry. 

    
# Here are the images 
ls -1 /home/sco/A_wcsf/Run_apr17/local_red/WCS/20*fits > list.ZP0
The 14 images are in list.ZP0

Actually, an easier run for demo purposes involves only 1 image file: 
% pwd
/home/sco/A_wcsf/ZP_may02
% cat S/list.3
/home/sco/A_wcsf/Run_apr17/local_red/WCS/20180206T070902.2_acm_sci_proc.fits

To process the image for ZP derivation: 
% usno_photcal_fitslist ./S/list.3 N 

*** At the end of the run: 
Choice for X param:  BmR Peak AVap None
Enter desired X param: None 

Fits base name = 20180206T070902.2_acm_sci_proc
Enter any key to continue:
Ready to build instrumental mag cdfp. 
Send this ZP table (table,archive) archive? (Y/N): Y

Local image is:  20180206T070902.2_acm_sci_proc.fits 

Send this image to the final ZP archive? (Y/N): N

***** The following files show up in /home/sco/A_wcsf/ZP_may02/local_red/ZPTAB
20180206T070902.2_acm_sci_proc.cdfp  20180206T070902.2_acm_sci_proc-usno.cdfp  20180206T070902.2_acm_sci_proc_zp.table
20180206T070902.2_acm_sci_proc.rdxy  20180206T070902.2_acm_sci_proc_zp.parlab

Brief of important files descriptions: 
20180206T070902.2_acm_sci_proc.cdfp 
  Contains the mido-derived apparent magnitudes using the apertures drawn when 
  you ran usno_photcal_fitslist. These magnitudes are the same as those in Column 11 
  of the *.table file. The magitude is:    

     magi = 30 - 2.5*alog10(F) + 2.5*alog10(EXPTIME) 
  where F is the total sky-subtracted flux in the aperture. 

Note also that Column5 in of the *.table file is the first part of this magnitude: 

     magi = 30 - 2.5*alog10(F)  
  
20180206T070902.2_acm_sci_proc.rdxy:
  Conatins the Ra,Dec,equinox,X,Y values of the final list of starss in the *.cdfp file 
  above. These Ra,Dec values will be used to search the PS1 database for gri magnitudes.
    Recall that we can use the "xyplotter_auto" routine and the *.table,parlab files to plot all sorts of photometric data. Otherwise, the table file is not used further. The *.cdfp file will be used in he final ZP derivation, and the Ra,Dec data in the *.rdxy files will be collected by make_panstars_file to creat a file for sending to the PS1 webtool. It should be notes that we run this next procedure is run in the top reduction diretory (i.e. where the local_red subdirectory is located). 
    
% pwd
/home/sco/A_wcsf/ZP_may02

% make_panstars_file N 
 
Use the file find.PanSTARRS in fuf mode in  https://archive.stsci.edu/panstarrs/search.php?form=fuf

% cat find.PanSTARRS
10 55 03.955,+05 27 36.01
10 55 08.133,+05 24 43.64
10 55 04.242,+05 27 30.51
10 55 04.437,+05 26 28.09
10 55 07.696,+05 26 17.18
10 55 07.653,+05 27 25.83
10 55 11.184,+05 26 09.82
10 55 11.856,+05 26 17.26
    The "make_panstars_file" script constructs the "find.PanSTARRS" file. It gives you a reminder message about how the file will be used ("Use the file find.PanSTARRS in fuf ...."). I discuss this step in the next section.



    How to query PS1.

    PS1 refers to the first data release of PanSTARRS. I will send the file "find.PanSTARRS" to a PS1 webtool to collect available gri photometry in the aperture positions marked in the previous section. The figure below gives the URL for this PS1 tool. I show some typical seetings. It is my goal to eventually find a puely command-line driven method for this step.



    The query page I used to pull data in a find.PanSTARRS file. The returned file is shown below. 
    
URL for this page is: https://archive.stsci.edu/panstarrs/search.php?form=fuf
I used the Browse button to navigate to my "find.PanSTARRS" file. 

Important points to note: 
  - use "Suppress null result message" 
  - use degree ra,dec
  - use space spearated values on output 
  - use search radius (per star) of 0.0333' (2") 

After I hit the "Search" button I usually have to wait a few seconds before 
a downloads pop-up box appears.  I can retrieve the retruned PS1 data file 
with a command like: 
% mv ~sco/Downloads/panstarrs_search.txt panstarrs_search.list  
% cat panstarrs_search.list 
raMean decMean gMeanPSFMag gMeanPSFMagErr rMeanPSFMag rMeanPSFMagErr iMeanPSFMag iMeanPSFMagErr
ra dec float float float float float float
163.7663291 5.4600063 17.349 0.005 16.139 0.005 15.100 0.002
163.7837866 5.4121244 14.138 0.005 13.822 0.004 13.749 0.001
163.7675708 5.4584708 17.430 0.006 17.542 0.006 17.716 0.004
163.7684212 5.4411560 18.733 0.009 17.988 0.005 17.705 0.003
163.7817986 5.4379801 18.826 0.005 17.946 0.004 17.582 0.003
163.7818405 5.4569644 19.953 0.011 18.757 0.005 17.669 0.006
163.7963309 5.4358980 19.543 0.010 18.362 0.005 17.770 0.004
163.7991610 5.4381320 18.665 0.010 17.513 0.005 16.686 0.002

    The next step will be to take the above PanSTARRS file (panstarrs_search.list) and create a cdfp file. Then I can use this to cross-match to my image.cdfp files stored in the ./local_red/ZPTAB storgae directory. I have a routine for positionally cross-matching two cdfp files (cdfpmatI.sh is a pretty fast OTW code).



    Build a cdfp file with the gri data.

    Using a returned PS1 data like that shown in the previous section we build a cdfp file that be used as our source of standard star photometry. A major step here is tranforming the PS1 gri to BVR. The transformation sets I selected were derived specifically for PS1 and are from Kostov and Bonev 2017 astro-ph 1706.06147v2. I also collected a boatload of transformations from the SDSS web pages . 

    
% cdfp_panstarrs.sh --help 
cdfp_panstarrs.sh:
  Transform gri from the PS1 webtool to a cdfp file. 
We use published transformations (see code) to predict BVR values. 

EXAMPLE: 
% cdfp_panstarrs.sh panstarrs_search.list 

The input file derived from the PS1 webtool run is: 
% cat panstarrs_search.list
raMean decMean gMeanPSFMag gMeanPSFMagErr rMeanPSFMag rMeanPSFMagErr iMeanPSFMag iMeanPSFMagErr
ra dec float float float float float float
163.7663291 5.4600063 17.349 0.005 16.139 0.005 15.100 0.002
163.7837866 5.4121244 14.138 0.005 13.822 0.004 13.749 0.001
163.7675708 5.4584708 17.430 0.006 17.542 0.006 17.716 0.004
163.7684212 5.4411560 18.733 0.009 17.988 0.005 17.705 0.003
163.7817986 5.4379801 18.826 0.005 17.946 0.004 17.582 0.003
163.7818405 5.4569644 19.953 0.011 18.757 0.005 17.669 0.006
163.7963309 5.4358980 19.543 0.010 18.362 0.005 17.770 0.004
163.7991610 5.4381320 18.665 0.010 17.513 0.005 16.686 0.002

The resulting file: 
% cat cdfp_panstarrs.out  
RA_hrs,DEC_deg,g,r,i,B,V,R,                                                     
  10.917755127   5.460006237    17.349    16.139    15.100    18.222    16.608    15.825
  10.918918610   5.412124634    14.138    13.822    13.749    14.509    13.932    13.668
  10.917838097   5.458470821    17.430    17.542    17.716    17.561    17.480    17.429
  10.917894363   5.441155910    18.733    17.988    17.705    19.345    18.270    17.799
  10.918787003   5.437980175    18.826    17.946    17.582    19.514    18.283    17.744
  10.918789864   5.456964493    19.953    18.757    17.669    20.818    19.221    18.434
  10.919754982   5.435897827    19.543    18.362    17.770    20.400    18.820    18.122
  10.919943810   5.438131809    18.665    17.513    16.686    19.505    17.959    17.234
    The gri data are used to predict transformed BRI magnitudes. Note that I typically run this script in the top reduction directory. The output fle (named cdfp_panstarrs.out) is written to this directory. However, I also have the run script (cdfp_panstarrs.sh) also copy this file to the ./local_red/ZPTAB subdirectory with the name "griBVR.cdfp". This file contains all of the photometry returned by PS1. Hence, the real result of our efforts here are in: 
    
./local_red/ZPTAB/griBVR.cdfp
    This is a standard cdfp file that can be used by a variety of cdfp_ tools. Note that at this point you may clean out the file in your top-level reduction directory. All of the files we need for the file ZP calibration step should now be stored in ./local_red/ZPTAB/.



    Cross-match catalogs and compute ZP values.

    Here we compute the zeropoint values. 

    
% cdfpmatI.sh 20180206T070902.2_acm_sci_proc.cdfp griBVR.cdfp 2.0
Resulting file is:   cdfpmatI.cdfp

I have renamed this file "zp_data.cdfp" and I am using it to develop zp_table_make.
This code with build a table that I can use in zpadopt (what I used in the USNO 
pipeline for zp derivation). Basically, I need to make a table file that has 
the ZP. some colore (like g-r), and values like Peak and AnnAv, etc....
See this filein: 
/home/sco/A_wcsf/ZP_may02/test2
/home/sco/sco/scohtm/scocodes/munge/ex2_zp

Usage: zp_table_make zp_data.cdfp g Y 
arg1 - basename of input CDFP file  
arg2 - name of magnitude system to be transformed to 
arg3 - run in debug mode? (Y/N) 

This creates:   ZPSEC.params  ZPSEC.parlab  ZPSEC.table

Run the table_xy_boxclean script on this table fle to get a mean zp value 
Usage: table_xy_boxclean ZPSEC Peak ZPSEC Y 
arg1 - basename of the table file (A for A.table)  
arg2 - parameter name for X axis  
arg3 - parameter name for Y axis  
arg4 - run in debug mode? (Y/N)
    I have preared a more general script that handles the ZP calculation after the gri data are retrieved from PDS1 and processed into a combined cdfp file named "griBVR.cdfp". This file resides in ./local_red/ZPTAB. The code is run from the top level directorey (where ./local_red is): 
    
% zpgriBVR --help 
Usage: zpgriBVR 20180206T022042.6_acm_sci_proc ./local_red/ZPTAB N 
arg1 - fullpath name of FITS image 
arg2 - path to local archive directory fr ZPTAB (fdir3) 
arg3 - run in debug mode (Y/N)

Note that this code can be run manually, but it is mostly run from 
within the routine:    photcal_fitslist

% photcal_fitslist --help 
Usage: photcal_fitslist ./S/list.0 usno N 
arg1 - Name of file with list of FITS images (can be full path)
arg2 - photometry catalog to use (usno,ps1)
arg3 - run in debug mode (Y/N)
    I note that photcal_fitslist is now set up to calibrate images using either USNO or PS1 photometry.



    Appendix 1: An early manual query for PS1 data.

    PS1 refers to the first data release of PanSTARRS. 

    

How to get PanSTARRS photometry 
===============================
I start with an image that has good astrometry: 
/home/sco/A_wcsf/images/20180206T070902.2_acm_sci_proc.fits

Just make a file of RA,DEC 
   % ds9 /home/sco/A_wcsf/images/20180206T070902.2_acm_sci_proc.fits
   % mv ds9.reg file.1 
edit to get just ra,dec, I remove the colons the file is "file.1" 

Go to URL:   https://archive.stsci.edu/panstarrs/search.php?form=fuf
NOTE: fuf = File Upload Form 

Navigate to file.1 using the Browse button. 
Set output columns   (I use ra,dec, and PSFmags for g,r,i) 
Set output format    (I used "File: comma separated values") 
Hit the "Search" button. 

In a couple seconds I see: 
% ls -lt ~sco/Downloads/ 
total 572
-rw------- 1 sco sco   9588 Apr 29 15:50 panstarrs_search.txt
 
Here is my file.1 and my returned file: 

% cat file.1  
10 55 04.359,+5 26 23.91
10 55 04.165,+5 27 26.03
10 55 03.841,+5 27 31.13
10 55 01.629,+5 25 53.10
10 55 07.562,+5 26 12.49
10 55 11.010,+5 26 04.72
10 55 07.586,+5 27 20.96
10 55 11.778,+5 26 13.64
10 55 07.984,+5 24 39.12
10 55 05.354,+5 25 43.29

% cat panstarrs_search.txt.gri
Input line 1: 10 55 04.359,+5 26 23.91
raMean,decMean,gMeanPSFMag,rMeanPSFMag,iMeanPSFMag
ra,dec,float,float,float
10 55 04.421,+05 26 28.16,18.733,17.988,17.705
Input line 2: 10 55 04.165,+5 27 26.03
10 55 04.208,+05 27 23.02,-999.000,21.932,21.393
10 55 03.888,+05 27 24.64,-999.000,-999.000,21.593
10 55 04.217,+05 27 30.49,17.430,17.542,17.716
Input line 3: 10 55 03.841,+5 27 31.13
10 55 03.919,+05 27 36.02,17.349,16.139,15.100
10 55 04.217,+05 27 30.49,17.430,17.542,17.716
Input line 4: 10 55 01.629,+5 25 53.10
10 55 01.692,+05 25 57.23,19.703,18.805,18.646
Input line 5: 10 55 07.562,+5 26 12.49
10 55 07.632,+05 26 16.73,18.826,17.946,17.582
Input line 6: 10 55 11.010,+5 26 04.72
10 55 11.119,+05 26 09.23,19.543,18.362,17.770
Input line 7: 10 55 07.586,+5 27 20.96
10 55 07.642,+05 27 25.07,19.953,18.757,17.669
10 55 07.319,+05 27 17.16,-999.000,20.495,-999.000
Input line 8: 10 55 11.778,+5 26 13.64
10 55 11.799,+05 26 17.28,18.665,17.513,16.686
Input line 9: 10 55 07.984,+5 24 39.12
10 55 07.978,+05 24 40.80,-999.000,-999.000,20.145
10 55 08.096,+05 24 39.96,-999.000,-999.000,20.001
10 55 08.109,+05 24 43.65,14.138,13.822,13.749
Input line 10: 10 55 05.354,+5 25 43.29
10 55 05.439,+05 25 45.58,13.846,-999.000,13.205
10 55 05.496,+05 25 46.87,12.288,11.944,12.692

NOTE: 
Much easier file to read if I: 
Much easier file to read if I: 
  - use "Suppress null result message" 
  - use degree ra,dec
  - use space spearated values on output 

When the search is done: 
mv ~sco/Downloads/panstarrs_search.txt panstarrs_search.list
    Notice that the example above was an early version. I later learned to click the "Suppress null result message" message to de-clutter the output list. I also changed to Ra,Dec in degrees instead of sexigecimal format.



    Back