In late Dec2018 I began to solve a shitload of silly matplotlib problems associated with using an interactive cursor. In the end, this also turned into a python refresher. Here are some notes.
Thi file was: /home/sco/README.Dec21_2018 (on scohome)
Dec2018: Where things stand
===========================
My image processing codes (ds9_profiles,imstats) are well testsed and workng on
several machines. Questions remain about how best to subtract sky, extract photometric
parameters, etc...., but the basic code infrastructure is in place.
I have a script (wcsf_imgcat) that will process a list of FITS images to the point
of creating sky maps and other images as well as a simple catalog of moment-based
image parameters. The "wcsf" part of the name indicates this was build for finding
sources with which to astrometrically calibrate the images.
After some thought, it seems that I really want a system hat starts with the catalog.
The first set of parameters could be moments-based, but eventually I want a catalog of
parameters that can be used for image classification (i.e. star-gal sep). With such a
catalog I can then proceed to higher quality WCS and ZP calibration. U now have a working
insteractive cursor and plotting setup. Hence, it seems a good time to start with wcsf_imgcat
and create a concise catalog builder with a goal of image classes and "best-estimate" sky
maps.
The state of scohome:
I have two hard disk copies of my system:
/dev/sdb1 1922726704 576614400 1248436736 32% /media/sco/470f7c55-8ab2-4ed4-bc7f-b21445176bce
/dev/sdc1 1922726704 576614420 1248436716 32% /media/sco/fb0d7e99-4ac6-4b31-be53-b63d101a3b6c
For example, I can go to the bash script area and see my codes:
% cd /media/sco/470f7c55-8ab2-4ed4-bc7f-b21445176bce/home
% ls
sco/ scob/ sco_back/ scoc/
% cd sco/Installs/install_sco_20180522/codes/bash
*** I see my bash scripts here.
Note also that I now have working versions of my codes on buckaroo and banzai.
I have stored all of the data I care about on the buckaroo scodata disk (and
most everything on banzai also).
===============================================================================
A simple test using the Test data from ds9_profiles
% cd Test_ds9_profiles
% mkdir tcat1
% cd tcat1
% imgcat0 ../images/Rsco2039.fits pfc 10.0
**** it does not run anymore!
/home/sco/Installs/install_sco_20180522/bin/binc/otw/skyhistfit.sh: line 55: poly3.x_for_miny: No such file or directory
*** So, once again I have modified a code and fucked up other codes.
I do make lots of calls to imgcat0 in various scripts, but these are fairly old
and most of them should be considered obsolete. However, in the spirit of preserving
old thinsg that (mostly) work, I am going to build imgcat1: the initial change will be
to simply add a debug flag to the command line.
*** This was an easy fix. I just had to change one file name in the skyhistfit.sh script.
In making this fix I used imgcat1. This really only adds a debug argumnt to the command line.
Sat Dec 22 09:12:08 CST 2018
============================
In the course of the abover work, I made a simple plot using the Rsco2039.table file and
xyplotter_auto. I saw an interesting star-gal sep space, but this immediately raised a desire
to have a tool that I think I can now cobble together without a lot of hassle now that I have
a working interactive cursor routine (like the table_point_selector routine used in xyfitter).
The tool I envision:
1) Plots a parameter space from a table file and lets user select a point
2) the point is dentified and the i1,2,j1,j2 coordinates are extracted
3) this box area is zoomed to in a specified ds9 frame where the image is displayed.
Mon Dec 24 09:40:14 CST 2018
============================
I have:
imgcat0 = makes a reasonable catalog based on zeroeth and first image moments
ds9_imgcat0 = displays the diagnositic images from imgcat0
table_point_selector = view (X,Y) space from a table file and select points
I'll just manually fuck around with things for a while this morning.
I can run table_point_selector (tps) on the imgcat0 table file = Rsco2039.table
table_point_selector Rsco2039 logEqRad isomag30 N
At the end of each run (after I enter "E") I have:
table_point_selector.X,Y,LineNum,Xstats,Ystats
The file of importance here is: table_point_selector.LineNum
It gives me the line numbers of the points I selected ("s")
In my example run I selected 3 big (presumed) galaxies:
% cat table_point_selector.LineNum
1
2
4
I want to get the table lines for these (i.e. the lines after "# data")
% lineget.py Rsco2039.table 1
1 5911 943 1038 1016 1101 13.697 990.81 1057.88 15.6 0.873 -67.68 76.88 1.6373
% lineget.py Rsco2039.table 2
2 3815 652 738 1146 1222 14.211 695.20 1183.03 15.2 0.577 -50.71 61.77 1.5422
% lineget.py Rsco2039.table 4
4 877 538 603 914 939 16.661 573.98 925.63 13.0 0.386 82.96 29.61 1.2229
i1 i2 j1 j2 Xc Yc
To look at the field:
% ds9_imgcat0 Rsco2039.fits N
*** I verify, using circle regions with tthe Xc,Yc manually entered that these
are my 3 big galaxies.
To zoom in with xpa:
% xpaset -p ds9 pan to 573.98 925.63 physical # centers N3389
% xpaset -p ds9 zoom 4 # zooms in 4 times
Then I can pan to another galaxy and retain the same (visually set) zoom factor
% xpaset -p ds9 pan to 990.81 1057.88 physical # centers N3379
% xpaset -p ds9 pan to 695.20 1183.03 physical # centers N3384
Using the box size (i1,i2,j1,j2) does not seem to be that critical. I
just pan to the position and manually set the zoom for each object (which
will be about the same for most objects).
I'll kluge this up in one script: imgcat0_viewer Rsco2039.fits N
------> I have a working version of imgcat0_viewer!
Once again, some information on using show() would be so helpful.
I submitted a question to a site (forum?) called: edureka
https://www.edureka.co/community/34032/record-limits-displayed-plot-using-matplotlib-show-module
Probably won't get an answer, but you never know.
I also found some intersting stuff googling: "matplotlib forum show()"
See: https://stackoverflow.com/questions/458209/is-there-a-way-to-detach-matplotlib-plots-so-that-the-computation-can-continue
Another very useful page: https://realpython.com/python-matplotlib-guide/#why-can-matplotlib-be-confusing
Near the bottom there is some interesting stuff on interactive mode.
Then I looked at: https://matplotlib.org/faq/usage_faq.html#what-is-interactive-mode
and tried this (very interesting)
import matplotlib.pyplot as plt
plt.ion()
plt.plot([1.6, 2.7])
************** THIS COULD BE WHAT I HAVE BEEN LOOKING FOR!
I did these steps and interactively built a plot:
>>> import matplotlib.pyplot as plt
>>> plt.ion()
>>> plt.plot([1.6, 2.7])
[]
>>> plt.title("interactive test")
>>> plt.xlabel("index")
>>> plt.ylabel("boodle")
>>> quit()
****** I now have, in icurs3.py, a code that is much more efficient in
identifyng points.
Another thing that would be extremely useful is being able to learn
value for (or) lot properties. I keep seeing references to rcParams.
This is importat I think, but as usual, no clear explanations.
It seems that rcParams is some entity (list?, dictionary?) that sets properties:
-------------------------------------------------------
def visualize_2D_trip(self, trip):
plt.figure(figsize=(30,30))
rcParams.update({'font.size': 22}) # I guess the sets a new font size.
# Plot cities
plt.scatter(trip[:,0], trip[:,1], s=200)
-------------------------------------------------------
Maybe I just fuck around in python:
% python
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
>>> import matplotlib.pyplot as plt
>>> type(rcParams)
Traceback (most recent call last):
File "", line 1, in
NameError: name 'rcParams' is not defined
>>> type(plt.rcParams)
# so RcParams is a class
>>> len(plt.rcParams)
179
>>> print plt.rcParams
{'figure.subplot.right': 0.9, 'animation.frame_format': 'png', 'mathtext.cal': 'cursive', .....
I seem to get this huge list of stuff
Maybe I just use this to query? What about figure.subplot.right?
What the fuck is { } ?
Ans: It is a dictionary, see: https://www.tutorialspoint.com/python/python_dictionary.htm
========================================================================
Fiarly clear text on lists and tuples: https://realpython.com/python-lists-tuples/
Define a list:
a = ['foo', 'bar', 'baz', 'qux'] # lists use [ ]
Define a tuple:
t = ('foo', 'bar', 'baz', 'qux', 'quux', 'corge') # tuples use ( )
========================================================================
How do I use a dictionary?
To set the values:
dict = {'Name': 'Zara', 'Age': 7, 'Class': 'First'}
print "dict['Name']: ", dict['Name']
print "dict['Age']: ", dict['Age']
Here is how it works in my example above:
>>> print "rcParams[ Andale Mono ]"
rcParams[ Andale Mono ]
>>> import matplotlib.pyplot as plt
>>> type(plt.rcParams)
>>> str(plt.rcParams) # big list
>>> f = "rcParams[ Andale Mono ]"
>>> print f
rcParams[ Andale Mono ]
>>> plt.rcParams.keys() #creates a list of just the keys
Once I know a valid key name, I can get the value with:
>>> plt.rcParams.get('axes.titlesize')
'large'
So, to get a listing of all keys in rcParams:
>>> plt.rcParams.keys()
# This creates an enormously long single line. How do I get
a list of the key value with one key value per line?
=====================================================
Now I am just looking at examples
To reste valeus:
params = {'savefig.dpi' : dpi,
'savefig.format' : 'png',
'axes.formatter.limits': [-3, 3]}
rcParams.update(params)
=====================================================
=====================================================
Here are two ways to print the (big!) plt.rcParams dictionary:
import matplotlib.pyplot as plt
method1:
for keys,values in plt.rcParams.items():
print "\n"
print(keys)
print(values)
method 2:
import json
print(json.dumps(plt.rcParams, indent = 4))
method 3:
from pprint import pprint
pprint(plt.rcParams)
-------------------------------------------------------------------
Practical point: It is difficult (for me) to enter this
interactively, so I make a file and then run it:
% cat p1.py
#!/usr/bin/env python
import matplotlib.pyplot as plt
ffc = open('File.rcParams', 'w')
for keys,values in plt.rcParams.items():
ffc.write( "\n %s\t%s \n" % (keys,values) )
ffc.close()
% python p1.py
% ls
File.rcParams p1.py S/
% cat File.rcParams # this s a viewable file!
-------------------------------------------------------------------
=====================================================
When I measure the figure made by icurs3.py it is not 8x6
figure.figsize [8.0, 6.0]
params = {'savefig.format' : 'png',
'axes.formatter.limits': [-3, 3]}
rcParams.update(params)
I do get a new, larger (but bot 9x9) plot inicurs3.py with:
# experimental
paramso = {'savefig.format' : 'png',
'figure.figsize' : [9.0, 9.0],
'axes.formatter.limits': [-3, 3]}
plt.rcParams.update(paramso)
**** So, I am at least able to change soemthing I want changed!
*************** Fiannly ********************************************
Here is how I can query the current box plot limits.
print "Current X1,X2,Y1,Y2 limits: %s %s %s %s \n"% (plt.axis())
**********************************************************************
***** Finally, here is an example of how to make a nice list of rcParam *****
#!/usr/bin/env python
'''Print a readbakle table of the default rcParams dictionary'''
import matplotlib.pyplot as plt
ffc = open('File.rcParams', 'w')
for keys,values in plt.rcParams.items():
ffc.write( " %-30s %-50s\n" % (keys,values) )
ffc.close()
Fri Dec 28 10:11:03 CST 2018
============================
A note from Greg Z:
Hi Steve,
I don't do interactive mode often, but I found this brief example set that
may or may not be useful: https://www.datacamp.com/community/blog/bokeh-cheat-sheet-python
This is an alternative to matplotlib, and should be included in anaconda.
I haven't used it, but there is a cheatsheet:
https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Python_Bokeh_Cheat_Sheet.pdf
I haven't ever used it, but I'm happy to talk with you (via zoom or skype)
and I could at least see exactly what you are trying to do and may be of
more help. Let me know.