Thursday, September 1, 2011

I think it's finally done.

My contrast curve plotter/point source detector finally works as intended!

To detect a point source companion or plot a contrast curve requires the program to first determine the background noise in the image. The program does so by first determining the aperture of the star (radius of one FWHM) and then determining the pixels that make up annuli that are 2 FWHM wide around the star. The radius of each annulus is defined as the midpoint between the inner and outer radii that define the annulus. Each annulus increases in radius by FWHM/2. This overlap of annuli allow us to find point sources that might have been on the edge of an annulus and otherwise missed.

For the point source detector, we further break up each annulus into spheres arranged in a ring in the annulus. Each sphere has a radius of 1 FWHM and the spheres overlap each other by half a FWHM. We determine the background of the spherical regions and then find point sources according to the darkpix, thresh, and adj values I've mentioned in previous posts. By breaking up the annuli into regions, we can more accurately determine the local background noise in each region and see if any region contains a point source.


(To refresh: darkpix is a decimal that determines the percent of darkest pixels used to find the mean noise of the background. thresh is the threshhold, or the multiplier we use with sigma, the standard deviation of the flux. Only pixels with intensity values greater than thresh*sigma+background mean will be labeled as potential point sources. adj determines the number of adjacent pixels greater than thresh*sigma+background mean a potential point source has to have in order for a pixel to be labeled as the position of the point source. )


For the contrast curve graph, we measure the difference in magnitude in each annulus with the peak flux of the central star. We call this difference contrast, or delta M. The graph plots contrast with respect to the radius of each annulus (the midpoint between the inner and outer radii). The magnitude value in each annulus is calculated in this way:

After getting the mean background of the regions in each annulus, we average the values of each region to get the average intensity of the annulus itself. We then convert this to a delta mag number by using the equation:

Contrast = -2.5*log10(flux1/flux2)


Flux 1 is the average flux of the annulus; flux 2 is the peak flux of the star. Finally, we plot the contrast with respect to magnitude on our graph.

If there is no companion point source, the program will plot a contrast curve; otherwise, the program will label the location of the main point source and its companion(s). The user can opt to save either of these figures by pressing "w". A text file with some important information will be saved along with the figure.
-----
This serves as a general description of what my program does. There are some specific details that I also need to mention:

- The darkpix value in my program, when used to detect extra point sources, has been set as 0.8 to increase robustness of the point source detector. The darkpix value is generally set to 0.95 when determining which pixels constitute the "background" of the FITS file.
- The ratio of arcseconds to pixels changes for each fits file. However, due to the variety of FITS files I am working with (some of them without headers as well), I cannot create subclasses for each FITS file (as I originally intended) and automatically make my program look for the arcseconds to pixels ratio. Thus the user needs to manually type it in into the phot4.py module. Hopefully this process will become more user-friendly as I clean up the program a bit.
-When the user enters multiple fits files into the command line when working with IPhot, the user needs to make sure that all files are working correctly. If not, the program will exit as it reaches that file. I have decided as of now to not let my program simply "pass" over these files as it would let a silent error through.
-I will post more as I find more errors.


Friday, August 26, 2011

Matching Justin's Results

So it turns out that I actually wasn't able to match Justin's results because we forgot that Justin had sent me images that had intensities on a square-root scale (square-root because it makes fainter companions stick out). Thus, all I needed to do to correct for this would be to square the original FITS array before making some contrast curves.

So: here are the new results for two different contrast curves:

(Properties: threshold=3 sigma)

KIC109:


KIC118:


Compare to Justin's results: 
At radial distances of 0.25", 0.5", 1", 2", 4" respectively:
KIC109   3.7, 5.6, 7.8, 8.6, 8.6  
KIC118   4.2, 5.7, 7.2, 7.6, 7.6


Unfortunately, I can't usually go past 1" on my graphs yet due to the way my point source finder works (it'll crash my computer if I try). However you can see that this is generally sufficient distance to make the contrast curve flatten out by the end. 


You may also have noticed that my delta mags tend to be slightly lower than Justin's values. This could be due to the fact that Justin took a thin slice of the picture and calculated the delta mags based on distance in one direction, while I averaged the noise at a certain distance away in all directions. But overall, the graphs seem to agree with Justin's results pretty well (It seems accurate up to 1").


To-Do:
-I will attempt to work out some kinks in my contrast curve calculator to make it run a bit more efficiently.
-Will continue working on a way to find point sources, as fitting 2D-gaussians still doesn't seem to work.

Thursday, August 25, 2011

Accepting Directories

Alright, now that I've finally adjusted to moving from Caltech to home (and learnt how to deal with my dust allergies), IPhot now accepts directories as well as individual .FITS files in one directory. (I guess I have to change the usage() description of IPhot as well.) Here are the three different ways IPhot can process and find .FITS files in command line:


  1. By default, if no additional argument is written after 'IPhot', IPhot will assume the directory is the current directory. Any .FITS files in this directory will then be processed. Let's say you have a file called fitsimage.fits in this directory. Typing in IPhot will give you this:

    C:\[Your path here]> IPhot
    fitsimage.fits


    The image will open up and you'll be able to click on it and such just as before.
  2. You can specify a directory after IPhot. IPhot will go through this directory and process all the .FITS images in that directory.

    C:\[Your path here]> IPhot "C:\\Example\\Path\\To\\Your\\Images\\" -d 0.95 -t 4
    ImageFromThisPath.fits

    ImageFromThisPath.fits and any other .fits images will then show up on the screen like before, and will create contrast curves in the way specified by the options following the path. You can still press 'q' to quite or any other key for the next image. Note the specific way you have to type the paths - it has to have quotes around it (since it's a string, and also because some path names have spaces inside them, like "Documents and Settings" that PC's use), and you should also write double \\'s to be safe.
  3. This is the way I originally implemented this program to handle .FITS files.
     
    C:\[Your path here]> IPhot CK00012_snap_20110412.fits -d 0.97 -t 5
    CK00012_snap_20110412.fits

    The program will go through the .FITS files one by one and create contrast curves for them (if you make it do so) according to the options following the .FITS files. Note that you can still type in multiple individual .FITS file names. 
I gotta say, this was pretty fun to work on. But I'm still not done with my point sources finder -- it's rather reluctant to actually find some point sources (simulated or 'real' ones) -- so I still can't test this code very extensively to make sure that it works brilliantly. For now though, I'll say that it seems to work fine.

Thursday, August 18, 2011

Contrast Curve Function 2.0 III and Point Source Detector


So it turns out that actually, because of various factors like dithering and such, my contrast curves are probably correct.
...
Yay! That's actually pretty exciting stuff.

However, now my point source detector is far too insensitive. I am trying to search for regions that have point sources, and then use the fitgaussian(...) function to detect those point sources. However, the fitgaussian(...) runs very slowly over large arrays and doesn't seem to be all that effective at pinpointing the point sources. Even after I've put down an artificial point source, it still doesn't detect anything in any of the regions. Perhaps it's the way I've implemented it, and I just have to keep working towards a solution.

Learned:
-What dithering means.
To-Do:
-Find a way to make my point source detector work correctly, whether by using the fit gaussian function, or some other function.

Tuesday, August 16, 2011

Contrast Curve Function v2.0 II

I finished the new contrastcurve(...) function today.


def contrastcurve(self,ann_space=0,darkpix=0.95,thresh=5): Returns list of radii and corresponding noise (thresh*sigma+mean background noise) and point sources (if any). It detects point sources by fitting gaussians over regions that have, on average, greater intensity than the noise and searches for peaks in those regions.


And it........... still doesn't work right. 


Here's an example though, of what the contrast curve looks like for KIC109:

Interestingly, however, if I calculate magnitude using log instead of log base 10, I get this result:



This agrees much more closely with Justin's data. Weird! I still haven't figured out why. 


By the way, Justin's data is this:
KIC109   3.7, 5.6, 7.8, 8.6, 8.6  at 0.25'', 0.5'', 1'', 2'', 4'' respectively.
(This is for thresh=3; mine are produced with thresh=5, but I ran the code again and this barely makes any difference). 


To-Do:
-Start testing how my code behaves around images with bright point sources (yes I said I would do this today, but then the contrast curves looked funny...) while I try to figure out why my contrast curves are behaving this way.
Learned:
-I actually learned something interesting today: If you have a function that returns multiple things in an array, it's much more efficient to do thing1, thing2, thing3=function() 
as opposed to thing1, thing2 = function()[0], function[1] even if you only want to save thing1 and thing2. This is probably because in the first case, Python will run the function once and save the corresponding variables, whereas in the second case, Python will run the function once, save the 0th element, and then run the function again and save the first. So I guess that ultimately what I learned is that if you have a function that takes a while to run, it's better to use the first case to save time, but if you have a function that's relatively fast for large amounts of data, but that returns a bunch of stuff, it's better to use the second case to save memory. If you have a slow function that returns lots of data.... well then good luck to you.

Monday, August 15, 2011

Contrast Curve Function v2.0 I

Today I finished most of the contrast curve function I talked about in my previous post. I haven't gotten it to return a list of detected point sources yet (it only returns radii and respective 5*sigma as of now), but it should integrate relatively well into all the other code I've written so far.

I have also been given a rough timeline of what I'm supposed to do for the rest of the summer (yes, although officially ten weeks, my project will continue till the end of summer/beginning of the fall term). So here it is:

Goals:
1) Rewrite my code and try and get the same results that Justin did.
2) Start using my code for all the fits files I have.
3) Detect problems and check them.
4) Repeat as necessary (This is not a fun step).
5) Make my code more user-friendly.
6) At some point start putting contrast curves up on a website.
.
.
.
n) Somehow help with the writing of a paper.

To-Do:
-Tomorrow I'll have to figure out how this new contrast curve function will actually detect point sources. Since we're talking about regions now, I might have to also change the way the program points out detected sources to the user.
-I also have to start writing code so that I can upload directories of .FITS files into the program. That way, the user won't have to spend a long time typing in the names of all the .FITS images.
-At some point I'll have to see what I can do about adapting my program to various arcsecond/pixel ratio. That is, I want my program to be able to recognize a type of image by its file name and automatically assign the correct arcsecond/pixel ratio.
-Finally, if possible/not too aesthetically ugly, I should try and make all the pictures axes display units in terms of arcseconds instead of pixels.

Friday, August 12, 2011

New Implementation

I talked with Tim and Justin about my code. The problem seems to be that in my current program, I am overestimating the amount of background noise since I'm calculating noise by averaging the flux from individual pixels. What I should do instead was to average noise from little regions and calculate the noise from the average of those regions. It seems like I'm just working with slightly bigger pixels, but by first averaging the flux from each region, I am able to smooth out my background and ultimately decrease the noise in each image.

So now, I have to code a different approach to finding point sources. It will still use much of what I have already written, but it's definitely the biggest change yet to my code.

Here's a rough outline of what I'm about to do:
I will partition each annulus into little circular regions of data. I will then average the flux in each region, and then find the standard deviation from the regions. This will be used to create sigma. The background mean will also be calculated from the regions. Detection of a point source apart from the main source will determined by if the a region has an average intensity greater than thresh*sigma. My program might pinpoint the specific location of the point source less precisely, but at least it won't detect fake point sources that turned out to just be speckles. The contrast curve will still be created in much the same way. It's just the way I determine background noise that will change the most significantly. 


I'll start working on this next week. 

Thursday, August 11, 2011

Sanity Check

I've been debugging my program for the last few days without much success. I've sent some preliminary testing results to Tim, but the results are definitely not good.

Here are two main problems I noticed:
-The point source finder is too sensitive. It detects almost anything that is greater than thresh*sigma. Unfortunately, there are many pixels in the picture that seem rather "grainy" and are registered as false point sources. I need to somehow eliminate these false point sources.
-The range of delta mags (contrast) for the contrast curves are way too small. The delta mags themselves are also high values, often hovering just below 0. This indicates that my program is not finding too much contrast within the annuli of the star compared to the central flux of the star, which seems intuitively wrong. However, this is most likely caused by the small size of the cropped image. The small images size makes my program calculate the background noise as part of the annulus of the star, which is incorrect as there is light pollution from the star. The problem is somewhat fixed when I increase the size of the cropped image; however, compared to Justin's results the delta mags still seem too small.

Professor Johnson suggested I do some sanity checks on my math yesterday, and I've been trying to figure out what really is going on with the math behind the contrast curves.

I'll post some more if I find anything interesting, but for now, it is just alone time for me and a buggy program.

Tuesday, August 9, 2011

Code Testing

Another colleague, Justin, sent Tim and I a MATLAB fitswriter file, which turns matlab .fig files into .fits files.

I have been setting up Matlab and using it to convert images into .fits files. I am then able to test my code on these new files, and compare my results with those of Justin's. Unfortunately, they don't seem to match up. The images that Justin sent me also seem to focus on objects with large FWHM's, which ultimately means that I obtain less points for my contrast curve because of the way I've implemented my code.

I am currently trying to compile a .pfd document containing the contrast curves I obtained using my code, and comparing my data to Justin's data. Hopefully this will give me a clearer indication of what the differences are between our results, and how I should change my code.

Friday, August 5, 2011

Saving Text Files and Figures

Today I finished a function that will allow me to save text files of some important data associated with the contrast curve plots I made.

When a user saves a figure, an associated text file will save the lists of data returned from phot3.contrastcurve(...). That is, it will save a list of points used in the creation of a contrast curve as well as any point sources the program may have found in the annuli of the main star. The text file will also save the center coordinates of that star.

The text file will save to the name of x-y_fitsfilename-results.txt, where x,y are the center coordinates, and fitsfilename is the name of the fits file.

Here's an example of the kind of text file it will save:

C:\[Your path here]>IPhot CK00012_snap_20110412.fits
CK00012_snap_20110412.fits
Press Q/q to quit, or any other key to load next image.

Clicking on point (22,100) and pressing W will save this picture... 
...as well as a .txt file containing this information:


Center Pts
(22, 100)
Coordiates (Radius,Contrast)
[(1.0310396072149, 1.2019494856629946), (1.2028795417507165, 1.8693450269793654), (1.3747194762865333, 2.1077248753625968), (1.5465594108223499, 2.3778325825061946), (1.718399345358167, 2.4598854750345436), (1.8902392798939833, 2.5393921380320639), (2.0620792144297999, 2.5689754466690302), (2.2339191489656169, 2.5851800258224134), (2.4057590835014335, 2.5950414542085398), (2.5775990180372501, 2.5998551889539634), (2.7494389525730667, 2.6055135477309928), (2.9212788871088833, 2.6096344474456359), (3.0931188216446999, 2.6104568644975972)]
Additional Point Sources at Each Radius (if any)
[[], [], [], [], [], [], [], [], [], [], [], [], []]


Saved to C:\Documents and Settings\Administrator\Desktop\CC22-100_CK00012_snap_2
0110412.png
Saved to C:\Documents and Settings\Administrator\Desktop\22-100_CK00012_snap_201
10412-results.txt
q
Quitting program...

C:\[Your path here]>

-----
Note that I also created a little compass on the lower right corner of each Image object. =]

To-Do:
-Somehow scale the axis ticks of the subplot images to arcseconds as opposed to pixels.
-Maybe at some point figure out how to load directories of FITS files into my command line as opposed to a list of file names (which are very tedious to type).

-----
EDIT:
I have changed the output of the text file so that it no longer returns a list of lists of point sources. It will simply return a list of point sources, or an empty list if there are none.

Wednesday, August 3, 2011

IPhot Changes

I successfully linked the two functions together in IPhot's createfigure(...) today after working on it for the past two days.

def createfigure(x,y): Creates a figure with a cropped image centering on the clicked point source by using phot3.plotptsmarking(...). If there are no other point sources in the cropped image, this function will plot a contrast curve using phot3.plotcontrastcurve(...), and the cropped image will be an inset on the top right corner of the contrast curve. Both of these functions were described in the previous post. x,y are the x and y coordinates of the mouse click. The cropped image specifications are specified by what the user enters in the command line and the mouse click, which indicates which point to center on. 


Example: (Using command line)


C:\[Your path here]>IPhot CK00012_snap_20110412.fits -s 2
CK00012_snap_20110412.fits
Press Q/q to quit, or any other key to load next image.


Clicking on the center point source at around (94,72), we get this:
Detected point source: (94, 76)

Clicking on the point source at around (22,100), we get this:

q
Quitting program...


C:\[Your path here]>


Note that I used an annulus spacing of 2. This shows up in both of the cropped images. The inner circle around the point source represents the aperture radius. The larger circle around the point source indicates the innermost annulus radius, or where the program will begin to sample the contrast between the point source and the background around it. 

In the first cropped image, there is also a smaller circle (with a fixed radius of 1) that indicates where other point sources are in the image.

Learned:
-How to create inset graphs (in two ways actually!). I can either use the axes(...) function from pylab or the inset_axes(...) function from mpl_toolkits
To-Do:
-Test my program and debug it. There are probably still some minor problems or aesthetic issues I still need to fix. 

Tuesday, August 2, 2011

Phot3.py - Two Functions

Today I finished coding two plotting functions.
def plotcontrastcurve(self,ann_space=0,darkpix=0.95,adj=3,thresh=5,fig=1): The original plotcontrastcurve(...) function wasn't nearly as fancy as this one. This function also allows the user to specify the number of adjacent pixels (adj) and the number of deviations from the mean background (thresh) that determines whether a pixel is a point source or not. The math behind the contrast curve has also changed. The shape looks pretty good, but I'm concerned that a lot of the curve is up in the negatives for contrast...

def plotptsmarking(self,cx,cy,filename="",ann_space=0,darkpix=0.95,adj=3,thresh=5): This function makes one of those subplot images that circles the aperture, the innermost annulus, and other point sources in the picture (if possible). It also has arguments for thresh and adj

Tomorrow I will link these two methods together in IPhot.

Friday, July 29, 2011

Phot3.py and Other Major Changes

Due to my recent major change to my findptsources(...) function, I had to also implement major changes to my overall phot2.py module. The result was that I decided to just start over and make a new module called phot3.py. It is essentially the same module, but I've streamlined it a bit and deleted some functions that I now don't need because of my more efficient findptsources(...) function. 


I've also been given suggestions to present my data in a more space-conserving and visually appealing way. 


I am working on creating a subplot image within a contrast curve plot. The subplot image will be like it is now - it will center on the point source and indicate where its aperture extends to and where its innermost annulus begins. 


I am also working on changing what happens when a user clicks a point source. When a click is registered, a cropped image centering around the point source will appear. If there are no point sources detected within the annuli of that cropped image, a contrast curve will appear as mentioned above. If there are point sources detected within the annuli, they will be marked with respective circles and the program will return a printed list of detected point source coordinates.

I'll continue working on this next week.

Thursday, July 28, 2011

Editing More Code...

This week has just been a major code-editing week.

Today I made a major change to the way I find point sources.

def findptsources(self,datalist,darkpix=0.95,adj=4,thresh=5): Finds a list of point sources (points brighter than their surrounding points where the number of adjacent pixels (adj) is brighter than thresh*sigma).

The new findptsources(...) function can be adjusted in sensitivity:

thresh gives the user the ability to specify how many deviations from the mean background noise the flux of a point has to be to note it as a potential point source. Thus, instead of only having 5*sigma, we can also ask the program to search for all potential point sources in which a point source is defined by any point greater than 3*sigma.

The new findptsources function can be adjusted in sensitivity by setting values for adj (0-4). It indicates how many of the neighboring points have to also be greater than thresh*sigma in order for the central point to be flagged as a point source.

You may note that the new findptsources(...) requires an array input, which seems kind of silly at first glance. However, this function is also being used to find the point sources in each annulus around a main point source, which means that it needs to be able to take the annulus array as an input, even though the main FITS file array might be different.

The new point source finder kind of already acts like a point source detector because it can look for point sources within annuli, which means that I don't need my old point source detector code I was working on earlier this week.

I tested this new point source finder, and these are the results:
>>>im=phot3.Image(arr)

>>> im.findptsources(im.data)
[(116, 22), (83, 40), (25, 53), (94, 72), (94, 76), (22, 100), (52, 128)]
>>>


As you can see, this is exactly the same array as before. I haven't had time to extensively test this yet, but I believe I'm at least on the right track. 

Learned:
-Neat way to use booleans as integers (0 and 1's) to meet threshold requirements.
To-Do:
-Debug more (I think you're seeing the trend by now...)

Wednesday, July 27, 2011

Still editing that point source detector...

Functions created: 

def addsource(self,cx,cy,peak_flux,FWHM=1): Adds a point source to current Image object and returns a new Image object. cx, cy are the coordinates for the point source, peak_flux is the maximum flux at (cx,cy), and FWHM is... well, the full-width-half-max. 

Other than that, today was mostly spent trying to correct tiny errors and figuring out how to correctly implement my Point Source Detector so that it looks for 3 points that are brighter than 5*sigma (the 5*sigma of the annulus it is positioned in). The current "filler" function I'm using for my Point Source Detector is the point source function I use to find my main point sources. However, this is technically incorrect, since the background noise affecting the main point sources is the background noise in the sky. The background noise for the non-main point sources (the faint point sources that might be polluting the light from the main point source) depends on the brightness of each annulus around each main point source. 

So I guess I'll just continue to work on this tomorrow. 

Tuesday, July 26, 2011

I feel like a happy frog...

...because I caught a major bug in my code today.
Well, I suppose that means I did something horribly wrong in the past weeks, but it definitely feels good to at least know that I've fixed it. Anyways, I again edited my code for findptsources(...). You may have noticed that in my previous posts dealing with point sources, the aperture rings were off-center. They're fine now. See the picture below:
The aperture ring is very nicely centered on the brightest parts of the object. This is because the array of point sources changed. If I open up the Python interpreter and type:

>>> from startup import*
>>> im=phot2.Image(arr)
>>> im.findptsources(darkpix=0.95)

I get this:
[(116, 22), (83, 40), (25, 53), (94, 72), (94, 76), (22, 100), (52, 128)]

All the point sources are isolated, and there are very few of them, which is nice to start with. Also notice that (94,72) instead of (92,72) is the point source, which makes sense, since if we type in:

>>> im.maxx
94
>>> im.maxy
72

we get the x and y coordinates of the pixel in the graphic with the maximum flux, and thus the point source (assuming of course it's not a cosmic ray).

So, hopefully this will be the last time I edit the point source finder code. And hopefully tomorrow I can just focus on making my point source simulator.

-----
I also worked a little bit on the Point Source Simulator today and cleaned up some coding for my Point Source Detector. But then I got distracted by the bug mentioned above...
-----
Learned (the hard way):
-Array elements are organized differently than pixel coordinates! >.<
To-Do:
-Finish the Point Source Simulator
-Edit some more code for my Point Source Detector and make it more precise in detecting point sources (as in, not make it accidentally detect cosmic rays)

Monday, July 25, 2011

Point Source Detector

Hello!

I finished the basic code for the Point Source Detector today. (Though I haven't quite had time to test it yet).

def ptsdetector(subim, aperture, darkpix=0.95): Given a subplot of an Image object (cropped Image object basically), the aperture around the main point source, and an optional darkpix argument, we can detect whether there are any point sources near or within the aperture of the main star.

Now, if we load the main FITS file and click on it, this figure will pop up:
If there are actual point sources within the aperture (significant addition to the light of the main point source), the program will print an additional message stating "Detected point source within aperture: (x,y)" and indicate the x and y coordinates where that point source was found. 

Note that there are two smaller circles. These denote where the point sources are in the picture. The concentric circles indicate that this is the main point source we are focusing on (the point source that was selected by the user). The other small circle indicate that there is another point source near the main point source. 

In this case, however, it should be noted that the slit in the picture is messing around with the object's light, and so we are given another "point source" when in fact it is just the light from the main source on the other side of the slit. I'm eager to test this with some other images that hopefully do not have slits on the main point source. 

Learned:
-Not much new stuff.... today was mainly coding
To-Do:
-Clean the point source detector code a little
-Start on the point source simulator

Friday, July 22, 2011

Refined: findptsources(...)

I finished the refined version of findptsources(...). Unfortunately, there are a lot of cases to take care of, and I'm not sure I accounted for all of them. I tested my code with the two FITS file images I had, and the code worked well for both of those images. Hopefully the code works well for all FITS files, though I'm currently not 100% sure. I'm trying to get some more FITS files to work with at the moment.

Anyways, here's some examples:


>>> from startup import*
>>> im=phot2.Image(arr)
>>> im.findptsources()
[(115, 22), (116, 22), (83, 40), (25, 53), (90, 72), (91, 72), (92, 72), (93, 72), (94, 72), (92, 76), (93, 76), (94, 76), (18, 100), (19, 100), (20, 100), (21, 100), (22, 100), (51, 128), (52, 128)]


[(115, 22), (83, 40), (25, 53), (92, 72), (93, 76), (20, 100), (52, 128)]

As you can see, the second list of point sources is much shorter!
In case I haven't explained what my edited code does clearly, here's another attempt.

As you can see, the lists contain a bunch of tuples that represent coordinates of pixels in the FITS image. In the first list, many of these tuples have the same 'y-coordinate' (second number in each tuple), meaning that the pixels are aligned in a horizontal line (excuse my handwriting):
It's easy to see now that the second list of tuples takes the 'midpoint' of a line of pixels. By midpoint I mean that for any line of odd pixel length, it takes the mid point, but for any line of even pixel length, it picks one of the two middle most points. For a single point, the program still includes it, even though it might just be a cosmic ray. You can check that this is true with the second FITS image as well:

>>> im2=phot2.Image(arr2)
>>> im2.findptsources()
[(42, 64), (43, 64), (91, 72), (92, 72), (93, 72), (94, 73), (95, 73), (94, 76), (95, 76), (91, 77), (92, 77), (93, 77), (37, 79), (52, 95), (98, 100)]


[(42, 64), (92, 72), (95, 73), (95, 76), (92, 77), (37, 79), (52, 95), (98, 100)]


Obviously a lot of these points could be cosmic rays or point sources we are not concerned with (as in they don't affect the total flux of the main point source we are looking at). But my program includes these points as well because I don't want to make it rule out anything potentially interesting. 


On Monday I'll start working on the point source detector code, and perhaps the point source simulator as well.

Learned:
- How to write complicated if statements.
- How to follow complicated if statements.
To-Do:
- Write the point source detector code
- Create the point source simulator to test my code

Thursday, July 21, 2011

Array Splicing

I'm currently refining my code that finds point sources. However, I've been stuck on this same problem the whole day:

I have here a list of point sources from one of the FITS images:

[(42, 64), (43, 64), (91, 72), (92, 72), (93, 72), (94, 73), (95, 73), (94, 76), (95, 76), (91, 77), (92, 77), (93, 77), (37, 79), (52, 95), (98, 100)]
Each tuple is a coordinate (x,y). If you look carefully, you'll note that there seems to be many rows of "point sources". This is because at these points, the FITS array recorded the same amounts of photons. But a point source should be a single point that is brighter than all its surrounding points. It should be the point of the star (or whatever object you are looking at). So I need to somehow sum up all the x coordinates for each row and then return the average of the x coordinates to find the midpoint of each "row". For example, instead of returning (91, 77), (92, 77), (93, 77), the program should return only (92, 77) since it's the midpoint of the row at y coordinate 77. But to do this, I need to figure out a way to splice my array based solely on the row number (the y coordinate). This proves to be rather difficult - especially if I want to avoid as many for loops as I can. 


I'll think about this tomorrow. 


Learned:
-A bunch of array manipulation functions
To-Do:
-Finish this task
-Create a point source simulator

Wednesday, July 20, 2011

Save Function and Point Source Detector

I finished another save function today. It lets users save contrast curves by selecting the graph with the contrast curve and pressing "w" (w for 'write' since s is already taken).

For example, CK00012_snap_20110412.fits saves as CC94-72_CK00012_snap_20110412.png. 'CC' indicates that it is a graph containing a Contrast Curve. 94-72 indicates that it was saved at the original FITS files coordinates of (94,72). This way, users can save multiple contrast curve images from the same fits file without accidentally writing files over.

I addition, I have been outlining and planning out how to make a Point Source Detector that will detect if there is a point source close to the main point source in the picture. This, however, also requires me to fix up my existing code that finds point sources. "Close" is defined as probably within 1 FWHM of the main point source. That is, its light is merged with the main point source's light since it's within the main point source's aperture.

Learned:
-How to save files
-Key press functions
To-Do:
-Create my point source detector function
-Eventually create a point source simulator and test my detector function

Tuesday, July 19, 2011

Save Function and ... some other things.

Yesterday I was working/finishing up on a proposal for a competition that gives SURF students a chance to use the Hale Telescope at Palomar for a well-designed project. ...Yep.
-----

Today I attempted to learn a little bit about Python's (well...pylab's) savefig(...) function. It lets you save the current figure to a file. However, halfway through I realized that Python has its own save figure code. Simply by pressing 's', you can save the current figure that is selected!

The thing is, the save window will pop up and prompt you for a file name in this case. I'm hoping I can create a function that will allow me to save the figure with the press of a key, and automatically give it a filename without me having to specify one. Maybe something like figure_x, where x is the figure number. Unfortunately, this seems to be difficult to do since I also have to make sure I don't save over any previously saved data.
-----

Results from testing indicated that my code was, unfortunately, not mac compatible (so not much got tested). Apparently it had something to do with end-line characters. I wrote some more lines of code and sent my program back. This time, I'm hoping that it will work.
------

Learned:
-How to save things in Python
-Certain compatibility issues

Friday, July 15, 2011

Beta Testing of IPhot?

I sent my program to Tim, who will test it over the weekend. I hope there won't be any serious bugs...

Anyways, IPhot now *ideally* comes with mac os compatability, the ability to upload multiple FITS files, and quick contrast curve plotting with a single click (or two)!

Example:


C:\>cd [your path here]


C:\[your path here]>IPhot CK00012_snap_20110412.fits CK00350_snap_20110524.fits -d 0.95 -w 70 -h 40 -s 0
CK00012_snap_20110412.fits
CK00350_snap_20110524.fits
Press Q/q to quit, or any other key to load next image.


The two lines starting with CK... indicate the FITS files that are loaded. At this point, this image pops up:


Clicking near point (93, 72) gives:

To go to the next image and close the current figure, I just press any key:


n
...loading next image
This image loads:
Clicking near the center gives me this:

Press Q/q to quit, or any other key to load next image.
n


I pressed n again, but as it doesn't have another Image to load, it automatically quits the program:


exiting...


C:\[your path here]>


-----


So there you have it! There are probably a few kinks I haven't worked out yet with the program, but hopefully nothing major. 


I haven't yet added a save function either as I couldn't easily figure out how to best add a save button without making the process of plotting contrast curves and loading images too slow. I'm trying to aim for a streamlined process of saving images, but at the same time, I don't want the user to feel rushed or confused while using my program.  

Thursday, July 14, 2011

Command Line Program - IPhot.py v 1.0!

I finally finished coding IPhot.py (previously called InteractivePhot.py, but I got sick of typing it into command line every time). So here's the long-awaited example:

Opening up my Windows command line... (not Python's command line)
C:\>cd [my path here]

C:\[my path here]>IPhot CK00012_snap_20110412.fits -d 0.95 --width=74 -h 60 -s 3
Press enter to exit.


The FITS image will pop up:

Upon clicking, I'll get this:

Notice that I actually have the name of the image on the second figure now =]

Also, users can specify as many or as few of the options as they want in any order they want. So:

C:\[my path here]>IPhot CK00012_snap_20110412.fits -h 60 -s 3 -w 74
Press enter to exit.

will give the exact same thing as the image above. (Note that -d or darkpix has a default value of 0.95)

To reiterate:
-d, --darkpix darkest x percentage of pixels
-w, --width = width of sub image (xsz variable in my python code)
-h, --height = height of sub image (ysz variable)
-s, --ann_space = annulus spacing, or the space between the aperture and innermost annulus radius (ann_space variable)


I also created a file called IPhotParams.py that is just a list of global variables with default values. A user can then edit the values in that file to quickly set the default values of darkpix, ann_space, etc without having to dig through the actual code.

Learned:
-More about RE
-A bit about setting paths
To-Do:
-Make an option to save the contrast curves
-Allow the program to sequentially load multiple FITS files; another will open once the user presses enter (or some other button)
-Start thinking about making the program mc os compatible.

Wednesday, July 13, 2011

Command Line Program II

More command line coding!

Today I learned about this function called getopt(), which will hopefully allow me to enter arguments (optional arguments) that will specify things such as the spacing between an annulus and aperture, the size of a cropped image, etc.

I've received another challenge: To make my program accept an arbitrary number of fits files in the command line window. Basically, we're hoping the input will look something like this:

C:\[your directory to saved InteractivePhot file]> InteractivePhot file1.fits file2.fits -d 0.95 -w 40 -h 40 -s 0


Where:
-d = darkpix percentage
-w = width of sub image (xsz variable in my python code)
-h = height of sub image (ysz variable)
-s = annulus spacing, or the space between the aperture and innermost annulus radius (ann_space variable)


If these are not specified, the program will run them with default values and create graphs of all the fits files entered preceding these arguments.


I haven't quite completely finished coding yet, so I won't be putting an example here today; however, I hope to get it done by tomorrow and have something more interesting to blog about. =]

I also learned about this neat thing called regular expressions. Not the Perl kind, but the Python kind. Basically regular expressions is a pretty powerful tool used to search for strings. You can set all sorts of restrictions to make sure that you are searching for the string that you want. I'm hoping to use regular expressions to create a list of all the .fits files the user has entered in command line so that I can make those graphs one by one.

Learned:
-Regular expressions
-How to use getopt(...)
To-Do:
-Finish program

Tuesday, July 12, 2011

Command Line Program

I've been working on making the contrast curve program into a program that can be executed directly from the command line.

Yesterday I attempted to at first to make a rectangle selector that allowed users to make a subplot and contrast curve by creating a rectangle on the FITS image. Unfortunately... the code was difficult for me to understand and since I couldn't find the source code, it ended up being easier to just abandon the whole idea.

But today I'm definitely making much more progress! This morning I read up on some different command line functions and how to write Python files that required input from a command line. The current problem I have is that it's hard for me to load the actual FITS file onto command line, so I can't show you any examples just yet (since I can't actually make my code work on a file).

Tim has been kind enough to teach me a bit about exception handling and the such. I'm assuming I'm doing something simple wrong and will continue to figure out why my fits file directories are "never found".

**Edit:
The fits file load now so that the programs will execute without error. However, I barely even have a chance to see anything load up before the program terminates again. Huh. I need to figure out how to make the program "pause".

Learned:
-Loading Python scripts in command line
-Using functions in Python that allows users to input info from command line
-Exception Handling
Problems/To-Do:
-Finish up the program.
-Figure out how to "pause" it.
-Test my contrast curve program against other contrast curves!

Friday, July 8, 2011

Fancy Contrast Curve Plotting

Exciting day!
Today was code clean up/make graphs look pretty day. And by make graphs look pretty, I sure do mean I made some awesome-looking, well organized graphs.
Here's an example:


>>> from startup import*
>>> import InteractivePhot
>>> im=phot2.Image(arr)
>>> InteractivePhot.start(im)
Please enter percent darkpixels (0-1):  0.95


After entering the percent darkpixels, the main FITS file image will pop up: 

It waits for a mouse click on a pixel (Clicking outside the image will unfortunately still give an error....)
I clicked near the center object, and this figure popped up: 


Pretty neat huh!? I'm so excited it turned out looking so pretty. 


Unfortunately though, there still needs to be a little tweaking/debugging, which I'll continue to work on next week. 


Learned:
-I can make beautiful graphs! :D
-How to align text inside a graph. (The text on the upper right corner of the Contrast Curve graph doesn't depend on on the coordinates of the graph. Instead it "clips on" to the maximum graph height and width.)
-How to create subplots
-How to customize tick marks and text
Problems/To-Do:
-Allow users to specify annulus space between the aperture and the innermost annulus radius (set to 0 as of now)
-Allow users to specify the dimensions of the cropped image (set to 40x40 as of now)
-Might want to try and make a dragging box that crops the image as opposed to a click that defines the center point... (that'll bypass the problem I have with allowing users to specify the dimensions of the cropped image)
-Somehow allow users to save the images?
-Somehow create a python file that'll run all my code in command line.
-I'm not actually sure that I'm plotting the right contrast curve (especially since the beginning of many curves tend to poke above 0 up to negative magnitudes, which seems a little odd...). I'll have to check my math eventually.

Thursday, July 7, 2011

Interactive Photometry

Nothing interesting happened yesterday. I read up a lot on how make methods that register mouse events. I also started to pseudo-code functions needed to make an interactive photometry program. Today I got a little farther.
-----
The object of this program is (as I briefly stated in my previous post) to create a simple photometry program that will plot contrast curves. More specifically, I will create a program that will allow the user to 'upload' a FITS file image. The user will then be able to click around on the FITS file. One the program registers a click, it will look around for the nearest point source and plot a contrast curve centered on that point source.

To achieve this, I have created two methods in the file InteractivePhot.py
def start(image): This method takes an Image object as an argument. It will plot the FITS image, and ask the user to specify parameters such as darkpix and ann_space. This method will be the basic startup function that the Interactive Photometry program will use.

The second method takes care of what happens when a user clicks on the FITS image.
def onclick(event): This function will (eventually) create a contrast curve centered on the closest point source. The user will be able to specify the size of the subplot image. The default size will be 40x40 pixels.

To allow these methods to use Image objects, I've created global variables to hold attributes of Image objects. (There might be better ways to do this since apparently global variables are a little iffy to work with)
-----
A lot of errors in my code relate to the way the coordinates are defined in the picture and the indices of the FITS array. Because of the way 2-D arrays work in python, the first index is actually the "y-coordinate" of the picture, and the second is the "x-coordinate". I switched some things around in my contrast curve codes to align everything with the image's coordinate system as opposed to the array system. This makes it easier for user to set arguments when they call a function.

Learned:
-How to use global variables
-More about events/handling in python
To-Do/Problems:
-Finish up my two methods in InteracticePhot.py.

Tuesday, July 5, 2011

Contrast Curve Plot

Created my contrast curve plot today:

Methods:
def contrast(flux1,flux2): Finds the difference in magnitude (contrast) between two fluxes. It returns this: -2.5*log10(flux1/flux2). This is not an Image class method; it's a function outside the class inside the phot2.py module.

def plotcontrastcurve(self,ann_space=0,fig=1):  Plots a contrast curve of a star in relation to the radial distance from the star. This method uses the function above. 

Ex: 
>>> from startup import* #This imports a bunch of stuff; most importantly, it creates an array arr from the fits file we're using. 
>>> im=phot2.Image(arr)
>>> sim=im.subplot(21,99,40,40) 
>>> sim.plotcontrastcurve()

Notice that the axes are flipped =] Also, note that I had to crop the original fits array. This is because the original array contains many point sources. In order for plotcontrastcurve(...) to work correctly, we need to crop the image so that it only contains one point sources; otherwise, the fitgaussian(...) function plotcontrastcurve(...) is using won't work correctly. 

Learned:
-More about classes/inheritance
-There's this neat little function in pylab called gca(). It returns the current axes, or creates a new axes if there isn't one. This means that I can plot and modify axes without making a figure container that messes with my utils.setfig(...) code! 
-Using this code to create a new pair of axes, I then used a function called invert_yaxis() to invert the labels on the y-axes. 
Problems/To-Do:
-The subclass I created handles all the header information in the FITS file; however, I would like to access it in the ancestor class in order to title the contrast curve I created with the name of the object. 
-Eventually, I'll have to start making a stand-alone program (I guess what I more accurately mean is one that doesn't require a Python Interpreter to run). This program will allow me to open a FITS image, click on/near a point source, and immediately return a contrast curve and some other information about that point source. 

Wednesday, June 29, 2011

Contrast Curves II

Hello!

I finally completed my contrast curve method today, but first:

One of the things Professor Johnson told me to do was to compartmentalize my code. If there are recurring actions I'm doing, I should just write a separate function for those actions. So today I redefined some of my original functions. It definitely makes my code look way neater, and will certainly save me a lot of typing later on.

Methods:

def annmask(self,cx,cy,ann_in,ann_out): Creates an annulus of 1's (other points outside annulus are set to zero)

def annulus(self,cx,cy,ann_in,ann_out): Creates an annulus of data (other points outside annulus are set to zero). This function just multiplies the array of 1's from annmask by the data values in the image.

def ann_meanbg(self,cx,cy,ann_in,ann_out): Finds the average bg noise of an annulus. This function uses the values taken from the annulus function and finds the mean of those points.

def ann_stdbg(self,cx,cy,ann_in,ann_out): Finds the standard deviation of brightness values inside an annulus. This function uses the values from the annulus function and finds the standard deviation of those points.

def phot(self,cx,cy,rad=5,ann_in=8,ann_out=10): This function was originally the aper_phot(...) function mentioned in this post: http://exolab-sandra.blogspot.com/2011/06/aperture-photometry.html. It now borrows heavily from the annulus(...) and annmask(...) functions.

Now, back to contrast curves.

def contrastcurve(self,ann_space=0): Returns an array of radii from star (rs) and corresponding sigma for annuli (lim). The annuli are two FWHM's wide and each element in rs is the average of the inner and outer radius of each annulus. ann_space is the space between the aperture and the innermost inner annulus radius. 

Example:
Using the same two images we used yesterday (with sim being the cropped image), we have:

>>> sim.contrastcurve()
(array([  3.43701872,   4.00985518,   4.58269163,   5.15552809,
         5.72836454,   6.30120099,   6.87403745,   7.4468739 ,
         8.01971036,   8.59254681,   9.16538326,   9.73821972,
        10.31105617,  10.88389262]), 
array([ 579.53765955,  191.44800124,  124.88768112,  52.21637822,
         29.54445459,   12.46020976,    8.8020379 ,   6.49041072,
          5.61804079,    5.24630489,    4.67690715,   4.30710904,
          4.28717292,    4.07445331]))


The first array contains the radii, and the second contains the corresponding sigma for each radius.

Learned:
-More about how fitgaussian(...) is defined.
To-Do:
-Have fun in Yosemite! (That's right, I'm going on a trip this weekend :D)

Tuesday, June 28, 2011

Contrast Curves (Monday-Tuesday)

Yesterday wasn't too exciting (as in I thought more than twice the work I produced), so I didn't blog.
First of all, I fixed up my point sources function. Now there are less points that are likely point sources, but there are still a few too many. I also started working on a contrast curve function yesterday, but didn't get far on the actual coding.

I'm still working on the contrast curve function(s) today... and will tomorrow as well.
-----
The idea is to create a contrast curve that plots contrast (basically magnitude of flux against some background noise) versus radial distance away from the star's center. The center is defined by fitting a Gaussian to the data and finding the x and y coordinates of the peak. The actual gaussianfit(...) function is found here: http://www.scipy.org/Cookbook/FittingData.

Now, in order to plot the magnitude and account for some differences in background noise near the star, we need to find annuli around the PSF of the star, and figure out the "background noise" in each annulus. I say this with quotes since a bit of the star's flux will leak out into this region and affect the overall flux level. We need many different concentric annuli since the flux varies in this region.

The annuli will have a width of two FWHM's to get a complete curve of other stars' PSF (if there are other stars in the picture). I will also calculate the background noise of overlapping annuli so that if we do see another star in the picture, we can hopefully get a PSF that isn't cut off by at least one annulus.

Anyways, I made a function called ann_meanbg(...). Given a center (cx,cy) and inner and outer radii (ann_in, ann_out) it can find the mean background flux of an annulus defined by the center coordinates and radii. Since there might be times where the annulus contains another fainter star that significantly increases the background flux, I have included a darkpix factor that will only take the darkest darkpix percentage of pixels in order to ignore the fainter star. 


def ann_meanbg(self,cx,cy,ann_in,ann_out,darkpix=1): Finds the average background noise of an annulus.

I also started a contrastcurve(...) method. It is supposed to return one array of radii that is the average of ann_in and ann_out, and one corresponding array that contains the annulus background noise at each radius. 


A third function I made is called subplot(...):
def subplot(self,cx,cy,xsz=50,ysz=50): Makes a cropped image from the original image with cx, cy as center coordinates and with a width of xsz and height of ysz.

Finally, because of the way my original class was defined, I had to completely redesign it so that contrastcurve(...) would be able to be a function of the class. Now the Image class takes the data array of the FITS file as opposed to the FITS file itself. I have defined a subclass called GuiderImage(Image) to take care of the FITS file header. 


Example: (assuming we have an Image (the new class) object called im)
>>>im.show()
This returns:
>>>sim=im.subplot(21,99,40,40)
>>>sim.show()
This returns:
>>>a=u.fitgaussian(sim.data)
>>>a
array([685.03745756, 20.29637509, 19.94028319, 27.96468548, 27.93414018])
>>>sim.ann_meanbg(a[1],a[0],5,8)
557.72357723577238

Learned:
-A little bit about the Gaussian/Bi-Variate Normal Distributions
-Python subclasses
-The use of lambda in Python functions
-How to dezero an array (take out all the zeros in an array) efficiently - using a[a!=0]
-Background about Contrast Curves
Problems/To-Do:
-Complete the contrastcurve(...) function
-Figure out how to find the difference in magnitude and plot the contrast curve