Thursday, August 18, 2011

Contrast Curve Function 2.0 III and Point Source Detector


So it turns out that actually, because of various factors like dithering and such, my contrast curves are probably correct.
...
Yay! That's actually pretty exciting stuff.

However, now my point source detector is far too insensitive. I am trying to search for regions that have point sources, and then use the fitgaussian(...) function to detect those point sources. However, the fitgaussian(...) runs very slowly over large arrays and doesn't seem to be all that effective at pinpointing the point sources. Even after I've put down an artificial point source, it still doesn't detect anything in any of the regions. Perhaps it's the way I've implemented it, and I just have to keep working towards a solution.

Learned:
-What dithering means.
To-Do:
-Find a way to make my point source detector work correctly, whether by using the fit gaussian function, or some other function.

Tuesday, August 16, 2011

Contrast Curve Function v2.0 II

I finished the new contrastcurve(...) function today.


def contrastcurve(self,ann_space=0,darkpix=0.95,thresh=5): Returns list of radii and corresponding noise (thresh*sigma+mean background noise) and point sources (if any). It detects point sources by fitting gaussians over regions that have, on average, greater intensity than the noise and searches for peaks in those regions.


And it........... still doesn't work right. 


Here's an example though, of what the contrast curve looks like for KIC109:

Interestingly, however, if I calculate magnitude using log instead of log base 10, I get this result:



This agrees much more closely with Justin's data. Weird! I still haven't figured out why. 


By the way, Justin's data is this:
KIC109   3.7, 5.6, 7.8, 8.6, 8.6  at 0.25'', 0.5'', 1'', 2'', 4'' respectively.
(This is for thresh=3; mine are produced with thresh=5, but I ran the code again and this barely makes any difference). 


To-Do:
-Start testing how my code behaves around images with bright point sources (yes I said I would do this today, but then the contrast curves looked funny...) while I try to figure out why my contrast curves are behaving this way.
Learned:
-I actually learned something interesting today: If you have a function that returns multiple things in an array, it's much more efficient to do thing1, thing2, thing3=function() 
as opposed to thing1, thing2 = function()[0], function[1] even if you only want to save thing1 and thing2. This is probably because in the first case, Python will run the function once and save the corresponding variables, whereas in the second case, Python will run the function once, save the 0th element, and then run the function again and save the first. So I guess that ultimately what I learned is that if you have a function that takes a while to run, it's better to use the first case to save time, but if you have a function that's relatively fast for large amounts of data, but that returns a bunch of stuff, it's better to use the second case to save memory. If you have a slow function that returns lots of data.... well then good luck to you.

Monday, August 15, 2011

Contrast Curve Function v2.0 I

Today I finished most of the contrast curve function I talked about in my previous post. I haven't gotten it to return a list of detected point sources yet (it only returns radii and respective 5*sigma as of now), but it should integrate relatively well into all the other code I've written so far.

I have also been given a rough timeline of what I'm supposed to do for the rest of the summer (yes, although officially ten weeks, my project will continue till the end of summer/beginning of the fall term). So here it is:

Goals:
1) Rewrite my code and try and get the same results that Justin did.
2) Start using my code for all the fits files I have.
3) Detect problems and check them.
4) Repeat as necessary (This is not a fun step).
5) Make my code more user-friendly.
6) At some point start putting contrast curves up on a website.
.
.
.
n) Somehow help with the writing of a paper.

To-Do:
-Tomorrow I'll have to figure out how this new contrast curve function will actually detect point sources. Since we're talking about regions now, I might have to also change the way the program points out detected sources to the user.
-I also have to start writing code so that I can upload directories of .FITS files into the program. That way, the user won't have to spend a long time typing in the names of all the .FITS images.
-At some point I'll have to see what I can do about adapting my program to various arcsecond/pixel ratio. That is, I want my program to be able to recognize a type of image by its file name and automatically assign the correct arcsecond/pixel ratio.
-Finally, if possible/not too aesthetically ugly, I should try and make all the pictures axes display units in terms of arcseconds instead of pixels.

Friday, August 12, 2011

New Implementation

I talked with Tim and Justin about my code. The problem seems to be that in my current program, I am overestimating the amount of background noise since I'm calculating noise by averaging the flux from individual pixels. What I should do instead was to average noise from little regions and calculate the noise from the average of those regions. It seems like I'm just working with slightly bigger pixels, but by first averaging the flux from each region, I am able to smooth out my background and ultimately decrease the noise in each image.

So now, I have to code a different approach to finding point sources. It will still use much of what I have already written, but it's definitely the biggest change yet to my code.

Here's a rough outline of what I'm about to do:
I will partition each annulus into little circular regions of data. I will then average the flux in each region, and then find the standard deviation from the regions. This will be used to create sigma. The background mean will also be calculated from the regions. Detection of a point source apart from the main source will determined by if the a region has an average intensity greater than thresh*sigma. My program might pinpoint the specific location of the point source less precisely, but at least it won't detect fake point sources that turned out to just be speckles. The contrast curve will still be created in much the same way. It's just the way I determine background noise that will change the most significantly. 


I'll start working on this next week. 

Thursday, August 11, 2011

Sanity Check

I've been debugging my program for the last few days without much success. I've sent some preliminary testing results to Tim, but the results are definitely not good.

Here are two main problems I noticed:
-The point source finder is too sensitive. It detects almost anything that is greater than thresh*sigma. Unfortunately, there are many pixels in the picture that seem rather "grainy" and are registered as false point sources. I need to somehow eliminate these false point sources.
-The range of delta mags (contrast) for the contrast curves are way too small. The delta mags themselves are also high values, often hovering just below 0. This indicates that my program is not finding too much contrast within the annuli of the star compared to the central flux of the star, which seems intuitively wrong. However, this is most likely caused by the small size of the cropped image. The small images size makes my program calculate the background noise as part of the annulus of the star, which is incorrect as there is light pollution from the star. The problem is somewhat fixed when I increase the size of the cropped image; however, compared to Justin's results the delta mags still seem too small.

Professor Johnson suggested I do some sanity checks on my math yesterday, and I've been trying to figure out what really is going on with the math behind the contrast curves.

I'll post some more if I find anything interesting, but for now, it is just alone time for me and a buggy program.

Tuesday, August 9, 2011

Code Testing

Another colleague, Justin, sent Tim and I a MATLAB fitswriter file, which turns matlab .fig files into .fits files.

I have been setting up Matlab and using it to convert images into .fits files. I am then able to test my code on these new files, and compare my results with those of Justin's. Unfortunately, they don't seem to match up. The images that Justin sent me also seem to focus on objects with large FWHM's, which ultimately means that I obtain less points for my contrast curve because of the way I've implemented my code.

I am currently trying to compile a .pfd document containing the contrast curves I obtained using my code, and comparing my data to Justin's data. Hopefully this will give me a clearer indication of what the differences are between our results, and how I should change my code.

Friday, August 5, 2011

Saving Text Files and Figures

Today I finished a function that will allow me to save text files of some important data associated with the contrast curve plots I made.

When a user saves a figure, an associated text file will save the lists of data returned from phot3.contrastcurve(...). That is, it will save a list of points used in the creation of a contrast curve as well as any point sources the program may have found in the annuli of the main star. The text file will also save the center coordinates of that star.

The text file will save to the name of x-y_fitsfilename-results.txt, where x,y are the center coordinates, and fitsfilename is the name of the fits file.

Here's an example of the kind of text file it will save:

C:\[Your path here]>IPhot CK00012_snap_20110412.fits
CK00012_snap_20110412.fits
Press Q/q to quit, or any other key to load next image.

Clicking on point (22,100) and pressing W will save this picture... 
...as well as a .txt file containing this information:


Center Pts
(22, 100)
Coordiates (Radius,Contrast)
[(1.0310396072149, 1.2019494856629946), (1.2028795417507165, 1.8693450269793654), (1.3747194762865333, 2.1077248753625968), (1.5465594108223499, 2.3778325825061946), (1.718399345358167, 2.4598854750345436), (1.8902392798939833, 2.5393921380320639), (2.0620792144297999, 2.5689754466690302), (2.2339191489656169, 2.5851800258224134), (2.4057590835014335, 2.5950414542085398), (2.5775990180372501, 2.5998551889539634), (2.7494389525730667, 2.6055135477309928), (2.9212788871088833, 2.6096344474456359), (3.0931188216446999, 2.6104568644975972)]
Additional Point Sources at Each Radius (if any)
[[], [], [], [], [], [], [], [], [], [], [], [], []]


Saved to C:\Documents and Settings\Administrator\Desktop\CC22-100_CK00012_snap_2
0110412.png
Saved to C:\Documents and Settings\Administrator\Desktop\22-100_CK00012_snap_201
10412-results.txt
q
Quitting program...

C:\[Your path here]>

-----
Note that I also created a little compass on the lower right corner of each Image object. =]

To-Do:
-Somehow scale the axis ticks of the subplot images to arcseconds as opposed to pixels.
-Maybe at some point figure out how to load directories of FITS files into my command line as opposed to a list of file names (which are very tedious to type).

-----
EDIT:
I have changed the output of the text file so that it no longer returns a list of lists of point sources. It will simply return a list of point sources, or an empty list if there are none.