I believe we have found a clue in the missing sources, we have plotting some interesting values and have created the following plot:

In the redshift range of 3.2 - 3.5, the missing sources are showing up in the center and the bottom left corner.













This is very peculiar, but we have concluded that this may not be a result of our code.  It may be the result of the instrumentation of our HETDEX project.  At this point, there may be nothing (programming-wise) we can do to fix this.  We'll have to look at the different parts of the cure repository more closely and find out what the problem is.

This week Maverick finally had the bus errors resolved (thank goodness it didn't have to do with our code).  This is extremely helpful!  We can run larger batches of data points on Maverick without running up on limits.

I've also been able to create some more intuitive plots from the larger batched of data we've gotten.  Emily told me to experiment with the chi2 fit value as a criterion for the detection algorithm.  I've generated the following plots thanks to Dustin:

 These first two plots in between are the controls.  I made sure the seed worked and that I got the exact same results for the same sets of data points.

The next plot to the left is with the chi2 value upped from the original 1.5 to 1.8.  It is a very small difference, but the largest values jumped from 97.6 to 98.2 in the number of recovered sources versus input line flux, a total of 50 more recovered sources out of the 32000.  This is an increase in 0.15% (virtually nothing).





These last two plots are with the chi2 value at 2.0 and 100,000 respectively.  They are identical.  However, note the difference between the 1.8 value and 2.0 value.  There were only 10 more recovered sources from the previous chart.  Overall, the chi2 value is not the issue with the missing sources in the plots.  We are aiming for 99% accuracy in picking up sources around the middle of the bar plot on the left and the chi2 value isn't the limiting factor.



Further analysis will be made to find out what is happening to our missing sources.

After debugging the python code and making changes to the find detection executable, we have begun to run into some more bus errors.  Some reasons for this may be that we do not have enough space on Maverick to run the number of tasks we have.  Another may be because TACC is busy with the other programs running things on Maverick throughout the day.

I have also started to get some bus errors as well.  Once the bus error problem is solved, we can continue testing the robustness of our detection code.  I will look into the chi2 values and see if that is the limiting problem with why we aren't detecting as many sources.

I have gathered my first set of data and created a few plots.  The first plot that I experimented with is the output flux (in electron) and the input line flux (in ergs/sec/cm^2):

The next plot is detected x-value versus input x-value:

The following plot is detected y-value versus input y-value, notice that this one is very similar to the previous plot because the detection parameters for both the x and y values are the same in the detection algorithm:

This final plot is detected z (redshift) versus input z:

Notice that this final plot has a much thinner line than the previous two.  This is due to the fact we only accept an x or y value that is within plus or minus 0.25 arcseconds, and we only accept redshift (z) values that are within plus or minus 0.01 arcseconds.  More plots should come up in the following week with more intuitive and helpful data.

We were able to find the bug and the reason for the bus errors.  In our python file that distributes random sources to our frame, we discovered a breakdown in the if statement that would cause bus failures and further problems.

We fixed this issue: the bitwise '&' operator in python versus the boolean 'and' operator.  This is a small fix, but a necessary one that helps with fresh eyes to our code.

On Thursday and Friday of this week, Dustin and I ran some tests to make sure we weren't getting any more bus errors on Maverick and tried to make sure that we weren't having any code issues with the cure repository.  Further results to be found next week.

We had our first meeting as a whole group today.  Over the course of a couple of weeks, Dustin has been able to debug the makerand_z_flux.py file, so we are no longer getting bus errors on Maverick.  I have been helping him test the scripts and make sure they run to completion without the issue of running into our memory usage limits.  This is a step to start generating plots and analyzing the data we find in the blank images that we create.

If you want to know more background of the HETDEX project, just click on the hyperlinked text.  This will give you a more detailed outline of the conceptual tasks at hand for the project and will ease you into astrophysics better that I can describe.

After my first week in training, I have really enjoyed being in Austin.  The research I will be conducting will be with Dr. Karl Gebhardt, Emily McClinden, and Dustin Davis in the Astrophysics department.  We will be primarily testing the robustness of the detection algorithm for the HETDEX unit and we will be creating plots based on the data we find in order to determine the best parameters we need to use to maximize detection of the point sources we provide to the data frame.

The main goal of this project is to test theories regarding dark energy, the force that drives our universe to expand faster and faster as it ages.  By observing this data, we can determine which theories on dark energy can be thrown out and which ones can be seen as plausible.  Our task is to narrow down these theories and gain a better understanding of this force which dominates our universe.

Austin is a beautiful city and we had the pleasure of visiting the state capital today!