I believe we have found a clue in the missing sources, we have plotting some interesting values and have created the following plot:

In the redshift range of 3.2 - 3.5, the missing sources are showing up in the center and the bottom left corner.













This is very peculiar, but we have concluded that this may not be a result of our code.  It may be the result of the instrumentation of our HETDEX project.  At this point, there may be nothing (programming-wise) we can do to fix this.  We'll have to look at the different parts of the cure repository more closely and find out what the problem is.

This week Maverick finally had the bus errors resolved (thank goodness it didn't have to do with our code).  This is extremely helpful!  We can run larger batches of data points on Maverick without running up on limits.

I've also been able to create some more intuitive plots from the larger batched of data we've gotten.  Emily told me to experiment with the chi2 fit value as a criterion for the detection algorithm.  I've generated the following plots thanks to Dustin:

 These first two plots in between are the controls.  I made sure the seed worked and that I got the exact same results for the same sets of data points.

The next plot to the left is with the chi2 value upped from the original 1.5 to 1.8.  It is a very small difference, but the largest values jumped from 97.6 to 98.2 in the number of recovered sources versus input line flux, a total of 50 more recovered sources out of the 32000.  This is an increase in 0.15% (virtually nothing).





These last two plots are with the chi2 value at 2.0 and 100,000 respectively.  They are identical.  However, note the difference between the 1.8 value and 2.0 value.  There were only 10 more recovered sources from the previous chart.  Overall, the chi2 value is not the issue with the missing sources in the plots.  We are aiming for 99% accuracy in picking up sources around the middle of the bar plot on the left and the chi2 value isn't the limiting factor.



Further analysis will be made to find out what is happening to our missing sources.

After debugging the python code and making changes to the find detection executable, we have begun to run into some more bus errors.  Some reasons for this may be that we do not have enough space on Maverick to run the number of tasks we have.  Another may be because TACC is busy with the other programs running things on Maverick throughout the day.

I have also started to get some bus errors as well.  Once the bus error problem is solved, we can continue testing the robustness of our detection code.  I will look into the chi2 values and see if that is the limiting problem with why we aren't detecting as many sources.