Laser Pointer Tracker

A program to detect the foreground target (laser spot) using histogram analysis and frame differencing, and lock on target by overlaying a crosshair marker on the center of the detected spot.

Example: Object tracking

Input video

in_video

Output video

out_video

build and run

To perform the histogram analysis and set an appropriate thresholds, run image_intensity_histogram.py using Google colab for conveience (or use python 3 and required python modules locally). Change line #12 to specify the path of the of the frame to be inspected.
For object detection:
1. Set threshold per your findings in the preivous step. See line #23 in lock_on_target.cpp.
2. use the attached makefile to build the source code as follows:
```
 make
```
3. run the executable and provide the input video as a command line argument. For a video file name in_video.mpeg that has been sliced into individual frames (see rgb2gray_slicer) in pgm format and saved under the folder named PATH_TO_FRAMES_FOLDER, use:
```
 ./lock_on_target ./PATH_TO_FRAMES_FOLDER
```
4. Consider usinf ffmpeg to stitch the output frames back into a single video. E.g., ffmpeg -start_number 0 -i frame_out%d.pgm -vcodec mpeg4 out_video.mp4.

Example: Histogram analysis

The analysis shown below exposed the difference between full-saturation level and the near-full-saturation levels. The difference is large enough to allow us to solve the problem without the need for edge enhancement. This is based on this example only, and edge enhancement methods may be required for other images (e.g., RGB scale, with cluttered background.. etc).

The first frame was analyzed per-pixel for each pixel’s intensity level to determine an appropriate threshold. A combination of OpenCV, Matplotlib, NumPy in Python 3.7 (using Google Colab) were used to perform the preliminary analysis. The first frame in gray scale (see original file at: q5/frames_out_chan_0/frame_out0.pgm):

histogram_analysis_frame_used

Since OpenCV in Python stores each image object (equivalent to Mat object in C++) as a NumPy array, using NumPy and Matplotlib, the full array of pixels was easily plotted as a histogram.

histogram_full_spectrum

However, note that due to the count of pixels with intensity values of ~0, we are not able to see any other bins easily. One way to get around this is by looking at the upper half of the saturation level (128-255, as opposed to the full range of 0-255).

histogram_upper_half_intesity

Alternatively, we could limit the top of our y-axis to a reasonable value (~300). Note that this will introduce some cropping in the bar’s height in the lower saturation levels.

histogram_lower_frequency

To further isolate the area of interest to us for the purpose of solving this question, we can limit both the x (240-255) and y (0-300) axis.

histogram_lower_frequency_upper_half_intensity

Now, let’s inspect the exact count of the most frequent saturation levels (top 15 shown below).

histogram_stats

From the analysis shown above, setting our thresholding function as follows seems reasonable:

histogram_analysis_threshold_function

And for our histogram, this would look like the shaded red region as shown below:

histogram_region_of_interest

Future work

Match frame rate 100% in real-time.
Enable frames stitching from the object detection program or via a bash script.
Incorporate all programs to be ran consecutively using a BASH shell script. Wait for user input between steps. Share threshold value with program using a pipe.