Classic CV + Object Tracking

Neato Golf

The goal of the project was to design and implement an autonomous robotic system capable of locating a golf ball within a predefined area and pushing it onto our target. We wanted to explore how computer vision, object tracking, and motion planning could be combined to complete a task. However, we wanted to experiment with a camera that is not attached to the mechanism that is directly interacting with the world; our camera will be placed in an independent location that oversees the world. This reduces complexity with needing to track changing transformations from camera frame to world frame. We can see this method of computer vision being used in CTVs working with robots (for example, in factories).

Some skills in the project are:

ROS2 project architecture
Classical CV techniques
Simple Online and Realtime Tracking

github: https://github.com/swisnoski/neato_golf

Brief methodology for this project is shown below. Full writeup is in Github README.

Simple Online Realtime Tracking (SORT)

SORT (Simple Online and Realtime Tracking) is a lightweight multi-object tracking algorithm that links object detections across frames. It begins by detecting objects and initializing a tracker for each new detection. Using a Kalman Filter, SORT predicts each object’s next position, then compares predictions with new detections using IoU overlap. The Hungarian algorithm assigns detections to trackers based on best matches. When a prediction and detection overlap sufficiently, the tracker is updated; if not, unmatched trackers age and may be removed. SORT is fast, simple, and effective for real-time applications where accurate short-term tracking is needed.

Classical CV

This pipeline uses classical computer vision techniques to detect and analyze a robot’s pose and shape. First, the image is converted to grayscale and thresholded to isolate bright, high-contrast regions. Morphological operations clean noise and refine the robot’s silhouette. Contour detection extracts the object’s outline, and the largest contour is selected as the robot. From this contour, geometric features such as the centroid, orientation, and shape boundary are computed. Edge detection further refines the outline, enabling accurate identification of the robot’s front edge or heading. Altogether, these steps provide a fast, reliable, learning-free way to track and measure the robot.

Heading Calculation

This method estimates a robot’s heading using contour geometry. After extracting the robot’s outline, the front edge is identified as a short line segment with endpoints. The slope and intercept of this detected edge are computed, and a perpendicular “heading line” is derived from it. Using the robot’s centroid as a reference, the intersection between the heading line and the detected edge is calculated by solving the two line equations. The vector from the centroid to this intersection gives the robot’s forward direction. Finally, the heading angle is obtained using the arctangent function, θ = atan2(yi − yc, xi − xc), producing a stable orientation estimate.

Setup

SORT Demo

Full Project Demo