DHS Undervehicle Scanner
Project Manager
Sep - Dec 2020
The Client.
The U.S. Customs and Border Patrol division of the Department of Homeland Security.
The Problem.
Collecting under vehicle imagery using conventional means is very tough. Either the border patrol agent is going around the car with a mirror or selfie stick, or a mirror-camera system is used that causes image distortion. The hardware team opted to use a direct-capture camera system (with cameras lined up), which is great for image quality, but terrible for composite image creation. In the case of the single-camera + mirror system, it's 1 image, just very distorted. In the case of the multi-camera direct capture system, there is a multitude of cameras that capture "slices" of the vehicle's underside.
If you've ever used "pano" mode on the iPhone, you know that a sequence of images is captured and then "stitched" together as a post-processing step. As you would guess, that means to create a composite image of the underside of a vehicle we have to "stitch" the images together.
We tried to use the conventional SIFT algorithm to stitch the images together, but it faltered. With that, we turned to AI.
The Solution.
Use AI to predict transformations between images and stitch them together.
The Process.
The first thing that popped into my mind when the SIFT algorithm failed was to use AI to predict the transformations (mostly vertical and horizontal translation) between image tiles.
SIFT as well as other image stitching alfgorithms are completely manual, whereas AI is automatic. We don't have to sit and bang our heads against the table to see what'll work; Rather, we collect data, show it to a capable model, and let the model figure out how to connect the dots.
Therefore, we got a small dataset together, fine-tuned an AI model on it, and boom: Best under-vehicle image stitcher in the world.
The Result.
An incredibly capable AI-based image stitcher that generalized well across real world data.
