This post is more of a re-cap of where we are in the pipeline, and a gut-check on what is left to be done. It takes about half a day in total to complete steps 1-5 end-to-end.
Pipeline so far:
- Capture Pictures
- four directions (horizontal, vertical, diagonal positive, diagonal negative) for a base location of the glitter and some moved location of the glitter (8 sets of images total)
- Extract Centroids
- using an intensity threshold (150) and some distance threshold (2) - 2 centroids are considered the same if they are less than the distance threshold away from each other. A cluster of pixels is considered a centroid if the pixels are all above the intensity threshold - do this for all 8 sets of images
- Reduce Centroids
- look at the intensity plots for each centroid, and any centroid with more than 1 intensity peak is thrown out (these could be pieces of glitter that are bent, or two pieces of glitter overlapping) - do this for all 8 sets of images
- Match Centroids Across Directions
- find the centroids that appear 'lit' in some frames in all 4 scan line directions - should be left with two sets of centroids, one for the base location of the glitter and one for the shifted location of the glitter
- base: 30,805 centroids
- shifted_up: 11,818 centroids
- Screen Map
- solve for the point on the screen which causes each centroid to be 'lit' using the intersection of the scan lines which produced the highest peak of intensity for each centroid
- base: 29,327 centroids
- shifted_up: 10,677 centroids
- (TO DO) Measure Physical Setup
- convert all screen locations and centroid locations to real-world 3D coordinates
- (TO DO) Compute Surface Normals
- most likely in the 'base' location of the glitter using real-world coordinates
- (TO DO) Match Centroids Across Glitter Locations
- should be similar to the process for matching across scan line directions
- I should do this sooner rather than later because we need to see how many centroids survive this step and actually are seen in both sets of images
- (TO DO) Surface Normal Error Analysis
- using the surface normals from the base location of the glitter, estimate the screen map for the shifted centroids and compute the error to the actual screen map (or something like that)
- ...more stuff...
- (TO DO) CALIBRATION!
These images below show the screen mappings of a few centroids from the base location of the glitter:
This last image shows an example of a centroid that has a 'bad' intersection of lines. The threshold being used to determine if the point of intersection is 'bad' is whether it is within 2*gaussian standard deviation of the line is furthest from or not. There are also some cases where the point of intersection has a negative value (when either the horizontal or vertical lines are on the edge) - there are about 15 such centroids (not pictured)