A bit late in posting it, this entry will provide a brief description of the sparkly happenings of Glitter Summer Week 4, in which we did in fact complete a useable glitter characterization! As described in the week 3 post, there were a couple known issues with the first full characterization. Addressing those and spending a couple days debugging gave a full characterization I trust (within some reasonable error).
Recall that a glitter characterization is knowledge of the *location* of many glitter specs on the sheet as well as the *direction* that they are facing (their surface normals). How do we test a glitter characterization? We shine a light on the glitter, take a picture, and then measure and record where the light, camera, and glitter were positioned relative to each other. Then, our glitter characterization allows us to trace rays from the measured camera position, reflect them off all the specs that were bright in the image, and then see if they intersect roughly where the light was positioned. Here is a diagram showing just that.
Hurray! Many of the specs really do go to the light position on the (green) monitor as we hoped! Some don't, and this could be due to many different sources of error. (If you're wondering, the stubby red lines are the surface normals of the specs we're reflecting the light off.) Ok, how else can we test the characterization?
Well, for the same image as before, we should be able to trace rays from the known light position to ALL of the glitter specs we characterized. Then, we should be able to measure which ones end up reflecting really close to the known camera position. The specs that have their reflected rays go really close to the camera location should be the specs that are sparkling in our image! If I just show the whole glitter image, you can't really make out anything since there are so many tiny sparkles, and so I zoom in to show smaller sparkle patterns. On the left is my predicted pattern (just white for sparkle and black for no sparkle), and on the right is the actual image we captured.
Pretty nice, isn't it? Sometimes we didn't do as well though. Here is an example where we predicted four sparkles but only two appear in the actual image captured.
In a few spots in the image, the predictions are quite bad. Here's an example of that.
We did some other testing as well where the camera, light, and glitter assumed totally new positions relative to where they were during the characterization. This allowed us to make sure we weren't only getting the answers right because of some weird glitch in the geometry our code was depending on.
Finally, there is one more game that we can play, arguably the most important. Remember that the overall goal of the summer is to use glitter to calibrate cameras with a single image. A key part of camera calibration is finding the location of the camera in 3d space. So, we illuminated the glitter with a known light source, traced the rays from light source to shining specs in the image and back into the world, and then we saw that many of them really did intersect where the camera was positioned. In this diagram, we show just the reflected rays (from the sparkling specs back into the world).
Week four ends here, where we quickly implement the code to estimate camera position from these reflected rays, and we get an answer (just for this single first test image) within 1 centimeter of the correct position. Very exciting!