With Hong and Abby's help, we were able to get the heatmap up and running on our model. Here you can see the similarity visualization across a couple of our classes as time passes (Left image is always the first image in the class, the right image advances timewise):
class80:
class120:
It appears as if these blobs are following groups of cars/ specific cars. Which would mean that it must be learning something about cars themselves. Its kinda hard to see how specifically it is paying attention to certain cars when they are close together, due to the fact that the resolution of the heatmap is 8x8.
One concern I have is that it only seems to care about some cars and not others. Would anyone have any insight as to why? This could mean that, if we were to use the embedding to identify cars for our amber-alert problem, we could get 'unlucky' and miss the car we actually care about.
Next week:
- Do this visualization on our training data too
- Figure out our next steps
