We modified our code a bit, to use ResNet without the softmax layer on the end. We then did our 'over training check', by training on a minimal number of triplets, to ensure the loss went to 0:
We then tried a test with all of our data (~500 images), to see if our loss continued to decrease:
One thing we found interesting is that when we train with all of our data, the val_loss converges to the margin value we used in our triplet loss (in this example margin = 0.02, and in other examples where we set margin=0.2, val_loss -> 0.2).
We believe this could be troubling if you look at this equation for triplet loss (https://omoindrot.github.io/triplet-loss):
L=max(d(a,p)−d(a,n)+margin,0)
It appears to me that the only way for L = margin is if the distance from the anchor to the positive is the exact same as the distance from the anchor to the negative, whereas we want the positive to be closer to the anchor than the negative.
Dr. Pless recommended that we visualize our results to see if what we are training here actually means anything, and to use more data.
We set up our data-gathering script on lilou, which involved installing firefox, flash, and selenium drivers. The camera that we were gathering from before just happened to crash, so we spent some time finding a new camera. We are currently gathering ~20,000 images from a highway cam that we'll use to train on.
After this we will visualize our results to see what the net is doing. However we are a bit confused on how to do this. We believe that we could pass a triplet into the net, and check the loss, but after that, how could we differentiate a false positive from a false negative? If we get a high loss, does this mean it is mapping the positive too close to the anchor, or the negative too far from the anchor? Do we care?
Is there some other element to 'visualization' other than simply looking at the test images ourselves and seeing what the loss is?