Glitter

I have solved some more issues with the Double Glitter screen mapping. First, the positives. Below is a more reasonable image where, upon looking at individual pieces of glitter, these screen mapping points seem reasonable.

There is a more dense cloud of points near the middle of the screen, but there is a decent spread and no points are off the screen.

With that said, there is still quite a few glitter pieces that are not being mapped correctly. Specifically, there is about 1800~ pieces of glitter not being mapped correctly out of 5300. Looking at certain pieces of glitter that are failing to map, I am seeing something like the image below:

This is an image of mask. The yellow shows the low frequency masking, and the X,Y location is what the high frequency mask says the point should be. Since that X,Y is not in the yellow, the final mask is all 0's so no mapping is made for that glitter piece.

I have fixed the comparing of floating point values issue, but for this current problem, perhaps revisiting color correction is appropriate. Either that or simply just say that if you are "close enough" to the yellow square, count it.

Starcraft

I have successfully created a very basic pipeline that can take images from a live stream and obtain their map images to use in my prediction model. I used the Twitch API to obtain a preview image of the stream and pass that onto a python script that resizes the image and predicts using my model. For example, here is a picture of a Korean Starcraft Player streaming his game!

I am interested in exploring running a program on the player's computer to obtain more data easily (such as how many resources that player has or that player's apm), as other Twitch extensions do something similar. For the moment, I will refine this pipeline and once it is at a more stable state I will revisit my model and explore LSTMs more as a potential new model. In addition, I would like to explore using more data (if I can get it) besides just the map image.