Based on last weeks discussion, I am testing the accuracy of using purely in game data that I can obtain both from replays and from a live stream. This includes Resources (Minerals and Gas), Current Population, Current Max Population, Race, GameID (to track data across one game), Current In Game Time, and finally who wins. No image data is being processed here.

From the replays I collected 53k snapshots over 406 games. Removing snapshots that had a current time of < 60 seconds (because the beginning of the game is not interesting), I was left with 48k snapshots.

Next I spent time trying to find a model that works well with the data.

Running a grid search with SVM, I found the RBF kernel with C=1000 and gamma =0.001 to be the best parameters. I achieved an accuracy of ~66%, but there is waaaaaaay too much variance at the moment. For example:

Running AdaBoost with 100 estimators did much better with ~75% accuracy:

Note:

Both graphs were created with no normalization of data (scaling ) and with keeping the column GameID in the dataset for training.

Questions:

• With the above models, I am including the GameID, such that a single game with say 10 snapshots will all have the same GameID. Without this column AdaBoost performs about 10% worse. What is the justification for keeping other than "it gives me better accuracy"?
• Currently nothing in the data is normalized. Should values such as resources or population be normalized over the dataset? Should it only normalize per game?

For Next Week:

• Keep working on the models
• test model over one game and see how the prediction does
• Compare to visual data classifier and look into how to possibly combine them together.

Starcraft

When retraining my model on a full set of 55k images of Map Data, I was still preforming badly at around 58% when a model has full vision of the map from both player's perspective. Further inspection in the images reveals that the screenshot capture library seems to be inconsistently failing for patches of time. For example, it will "capture" the same image file for 1000 images worth of data. So I am checking to see if that is on me more something in the library.

Additionally, I am testing how the model does at various times in the game to determine if it fails more in beginning of the game or is it later. But I should have a better model when the data is correct 😉

Glitter

I successfully have a script that can take an image (or batch of images) and display them onto the screen and have the Nikon camera take pictures of them! My current feature request I am working on is being able to set certain camera settings when taking those pictures, such as ISO, focus, zoom, etc. Any other suggestions are welcomed!

Whenever the new Nikon camera comes in, I can properly configure the script to handle the camera settings for that one as well (assuming the camera settings are in a slightly different format).

Glitter

I have solved some more issues with the Double Glitter screen mapping. First, the positives. Below is a more reasonable image where, upon looking at individual pieces of glitter, these screen mapping points seem reasonable.

There is a more dense cloud of points near the middle of the screen, but there is a decent spread and no points are off the screen.

With that said, there is still quite a few glitter pieces that are not being mapped correctly. Specifically, there is about 1800~ pieces of glitter not being mapped correctly out of 5300. Looking at certain pieces of glitter that are failing to map, I am seeing something like the image below:

This is an image of mask. The yellow shows the low frequency masking, and the X,Y location is what the high frequency mask says the point should be. Since that X,Y is not in the yellow, the final mask is all 0's so no mapping is made for that glitter piece.

I have fixed the comparing of floating point values issue, but for this current problem, perhaps revisiting color correction is appropriate. Either that or simply just say that if you are "close enough" to the yellow square, count it.

Starcraft

I have successfully created a very basic pipeline that can take images from a live stream and obtain their map images to use in my prediction model. I used the Twitch API to obtain a preview image of the stream and pass that onto a python script that resizes the image and predicts using my model. For example, here is a picture of a Korean Starcraft Player streaming his game!

I am interested in exploring running a program on the player's computer to obtain more data easily (such as how many resources that player has or that player's apm), as other Twitch extensions do something similar. For the moment, I will refine this pipeline and once it is at a more stable state I will revisit my model and explore LSTMs more as a potential new model. In addition, I would like to explore using more data (if I can get it) besides just the map image.

Double Rainbow

With Abby's help, I have modified my code to calculate the screen mapping by applying masks to both the low and high frequency rainbow images. By then combining the low and high frequency mask, I can determine the location of the screen mapping. Below are the results of comparing predicted versus measured hue values using our double rainbow method.

It is still really bad. Here is the 3D image figure:

Most of the screen mapping is going to the top left corner. There is something weird going on with comparing hue values to generate our masks.

What's Next:

• fix this stupid screen mapping
• Clean up code and document everything better so that it can be used by the lab.

Starcraft Machine Learning

I am learning about the twitch (video streaming service that 'televises' Starcraft games) API service and how to interface with a stream so that I may use my Starcraft Win Predictor on live data.

I am also cleaning up the code so it is more useable for the a server.