As mentioned in the previous blog post about CS 231n, it covers some of the most important aspects of CNNs for visual recognition (and thankfully to me, starts from the basics).
The previous post went through a toy example of a simple two-layer neural network and how its performance differs from that of a linear softmax classifier, which was very helpful in understanding the basics of neural networks. The further lectures, modules, notes and assignments of the course go into more advanced topics of the subject matter. I'm still trying to figure out exactly what is happening in every step described in the tutorials, but due to the density of the information I try to focus on topics that seem to be more applicable and useful for the future. One of such topics is PyTorch, which is a system for executing dynamic computational graphs over Tensor objects that behave similarly as numpy ndarray. In short, it allows to eliminate some of the steps we otherwise would have to complete manually. Being unfamiliar with this library, I chose to jot some things down about it in this blog post (and I might go further in-depth into Tensorflow in the next post). PyTorch is a library I hadn't heard of before starting to work in the lab, so pretty everything about it is new to me.
Here are some things I noted about PyTorch:
- When using a framework like PyTorch you can harness the power of the GPU for your own custom neural network architectures.
- When creating a neural network from scratch you have to download the dataset, preprocess it, and iterate through it in minibatches. PyTorch, similarly to Tensorflow, automates and therefore simplifies this process.
- This is usually done through the torchvision package.
- For a simple fully-connected ReLU network with 2 hidden layers and no biases PyTorch allows to compute the forward pass using operations on PyTorch Tensors, and uses PyTorch Autograd to compute gradients.
- A PyTorch Tensor is similar to a numpy array: it is an n-dimensional grid of numbers.
- The tensor needs to be flattened before being passed onto the network to be 2-dimensional.
- A 2-layer neural network can be trained in PyTorch in much less lines of code than when created from scratch!