Using PyTorch to create a Convolutional Neural Network for Image Classification of Numbers

Introduction

Just a few notes before I get started:
1) Be on the lookout for overfitting in your models. Do not use to many layers, because when you eventually test it, it will be so tuned so much to your trained dataset that it will actually provide the wrong answer during your test (personal experience).
2) .to('cuda') vs .to('cpu'): if you have a compatible NVDIA graphics card with the CUDA drivers downloaded, you should check if you can use this function before proceeding with my code. If you don't have an NVDIA graphics card or don't want to, you can use ".to('cpu')" to use your CPU instead.
3) I am using the MNIST Number Dataset on Google Colab with a T4 GPU and have tested it on a RTX 3060. You can use ".to('cuda')" for either.

Framing + Implementation 

Our goal is to get the computer to understand integers 0-9. To do this, we will use a Convolutional Neural Network. Personally, I will be using the PyTorch library to accomplish this, since it's simple and already has an inbuilt function to download this dataset.

First, import these statements. We will use them for the entire project, 
 


Next, lets load our testing data in,



*Note, transforming to tensor is very crucial to making the program run. I used 32 images for the batch size, or the amount of data that will be processed in one iteration in the network.

Next, we have to create our Neural Network, specifically using convolutions. Create a class, 

Next, we're going to build the real network. We will use a sequential model to build our neural network, saving it under self.inital_model. Here's what I mean,

This container will contain the model, holding the Convolutional Layers, flattening them, and converting the image, and applying the ReLU activation function. Here's how our first layer will look like, 





The last part is what we use to transform the image. The reason we subtract 2 pixels is because each layer, we shave 2 pixels in this situation. Let's create 2 more layers (for a total of 3), 















Next is the forward method, which will make sure the images pass through the Neural Network. 
*Note: There is an indention error for the forward method. Make sure you indent it one more time. 

We're going to print the model's features. 




You should expect these results, 








We now need to build our backpropagation portion of the network. We, first, will use the Adam Optimizer (it's really good for learning and helping fine-tune the parameters) and the Cross Entropy Loss (classify probability values between 0-1).

LR stands for "Learning rate," and "1e" (I think), stands for 10^1. 3 is subtracted after that ([10^1]-3).

Next, we're going to create our training flow to, obviously, train our model. We are going to store this in a file called "model_state.pt". I will be using 5 epochs














Once this is all done, make sure to run your code! I'm going to go ahead and test this network. My number is going to be number 2.  
It's right! Congrats, you just learned to make your first CNN!!!

Comments

Post a Comment

Popular posts from this blog

What is a Multimodal LLM?

Image
Introduction Multimodal is the ability for a model to process several different types of inputs. For example, ChatGPT can support text input as well as image input as one combined input. You could also have a model take in .pdf, .md, and input text as inputs (yes, these count as different modalities).  This allows for users to give represent different ideas in its rawest form possible, such as an image, instead of describing it. If you try to describe it, you could hit hurdles in trying to properly explain it and making sure it interpreted the description correctly.  How do Multimodal LLMs work?  Typically, these modalities are "fused" together. Here's an architecture diagram,  Image from NVIDIA In this case, we have an encoder for a specific modality (e.g. video encoder, image encoder, audio encoder). This encodes them into embeddings the LLM can understand and use. After encoding each specific modality, we then fuse them together. Then, the output is projected and ...
Read more

A Brief Introduction into Model Quantization

Image
  Llama. Lots of popular packages and models have "llama" in the name (I'll be using a model and package that have llama in their name!) ( Flickr user Katherine ).   Introduction Have you ever wondered how people get 20B parameter models working on their laptops? Well, they have to quantize a model. Quantization is when you lower the accuracy of the model weights (from floating point 32 bit to floating point 16 bit). Although you lose some precision, researchers have found that this loss is minimal. In fact, accuracy still rivals the pre-quantized model. There also comes an added bonus of less storage.  Researchers are even pushing the limit to 1 bit ( Wang et. al. ) and 1.58 bit ( Ma et. al. ) LLMs.  Different Quantizations Terminology Q1, Q2, Q3, etc.  The number next to the Q refers to how many bits the model will be quantized to. For example, Q6 means 6 bit, Q8 is 8 bit, and Q1 is 1 bit.  _K, _0, _1 This tells us how the model rounds.  Type-0 ...
Read more

A Mathematical Explanation of Gradient Descent

Image
Example of a close-to-optimal Gradient descent for a quadratic function. Credit: Dylan Introduction Although I wrote an article recently explaining Gradient Descent, I feel like a mathematical explanation of how it works would be beneficial not only to understand how it works, but also how you derive the functions.  Explanation First, we need a way to calculate how far off the model is. For our example, we will use the L2 Loss Function,  Let N represent the total amount of data points we have, i the iteration we are on, yᵢ represent the expected output, and ŷ represent the output predicted by the model.  Let's look more into ŷ. Since ŷ is the predicted value, this is the function that needs to have their parameters changed. Our goal is to change these parameters enough to minimize the loss of ŷ. For our example, let's set ŷ to be a linear line.  The values in "a" and "b" are our parameters. These will be the values we will have to change within f(x) to chang...
Read more