Convolutional Neural Network (CNN)
Budget: $10 – $30 USD
Your task is to build and train convolutional neural networks (CNN) on Cifar100 dataset for object classification. More details about this data can be found at https://www.cs.toronto.edu/~kriz/cifar.html .
The dataset has 100 classes but we will be selecting only fourtenn classes so that it can be trained relatively easily on your machine.
Build a NN network specified in the figure below and initialize the parameters. In the figure, "8, 3×3 " convolution stands for a 2D convolution operation with 8 filters with 3×3 kernel. Set the padding in the convolution filters such that the spatial size of the output is an integer division of the input size. See the pytorch documentation for torch.nn.Conv2d, torch.nn.BatchNorm2d, torch.cat, and torch.nn.AdaptiveAvgPool2d for convolution, batch-normalization, and concatenate, and global average pooling operations, resepectively.
The last layer is a fully connected layer and the input shape for this layer must be ?×? wheras the output of its preceding average pooling layer is 4D if dimension is preserved and will have a shape of ?×?×?×? . You need to reshape the output of this convolution layer before you feed it into the linear layer (torch.flatten can be used for this.).
Initialize the weights of convolution layers with zero mean and variance = 2?????????????? . For FC, the variance is 1?????????????? . Initialize all biases to zero.
Do forward propagation for a minibatch and verify that the output shape equals the number of samples in a minibatch by the number of classes.
Set up the loss and optimizer. Train using Stochastic gradient descent with momentum for 50 epochs with a learning rate of 0.01, momentum of 0.2, and weight_decay of 1?−8 . Note that nn.CrossEntropyLoss class in PyTorch combines both softmax and crossentropy. That is why, we did not include softmax operation in our network.
Do the training and validation. Follow the instructions in the code block to complete this part.
Do a grid search for hyperparameters - learning rate, momentum, and weight decay. Try 1?−1,1?−3 for learning rate, 0.5,0.9 for momentum, and 0,1?−4 for weight decay. See torch.optim.SGD for how to configure these hyperparameters in the optimizers. You need to train for all possible combination of learning rate, momentum, and weight decay.
Save the model that yields the best validation accuracy along with its hyperparameters. See the section, "Saving & Loading a General Checkpoint for Inference and/or Resuming Training" at https://pytorch.org/tutorials/beginner/saving_loading_models.html for tutorial on saving and loading models.
Load the best model and evaluate its performance (e.g., accuracy) on the test data.
The dataset has 100 classes but we will be selecting only fourtenn classes so that it can be trained relatively easily on your machine.
Build a NN network specified in the figure below and initialize the parameters. In the figure, "8, 3×3 " convolution stands for a 2D convolution operation with 8 filters with 3×3 kernel. Set the padding in the convolution filters such that the spatial size of the output is an integer division of the input size. See the pytorch documentation for torch.nn.Conv2d, torch.nn.BatchNorm2d, torch.cat, and torch.nn.AdaptiveAvgPool2d for convolution, batch-normalization, and concatenate, and global average pooling operations, resepectively.
The last layer is a fully connected layer and the input shape for this layer must be ?×? wheras the output of its preceding average pooling layer is 4D if dimension is preserved and will have a shape of ?×?×?×? . You need to reshape the output of this convolution layer before you feed it into the linear layer (torch.flatten can be used for this.).
Initialize the weights of convolution layers with zero mean and variance = 2?????????????? . For FC, the variance is 1?????????????? . Initialize all biases to zero.
Do forward propagation for a minibatch and verify that the output shape equals the number of samples in a minibatch by the number of classes.
Set up the loss and optimizer. Train using Stochastic gradient descent with momentum for 50 epochs with a learning rate of 0.01, momentum of 0.2, and weight_decay of 1?−8 . Note that nn.CrossEntropyLoss class in PyTorch combines both softmax and crossentropy. That is why, we did not include softmax operation in our network.
Do the training and validation. Follow the instructions in the code block to complete this part.
Do a grid search for hyperparameters - learning rate, momentum, and weight decay. Try 1?−1,1?−3 for learning rate, 0.5,0.9 for momentum, and 0,1?−4 for weight decay. See torch.optim.SGD for how to configure these hyperparameters in the optimizers. You need to train for all possible combination of learning rate, momentum, and weight decay.
Save the model that yields the best validation accuracy along with its hyperparameters. See the section, "Saving & Loading a General Checkpoint for Inference and/or Resuming Training" at https://pytorch.org/tutorials/beginner/saving_loading_models.html for tutorial on saving and loading models.
Load the best model and evaluate its performance (e.g., accuracy) on the test data.