Deep Learning Workflow in PyTorch

1. Prepare data.
2. Prepare (untrained) model.
3. Train model.
4. Evaluate model.
5. Save model.

1. Prepare data.

1. Get data like images, video, sound, text, etc.
2. Divide the data into the one for training(Train data) and the one for testing(Test data). *Basically, train data is 80% and test data is 20%.

2. Prepare (untrained) model.

1. Select the suitable layers activation function for the data.
2. Select the activation function for the data if necesarry.

3. Train model.

1. Select the suitable loss(cost) function and optimizer for the data.
   *Memos:
   • A loss function is the algorithm which can get the gap between predictions and train data.
   • An optimizer is the algorithm which can minimize the loss between predictions and train data with gradient descent. *Gradient Descent(GD) is the algorithm which can find the minimum(or maximum) gradient(slope) of a function.
   • There are loss functions in PyTorch such as nn.L1Loss(), nn.MSELoss(), nn.CrossEntropyLoss(), etc according to the doc.
   • There are optimizers in PyTorch such as optim.SGD(), optim.Adam(), optim.Adadelta(), etc according to the doc.
2. Calculate predictions with train data.
3. Calculate the loss between predictions and train data with a loss(cost) function.
4. Zero out the gradients of all tensors every epoch for proper calculation. *The gradients are accumulated in buffers, then they are not overwritten until backward() is called.
5. Do backpropagation. *Backpropagation is the algorithm which can minimize the loss between predictions and train data, working from output layer to input layer.
6. Optimize the model to minimize the loss between predictions and train data with an optimizer.

*Repeat the epoch of 2., 3., 4., 5. and 6. to minimize the loss between predictions and train data.

4. Evaluate model.

1. Calculate predictions with test data.
2. Calculate the loss between predictions and test data with a loss(cost) function.
3. Check the loss with train and test data by text or graph.

5. Save model.

Finally, save the model if the model is the enough quality which you want.