Digit Detecting Neural Network from Scratch

2. Implementing Backpropagation & Gradient Descent

The core of the project was the implementation of the backpropagation algorithm to train the network.

I implemented a forward pass to calculate activations, and a backward pass to compute the gradients of the loss (Mean Squared Error) with respect to every weight and bias.

The algorithm used stochastic gradient descent with a fixed learning rate to iteratively update the parameters. Handling the matrix operations and ensuring correct gradient flow through the layers was the most complex part of the project.

I created a dedicated Network struct to hold the weights, biases, and layer sizes, using pointers to pass this structure efficiently between functions during training and inference.

3. Curating a Custom Dataset to Solve a Real Problem

I identified a critical issue: a model trained on a standard dataset (like MNIST) would perform poorly on my drawing program's distinct style (binary pixels, thin lines).

• Solution: I developed a data collection tool that presented a random digit to draw and saved my 28x28 binary drawing to a labeled file.

• Outcome: I hand-drew a custom dataset of over 3,000 digits. This ensured the training data distribution matched the real-world application, a key factor in the model's high final accuracy.

4. System Integration and Testing

The final step was integrating the trained model with the drawing program.

I modified my existing drawing application to process the canvas, segment multi-digit numbers, and feed each 28x28 segment into the neural network for classification.

Upon pressing 'g', the program displays its prediction in real-time.