Handwritten digit recognition, experiment

---

Already prepared initial data for training and testing a neural network that recognizes handwritten numbers from 0 to 9 were found on the Web. 

The files include a set of 60,000 training images of numbers, a training set of labels, a set of 10,000 test images and their corresponding test labels.

Training and test images are black-and-white pictures sized 28*28 pixels. Label files are simple sets of bytes with values from 0 to 9 that carry the true meanings of the numbers shown in the pictures.

The picture on the right shows an example of what the test and training pictures look like.

The training data is used in the training phase of the perceptron. Test - the final check of the trained perceptron.

During the learning process, at least one minimum was found. Perhaps this was a local minimum, but on the test set, the found set of perceptron coefficients produced only 194 errors per 10,000 test images presented, i.e. less than 2%

The software, binary weight file and export to hls tables are available for download at the bottom of the page.

All computational operations in the neuroblock are performed in single precision floating point format. The operating time of the neuroblock when calculating the perceptron on forward and backward propagation is counted by the hardware timer of the neuroblock. The operating frequency of the processor is 170 MHz. Board: Stratix V DE5-NET 

Forward propagation.

1024 neurons in the first layer, 784 multiplication operations and the same number of additions to accumulate the sum, then the calculation of the sigmoid function - 3 operations.

Total: the first layer 1024*(784*2+3)=1608704 operations.

10 neurons in the second layer, 1024 multiplication operations and the same number of additions + sigmoid calculation. Total: 10*(1024*2+3)=20510

Total number of operations: 1629214 were completed in 9.4E-5 sec, which is equivalent to 17,332,063,829.787 floating point operations per second /17.33 Gflops/. 94µs character recognition time, recognition rate 10638 characters/sec. It should be noted that the theoretical maximum performance is 21.76 Gflops

Back propagation.

1024*(10+9)+1024*(4+784*2)+10*(4+1024*2)=1649704

calculation of errors of the first layer, recalculation of coefficients of the first layer and recalculation of coefficients of the second layer. Execution time 0.1007814sec.

performance 16,369,131.6 operations per second.