Independent AI Engineer Karthik Rao Aroor presents the “Improved Data Sampling Techniques for Training Neural Networks” tutorial at the May 2024 Embedded Vision Summit.
For classification problems in which there are equal numbers of samples in each class, Aroor proposes and presents a novel mini-batch sampling approach to train neural networks using gradient descent. His proposed approach ensures a uniform distribution of samples from all classes in a mini-batch. He shares results showing that this approach yields faster convergence than the random sampling approach commonly used today.
Aroor illustrates his approach using several neural network models trained on commonly used datasets, including a truncated version of ImageNet. He also presents results for large and small mini-batch sizes relative to the number of classes. Comparing these results to a suboptimal sampling approach, he hypothesizes that having a uniform distribution of samples from each class in a mini-batch is an optimal sampling approach. His approach benefits model trainers by achieving higher model accuracy with reduced training time.
See here for a PDF of the slides.
