Ramit Pahwa, Machine Learning Scientist at Rivian, presents the “Removing Weather-related Image Degradation at the Edge” tutorial at the May 2024 Embedded Vision Summit.
For machines that operate outdoors—such as autonomous cars and trucks—image quality degradation due to weather conditions presents a significant challenge. For example, snow, rainfall and raindrops on optical surfaces can wreak havoc on machine perception algorithms. In this talk, Pahwa explains the key challenges in restoring images degraded by weather, such as lack of annotated datasets, and the need for multiple models to address different types of image degradation.
Pahwa also introduces metrics for assessing image degradation. He then explains Rivian’s solutions and shares results, demonstrating the efficacy of transformer-based models and of a novel, language-driven, all-in-one model for image restoration. Finally, he highlights the techniques used to create efficient implementations of Rivian’s models for deployment at the edge—including quantization and pruning—and shares lessons learned from implementing these models on a target processor.
See here for a PDF of the slides.
