Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment

Nazia Attari, Ferda Ofli, Mohammad Awad, Ji Lucas, Sanjay Chawla

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

20 Scopus citations

Abstract

We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage. To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines.

Original languageEnglish
Title of host publicationProceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages50-59
Number of pages10
ISBN (Electronic)9781509050048
DOIs
StatePublished - Jul 2 2017
Externally publishedYes
Event4th International Conference on Data Science and Advanced Analytics, DSAA 2017 - Tokyo, Japan
Duration: Oct 19 2017Oct 21 2017

Publication series

NameProceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017
Volume2018-January

Conference

Conference4th International Conference on Data Science and Advanced Analytics, DSAA 2017
Country/TerritoryJapan
CityTokyo
Period10/19/1710/21/17

Fingerprint

Dive into the research topics of 'Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment'. Together they form a unique fingerprint.

Cite this