We propose to leverage concept-level representations for complex event recognition in photographs given limited training examples. We introduce a novel framework to discover event concept attributes from the web and use that to extract semantic features from images and classify them into social event categories with few training examples. Discovered concepts include a variety of objects, scenes, actions and event sub-types, leading to a discriminative and compact representation for event images. Web images are obtained for each discovered event concept and we use (pretrained) CNN features to train concept classifiers. Extensive experiments on challenging event datasets demonstrate that our proposed method outperforms several baselines using deep CNN features directly in classifying images into events with limited training examples. We also demonstrate that our method achieves the best overall accuracy on a dataset with unseen event categories using a single training example. [full paper]

 
 

Datasets

 

1. Social Event Image Dataset (SocEID)

This is the dataset we created in-house. We collected images of the following social events: birthdays, graduations, weddings, marathons/races, protests, parades, soccer matches and concerts. We queried Instagram and Flickr with a tag related to the event itself (‘wedding day,’ ‘Graduation 2014’ etc.) and downloaded public images in chronological order determined by post date. Our dataset includes some relevant images from the NUS-WIDE dataset [1] and the Social Event Classification subtask from MediaEval 2013 [2]. [Download Dataset]

 

2. Rare Events Dataset (RED)

This is another in-house dataset we collected by querying MS Bing image search engine with a set of 26 ‘rare’ event categories. We call them rare not on the basis of how frequently they occur in the world but on how seldom they are found in large labeled event image datasets. These event categories comprise of recent news events such as: Justin Trudeau elected, election campaign Trump and natural disasters such as Hurricane Katrina, Hurricane Sandy and Nepal earthquake. [Download Dataset]

 

 

If you find these datasets useful, please cite our paper:

@article{ahsan2017complex,
  title={Complex Event Recognition from Images with Few Training Examples},
  author={Ahsan, Unaiza and Sun, Chen and Hays, James and Essa, Irfan},
  journal={arXiv preprint arXiv:1701.04769},
  year={2017}
}

 

 

References

[1] T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng. Nus-wide: A real-world web image database from national university of singapore. In Proc. of ACM Conf. on Image and Video Retrieval (CIVR’09), Santorini, Greece., July 8 -10, 2009.

[2] T. Reuter, S. Papadopoulos, G. Petkos, V. Mezaris, Y. Kompatsiaris, P. Cimiano, C. de Vries, and S. Geva. Social event detection at mediaeval 2013: Challenges, datasets, and evaluation. In Proceedings of the MediaEval 2013 Multimedia Benchmark Workshop Barcelona, Spain, October 18-19, 2013, 2013.