Efficient Image Ranking in Heterogeneous Social Media Networks

Abstract

Social media websites such as Flickr and Facebook are pervading our lives today. These fast-evolving Internet communities are characterized of the presence of large amounts of images and videos, which has opened up interesting research avenues within the multimedia and computer vision domains. Social media data is highly interconnected and heterogeneous and associated with a variety metadata (e.g. HTML tags). In this thesis, we investigate three important problems in efficient image ranking in the context of heterogeneous social media networks. We first study the problem of image and tag co-ranking by utilizing graph structures in image collections and orderless tags. A prototype of exploring mutually reinforcing relationships between image and tag graphs is developed which is immediately applicable to image/tag ranking and significantly boosts the performance compared with previous work. In real-world image search engines, images from databases are returned ordered by their relevance to the issued query. There is often significant redundancy in the top-matching images; it would be desirable to remove the redundancy and present a more diverse range of results, to better cover the search topic. To address the problem of diversifying image search results, we develop a novel yet efficient framework, based on non-uniform matroid constraints, to jointly capture the relevance and diversity. Finally, we study the problem of landmark photo retrieval over social media networks. Observing that a landmark query issued by a specific user cannot generally display distinctive landmark features, we develop novel algorithms to expand the unary query to be a multi-query set over which regular landmark features can be mined out. An effective landmark specific mid-level representation is presented to support retrieving relevant landmark photos in a scaled way.