Combining computer vision and knowledge acquisition to provide real-time activity recognition for multiple persons within immersive environments

Download files
Access & Terms of Use
open access
Copyright: Sridhar, Anuraag
Altmetric
Abstract
In recent years, vision is gaining increasing importance as a method of human-computer interaction. Vision techniques are becoming popular especially within immersive environment systems, which rely on innovation and novelty, and in which the large spatial area benefits from the unintrusive operation of visual sensors. However, despite the vast amount of research on vision based analysis and immersive environments, there is a considerable gap in coupling the two fields. In particular, a vision system that can provide recognition of the activities of multiple persons within the environment in real time would be highly beneficial in providing data not just for real-virtual interaction, but also for sociology and psychology research. This thesis presents novel solutions for two important vision tasks that support the ultimate goal of performing activity recognition of multiple persons within an immersive environment. Although the work within this thesis has been tested in a specific immersive environment, namely the Advanced Visualisation and Interaction Environment, the components and frameworks can be easily carried over to other immersive systems. The first task is the real-time tracking of multiple persons as they navigate within the environment. Numerous low-level algorithms, which leverage the spatial positioning of the cameras, are combined in an innovative manner to provide a high-level, extensible framework that provides robust tracking of up to 10 persons within the immersive environment. The framework uses multiple cameras distributed over multiple computers for efficiency, and supports additional cameras for greater coverage of larger areas. The second task is that of converting the low-level feature values derived from an underlying vision system into activity classes for each person. Such a system can be used in later stages to recognize increasingly complex activities using symbolic logic. An on-line, incremental knowledge acquisition (KA) philosophy is utilised for this task, which allows the introduction of additional features and classes even during system operation. The philosophy lends itself to a robust software framework, which allows a vision programmer to add activity classification rules to the system \textit{ad infinitum}. The KA framework provides automated knowledge verification techniques and leverages the power of human cognition to provide computationally efficient yet accurate classification. The final system is able to discriminate 8 different activities performed by up to 5 persons.
Persistent link to this record
Link to Publisher Version
Link to Open Access Version
Additional Link
Author(s)
Sridhar, Anuraag
Supervisor(s)
Sowmya, Arcot
Compton, Paul
Creator(s)
Editor(s)
Translator(s)
Curator(s)
Designer(s)
Arranger(s)
Composer(s)
Recordist(s)
Conference Proceedings Editor(s)
Other Contributor(s)
Corporate/Industry Contributor(s)
Publication Year
2012
Resource Type
Thesis
Degree Type
PhD Doctorate
UNSW Faculty
Files
download whole.pdf 4.13 MB Adobe Portable Document Format
Related dataset(s)