Dr. Dolittle Project: A Framework for Classification and Understanding of Animal Vocalizations

Granting Agency: National Science Foundation
Award number: 0326395
Investigators:

  • Michael T. Johnson
  • Michael Darre
  • Anne Savage
  • Peter Scheifele
  • Elizabeth von Muggenthaler
Award date: October 1, 2003 – September 30, 2009
NSF programs: Information Technology Research; ITR Medium (Group) Grants; Human-Centered Computing; Collaborative Systems

Abstract
The fundamental goal of this research is to develop a broadly useable framework for pattern analysis and classification of animal vocalizations, by integrating successful models and ideas from the field of speech processing and recognition into bioacoustics. Current approaches to automated methods for analyzing and classifying animal vocalizations are still significantly behind the capabilities that exist in the field of human speech processing, and cross-fertilization with other fields offers a tremendous opportunity for making advancements in this area. The PI and his team plan to employ an innovative cross-disciplinary strategy to develop a framework based on robust feature analysis and selection and Hidden Markov Model classification, with additional expertise drawn from such fields as psychology, biology, linguistics, machine learning, and signal processing, for the purpose of making significant advancements to the current state-of-the-art in bioacoustics algorithms and animal communications research. The algorithms developed through this process will be applied to a wide range of important tasks, including automatic vocalization classification and labeling, individual identification, call type classification, behavioral-vocalization correlations, stress analysis, language acquisition, and seismic infrasonic communication. Species being targeted for study include domestic and agricultural animals, marine mammals, and several endangered species. The primary contribution of this work will be the development of well-founded models and methods for improving our understanding of animal communication and behavior. Experimental tasks of significance to a wide variety of research problems within this field are also planned. In addition, many aspects of the HMM classification framework, including hierarchical feature analysis and selection, variable frame sizing, and learnable model topologies, offer the potential for advances in machine learning as well as speech and signal processing.
Broader Impacts: The development of an easily adaptable framework for applying speech-processing techniques to animal vocalizations will allow other researchers to create systems that pertain to the specific species they are studying. This will speed the development effort of these systems and allow those not familiar with speech and signal processing techniques to incorporate advanced methods in their research. A primary underlying motivation for such research is the preservation of endangered species and the improvement of careand habitats for animals in captivity, and this project has the potential for significant, long-term impact toward these goals.

Link to NSF award abstract

Follow

Articles

PDF

Acoustic model adaptation for ortolan bunting (Emberiza hortulana L.) song-type classification, Jidong Tao, Michael T. Johnson, and Tomasz S. Osiejuk

Link

African Elephant Vocal Communication I: Antiphonal Calling Behaviour Among Affiliated Females, Joseph Soltis, Kirsten Leong, and Anne Savage

Link

African Elephant Vocal Communication II: Rumble Variation Reflects the Individual Identity and Emotional State of Callers, Joseph Soltis, Kirsten Leong, and Anne Savage

Link

Antiphonal exchanges in African elephants (Loxodonta africana): collective response to a shared stimulus, social facilitation, or true communicative event?, Katherine A. Leighty, Joseph Soltis, and Anne Savage

PDF

Generalized Perceptual Linear Prediction (gPLP) Features for Animal Vocalization Analysis, Patrick J. Clemins and Michael T. Johnson

Link

GPS determination of walking rates in captive African elephants (Loxodonta africana), Katherine A. Leighty, Joseph Soltis, Christina M. Wesolek, Anne Savage, Jill Mellen, and John Lehnhardt

Link

Infant African Elephant Rumble Vocalizations Vary Accourding to Social Interactions with Adult Females, Christina M. Wesolek, Joseph Soltis, Katherine A. Leighty, and Anne Savage

Link

Moral Psychology Must Not Be Based on Faith and Hope: Commentary on Narvaez (2010), Jonathan Haidt

PDF

Perceptually Motivated Wavelet Packet Transform for Bioacoustic Signal Enhancement, Yao Ren, Michael T. Johnson, and Jidong Tao

Link

Rumble vocalizations mediate interpartner distance in African elephants, Loxodonta africana, Katherine A. Leighty, Joseph Soltis, Christina M. Wesolek, and Anne Savage

Link

The Expression of Affect in African Elephant (Loxodonta africana) Rumble Vocalizations, Joseph Soltis, Katherine A. Leighty, Christina M. Wesolek, and Anne Savage

Link

The expression of emotion in the voiced sounds of rhesus monkeys and African elephants., Joseph Soltis, Christina M. Wesolek, Katherine A. Leighty, Anne Savage, and John D. Newman

Audio Data

File

African Elephant Vocalizations, Michael T. Johnson

File

Beluga Whale Vocalizations, Michael T. Johnson

Conference Proceedings

Link

An Improved SNR Estimator for Speech Enhancement, Yao Ren and Michael T. Johnson

Link

Application of Speech Recognition to African Elephant (Loxodonta Africana) Vocalizations, Patrick J. Clemins and Michael T. Johnson

Link

Auditory Coding Based Speech Enhancement, Yao Ren and Michael T. Johnson

PDF

Automatic Classification of African Elephant (Loxodonta africana) Follicular and Luteal Rumbles, Michael T. Johnson and Patrick J. Clemins

Link

Optimal Distributed Microphone Phase Estimation, Marek B. Trawicki and Michael T. Johnson

Link

Stress and Emotion Classification Using Jitter and Shimmer Features, Xi Li, Jidong Tao, Michael T. Johnson, Joseph Soltis, Anne Savage, Kirsten Leong, and John D. Newman

Link

Unsupervised Validity Measures for Vocalization Clustering, C. Adi Kuntoro, Kristine E. Sonstrom, Peter Scheifele, and Michael T. Johnson

Conference Proceedings

Link

Automatic Song-Type Classification and Speaker Identification of Norwegian Ortolan Bunting Emberiza Hortulana Vocalizations, Marek B. Trawicki, Michael T. Johnson, and Tomasz S. Osiejuk

Link

Generalized Perceptual Features for Vocalization Analysis across Multiple Species, Patrick J. Clemins

Dissertations/Theses

Link

Automatic Frame Length, Frame Overlap and Hidden Markov Model Topology for Speech Recognition of Animal Vocalizations, Anthony D. Ricke

Link

Hidden Markov Model Based Animal Acoustic Censusing: Learning from Speech Processing Technology, C. Kuntoro Adi

Link

SPEech Feature Toolbox (SPEFT) Design and Emotional Speech Feature Extraction, Xi Li

Link

The Classification of Vocalizations to Identify Social Groups of Beluga Whales in the St. Lawrence River Estuary Using a Hidden Markov Model, Kristine E. Sonstrom

Presentations

Link

The Classification of Vocalizations to Identify Social Groups of Beluga Whales in the St. Lawrence River Estuary Using the Hidden Markov Model, Kristine E. Sonstrom, Peter M. Scheifele, Michael Darre, and Michael T. Johnson

Project Websites

Link

Dr. Dolittle Project [project website], Michael T. Johnson and Marquette University College of Engineering