Research Interests

Video     Surveillance

James W. Davis, Dept. of Computer Science and Engineering, is developing an advanced video surveillance system that uses computers equipped with video cameras to not only detect the presence of people, but also to identify their activities. The research has broad implications for Homeland Security as well as search and rescue, border patrol, law enforcement and many other types of military applications. The system combines a thermal camera with machine learning methods, enabling the computer to perform the kind of visual recognition that seems effortless for humans. Davis' work in investigating computer vision methods was recently recognized by the National Science Foundation with the prestigious NSF Faculty Early Career Development (CAREER) Program award. Support for this research (past and present) has been provided by the National Science Foundation, U.S. Air Force, Los Alamos National Lab, U.S. Army Night Vision Laboratory, Intel, and Ohio Board of Regents. Technical Areas: Computer Vision, Visual Perception, Human-Computer Interaction, Motion Capture, Artificial Intelligence.

Media and Press

TV/Video: OSU Video, "Better Eyes" (released January 9, 2009) [WWW] OSU Video, "'Smart' Surveillance System May Tag Suspicious or Lost People" (released 12/08) [mov video clip] Frontline: "Spying on the Home Front" (Aired 05/15/07) Fox-28/ABC-6 News, "They're Watching You" (Aired 05/10/04) [wmv video clip] Fox-28/ABC-6 News, Interview on thermal technology (Aired 05/08/03, 05/09/03) [wmv video clip] NBC-4 News, (aired 05/09/03) CBS-10 News, (aired 05/04/03) OSU Video,"Researcher developing video technology for surveillance: Cameras being trained to `see' people and recognize their activities" (released 04/30/03) [web page and video], [news article] Radio: AAAS Science Update radio feature, "Internet Video: A new technology could make watching video on the Internet less frustrating" (aired January 12, 2005) [MP3 audio clip], [web page and audio] Print: SPIE Newsroom: Electronic Imaging & Signal Processing, "A Feature-Selection Approach to Fusing Electro-Optical and IR Imagery" (February 25, 2009) [PDF] [WWW] Photonics Spectra, Tech News, "Neighborhood Watch, Sans Neighbors" (February, 2009; pp. 18-20) [PDF] MSNBC, Technology & Science, "Computer has eye for Suspicious Behavior" (January 26, 2009) [PDF (no pictures)], [WWW] Columbus Dispatch, "Computers Might Spot People up to no Good" (January 24, 2009) [PDF], [WWW] onCampus, Discovery Briefs, "'Smart' Surveillance System May Tag Suspicious or Lost People" (January 8, 2009) [PDF] UWeekly "Big Brother No, Smart Camera Yes" (February 18, 2009) [PDF] The Lantern, "Under Surveillance" (January 27, 2009) [PDF] The Lantern, "'Smart' Cameras are Watching You" (January 9, 2009) [PDF] OSU News Release, "'Smart' Surveillance System May Tag Suspicious or Lost People" (released 12/08) [PDF], [WWW] The Economist, Science & Technology section, "Surveillance Technology: If Looks Could Kill", October 23, 2008 [PDF] OSU CSE Newletter, Focus on Research Article, "Jim Davis: Vision for the Future of Surveillance", Autumn 2007, pages 2-3 [PDF] OSU Alumni Magazine, Research Report Article, "Watch the Hands and Face", May/June, 2005, page 8 [PDF] OSU News In Engineering Research Update article, "Advancing Technologies for Internet Video", Vol. 77, No. 1, Spring 2005 [PDF] ACM TechNews Feature, "Researchers Find Way to Make Internet Video More Appealing", Vol. 6, Issue 734, December 22, 2004 [PDF], [WWW] OSU News Release, "Researchers find way to make internet video more appealing" (released 12/21/04) [WWW] Homeland Security & Defense, "HSARPA seeking video software technology to interpret complex surveillance stream", Vol. 3, No. 16, April 21, 2004 [PDF] Columbus Business First, Tech Now feature article, "Hot on Surveillance: Thermal technology grows in popularity, importance in post-911 world", Vol. 20, No. 7, Oct. 17, 2003 [PDF]

Recent News

SPIE Newsroom: Electronic Imaging & Signal Processing, "A Feature-Selection Approach to Fusing Electro-Optical and IR Imagery" (February 25, 2009) [PDF] [WWW] MSNBC, Technology & Science, "Computer has eye for Suspicious Behavior" (January 26, 2009) [PDF (no pictures)], [WWW] Columbus Dispatch, "Computers Might Spot People up to no Good" (January 24, 2009) [PDF], [WWW] The Economist, Science & Technology section, "Surveillance Technology: If Looks Could Kill", October 23, 2008 [PDF] UNDERGRAD AND GRAD POSITIONS AVAILABLE IN THE COMPUTER VISION LAB. "A Fast Linear Registration Framework for Multi-Camera GIS Coordination", IEEE International Conference on Advanced Video and Signal Based Surveillance, September 2008. "3D Occlusion Recovery using Few Cameras" and "A Recursive Filter For Linear Systems on Riemannian Manifolds", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2008, Anchorage, Alaska. "Integrating Appearance and Motion Cues for Simultaneous Detection and Segmentation of Pedestrians", IEEE International Conference on Computer Vision (October, 2007; oral accept rate 3.9%). OSU CSE Newletter, Focus on Research Article, "Jim Davis: Vision for the Future of Surveillance", Autumn 2007, pages 2-3 [PDF] Video clips of our research on tracking people in surveillance videos shown in Frontline: "Spying on the Home Front" (Aired 05/15/07) [wmv video clip (no audio)], credits "Simultaneous Detection and Segmentation of Pedestrians using Top-down and Bottom-up Processing" and "Kernel-Based 3D Tracking", IEEE International Workshop on Visual Surveillance, June 2007. "Next Steps in Automatic Video Surveillance", invited seminar at ASIS International Annual Seminars and Exhibits, May 17, 2007 [brochure]. "Background-Subtraction using Contour-based Fusion of Thermal and Visible Imagery", Computer Vision and Image Understanding, Vol. 106, No. 2-3, 2007, pp. 162-182. "An Adaptive Focus-of-Attention Model for Video Surveillance and Monitoring", Machine Vision and Applications, Vol. 18, No. 1, 2007, pp. 41-64. "Building Adaptive Camera Models for Video Surveillance", IEEE Workshop on Application of Computer Vision, February 2007. "Extraction of Person Silhouettes from Surveillance Imagery using MRFs", IEEE Workshop on Application of Computer Vision, February 2007. "Fusion of Multiple Camera Views for Kernel-Based 3D Tracking", IEEE Workshop on Motion and Video Computing, February 2007. "Tracking Mean Shift Clustered Point Clouds for 3D Surveillance", and "Multiview Fusion for Canonical View Generation based on Homography Constraints" ACM Mulimedia Workshop on Video Surveillance and Sensor Networks, October 2006. "Background-Subtraction in Thermal Imagery Using Contour Saliency", International Journal of Computer Vision, Vol. 71, No. 2, 2007, pp. 161-181. "Feature-level Fusion for Object Segmentation using Mutual Information", IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum, New York, NY, June 22, 2006. "Minimal-Latency Human Action Recognition using Reliable-Inference", Image and Vision Computing, Vol 24, 2006, pp. 455-472. Co-Chair, for IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum (OTCBVS), June 22, 2006. "Sampling Representative Examples for Dimensionality Reduction and Recognition - Bootstrap Bumping LDA", European Conference on Computer Vision (ECCV), Graz, Austria, May 7-13, 2006. "Why Direct LDA Is Not Equivalent to LDA", Pattern Recognition, Vol. 39, 2006, pp. 1002-1006. Promotion to Associate Professor with Tenure (October 1, 2005). Guest Editor (w/ Riad Hammoud) for CVIU Special Issue on Advances in Vision Algorithms and Systems Beyond the Visible Spectrum. [CALL FOR PAPERS] "Modeling Behavior Trends and Detecting Abnormal Events using Seasonal Kalman Filters", Workshop on Human Activity Recognition and Modeling, Oxford, UK, September 9, 2005. OSU News In Engineering Research Update article: "Advancing Technologies for Internet Video", Vol. 77, No. 1, Spring 2005. Appointed as Associate Editor for Machine Vision and Applications journal (May 2005). OSU Alumni Magazine Research Report Article: "Watch the Hands and Face", May/June, 2005, page 8. Recipient of 2005 OSU College of Engineering Lumley Research Award. "Fusion-based Background-Subtraction using Contour Saliency", IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum, San Diego, CA, June 20, 2005. AAAS Science Update Radio Feature: "Internet Video: A new technology could make watching video on the Internet less frustrating" (January 12, 2005) [MP3 audio clip]. "A Two-Stage Template Approach to Person Detection in Thermal Imagery", Workshop on Applications of Computer Vision, Breckenridge, Colorado, January 7, 2005. ACM TechNews Feature "Researchers Find Way to Make Internet Video More Appealing", Vol. 6, Issue 734, December 22, 2004 [pdf]. OSU News Release (12/21/04): "Researchers find way to make internet video more appealing". Benchmark and Website Chair for IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum (OTCBVS), June 20, 2005. Call for participation in OTCBVS benchmark data collection. "Differential Video Coding of Face and Gesture Events in Presentation Videos", Computer Vision and Image Understanding, Vol. 96, No. 2, 2004, pp. 200-215. "Robust Detection of People in Thermal Imagery", International Conference on Pattern Recognition (August 2004). NSF ITR grant award ($1.3M) for proposal "Multi-Level, Active Attention Surveillance" (2004-2007). "Sequential Reliable-Inference for Rapid Detection of Human Actions", IEEE Workshop on Detection and Recognition of Events in Video (July 2004). "Robust Background-Subtraction for Person Detection in Thermal Imagery", IEEE International Workshop on Object Tracking and Classification Beyond the Visible Spectrum (July 2004). "Detection and Recognition of Human Activity in Thermal Video", Presentation at U.S. Army Night Vision Laboratory, Fort Belvoir, VA, June 28, 2004. "Gender Recognition from Walking Movements using Adaptive Three-Mode PCA", IEEE Workshop on Articulated and Nonrigid Motion (June 2004). TV interview: "They're Watching You" (Aired 5/10/04) [wmv video clip]. "Video Surveillance of Tomorrow", invited seminar at ASIS International Annual Seminars and Exhibits, May 13, 2004 [brochure]. "An Expressive Three-Mode Principal Components Model for Gender Recognition", Journal of Vision, Vol. 4, No. 5, 2004, pp. 362-377. Interview in feature article, "HSARPA seeking video software technology to interpret complex surveillance stream", Homeland Security & Defense, Vol. 3, No. 16, April 21, 2004. "Representing and Recognizing Human Actions", Presentation at Robotics Institute, Carnegie Mellon University, March 26, 2004. "Computer Recognition of Human Actions", Presentation at Siemens Corp., Princeton, NJ, Dec. 15, 2003. "Computer Recognition of Human Actions", Presentation at Boston University, Oct. 30, 2003. "Computer Recognition of Human Actions", Presentation at University of Illinois at Urbana-Champaign, Oct. 24, 2003. Interview in Tech Now feature article, "Hot on Surveillance: Thermal technology grows in popularity, importance in post-911 world", Columbus Business First, Vol. 20, No. 7, Oct. 17, 2003. "Recognizing Human Action Efforts: An Adaptive Three-Mode PCA Framework", International Conference on Computer Vision (ICCV), Oct. 13-16, 2003, Nice, France, pp. 1463-1469. "An Expressive Three-Mode Principal Components Model of Human Action Style", Image and Vision Computing, Vol. 21, No. 11, 2003, pp. 1001-1016. "A Reliable-Inference Framework for Recognition of Human Actions", IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 169-176, July 21-22, 2003, Miami, FL. OSU News In Engineering article (and cover): "Future Innovators", Vol. 75, No. 1, Spring 2003. OSU Lantern article: "Night Vision: Advanced thermal surveillance system used to identify and catalogue motion" (5/12/03). TV Interviews: Fox-28/ABC-6 News (aired 05/09/03, 05/08/03) [wmv video clip] NBC-4 News (aired 05/09/03) CBS-10 News (aired 05/04/03) Press Release (04/30/03): "Researcher developing video technology for surveillance: Cameras being trained to `see' people and recognize their activities" [Video News Release]. Recipient of 2003 NSF Career Award (March 2003). "A Three-Mode Expressive Feature Model of Action Effort", IEEE Workshop on Motion and Video Computing, Dec. 5-6, 2002, pp. 139-144. [Received a Lockheed-Martin Best Paper Award]

Recent Speakers at OSU

Kevin Bowyer (ND), "Adventures in Three-Dimensional Face Recognition" (May 3, 2007) Yiannis Aloimonos (UMD),"HAL: Human Activity Language" (Feb 8, 2007) Simon Baker (CMU), "Shape-From-Silhouette Across Time" (February 7, 2006) Stan Sclaroff (BU), "Computer Vision Techniques for Analysis of Human Gesture" (May 24, 2005) Robert Collins (CMU/PSU), "Topics in Automated Video Surveillance" (May 12, 2005) Bruce Flinchbaugh (TI), "Smart Camera Systems Technology Roadmap" (April 28, 2005) Fernando de la Torre (CMU), "Component Analysis for Multimodal Diaries" (April 26, 2005) Gerard Medioni (USC), "Tensor Voting Review, Applications to Computer Vision and Machine Learning" (April 7, 2005) Ben Shneiderman (UMD), "The Eyes Have It: User Interfaces for Information Visualization" (November 9, 2004) Mubarak Shah (UCF), "Object Tracking" (October 13, 2004) Chris Bregler (NYU), "Capturing Muybridge, Kangaroos, and Cartoons" (April 16, 2004) Bir Bhanu (UC Riverside), "Biometrics for Human Identification" (March 12, 2004) Zoran Popovic (Univ. Washington), "Rapid Design of Realistic and Expressive Animations" (November 14, 2003) Irfan Essa (GaTech), "Analysis and Synthesis of Video for Special Effects and Animation" (October 3, 2003) Bruce Flinchbaugh (TI), "Introduction to Digital Video Systems" (August 29, 2003) Tom Huang (UIUC), "Relevance Feedback in Content-Based Image and Video Retrieval" (May 27, 2003) Jessica Hodgins (CMU), "Animating Human Characters" (April 25, 2003) Larry Davis (UMD), "Monitoring Human Activity using Computer Vision" (April 17, 2003) Jake Aggarwal (UT), "Human Motion: Tracking and Recognition of Actions and Interactions" (April 11, 2003) Norm Badler (UPenn), "Representing and Parameterizing Embodied Agent Behaviors" (March 14, 2003)

Related Links

Publications People Teaching Personal Web Page