Head and Shoulder Detection using Convolutional Networks and RGBD data


To develop a method for the real time detection of the heads and shoulders of humans in indoor environments using embedded systems.


A complete system capable of high accuracy head and shoulder detection using RGBD data.


  • Run in real-time
  • Low space and computation requirements to be able be ported to a dedicated image processing chip
  • Perform reliably to be released as a consumer product

Core Model Architecture

Core model architecture showing the end to end system.

A fast image processing algorithm for generating object proposals using the CHL algorithm, and a small convolutional neural network that processes the proposals and classifies them. For proposals predicted as belonging to humans, a head bounding box and shoulder keypoints are output.


Image credit: Sandipan Mukherjee, WPforms

This technology can be used in generating business analytics (number of customers, serivce time...), people counting, and congestion analysics.

Sample results