Kinect Head Pose Estimator

This code is originally from a paper published by Gabriele Fanelli. Chris Peplin added detailed instructions for compiling and running the code in Ubuntu 11.10.

head_pose_estimation

The actual code for head pose estimation is contained in the CR* files present in this folder. The main.cpp file shows an example use of the code on (compressed) depth images as the ones provided in the data/ folder.

A forest is provided in the trees/ folder, which was trained on depth images of people turning theirs heads in front of the Kinect, sitting at around 1 m distance from the sensor; therefore, the forest will give optimal results for similar scenarios and will fail on completely different settings.

NB: This demo contains the code and forest related to our DAGM paper, i.e., the provided trees are trained on the Biwi Kinect Head Pose Database (all of it), and should therefore not be used for comparison to other methods using high quality depth scans, like the ETH Face Pose Range Image Data Set of Breitenstein et al.

In order to compile, OpenCV2.3 is required, please modify the CMakeLists.txt file according to your local OpenCV installation.

To run the example code, type:

./head_pose_estimation config.txt data/frame_XXXX_depth.bin

The config.txt file contains all parameters needed for the head pose estimation, e.g., the path to the forest, the stride, and z threshold used to segment the person from the background. Please note that a depth.cal file should be present in the same folder as the depth image.

head_pose_estimation_demo

In the demo folder, an additional example is provided which runs on a stream of depth images coming from a Kinect sensor. The demo requires (in addition to OpenCV), the OpenNI, freeglut, and OpenGL libraries. See below for instructions on installing these depenedencies in Ubuntu 11.10.

To run the demo, type:

./head_pose_estimation_demo config.txt

Installation in Ubuntu 11.10

Install the dependencies that are available in the central repository:

$ sudo apt-get install cmake freeglut3-dev libusb-1.0-0-dev build-essential

OpenCV

Add a PPA required to get OpenCV 2.3 (only 2.1 is available in the central repository):

$ sudo add-apt-repository ppa:gijzelaar/cuda
$ sudo add-apt-repository ppa:gijzelaar/opencv2.3
$ sudo apt-get update
$ sudo apt-get install libcv-dev

If you install OpenCV by some other method, you will need to modify demo/CMakeLists.txt to reflect the correct path to the header and library files.

OpenNI

Download the unstable OpenNI binary OpenNI Unstable Build for Ubuntu 10.10 x64 (or x86 if you're using a 32-bit OS) v1.5.2.23.

Extract the .tar.bz2 file that you've downloaded and install it:

$ tar -xjf openni-bin-dev-linux-x64-v1.5.2.23.tar.bz2
$ cd OpenNI-Bin-Dev-Linux-x64-v1.5.2.23/
$ sudo ./install.sh

If you get an "illegal instruction" error, it most likely means the CPU you are using doesn't support the SSE3 instruction set. You need to build OpenNI from source:

$ sudo apt-get install default-jdk
$ git clone https://github.com/OpenNI/OpenNI
$ cd OpenNI/Platform/Linux/Build
$ chmod a+x ../CreateRedist/install.sh
$ chmod a+x ../CreateRedist/RedistMaker
$ SSE_GENERATION=2 sudo -E make install
$ cd ../Redist/OpenNI-Bin*
$ sudo ./install.sh

OpenNI SensorKinect

Clone and install the Kinect sensor module for OpenNI:

$ git clone https://github.com/avin2/SensorKinect.git
$ cd SensorKinect/Bin
$ tar -xjf SensorKinect091-Bin-Linux64-v5.1.0.25.tar.bz2
$ cd Sensor-Bin-Linux-x64-v5.1.0.25
$ sudo ./install.sh

If you get an "illegal instruction" error, it most likely means the CPU you are using doesn't support the SSE3 instruction set. You need to build the sensor module from source, too:

$ cd SensorKinect/Platform/Linux
$ chmod a+x

$ git clone https://github.com/OpenNI/OpenNI
$ cd OpenNI/Platform/Linux
$ chmod a+x CreateRedist/install.sh
$ chmod a+x CreateRedist/RedistMaker
$ cd Build
$ SSE_GENERATION=2 sudo -E make install
$ cd ../Redist/Sensor-Bin*
$ sudo ./install.sh

USB Device Setup

Block the Kinect webcam driver that's new in the 3.0 Linux kernel from loading by adding this to the end of /etc/modprobe.d/blacklist.conf:

blacklist gspca_kinect

Fix USB permissions by copying the file SensorKinect/Platform/Linux/Install/55-primesense-usb.rules to /etc/udev/rules.d. Finally, restart udev and plug in the Kinect:

$ sudo /etc/init.d/udev restart

Compiling

Back in the head-post-estimator directory, generate the Makefile and run the first example:

$ cmake .
$ make
$ ./head_pose_estimation config.txt data/frame_00100_depth.bin
...this should print out the head it detected in the data

Next, compile and test the live Kinect example:

$ cd demo
$ cmake .
$ make
$ cd ..
$ ./head_post_estimation_demo config.txt

At this point, a window should pop up showing the live depth view from the Kinect and a head pose estimation.