InSight  3.2.3
The Face Analysis Toolkit
InSight SDK


InSight SDK is a cross platform software library which can be used to seamlessly integrate face analysis and facial expression recognition in third party applications. The SDK is extremely flexible, and allows retrieving tailored information depending on the specific requirements of the companies or individuals. Via a simple webcam the InSight SDK can be used to automatically analyze face videos in real time, and communicate the resulting information to a third party application. The information ranges from seven emotional categories (neutral, happy, surprised, sad, afraid, angry and disgusted) to head pose, gaze, eye movements and age and gender of the subjects. The SDK is based on the following state-of-the art technologies developed in collaboration with the University of Amsterdam:

  • Face Detection
  • Facial Features Detection
  • Motion Tracking
  • Head Pose Estimation
  • Eye Localization
  • Gaze Estimation
  • Emotion Recognition
  • Age and Gender Estimation

The SDK can be easily integrated in third party software as a collection of C++ shared libraries for Windows, Mac and Linux environments. The SDK requires an internet connection to function properly, as it will communicate with our API to validate the license.


InSight SDK works on a video sequence of frames. The first frame should be the initialization frame, after that all frames should be processes and the information about the user can be retrieved by using the getter functions. The following is the pseudo-code to retrieve and display the facial expressions of a user in each frame:

Initialize InSight and the capturing device/video input
for (;;)
capture >> frame; // get a new frame from the video stream
if (!insight.isInit())
insight.init(frame) // initialize InSight with the first frame
insight.process(frame) //process the new frame
insight.getEmotions(emotions) //retrieve the facial expressions
print(emotions) //print them to console

For the full code, refer to the example implementation provided with the SDK. Additional usage examples can be provided upon request. The following is the pseudo-code to calibrate and obtain eye gaze information on each frame:

// fill a vector with at least nine calibration points
std::vector<cv::Point> calibrationPoints = nine calibration points;
std::vector<cv::Point>::iterator it = calibrationPoints.begin();
// this vector will accumulate processed calibration points
std::vector<CalibInfo> calibCollection;
cv::Mat frame;
bool isCalibrated = false;
int drawing = 0;
while( grab( frame ) )
// initialize insight if necessary
if( !insight->isInit() && !insight->init( frame ) )
printError( insight->getError() );
// always process a frame
if( !insight->process( frame ) )
printError( "Failed to process" );
if( !isCalibrated )
cv::Point & calibrationPoint = *it;
if( it != calibrationPoints.end() )
// draw calibrationPoint on screen for 25 frames
// to allow the user to fixate on the point
if( drawing < 25 )
drawDotOnScreen( calibrationPoint );
drawing = 0;
// process calibration point
if( !insight->addCalibrationPoint( calibrationPoint, ci ) )
printError( "Failed to add calibration point" );
// add processed calibration point to collection
calibCollection.push_back( ci );
// move to next calibration point
// calibrate using the processed calibration point collection
std::vector<cv::Point2f> calibErrors;
if(!insight->calibrate( calibCollection, calibErrors) )
printError( "Failed to calibrate" );
isCalibrated = true;
// do something with the reprojection errors if required
printReprojectionErrors( calibErrors );
// insight get gaze
cv::Point estimatedGaze;
insight->getEyeGaze( estimatedGaze );
// do something with the acquired gaze point
drawOnScreen( estimatedGaze )


Platform requirements

The minimum platform requirements.

  • Intel Core 2 Duo 1.6GHZ or better.
  • 2GB RAM
  • Input frames with a resolution of at least 640 480
  • Active Internet connection

User requirements

InSight SDK is designed for a single-user scenario, in which the user is around 60 cm away from the camera (eg: like in front of a desktop or a laptop computer). The following user requirements should be met in order to use the InSight SDK.

  • The user should be approximately at the center of the camera image that is used for initialization
  • The user should sit at 60 centimeters from the screen and the camera (attached to the top center of the screen)
  • The user should assume a frontal face position with a neutral expression during initialization
  • For gaze estimation, the user should not move his head too much during calibration

Environment requirements

The following environment conditions should be met in order to use the InSight SDK.

  • Good illumination so face is clearly visible
  • No light source behind the user (windows, lamps etc)