Sightcorp logo

1.About CrowdSight Toolkit #

About CrowdSight Toolkit

CrowdSight Toolkit is a simple cross-platform plug and play solution that enables you to detect and analyze multiple faces using just one application!

Get insights into your viewers and optimize messaging in real-time with no need for hard coding. The software can be installed with ease by anyone within minutes!

1.1.Analyzed features #

  • Emotion Recognition
  • Age Estimation
  • Gender Estimation
  • Ethnicity Estimation[1]
  • ID Detection
  • Head Position/Head Gaze
  • Attention Time
  • Clothing Color

 

[1] Not available for EU

1.2.Key benefits #

  • 14 Analyzed features: age, gender, emotion, head pose, attention and more!
  • Easy and quick setup: the software can be installed with ease by anyone within minutes.
  • Affordable pricing: start from 15€/month!
  • CMS Integration: using http request in JSON format.
  • Real-time results: no need to wait for data to be displayed.
  • Anonymous analysis: no video is ever recorded and no identifiable facial features are stored.
  • Cross-Platform: Toolkit runs on Windows, Linux and OS X (Android coming soon!).
  • Processed Locally: all data is analyzed on the device.
  • Push mechanism: send messages from Toolkit to external applications.
  • Works with most cameras: from budget USB cameras to high definition IP cameras.
  • No internet needed: analysis runs offline (one-time authentication).
  • 1.3.Quick installation guide #

    Preparations for installation:

    Step 1. Make sure your device has Internet connection for license authentication (by browsing google.com, for example).

    Step 2. Make sure the web/inbuilt camera on your device is installed and is running as expected (by opening camera app, for example).

    Installation:

    STEP1

    Download the CrowdSight Toolkit for your preferred operating system.

    For Windows device, please allow the app to make changes to your device and run it.

    For Mac device, please drag and drop the file to your application folder and allow the security settings to download the software.

    STEP 2

    Follow the setup instructions and accept the licensing agreement to continue.

    STEP 3

    You can now access your application folder and double click to activate and initialize the Toolkit.

    STEP 4

    After the Toolkit is installed, you will need to enter a valid license.

    Go to the “Licensing” tab to add your license key.

    You are now ready to start using the CrowdSight Toolkit.

    2.Using the Toolkit #

    3.Functionalities #

    3.1.CSV export #

    A CSV file is a comma separated values file which allows data to be saved in a table structured format. The Toolkit can generate a CSV file which contains raw data gathered during the real-time analysis. This data can be used for post-processing.

    To enable CSV saving go to Output and make sure the checkbox Save data to CSV is checked. By default, the CSV file will be automatically generated in the Toolkit folder after each analysis. Users can also specify the location where the CSV file will be stored. Every row in the CSV file contains the analysis of 1 person.

    IMPORTANT: If you install the Toolkit on its default location (eg. “Program Files (x86)”, the default path for storing CSV files is within its installation directory and will need Administrator permission to write the files. You can start the Toolkit with Administration rights to write to this directory, or select another directory where you do not need Administration rights to write the CSV files to.

    Toolkit Output tab

    You can select a directory where to save the files to, and optionally choose to also dump the face landmarks. The landmarks will increase CSV file size significantly due to the number of landmarks. The first row of the CSV file contains the label of the respective column.

    When doing video analysis, you can choose to run analysis at the speed of the video or you can choose to process every frame of the video. Running the analysis on every video frame will be considerably slower. If you choose to analyse at the speed of the video, new frames are dropped until the processing of the input frame is finished. You will notice non-consecutive frame numbers in the CSV file.

    3.1.1.Item description #

    Frame: this is the frame number as analyzed from the video feed. The analyzed video is divided in single frames, and each frame is given a number in the sequence.
    Therefore the first frame being analyzed is frame 1, the second is frame 2 and so on. Only frames with analysis are saved to the CSV. If a frame does not produce an analysis, it will be skipped in the CSV.

    Timestamp: this is the time when the frame was analyzed, according to the PC clock. The time is expressed in unix timestamp as it is the most widely accepted way for expressing time.
    There are services like this to convert from unix timestamp to a human readable format. Excel, other tools and programming languages can also understand and convert the timestamp format to other formats.

    One frame is relative to one moment in time and therefore it appears in only one timestamp, but in one timestamp multiple frames can be analyzed. Multiple frames can be analyzed in the same second. If the frame rate (fps) is 10, then there will be 10 frames in the same timestamp (timestamp is expressed in seconds).

    ID: this is the unique ID assigned by the toolkit to a detected person. Therefore if a person with an ID appears in different frames, it is the same person, with the exception of ID 0 (zero). ID 0 means that the system has not enough information yet to understand if it is a person previously analyzed. By default, at least 3 frames are needed in order to assign an ID to a person, otherwise ID 0 will be given to all the detected faces which cannot be ‘recognized’. The EvidenceLevel can be changed in data/settings.ini such that an ID is assigned immediately or after a different amount of frames other than 3, however lowering this number will lower the confidence to track the same person.

    A unique visitor is identified by its ID. Please keep in mind that there’s a time limit in which the system can assign the same ID to the same person. If a person is detected again after (for instance) one hour, it will be given a different ID.

    Age: This value represents age estimation per detection and ranges from 0 to 99 in years.

    Gender: This value represents the confidence of a detection belonging to a certain gender category. The confidence ranges from -99 to 99, where -99 is considered a high confidence for the subject being male and 99 a high confidence for the subject being female. The closer the estimation is to 0, the more uncertain the estimation is.

    Ethnicity: The ethnicity estimations are grouped in four categories: African, Asian, Caucasian and Hispanic. Each number entry represents the estimation for ethnicity type per face detection.

    Headpose: The direction of where a person is looking at is represented in the head pose using pitch, yaw and roll in degrees.

    headpose example

    Viewing: This value shows if the subject looked towards the point of interest (e.g. camera, digital screen, etc.). Number 1 stands for ‘yes’ and 0 means ‘no’. This value does not include subjects who were detected but did not look towards the point of interest.

    Headgaze:
    The position of a person’s gaze on top of the point of interest. The origin of the position is considered the camera and the value is expressed as a point
    headGaze_x:
    headGaze_y

    Attention: This number represents the amount of time spent looking toward the point of interest. The time is expressed in milliseconds.

    Interest: This number represents the aggregate value of multiple people looking at the same point of interest. The higher the number, the more people are looking towards the same point of interest.

    Mood: This number represents a quantity of positive facial expression and ranges from 0 to 100.
    Happiness: one of the 6 basic facial expressions, ranging from 0 to 100 where 100 is the maximum happiness index.
    Surprise: one of the 6 basic facial expressions, ranging from 0 to 100 where 100 is the maximum surprise index.
    Anger: one of the 6 basic facial expressions, ranging from 0 to 100 where 100 is the maximum anger index.
    Disgust: one of the 6 basic facial expressions, ranging from 0 to 100 where 100 is the maximum disgust index.
    Fear: one of the 6 basic facial expressions, ranging from 0 to 100 where 100 is the maximum fear index.
    Sadness: one of the 6 basic facial expressions, ranging from 0 to 100 where 100 is the maximum sadness index.

    Face_x: top left x coordinate in pixels of the face location.
    Face_y: top left y coordinate in pixels of the face location.
    Face_width: the width of the rectangle enclosing the detected face.
    Face_height: the height of the rectangle enclosing the detected face.

    left_eye_x: x coordinate in pixels of the left eye position
    left_eye_y: y coordinate in pixels of the left eye position
    right_eye_x: x coordinate in pixels of the left eye position
    right_eye_y: y coordinate in pixels of the left eye position

    Mask points: 68 points that indicate different parts of the face, eg. eyebrows, eyes, nose, mouth and the yaw line.

    3.1.2.HTTP interface #

    You can connect your Business Intelligence platform or CMS to the Toolkit using the HTTP interface of the Toolkit. Go to ‘Output’ Tab and click on the field: ‘Enable HTTP Interface’ and ‘Send processed image data through socket’.

    You can now access the JSON response through a GET request on localhost for your external application. You can use this function for instance for Digital Signage or narrowcasting application which trigger custom and relevant ad content to the right people at the right time.

    Toolkit HTTP interface

    Example JSON output
    JSON output example

    3.1.3.Push mechanism #

    The new push mechanism introduced in the Toolkit allows for messages to be sent from the Toolkit to external applications (eg. CMS systems) instead of requiring the external application to actively ask the toolkit for changing status (pulling). Therefore it’s the Toolkit pushing data instead of the external application pulling data. If the “push mechanism” feature is enabled, the Toolkit will send every second an HTTP POST request to the endpoint defined by the user. The endpoint needs to be a valid HTTP server able to receive HTTP POST requests. The body of the HTTP POST request is a JSON object containing a summary of the last second of analysis.

    Toolkit push mechanism

    3.1.4.Auto-restart #

    It might be desirable to start the Toolkit on startup of the system. For both platforms it is required that you log in automatically when the system is turned on, eg. you are not required to enter a password and you’re instantly booted to the Desktop.
    Windows 10

    In the installation directory of the Toolkit there is a default start.bat script that can be used and modified for automatic startup of the Toolkit from the command line. Once the start.bat script has been modified in accordance to your preference, you can use this script for automatically starting up the Toolkit as follows:

    1. Use the Windows key + R keyboard shortcut to open the Run command
    2. Type shell:startup to open the Startup folder of Windows. All shortcuts placed in this directory will be called when Windows has started
    3. Right-click and select New > Shortcut
    4. Browse and select the start.bat file
    5. Double-check if the Toolkit starts up when you double-click the shortcut
    6. If the Toolkit launches in step 5, you’re all set for automatic startup

    Please see our step-by-step guide to learn more about how to automatically start Crowdsight Toolkit v3.2.0 after Windows reboot.

    3.1.5.Multiple configurations #

    This update comes with an improved system to store settings from your Toolkit configuration. Previously the settings were stored in Registry, making it difficult to run multiple instances of the Toolkit using different configurations on the same machine. With this update we now store the settings to .ini files in a configuration directory that can be selected through the UI.

    You can specify any directory to be a configuration directory and share this directory structure between Toolkit instances or launch multiple Toolkit instances pointing at different configuration directories. The start.bat script inside the installation directory shows how you can specify a configuration directory from command line and allows to launch multiple Toolkits using different configurations on startup of your system.

    3.1.6.Disable Windows updates #

    When running the Toolkit in a production environment, it is often desired to not have the system update and restarting itself which in turn stops the Toolkit from running and analysing camera feeds.

    Previous versions of Windows allow the user to simply disable the update service and do manual updates whenever convenient. Since Windows 10, Microsoft has made changes in their update policy making it more difficult to stop this process.

    Schedule downloads and updates

    The following steps will assure that you will be given the opportunity to schedule the download and installation of Windows updates:

    1. Use the Windows key + R keyboard shortcut to open the Run command.
    2. Type gpedit.msc and click OK to open the Local Group Policy Editor.
    3. Browse the following path:

    Computer Configuration\Administrative Templates\Windows Components\Windows Update

    windows_updates_path_directory

    4. On the right side, double-click the Configure Automatic Updates policy

    5. On the left side, check the Enabled option to enable the policy

    6. Under Options, you’ll find a number of ways to configure automatic updates, including:

    • Notify for download and notify for install.
    • Auto download and notify for install.
    • Auto download and schedule the install.
    • Allow local admin to choose setting.

    (While you can pick from any of the available options, your best choice is probably 2 – Notify for download and notify for install. This option will not only prevent the OS from downloading updates automatically, which can save you data when using a metered internet connection, but you’ll get a notification when new updates are available.)

    7. Click Apply

    8. Click OK to complete the task

    Disable updates

    It is not recommended to completely ignore all updates since some security updates should be mandatory. However, to turn off the update mechanism completely you will need to alter the type of connection your system has with the internet.

    1. Go to the Windows start menu on the bottom left and type Settings.

    2. Within Settings, go to Network & Internet.

    3. Under Status, select Change connection properties

    4. You will see the header Metered connection. If you set your connection as metered connection, Windows will treat it as a connection with limited data usage, preventing any updates to be downloaded.

    5. Switch Set as metered connection to On

    6. You’re done

    3.1.7.Logging system #

    The new logging system keeps track of start-up configuration, network issues and potential errors that can occur during the analysis. This allows for diagnostics and improves the effectiveness of our support service.

    Toolkit creates a log file in the log directory under your configuration directory. On Windows, if a custom configuration directory is not defined, then you can find the log directory under the C:\Users\\AppData\Local\CrowdSight-Toolkit\Log\ folder.

    3.1.8.Local reporting dashboard #

    COMING SOON!

    4.FAQ #

    Please refer to our help desk.

    5.Privacy policy #

    Please contact us for further resources.