ChronoSense

Overview

Combining learnings from prior work, I rebuilt the Lab In A Box software into a unified multimodal data capture system.
ChronoSense evolved from a Windows-only application written in C# to an OS-agnostic Electron application.
ChronoSense supports time-aligned data capture from multiple audio and video devices and has a plugin architecture to support integrating other devices via the Node.js C++ addon layer.

ChronoSense is an open-source software available at https://chronosense.ucsd.edu/.
ChronoSense has supported a wide range of research around multimodal human behavior analytics.

The Lab In A Box system first used a piece of software known as Kinect4ChronoViz to capture data from a Kinect v1 and save it to disk as a series of images from the color and depth camera and all body tracking via CSV files.
As I used Kinect4ChronoViz in different settings I learned about its utility and limitations for researchers and sought to improve the tool to be more widely accessible.
ChronoSense was officially launched with the launch of the Kinect v2, which brought to light many shortcomings of the old software with a new device that had higher framerates and resolutions.

ChronoSense

With the launch of Azure Kinect DK, I led the final ‘web-tech-ification’ of ChronoSense, moving over to Typescript, Electron, and Node.js, building a truly OS-agnostic version of the software.
The system was redesigned to support time-synchronized capture from any camera or audio source, using a combination of audio and video APIs as well as FFmpeg to handle encoding and compression.
The Node.js C++ addon layer was used to support additional capture of other devices and data streams, such as body tracking from the Kinect, EMG or IMU from wearables, etc.

These are works that I did not author that used ChronoSense: