The Sensor class lets you stream live data from various sources in real-time. More specifically, it allows you to specify the source, configure its properties, and capture frame data (color, depth, IMU, skeleton).
Sensor Configuration
The sensor configuration controls the basic characteristics of the input source.
Sensor Type
Specifies the type of the sensor from the list of the supported sensor types:
- Webcam: USB cameras, phone rear & front cameras
- Apple LiDAR: iPhone and iPad depth camera
- RealSense: Intel RealSense D415, D435, D435i, D455, D405, and L515 depth cameras
- Orbbec: Orbbec Femto, Femto W, and Astra+ depth cameras
- Structure: Occipital Structure Core depth camera
- Kinect: Microsoft Azure Kinect depth camera
- OAK-D: Luxonis OAK-D and OAK-D Lite depth cameras
- Video: Video file input (MP4, MKV, AVI, MOV, WMV)
Device Index
The index of the camera. When there are multiple cameras, the device index allows you to specify which one to use.
Requested Frame Rate
Many cameras can run on different frame rates. For example, RealSense can run at 15, 30, 60, or 90 FPS. Set the frame rate property to specify the desired rate. If you specify an invalid value, a default value will be applied.
Requested Color Resolution
Just like the frame rate, one camera may support different color resolutions (frame width & height). For example, a webcam may support 640×480 or 1280×720. Specify the resolution of your choice. If you provide unsupported values, a default resolution will be used.
Requested Depth Resolution
Similar to the color resolution, the depth resolution specifies the width and height of the depth frames. It’s only available on depth cameras.
Smoothing
Smoothing allows you to control or eliminate the amount of jitter. It’s a floating-point number between 0 and 1. 0 indicates there will be no smoothing applied and, as a result, data may be jittery. 1 indicates that the highest level of smoothing will be applied. We recommend a value between 0.1 and 0.3.
Smoothing Type
The smoothing type allows you to control the underlying method of reducing jitter. LightBuzz supports two types of smoothing:
- Dynamic: dynamic smoothing automatically adjusts to the speed of the movement while also considering previous data. It’s the recommended option for most scenarios.
- Legacy: legacy smoothing searches past and future frames to average and interpolate between them.
Enable Steady Rate
Specifies whether the Sensor should serve data at a steady or variable rate. Typically, the system needs time to process the raw data, so it serves skeleton frames right as they get processed. When turning this setting on, the system will use a separate thread to serve the frames at a consistent rate.
Video Path
When using a video file as the input source, the video path property specifies the absolute path to that file. The file needs to be in the local filesystem of the computer or phone, while the app needs to also have access (read) rights to it.
Sensor Methods
The Sensor class includes the following methods that give you control over the sensor:
Open()
Opens the input source and starts streaming data.
Close()
Closes the input source and stops streaming data.
Update()
The Update() method returns the most recent frame captured by the camera.
Sensor Properties
The Sensor properties include settings that can change at runtime.
Brightness (0 to 1)
The Brightness setting adjusts the lighting of the color image. 0 indicates no light, while 1 indicates very bright light. Increasing brightness is helpful in dark environments. Defaults to 0.5.
Contrast (-1 to 1)
The Contrast setting adjusts the contrast of the color image from -1 to +1. Defaults to 0.