C++ SDK Development Guide
About 3661 wordsAbout 12 min
Version: 0.0.2
Current package platform: windows_x64
This document explains how to integrate and use the binary SDK. Users do not need the SDK source code or to recompile the driver. The namespace used here is dexsense, and the public header is dexsense/dexsense.h.
1. Integration and Deployment
Find the package with find_package(DexSense 0.0.2 EXACT CONFIG REQUIRED), link dexsense::dexsense, and enable C++17. The included examples/CMakeLists.txt demonstrates the complete integration. Set CMAKE_PREFIX_PATH to the SDK root directory for the current platform. The CMake exported target provides the include directories and link information together.
A successful build does not mean the system can find the libraries at runtime:
| Platform | Used when linking | Runtime deployment |
|---|---|---|
| Windows x64 | Import libraries in lib/ and the exported target | Add this package’s bin/ to the current process’s PATH, or place the required DLLs beside the application; use a compatible MSVC runtime |
| Linux | Link names in lib/ and the exported target | Use LD_LIBRARY_PATH during development; for release, place the SO files in the application’s lib/ directory and set a relative RPATH on the executable |
One directory layout for a released Linux application is app/bin/my_camera_app and app/lib/lib*.so*. In the application’s CMake configuration, set INSTALL_RPATH "$ORIGIN/../lib" on the executable target. $ORIGIN refers to the directory containing that executable at runtime. Setting this value does not automatically copy the SDK libraries or change the dependency search paths of other libraries. During installation and deployment, retain the complete SO files and their chain of versioned symbolic links; do not copy only the unversioned .so link. ldd <application> can be used to check for missing runtime libraries.
Do not link Windows libraries on Linux, or mix x64 and ARM64 libraries. The headers, libraries, examples, and documentation in the same package correspond to the same version. If the binary ABI or system runtime is incompatible, choose a matching release package.
2. Objects and Lifecycle
The usual call sequence is Context → IDiscoveredDevice → open() → Camera → capture() → Frame:
Contextprovides discovery, lookup by IP, and logging configuration.IDiscoveredDevice::info()provides information available during discovery;open()returns an initializedCamera. Obtain detailed information after connection, such as the firmware version, fromCamera::getDeviceInfo().Cameracan be moved but not copied; it owns connection, heartbeat, and capture resources. When finished, stop streaming, wait for asynchronous tasks, and then calldisconnect()from an external control thread.Frameowns the data for that frame.getImage(),getDepthMap(), andgetPointCloud()return constant references to objects within the frame. When using these references anddata()pointers, ensure that the corresponding objects and underlying data remain alive. To retain data across threads for an extended period, an application can retain theFrameor copy the data itself.The reference returned by
camera.parameters()belongs to that camera and must not be used after the camera is destroyed.
The application must coordinate capture and parameter operations on the same camera so that they run serially. Do not treat a single camera object as generally thread-safe for arbitrary concurrent calls; in particular, avoid changing KV parameters from another thread during capture.
3. Discovery, Direct Connection, and Network Configuration
discoverDevices(timeout) uses broadcast to discover network cameras. An empty list means only that no camera was discovered during this attempt; it does not necessarily mean the camera is offline. Broadcast discovery can be affected by subnets, firewalls, network adapter selection, or the network environment. The example’s --discover option only lists information; it does not automatically connect to the first camera in the list.
const auto devices = context.discoverDevices(std::chrono::milliseconds(3000));
for (const auto& device : devices)
{
const auto info = device->info();
std::cout << info.model << " IP=" << info.ip << " MAC=" << info.mac << '\n';
}
std::cout << "Discovered " << devices.size() << " device(s).\n";When the IP address and model are known, prefer findByIp(ip, DeviceModel::Xema/Finch/..., timeout), followed by open(). This overload creates a device entry for the specified model without broadcasting or verifying that the device is currently reachable. The actual TCP connection, protocol handshake, and driver initialization occur in open(). The overload that takes only an IP address first performs device discovery and then filters the results by IP. It does not probe over TCP automatically or fall back to a direct connection. If broadcast cannot reach across subnets, specify the model explicitly.
The explicit-model overload passes timeout to the underlying connection. TCP connection establishment, the protocol handshake, and driver initialization share this time limit. After driver initialization, the normal operation timeout is restored. Wrapping the Camera may still read initial parameters afterward, so this is neither a strict upper bound on the total wall-clock duration of open() nor a timeout setting for each capture().
USB devices use a separate entry point: supported platforms and models can call findByDevicePath(), for example with a Linux KFC V4L2 device path. Do not pass a network IP address to the USB interface. findByUsbId() is a reserved entry point; its presence does not mean USB enumeration has been implemented.
setDeviceIp() and setDeviceDhcp() configure network addresses and should receive the complete DeviceInfo obtained through discovery. A successful call means only that the configuration message was sent from the host. Rediscover the device by MAC afterward and verify its final address. Changing the address may interrupt the current session, so perform it as a separate configuration operation, not within a capture loop.
4. Parameter Configuration: Whole Settings and the Key-Value Interface
The two interfaces use the same camera driver but serve different purposes.
| Need | Method | Considerations |
|---|---|---|
| Set exposure, gain, filter structures, or HDR groups | Obtain Settings, modify its fields, then call applySettings() | Changes are applied as a diff; check the returned result. This is not an atomic transaction |
| Interactively change one common parameter | Convenience methods such as setExposureMs() and setGainDb() | These still return ApplySettingsResult; do not discard it |
| Use device extension parameters | parameters().setFloat() / setInt() | Call hasKey() first; consult the interface reference and device descriptors for types and ranges |
| Check whether firmware or the driver provides a parameter | isParamSupported(ParamKey::...) | This does not mean every parameter supports readback |
| Build a parameter interface | getParamDescriptors() and parameters().keys() | Descriptor keys such as exposure_ms differ from KV keys such as ExposureTime and cannot be interchanged directly |
Settings::defaults() is a fixed SDK default template, not a command to restore device factory settings or a representation of the camera’s actual current parameters. For routine adjustments, copy the current cache from camera.getSettings() before modifying it. Settings::validate() performs basic local range checks; it does not guarantee that the current hardware and firmware will accept a value.
The unified interface’s Settings::setExposureMs() and the KV parameter ExposureTime use milliseconds. The FINCH extension RgbExposure uses microseconds. Do not use the same numerical value merely because both names contain “Exposure.” Use setInt() with the corresponding enum conversion for enum parameters. Boolean KV parameters generally use integers 0/1. Structure parameters are not read or written through KV; use their corresponding structures in Settings.
Check each result for critical settings:
ParamStatus | Meaning | Application handling |
|---|---|---|
Applied | The SDK successfully performed this write | Read back and verify critical parameters |
Skipped | The value matched the cache, so the diff was skipped | This does not mean a command was sent or a readback occurred during this call |
Unsupported | The current device does not support the parameter | isSuccess() can still be true; stop if the application requires this capability |
Failed | Applying this item failed | Inspect the error details and handle the partially successful state |
applySettings() processes items individually; a partial failure does not roll back parameters that were applied successfully. The current implementation synchronizes only applied device parameters to the Settings cache. Capture mode and repetition count are client-side settings and are updated separately in the cache. The overall result may also list other fields that this model does not support and that were not changed in this call. The quick start stops subsequent capture for any Failed item. For Unsupported, check whether it is a required parameter explicitly requested in this call, so an optional field the device never supported is not mistaken for a capture failure.
5. Cache, Readback, and Saving Configuration
You must distinguish “the desired setting” from “the value actually returned by the device”:
getSettings()does not access the device. It returns only the SDK cache and is not evidence that parameters were accepted.readSettingsFromDevice()attempts to read device parameters covered bySettingsand update the cache. The current batch read skips individual read exceptions. Fields that were not read successfully may retain default values, and pure KV extension parameters are not included. Therefore, receiving aSettingsobject does not mean every field was read back successfully.KV
getFloat()/getInt()call the driver to read values and preserve exceptions. Explicitly call them for critical readable parameters and compare the results with the target values. Client-side parameters or write-only parameters in a particular driver may not support device readback.KV
setFloat()/setInt()remember previously written values that succeeded; a repeated value may skip sending a command. Some standard parameters also update theSettingscache after a KV write. A KV read does not automatically merge all extension parameters intoSettings.
auto& parameters = camera.parameters();
if (parameters.hasKey("ExposureTime"))
{
// ExposureTime 在统一接口中的单位为 ms;调用驱动回读,不是 getSettings() 缓存。
const float actualExposureMs = parameters.getFloat("ExposureTime");
std::cout << "ExposureTime readback=" << actualExposureMs << " ms\n";
}hasKey() indicates that a KV parameter is available for the current device; it does not guarantee read support. Calling get*() on a write-only parameter may throw NotSupportedError. If an external program changes the device, the application reconnects, or different configuration entry points are used in turn, do not assume the cache has detected the change automatically. Read back critical values individually and, if necessary, create a new session before configuring the device again. Do not repeatedly set the same value as a way to force a command to be resent.
Settings provides saveToFile()/loadFromFile() and YAML and JSON serialization. These save the fields covered by Settings, not all pure KV extension parameters, and are not a lossless backup of device configuration. If the application uses extensions such as CaptureEngine or CameraSelect, save them in its own configuration format, check support when loading, and apply them again. Do not confuse serialized configuration with a device calibration document.
6. Capture, Images, Depth, and Point Clouds
Finish configuring parameters before calling capture(). Capture uses the current device parameters and client-side capture settings; it does not send all of Settings to the device again. After obtaining a frame, check isEmpty() separately for each type of data needed. frame.isEmpty()==false does not guarantee that an image, depth map, and point cloud are all present.
| Data | Access interface | Layout and units |
|---|---|---|
| Main structured-light image | frame.getImage() | Check width, height, channels(), and format(); channel count alone does not establish RGB/BGR order |
| Left and right stereo images | frame.getLeftImage() / getRightImage() | KFC stereo images use separate left and right images; an empty getImage() does not indicate capture failure |
| Depth map | frame.getDepthMap() | float, stored by row, in millimeters; at(row,col) accesses it by row and column |
| Point cloud | frame.getPointCloud() | Interleaved XYZ [x0,y0,z0,x1,y1,z1,...], in millimeters |
Grayscale images, color images, and raw Bayer images are distinct. Read PixelFormat to determine how to interpret channels. When displaying images through a third-party library such as OpenCV, handle RGB/BGR according to that library’s requirements rather than always swapping channels. The SDK’s savePng() / saveBmp() already handle the supported three-channel BGR conversion when saving. Raw Bayer data requires separate processing according to its format; changing the filename extension alone does not produce a color image.
Invalid depth values must be filtered. The SDK’s isValid(row,col) checks for a positive value that is not NaN; application calculations can also use std::isfinite() to exclude infinite values. Do not treat an invalid point as a zero-distance measurement.
frame.saveAll(directory) creates the directory and saves the content present in the frame: the point cloud as pointcloud.ply, the structured-light image as brightness.png or color.png, and stereo images as left.png and right.png. It does not save the depth array or Settings, and it does not automatically create a separate subdirectory for each frame. To save individual items, use Image::savePng()/saveBmp() or PointCloud::saveToPly()/saveToPcd(). Check output files and disk space according to the application’s needs.
The quick start adds a depth CSV with the following application-side function so the data can be checked directly. Saving a large image pixel by pixel as text is costly in production. You can choose your own binary format and record its width, height, floating-point type, and units alongside it.
/**
* @brief 按图像行列保存毫米单位的深度 CSV,无效点写为 nan。
* @param depth 待保存的深度图。
* @param path 目标文件,父目录必须已存在。
* @throws std::runtime_error 文件创建或完整写入失败。
*/
void saveDepthCsv(const dexsense::DepthMap& depth, const std::filesystem::path& path)
{
if (depth.isEmpty()) return;
std::ofstream output(path);
if (!output) throw std::runtime_error("Cannot create depth CSV: " + path.string());
output << std::setprecision(std::numeric_limits<float>::max_digits10);
for (int row = 0; row < depth.height(); ++row)
{
for (int col = 0; col < depth.width(); ++col)
{
if (col != 0) output << ',';
const float value = depth.at(row, col);
if (depth.isValid(row, col) && std::isfinite(value))
output << value;
else
output << "nan";
}
output << '\n';
}
output.close();
if (!output) throw std::runtime_error("Cannot finish depth CSV: " + path.string());
}getCaptureSettings() is an SDK settings snapshot associated with the frame. It does not contain the complete KV extension configuration, nor is it a state read from every device register at the instant of exposure. getTimestamp() returns a text timestamp provided by the SDK. Do not use it for strict multi-camera hardware synchronization before verifying the clock semantics of the specific device.
7. Selecting Modes and Parameters by Device
A unified interface does not mean every model, firmware version, and transport method supports the same features. First query isFeatureSupported(), isParamSupported(), KV hasKey(), and parameter descriptors, then consult the complete example for the corresponding model in the package.
XEMA: See
xema_capturefor basic capture.CaptureEngineselects the capture engine, whileSettings::brightness()configures brightness-image generation; they are not interchangeable. Main exposure and brightness-image exposure must also be treated as separate parameters. Color output depends on the actual camera hardware and capture path. Configure the required path as shown in the example, then check the returned image format. Setting a single “RGB switch” does not guarantee color output in every mode.FINCH: See
finch_capturefor the complete parameter workflow.CameraSelectusesCameraSelect::Gray/RGBHalf/RGBFullto select a path. Check support before using extensions such as Gamma, RGB exposure/gain, block outlier filtering, and depth completion. Features in new firmware may be absent from older firmware; the model name alone does not establish that every parameter is available.KFC: Obtain stereo data from the left and right image interfaces. The current generic frame does not provide structured-light depth and point clouds where none exist. Use the corresponding examples for the USB and Ethernet paths. Capabilities depend on the transport method and firmware.
HDR, repeated capture, and streaming capture: Use
CaptureModetogether with device capabilities such asFeature::HDRandFeature::Streaming. See the model examples andhdr_yamlfor HDR groups and combinations of LED and exposure settings. Check capability first, then apply the settings and inspect the result.AI structured light: Use a device that supports this capability,
Feature::AiStructuredLight, and the multimodal interface.captureMultiModal()returns information about roles and data categories; do not treat the subimage list as ordinary RGB images in a fixed order. Use dedicated interfaces to read on-camera calibration and lookup tables. See the relevant model examples included in the package, if this platform’s package contains them.
The parameter values shown in examples are not production defaults suitable for every target. In particular, validate exposure, LED, and filtering against your own image and point cloud results. For a device and firmware combination that has not been tested on hardware, successful compilation does not establish that it will produce equivalent results.
8. Asynchronous and Continuous Capture
captureAsync() returns std::future<Frame>: submit one capture, process the previous frame on the current thread, and finally call future.get() to obtain the result. Capture exceptions are rethrown at get(). Do not move, destroy, or disconnect the Camera referenced by an asynchronous task before that task finishes. Do not submit multiple capture requests at once expecting the same device to expose images in parallel.
Before startStreaming(callback), check isFeatureSupported(Feature::Streaming). Complete configuration before starting the stream. An SDK worker thread invokes the callback for each frame. Keep work in the callback brief, such as passing the frame to a capacity-limited work queue, so disk writes or time-consuming algorithms do not block capture.
The current implementation holds an internal camera lock while invoking the frame callback. Therefore, do not call that camera’s capture(), parameter configuration methods, getSettings(), stopStreaming(), or disconnect() from within the frame callback; doing so may cause lock waits or a thread to wait for itself. Have an external control thread observe state or notifications, call stopStreaming() and wait for the stream to exit, and only then change parameters, restart, or disconnect. A disconnect callback should likewise only report the event; an external thread should handle recovery. The application must keep objects captured by callbacks valid until stopping is complete and handle exceptions thrown by its callbacks appropriately.
Observe stream errors through logs, stream state, and possible disconnect notifications. They do not automatically become exceptions on the thread that called startStreaming(). Stop the stream explicitly before the application exits, and handle frames still pending in the queue.
9. Reading and Writing Calibration
getCalibration() provides a calibration view for general calculations; it is not a complete, lossless device backup. Use exportActiveCalibration() / exportOriginalCalibration() for lossless export, after querying capability with isCalibrationOperationSupported(). CalibrationDocument contains device identity, firmware version, payload format, raw payload, and checksum. isValid() validates the payload and checksum.
Calibration reads and writes must be treated differently. Before calling importActiveCalibration(), persist the complete document returned by the current exportActiveCalibration() to reliable storage, retaining its device identity, format, and checksum. Do not overwrite this pre-write backup with the document to be imported. The target document must belong to the current device. For a complete corrected-calibration write, use writeCorrectedCalibration() and all attachments required by that model. The presence of one field does not mean other required attachments can be omitted.
If a write, ACK, or readback fails, the device may already have changed partially. Do not assume the SDK rolled it back automatically, and do not blindly repeat the write. Check the device state before choosing a recovery procedure. Ordinary capture programs do not need to write calibration. The included calibration_inspect demonstrates the read workflow, while kfc_calibration_import demonstrates controlled import and backup requirements.
getCalibrationFrameDescriptors() queries the roles, indexes, dimensions, and raw pixel formats of named calibration frames without triggering a capture. captureCalibrationFrames() returns the actual data. Match frames by role and index; do not guess which are left or right images or phase-shift images from their array positions.
10. Exceptions, Logging, and Common Issues
SDK exceptions inherit from dexsense::Exception, which provides what() and errorCode(). Connection failure, device not found, capture failure, parameter errors, unsupported operations, and timeouts have their own exception types. ConnectionError includes a raw protocol code only when hasProtocolCode() is true. For batch parameter failures, you must also inspect ApplySettingsResult; try/catch alone is insufficient.
Filesystem and application code can also throw standard exceptions. The quick start’s handling example retains the SDK error code and attempts to disconnect after a failure:
catch (const dexsense::ConnectionError& error)
{
std::cerr << "Connection error: " << error.what() << " code=" << error.errorCode();
if (error.hasProtocolCode()) std::cerr << " protocol=" << error.protocolCode();
std::cerr << '\n';
exitCode = 1;
}
catch (const dexsense::Exception& error)
{
std::cerr << "SDK error: " << error.what() << " code=" << error.errorCode() << '\n';
exitCode = 1;
}
catch (const std::exception& error)
{
std::cerr << "Application error: " << error.what() << '\n';
exitCode = 2;
}
// 发生 SDK、文件或命令行异常后也执行断开;Camera 析构进一步释放资源。
try
{
camera.disconnect();
}
catch (const std::exception& error)
{
std::cerr << "Disconnect error: " << error.what() << '\n';
exitCode = 1;
}| Symptom | Check first |
|---|---|
find_package cannot find the package | Whether the platform root directory is correct, whether it contains lib/cmake/DexSense, and whether the required version matches |
| The program cannot find a DLL/SO | Runtime library path, processor architecture, chain of versioned links, and system runtime compatibility |
| Discovery returns nothing, but the IP is reachable | Broadcast reachability; if the model is known, connect directly using IP + model |
| Connection is refused or times out | Model, IP, network, firewall, another session occupying the device, device state, and complete error logs |
| A setting reports success, but the image does not change | Whether it was actually Applied, whether the wrong unit or a different capture path was used, whether critical values were read back, and whether only the cache was inspected |
| A single-channel image is returned instead of the expected color image | Actual hardware, engine/camera selection, brightness-image path, and pixel format; do not infer parameter semantics from the array shape |
| There is no point cloud or depth file | Whether the current device/mode provides the data; saveAll() does not write the depth array |
| Streaming capture stalls | Whether the callback takes too long, reenters camera methods, or blocks on the queue |
When troubleshooting, record the SDK VERSION, platform, camera model, firmware version, connection method, application parameters, complete error information, and a minimal reproduction command. Use Context::setLogLevel() to adjust the SDK log level.