Skip to content

Object Detection Benchmark

This script can be used to an Object Detector like Tensorflow-Lite (Coral), CoreML (Mac and Apple Silicon), and OpenVINO (Intel).

WARNING

The script below runs the benchmark on OpenVINO. Modify the script to run it on a different Object Detection backend(s).

Reference Times

Below are benchark times that can be expected from various servers.

Serveryolov8n 320EfficientDet-Lite0yolov9c 320yolov6n 320
Apple Silicon M1 Ultra5 secondsN/A7 seconds5 seconds
Intel 13500H10 secondsN/A43 seconds11 seconds
2 x Mini PCIe Coral19 seconds?25 secondsN/AN/A
Intel N10025 secondsN/A152 seconds27 seconds
1 x Mini PCIe Coral21 seconds50 seconds21 seconds20 seconds
1 x USB Coral20 seconds89 secondsModel Too Large20 seconds

TIP

Tensorflow-Lite uses the EfficientDet-Lite0 model by default, since yolov8n suffers from accuracy loss on int8 quantization. The yolov8n benchmark is listed for reference purposes.

Script

This script will run 250 iterations of 8 detections at a time (to test concurrency and batching). The test includes the time it takes to upload the input image to the object detection processor.

ts
const mo = await mediaManager.createMediaObjectFromUrl('https://user-images.githubusercontent.com/73924/230690188-7a25983a-0630-44e9-9e2d-b4ac150f1524.jpg');
const image = await mediaManager.convertMediaObject<Image & MediaObject>(mo, 'x-scrypted/x-scrypted-image');

const detectors = [
    // '@scrypted/coreml',
    '@scrypted/openvino',
    // '@scrypted/tensorflow-lite',
];

const detectIterations = 250;

for (const id of detectors) {
    const d: ObjectDetection = systemManager.getDeviceById<ObjectDetection>(id);
    console.log('starting', id);
    // await d.detectObjects(image);

    const model = await d.getDetectionModel();
    const bytes = await image.toBuffer({
        resize: {
            width: model.inputSize[0],
            height: model.inputSize[1],
        },
        format: model.inputFormat,
    });

    // cache a preconverted image to remove that from benchmark.
    const media: Image & MediaObject = await sdk.mediaManager.createMediaObject(bytes, 'x-scrypted/x-scrypted-image', {
        sourceId: image.sourceId,
        width: model.inputSize[0],
        height: model.inputSize[1],
        format: null,
        toBuffer: async (options: ImageOptions) => bytes,
        toImage: undefined,
        close: () => image.close(),
    })

    const start = Date.now();
    for (let i = 0; i < detectIterations; i++) {
        await Promise.all([
            d.detectObjects(media, { batch: 4 }),
            d.detectObjects(media),
            d.detectObjects(media, { batch: 4 }),
            d.detectObjects(media),
            d.detectObjects(media),
            d.detectObjects(media),
            d.detectObjects(media),
            d.detectObjects(media),
        ]);
    }
    const end = Date.now();
    const ms = end - start;
    console.log(id, 'done', ms, 'ms', (detectIterations * 2) / (ms / 1000), 'detections per second');
}