# Object Detection Explained: From Model to Real-World Deployment

> How object detection works, what it takes to train a model on your own camera footage, and the deployment choices that decide whether it works in the field.

- URL: https://weaveward.com/blog/object-detection-explained
- Published: 2024-08-27
- Updated: 2024-08-27
- Category: Computer Vision
- Tags: object detection, computer vision, machine learning, edge ai, cctv analytics
- Author: Weaveward Team

Object detection is the computer vision technique that finds specific things in an image or video frame and draws a box around each one, with a label and a confidence score: three motorcycles, one truck, two workers without helmets. It is the building block behind vehicle counting, shelf monitoring, safety checks and many other camera-based systems.

Getting a detection model to work on sample images is now fairly quick, thanks to open pre-trained models. Getting it to work every day, on your cameras, in heavy rain and at night, is the real project.

This guide explains how detection works in plain terms, walks through the path from a first model to a deployed system, and lists the places where real-world projects usually stumble. For a broader introduction to the field, start with [what computer vision is and how businesses use it](/blog/what-is-computer-vision).

## What object detection does, and what it does not

Object detection is one of several related vision tasks. Knowing the differences helps you ask vendors the right questions.

| Task | Question it answers | Output | Example |
|---|---|---|---|
| Image classification | What is in this picture? | One label for the whole image | Is this product photo defective or not? |
| Object detection | Which objects are where? | Boxes, labels and confidence scores | Count cars and motorcycles entering a parking area |
| Segmentation | Which exact pixels belong to each object? | Pixel masks | Measure the area of a crack or a spill |
| Tracking | Is this the same object as in the last frame? | IDs that persist across frames | Count each shopper once as they pass a door |

Detection on its own answers "what and where" in a single frame. Most business questions, such as "how many trucks were loaded today", need detection plus tracking plus business rules, for example counting only objects that cross a line in one direction. When a vendor demo shows colorful boxes on a video, remember that the boxes are the start of the answer, not the answer.

### How a detector works

A detector is a neural network trained on many labeled images. For each image it proposes boxes, assigns a class and a confidence score to each, and removes overlapping duplicates. Well-known families include one-stage detectors such as the YOLO series, which are fast enough for real-time video, and transformer-based models such as DETR.

Most projects do not train from scratch. They start from a model pre-trained on a large public dataset such as COCO, which covers 80 everyday object categories, and fine-tune it on their own images. That shortcut is why a first feasibility model can be trained quickly once you have labeled images from your own cameras.

## From idea to working model in six steps

1. **Define the decision and the classes.** "Detect helmets" is vague. "Alert the supervisor when a person in the scaffolding zone has no helmet for more than 10 seconds" is buildable. Decide the classes, such as person, helmet and safety vest, and agree on what counts as a mistake
2. **Collect images from the real cameras.** Use the actual camera positions, lenses and lighting, across times of day and weather. Images from the internet look different from your CCTV feed
3. **Label consistently.** Draw boxes in an annotation tool such as CVAT or Label Studio, following written rules for partly hidden objects, reflections and people far away. Inconsistent labels limit accuracy more than the choice of model does
4. **Fine-tune a pre-trained model.** Choose the model size based on the hardware it will run on, not on leaderboard rankings
5. **Evaluate on images the model has never seen,** ideally from different days and cameras than the training set
6. **Set thresholds based on the cost of each error type,** then test the full system, including the counting or alert rules, on recorded video

### How much data do you need?

There is no universal number. When fine-tuning a pre-trained model, a feasibility pilot can start with a modest labeled set from your own cameras, then grow by adding examples of the situations where the model fails. Production systems usually need far more images than the pilot, especially for rare classes, small objects and classes that look alike. The better question is coverage: do your images include night, rain, glare, crowds and every camera angle you plan to use?

### Reading the metrics

- **Precision:** of all the boxes the model drew, how many were correct
- **Recall:** of all the real objects in the images, how many the model found
- **IoU (intersection over union):** how much a predicted box overlaps the true box. A detection usually counts as correct above a threshold such as 0.5
- **mAP (mean average precision):** a single score across classes and thresholds, useful for comparing models but hard to connect to business impact

Translate metrics into business terms before you sign off. In a safety system, a missed violation (low recall) may be worse than a false alarm. In a system that bills customers per detected item, a false detection (low precision) costs money directly. The confidence threshold you choose moves the balance between the two, so set it with the people who live with the consequences.

## Choosing where the model runs

Where inference happens shapes cost, privacy and reliability as much as the model does.

| | Edge device at the camera | On-site server | Cloud |
|---|---|---|---|
| Latency | Lowest | Low | Depends on upload speed |
| Bandwidth needed | Minimal; sends results only | Local network only | Continuous video upload |
| Video leaves the site | No | No | Yes |
| Typical hardware | Small accelerators such as NVIDIA Jetson | GPU server | Rented GPU instances |
| Maintenance | Many devices to update | One machine to maintain | Provider manages hardware |
| Good for | Many sites with few cameras each, weak links | Many cameras in one location | Photo uploads, batch analysis |

Before deployment, models are usually converted and optimized for the target hardware, for example exported to ONNX and run with TensorRT on NVIDIA devices or OpenVINO on Intel processors. Smaller models trade some accuracy for speed, so test on the actual device with your actual camera streams. Our [cloud vs on-premise guide](/blog/cloud-vs-on-premise) covers the wider infrastructure trade-offs, including internet reliability at remote sites.

## What breaks in the real world

### Lighting and weather

Tropical downpours, glare on wet roads, headlights at night and the switch to infrared mode on many CCTV cameras all change how objects look. Include these conditions in both training and testing, or the model will meet them for the first time in production.

### Camera placement

Mounted too high, too far or with too wide a lens, small objects shrink to a few pixels. Moving a camera after deployment can quietly break accuracy, so fix camera positions and recheck after any maintenance.

### Crowding and occlusion

Motorcycles packed at a traffic light, shoppers behind shelves and stacked boxes are harder to separate and count. Tracking across frames helps, but expect lower accuracy in dense scenes and test them specifically.

### Change over time

New packaging, new uniforms or a repainted loading bay can lower accuracy without anyone noticing. A practical safeguard is a monitoring loop: save a sample of low-confidence detections each week, have someone review them, and add the hard cases to the next training round. Our article on [retail CCTV analytics](/blog/retail-cctv-analytics) shows how this applies to in-store cameras.

### Licensing

Some popular detection frameworks are released under the AGPL-3.0 license, which can oblige you to publish your own source code when you distribute the software or offer it over a network, unless you buy a commercial license. Check the license of every model and library before building a product on it.

## Privacy and responsible use

Cameras capture people, and images of identifiable people are personal data under Indonesia's Personal Data Protection Law, Law No. 27 of 2022 (UU PDP), whose two-year transition period ends in October 2024. Biometric data, such as facial images processed to identify someone, is classed as specific personal data with stricter requirements.

Good practice keeps both risk and cost down:

- Detect only what you need. Counting helmets does not require knowing who wears them
- Process video at the edge where possible, and store results rather than footage
- Set retention periods for recordings and snapshots, and delete on schedule
- Put up clear notices where cameras analyze people
- Limit who can view raw video, and log that access

## How Weaveward can build this for you

We would start with a feasibility check on your own footage: recordings from the actual cameras, a labeled sample and a baseline model, so you see real precision and recall under your conditions before committing to a full build. If the numbers hold up, we would deliver a pilot at one site with detection, tracking and the business rules that turn boxes into counts or alerts, running on edge or on-site hardware sized for your cameras. The pilot would include a review screen where your team confirms or corrects detections, and that feedback becomes the training data that keeps accuracy up after rollout.

See how we approach [computer vision projects](/services/computer-vision), or [talk to our team](/contact) about the cameras and questions you have in mind.