Cameras and AI: How people, zones and routes count

Video analytics has a huge advantage over all other technologies in this series: there are already cameras on the site. From this comes the most tempting offer on the market — "we will connect to your cameras and give analytics", without installation, without cable, without tags.

Sometimes that's exactly how it works. Sometimes — no, and the reasons for the refusal lie not in algorithms, but in physics: in a pair above the font, in the glare on the water, in the contrast of panoramic glazing and in the fact that you can not put a camera in the locker room.

This article is about how modern video analytics works inside, what numbers are behind the word "neural network will count", and where is the boundary between tasks that the camera solves perfectly and tasks that are sold to it, but it does not solve them.

Conveyor: four stages that make up any system

Any system of video analytics of people is a sequence of four steps, and each has its own limitations, its own cost and its own accuracy.

1. Detection. There are people in every frame. The result is frames around silhouettes.

2. Tracking. Frames are linked between frames: It's the same person walking from left to right. The result is a trajectory within the same camera.

3. Re-identification. A person who leaves the field of view of one camera and appears in another is identified as the same. The result is a through route through the item.

4. Metrics. Trajectories turn into numbers: so much has passed, so much has been delayed, the zone is loaded like this.

It is critically important that complexity and fragility grow with each step not linearly, but in leaps and bounds. The first step is a solved problem. The second one works well at a reasonable density. The third is the weakest point of the entire industry, and it is sold as the main value. The fourth is not technical, but managerial: this is where projects most often die, and a separate article in the series is devoted to this.

Detection: which models and what do their numbers mean

The YOLO family remains the industry standard for real-time detection. In January 2026, the YOLO26 version was released, and reviews of the following appear. The main competitor is transformer detectors, primarily RF-DETR.

Open comparisons on the COCO reference set give the following picture:

Model	Accuracy (mAP)	Time per frame	Parameters	License
RF-DETR Medium	54,7	4.4 ms	more than 30 million	Apache 2.0
YOLO26m	53,1	4.7 ms	—	AGPL-3.0
YOLO26n	lower by about 7.5 mAP than RF-DETR Nano	1.7 ms	about 2.4 million	AGPL-3.0

Three practical conclusions follow from this table, and none of them is about accuracy.

First: the license may cost more than the model. YOLO26 is distributed under AGPL-3.0. This means that for commercial use without a separate commercial license, there is a requirement to open the source code of derivative works. For the contractor who puts the system up to you, this is a matter of license purity, and for you, it's a question of what will happen to your system if you haven't bought a license. Direct question to the supplier: what model is used, under what license, is there a commercial agreement. RF-DETR in the main versions is distributed under Apache 2.0, which is simpler; but the largest version has a different license.

Second: transformers require a graphics processor. RF-DETR with more than thirty million parameters does not have a practical execution path on the processor; YOLO26 in the younger version has about two and a half million parameters and runs on inexpensive hardware. This does not determine the accuracy, but the cost of the server side and the number of cameras per box.

The third: Map on COCO is not your metric. COCO is a set of photos from the Internet with eighty classes of objects. Your task is one class (person), from above, in a pair, in a swimsuit, with a background light. mAP numbers are useful for comparing models with each other and are useless as a prediction of accuracy on your object. The only honest forecast is the measurement on your video.

Separate classes of tasks. Detection by frames is not the only way. When there are a lot of people in the frame and they overlap each other, detection loses its meaning, and dense crowd counting models are used: they estimate the number of people on the density map without highlighting each one. For falls and to determine "whether a person is lying down", a pose assessment is used — the model finds key points of the skeleton. This is important for an article about critical events.

Tracking: ByteTrack, BoT-SORT, OC-SORT and what their metrics mean

The tracker links detections between frames. The main open solutions are ByteTrack, OC-SORT, BoT-SORT and their variants with an appearance model.

The most useful thing here is not advertising statements, but a fresh comparative analysis on closed premises. In the work of 2026 IndoorCrowd, combinations of detectors and trackers have been tested, and the results are as follows:

• RT-DETR-L with OC-SORT — 56.2 MOTA, the best overall result;
• RT-DETR-L with BoT-SORT — 55.5 MOTA and 71.6 IDF1;
• BoT-SORT consistently outperforms ByteTrack with any detector;
• in terms of speed: RT-DETR-L is about 32 frames per second versus about 120 for YOLOv8n.

What these metrics mean in practice. MOTA answers the question "how many people the system did not notice in total, invented and confused." IDF1 answers the question "how stable is the system keeping the same ID on one person".

For a management report, this translates as:

• low MOTA — the score is lying, you can't trust the download;
• a low IDF1 score may be correct, but the time of stay and routes lie: one person is counted as three, because the track broke three times.

And here's the key: 56 MOTA is not "the system works at 56%," but it's not 95% either. This is an honest level of modern video analytics indoors with people, and it means that the data is suitable for comparing periods and identifying trends, but not for accounting with the accuracy of the cash register. Any supplier that promises 99% on routes in a crowded room promises something that is not in open research.

A separate caveat about speed: the difference of 32 versus 120 frames per second is a difference of four times in the number of cameras on one server, that is, in the cost of hardware. Therefore, the choice of a model is not a choice of "who is more accurate", but a choice of a point on the curve "accuracy versus server price".

Re-identification: the most fragile promise

To collect a guest's route around the object, you need to recognize a person on different cameras. This is a separate model that builds an "appearance fingerprint" and compares the prints.

The task is fundamentally more difficult than tracking: the angle is different, the lighting is different, the person turned around, time passed. And if he also changed his clothes, the task changes qualitatively, and in the scientific literature this is highlighted in a separate direction.

Figures from this direction should be known to everyone who is promised a through route. In the work of February 2026, claiming to have reached the current level on the LTCC reference set, the result is 58.8% according to the Rank-1 metric and 25.4% according to mAP. Rank-1 is the proportion of cases when the right person was first in the list of similar ones. That is, the best result of a scientific article of 2026: in four cases out of ten, the first candidate is incorrect.

Now let's apply this to our objects.

The thermal spa complex and the water park are the worst possible case. Re-identification mainly relies on clothes: color, style, bag, shoes. There are no clothes in the water park. All the guests are in swimsuits, wet, with a similar color scheme of leather and fabric, some with towels, some without. The model loses the main feature. This is not a question of vendor quality: it is the physical absence of information on which the method works.

A fitness club is an average case. There are clothes, but monotonous: black leggings and gray T-shirts. The visibility is low.

A shopping mall and a museum are the best case. A variety of outerwear, bags, strollers. This is where end-to-end tracking works best, and that's where the beautiful demonstrations come from.

Practical conclusion. If you need a guest's route through the thermal spa complex, you need to collect it not by video, but by bracelet: the marks on the reading points give accurate, indisputable links of the route. Video in this task is an expensive way to get an unreliable result. For the shopping center, the picture is reversed: there is no bracelet, and video is the only way, adjusted for real accuracy.

Iron: what counts and where

Three architectural options, and the choice between them is a matter of money and delay.

On camera. Some modern cameras consider detection and simple analytics to be a built-in processor. Plus: you don't need a server and a channel, analytics lives when the network is broken. Minus: the models are limited by the camera's capabilities, it is difficult to change them.

On the edge, in a separate box. A common variant of 2026. Typical solutions are Nvidia Jetson Orin Nano and Hailo accelerators. According to open descriptions, Hailo-8 gives about 26 TOPS with a power consumption of about 2.5 Watts — that is, several high—definition streams on a very modest power and heat budget; the payback is a specialized compiler and a narrower set of supported models. Jetson is chosen when model flexibility is more important than price.

On the server or in the cloud. Heavy models: re-identification, face recognition (with legal reservations), archive search, correlation between objects. A typical scheme of 2026 is detection on the edge, heavy on the server.

To this is added what is remembered at the end of the project: a video archive. Analytics requires you to keep records on which you can disassemble the incident and check the operation of the system. This is disk storage, archive depth calculation and a separate budget line — along with a server, uninterruptible power supply and PoE switches.

Multimodal models: what is really useful from them

The most discussed topic of 2026 is models that "understand" the video and answer questions about it: Qwen3—VL (under the Apache 2.0 license, that is, suitable for commercial use), InternVL3 and analogs. For video surveillance, there is even a specialized assessment kit — SurveillanceVQA-589K.

What you need to know before it appears in the offer.

Resources. A model with 8 billion parameters in full accuracy requires about 17 GB of memory only for weights, and under load memory problems are manifested not by an error, but by timeouts. This is not what is put in the flap at the entrance.

The delay. The generative model responds in seconds. This is not applicable for online detection.

Where it really works. Archive search and incident analysis: "find fragments where a person climbed over the fence," "show when there was a queue of more than five people at the counter." This saves operator hours and does not require real time. This is an honest application of multimodal models at the facility in 2026; everything else is still demonstrations.

Where Video Analytics breaks Down in our Environments

The most practical section of the article. All these problems are physical, they are not solved by changing the vendor.

Steam. In the thermal zone and at the font, steam diffuses light. The camera sees milk, the detector does not find people. Detours: take the camera out of the steam zone, direct it along the flow, not across, choose a shooting point above the steam layer, in critical places switch to another physics — radar or thermal sensor.

Condensation on the lens. The classic reason is "the system worked yesterday, but not today." Detours: sealed heated casings, removal of the camera to a dry area, wiping regulations.

Glare and refraction on the water. The reflected sun gives illumination, and refraction distorts the shape of the body under water. This is a key problem for water safety tasks, and specialized systems are used there — see the article about critical events. Detours: polarizing filters, angle selection, underwater cameras.

Kontrazhur. Panoramic glazing of the recreation area is an almost guaranteed detection failure against the background of the window. Detours: extended dynamic range of the camera, change of angle, additional illumination.

Density. At the bar during rush hour, people overlap each other, the track breaks, IDF1 falls. Detours: camera from top to bottom, not at an angle; transition to the model of counting a dense crowd; for the input group — a separate counter that does not depend on general video analytics.

Children. The child in the frame is smaller, often partially hidden by an adult and moves unpredictably. Detectors trained on adult pedestrians lose children. Direct question to the supplier: how the work on children has been tested, and ask them to show it on your video.

Darkness and uniformity. Hammam, areas with dimmed lights, semi-darkness in recreation areas. Infrared illumination partially solves the problem: in a pair it gives illumination.

Swimsuits. It has already been discussed: re-identification of the main feature is deprived.

The conclusion is not that the video does not work well. The conclusion is that video analytics at an object with wet zones has a map of applicability, and it needs to be drawn before purchase: lobby, reception, entrance group, corridors, recreation areas, cafes — works well; thermal zone, swimming pool, showers, changing rooms — from "bad" to "impossible".

Russian platforms and one legal mine

Foreign platforms have mostly left the Russian market or are not officially available; according to market reviews, the existing Russian platforms are Macroscop, TRASSIR (DSSL), Axxon (ITV), SecurOS (ISS), NtechLab (FindFace), VisionLabs (LUNA), VOCORD, Tevian.

Functionally, they have what they need for flow analytics. For example, Macroscop offers counting visitors with reports by hour, day and month and uploading, a heat map of traffic intensity and tracking; the manufacturer declares inclusion in the register of Russian programs (the entry number should be checked in the register of the Ministry of Digital Services).

And now an important warning that is worth reading before signing the contract.

The module "counting unique visitors" in video analytics is usually based on facial recognition. The logic is clear: in order not to count one person twice, you need to recognize him. In the Macroscop documentation, for example, it is explicitly stated that the corresponding module works on the basis of face recognition and allows you to exclude faces from the database from counting — for example, employees.

From the point of view of Russian legislation, this translates the task of "counting visitors" into a completely different category: the processing of biometric personal data, with the requirements of 152-FZ and 572-FZ, the issue of a single biometric system, separate consent and strict responsibility. That is, a seemingly harmless item in the specification ("counting unique visitors") may turn out to be the most binding part of the entire project.

What to do about it. Separate tasks:

• we need a stream and a download — impersonal counting of silhouettes or counters at the entrances, no biometrics;
• it is unique visitors and repeat visits that are needed — this is the task of the loyalty program and the bracelet or card, where the guest himself presents the identifier, and not the task of the camera;
• face recognition is only done consciously, with a lawyer, with a well—developed basis and understanding why the business needs it so much to accept the risk.

A detailed analysis is in the legal article of the series.

How to accept video analytics

Acceptance here is simpler than that of radio systems, and it is almost never done.

The standard is manual counting by recording. An hour of video is taken at rush hour and an hour at a quiet time, two people independently count people manually by recording, the results are checked. This is the truth with which the system is compared.

What is fixed:

Indicator	How to count	A reasonable benchmark
The accuracy of counting at the input	Deviation from manual counting	Within 2-5 % during rush hour
The accuracy of counting in the zone	The same, for frames with an interval	Worse than the input; fixed as is
Track stability	Number of ruptures per person	The smaller, the more reliable the stay time
False objects	Reflections, mannequins, shadows	Enumerate and eliminate by setting up zones
Behavior in problematic conditions	Steam, counterattack, crowd	A separate protocol for each zone
Working on children	The proportion of missed	It is checked on a real video of the object

The acceptance condition is formulated by zones, not by object. "In the lobby and at the entrance group, the counting deviation is no more than 5% during rush hour; in the thermal zone, the system is used only to record presence, the accuracy of counting is not normalized." This is an honest wording that protects both you and the contractor.

Checklist for talking to the supplier

1. What models are used for detection, tracking and re-identification?
2. What licenses do these models have and do you have a commercial agreement?
3. Where is the calculation performed: on the camera, on the edge, on the server, in the cloud? What if the connection is broken?
4. How many threads does the proposed hardware handle and with what margin?
5. What is the accuracy of the calculation confirmed by manual measurement at an object of our type?
6. How does the system behave in pairs, with a counter-image, on the glare from the water?
7. Has the work been tested on children and is it possible to show the result on our video?
8. Does the system build an end-to-end route and what is the basis for re-identification?
9. Does any module use face recognition - directly and unambiguously?
10. What data is stored: videos, events, impersonal numbers? The timing and scope of the archive?
11. How do we know if the camera or analytics has stopped working?
12. What reports are uploaded and is there an API for our accounting system?
13. Are you ready to conduct a pilot on our cameras and record the result in the act?

Sources

Sources

• IndoorCrowd: Comparison of indoor detectors and trackers, 2026
• BoT-SORT, ByteTrack, Deep OC-SORT trackers and their published results
• Overview of deep multi—object tracking - the state of the direction
• Comparison of RF-DETR and YOLO26 — numbers and licenses (material of the vendor of one of the models)
• Re—identification when changing clothes, 2026 - the current level on the LTCC set
• SurveillanceVQA-589K — a set for evaluating multimodal models on video surveillance
• Macroscop: counting unique visitors — a module based on face recognition (manufacturer's documentation)
• Overview of the video analytics market in Russia