The faster part is not always the part that helps
What you will learn
- Why the obvious part to speed up is usually not the part that is slow
- How to split a job into stages and find out which one is actually costing you
- How to test a hardware shortcut honestly before you commit an evening to it
A machine watching video all day was running hot and working its main processor flat out. The obvious answer was sitting right there. The chip has a built in graphics unit, doing nothing at all. Move the hard work onto that. Graphics units are for hard work.
It did not work, and not in a way I could grind through with better settings. The program I was using to recognise things in the video had no route onto that particular graphics unit at all. The maker's own toolkit was the supported route, and that toolkit expects a different family of hardware. There was one general purpose route that did exist, and when I measured it, it was slower than the processor I was trying to relieve.
So I had spent an evening on a dead end and the machine was still hot. Then I did the thing I should have done first. I measured which part of the job was actually expensive.
A kitchen running late is not always slow at cooking. Sometimes everything is waiting on one person at the sink. Buying a better oven does not clear the washing up.
Watching video is at least two jobs wearing one name
First the squashed video has to be turned back into pictures. Then something has to look at the pictures and decide what is in them. I had assumed the looking was the expensive half, because the looking is the clever half, and clever feels costly.
It was the other one. Turning squashed video back into pictures, continuously, for several cameras at once, was most of the heat. And that job did have a hardware route on the graphics unit I already owned, because unsquashing video is a dedicated piece of circuitry that exists for exactly this. Switching it on moved the load off the main processor and the temperature came down.
The recognising stayed on the main processor. It is still there. It was never the problem, so making it faster would have bought me nothing, and I would have been able to write a confident guide about the hardware that fixed everything while being wrong about why.
The lesson is not "turn on hardware video decoding". It is that the obvious knob is often the wrong knob, and the only way to find out is to measure which stage is hot before you spend money or an evening.
Buying a better oven does not clear the washing up.
How to tell the cost of unsquashing video apart from the cost of recognising things in it, and how to test a hardware route honestly. The measuring approach works for any job with stages where you are about to speed up the one that feels expensive.
1. Separate the stages before you measure anything
What this does: writes the job out as distinct stages with names.
Why it is first: you cannot blame a stage you have not named.
Watch the third one: a conversion sitting between unsquashing and recognising can cost more than either, and it never appears in the diagram you drew in your head.
# pipeline.md
# A. receive the video (network, cheap)
# B. unsquash it to pictures (main processor, or dedicated circuitry)
# C. resize or reformat (often forgotten, often expensive)
# D. recognise things in it (the clever bit)
# E. act on the result (cheap)2. Measure the starting point properly, not with a glance
What this does: samples how hard the machine is working, repeatedly, over a window of time, while the real job is running.
Why repeatedly: a single reading taken when you happened to look is not a measurement.
Worth knowing: the effort figure in a simple process list is an average over that program's entire life, so something that was busy an hour ago and is idle now still reads as busy. Sample several times and use the later samples.
# sampled, not a lifetime average
top -b -n 5 -d 2 | grep <your-program>
# temperature over the same window
sensors
# write the numbers down, because you will need to compare later3. Measure the unsquashing on its own, with nothing clever attached
What this does: unsquashes the same video and throws the pictures straight away, so nothing else is running.
What the answer tells you: if this alone accounts for most of your starting figure, your cost is the unsquashing and the clever part is not your problem. This is the test that answered my question, and it takes two minutes.
# unsquash only, discard the output, no hardware help
ffmpeg -i <your-video-source> -f null -
# watch the effort figure while that runs4. Measure the clever part on its own, using pictures you already have
What this does: feeds the recogniser still pictures from disk, so no unsquashing happens at all, and times it.
What you now have: two numbers that roughly add up to your starting figure, and the bigger one tells you where to spend your effort. Everything before this step was guessing.
# time a batch of recognitions on pictures already saved to disk
# fast here and slow in step 3 -> stop improving the recogniser
# slow here -> now you have a real reason to look5. Check a hardware route exists for your hardware before trying to use it
What this does: asks the system what it actually supports, rather than assuming that having a graphics unit means having a route for your job.
Why these are different questions: they often have different answers, and this is exactly where an afternoon disappears.
Read the list against the video format you really receive. Support for one format says nothing about another.
# Linux: is there a video acceleration device, and what can it do?
ls /dev/dri/
vainfo
# and does your tool have that route built in?
ffmpeg -hwaccels6. Test the hardware route and compare the same numbers
What this does: repeats step three using the hardware route, over the same window, measuring the same things.
Why identical conditions: a change you cannot compare is a change you are taking on faith.
And check one more thing: whether pictures were quietly thrown away. A hardware route that lowers the effort figure by dropping frames has not helped you, it has changed what you are measuring. Count the output, not just the load.
# unsquash only, hardware route
ffmpeg -hwaccel <method> -i <your-video-source> -f null -
# compare against step 3: effort, temperature, and frames produced7. Accept the answer even when it is boring
What this does: writes down the result, including the dull version.
Mine was dull: leave the recogniser where it is, move the unsquashing, done. No new hardware, no exotic toolkit, one setting.
Why write it down anyway: the satisfying answer and the correct answer are frequently different, and preferring the satisfying one is how people end up with hardware that speeds up nothing.
# results.md
# unsquash, software : ....... effort, ....... temperature
# unsquash, hardware : ....... effort, ....... temperature
# recogniser alone : ....... per picture
# decision: moved the unsquashing only. recogniser left alone on purpose.Hardware unsquashing is not free. It moves the load onto circuitry with limits of its own, and enough cameras will use all of that up too. You have moved the ceiling, not removed it, so write down where the new one is.
Related reading: measure before you delete is the same instinct pointed at disk space instead of heat.
This page in the original Kyber Cypher voice: The accelerator you own is not always the one that helps