Finding the max GOP length of a video file

I’ve had an interesting request today: Analyze a random video file and find out if the maximum GOP length is above some threshold. This took a bit of experimenting and thinking, so I want to preserve this method for anyone who might need to do the same.

What’s a GOP?

A very simplified explanation: Modern video compression generally follow the schemes of saving one full video frame (think of it as a picture, called a key-frame or “I-frame”) and after that for a number of frames just “movement” information (called “B-frame” or “P-frame” - their difference is irrelevant in this context).

So, instead of encoding 25 full pictures, we can just encode one full picture, and in the next 24 pictures we just say “the pixels depicting a car have moved from their original location 10 pixels to the right”. This interval is called a “Group of Pictures”, short GOP. This length, i.e. the amount of frames in one GOP has several implications for further processing and playback:

Why do we need to calculate this?

Okay, so this seems like a pretty standard concept so far. Now comes the tricky part: Though most of the time your GOP length will be fixed, there are some special cases that have varying GOP length. Many security Cameras for example will try to produce very large GOPs because this will increase the compression ratio a lot. If you think about it: 2 hours of an empty parking lot will not have much movement, and thus very little changing information. Increasing the GOP for this is desirable. But as soon as a car enters the scene, the amount of information increases and you will want to produce shorter GOPs.

There are some reasons why you want to find out the maximum GOP length - mainly the afore-mentioned cutting precision and streaming start time. You may want to define a maximum length, and if necessary, re-encode the video file to adhere to the maximum.

How do we do this?

My toolchain of choice, ffmpeg, sadly will not tell you the GOP length because it might vary. Mediainfo won’t help you here either. So we will need to do some manual labor to find out the length. My (crude) method can be summarized like this:

  1. let ffprobe analyze all frames in a file, and tell us wether a frame is a key-frame or not.
  2. count the length of all not-key-frame groupings (separated by a key-frame)
  3. sort these lengths
  4. the maximum length is the last value

ffprobe

This is the heaviest step computationally: ffprobe will have to walk through the entire video stream and decide wether a frame is a key-frame or not. This can be done with this command:

$ ffprobe -v quiet -show_frames -select_streams v:0 <file.mp4> | grep key_frame
key_frame=1
key_frame=0
key_frame=0
key_frame=0
[…]

That’s a bit crude though, we have to reduce the output of ffprobe with grep. Let’s tell ffprobe to only give us the relevant info to begin with:

$ ffprobe -v quiet -select_streams v:0 -show_entries frame=key_frame -of copact=nk=1:p=0 <file.mp4>
1
0
0
0
0
[…]

The 1 indicates a key-frame, 0 indicates any other frame.

Count the GOP lengths

We now have a stream of ones and zeros. We are only interested in the amount of zeros in each grouping, so, count the zeros between each one:

$ uniq --count --repeated
    11 0
     9 0
    24 0
     2 0
    11 0
[…]

Uniq removes duplicates from our stream, the --count option will tell us how many duplicates were found. The --repeated option will print only duplicates and omit single occurences - this essentially filters out the ones.

Sorting the lengths

We now want to find the maximum length, so we need to sort the output of uniq.

$ sort --ignore-leading-blanks --numeric-sort
     2 0
     9 0
    11 0
    11 0
    24 0
[…]

uniq will leave us with a huge amount of leading whitespace, which we really don’t need, so ignore that via --ignore-leading-blanks. Using --numeric-sort we can sort by the numeric value instead of sorting via the string characters, which is what we want.

Grabbing the maximum length

The last step is easy: We only need the first value of the last line:

$ tail --lines 1 | awk '{print $1}'
24

TL;DR - what’s the command?

Let’s pack all of this into one nice command with abbreviated parameters:

$ ffprobe -v quiet -select_streams v:0 -show_entries frame=key_frame -of copact=nk=1:p=0 <file.mp4> | uniq -cd | sort -bn | tail -n 1 | awk ‘{print $1}’
24

Conclusion

This is not a perfect solution! It will take a lot of time to calculate the maximum GOP length. Also, this is 0-indexed because the key-frame will not be counted towards the length. :) So in this case, the actual GOP length is 25 instead of 24 frames. But, that is a problem for someone else. You could just as well use awk to add 1 to it. Or maybe compress all those uniq, sort, tail commands into awk. Certainly possible. Do I have the nerve to do it? No. It works as is.

One option to drastically reduce the runtime of this command: Only look at the first minute of the video file. If you only want a “guesstimate” you can use this, because most files will have a somewhat fixed GOP length. Add the parameter -read_intervals %+60 to the ffprobe section.