A cheap and automatic way to catch lightning

Taking pictures of lightning is hard. Usually you would set up the camera on a tripod, take many shots with an exposure time of several seconds, and then sift through all the pictures later. You might even need a remote to avoid shaking, and if the camera doesn’t have a suitable interval mode you might have to stand behind your camera all the time and press the shutter button every couple of seconds. Even with interval mode, you lose the time between two shutter releases. Clouds moving too fast might interfere with the long exposure time.

One of the most beautiful storms I have seen in years was passing over our building a while ago, but the time between two close-by strokes was up to 20 seconds, it always happened between two shutter releases (of course!), and the clouds were moving quite fast (of course!). So I didn’t really feel like standing on the balcony for the next hour, pressing a button and still missing some of the best shots. There had to be a more comfortable solution.

The beautiful thing about my Nikon D750 is that it still performs extremely well in low-light conditions, so I thought: What if don’t take a long exposure picture, just switch to video mode instead and extract the “good” frames later? Onviously I had to consider the following things:

• I would end up with 216,000 frames per hour at 60 fps.
• The Nikon D750 only does FullHD video, so the frames would have a resolution of just 2 megapixels (1920×1080).
• One frame takes at most 1/60 second at 60 fps, so the moving clouds wouldn’t be such a problem, but a single stroke would have to be reconstructed from multiple consecutive frames.
• Videos are encoded, so I would lose the 14 bit color information I would have gotten in RAW mode, which might create problems in post-processing.

Manually sifting through several hundred thousand frames was a no-go, so the idea was to let the computer do it. There’s a pretty big difference between a frame with a lightning stroke on it and one without, while most frames withouth a lightning stroke should be very similar to each other.. Detecting a stroke should not be that hard. So I just tried.

Getting the frames

I managed to take about 45 minutes of footage until it started to rain and I had to take the camera inside. I converted the videos to individual images using ffmpeg:

ffmpeg -f image2 DSC_7338/img%4d.ppm -i DSC_7338.MOV

Yes, PPM has its problems and PNG would have been better, but PNGs are compressed, converting and storing 160,000 frames at 6 megabytes per file was already enough of a challenge without compression, and there’s plenty to improve everywhere.

Filtering the frames

I came across ImageMagick’s compare utility, which compares two pictures and calculates a number of metrics. The biggest difference between frames with lightning strokes on them and ones without seemed to be on the red channel, so after some testing I decided that the MAE metric for the red channel could be used to split the frames into two groups.

I came up with the following script:

#!/bin/bash

REFERENCE=${1}
FRAME=${2}

METRIC=$(compare -metric mae -channel red "${REFERENCE}" "${FRAME}" /dev/null 2>&1 | cut -d' ' -f 1 | cut -d'.' -f 1)

if [ ${METRIC} -gt 6000 ]; then
    echo "${FRAME} has ${METRIC}, matches"
    mkdir -p matches
    cp ${FRAME} matches/
else
    echo "${FRAME} has ${METRIC}, doesn't match"
fi

It calculates the difference between the two pictures ($REFERENCE and $FRAME) specified as command line arguments and checks it against a pre-defined threshold. If the metric is above the threshold, $FRAME is copied to a sub-folder called matches.

Now all I had to do was choose a reference picture and let the computer do the work, in parallel:

ls | xargs -n 1 -P 12 ../compare.sh img0001.ppm

That took a while, compare is not the fastest tool and scheduling 200,000 bash scripts was also not the best idea. But it did the job. I then sifted through the matches, used GIMP to blend consecutive frames together and used darktable to do some very minor corrections. This resulted in the quite beautiful pictures on this page.