Back in late December, I planted a Raspberry Pi camera at a cottage on Georgian Bay, in Northern Ontario, set to take a picture once every two minutes. I had been planning the shoot for a couple months prior to the deployment: There were two Raspberry Pi’s involved, in case one failed somewhere during the winter. One of the Pi’s was set to reboot once a week, just in case the software crashed but the Pi was still awake. I had also written some software for the time-lapse to ensure that pictures were only taken during the day time, and to try to maintain a balance of well-lit, consistent images over the course of each day.
In spite of all the planning, I had a sense that something would go horribly wrong, and, indeed, when we showed up to the cottage, the windows were completely frosted over. The cameras had to be placed inside, so we figured we would mainly see the back-side of an icy window when we retrieved the cameras. Or that the camera boards would literally freeze after about a week of sub-zero temperatures in the unheated cottage. Or that a raccoon would find it way in and gnaw through the shiny Lego cases. Or something else entirely unplanned for.
So it was a bit of a surprise when it turned out that the shoot went perfectly. We retrieved the cameras about a week ago, on May 7th, and found over 42,000 photos waiting for us on one of the cameras and somewhat fewer on the other. Both cameras had survived the winter just fine!
All told, I think the result was really cool! The video at the top is the ‘highlights’ reel, with all of the best days. It comes to 13 minutes at 18 frames per second. Turns out it was a fantastic winter for doing a time-lapse, with lots of snow storms and ice. There’s even the occasional bit of wildlife, if you watch closely. I’ll post the full 40-minute time-lapse on Youtube sometime next week.
DIY Image Manipulation Algorithms
This week, I’ve been writing bits of software to do post-processing of the images. The software I’d written for taking the images had done a pretty good job of taking pictures in a consistent way (slowly changing the shutter speed and ISO over the course of the day, in response to the available light), but there was still a significant amount of ‘flicker’ in the final images. Here’s a picture to give a sense of what I was correcting:
This picture contains all of the histograms of 346 images taken on one particular day. Each horizontal row is the (color) histogram for one image, and you can see how they change over the course of the day. The three vertical regions are the three different colour channels for each image: red, blue, and green. There are 768 pixels in each row; 256 for each color channel. The intensity of a pixel tells us how many pixels in the original image had a certain intensity, with dark blue meaning ‘none,’ and red meaning ‘lots.’
You can see that there’s a significant amount of ‘noise,’ jaggedness in the histograms, which is exactly the flickering that I was trying to reduce. To fix it, I separated each image into its three color components, and kept a running average over about twenty nearby images to get an ‘average’ histogram that we wanted to match. I then figured out the ‘expected value’ of the ‘averaged’ histogram and compared the expected value of the histogram of the image, and applied an additive shift to the image to get the expected values to line up.
Simultaneously, I used a similar kind of running average to find ways to give the images more contrast. Basically, we find a number L between 0 and 256 for each image such that 5% of the pixel intensities are less than L, and a similar upper bound H. (For my calculus students: hey, look, it’s an integral!) Then we ‘stretch’ the image so that the range between L and H fills the entire 0 to 256 range. This ensures that 5% of the pixels are black, and 5% are white. It’s more-or-less what ‘auto-levelling’ does in Gimp (or photoshop).
So one could do this kind of thing in Gimp, but a) I wanted the ‘filters’ to keep in mind the data in the other nearby images, and b) that there were 42,000 photos to apply the filters to. Neither of which are the standard photo editor use-cases. So I just used a bit of Python and a lot of processing time. The final result isn’t perfect, but it’s definitely an improvement.