Rebuilding bad frames using OFlow

THE PROBLEM

This is a tool I recently built for helping fix footage that was shot on an octocopter. However, it could equally be used for any situation where you have footage that has good frames interspersed with bad frames i.e. a CG render that broke on the farm.
In this case the problem was that the octocopter suffered from intermittent vibrations, causing the footage to become blurred every few frames. I found myself going through and fixing the blurry frames one at a time, which became hugely time consuming and called for a more automated method.

THE SOLUTION

OFlow is a great tool at creating new frames based on morphing between two existing ones. This would normally be done when retiming – i.e. if slowing footage down to 50% speed, then a 100 frame clip would become 200 frames long, and the alternate missing frames would be rebuilt by OFlow.

This is easy to calculate if applying a uniform retime over an entire clip, but I needed a way of arbitrarily telling OFlow which frames to keep and which to rebuild without actually retiming anything.

The solution was to first establish which frames were good. In my gizmo I created the “Frame” input, and then went through and created a keyframe on every frame that was to be kept.

Then I added a Python Button that runs a little ‘Rebuild’ script. Basically, a TimeWarp node is fed the frame numbers of the good frames, and the non-keyframed frames (i.e. the blurry ones) are skipped. So in this case, the first keyframe occurs on frame 100 and is copied into frame 1 of the TimeWarp. The eighth keyframe occurs on frame 115 and is copied into frame 8 of the TimeWarp, and so on.

Next, an OFlow node inverts this process – retiming the TimeWarp’s output back to normal speed, and using optical flow to rebuild the skipped frames.

I dropped a couple more buttons on the gizmo to help speed up the process as much as possible, but really the most time consuming part is manually going through and keyframing your good frames – after that OFlow does all the heavy lifting.

As you can see, the end result is very impressive (even with some minor optical flow artifacts that need to be fixed)

THE NUKE NODE

Here is the GitHub link for the node – just select the text, copy it then, then paste it into Nuke.

CREDIT

The footage used in the demo comes from stockfootageforfree.com