Production Company: Paranoid US
Director: Nieto

My work on this spot encompassed VFX Supervision on set, offline editorial, and finishing artist/lead compositor duties.  For VFX Supervision, I worked closely with the director and AD to identify what elements would be required to create the finished product. I also created 3D assets on set using modo and Element 3D to provide on set previs of certain sequences I captured HDR light probes of the set, lens distortion charts for all focal lengths used in the spot and orthographic reference photos of all props.

The spot was shot as a series of motion control passes that needed to be lined up so that the finished spot appeared to be a single seamless shot. Adding a wrinkle to this challenge was that the approved ending of the spot required a CG oreo to drop out of frame and start an MC Escher-like domino cascade down the steps and risers of a fully CG staircase, ultimately revealing the end card, which itself was shot practically.

In the offline editorial stage, the primary concern was timing. The music had yet to be finalized and one of the main conceits of the campaign was a device called “Oreoke,” where words on screen would appear at the same time you’d hear that word sung in the lyrics of the song. Many of these were captured practically on set, with a small handful expected to be added as tracked 3D elements in the finish. After lining up all motion control plates and roughing in some effects to help populate the space with more oreos, the task then became dialing in timewarps that would line up the motion control move up with certain parts of the song. As the song selection changed, so would the retiming. Some of the retiming became so extreme that it became necessary to consider cutting certain parts of the shot out, which was accomplished using some of the 3D assets created on set as wiping devices that could appear to be in the extreme foreground and wipe us from one section of the shot to another without feeling as though there had been a cut.

In the end, the solution that approved turned out to be relatively simple from an offline standpoint, but complex from a finishing standpoint. All of the oreoke moments captured practically on set were to be removed and new ones created as 3D assets that could be lined up to whatever song was ultimately selected as the approved music.

During the finish, I worked closely with Maya generalist Del Depierro, who refined my 3D assets and those provided by the agency as approved Oreoke designs, laid them out in the scene using a tracked camera provided by TraceFX, and rendered them using Mental Ray. I performed extensive compositing work in Nuke to line up motion control plates, rig remove some of the seams in the set used to puppeteer the oreos and generate clean plates throughout the spot so that the new Oreoke elements could be inserted wherever the agency chose.

Some of the most challenging work was cleanplating and animating the Oreos down the eight ramps that lead to the CG mouth about 1/3 of the way through the spot. The agency wanted that moment to have more kinetic energy, but on set the oreos were pinned to the ramps and we didn’t have time in our finishing schedule to create and animate a stampede of oreos flowing down the ramps. I used a combination of keying, rotoscope and paint work to remove every single oreo and then replace them and their shadows to make it appear that they were stepping towards the mouth at the center of the pinwheel.

The agency wanted to experiment with different colors for the Oreoke, so Del would provide multi-pass renders of all CG assets so that I could grade them in Nuke and dial in occlusion and shadows as needed. Extensive color and depth of field work was used on the CG staircase as all of the oreos and their fillings had to perfectly match the colors of the practical oreos captured on set. Extra oreos were added below the pinwheel, in the spiral, and into the “wave” of oreos that are suspended in the air towards the end of the spot.

Working to our advantage was the fact that the oreo sculpture was so wild that even the practical footage had a nearly CG feel to it. The passes Del rendered proved to be invaluable as the agency wanted to experiment with transparency, reflections and shadows throughout the spot. The 3D camera solve was also extremely helpful throughout as it was used in conjunction with Nuke’s model builder node to quickly generate roto masks that seamlessly matched on set elements and relatively easy work of everything from adjusting the brightness of the windows to masking out objects as they passed in front of our CG elements.

Because of the spot needing to ultimately feel as though it was a single long shot and the footage being shot at 48 fps to help with the retiming work, the master Nuke script was over 1500 frames long. Organization was extremely important and a pipeline was designed and refined throughout that would allow us to keep all the various pieces that were feeding into the master script in order. One of the challenges in this regard was that Del was running off a rental PC while I was working off of a Mac Pro system. In the end we probably spent about two total days getting the pipeline set, which probably saved us over a week’s worth of work as we were able to easily pass elements between us and manage renders.