Lexus | Trace Your Road

Fake Love

I worked as the Lead Creative Technologist on this project for Fake Love. We were tasked with creating a life sized video game for a small group of participants and a professional F1 Driver to show the capabilities of a new hybrid model from Lexus.

The participants were selected from Facebook and invited to an airplane hangar to participate in this one-of-a-kind experience. They were seated in the passenger seat of the car and given a tablet. A timer began counting down, and they were instructed to trace a "road" on the tablet to a virtual checkpoint. As they did this, their path was projected in real time on the floor of the airplane hangar. The F1 driver would follow their projected path as quickly as possible, and overhead tracking cameras would determine when the driver would hit the checkpoints and trigger the software to advance the game. The fastest participant to collect all the checkpoints was declared the winner.

The entire experience was shot and edited together as a commercial. The finished piece was a viral hit - it received over 2.3 million Youtube views, and resulted in a measurable uptick in dealership visits for Lexus.

Technical Approach:

This project had about 6 weeks of development time from concept to launch. It was a very large production and had a lot of technical challenges to overcome in a short amount of time. Additionally, it was completed in 2013 on a lower end budget when a lot of hardware and software tools for this kind of work were still a bit underpowered for this application. In short, we made it work with what we had.

The on-site prep was about 4 days, the projectors weren't fully installed and usable for about 2 days, and the entire event occurred over one long day from about 8am-10pm.

The tech stack was as follows:

• Software:
  • Primary Development Environment: openFrameworks 0.7.3 for MacOS and iOS
  • Communication Protocol: OSC (Open Sound Control) - we actually developed an open source addon for this called ofxOSCSync
  • Total apps: 4.
    • Master controller - handled drawing the road, checkpoints, projection mapping correction and blending
    • Renderer - a modified version of the master controller
    • iOS Participant app - sent drawing commands to the master controller and controlled the overall game loop
    • Computer Vision Tracking App for car position - tracked the car and sent position to the Master App
    • Keeping apps separate and only communicating important bits over OSC allowed us to work quickly without worrying about integrating everything into one mega app.
• Hardware:
  • 5x Mac Minis - 1 Master Controller and 4 Renderers. The master would render a low resolution preview of the entire projection canvas for monitoring during runs, and we would use it send commands to each of the 4 renderers that rendered their individual piece of the canvas to each of the 12 projectors
  • 4x Matrox TripleHead2Go to create 3x 1280x1024 outputs from each of the 4 Render Mac Mini's. Final canvas resolution was 5120 x 3072px.
  • 1x Mac Pro - w/ HD-SDI capture for IR Camera and Computer Vision tracking
  • 1x iPad running the participant software
  • 1x Linksys Wireless Router
  • 1x IR Camera over HD-SDI with a fisheye lens
    
  • 12x 20,000 Lumen FLM20HD Barco Projectors with 0.73 short throw lenses and 4:3 aspect ratio
  • DVI to Fiber converters to run video signals from the control booth to the 12 projectors in the ceiling
• Other noteable tech elements/things I would do differently:
  
  • Team/my role:
    • I don't have everyone's name at this point, but on the Fake Love side, Layne Braunstein was the chief creative, Marie Celine Merret was our excellent producer, Dan Moore was the primary software developer for the controller and renderers and contributed heavily to overall architecture, and Ekene Ijeoma made some fantastic additional visuals. I developed the iOS app and the computer vision component, as well as the overall technical approach and documentation. Another vendor did all the the projector installation and alignment.
  •  Projection:
    • Finding a suitable location with a big enough surface and a high enough ceiling was the biggest challenge. The production ended up identifying a miltiary airplane hangar outside of Rome, Italy.
    • The projection surface was roughly 220ft x 120ft and required a ceiling height of 45ft. Each projector at that height covered roughly 60ft x 40ft of floor. They were arranged a 4x3 grid. The floor of the airplane hangar was grey concrete and ate a lot of light. Even with 20,000 lumen projectors, our projected brightness was not the best, especially considering the need to film everything. Very expensive lenses with very low f-stops had to be used to capture enough light.
  • Car tracking:
    • The car had bright IR lights covertly installed in the rear window so that they couldnt be seen during filming.
    • Other tracking elements had been considered, like LiDAR, GPS, and multi-camera setups. The tight timeline on site and the relatively demanding calibration needed for those solutions, as well as their high cost made them not feasible. In hindsight it could be done, but it is important to keep in mind how quickly the project moved and how small the team was.
    • The single camera solution had a 160º fisheye lens on it to capture the entire canvas from a height of 45ft. 
    • Unfortunately, testing the fisheye at scale prior to arriving was not possible. When we arrived, our lens correction algorithms weren't working as planned and tracking did not align properly to the car.
    • As a quick fix to doing automatic tracking, we actually ended up building another app that would allow us to manually track the car using a modified app and a mouse. I sat up in the ceiling of the hangar during live runs and moved a subtle blue dot along with the car to make sure that it still triggered the checkpoints.
  • Wireless challenges
    • Looking back, using a single wireless router to cover such a large distance was probably not the best choice. It did work fine in testing, but the day of the shoot, the crew installed about 8 wireless GoPro cameras in the car to capture internal footage. This combined with the long distance began to cause issues with the smoothness of the controller iPad. We ended up having to move the router much closer to the driving spot and away from the control room.
  • Master/Renderer Computers
    • In hindsight (and with more budget) this could have been done better, especially in the present day. Rendering 12 outputs from a single computer was difficult in 2013, and would have required a custom PC or other expensive solution. I had experience doing something similar with MacMini's for our work with Shen Wei at the Park Avenue Armory and used a very similar approach.
    • The Matrox TripleHead2Go was particularly finicky with MacMini's back then and we had an emergency when 2 of the ones we brought with us ended up just totally not working after a couple days and we had to locate the only two Matrox TH2Go's within a couple hours of Rome - not an easy task.
  • Project Documentation Highlights:
    • Below is the Hardware Wiring Diagram and the Projector Layout Plan for the project

Contact

©Blair Neal 2020