FPS Immersive Experience

The most exciting and challenging project to date by far was an Immersive Experience project we were hired to film for Fleming’s Prime Steakhouse. In September 2018 we were to set out on a magnificent journey along the Pacific Coastline from Ensenada, Mexico up to Humboldt, California. Along the way we were to film with various producers of ingredients for Fleming’s seasonal menu items. The goal was to create an Immersive Experience to be showcased in restaurant event spaces that would transport guests to the places where ingredients and wines were sourced from. The trip was to take place over 7 days with 5 production/travel days in the middle so we knew we were definitely in for a challenging time, but we were up to the task.

Pre Production

The first thing on the agenda was update our expired or expiring passports within a very short time frame. This meant we had to expedite the process just to ensure they would arrive in time. We found a company online that specialized in this and surprisingly everything went smoothly so we would be ready to travel without concern. It cost a pretty penny but we were rather limited on options seeing as I would be on another trip leading up to this job.

There were numerous unknowns we needed to address before the project and since the trip meant crossing the border into Mexico, we wanted to know what steps needed to be taken, if any, to cross with the amount of gear we had. The answer was certainly, yes. After doing research we felt the best option was to secure a carnet, which is a customs document you present when entering another country. It is also known as a merchandise passport. It basically lists your equipment including serial numbers and value of the items. Of course there are costs associated with obtaining this document but when it comes to gear, especially expensive gear, we weren’t wanting to take any chances and be forced to leave equipment behind.

The next question we needed to address was if there were any steps that needed to be taken for us to fly our drone in Mexico. At the time there wasn’t a lot of information available online so we did our best to reach out to any local production companies we could find as well as have a Mexican National dig into the Federal Civil Aviation Agency (Mexico’s equivalent of the FAA). We discovered no apparent issues with us having to submit any official paperwork but we did our best to inform their agency where and when we would be working just to be safe.

With our two main logistical questions answered we were then able to focus on the unique technical requirements of this project. The biggest challenge was the resolution of the final deliverable being around 16,000×1,000. This was so that the visuals could be projected onto all four walls surrounding the guests in the event space.

Part of the project was to capture 360° video at select locations and we made the suggestion that we might as well capture ambisonic audio for a truly immersive experience. This was our first time delivering anything in 360° so we made sure to research as much as possible in preparation for the assignment. We looked into various camera options and opted to go with Insta360 for the camera and we picked up a nice kit specifically for ambisonic audio capture that included the Sennheiser AMBEO VR Mic and the Zoom F8 MultiTrack Field Recorder.

Other gear we used for this trip included Canon C200 cameras, Canon L series lenses, a DJI Inspire 2 and the Teradek Serv Pro. The Serv Pro was the perfect monitoring tool for this job as it would allow the agency and client to all monitor what we were filming without needing to set up a video village in such a run and gun environment. Being able to look on their phones or tablet through the VUER app at what the camera was filming would really help us keep moving quickly and efficiently.

Production

Day 1

We flew into San Diego, California, then made our way down to the Tecate border crossing to enter Mexico. I had arranged for my cousin to help us navigate the carnet process with the Mexican federal police as well as escort us through Mexico without incident as he had spent their entire life in the area. Before being allowed to proceed on our trip we had to take out all of the gear listed on the carnet document and cross check it in front of the agency officer. We are talking over 100 pieces of gear so needless to say it was quite a long process, especially when their agency was not in a particular hurry to help speed things along. The agency team and client team waited at a nearby restaurant for us to clear the inspection and after about ninety minutes we were finally made it through and started the drive down to Ensenada. Upon arriving at the hotel we grabbed a quick dinner then began preparing for the day ahead. Knowing we would be chartered to the island on a boat, we had to really only bring the gear we knew we would need.

Day 2

The following morning we boarded the boat and headed off for the hour long journey to this island where they farm Striped Sea Bass. The first thing we noticed as we neared the island was the incredibly high numbers of birds swarming about. We did a quick scout then started off capturing 360° content. The chef was preparing lunch for the workers as well as our entire group which led to some pretty amazing B Roll as he prepared some of the Striped Sea Bass right before our eyes.

After a delicious meal we boarded a smaller boat that was to take us out onto the water to interview one of the Pacifico Aquarium workers. The boat ride was equal parts exciting and nerve-racking as this was a small motor boat quite smaller than the one that brought us to the island. Stepping onto this platform floating on the ocean with the team and our equipment certainly had us on edge. Definitely not a good place to drop something and fortunately no one did. We wrapped up our filming on the platform and made it back to the island.

While filming aerial footage of the island and surroundings, we began to notice some of the birds getting closer and closer to the drone. They were becoming territorial so we were doing our best not to instigate them further while just hoping that they didn’t get bold enough to try and take the Inspire 2 down completely. We managed to avoid a disaster and brought it in as we prepared to go film another interview.

It was about a half day spent on the island before we wrapped up and headed back to the mainland to gather our things from the hotel and head back to the states. Upon returning to the US we needed to go through customs on the Mexico and US side to account for all of the equipment one last time. Fortunately the same Mexican agency official who we dealt with on our way in was there to help us and she seemed to be much more relaxed about the process this time around. We crossed through without any problems and after clearing US Customs we set out to make the long drive up to Paso Robles.

Day 3

Paso Robles was a two stop trip. First we visited the Javadi Farm where we had previously filmed the Mina Mesa film for Total Wine & More. The Javadi family not only make exceptional wine but they also happen to raise cattle as well so we filmed some B Roll of the cattle before heading over to DAOU mountain to film our 360° content from one of their incredible vantage points. After a few long takes, we loaded up the gear and made our way to Fresno to check into the hotel and prep gear for the following day.

Day 4

The trip also too us to a fig farm in the middle of California’s agricultural heartland. We were surrounded by fig trees, which came with some nostalgia as I grew up with a fig tree outside my bedroom window. The aerial footage of the farm was impressive specifically because the vast fields of trees on the property that included tall groves lining the roadways. The sun breaking through the branches and leaves provided some equally stunning B Roll footage of workers and we finished up with an interview with the property’s manager which included a very curious cat sitting right smack in the middle of it all. We next headed over to an artichoke farm near Monterey for some additional B Roll and aerial coverage before making the drive up through San Francisco to our hotel in Napa.

Day 5

We first visited Robert Mondavi Winery to interview one of their team members as well as B Roll and aerial footage. Next up was Freemark Abbey Winery to capture 360° content. Finishing up Napa was Far Niente for B Roll and another 360° content capture. This was where the client portion of the journey came to a close, leaving us to travel up to Arcata and Humboldt with just the agency team of three. We were all tired and running simply on the adrenaline of adventure but we could start to see the finish line now.

Day 6

The beaches of Arcata provided some incredible coastline imagery and not just from the birds eye view of the drone. There were visual opportunities everywhere we looked. So we set up for our final 360° content right on the sand with ocean to the west and Northern California cliffs to the east. It was so calm and soothing and to have this be the last leg of the trip made it extra special. I believe we all sat back and soaked it all in, teetering between being right there in the moment, and reminiscing about the days and miles between us and the beautiful Ensenada waters where it all began.

Day 7

The next day we would make the long drive back down to Oakland where we could fly back home to Phoenix, Arizona. It seemed like it took so much longer getting back as by now, the adrenaline was all but gone and the reality of how much we had put into the past week had caught up with us. I wish I could say that the hard part was behind us but little did we know what Post Production had its own challenges in store for us.

Post Production

Within a couple of weeks scripts and asset details began to come through. It was time to begin building towards the final deliverables. Up to this point, I had been editing entirely on my laptop. I knew that was not going to be an option with this project due to the amount of content we captured as well as the unique resolution for deliverables. We went to work on building a beast of a desktop machine.

Once scripts and storyboards came in we began to build out the videos. After some rounds of revisions we began to finesse the cuts before preparing to then work the edits into the unique resolution. The final output resolution for delivery was to be 16,000 x 1,000. Initially the plan was for each video to wrap around the entire room but that proved to create its own challenges due to the layout of the room as well as the corners of the walls not really complimenting the imagery. Another big issue with the single image approach was just how much of the image we would be cropping out on the vertical height. Igloo, the company they partnered with to map out the project and install the system had mainly been accustomed to running their projections in round rooms that would be set up for special events. They certainly had their own challenges customizing their system to the event space but everyone was working hard to see this through.

Eventually the decision was made to project the same image onto the four main walls of the room so that no matter which way guests looked, they were seeing the same imagery. This meant the crop factor would not be as intense and allowed us to keep more of the great imagery we had captured on the screens. So using a template provided by the agency we had to then take each video and resize it to four varying sizes that represented each wall in the event space. These four dimensions would add up to the unique resolution of 16,419 x 1,000. The new challenge this presented was accuracy in measuring the walls as they were not all uniform.

In order to ensure accuracy we scheduled a test run prior to launch to allow us time to address any possible issues. I flew to Orange County, California to finally see this concept brought to life. So much emotion leading up to this moment after encountering frustration and challenge throughout the editing process that we couldn’t fail now. We weren’t sure what to expect due to so much back and forth between the agency and igloo regarding an audio situation. It turned out the system they had been installing wasn’t equipped to actually present surround sound. This was quite a disappointment due to the work we put in to capture the ambisonic audio. Fortunately we were able to provide an AV tech to install the surround sound system that would compliment the projector system.

So we loaded up our files and prepared to witness something truly incredible but it quickly became apparent that things were not ready to go live. The audio wasn’t working, the visuals were off centered and all I could do was try and think if I may have made any mistakes with the HAP encode process necessary for their software to play the files. The agency was concerned as well and reached out to igloo who put us in touch with the head of their tech support in Australia, which presented its own challenges due to the time difference. Fortunately he confirmed that I had been doing everything correct on my HAP encode then said he was going to work on a possible fix.

We spent the next week trying to figure out a solution when igloo finally sent over an update to their software system that was supposed to fix all of the issues. We went back to the restaurant to se for ourselves, not getting our hopes up too high. What followed was a moment of absolute emotional free flow as we saw the imagery up on all four walls with full surround sound working and it was mind blowing. All of the hard work we had put in had finally paid off and tears ran down from my eyes as I pulled my phone out and began recording the experience. I thought to myself, it doesn’t get any better than this.

The agency, however, had other ideas as to how they could elevate the experience even further. Days before the official public launch, they invited select guests to take part in this experience as a sort of official dry run. The agency had sourced specific items such as seashells from Pacifico Aquarium, soil from DAOU mountain, fresh herbs and more that was placed into a touch box for each guest so that they could hold the items and smell them as well as taste them to elevate the experience from something visual and audible to then reach all of the senses. We filmed the test run to create a trailer for the experience which you can see here.

Everyone was so proud of how well it had all turned out and without a doubt we were all a part of something truly special. Our work was done, for now. The official launch was a great success and there was discussion about taking the experience to other restaurant locations as well as creating fresh content regularly. It would be a huge undertaking as it would mean that restaurants would need to come up with the extraordinary budget to install the project mapping system as well as surround sound.  We are still hopeful that there are more opportunities to see this through but the 2020 pandemic certainly slowed down those discussions.

To date this is still the most memorable, albeit whirlwind of a project we have taken on. Being able to be a part of something so different and unbridled was equally terrifying and exciting at the same time. We learned so much about our capabilities throughout the project and developed some great work flows that we still apply to this day. The main takeaway is that when you are hired to do any task, ask yourself, how can we make it even better.