Red Bull

Exploring Augmented Reality with Red Bull!

Over the last few years we’ve done a lot of work alongside the Red Bull marketing team developing cool new AR experiences. This was a real learning experience for us, as initially we were totally new to the world of augmented reality. The plan at the very beginning (all the way back in 2019!) was to use the can itself as the tracking point for the AR elements, with ideas like taking a photo of yourself to be incorporated into the design of the real can.

It pretty quickly became clear that the technology just wasn’t quite where it needed to be though, the issue being that with the Red Bull can being as reflective as it is, the software could just never quite get a read on the can. The tracking algorithm bases what its looking for on a still image, so since the Red Bull can will never look the same way twice, even with thousands of images we wouldn’t be able to allow for every eventuality. So after much investigation (including a brief foray into machine learning), we decided a different approach was necessary.

Luckily, the team at Red Bull had produced some handy flyers, complete with QR codes that would take the user directly to the AR experience without needing to download anything. These flyers would sit on tables in pubs and bars, and because they’re matte and high contrast, the tracking software has no trouble at all picking them up. Below are a couple of examples of the different routes we took with this new technique…

Shark and Fish Cartoon

A lot of the work we did revolved around converting the classic Red Bull cartoon style from 2D into 3D, while still keeping what makes it quintessentially Red Bull; hectic, wiggly outlines and all. Our eventual solution was to basically rotoscope all of the individual elements of the cartoons (which got really fiddly at times, especially on things like the school of fish in this animation) and keep them as 2D planes moving around in 3D space. This meant we could keep as much of the great work done by the Red Bull animators as possible, whilst still adding our own flair on it. Some elements were converted into full 3D objects, like the treasure chest and rocks in this scene. We cut out parts of frames from the animation and applied them to simple 3D models to dot around the scene and fill out the environment a bit more.

This approach was key for performance reasons as well. Not only does AR on mobile devices have quite strict boundaries for the complexity of scenes due needing to run in realtime, the fact that the end user would need to load the website hosting the experience and stream all the necessary elements on the spot meant file size was a major concern. The fact that we needed to use video textures to show the animated characters meant a lot of that file size cap was already accounted for.

Superhero Cartoon

Next up is a more advanced version with the same general gist. We explored a few new techniques in this one, such as using the flyer as a base to build a bigger stage around. This allowed us to build a more immersive world, as well as more effectively simulate an indoor setting. Some fun masking trickery let us give the effect of a whole cityscape outside of the window that our hero flies through. Some great animation of a really intricate, origami-esque model by Jamie led to that super cool transition between the indoor and outdoor scenes.

Otherwise, this is all the same solutions we’d come up with for the sharks and fish cartoon, just leveled up for a more impressive final result!

Adding interactivity!

Where do you go from that superhero cartoon? Simple, instead of just having the experience play out in front of the viewer, they actually get to interact with it. In this experience, we let the user move a slider to determine the level of WIIIINGS! We go from a chilled out, post-work drinks, to a rooftop party, all the way to sprouting wings and flying above the city.

Another innovation here is that we managed to do away with video textures entirely. By splitting the characters up like paper puppets, we could animate their limbs individually in Cinema 4D, as long as we kept all their constituent parts on the same plane in 3D space. This was a huge space saver, and let us get much more variety into the scene in terms of background assets and number of characters. It also meant we weren’t relying solely on a pre-existing cartoon and could make our own experiences out of various flyers, cartoons and other promotional material.

The final thing that sets this experience apart is we’re no longer using the flyer. Instead we’re using whats called SLAM, or ‘Simultaneous Localization and Mapping’. The AR tracker just picks up whatever flat surface you choose, and projects the 3D experience directly on to it. This meant we weren’t confined to building the experience around the flyer, but could go with whatever form we wanted (the flyer was still necessary though as we still need the QR code).

Something slightly different – Red Bull Summer Edition

Just a quick look into some of the other styles we worked on with Red Bull, not connected to the cartoons. We had a lot of fun with this one, seeing what we could achieve while staying within the confines of what AR could manage at the time. Again using SLAM, we had a 3D can that bursts out of a watermelon, before filling a glass up a glass. The challenge here was the liquid, without a very complicated 3d simulation, which the AR environment wasn’t able to handle, we needed an alternative solution. We opted for a quite stylized look, with the liquid pour being a 2D video texture on a 3D plane, like the cartoon elements from the previous examples. The various fizzes and splashes are done with the same technique.

The watermelon itself was a challenge too, since you can’t do physics simulations in AR, we had to bake the simulation into keyframe data on all of the individual watermelon pieces. These pieces are hiding invisible inside the watermelon until the can starts slicing it up, at which point we toggle the visibility of the whole melon off, and the pieces on.

There are so many more examples we could share, we ended up developing around 20 unique experiences over the years, all with their own unique challenges that needed to be solved. By the end, we had gone from AR amateurs to aficionados!

Client – Red Bull