Flying high with innovative camera technology

Established as a world championship in 2005, the Red Bull Air Race is an international series of events in which competitors navigate with aeroplanes a course of obstacles often over water in the fastest time. Each race involves pilots making complete tight turns up to 10 G through a slalom course comprising a pattern of pylons, known as Air Gates.

Broadcast coverage of such fast moving and complex events imposes considerable demands on everyone involved. Camera operators have to be top of their trade. Directors need an in depth understanding of what is involved. And graphic operators must remain vigilant when it comes to providing the very latest augmented on-air graphics – for such information has become an accepted necessity when it comes to describing high profile events. In the case of the Red Bull Air Race, the graphics overlay must reveal details of the gate number, height of the plane, wind speed and direction and GPS information during each flight.

Of course, to be effective, the positioning of these added – or augmented - graphics must appear ‘pinned’ to the live background of the planes in flight. And to achieve that result, the output of the broadcast camera must be processed with precise tracking technology.

Aerial challenge

“When the graphics company netventure – the French and German broadcast graphics service provider and distributor of Vizrt solutions - approached us with the challenge, we provided a solution based on our experience with gyro stabilization,” explains Michael Geissler, CEO of London based Mo-Sys Engineering.

According to Geissler, there are two major challenges when it comes to creating the precision technology needed for such events. “The first challenge involves the height from which the main coverage camera must operate. Filming a plane with cameras looking up is quite boring for the viewer and does not convey the speed of the event. To generate that excitement the key cameras need to look down in order to show the planes in relation to the terrain and sea below. Such cameras, therefore, need to be mounted on cherry pickers - perhaps some 50-100 metres above the ground – and that means a certain amount of swaying and wobbling is inevitable. And that makes referencing for placing graphics in the correct position very hard.”

The second challenge stems from the fact that such races stretch over distances of up to two kilometres – requiring very long zoom lenses to be employed. Viewing angles are down to half a degree, and for HD production this means that the graphics tracking systems need to be accurate to 1/4000th of a degree in real time. “That is a challenge enough, but added to the movement of the camera crane, means even more accuracy in tracking is required,” admits Geissler.

Up until recently, this level of precision has been almost impossible to achieve in challenging situations such as air races. Previously used optical tracking systems fall short for a number of reasons. For example, such systems generally operate with just a 120 degrees wide view. That means if the system is referenced to half a degree, the result is 240 times the noise. And where the background – in the case of the Air Race, the water - is moving, optical systems find it hard to locate non moving, fixed reference points.

Stabilizing effect

Geissler continues, “Working closely with netventure, we concluded that an inertial tracking technology with high spec gyro sensors in combination with our existing tracking technology was the answer and we called it GyroTracker. The GyroTracker sensors are mounted on the box lenses of the camera to provide accurate motion data of the tiniest movement - even when the camera is swaying many metres in the air. In fact, so sensitive is the system that even the tap of one finger on the lens is enough to record a movement.”

The fact that cameras are mounted so high is actually a benefit to the system since the birds’ eye view provides a good method of lining up objects that are used as reference point for the positional graphics.

Geissler says that they found a method of managing the ‘drift’ from which all gyro systems will experience. “We reduced the drift to as little as 1/50 speed the of the earth rotation. But, what’s more, we developed a method that compensated for the earth rotation by calculating 2000 times per second the effect of the earth rotation based on the cameras position on the surface of the globe and the direction the camera is pointing. The system tracks precisely up to half an hour before needing to be re-homed. The procedure was so easy that at the end of each race, the camera operators chose to reset using this re-homing procedure by focusing on an agreed point and pressing a button. The system immediately knows where it is located and can allow the graphics to be positioned accurately.”

This Mo-Sys GyroTracker technology premiered during early 2014 in Austria as a proof of concept and was further tested in Spain later that year. The first real outing was in February 2015 in Abu Dhabi, the initial stop of the new Red Bull Air Race season. After further tests, revision of software as well as electronics, GyroTracker achieved a tracking result that met everyone’s expectations and became after the race in Rovinj a firm part of the Air Race production technology.

Close cooperation

Geissler says he is very thankful to both netventure and Red Bull Air Race for allowing the development of the system throughout the season and so provide the impetus for the tracking technology to mature. “We all worked very closely together. netventure, expanded its existing knowhow and also constantly improved the workflow and set-up procedure to deliver live augmented broadcast graphics fast and reliably. netventure developed its own application which works in conjunction with Bing Maps to align the location of the race, gates, cameras and virtual objects resulting in speeding up the process of the setup. It all added to the viewer experience to make the races even more exciting.”

He adds, “What made this project possible in a short time was the iterative, repeated testing and maturing cycles. Every four to six weeks there was a race where we were testing, revising and improving the technology in real life situation, not just in our lab. After a race there was a revision and we had to the next race to improve.”

Alongside the GyroTracker, the production facility for the outside broadcast acquired a Mo-Sys L40 remote head for the virtual informational graphics. With the L40, ultra precise tracking information, including lens data, plus a responsive and intuitive operation is possible through single mode fibre pair. The tracking data from L40 and GyroTracker is send by a separate fiber pair and then collected by a specially designed triple gateway with fibre conversion, and fed into the Vizrt Tracking Hub via RS422.

Geissler anticipates that such tracking technology will become more widely used on other sporting events, especially where long lenses from shaking camera platforms or grandstands need to provide tracking information for precise positioning of augmented graphics to make the broadcasting more dramatic.