Real-time tracking in Production of Rock Opera “Crime and Punishment” by Andrei Konchalovsky

Moscow Musical Theater

InAVation Awards – 2017

Table of Contents

1. General Description of the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Brief Description of the Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

About the Production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

About the Project Team. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2. Goals, Tasks, and the Purpose of the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3. Requirements and Customer Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4. Stages of Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5. Solution Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5.1. Brief Description of the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5.2. The Principle of System Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5.3. Stage Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

6. Problems during Project Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

7. Contact information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1. General Description of the ProjectPolymedia, a technology partner of the Musical Theater in the performance “All about Cinderella” musical, yet again acted as a technology partner in a new production in the fifth theatrical season. The company developed and implemented a project for the premiere of Andrei Konchalovsky’s rock opera “Crime and Punishment” based on the novel by F. M. Dostoevsky and integrated an innovative solution into the production to implement an unconventional idea of the stage director: Real-time tracking.

For the first time in Russia, an innovative Black Trax real-time tracking system was implemented in a production. This allowed to enable video mapping on moving objects in real time. So far, this technology was used only by foreign producers and brought world-wide renown to such shows as Madonna’s concerts and performances by Cirque du Soleil.

Polymedia implemented its project in collaboration with Canadian-based CAST Group (Black Trax) and d3 Technologies, whose video server was used to process data from tracking devices in order to map the content .

“Our goal was to create a solution that is most suitable for this talented ultramodern production whose challenge, among many others, was to reach out to the younger generation within the audience by interpreting the classic novel anew. We wished to reflect the novelty of the format in everything, including visualization technologies. So far, not a single director in our country has resorted to a Real-time tracking tool; and we are glad to be the first to implement this technological breakthrough in the theater,” Helen Novikova (CEO, Polymedia) says.

Brief Description of the Object

Moscow Musical Theater, directed by Mikhail Shvydkoy, is the first musical theater in Russia. The theater is located inside the Gorbunov Palace of Culture (“Gorbushka”) building. Over the course of a year, the international team created a new theater in the completely renovated “Gorbushka” building. The theater features a contemporary hall for 1250 seats, new comfortable chairs, spacious well-lit halls, comfortable cafés and bars, a spacious lobby with a ticket office, a vast backstage area, and a convenient parking lot. The hall is fit out with up-to-date sound and light equipment. The stage of the theater is moveable and mutlitiered. All of the musicals in the new Moscow Musical Theater are accompanied live by a symphonic orchestra.

About the Production

A famous American-Russian film director, Andrei Konchalovsky began his work on the production of the music performance Crime and Punishment in September 2014, calling it a “poly-genre fusion performance”.

The idea of the production crossed Konchalovski’s mind more than 35 years ago: co-authored by the film director Mark Rozovsky and the poet Yuri Ryashentsev he wrote a libretto based on a work by Dostoevsky in 1979. The author of the music is the composer Eduard Artemyev.

The decorations were created based upon drafts by Matthew Dilley, a set designer who has been actively working together with a number of theaters across the world.

According to the idea of the film director, the events of the novel are moved from the XIX century to timeless space. The opera written five years ago for the libretto of Mark Rozovsky and Andrei Konchalovsky was decided to be transformed into a musical performance for the sake of the new generation of audience, which has people who are not familiar with the most well-known work by Dostoevsky.

The performance was premiered on the 17th of March 2016.

About the Project Team

The Russian system integrator Polymedia, known for its previous involvement in the production of the musical “All about Cinderella”, was invited to implement the technical part of the project. Based upon technical goals and tasks, the offer to the theater from Polymedia featured the most up-to-date technologies that became available owing to inviting two world’s leading high-technology solution developers of the entertainment industry: CAST Software (Canada) and d3 Technologies (UK).

2. Goals, Tasks, and the Purpose of the Project• Development of an unconventional solution that corresponds to the stipulated production

innovativeness

• Creation of a solution for mapping based upon projection technologies

• The possibility of accurate positioning of video content on a decoration that moves about the stage with no predefined trajectory must be found

3. Requirements and Customer Feedback1. Mobility of installations. Easy and quick assembly and mounting that requires no contractors

2. Effectiveness and innovation of the solution

3. Freedom. The solution must not limit the imagination of the director

Feedback

“We all were lucky to see the great performance, which is as much the merit of our theatre team and creative group, as it is of the Polymedia company team, since it was their engineering that made the rock opera “Crime and Punishment” performed in Musical Theatre one of the most technology-savvy audiovisual shows in Russia.”

Art Director of Moscow Musical Theatre, Mikhail Shvydkoy

4. Stages of Implementation

5. Solution Description5.1. Brief Description of the Project

The project of the production consists in integrating two systems, tracking and video-mapping.

The video-mapping system is realized on the technology of reverse and front projection using Panasonic equipment: four PT-EX16KE projectors, four projector of the series PT-DW11KE, and short-throw lens rear projections of D75LE90 series. This set of equipment has allowed to unite scene and scenery in a unique mapping space.

The tracking system detects and follows the position of moving objects and spinning decorations on the stage by using special cameras and sensors, and the read data with the shape and configuration of the moving objects are transmitted to the video-mapping system, which, in its turn, processes the content in real time so that the projection on decorations is displayed according to their position and dynamic rotation. Integration of this smart system and audio and visual stage equipment creates a dynamic 3D effect, but unlike a 3D film, it is implemented online without editing.

The real-time 3D tracking Black Trax system from CAST Software was chosen to implement the project. This system allows to track objects in space in real time with a subsequent data processing on a video server.To process tracked data, a video server from d3 Technologies, which is a well-known British company, was chosen.

5.2. The Principle of System Operation

To track objects (in this case, stage decorations) in space, Polymedia specialists have integrated the BlackTrax tracking system. Decorations have infrared emitters (IR beacons) that are invisible to the human eye attached onto them. These emitters (beacons) send the location data for objects (decorations) that move about in space. To read signal data from the beacons, special high-speed IR BlackTrax cameras are used.

ID Phase of the project Duration(days)

Start Termination

1 Development of project concepts 90 days Tue 01.09.15 Mon 04.01.16

2 Concepts approval 2 days Tue 05.01.16 Wed 06.01.16

3 Designing installation 14 days Thu 07.01.16 Tue 26.01.16

4 Equipment and supplies order and delivery 21 days Wed 27.01.16 Wed 24.02.16

5 Routing cables, embedded parts 7 days Wed 27.01.16 Thu 04.02.16

6 Tracking system installation 1 day Fri 05.02.16 Fri 05.02.16

7 Projectors installation 2 days Fri 05.02.16 Mon 08.02.16

8 Deployment of video server and workplace 2 days Tue 09.02.16 Wed 10.02.16

9 System setting and programming 1 day Thu 11.02.16 Thu 11.02.16

10 Prototype testing (first trial runs with theatrical scenery) 22 days Fri 12.02.16 Mon 14.03.16

11 First performance 1 day Tue 15.03.16 Tue 15.03.16

2901040710131619222528010407101316192225283103060912151821242730030609121518212427300205081114172023262901040710131619222528020508111417September 2015 October 2015 November 2015 December 2015 January 2016 February 2016 March 2016

Задача

Прерывание

Веха

Суммарная задача

Суммарная задача проекта

Внешние задачи

Внешняя веха

Inactive Task

Inactive Task

Inactive Milestone

Inactive Summary

Manual Task

Duration-only

Manual Summary Rollup

Manual Summary

Start-only

Finish-only

Ход выполнения

Крайний срок

Приложение к письму о подаче офертыот "25" апреля 2014 г. №______

Календарный график выполнения работНаименование и адрес участника аукциона: Закрытое акционерное общество "Полимедиа", 109240, г. Москва, ул. Верхняя Радищевская, д.4, стр.3-4-5

Директор по работе с корпоративными заказчиками __________________________ Чебышев Д.В.

To locate objects on the stage accurately, Polymedia has developed an IR-camera placement map. Thus, when objects move about the stage, the system obtains fully outlined decoration contours, which allows to transmit the coordinates of the decorations to the d3 media server to render the whole video content in real time with a subsequent projection. The d3 system also performs edge blending of both frontal and rear projections, taking into account all of the content, balancing the brightness, and juxtaposing images. The principle of the visualization server is simple: it is rendering and edge-blending images in real time by using the signals from sensors.

The total number of projectors on the stage is eight; during the performance the stage scenery changes its position 36 times, and the mapping content also changes 36 times. The integration of such an intelligent system with audiovisual equipment creates a dynamic 3D effect, but, unlike a 3D movie, it performs online, with no need to cut.

IR BlackTrax beacon

5.3. Stage Equipment

Position of the Projection Equipment on the Stage

6. Problems during Project Implementation7.1. Coordination of Projection Mapping for Decorations and Trajectories of Decoration Movement

Inasmuch as decorations are moved by the stage staff, which does not happen automatically, there is always a possibility of inaccuracy: the objects that are visually placed directly upon points A or B have their real positions always shifted relative to their respective points. It was important that the content should be projected without inaccuracy right onto specific objects, regardless of the objects location or their trajectory. In order for this to happen, the tracking system was developed in such a way that it follows the decorations in space and sends the exact coordinates to the operator to position the content accurately.

7.2. Integration of Audiovisual Equipment with Lighting Devices The small size of the stage and a high density of placement of lighting technology and theatrical objects caused difficulties when mounting IR cameras of the BlackTrax system onto decorations. Vibration from the equipment affected the IR cameras, and they changed the accuracy of adjustment. Dust, halogen lighting, and other external factors significantly affect the calibration. An accurate fine tuning for each camera, tracked object, and IR sensor was required. To solve the problem, Polymedia developed a “Coverage Map”, which provided an accurate calculation of the camera layout relative to all of the stage decorations. Polymedia, together with actors, played over different situations that caused loss of calibration and achieved the ideal adjustment that included all of the possible deviations. The “Coverage Map” was endorsed by the technical administration of the theater and will serve as a guide during next installations, assembly, maintenance, theatrical seasons, and tours of the theater.

7.3. The Regular Recharge Requirement for Infrared Beacons Polymedia specialists solved the problem by developing a plug-and-play system. This is a unified system for charging all of the beacons on each decoration, which required only an extension cord for plugging the power. Developed by Polymedia specialists, this quick-charge module is capable of recharging beacons in mere minutes instead of three-four hours that were required for the stock charging hub.

7. Contact information

Polymedia

29 bld.1, Krzhizhanovskogo str., Moscow, 117218, Russia Tel: +7 495 956 85 81 Fax: +7 495 956-85-84 info@polymedia.ru www.polymedia.ru