7thSense Design Delta Media Server

Mesh Mode

Eyepoint Projection and

Textured Modes

Copyright © 7thSense Design Ltd

Trademark Information Delta, Delta Media Server, Delta Nano, Delta Lite, DFM, and Delta Audio Server are trademarks of 7thSense Design LTD. Brand or product names may be registered trademarks of their respective owners. Windows is a registered trademark of Microsoft Corporation in the United States and other countries. Copyright Information All Rights Reserved. These documents are copyrighted © by 7thSense Design LTD and shall not be reproduced or copied without express written authorisation from 7thSense Design LTD. The information in these documents are subject to change without notice. 7thSense Design LTD assumes no responsibility for errors, and/or omissions contained in this information.

2

Mesh Mode

• Projection onto a 3D object, which may be a regular or irregular shape (e.g. a building)

• Uses 3D model of the object as the Delta ‘mesh’

3

Prepare a Model

• Create a CAD/3D model of the object (Designer)

• Save as .obj file type

• Can be simple or very complex shapes – but for best results must be accurate!

• Origin: set at a central point of the object/model

• Eyepoint: the point from which the real object/media projection is to be viewed (projected mode)

Note: Physical object must have identifiable 3D calibration points (at least 7) to match CAD/3D model vertices for accurate projection…

…or the object must be accurately positioned in a 3D calibrated ‘space’ 4

Store files on Delta Server…and Mesh file on GUI too!

• OBJ files in C:\Meshes\...

• Image files in C:\Images\...

• Movie files in E:\Movies\... 5

Error! Mesh not stored on GUI machine

For 3D calibration/shows, you must have a local copy of the mesh on your GUI C: drive as well as on your server

If there is no local copy of the mesh, you will get an error message and the GUI may crash!

6

Textured Mode • Media is ‘baked in’ to the

model so that it effectively wraps around the object

• Used to project media around

a 3D object for up to 360 coverage

Projected Mode • Media is ‘flat’ and (usually)

projected onto a single face of an object from a specific eyepoint

• Could be multiple projections

on multiple meshes but each will be ‘flat’ projections, directed from a single eyepoint

Which Mesh Mode?

7

What is Projected Mode?

• 2D media projection

– May be created specifically to fit the object

• Viewed from a specific eyepoint

• Projector(s) may be positioned offset from the eyepoint to cover required projection space

• More than one eyepoint may be used if multiple objects/media are used

8

Projector locations • One or more projectors may be used (depending upon requirements)

• Lens and other settings should be noted for Delta configuration

• X Y Z projector locations are used in Delta to assign ‘sections’ of the media for projection

Remember:

• Single media eyepoint (cannot project round full 360)

• Projections can cross over to allow for blending

+x -x +z

+y

= Origin

9

Projected Mode - requirements

1. An object to project onto

[Note: for this example we are using boxes arranged to resemble a Pyramid shape: a real example could be a full-size building or very complex shape]

2. An image file: which is usually created to specifically fit the object (as seen here)…

2. 1.

10

A Simple Example – Projected Mode

…or other media, such as a movie, may be projected onto the object

www.caminandes.com

11

Limitations

• Model must be accurate and to scale for best projection of media

• Physical object must have identifiable 3D calibration points (at least 7) to match CAD/3D model vertices for accurate projection…

• …or the object must be accurately positioned in a 3D calibrated ‘space’

• Media only projects on the surface you can see from the eyepoint. From another eyepoint (view), distortion or no media may be visible

• The object is fixed and has no dynamic movement – If the object is moved, projection will not line up

12

Projected Mode: Eyepoint

The Eyepoint must be set for Projected Mode as the media is directed to the object from that precise angle/distance (even if projectors are offset from this point)

Note: the GUI model may be viewed from different angles, but the ‘eyepoint’ is maintained for projecting the media

Eyepoint

13

Media distortion

If the object is viewed from another angle (not the eyepoint), the media may be distorted or there may be areas where no media is visible

14

Projected Mode example:

In Delta GUI, create a new file

• Click File > New

• In the popup window, select Resources and Configuration

• The system will prompt you to clear the loaded file and start new – click Yes

• Click on Config then Display

• Select Mesh Mode 15

Mesh Mode environment

Channel and Mesh panels:

• Settings used to set up positioning relative to the eyepoint

Grid (Off…100%):

• Allows the grid to be viewed on the Playback Window (for alignment purposes)

16

Load mesh file (OBJ)

• Click the Files button

• Double-click an ID number to attach (or edit) a file

• Browse and select your file

• Click to select Y for Visible and Project on Mesh options

• [Repeat to load further files if required]

17

Mesh Loaded You should now see the name of your mesh listed in the MESH panel

Note that you will not see your mesh until you change your view (see 3D Projector Calibration)

Check that your Active Canvas matches your output setup

18

Channel Settings

• Set each channel (usually one per projector or group of projectors) to Active

• The settings for each will update automatically as you calibrate

These settings should default in according to your mesh/3D model so you should not need to do anything with them, but you can scale the mesh or position it as required here

Mesh Settings

Projection Settings

19

Projected Mode: set Eyepoint!

In the MESH panel, click EYEPOINT

The FILES button will change to read EYEPOINTS

Change the eyepoint settings to reflect ‘real’ measurements of the object in relation to the eyepoint

20

Eyepoint Details

To check/change Eyepoint Details:

• Click the EYEPOINTS button

• An Eyepoint should be listed as Projected (as default)

• Double-click to check/amend Eyepoint Details if required…

• Click OK [OK] to accept settings and return to main GUI screen

• Untick “Use texture coordinates” to enter projected mode

Advanced functionality:

• Further Eyepoints may be added (double-click a new ID) for example, if you want to project multiple images on specified areas or multiple objects via multiple channels

• Each eyepoint must be set up according to planned perspective (direction/distance etc)

• Specific meshes may be applied to each eyepoint in the relevant Eyepoint Details by ID screen

• See variations in ‘Textured Mode’ section to see how multiple channels are set up

21

Change View to 3D Calibration Screen

In the MESH panel

• Select 3D Projector Calibration

• Click Enable use of 3D Calibration Data icon

• Click to Select Reference Mesh from the list of meshes that you have loaded 22

Move mesh into view

Use your mouse controls to zoom/drag your mesh into view:

• Left-click and drag = pan and tilt

• Right-click and drag = shift

• Centre-click and drag = zoom

Note: moving your mesh around to how you want to view it may take some practice!

23

Physical setup

Ensure that projectors are set up according to planned eyepoint/projection onto your object

24

Show Help

The Show Help button gives on-screen advice if you need it for calibration…

25

Grid On

The Grid defaults to Off but can be switched on if it helps you to calibrate the projection

Note: the grid/projected image will only show on your model once you have calibrated at least 6 points

26

Add Media to Timeline

In GUI, click on TIMELINE and select your media (image/movie) from those listed – these are stored on the server

Note: one or media may be added according to requirements but you probably only need one ‘active piece’ to set up and test!

27

Configure 3D Points: CONFIG > Display > 3D View

• Click on an appropriate calibration point (usually a ‘corner’)

• The point will show in RED

• Press Shift+D to make the dots bigger or D to make them smaller

Note: You do not have to view the mesh precisely from the eyepoint but it helps if it is in a similar orientation to the actual object to enable accurate calibration

28

Line up Cross-hair on Object

• Switch to 2D View in the GUI

• Your mesh will disappear and a cross-hair will be in view – your projector(s) should show a cross-hair on or around the object

• Use your mouse to move the cross-hair projection to the point on the object that correlates with the selected point on your 3D mesh

• Ctrl + left-click selects the point on the object. The projected cross-hair will turn green.

29

Finer Tuning

You may not be able to get it precise with your mouse, so once selected (green) you can use keyboard controls for finer movement…

• Ctrl + arrow keys for small movements

• Ctrl + Shift + arrow keys for even finer movement

30

Media/grid shows at 6 Vertices…

Repeat the process for each of 6 vertices on your model :

• 3D view > select vertex on mesh

• 2D view > select vertex on object (refine if necessary)

Selected vertices show as green dots on the mesh and the ‘active’ vertex show in red

Your loaded media or grid should show on the object after 6 vertices have been calibrated… 31

Grid vs Media

To help finish alignment, remember that you can:

1. Enable/disable your media in TIMELINE

2. Switch Grid between Off and 100% as required

1.

2.

32

Troubleshooting: Adjust the vertices

1. If your grid/media does not show after 6 point calibration, it may be because your points aren’t yet aligned correctly – in 2D view, use arrow keys and Ctrl+arrow/Shift+Ctrl+arrow for finer adjustment.

2. Add more points if required… often calibrating 7 or 8 points (or even more) may help to give a more perfect finish!

33

34

Troubleshooting: Image is still skewed

1. Check your Eyepoint settings are correct (see slide 19 to set)…

2. Adjust your Lens Modes…

35

Troubleshooting: Adjust Lens Modes

In the GUI:

Above your listed ‘3D Locations’ of the vertices there are several Lens Modes options:

• View your projection whilst checking and un-checking these settings

• It is not necessary to ‘know’ the best settings required here - a good adjustment can usually be made simply by using ‘trial and error’!

For this model, ticking Zero Tangential and K1, K2, K3 gives optimum effect

36

Troubleshooting: Apply Warp

• In TIMELINE drag and drop a Flat Geometry resource onto the layer that contains your media

• Click CONFIG > Warp > Surface and select the Flat Geometry that you have added

• Dots should appear on your GUI and on the object itself (though they may not be very visible on the object)

• Click on a dot (or use arrow keys) to make it active, then left-click + drag or Ctrl + arrow to warp that section of the projection

• Shift + Ctrl + arrow gives even finer warp

• H+ H- V+ V- allow you to add and remove control dots for further warp opportunities

37

Troubleshooting: more about Surface Warp

Surface Warp has two different interpolation methods. Note how the Spline method moves points in smooth arcs, warping a greater area than the linear method, which moves the point(s) in straight lines, within its own area, and does not affect the rest of the image. These two methods can have a very different affect on your media output so experiment to see which works the best in each scenario!

38

Troubleshooting: Using Point Distortion

• Point distortion may be applied to an individual or group of points (click and drag to select multiple points for adjustment).

• You can get some fun effects using this but generally you should use it sparingly to tweak small areas of your media to fit.

39

Add Blends

To blend off ‘sharp edges’ and merge projections smoothly (when using more than one projector), you can apply blends to specific areas.

In TIMELINE:

• Drag and drop a Flat Blend onto your timeline

• It will automatically be applied to the ‘BACKGROUND & DISPLAY LAYER’

40

Apply Blends

Under CONFIG > Blends there are lots of options to enable you to ‘smooth’ your projection(s)

• Click on the top set of icons to select left, top, right or bottom of the channel that you are viewing

• Select a type of blend that you want to apply from the blends panel below

• Move the scroll bar to adjust blending for each area/blend type

• View results on your projected media on the object as you adjust

• Experiment to see what works best for each setup. 41

Troubleshooting: Projected Mode Eyepoint!

Remember:

If you project ‘flat’ media, it will never be perfect from every angle, and quality will depend upon positioning of the projector(s). For full 360 projection, you need to use Textured mode!

42

View the media from the original viewpoint for best results!

43

What is Textured Mode?

• 3D media projection (up to 360)

• Media is UV mapped to the model

– ‘Flat’ media is designed specifically to ‘wrap around’ the object

• Can be viewed effectively from any angle

• Multiple projectors positioned around the object for full 3D effect

44

UV Mapping

The media is create to fit around the object (rather than project onto a single face) enabling Delta to project media around the whole object if required.

Image from: https://en.wikipedia.org/wiki/UV_mapping

45

Textured Mode Example

• Instead of projecting media onto a single face of the object, the 3D model is created with the UV mapped media ‘baked in’.

• Delta matches the 3D mesh with the UV mapped image, effectively ‘wrapping’ the projected media around the object (from strategically placed projectors).

• There is no specific eyepoint and the result can mean that there is full 360 coverage.

46

TEXTURED MODE IS CONFIGURED IN MUCH THE SAME WAY AS PROJECTED MODE WITH A FEW VARIATIONS TO CONSIDER…

1. Store mesh and media files on the server

2. Set up basic show in the same way as projected mode

3. Multiple projections need multiple channels (slide 45 for variation)

4. Set eyepoint settings (slide 46 for variation)

5. 3D Calibration – load mesh and calibrate points (see variations below for multiple channels/projector settings)

6. Refine with warp and blend functionality

7. Enjoy!

Variations to follow…

47

Channel Settings: Multiple Channels/Projections

• Ensure that your selected channels Active – usually one channel per projector

• Interactive view shows the frustums for each channel (projection cone)

• Calibration must be done for each channel to reflect the ACTUAL position of the projectors (Do not change settings here – they will generate automatically on calibration!)

48

Eyepoint Settings: Textured Mode

• Do not set up Eyepoint details in the MESH panel

• Click EYEPOINTS button to set to Textured Mode

• ID [1] will default to Projected: double-click to open Eyepoint Details screen

• Tick Use Texture Coordinates and Repeat Texture to ensure that the media will fit around the 3D object with no gaps

49

3D Calibration 3D calibration must be set for EVERY channel (6+ points for each)

Each calibration may be set up from a different viewpoint to cater for the projection positioning

50

Full 3D Media Viewing

Once complete, textured mode will enable you to view media from all viewpoints, potentially around the whole object, with equal clarity.