commonsense: software for displaying full motion … · commonsense: software for displaying full...

14
CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense & Aerospace Abstract – This paper presents an overview of Barco’s CommonSENSE software library which provides GPU-accelerated video capabilities for operator positions in a networked environment. CommonSENSE performs optimized, high performance video manipulation and presentation operations and simplifies the rapid development of visual application programs involving Full Motion Video. This software is offered as an option for multiple Barco rugged computing and smart display products and establishes a common Application Programming Interface for multiple types of equipment. Following a brief discussion of networked visualization concepts, an overview of CommonSENSE benefits, architecture, and capabilities is presented. Typical applications are also summarized in terms of related working positions for Unmanned Vehicle Systems, Naval C4ISR, and Army ground vehicles. INTRODUCTION For mission-critical systems, operators and decision-makers at multiple working positions must have quick and easy access to visual information from a rapidly increasing number of sources. Command and control information, computer generated imagery, and sensor feeds all provide critical input during surveillance/recognizance and combat management decision loops. Goals for reduced crew sizes, multi-mission flexibility, and increased interoperability between multiple systems also place growing demands for Processing, Exploitation, and Dissemination (PED) of information involving Full Motion Video (FMV). Networked visualization systems provide the means to effectively distribute and present visual information to people. These systems include displays, graphics, video, and radar processing capabilities to create situation awareness and understanding from complex data. The following paragraphs provide an overview of Barco’s CommonSENSE (CS) software and discuss the application and benefits for mission-critical networked visualization systems. System designers and visual application developers can utilize Barco’s smart displays with CS software to easily and rapidly integrate reliable, multi-channel, full frame rate FMV into a variety of operator working positions. NETWORKED VISUALIZATION CONCEPT Networked Approach In a networked visualization architecture, all information sources and user end points interface to a common network infrastructure. Therefore, all sources can be logically connected to all destinations and the functional system configuration is defined by system management software. For an ideal system, access to all data sources could be readily available at all working positions. In reality, varying security levels, resource limitations, and deterministic performance needs may induce some level of segregation involving multiple networks. However, controlled connections between multiple sub-system networks can be achieved using gateways.

Upload: dangnga

Post on 03-Apr-2018

222 views

Category:

Documents


4 download

TRANSCRIPT

Page 1: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

CommonSENSE:Software for displaying Full Motion Video

for mission-critical C4ISR working positions

Jeff Malacarne Barco Defense & Aerospace

Abstract – This paper presents an overview of Barco’s CommonSENSE software library which provides GPU-accelerated video capabilities for operator positions in a networked environment. CommonSENSE performs optimized, high performance video manipulation and presentation operations and simplifies the rapid development of visual application programs involving Full Motion Video. This software is offered as an option for multiple Barco rugged computing and smart display products and establishes a common Application Programming Interface for multiple types of equipment. Following a brief discussion of networked visualization concepts, an overview of CommonSENSE benefits, architecture, and capabilities is presented. Typical applications are also summarized in terms of related working positions for Unmanned Vehicle Systems, Naval C4ISR, and Army ground vehicles.

INTRODUCTION

For mission-critical systems, operators and decision-makers at multiple working positions must have quick and easy access to visual information from a rapidly increasing number of sources. Command and control information, computer generated imagery, and sensor feeds all provide critical input during surveillance/recognizance and combat management decision loops. Goals for reduced crew sizes, multi-mission flexibility, and increased interoperability between multiple systems also place growing demands for Processing, Exploitation, and Dissemination (PED) of information involving Full Motion Video (FMV).

Networked visualization systems provide the means to effectively distribute and present visual information to people. These systems include displays, graphics, video, and radar processing capabilities to create situation awareness and understanding from complex data. The following paragraphs provide an overview of Barco’s CommonSENSE (CS) software and discuss the application and benefits for mission-critical networked visualization systems. System designers and visual application developers can utilize Barco’s smart displays with CS software to easily and rapidly integrate reliable, multi-channel, full frame rate FMV into a variety of operator working positions.

NETWORKED VISUALIZATION CONCEPT

Networked ApproachIn a networked visualization architecture, all information sources and user end points interface to a common network infrastructure. Therefore, all sources can be logically connected to all destinations and the functional system configuration is defined by system management software. For an ideal system, access to all data sources could be readily available at all working positions. In reality, varying security levels, resource limitations, and deterministic performance needs may induce some level of segregation involving multiple networks. However, controlled connections between multiple sub-system networks can be achieved using gateways.

Page 2: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

A networked Service-Oriented Architecture enables the deployment of resources where they are needed most. For example, centralized servers can provide powerful on-demand execution of compute intensive algorithms and data processing operations. Live sensor streams can be multicast directly to all subscribing working positions as well as to server-based image processing services for complex automated analysis. Client working positions with visualization processing and presentation capabilities can be utilized to provide the smooth user interaction with graphics applications and sensor streams.

Figure 1 – Manned working positions in a networked visualization environment provide user access and interaction with a variety of sensor and computing services.

Networked Visualization ClientsClients in a networked visualization environment include several types of individual working positions as well as collaborative displays for sharing information among multiple people. For mission-critical C4ISR systems involving FMV, accelerated graphics and video processing at the client can greatly enhance system interaction and performance. Utilizing visualization horsepower at the client focuses visual computing operations at the display where the output is needed. For many system configurations, this architecture provides the highest visualization performance, ensuring that time-critical operation and interactivity are maintained.

Barco provides several products for implementing Networked Visualization Clients including Display Processor Modules (DPM), SmartView intelligent displays, and Vista Consoles. Advanced computing, graphics generation, and video processing capabilities are provided in or near the display to create workstation or thin-client working positions with powerful visualization performance. The latest computing elements such as Intel processors and Nvidia graphics modules are offered at various levels of ruggedization.

Figure 2 – Barco’s display processing equipment provides powerful visual computing capabilities for harsh environments.

Data/SensorProcessingServices

Video

Radar &other sensors

CollaborativeDisplay

Mannedworking positions

Link

Gateway

OtherServices

Data/SensorProcessingServices

Video

Radar &other sensors

CollaborativeDisplay

Mannedworking positions

Link

Gateway

OtherServices

Display ProcessingModules

IntegratedSmart

Displays

DPM-2

DPM-3

SmartView

Consoles

Vista 4500

Display ProcessingModules

IntegratedSmart

Displays

DPM-2

DPM-3

SmartView

Consoles

Vista 4500

Page 3: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Networked Visualization Clients for C4ISR systems must integrate multiple HD video channels and computer graphics while providing mission-critical performance and reliability. CommonSENSE software provides this capability for Barco’s rugged computing and smart display products without requiring any special or non-standard hardware.

CommonSENSE OVERVIEW

CommonSENSE (CS) software provides GPU-accelerated FMV manipulation and presentation for Networked Visualization Clients. Supported capabilities include video windowing, integration with graphics, and image processing, as well as screen distribution and recording. CS software resides at each client position and provides an integrated and uniform mechanism for integrating FMV and screen image operations into visual application programs.

Customer-developed application programs specify video processing and display formatting operations via a network-transparent C++ Application Programming Interface (API). Control using a language-independent browser-oriented interface is also supported. CS provides services for decoding camera inputs and network streams, accessing recording files, applying visual processing operations, and generating/capturing the resulting on-screen display. Standard components and interfaces are employed within the CS software to provide a variety of capabilities that are accessed via a simplified higher level API. The latest standard GPU technology provides the acceleration needed for smooth and reliable operation.

Figure 3 – CommonSENSE provides GPU-accelerated FMV visualization capabilities based on open standards for mission-critical applications.

BenefitsCommonSENSE offers the following benefits for integrating Full Motion Video into operational display system applications:

Scree

nSha

ring

&Rec

ording

Image

Pro

cessin

g

Inte

gra

tion

with

Gra

phic

s

Video

Window

ing

FileMgmt

DisplayOutputNet

workin

g

Visual

Proc

essin

g

Vid

eo

Input

Codecs

Unmanned

Vehicle Systems

NavalC4ISR

Ground VehicleSituation Awareness

Applications

GPUProcessingH

.264/

RTP

OpenGLLi

nux

&W

indows

Ethernet

CUDA

DS/

V4L2

API

Services

Standards

CapabilitiesCommonSENSE

Hardware

C+

+Lib

rary

url/h

ttp

Scree

nSha

ring

&Rec

ording

Image

Pro

cessin

g

Inte

gra

tion

with

Gra

phic

s

Video

Window

ing

FileMgmt

DisplayOutputNet

workin

g

Visual

Proc

essin

g

Vid

eo

Input

Codecs

Unmanned

Vehicle Systems

NavalC4ISR

Ground VehicleSituation Awareness

Applications

GPUProcessingH

.264/

RTP

OpenGLLi

nux

&W

indows

Ethernet

CUDA

DS/

V4L2

H.2

64/

RTP

OpenGLLi

nux

&W

indows

Ethernet

CUDA

DS/

V4L2

API

Services

Standards

CapabilitiesCommonSENSE

Hardware

C+

+Lib

rary

url/h

ttp

Page 4: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Shortens time-to-market for application development – CS provides an extensible set of controls for on-screen video operations that enable rapid development of a wide variety of visualization scenarios. Complex video and imaging operations are reduced to a handful of function calls that abstract low level implementation details to a generic set of functional controls. All video data is handled by CS and only basic controls need to be specified.

Provides optimized performance for video operations – Customer software programs can leverage CS’s off-the-shelf performance to achieve reliable, multi-channel, full frame rate video. Video decoding or processing operations are accelerated using Barco’s ruggedized GPUs. The latest GPU technology is used to provide standard video decoding or programmable processing of multiple pixels in parallel.

Minimizes the impact of video on other computing operations – Video operations are also tuned to minimize the utilization of computing platform CPU resources. In several cases, little or no CPU is required for displaying multiple video channels. Executing video decoding or processing operations on the GPU also serves to offload the CPU and provide a more balanced use of processing resources. This leaves more computing power for application level data processing and communications as well as spare compute resources for future evolution and growth.

Enables seamless hardware evolution without changes to application software –CS supports the insertion of future computing and visualization technology without requiring changes to application software programs developed today. Generic video controls are utilized, enabling the seamless technology refresh as new platforms, interfaces, and components become available.

Provides a common software interface (API) for multiple networked Barco products – Application programs can use a common interface for controlling multiple types of Networked Visualization Clients from Barco. Systems involving different products and configurations can use a uniform mechanism to perform dynamic control operations as well as setup pre-defined screen layouts. These layouts can then be invoked and controlled at various working positions independent of the type of client equipment.

ArchitectureCS software runs on each Networked Visualization Client to provide FMV visualization operations for that working position under the control of customer developed application programs. In general, the CS API provides controls for video operations at a similar software level as common graphics APIs such as X Windows and OpenGL.

Figure 4 – Generalized CommonSENSE Architecture

GPUVideo Input

Graphics Library/Service-X, DirectDraw

-OpenGL-CUDA/OpenCL

Application Programs

NetworkStreams

Cameras Displays

CommonSENSEVideo Library/Service

CommonSENSEVideo Library/Service

CommonSENSEVideo Library/Service

CommonSENSEVideo Library/Service

Page 5: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Two major application models are supported: 1. Networked service for multiple application programs – Much like the X Window System

model (xserver/xlib), an independent CS executable (csServer) can perform all video operations as directed by multiple networked application programs. Video channels are handled end-to-end by the csServer while controls for input selection, processing operations, and on-screen formatting are specified by application software. Multiple control applications can be run locally or anywhere on the network in any combination. Control over individual video channels can be isolated or shared among multiple applications. Socket-based communication to the csServer is provided by a thin C++ library (csLibRemote) which is linked with the customer application software.

2. Video library for individual application programs – CS capabilities (csLibLocal) can be directly linked with an application program to provide tightly integrated video and graphics operations. Video channels can be handled end-to-end by the library while controls for input selection, processing operations, and on-screen formatting are specified by application software. Additionally, directly linking with csLibLocal enables high performance sharing of video textures and graphics contexts. This greatly simplifies application development for use-cases including the following:

Integration with interactive annotation graphics (e.g. telestrator-style highlighting).

Integration of multiple video channels into 3D space along with maps/terrain to provide a geospatial reference for the sensor field-of-view.

The development of a single full screen application that integrates all on-screen graphics and video for achieving real-time performance when using a non-real-time environment (e.g. Linux or WindowsXPe).

Figure 5 – Running CommonSENSE as an independent executable provides network-transparent control of video processing operations from multiple application programs.

Figure 6 – Using CommonSENSE as a video library provides direct access to video content and integration with dynamic graphics processing.

Application Program

GPUVideo Input

Graphics Library/Service-X, DirectDraw

-OpenGL-CUDA/OpenCL

NetworkStreams

Cameras Displays

CommonSENSEVideo Library-csLibLocal

CommonSENSEVideo Library(csLibLocal)

CommonSENSEVideo Library-csLibLocal

CommonSENSEVideo Library(csLibLocal)

Control & Video

Application ProgramsApplication Programs

Application Programs

xlib, glxlib, othercsLibRemote

GPUVideo Input

Graphics Library/Service-X, DirectDraw

-OpenGL-CUDA/OpenCL

NetworkStreams

Cameras Displays

Control

CommonSENSEVideo Library/Service

CommonSENSEVideo Service

(csServer)

csLibRemote

Application ProgramsApplication Programs

Application Programs

xlib, glxlib, othercsLibRemote

GPUVideo Input

Graphics Library/Service-X, DirectDraw

-OpenGL-CUDA/OpenCL

NetworkStreams

Cameras Displays

Control

CommonSENSEVideo Library/Service

CommonSENSEVideo Service

(csServer)

CommonSENSEVideo Library/Service

CommonSENSEVideo Service

(csServer)

csLibRemote

Page 6: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Both models use an identical C++ API for all functions and both can run simultaneously on a single Networked Visualization Client. When running CS as an independent executable (csServer), a browser-oriented url/http interface is also supported for language-independent control without the need for Barco libraries. C++ programming samples and a browser-based utility are provided to simplify evaluation and training. Additionally, scenarios can be pre-defined and saved, then loaded and manipulated during normal operation.

Like DirectShow, gstreamer, and other available video frameworks, CS provides a means for adding video capabilities into visual application programs. Unlike these frameworks, CS offers a fully integrated capability with verified reliability and performance. These commercial software frameworks and components are utilized within CS where beneficial.

CommonSENSE CAPABILITIES

CommonSENSE provides application controls for video windowing, integration with graphics, image processing, and screen image operations.

Video WindowingControls for selecting video inputs and formatting onto a display screen are supported.

Selecting input sources – Multiple directly connected cameras and network streams with different resolutions and formats can be selected for processing and display. The types and number of inputs that can be processed simultaneously depends on the equipment configuration.

Adjusting camera inputs – Controls for adjusting camera video inputs are provided including brightness contrast, hue, saturation, and multiple de-interlacing modes.

Positioning video on the screen – Program controls for positioning and sizing the video on the screen are provided. Each video input can be displayed as a user controllable video window with standard window dressing or as a program controlled “picture-in-picture” overlay with no window system dressing or interactive user controls. Program control of video window stacking order is also supported to ensure critical sources remain visible on-screen.

Specifying video content to be displayed - Flexible controls for mapping the input video content into the on-screen window are provided. Two methods are supported:

o Select input Region-of-Interest (ROI) – A rectangular region of the input video content can be selected for display in the output window. The corners of the selected region are mapped to the corners of the output window. Manipulating the output window will effectively scale the selected region to fit the output window size. The selected input region can be rotated by 90, 180, or 270 degrees to accommodate various camera orientations. Mirroring is also supported for rear-view applications.

o Specify pan/zoom/rotate (TSR) - Input video can be mapped into the output window using two-dimensional translate (pan), scale (zoom), and rotation parameters. The translated center of the input video remains positioned in the center of the output window and manipulating the size of the output window will not change the specified scaling. Rotations at any angle around the center of the input video are supported for aligning video from skewed camera mounts. Mirroring is also supported for rear-view applications.

Page 7: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Figure 7 – Multiple video windows can be displayed as overlays or managed windows, often with little or no CPU utilization

Figure 8 – Application programs can construct complex visualization scenarios with a few CommonSENSE function calls.

“Picture-in-picture” overlay

Managed window

NOTE:CS screen snapshotprocessing an HD H.264 video streaminto two windows with <1% CPU utilization

“Picture-in-picture” overlay

Managed window

NOTE:CS screen snapshotprocessing an HD H.264 video streaminto two windows with <1% CPU utilization

Page 8: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Integration with GraphicsThe purpose of FMV for C4ISR applications is to provide information, not just high quality imagery. Communicating video-related information to people is greatly enhanced by the addition of accompanying references, tags, and highlights. This is often achieved by rendering associated metadata over video or supporting interactive annotation for highlighting content of interest. Additionally, video can be mapped onto 3D terrain or integrated with synthetic vision graphics to increase understanding of the spatial orientation of the sensor’s field-of-view.

Mixing video and graphics in a common window on the screen involves combining and synchronizing image content from multiple sources into a singe output. Video content is constantly changing at a periodic rate. Graphics content changes at various rates, depending on the type and purpose. Generating a common output image (without special hardware or system dependencies) requires that graphics be rendered on top of each new video frame.

CS supports multiple mechanisms for mixing application-generated graphics with CS-generated video:

Let CS do it – CS can blend user rendered graphics images, such as text strings or icons, onto CS video windows. Application programs can load, position, and enable/disable multiple images and CS will take care of rendering and blending graphics with each new frame of video. This mechanism is designed to support metadata overlays with a mix of static and dynamically changing text or other 2D graphics content. Images are generated by the application program using any off-screen rendering system available (including X Windows or DirectDraw), then passed to CS for layering over video. Updates are loaded or selected by the application program only when the overlaid information needs to change.

Let the application program do it – CS can provide access to video content directly to the application program in the form of OpenGL texture handles. Application programs can then apply the video textures as desired within the GPU without any need to read back the video content. This provides a high performance path without any special video processing required on the CPU. Presentation of video onto the screen must be performed by the application program within its own output window. This method is designed to support integration with complex graphics such as rendering video over 3D terrain. If desired, the application program can read back the texture content from the GPU for additional processing.

Work together – Application programs can synchronize their dynamic graphics rendering with the CS video pipeline using a shared OpenGL context. The application program can control the synchronization with graphics updates and can directly render OpenGL graphics over each CS video frame. CS presents the combined output to the screen within a CS video window.

Figure 9 – Examples of video integrated with graphics - Video integration with dynamic graphics applications is simplified using CommonSENSE.

Metadata graphics over video

Interactive graphicsannotation

3D terrain textured with video

This one!This one!This one!This one!

XyAltRsvpalt

Signon

XXXXXX

ASDFDGHtJHFasdf

AsdfasdfAdsfasdfasdfasdfAsdfasdf

asd

XXXXXX

XyAltRsvpalt

Signon

XXXXXX

ASDFDGHtJHFasdf

AsdfasdfAdsfasdfasdfasdfAsdfasdf

asd

XXXXXX

Page 9: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Image ProcessingCS supports GPU-accelerated imaging processing for enhancing and combining video sources. A small set of baseline capabilities and interfaces are available off-the-shelf as an example of the performance achievable for typical operations. Custom operations can be developed rapidly on a project basis to address system-specific needs. In general, any video transformation operation (with video in and video out) can be added to the CS video pipeline as a GPU program, often with little or no added latency or CPU utilization. Many high performance imaging operations are possible using CUDA/OpenCL such as contrast enhancement, noise reduction and others. For more complex image analysis capabilities, CS can share video content with 3rd party application software via OpenGL texture handles.

Baseline capabilities to support image processing operations include:

Image enhancement – Imaging operations such as stabilization can be applied to improve video quality. Sensor or system specific enhancements can be developed on a project basis.

Fusion of multiple sensors – Video from different types of sensors observing the same view, such as daylight and infrared cameras, can be overlaid and fused to combine the content into one image. Capabilities for aligning multiple inputs are supported, such as lens distortion correction, along with sample blend modes including alpha blending and red/blue coloring for monochrome inputs from infrared cameras. Sensor or system specific blending operations can be developed on a project basis.

Integration with imaging applications – CS video can be shared with video analysis applications that provide capabilities such as intelligent motion detection, target tracking/or other operator aids for increasing image understanding. Application programs can also use CS to highlight results as graphics over video within a CS video window.

Figure 10 – Image processing capabilities increase the understanding of sensor inputs.

Screen OperationsScreen content from any position on the network can be captured and streamed for remote display or recording at other working positions. Unlike many commercial software-based remote desktop or recording implementations, CS provides the performance needed for distributing and recording screens involving dynamic 3D graphics and live sensor imagery such as FMV and radar. GPU-accelerated processing is used for screen change detection and compression. Once captured, screen content can be recorded locally or can be streamed for remote live display and recording.

Day/nightsensorfusion

Application generatedmotion detectionhighlights over videoStabilization

Day/nightsensorfusion

Application generatedmotion detectionhighlights over videoStabilization

Page 10: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

Live remote display - Any working position screen can be streamed to any networked working position for collaboration using visual information. The entire screen or multiple regions/windows can be streamed simultaneously. Multicasting can be used to distribute contents to multiple positions simultaneously. This supports sharing an individual screen on a large area display for viewing by a group as well as interaction between individual working positions. For example, when combined with CS’s graphics integration capabilities, text/graphical annotations can be shared interactively between multiple positions to visually communicate aspects of interest.

Recording – Recording of dynamic screen content can be done on the local working position or remotely on any supported networked equipment. CS execution on networked storage devices for recording is also supported on a project-basis.

Playback - Recorded screen content can be played and displayed locally or remotely at any position. Playback streams can be multicast to view recordings at multiple positions simultaneously. Playback controls include start/stop positions, play speed, and looping.

Barco has integrated CS recording and playback capabilities into the MR-300 Mission Recorder product to provide a complete screen recording solution with a higher level recording API and sample GUI.

Figure 11 – Dynamic screen content can be shared and recorded.

Fullscreensharing

Individualwindowsharing

Remote Collaborative Displays

Other Working Positions

Source Working Position

NetworkedScreen

Recording & Playback

Page 11: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

APPLICATIONS Barco’s Networked Visualization Client products running CS software can be deployed into a variety of manned positions for mission-critical systems. Related positions for unmanned vehicle systems, Naval C4ISR systems, and Army ground vehicle working are summarized in the following paragraphs.

WORKING POSITIONS for UNMANNED VEHICLE SYSTEMS (UVS) Visualization is a critical component for UVS Processing, Exploitation, and Dissemination (PED) operations. Typical environments that utilize visualization clients include:

Mobile Shelters – vehicle mounted Ground Control Stations involving multiple manned working positions for flight and payload control with full system-level communication. For visualization, relatively powerful semi-rugged servers, workstations, and thin-clients are used, often with multiple large high-resolution displays (e.g. 30” with 2560x1600 pixels).

Dismounted Terminals – portable Ground Control Stations with flight and payload control, but with limited communications capability. Processing and display equipment must provide useful visualization capabilities within size, weight, and power (SWaP) constraints. Fully rugged equipment is required to withstand exposure to the elements and rough handling.

Transportable Operations – truck and tent-based command posts used to guide tactical operations during a mission. Semi-rugged transportable computers and displays are used, often in conjunction with larger screen collaborative viewing sub-systems.

Command Centers – permanent facilities providing centralized decision making. Commercial compute servers, workstations, and displays are used with significant communications connectivity. Large screen displays and walls are used for center-wide collaborative viewing.

Analyst Workstations – office environment working positions for detailed exploitation analysis of near-real-time or recorded video. Professional grade computing and display systems are deployed with ultra-high resolution capability for maximum viewing fidelity.

Each environment and use-case involves varying needs for visualization processing and display. However, compatible video feeds and recordings may need to be viewed and shared between any of these client positions. In addition to viewing raw or processed video sources, each position can add tags or other annotation information that is communicated along with the video via metadata. This is an important capability for communicating critical information in near-real-time or enabling later searching for recorded segments of interest.

Figure 12 – Networked visualization positions in a generalized UVS environment.

Command Centers

Transportable Ops Centers

Mobile Shelters

DismountedTerminals

LAN/WAN

Analyst Workstations

Page 12: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

WORKING POSITIONS for NAVAL C4ISR

The primary function for a Naval visualization system is to deliver a visual presentation of all operational data used to describe the current tactical and strategic situation when and where it can be used most effectively. Therefore, effective presentation capabilities at all related manned positions play a critical role in mission success. Networked Visualization Client equipment at these positions implements the functions necessary for generating or accessing distributed graphics imagery as well as receiving and presenting video and other sensor sources. Typical working positions that utilize networked visualization for naval systems include:

Combat Information Center (CIC) positions – The CIC is the ship’s central command center executing the collection, processing, display, evaluation, and rapid dissemination of tactical information and intelligence. This involves various stations for command & control, communications, surveillance/reconnaissance, weapon system control, and mission planning.

Platform management stations – Positions for monitoring, alarm management, and control for the ship’s main engines and other on-board systems, as well as damage control functions.

Bridge – Positions for ship command and navigation.

Auxiliary positions - Briefing Rooms – A collaborative center with a large area display for communicating

situational information to a group. - Officer Cabins – Remote informational positions with standard PC level equipment. - Portable Terminals – Portable system access points using laptops or tablets, but with

limited communications capability.

Each working position and use-case involves varying needs for visualization processing and display. Viewing and sharing graphics, video feeds, and recordings between positions can increase situation awareness and mission effectiveness.

Figure 13 – All manned positions involving computer displays are candidates for networked visualization processing.

Page 13: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

WORKING POSITIONS for GROUND VEHICLES

Inside armored vehicles, access to information and correct interpretation and evaluation is crucial. With an advanced situation awareness capability providing real-time, all-round perception of the local environment in mission-critical situations, decision-makers have the information they need to anticipate risks and to act accordingly. Typical vehicle and support positions that utilize networked visualization for situation awareness include:

Driver – Driver access to real-time graphical vehicle instrumentation and platform information is required. Additionally, video feeds for driver vision are also required when driving blind. Low latency display of day and night sensor information from multiple cameras is mandatory to drive in all possible situations

Commander – Display of battle management information is critical at Commander’s positions to direct vehicle operations. Video feeds and access to screens from other positions is also beneficial for additional situation awareness.

Gunner – Remote Weapon Systems are utilized for Gunner positions with strict requirements for low latency video feeds.

Auxiliary positions - Observers and other crew positions – Driving and personnel compartments often

carry additional personnel with needs/benefits for accessing visual information such as monitoring surveillance cameras.

- Mission planning and review operations – Support personnel at remote facilities can be involved in collaborative planning, training, and review of recorded missions.

When utilizing smart Multi-Function Displays, viewing and sharing video feeds, sharing display content (and control of positions), recording display content for training, and mission debrief are all possible using common equipment. Core vehicle positions require low-latency access to sensor information to provide real-time interaction with steering, weapons, and decision making systems. Auxiliary working positions can tolerate some level of delay from the live sensor, but may need to integrate or fuse the vehicle data with other information sources.

Figure 14 – Networked visualization positions in a ground vehicle environment.

Gunner

Commander

Driver

Missionplanning& review

Other crew positions

Hard-real-time network

Near-real-time& recorded

access

ww

w.d

vid

shub.n

et

Page 14: CommonSENSE: Software for displaying Full Motion … · CommonSENSE: Software for displaying Full Motion Video for mission-critical C4ISR working positions Jeff Malacarne Barco Defense

CONCLUSION

Networked visualization capabilities provide significant benefits for enhancing C4ISR mission effectiveness. Barco’s Network Visualization Client equipment provides the performance and reliability required for mission-critical operations. For these rugged computing and smart display platforms, CommonSENSE software simplifies the rapid development of high performance visualization applications. Systems involving Full Motion Video, integration with graphics, image processing, screen sharing, and screen recording can utilize CommonSENSE to reduce time-to-market and get the best performance possible from standard CPU and GPU hardware.