denodo platform 6.0

Data Sheet

Denodo Platform 6.0

With the advent of big data and the proliferation of multiple information channels, organizations must store, discover, access and share massive volumes of traditional and new data sources. Data virtualization is a modern approach to data integration that transcends the limitations of traditional techniques. Denodo is the leader in data virtualization, providing agile, high performance data integration and data abstraction across the broadest range of enterprise, cloud, big data and unstructured sources as real-time data services at half the cost of traditional approaches.

DS-DenodoPlatform6.0-02 2Copyright© 2016, Denodo Technologies

What’s New in Denodo Platform 6.0?Denodo Platform 6.0 delivers breakthrough performance for big data, logical data warehouses and operational scenarios with the Dynamic Query Optimizer. The Denodo Platform now accelerates data virtualization adoption with cloud support, which enables business users to leverage self-service data discovery and exploration.

Dynamic Query OptimizerThis feature dynamically determines the lowest-cost query execution plan based on statistics. It is

the only data virtualization performance optimization engine in the industry that takes into account all the special characteristics of big data sources (e.g., the number of processing units or partitions), it can handle incremental queries and it enables connectivity to the broadest array of big data sources, including Redshift, Impala and Spark.

Denodo Platform for AWS The Denodo Platform 6.0 aims to accelerate the adoption of data virtualization as-a-Service with the introduction of Denodo Platform for AWS, another first to the industry which is ready-to-use and available on AWS Marketplace. Denodo Platform for AWS offers elastic infrastructure, flexible options such as rent-by-the-hour and a wide range of capacity choices.

Self-Service Data Discovery and ExplorationThe Denodo Platform 6.0 is built to expedite the availability, access and use of data for business users with the self-service data discovery and exploration functionality that comes with google-like search capabilities for data and metadata.

It works with all types of data sources, including big data sources, and it offers easy-to-use interfaces for browsing data and metadata as well as viewing data lineage. Users can also create ad-hoc queries using the new Advanced Query Wizard and save them as views in the Denodo Platform Server, and they can now view the graphical representation of entities and relationships.

Big Data ConnectivityThe Denodo Platform offers big data and cloud databases connectivity including Redshift, Apache Spark, HP Vertica and Impala.

Unified Security and GovernanceThe Denodo Platform 6.0 offers the following enhanced features to enforce security and governance policies.

• Kerberos southbound support for databases and Web Services• Kerberos pass-through support • An API for accessing view dependencies information and data lineage information

Resource ManagementThe Denodo Platform 6.0 offers the Resource Manager, which is a new UI based tool for enforcing workload management policies in environments with multiple consuming applications that have varying workloads and usage patterns.


The Denodo Platform for Data Virtualization Delivers the Following Fundamental Capabilities

Universal Connectivity – Data Access: The Denodo Platform allows you to connect to any data source including Web, semi-structured and unstructured, in any format or location. The Denodo platform offers high-performance optimized adapters for all the main sources as well as extension points for custom wrappers to unique data sources.

Universal Connectivity – Data Delivery: The Denodo Platform allows you to publish any combined information as reusable data services in multiple formats (SQL through JDBC and ODBC, SOAP WS, REST WS, XML, JSON, XHTML, etc.) and supported hybrid delivery modes (real-time, cache, batch, etc.) to consuming applications.

Optimized Performance: The performance features of the Denodo Platform include the Dynamic Query Optimizer. By always enabling the lowest-cost query execution plan, this feature significantly reduces the amount of data transferred from the data sources through the network, with the associated reduction in execution time. This feature provides greater benefits in scenarios such as logical data warehouse architectures. The Optimizer enables the lowest-cost execution plan using multiple optimization features including the following ones that are new to Denodo Platform 6.0:

• Full aggregation pushdown• Partial aggregation pushdown• On-the-fly data movement• Leveraging dimension tables replicated in several data sources to maximize query pushdown

Intelligent Caching: The Denodo Platform offers an advanced caching system that allows both a full-view materialization and query-by-query caching. The cache is configurable on a per-view basis, enabling hybrid query executions that are part real-time and part cached, so the cache can be applied when and where it is needed. The cache system supports manual, scheduled and triggered updates with full or incremental loads, for the flexibility to adapt to any use case. Incremental queries allow cached data to be combined with the most recent data from the data source to return the most up-to-date results with minimal impact on the data source.

Data Integration: The Denodo Platform flexibly automates web integration processes that are modeled using a library of pre-built templates and components for workflow, navigation and extraction as well as the structuring of Web, semi-structured and unstructured data.

Unified Virtual Data Layer: This capability provides an integrated modeling and execution environment that enables users to build powerful transformations and relationships. Because the Unified Virtual Data Layer uses common metadata and semantics, it helps to normalize, transform, improve quality and relate data across heterogeneous source types. The extended relational data model allows disparate data types to be represented natively in the virtual layer, minimizing effort.

Advanced Security: The Denodo Platform offers policy-driven row-based and column-based security management features that provide users with controlled access to information. Some of the salient features include role-based security access, schema wide and data-specific permissions, support for security protocols, including single-sign-on using Kerberos in both northbound and southbound connections, and pass-through authentication.

Unified Data and Metadata Governance: This feature offers a single, enterprise-wide entry point for data and metadata management, security, audit, logging and monitoring, enabled through built-in tools and instrumentation, which can also integrate with external data management tools.


DISPARATE DATAAny Source,any Format

BUSINESS SOLUTIONSAccess Information-as-a-Service

DENODO PLATFORMRight Information at the Right Time

Multiple Protocols,Formats

Query, Search,Browse

Request/Reply,Event Driven

SecureDelivery

SQL,MDX

WebServices

Big DataAPIs

Web Automationand Indexing

DOC

W

DATA VIRTUALIZATION

Databases & Warehouses, Cloud/Saas Applications, Big Data, NoSQL, Web, XML, Excel ,PDF, Word...

CONNECTNormalized

Views of Disparate Data

COMBINETransform,

Improve Quality,Integrate

PUBLISHReal Time

(Right Time)Data Services

DISPARATE DATA SOURCESMore Structured Less Structured

DATA CONSUMERSAnalytical Operational

Enterprise Applications, Reporting, BI, Portals, ESB, Mobile, Web, Users

Performance

Cache

Hybrid Integration

AgileDevelopment

Governance & Metadata

Security

ResourceManagement

Self-serviceInformation

Denodo Platform Architecture

Self-Service Data Discovery and Exploration: The Denodo Platform is a feature-rich, single product offering with user-friendly, self-service features that make business-related data accessible and extensively available to business and technology users alike. The Information Self-Service Tool enables business users to easily search and discover data and metadata using a Web interface. It provides browser-based search functionality that enables users to find critical data as well the lineage of the data.

Agile Development: The Denodo Platform offers rapid ways to deliver pervasive, self-service data services using graphical, wizard driven UI and discovery tools. The platform hides the complexity to application developers and business users, decouples consuming applications and data sources and enables the easy creation, extension and use of data services.

Hybrid Integration Mode: The Denodo Platform supports complex, hybrid integration processes, such as those that integrate ETL within a broader data virtualization approach, to enable the delivery of reliable, high performance virtual data services through the balanced orchestration of real-time, cached, scheduled batch or hybrid execution modes.


Feature Description

Adapters Point-and-click adapters to enterprise, semi-structured and unstructured data in any format or location. High-performance optimized adapters for all the main sources. Bi-directional read/write.

Databases • Relational databases such as Oracle, HP Vertica, DB2, Sybase, MS SQL Server, MySQL, PostgreSQL, Informix, MSAccess and Apache Derby, including graphical introspection of tables, views and stored procedures.

• In-memory relational databases such as SAP HANA and Oracle TimesTen.• Parallel databases and appliances such as Teradata, Netezza, Vertica, Oracle

Exadata, Sybase IQ, ParAccel and Greenplum. • Cloud data warehouse sources (‘DWaaS’) including Amazon Redshift and

Snowflake.

Big Data/NoSQL • Hadoop (Denodo is certified with multiple vendors such as Cloudera, Hortonworks, etc., included Kerberos-secured clusters, and multiple technologies such as Hive, Impala, sequence/maps/key-value/Avro files, HDFS, Map/Reduce, HBase), Mongo DB, CouchDB, Neo4J and MarkLogic.

• Apache Spark – access data in Spark data frames via SQL.

Web Automation Sophisticated tools to expose Web, semi-structured and unstructured data as virtual relational data/service.

Multi-Dimensional Sources Multidimensional OLAP engines such as SAP BW, MS SQL Server Analysis Services, Mondrian and Essbase.

Web Services SOAP / REST Web Services and data feeds, including support for XML, RSS, ATOM, JSON and CSV formats.

Flat and Binary Files CSV, pipe-delimited, MS Excel (xls & xlsx), MS Access, XML and JSON. Regular expression-parsed flat files. All files can be locally accessible or in remote filesystems, through FTP/ SFTP/FTPS, and in clear, zipped and/or encrypted format.

Active Directory Connect and introspect LDAP and Active Directory services as source data (apart from security access).

Cloud, SaaS, Web Sources with Simplified OAuth Security

Salesforce, Google, Amazon, LinkedIn, Facebook, Twitter via APIs with simplified OAuth integration (1.0, 1.0a and 2.0); Any website, form, WebApp via browser auto-mation.

Connect


Feature Description

JMS Queues Consume data from JMS queues by subscribing to the queue. JMS implementations include MQSeries, SonicMQ and ActiveMQ.

Semantic Repositories Semantic repositories in Triple Stores / RDF accessed through SPARQL endpoints.

Mainframe Mainframe/legacy connectivity through third-party adapters.

Packaged Applications Consume information from popular packaged applications such as SAP ERP/ECC, Oracle E-Business Suite or Siebel.

SDK for Custom Extension • Unlimited extensibility through a custom connector API that allows users to create connections to other information systems not accessible out-of-the-box.

• Custom data sources (wrappers), stored procedures, custom functions (both for VDP and ITPilot) and custom policies.

Combine

Feature Description

Canonical Business Views Transform, clean, combine disparate information into virtual canonical business views.

Extended Relational Model Natively represents and seamlessly combines relational, hierarchical, NoSQL and semantic representations of data into abstracted relational model; takes into account source capabilities and constraints for query optimization.

Web Integration • Automates extraction and integration of less-structured data from Web sites, forms, applications, PDF and MS Word.

• Automatic Web Navigation: Transparently handles dynamic sites with AJAX, JavaScript, authentication, secure servers, sessions, cookies, popup windows or sequences.

• Data Extraction: Example-based heuristic extraction of semi-structured content into defined schema from dynamic Web 2.0 content, PDF forms, documents.

• Search/Index: Built-in tools expose unstructured data as inverted indexes.• High Performance: Parallel Execution using MS IE or Denodo browser pools;

smart browsers load only optimal navigation sequences to minimize memory usage.

Content Integration CMS, file systems, Sharepoint, email servers, knowledge bases, indexes and ontologies.


Feature Description

Metadata Driven Integration

Graphical wizard-driven UI and tools to introspect siloed data, create unified logical data views, transform, cleanse, group, aggregate, modify output and define data workflow.

Semantic Integration Transform, relate and merge unstructured data with structured data using text mining, taxonomy filters and semantic tools for textual similarity.

Bidirectional Integration Supports read, write and transactions (2-phase commit and XA transactions).

Transformation, Quality, Matching

• Large library of built-in functions for SQL (including window functions), XSLT, XQuery, XPath, Java and semantic transformations with the ability to plug in external tools or custom functions in every step of the query execution lifecycle.

• Built-in data workflow tool for complex transformation or data quality processes. Support for arbitrary numeric precision for scenarios that require maximum accuracy.

Data Modeling and Model Bridge

• Data modeling to shape output, flatten or create new hierarchical data structures to match target schema.

• Support for using elements from other databases through the Model Bridge tool, which enables users to import data models created with modeling tools like ER/Studio, ERwin or PowerDesigner.

• Contract-first/top-down and bottom-up modeling for greater flexibility, parallel work and change governance.

Publish

Feature Description

Web Services • Fully RESTful interface exposes enterprise data assets as unique URIs accessible via search, browse or query drill-down using standard Web protocols, interfaces and HTTP verbs (GET, POST, PUT, DELETE)• Outputs in XHTML, XML or JSON for human and/or machine consumption.

• Publish SOAP web services that conform to contract-first schema. Supports XSLT tools, WS-security, ESB/JMS access and SOA catalog.

• Publish services in the Open Data (OData) protocol format. • Publish data via JMS (with JSON support), including MQSeries, SonicMQ and

Active MQ.

Database Views Delivers virtualized data as SQL views, data services or portlets/widgets to suit every need. Bespoke, optimized SQL views of unified virtual data layer accessed via ODBC, JDBC and ADO.NET.


Feature Description

Portal Widgets Publish SharePoint WebParts, Java portlets, AJAX widgets to use in major portal/mashup servers.

OData Protocol Supports the ability to publish data in the OData standard format so client applications can consume it easily using standard protocols.

Semantic Formats Answers SPARQL queries returning RDF (via D2R mappings).

Performance

Feature Description

Dynamic Query Optimizer • Makes automatic optimization decisions taking into account data source statistics, data source indexes, network transfer rates, etc.

• Reduces the amount of data transferred from the data sources through the network and execution time, for a dramatic improvement in performance.

• Maximizes delegation of operations to the data source.• Takes into account the special characteristics of data sources found in BigData

scenarios, like Hadoop clusters and parallel databases.

Automatic Query Rewriting Supports high-performance sub-queries, access to data sources using native high-performance protocols as well as automatic query rewriting for restructuring SQL sentences into optimal form.

Full Aggregation Pushdown Automatically transforms query execution plans to maximize delegation of aggregation to the source level, even when the aggregation must be performed on information obtained by joining disparate data sources.

Partial Aggregation Pushdown

Aggregation operations that cannot normally delegated are optimized by splitting them into two steps; then the optimizer is able to push down the most resource in-tensive step to the source while executing the less taxing step in the virtualization server.

Leveraging dimension tables replicas

When a table is replicated in several data sources (e.g. in several Data Warehouses), uses in each case the replica which maximizes query delegation.

On-The-Fly-Data Movement • In order for delegation to happen at the source level, a small data set from a data source is temporarily moved to another data source with a larger data set.

• The decision for data movement is made by the optimizer on its own without the user having to manually specify it.


Feature Description

Tunable Optimization Multiple features to manually tune the optimization engine; the user can analyze the behavior of the optimizer using the advanced visual tools for inspecting query execution traces and decide where and when to override the automated decisions.

Scheduling and Alerts A built-in scheduler, with support for external ETL processes, such as pre-fetching data to cache or materialized source. It also balances source latencies.

Intelligent, Multi-Mode Caching

Feature Description

Full Materializes the source data into a cached copy with full delegation pushdown and can be incrementally updated.

Partial Stores the results of the queries on a query-by-query basis to improve the performance of the most frequent queries without having to materialize the entire data set.

Incremental or Total Refresh The cache can be loaded incrementally so only the new rows in the source need to be copied into cache.

Event-Based or Scheduled Cache maintenance processes can be executed on a scheduled basis, configured to run when specific events are triggered by the sources, or executed manually.

Multi-Mode Caching The cache type can be configured at the view level. Fully cached, partially cached and non-cached views can be combined when necessary.

Incremental Queries When using ‘full caching’ mode, incremental queries can be used to retrieve the most recent data from the original data source and combine this with the cached data to deliver the most up-to-date results with minimal impact on the data source.

Security

Feature Description

Role-Based Access • User authentication using external LDAP, Active Directory and/or built-in user directory.

• Secure and differentiated role-based access control to data services and sources, enterprise tools for management and monitoring dashboards.


Feature Description

Single-Sign-On Single sign-on using Kerberos northbound and southbound connections.

Row/Column- Level Fine-Grained Authorization

Support for fine-grained security at virtual data view, column or row level and for masking sensitive data. Integrates with external systems for security, access control and SLA policy management.

Pass-Through Supports pass-through authentication to leverage security infrastructure in the data sources.

Data Encryption Support for importing and exporting encrypted data; communication between modules can be encrypted and authenticated using SSL also.

Firewall Support All components can be distributed in different network segments.

Resource Management

Feature Description

Resource Manager A UI based tool to enforce workload management policies in environments with multiple consuming applications that have varying workloads and usagepatterns.

Monitoring and Diagnostics Tool

• Real-time monitoring of groups of Denodo servers (‘environments’) or individual servers.

• View resource usage (CPU, memory, threads), query execution, cache activity, and error logs.

• Select time windows within log files to drill down and examine state of server for post-mortem diagnostics of past operations.

• View CPU, memory, query, cache, and thread information within selected time window.

Server Monitoring • Denodo Monitor to view real-time queries and historic audit/logs.• Support JMX, SNMP and WS-Management standards to integrate with leading

external monitoring tools including HP Openview, IBM Tivoli, Microsoft RM and Nagios.

Custom Policies for Workload Management

Workload management using custom policies to restrict or constrain service levels and workload based on external factors (source load, network, time).


Unified Data and Metadata Governance

Feature Description

Consistent Data Delivery Delivery of consistent, meaningful data services to users.

Metadata Repository Metadata repository with multiple visualization (tree view, linked data, attribute origin, source impact, catalog search, etc.); includes metadata API, model export and introspection from external systems.

Metadata Management • Discover, introspect and transform source metadata. • Refresh or propagate source metadata when it changes. • Flatten or create new hierarchical data structures.

Controlled Data Service Delivery

Data lineage, change impact, dependency analysis, metadata migration tools, version control and granular policy-based tiered security deliver a controlled data virtualization and enterprise data services capability.

Self-Service Information Discovery and Search

Feature Description

Universal Search • Google-like search capabilities for data and metadata.• Works on all types of data sources, including big data sources.

Data Lineage Intuitive, easy-to-use interfaces for browsing data and metadata and viewing data lineage.

Advance Query Wizard Create ad-hoc queries using the Advanced Query Wizard and save them as views in the data virtualization server.

Graphical Representation for Relationships and Associations of Data Entities

• Establishes relationships between virtual data entities using referential constraints (primary keys, foreign keys, varying levels of multiplicity, or conditionals).

• Associations can be introspected from external metadata/model tools for governance, browsed/traversed by users manually, or programmatically by applications using navigational queries.

Data Exploration Sandbox environment for exploring the data before publishing.


Agile Development

Feature Description

User-Friendly Development Environment

• Easy-to-use, enterprise-class tools. • Graphical, wizard-driven UI for all functions and modules.• Documented scripting for advanced users.

Platform Extensibility Platform extensions enabled via the Eclipse IDE plug-in for developing, testing, debugging and deploying custom functions, connectors and stored procedures.

Integration with Version Control

• Graphical Support for Git, Subversion and TFS Version Control Systems. • Support for checkout, commit and update of virtual entities with automatic

dependency control directly from within Denodo.

Graphical Support for Lifecycle Process Management

Graphical support for lifecycle process management (development, staging, production) and/or geographically dispersed environments.

VDP Admin Tool UI • New UI eases managing several open tasks at the same time.• Enables users to combine elements from several virtual databases. • Provides docking support, VQL highlighting and auto-complete features.

Hybrid Integration Mode

Feature Description

Denodo Scheduler • Enables complex, hybrid integration processes, integrating ETL within a broader data virtualization approach.

• Materializes unified data views by exporting to databases, warehouses, flat files, Excel, XML, etc. using the built-in Denodo Scheduler or external ETL tools.

Balanced Orchestration Deliver reliable, high performance virtual data services through the balanced orchestration of real-time, cached, scheduled batch or hybrid execution modes.

Task Persistence and Support

Supports persistent tasks through the continuation of queries after restart and transparent retries in case of failures, intermittent Web access or human intervention.

Task Orchestration Task dependencies allow linked tasks to start only when others have finished.


Denodo Platform Differentiators

• Broad Connectivity: The Denodo Platform provides universal connectivity and unified access to any type of data source—structured or unstructured—including big data, IoT and cloud.

• Operational and Analytical Use Cases: The Denodo Platform can be used for a broad spectrum of industry-specific or horizontal use cases covering both operational and analytical functions.

• Self-service discovery and search: The Denodo Platform is the only data virtualization software that allows business users to search for data in the data virtualization layer.

• Superior Performance: The Denodo Platform delivers superior performance through pushdown query optimization, advanced multi-mode caching and workload management. Denodo is the only data virtualization software that can automatically apply sophisticated optimization techniques which are crucial in Big Data scenarios, such as query aggregation, pushdown optimization and ‘on the fly’ data movement.

Benefits

• Faster Time to Market: A library of high quality data services speeds development on new projects. • Unified Ease-of-Use: A platform with unified access, to graphically model all data sources and publish

and manage data services easily and intuitively.

• Scalable Performance: Advanced automatic query optimization with options for manual override control, load balancing and modular clustered scalability.

• Data Services Reuse: Data is leveraged efficiently across transactional, analytical and informational applications to deliver the full benefits of data virtualization.

Denodo Platform Requirements

Operating Systems Microsoft Windows (32-bit and 64-bit platforms) • Windows XP, Windows Vista, Windows 7, Windows 8, Windows 10• Windows Server 2003, Windows Server 2008, Windows Server 2012

Linux (32-bit and 64-bit platforms) • Red Hat Enterprise Linux (RHEL) • Oracle Enterprise Linux (OEL)• Ubuntu Linux • CentOS Linux

UNIX (64-bit platforms) • Sun Solaris

Directory Services• LDAP v3 • Microsoft Active Directory 2003, 2008

Browser • Internet Explorer 10.x, 11.x.

Note: The Internet Explorer is only a requirement for using the ITPilot web browsing capabilities. The self-service interface also

requires a browser and will work with Firefox and Chrome in addition to IE.


Deployment Patterns

Denodo Platform 6.0 can be deployed in the data center or private cloud (either in physical or virtual servers) or in the public cloud (e.g. Amazon EC2). Strong Web Services support and available automation scripts make Denodo very cloud-friendly.

Multiple configurations supported:

• Basic single server configuration • Basic server with proxy configuration for firewalls • High availability clusters with load balancer in either active-passive (hot standby) or active-active

configurations • Clusters with shared cache or distributed local cache • Geographically distributed server environments • Multiple Denodo instances in peer-to-peer or multi-layered environments • Configurations can also be embedded in other product architectures (for example reporting and

dashboards, single-view applications, information services delivery platforms, or BPM/Workflow)

Visit www.denodo.com Email [email protected] twitter.com/denodoNA & APAC (+1) 877 556 2531 | EMEA (+44) (0) 20 7869 8053 | DACH (+49) (0) 89 203 006 441 | Iberia & Latin America (+34) 912 77 58 55

denodo platform 6.0

Documents