talend open studio for data integration release notes...talend open studio for data integration: new...

Talend Open Studio for DataIntegration Release Notes

7.3.1Last updated: 2020-02-26

Contents

Copyleft...........................................................................................................................3

System requirements.................................................................................................... 4

Talend Open Studio for Data Integration: new features..........................................5Data Integration: new features......................................................................................................................................5

Talend Open Studio for Data Integration: bug fixes..............................................15

Talend Open Studio for Data Integration: deprecated and removed items.........16Language support.............................................................................................................................................................16Data Integration: deprecated and removed items...............................................................................................16

Talend Open Studio for Data Integration: known issues and limitations............ 18Data Integration: known issues and known limitations.................................................................................... 18

Talend Open Studio for Data Integration: hints and notes...................................19Data Integration: hints and notes..............................................................................................................................19

Documentation............................................................................................................ 20

Reporting defects and issues.................................................................................... 21

Copyleft

Copyleft

Adapted for 7.3.1. Supersedes previous releases.

The content of this document is correct at the time of publication.

However, more recent updates may be available in the online version that can be found on TalendHelp Center.

This documentation is provided under the terms of the Creative Commons Public License (CCPL).

For more information about what you can and cannot do with this documentation in accordance withthe CCPL, please read: http://creativecommons.org/licenses/by-nc-sa/2.0/.

Notices

Talend is a trademark of Talend, Inc.

All brands, product names, company names, trademarks and service marks are the properties of theirrespective owners.

License Agreement

The software described in this documentation is licensed under the Apache License, Version 2.0 (the"License"); you may not use this software except in compliance with the License. You may obtaina copy of the License at http://www.apache.org/licenses/LICENSE-2.0.html. Unless required byapplicable law or agreed to in writing, software distributed under the License is distributed on an "ASIS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See theLicense for the specific language governing permissions and limitations under the License.

This product includes software developed at AOP Alliance (Java/J2EE AOP standards), ASM, Amazon,AntlR, Apache ActiveMQ, Apache Ant, Apache Axiom, Apache Axis, Apache Axis 2, Apache Batik,Apache CXF, Apache Chemistry, Apache Common Http Client, Apache Common Http Core, ApacheCommons, Apache Commons Bcel, Apache Commons JxPath, Apache Commons Lang, Apache DerbyDatabase Engine and Embedded JDBC Driver, Apache Geronimo, Apache Hadoop, Apache Hive, ApacheHttpClient, Apache HttpComponents Client, Apache JAMES, Apache Log4j, Apache Lucene Core,Apache Neethi, Apache POI, Apache ServiceMix, Apache Tomcat, Apache Velocity, Apache WSS4J,Apache WebServices Common Utilities, Apache Xml-RPC, Apache Zookeeper, Box Java SDK (V2), CSVTools, DataStax Java Driver for Apache Cassandra, Ehcache, Ezmorph, Ganymed SSH-2 for Java, GoogleAPIs Client Library for Java, Google Gson, Groovy, Guava: Google Core Libraries for Java, H2 EmbeddedDatabase and JDBC Driver, Hector: A high level Java client for Apache Cassandra, Hibernate Validator,HighScale Lib, HsqlDB, Ini4j, JClouds, JLine, JSON, JSR 305: Annotations for Software Defect Detectionin Java, JUnit, Jackson Java JSON-processor, Java API for RESTful Services, Java Agent for MemoryMeasurements, Jaxb, Jaxen, Jettison, Jetty, Joda-Time, Json Simple, LightCouch, MetaStuff, Mondrian,OpenSAML, Paraccel JDBC Driver, PostgreSQL JDBC Driver, Resty: A simple HTTP REST client for Java,Rocoto, SL4J: Simple Logging Facade for Java, SQLite JDBC Driver, Simple API for CSS, SshJ, StAX API,StAXON - JSON via StAX, The Castor Project, The Legion of the Bouncy Castle, W3C, Woden, Woodstox:High-performance XML processor, Xalan-J, Xerces2, XmlBeans, XmlSchema Core, Xmlsec - ApacheSantuario, Zip4J, atinject, dropbox-sdk-java: Java library for the Dropbox Core API, google-guice.Licensed under their respective license.

3

https://help.talend.com

https://help.talend.com

http://creativecommons.org/licenses/by-nc-sa/2.0/

http://www.apache.org/licenses/LICENSE-2.0.html

System requirements

System requirements

Users should refer to the Talend Installation and Migration Guide on the Talend Help Center (http://help.talend.com) for more information on Installation and System Requirements.

4

http://help.talend.com

http://help.talend.com

Talend Open Studio for Data Integration: new features

Talend Open Studio for Data Integration: newfeatures

Data Integration: new features

Feature Description

Online help for components on pressing F1 Starting from Talend 7.3, by default, pressing F1 on acomponent will open the relevant documentation on TalendHelp Center using the default Web browser.

To switch to offline help, go to Window > Preferences >Talend > Components and clear the Enable online help forcomponents check box.

Before using offline help, subscription users need to installthe Talend Help package by following the instructionswhen switching to offline help, or going to Help > InstallAdditional Packages.

Rotating encryption keys Talend Studio now allows you to rotate the following twoencryption keys by adding a new version of the key with anempty value in the encryption key configuration file.

• the system encryption key: for encrypting propertiesand nexus passwords

• the routine encryption key: for encrypting passwords ofgenerated Jobs

Log4j 2 support Talend Studio now supports Apache logging utility Log4j 2in components and allows you to enable or disable Log4j

5


Feature Description

2 loggers and customize the configuration globally for theproject.

Note: If you need to deploy your Jobs to Talend Cloud,the use of Log4j 2 requires Talend Remote Engine 2.8.1or higher.

Redshift SSO support with Ping Federate in metadata wizard The Login and Password fields have been added in theDatabase Connection metadata wizard for Redshift SSO.

Metadata Bridge version upgrade The version of Talend Metadata Bridge has been updated to10.1.

New components available The following new components are now available.

• tSnowflakeCommit, tSnowflakeRollback• tAzureAdlsGen2Input, tAzureAdlsGen2Output• tWorkdayInput

New option available: Connection timeout The Connection timeout option is added to the Advancedsettings view.

Components concerned:

• tFTPFileExist, tFTPPut, tFTPConnection, tFTPDelete,tFTPFilelist, tFTPFileProperties, tFTPGet, tFTPRename

6


Feature Description

Excel 2007 xlsm support This feature enables accesses to Excel 2007 files (withthe extensions of .xlsx and .xlsm) through the Readexcel2007 file format (xlsx / xlsm) and the Write excel2007file format (xlsx / xlsm) options.


• tFileInputExcel, tFileOutputExcel

Excel 2007 password support This feature allows the tFileInputExcel component toread password-protected Excel 2007 (and later) filesgiven that the passwords are available; it also allows thetFileOutputExcel component to set passwords for Excel2007 (and later) files.


• tFileInputExcel, tFileOutputExcel

tSetProxy: Https proxy with authentication This feature allows for connections to a basic authentication-enabled Https proxy server through the Use authentication, Proxy user, Proxy password, and Enable BasicAuthentication Header (in the Advanced settings view)options.

Component concerned:

• tSetProxy

7


Feature Description

tJDBCInput: prepared statements This feature enables you to query a database using aprepared statement through the Use PreparedStatementoption in the Advanced settings view.

Component concerned:

• tJDBCInput

AWS server-side encryption (SSE) for SQS queues Components concerned:

• tSQSConnection, tSQSInput, tSQSMessageChangeVisibility, tSQSMessageDelete, tSQSOutput,tSQSQueueAttributes, tSQSQueueCreate,tSQSQueueDelete, tSQSQueueList, tSQSQueuePurge

tPostgresqlOutput: new option available The tPostgresqlOutput component provides the Convertcolumn and table to lowercase option in the Advancedsettings view, to put column names of newly createdPostgres tables in lower case.

tServiceNowInput: combined filtering conditions This feature allows for combined filtering conditions(combined with AND and OR operators) for retrieving datafrom ServiceNow through the Advanced Query field in theAdvanced settings view.

8


Feature Description

tRunJob: customizable Child Job JVM argument This feature provides the following two ways in theAdvanced settings view for setting JVM arguments for theJob to be called or processed (Use dynamic job needs to beselected in the Basic settings view).

• Use child job JVM arguments• Overwrite child job JVM arguments

tJDBCSCDELT: End date configurable This feature allows you to set the End date for type 2 SCD.

Access control list for S3 components This feature allows you to set access control list (ACL) forthe current component through the Access control option inthe Advanced settings view.


• tS3Put• tS3BucketCreate

9


Feature Description

Redshift Bulk components: reuse S3 connection This feature enables Redshift components to reuse existingS3 connections.


• tRedshiftUnload, tRedshiftOutputBulk, tRedshiftOutputBulkExec, tRedshiftBulkExec

tSetProxy: three new options provided for using proxy The tSetProxy components provides the following threenew options.

• Use Java Virtual Machine Proxy Configuration• URIs to use proxy• Thread isolated proxy (in the Advanced settings view)

See Talend Components Reference Guide for relatedinformation.

Setting Kafka connections and configuring Azure EventHubas Kafka cluster

This feature enables you to set Kafka connections andconfigure an Azure EventHub as Kafka cluster.

Using a schema other than the schema of the connection Through the Use alternate schema option in the Advancedsettings view, this feature enables you to use a schemaother than the one specified by the component thatestablishes the database connection (Use an existingconnection needs to be selected in the Basic settings view).


• tAmazonOracleOutput• tRedshiftOutput

10


Feature Description

• tGreenplumOutput• tDB2Output• tInformixOutput• tMSSqlOutput• tOracleOutput• tParAccelOutput• tPostgreSQLOutput• tPostgresPlusOutput• tSAPHanaOutput• tSybaseOutput• tVerticaOutput

tSnowflakeConnection: key pair authentication The tSnowflakeConnection component provides the key pairauthentication.

Customizing SET clause This feature generates SET clauses for update operationsbased on the columns you selected in the Clause SET tablein the Advanced settings view (Update needs to be selectedfrom the Action on data drop-down list in the Basic settingsview).


• tELTTeradataOutput• tELTOutput

Extending AWS ECS role support to other components This feature makes instance profile credential valid both onAmazon EC2 instances and AWS ECS instances.

SHA2 support SFTP connection now supports SHA2 authentication algorithm.


• tFTPConnection• tFTPDelete

11


Feature Description

• tFTPFileExist• tFTPFileList• tFTPFileProperties• tFTPGet• tFTPPut• tFTPRename• tFTPTruncate

New Amazon S3 region options The following new Amazon S3 region options are provided:

• US East (Ohio)• Asia Pacific (Hong Kong)• Asia Pacific (Seoul)• Canada (Central)• China (Ningxia)• EU (London)• EU (Paris)• EU (Stockholm)


• tRedshiftOutputBulk, tRedshiftOutputBulkExec• tS3BucketCreate, tS3BucketDelete, tS3BucketExist

, tS3BucketList, tS3Connection, tS3Copy, tS3Delete,tS3Get, tS3List, tS3Put

New Amazon S3 region options The following new Amazon S3 region options are provided:

• US East (Ohio)• Asia Pacific (Hong Kong)• Asia Pacific (Seoul)• Canada (Central)• EU (London)• EU (Paris)• EU (Stockholm)

Components concerned: tSnowflakeOutputBulk,tSnowflakeOutputBulkExec, and tSnowflakeBulkExec

tRedshiftBulkExec: the Assume Role option available toother data source types besides S3

For the tRedShiftBulkExec component, in addition to S3data sources, the Assume Role option is now available tothese data sources: EMR, DynamoDB, and Remote host. Youcan thus set IAM role ARN chains for these four types ofdata sources.

tSQLTemplateMerge: the Database Type drop-down listprovides the Snowflake option

The tSQLTemplateMerge component now provides theSnowflake option in the Database Type drop-down list. Inaddition, the following system SQL templates are provided:

12


Feature Description

• SnowflakeCreateSourceTable• SnowflakeCreateTargetTable• SnowflakeDropSourceTable• SnowflakeDropTargetTable• SnowflakeMerge

tAmazonEMRManage: cluster version upgrade The tAmazonEMRManage component now supports thecluster version EMR 5.15.0, which is the default clusterversion when configuring the component.

Besides, the support for EMR 4.2.0, EMR 4.1.0, and EMR 4.0.0is dropped.

ActiveMQ 5.15.10 support Components concerned:

• tMomCommit, tMomConnection, tMomInput,tMomMessageIdList, tMomOutput, tMomRollback

• tSAPADSOInput, tSAPBApi, tSAPBWInput,tSAPCommit, tSAPConnection, tSAPDataSourceOutput, tSAPDataSourceReceiver, tSAPDSOInput,tSAPDSOOutput, tSAPIDocInput, tSAPIDocOutput,tSAPIDocReceiver, tSAPInfoCubeInput, tSAPInfoObjectInput, tSAPInfoObjectOutput, tSAPODPInput,tSAPRollback, tSAPTableInput

Snowflake JDBC driver 3.11.x support Components concerned in Standard Jobs:

• tSnowflakeBulkExec, tSnowflakeClose, tSnowflakeConnection, tSnowflakeInput, tSnowflakeOutput,tSnowflakeOutputBulk, tSnowflakeOutputBulkExec,tSnowflakeRow

Redshift JDBC driver 1.23.7.1061 support Components concerned in Standard Jobs:

• tRedshiftBulkExec, tRedshiftClose, tRedshiftCommit,tRedshiftConnection, tRedshiftInput, tRedshiftOutput, tRedshiftOutputBulk, tRedshiftOutputBulkExec,tRedshiftRollback, tRedshiftRow, tRedshiftUnload

MySQL driver 8.0.18 support (MySQL 4 support dropped) Components concerned in Standard Jobs:

• tMysqlCDC, tMysqlClose, tMysqlColumnlist,tMysqlCommit, tMysqlConnection, tMysqlInput,tMysqlInvalidRows, tMysqlLastInsertId, tMysqlOutput,tMysqlOutputBulk, tMysqlOutputBulkExec,tMysqlRollback, tMysqlRow, tMysqlSCD, tMysqlSCDELT,tMysqlSP,tMysqlTableList, tMysqlValidRows

Vertica DB 9.3.1 support Components concerned:

• tVerticaBulkExec, tVerticaClose, tVerticaCommit,tVerticaConnection, tVerticaInput, tVerticaOutput

13


Feature Description

, tVerticaOutputBulk, tVerticaOutputBulkExec,tVerticaRollback, tVerticaRow, tVerticaSCD

Postgres 12.1 support Components concerned:

• tPostgresqlCDC, tPostgresqlClose, tPostgresqlCommit, tPostgresqlConnection, tPostgresqlInput,tPostgresqlInvalidRows, tPostgresqlOutput,tPostgresqlOutputBulk, tPostgresqlOutputBulkExec,tPostgresqlRollback, tPostgresqlRow, tPostgresqlSCD,tPostgresqlSCDELT, tPostgresqlValidRows

MariaDB 10.x support (MariaDB JDBC driver upgraded to2.5.3)

Components concerned in Standard Jobs:

• tCreateTable• tELTMysqlMap• tMysqlBulkExec, tMysqlConnection, tMysqlInput,

tMysqlOutput, tMysqlOutputBulkExec, tMysqlRow,tMysqlSCD, tMysqlSCDELT, tMysqlSP, tMysqlCDC

• tJDBCClose, tJDBCColumnList, tJDBCCommit,tJDBCConnection, tJDBCInput, tJDBCOutput,tJDBCRollback, tJDBCRow, tJDBCSCDELT, tJDBCSP,tJDBCTableList

Default Salesforce endpoint updated The default Salesforce endpoint updated as: https://login.salesforce.com/services/Soap/u/47.0.

Netsuite endpoint updated Netsuite endpoint updated as: https://webservices.netsuite.com/services/NetSuitePort_2019_2 (API version 2019.2 required).

14

https://login.salesforce.com/services/Soap/u/47.0

https://login.salesforce.com/services/Soap/u/47.0

https://webservices.netsuite.com/services/NetSuitePort_2019_2



Talend Open Studio for Data Integration: bug fixes

Talend Open Studio for Data Integration: bug fixes

In addition to the new features, a number of minor improvements within the entire products andsignificant bug fixes have been made. See the corresponding Change Logs on our bug tracking systemfor more details on the individual issues:

Module Change Log

Data Integration https://jira.talendforge.org/issues/?filter=31259

15

https://jira.talendforge.org/issues/?filter=31259

Talend Open Studio for Data Integration: deprecated and removed items

Talend Open Studio for Data Integration: deprecatedand removed items

Language supportFrom the 7.3 release onwards, the community-based UI localization system (Babili) has beendecommissioned.

All localization of Talend products is now handled internally by Talend. This means that the scope oflanguages available in the products will be restricted to French, Japanese and (partially) Chinese sothat Talend can maintain and support them over time.

For the 7.3 release itself, all community languages and their partial translations are still available.Talend reserves the right to remove these languages in subsequent releases.

Data Integration: deprecated and removed items

Feature Description

Log4j 1 The Apache Log4j 1 logging utility in components isdeprecated from Talend 7.3 onwards.

MicrosoftAX components The following MicrosoftAX components are removed fromTalend 7.3 onwards:

• tMSAXInput• tMSAXOutput

Sage X3 components The following Sage X3 components are removed fromTalend 7.3 onwards:

• tSageX3Input• tSageX3Output

OleDB components The following OleDB components are removed from Talend7.3 onwards:

• tOleDBInput• tOleDBOutput• tOleDBRow

CentricCRM components The following CentricCRM components are removed fromTalend 7.3 onwards:

• tCentricCRMInput• tCentricCRMOutput

SugarCRM components The following SugarCRM components are removed fromTalend 7.3 onwards:

• tSugarCRMInput• tSugarCRMOutput

SAS components The following SAS components are removed from Talend7.3 onwards:

• tSasInput

16

Talend Open Studio for Data Integration: deprecated and removed items

Feature Description

• tSasOutput

SPSS components The following SPSS components are removed from Talend7.3 onwards:

• tSPSSInput• tSPSSOutput• tSPSSProperties• tSPSSStructure

17

Talend Open Studio for Data Integration: known issues and limitations

Talend Open Studio for Data Integration: knownissues and limitations

Data Integration: known issues and known limitationsWe encourage you to consult the JIRA bug tracking tool for a full list of open issues:


Limitation Description

• the SOQL query using the aggregate function• the SOQL query using the TYPEOF expression• the SOQL query using alias• Some SOQL queries are not supported for the guess

schema andthe prefix in the SOQL query is the same asthe module name after FROM

For example, SELECT Account.Name, (SELECTContact.LastName FROM Account.Contacts) FROMAccount is not supported, while SELECT Name,(SELECT LastName FROM Contacts) FROM Account thatperforms the same function is supported.

• the SOQL query with multiple modules after FROM

For example, SELECT Contact.Firstname, Account.NameFROM Contact,Contact.Account is not supported.However, the child-to-parent and parent-to-childrelationship queries are supported, so this query can beimproved to SELECT Firstname, Account.Name FROMContact.

• the SOQL query using toLabel()• the field with the underscore character '_' in its name

Java PSU is not supported. Talend is only supported on Java CPU Versions, not JavaPSU.

Some users reported problems using Talend Studio withJava PSU versions (such as jdk8_u92), but when revertingto a Java CPU Version (such as jdk8_u91) the problem wasresolved.

See also: http://www.oracle.com/technetwork/java/javase/cpu-psu-explained-2331472.html

Collation limitation with Access Since 6.0, the Access components in Talend Studio supportonly the General collation mode.

18


http://www.oracle.com/technetwork/java/javase/cpu-psu-explained-2331472.html

http://www.oracle.com/technetwork/java/javase/cpu-psu-explained-2331472.html

Talend Open Studio for Data Integration: hints and notes

Talend Open Studio for Data Integration: hints andnotes

Data Integration: hints and notes

Item Description

Oracle JDK version requirement If you are using Oracle Java 8, the JDK version should be1.8.0_161 or higher. If the installed JDK version is lowerthan 1.8.0_161, you need to install the additional resourceJava Cryptography Extension (JCE) Unlimited StrengthJurisdiction Policy Files.

Notes about Git support All branches/tags are visible for all projects on the same Gitrepository. Therefore, if you create some branches/tags forone project, all other projects on the same Git repositorywill have the same list.

About the unlock mode: Any item modified will be keptlocally, and will never be committed / pushed to the serveruntil the item is unlocked. Project items will be storedtemporarily on the stash system of Git if needed, so thatthey are not pushed until they are unlocked.

Use of recursive Jobs Avoid using recursive Jobs because this forces recursiveJobs with the same dependencies build. In a normal caseeach Job has its own Maven project, but with the use ofrecursive Jobs, all the Jobs linked might be compiled insidethe same project. This can easily cause some compilationissues between the Jobs.

Unable to select tables in MySQL components Due to a known issue of MySQL database driver 8.0.12 andsome earlier versions, you may have a problem openingthe list of database tables in MySQL components. If thishappens, use MySQL database driver 8.0.13 or a laterversion.

19

Documentation

Documentation

Talend Help Center

Find out more about how to get the most out of your Talend products on the Talend Help Center:http://help.talend.com.

Open issues

We encourage you to consult the JIRA bug tracking tool for a full list of open issues:


20

http://help.talend.com/


Reporting defects and issues

Reporting defects and issues

As a customer of Talend with a valid support contract, you can use our Talend Online Helpdesk or youcan contact our customer support by phone and e-mail as stated in your contract.

Often the following information is needed and would help us to more easily evaluate your issue:

• Product and Version which you used (for example, Talend Data Fabric 7.3.1)• Operating System, for example, Windows Server 2008 R2 SP1 (64bit)• Version of Java Platform JDK / JRE in use (for example, ORACLE JDK 1.8.0_161)• Log files and/or screenshots

Also, in order to ensure that the technical support team has sufficient information to help you,describe at least the following in detail:

• your actions up to the point when the problem occurred• the results you expected

In addition to our support helpdesk, you can also use our community support tools:

• Talend Community.• You can also look into the Talend Bug Tracker.• Just keep in mind that only cases you create via one of the Talend Support channels in our Talend

Support Helpdesk are treated under the service level agreements we provided in the relatedcontract to you.

21

https://community.talend.com/

http://jira.talendforge.org/

talend open studio for data integration release notes...talend open studio for data integration: new...

Documents