w m d fcut al ver ions p w is d t u t d t sf mer l op
TRANSCRIPT
12/6/2014 DSSharing: Understanding of Transformer Looping & Caching
http://dsshar.blogspot.in/2013/02/understandingoftransformerlooping.html 1/6
DSSharingFriday, 1 February 2013
Understanding of Transformer Looping & Caching
Understanding of Transformer Looping & Caching:
Power of transformer looping is introduced from IIS Data stage 8.5 and above versions. Transformer looping is made developers life easy indeveloping complex scenarios which was almost difficult in earlier versions.
loop variables:
You can use loop variables when a loop condition is defined for the Transformer stage. When a loop is defined, the Transformer stage can outputmultiple rows for every row input to the stage. Loop variables are evaluated every time that the loop is iterated, and so can change their value forevery output row. Such variables are accessible only from the Transformer stage in which they are declared. You cannot use a loop variable in astage variable derivation.
Loop variables can be used as follows:
They can be assigned values by expressions.
They can be used in expressions which define an output column derivation.
Expressions evaluating a variable can include other loop variables or stage variables or the variable being evaluated itself.
@ITERATION: System variable used for transformer looping
What is Caching in the Transformer?
Total Pageviews
9 6 4 0
prasoon
Hi Guys, Welcome to DSShare blog.My nameis Prasoon and i have created this blog toshare datastage concepts.I will appriciatedyour comments on posts.
View my complete profile
About Me
CNET News.comFeed TechnologyNews
0 More Next Blog» Create Blog Sign In
12/6/2014 DSSharing: Understanding of Transformer Looping & Caching
http://dsshar.blogspot.in/2013/02/understandingoftransformerlooping.html 2/6
The Transformer Cache is an inmemory storage mechanism that is available from within the Transformer stage and is used to help solvecomplex data integration scenarios. The cache is a firstin/firstout (i.e. FIFO) construct and is accessible to the developer via two newfunctions:
• SaveInputRecord: stores an input row to back of the cache
• GetInputRecord: retrieves a saved row from the front of the cache
I will try to explain the transformer caching and looping using below scenario:
Below is my input data:
Customer Store Items Value600785011 103 10 100600785011 104 20 200600785011 109 30 300
Output:
For each customer get a list of the stores he shopped, with this list output every combination of stores dropping customer from the output.
HomeStore Items Value
AwayStore Items Value
103 10 100 103 10 100103 10 100 104 20 200103 10 100 109 30 300104 20 200 103 10 100104 20 200 104 20 200104 20 200 109 30 300109 30 300 103 10 100109 30 300 104 20 200109 30 300 109 30 300
Earn MoneyOnline
CNETNews.comMom tries toFacebookshamedaughter, getspizza on faceNSA'sreported
► Tutorial► SQL Queries► CNet
Apple Google Microsoft
How Apple will use iPhones to automate ourhomesMacworld UKHome automation could benefit from Apple'strack record of taking a technology andmaking it user friendly and mainstream. At themoment those with smart gadgets in theirhome likely have a number of separate appson their phones and computers for ...
Steve Jobs Defends Apple in TapedDepositionWall Street JournalA federal judge weighed whether to end a 10year legal battle over iPod pricing, as a juryheard late Apple Inc. cofounder Steve Jobssay in a recorded deposition that he was “veryscared” about hackers of the company'siTunes service. Apple moved ...Related Articles »
Judge Questions Plaintiffs in Apple iPod CaseWall Street JournalA federal judge Thursday questioned whetherany of the plaintiffs in a longrunning antitrustsuit against Apple Inc. had actually bought theiPods at issue in the case. “What am Isupposed to do if I don't have a plaintiff?”asked a concerned U.S ...
12/6/2014 DSSharing: Understanding of Transformer Looping & Caching
http://dsshar.blogspot.in/2013/02/understandingoftransformerlooping.html 3/6
Below is Job Design:
SeqFile > TRFM1> TRFM2>TempDataset
TSFM1:
Define below stage variables in transformer:
NumSavedRows= SaveInputRecord
IsBreak= LastRowInGroup(to_tsfm1.Customer)
svCurrStore= If IsBreak Then to_tsfm1.Store:"|": svPrevStore Else to_tsfm1.Store
svPrevStore= If IsBreak Then to_tsfm1.Items else to_tsfm1.Items :"|": svPrevItem svCurrItem= If IsBreak Then to_tsfm1.Items :"|":svPrevItem Else to_tsfm1.Items
svPrevItem= If IsBreak Then to_tsfm1.Items else to_tsfm1.Items :"|": svPrevItem
svCurrValue= If IsBreak Then to_tsfm1.Value :"|": svPrevValue Else to_tsfm1.Values
svPrevValue= If IsBreak Then to_tsfm1.Value else to_tsfm1.Value :"|": svPrevValue
NumRows= If IsBreak Then NumSavedRows Else 0
Define below loop conditions & Derivation:
@ITERATION <= NumRows
OutputRecord= GetSavedInputRecord()
Output Derivations:
Store=Inputlink.Store
Item=InputLink.Item
Value=InputLink.Value
ConcatStore=svCurrStore
ConcatItem=svCurrItem
Related Articles »
The difference between Apple and everyoneelse, in one photoQuartzThen if you turn 180 degrees, this is the“Apple Shop”—a storewithinastore that'spresent in several BIC Camera locationsaround Japan. It's an amazingly calm sectionin an otherwise waytoobusy store, includingthe same sorts of fixtures, design ...
powered by
12/6/2014 DSSharing: Understanding of Transformer Looping & Caching
http://dsshar.blogspot.in/2013/02/understandingoftransformerlooping.html 4/6
ConcatValue=svCurrValue
Customer=InputCustomer
The logic of this transformer is as follows:
1 input data is read and stored in the cache via the "SaveInputRecord" function.2.key breaks are tested via the "LastRowInGroup.3.svCurrStore, svCurrItem, svCurrValue will concatenate with pipl all stores,Items & values in the same group.4.if there is a key break, the " NumRows " variable is set to the number of records on the cache and we'll skip to step 6 5.if there is no key break(NumRows=0) the Loop Condition will be false. This will cause the next input record to be read and processed .6.finally, the transformer will loop while "@ITERATION <= NumRows " and read the cache records.
TSFM2:Define below Stage Variable:
RowNum=Count(InputLink1.ConcatStore,”|”)
Define below loop conditions :
@ITERATION <= RowNum
Output Derivation:
Store=InputLink1.Store
Item=InputLink.Item
Value=InputLink.Value
AwayStore= Field(to_tsfm2.ConcatStore,"|",@ITERATION)
Item= Field(to_tsfm2.ConcatItems,"|",@ITERATION)
Value= Field(to_tsfm2.ConcatValue,"|",@ITERATION)
Note: I have just tried to explain better understanding of Transformer looping using this scenario in my blog,may be it can be done indifferent logics.I will appriciate if you can bring other logics too. ThanksPrasoon
12/6/2014 DSSharing: Understanding of Transformer Looping & Caching
http://dsshar.blogspot.in/2013/02/understandingoftransformerlooping.html 5/6
Posted by prasoon at 05:46
Location: Cape Town, South Africa
Recommend this on Google
Enter your comment...
Comment as: Google Account
Publish
Preview
1 comment:
MADHAVI LATHA 25 May 2013 01:45
hi. i have a scenario where I should implement the carry forward logic. Consider the below example.Account_number Balance Transcation_date 1 500 01May20131 100 10May20131 1000 15May2013I need the output as below:Account_number Balance Transaction_date 1 500 01May20131 500 02May20131 500 03May2013'''1 100 10May20131 100 11May2013''1 1000 15May2013
Please someone help me in finding the logic using transformer looping concept.Thanks in advance.
Reply
12/6/2014 DSSharing: Understanding of Transformer Looping & Caching
http://dsshar.blogspot.in/2013/02/understandingoftransformerlooping.html 6/6
Older PostHome
Subscribe to: Post Comments (Atom)
Simple template. Template images by gaffera. Powered by Blogger.