intelligent application networks webinar - ww2.amstat.org · mule • scatter-gather • choice...
TRANSCRIPT
![Page 1: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/1.jpg)
WITH MULE AND TENSORFLOWINTELLIGENT APPLICATION NETWORKS
![Page 2: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/2.jpg)
MACHINE LEARNING BASED AI WILL TRANSFORM 80% OF BUSINESSES IN FIVE YEARS
https://news.greylock.com/the-new-moats-53f61aeac2d9 https://sloanreview.mit.edu/projects/reshaping-business-with-artificial-intelligence/
![Page 3: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/3.jpg)
MACHINE LEARNING BASED AI WILL TRANSFORM 80% OF BUSINESSES IN FIVE YEARS
https://news.greylock.com/the-new-moats-53f61aeac2d9 https://sloanreview.mit.edu/projects/reshaping-business-with-artificial-intelligence/
![Page 4: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/4.jpg)
A MAJOR CHALLENGE IS MANAGING CONNECTIVITY AND CONSTANT CHANGE
Accelerating hyperspecialization of data, applications, and devices
![Page 5: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/5.jpg)
AVERAGE ENTERPRISE HAS 91 CLOUD SERVICES JUST IN MARKETING GREW FROM 150 TO 6800 SINCE 2011
HYPER SPECIALIZATION IN MARKETING
https://chiefmartec.com/2018/04/marketing-technology-landscape-supergraphic-2018/
![Page 6: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/6.jpg)
THIS TREND IS ACCELERATING IN EVERY INDUSTRY AS THE PACE OF CHANGE INCREASES
COMPANIES DOING “INITIAL COIN OFFERINGS ”
https://www.cbinsights.com/reports/CB-Insights_Blockchain-In-Review.pdf
![Page 7: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/7.jpg)
TO BUILD A SYSTEM OF INTELLIGENCE, START WITH AN “MINIMUM VIABLE MODEL” AND ITERATE
DATA TEAMS MUST ITERATE QUICKLYIdentify, Refine Problem
Deploy System
Collect Feedback
Improve Model
Train Model
Prepare training data
Collect Data
Explore Data
Evaluate Model
![Page 8: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/8.jpg)
MOST TIME-CONSUMING, LEAST ENJOYABLE DATA SCIENCE TASK
COLLECTING AND CLEANING DATA IS 80%
60%4%9%
19%3%5%
https://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says
Creating Training Set
Cleansing and organizing
Collecting Data
Mining for patterns
Other
Refining Algorithms
![Page 9: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/9.jpg)
WITH HYPERSPECIALIZATION CURRENT PROBLEMS WILL BECOME WORSE
▸ Custom code to get models into production grade applications
▸ Lack tight integration with user
▸ Long ETL cycles, different update cycles
▸ Poor discoverability
▸ Lack of labeled data
▸ Data lakes become dumping grounds
▸ Governance
▸ Poor metadata
FAILURE TO ITERATE RAPIDLY
![Page 10: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/10.jpg)
CONNECTIVITY AND ORCHESTRATION MEETS MACHINE LEARNING TO ENABLE RAPID ITERATION
SYSTEMS OF INTELLIGENCE
![Page 11: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/11.jpg)
CONNECTING APPLICATIONS, DATA, AND DEVICES INTO COMPOSITE APPLICATIONS
APPLICATION NETWORKS
▸ APIs and metadata
▸ Data types
▸ Capabilities
▸ Orchestration
▸ Composite applications
AWS
https://www.mulesoft.com/resources/api/what-is-an-application-network
![Page 12: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/12.jpg)
RAPIDLY ITERATE PRODUCTION MODELS AS SERVICES
Define Model
Train Params
Wrangle
Pipeline Orchestration Composite Application
Update APIBuild App
Publish API
Engage
Deploy as Microservice
Evaluate and Deploy
Connect
Collect Feedback
Mule Connectors
Connect Real-Time
![Page 13: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/13.jpg)
A CHURN PREDICTION: DESIGN AND IMPLEMENT AN API
API Spec and Type Definitions (RAML)
Querying the production server in Postman
![Page 14: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/14.jpg)
ONCE YOU DEPLOY AND PUBLISH, YOU’VE ADDED A NOTE TO THE APPLICATION NETWORK
Publish the API for other applications to use in
MuleSoft’s Anypoint Platform
Deploy as a micro service to MuleSoft’s PaaS (based on Kubernetes)
![Page 15: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/15.jpg)
LOAD KERAS MODEL AND WEIGHTS INTO SPARK AND CALL FROM MULE
![Page 16: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/16.jpg)
ORCHESTRATING THE ML PIPELINE WITH MULE FLOWS
Evaluate
R Packages • Yardstick
Mule • Scatter-Gather • Choice
Train
R Packages • Keras • Rsampler
Tensorflow • 35 inputs • 2 hidden layers • Binary output
Fetch
Mule Connectors
Preprocess
Mule • Scheduler • DataWeave
R Packages • Recipe • Dplyr • Farcats
Publish
Persist • Recipe • Model • Weights
Mule Connector
![Page 17: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/17.jpg)
BUILDING THE ML PIPELINE IN MULESOFT’S ANYPOINT STUDIO
R is invoked here
![Page 18: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/18.jpg)
PROCESSING THE DATA USING RECIPES
Adapted from: https://tensorflow.rstudio.com/blog/keras-customer-churn.html
Data Set Watson dataset on Telco Churn •20 predictors of churn •Demographics from Salesforce •Products from billing database •Transactions from S3 flat files
Preparing the training and test with R Recipes
![Page 19: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/19.jpg)
TRAIN THE MODEL WITH KERAS WITH TENSORFLOW BACKEND
Diagram here
Defining the model in R Keras
HIDDEN LAYER 1(20, 16), RELU
HIDDEN LAYER 2(16, 16), RELU
HIDDEN LAYER 2(16, 1), SIGMOID
Training the model in R Keras
![Page 20: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/20.jpg)
EVALUATE WITH YARDSTICK, “PUBLISH” UPDATE MODEL IF IT IMPROVES
![Page 21: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/21.jpg)
ADDING SYSTEMS OF INTELLIGENCE TO THE APPLICATION NETWORK
INTELLIGENT APPLICATION NETWORKS
▸ Specifications
▸ Capabilities
▸ Ontologies
▸ Orchestration
▸ Composite applications
▸ Knowledge Graph
▸ Systems of Intelligence
▸ Computational Graphs
AWS
TF
TF
TF
TF
![Page 22: Intelligent Application Networks Webinar - ww2.amstat.org · Mule • Scatter-Gather • Choice Train R Packages • Keras • Rsampler Tensorflow • 35 inputs • 2 hidden layers](https://reader030.vdocuments.us/reader030/viewer/2022041215/5e04463d4fba9e7616604f64/html5/thumbnails/22.jpg)
LINK TO PRESENTATION, VIDEO OF DEMO, AND CODE FOR EXAMPLE
Soren Harner is a principal data scientist with Permaling.
Permaling provides strategy consulting, implementation, and training for AI in the enterprise.
https://permaling.com
soeren [at] permaling [dot] com
https://github.com/sharner/sdss-mule
Permaling