workshop deep learning: ai applications and trends · deep learning future trends automl automated...

DEEP LEARNING:AI APPLICATIONSAND TRENDS

WORKSHOP

WHAT IS DEEP LEARNING?A TYPE OF MACHINE LEARNING CAPABLE OF ADAPTINGITSELF TO NEW DATA AND TRAINING ITS SYSTEMS TO LEARN ON THEIR OWN AND RECOGNISE PATTERNS.

DEEP LEARNINGINTRODUCTION

WHY DO WE USE DEEP LEARNING?DEEP LEARNING EXCELS ON PROBLEM DOMAINS WHERE THE INPUTS (AND EVEN OUTPUT) ARE ANALOG, I.E., NOT QUANTITATIVE DATA IN A TABULAR FORMAT BUT INSTEAD IMAGES OF PIXEL DATA, DOCUMENTS OF TEXT DATA OR FILES OF AUDIO DATA.

DEEP LEARNINGWHY DO WE USE DEEP LEARNING?

WHAT IS A NEURAL NETWORK?NEURAL NETWORKS (ALSO KNOWN AS ARTIFICIAL NEURAL NETWORKS) ARE A SET OF ALGORITHMS, MODELED LOOSELY AFTER THE HUMAN BRAIN, THAT ARE DESIGNED TO RECOGNIZE PATTERNS. THEY INTERPRET SENSORY DATA THROUGH A KIND OF MACHINE PERCEPTION.

BIOLOGICAL VS ARTIFICIAL NEURAL NETWORKSINTRODUCTION TO NEURAL NETWORKS

BIOLOGICAL NEURONS

ARTIFICIAL NEURAL NETWORKS

NEURAL NETWORKS IN USEHOW DO THEY WORK?

WHAT IS BACKPROPAGATION?A NEURAL NETWORK PROPAGATES THE SIGNAL OF THE INPUT DATA FORWARD THROUGH ITS PARAMETERS TOWARDS THE MOMENT OF DECISION, AND THEN BACKPROPAGATES INFORMATION ABOUT THE ERROR, IN REVERSE THROUGH THE NETWORK, SO THAT IT CAN ALTER THE PARAMETERS.

BACKPROPAGATIONSTEP BY STEP

WHAT IS TRANSFER LEARNING?WHILE MOST MACHINE LEARNING ALGORITHMS ARE DESIGNED TO ADDRESS SINGLE TASKS, TRANSFER LEARNING INVOLVES THE APPROACH IN WHICH KNOWLEDGE LEARNED IN ONE OR MORE SOURCE TASKS IS TRANSFERRED AND USED TO IMPROVE THE LEARNING OF A RELATED TARGET TASK.

TRANSFER LEARNINGADVANTAGES

TRANSFER LEARNINGFINE-TUNING PRE-TRAINED MODELS

CONVOLUTIONAL NEURAL NETWORKSA DEEP LEARNING ALGORITHM WHICH CAN TAKE IN AN INPUT IMAGE, ASSIGN IMPORTANCE (LEARNABLE WEIGHTS AND BIASES) TO VARIOUS ASPECTS/ OBJECTS IN THE IMAGE AND BE ABLE TO DIFFERENTIATE ONE FROM THE OTHER.

CONVOLUTIONAL NEURAL NETWORKSTHE PRE-PROCESSING REQUIRED IS MUCH LOWER AS COMPARED TO OTHER CLASSIFICATION ALGORITHMS.

RECURRENT NEURAL NETWORKSA TYPE OF NEURAL NETWORK WHERE THE OUTPUT FROM PREVIOUS STEPS ARE FED AS INPUT TO THE CURRENT STEP.

FOR INSTANCE, IN CASES LIKE WHEN IT IS REQUIRED TO PREDICT THE NEXT WORD OF A SENTENCE, THE PREVIOUS WORDS ARE REQUIRED AND HENCE THERE IS A NEED TO REMEMBER THE PREVIOUS WORDS.

RECURRENT NEURAL NETWORKSHOW THEY WORK

ATTENTION IN DEEP LEARNINGATTENTION MECHANISMS IN NEURAL NETWORKS ARE LOOSELY BASED ON THE VISUAL ATTENTION MECHANISM FOUND IN HUMANS, I.E., BEING ABLE TO FOCUS ON A CERTAIN REGION OF AN IMAGE WITH “HIGH RESOLUTION” WHILE PERCEIVING THE SURROUNDING IMAGE IN “LOW RESOLUTION”, AND THEN ADJUSTING THE FOCAL POINT OVER TIME.

ATTENTION MECHANISMSEXAMPLE: NEURAL MACHINE TRANSLATION (NMT) SYSTEMS WORK A BIT DIFFERENTLY FROM TRADITIONAL METHODS. IN NMT, WE MAP THE MEANING OF A SENTENCE INTO A FIXED-LENGTHVECTOR REPRESENTATION AND THEN GENERATE A TRANSLATION BASED ON THAT VECTOR.

APPLICATIONS OF DEEP LEARNING

GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text.

ImageNet is an image database organised according to the WordNet hierarchy, in which each node of the hierarchy is depicted by hundreds and thousands of images. Currently it has an average of over five hundred images per node.

APPLICATIONS OF DEEP LEARNINGFINANCIAL

APPLICATIONS OF DEEP LEARNINGFACIAL RECOGNITION

APPLICATIONS OF DEEP LEARNINGDRIVERLESS CARS

DEEP LEARNING IN TELECOMMUNICATIONS

ANALYSING CALL LOG DATA MOBILE PAYMENTS REDUCING NOISE

DEEP LEARNING IN TELECOMMUNICATIONS

SPEECH-TO-TEXT NATURAL LANGUAGEUNDERSTANDING

MODEL PRUNING

DEEP LEARNINGTECHNICAL LIMITATIONS

• Huge amounts of data required to ensure good performance

• High levels of computing power required to train deep learning models, as compared totraditional machine learning models

• Difficult to get error estimates or confidence levels on your predictions

• Black Box: very hard to explain why a certain decision was made by the algorithm

• Difficult to implement and train; a lot of tuning required

DEEP LEARNINGFUTURE TRENDS

AUTOML

Automated machine learning (AutoML) is the process of automating end-to-end the process of applying machine learning to real-world problems. AutoML was proposed as an AI-based solution to the ever-growing challenge of applying machine learning for non-experts.

Google’s AutoML lets you train custom machine learning models without having to code, exploring the capabilities of neural nets to design neural nets.

DEEP LEARNINGAUTOML

DEEP LEARNINGFUTURE TRENDS

NEURAL ARCHITECTURE SEARCH

Manually designing neural network architectures requires tremendous human effort, which is expensive and sub-optimal. Neural Architecture Search(NAS) focuses on automating the architecture design process.

DEEP LEARNINGNEURAL ARCHITECTURE SEARCH

DEEP LEARNINGFUTURE TRENDS

ONCE FOR ALL NETWORKS

Once For All (OFA) Networks have an efficient neural network design to handle many deployment scenarios, a new methodology that decouples model training from architecture search.

Instead of training a specialised model for each case, we propose to train a once-for-all network that supports diverse architectural settings.

DEEP LEARNINGONCE FOR ALL NETWORKS

QUESTIONS?LET’S TALK ABOUT AI :)

THANK YOU.

DR. ALPAN RAVALHEAD OF DATA SCIENCEWADHWANI AI

jerome@wadhwaniai.org