document digitization - qcon.ai · @nischalhp | document digitization | qconai sfo 2019 human and...

DOCUMENT DIGITIZATIONRethinking it with Machine Learning

Nischal Harohalli Padmanabha QConAI SFO 2019

@nischalhp | Document Digitization | QconAI SFO 2019

“The brain sure as hell doesn’t work by somebody programming in rule.”

- Geoffrey Hinton

PROBLEM


Understanding unstructured documents and extracting semantic information to automate claims handling.


DOCUMENT CLASS

Policy

POLICY NUMBER

H 54/16 307 728

CUSTOMER

Renolate GmbH10115 Berlin

AGENT

pma Insurance Broker48149 Nurnberg

RISK DESCRIPTION / INSURED LOCATION

Private liability insurance comfort plus Dog liabilityEnvironmental damage insuranceEmployees on premises

POLICY

Liability Protection

EFFECTIVE DATE OF CHANGE

22.12.2016 12:00TERMINATION

22.12.2019 12:00ANNUAL CHARGE

EUR 424,63

COVERAGES

Persons & property damage flatFinancial lossesEnvironmental damage basic flat

EUR 3.000.000EUR 100.000EUR 3.000.000

REWIND

TABULAR INFORMATION EXTRACTION


Writing a lot of rules

COURSE OF ACTION - ROUND 1

Initial results, gave us a lot of happiness. Evaluation on known Data

RESULT


In production 58% accuracy

RESULT


We failed, miserably.Rules became cumbersome & brittle.



Life or death situation for the project (and us engineers)


ADAPTIVE LEARNING THOUGHT PROCESS

How does a human solve the same problem?

Identifies Grouping of Text, to build Context

Eg: Tables, paragraphs, passages Given the context, domain knowledge and semantic understanding of text


Sounds straightforward, right?


TECH STACK CHECK

NEXT STEPS


Which algorithms to use?

What should we feed as input to the algorithm?What to annotate?

What are our deadlines?

Human and computation resources required?

How to agile this?


Which algorithms to use?


Supervised Learning

Unsupervised Learning

Computer Vision

NLP

Computer Vision

NLP

Using this technique to generate data for supervised training. Wrote implementations of Deep clustering, word / sentence / page / document embeddings

● Object detection● Messaging parsing networks● Custom CNN networks

● Implementation of Deep Topic modeling● Custom RNN + CNN networks with

domain adaptation

EMPHASIS ON SUPERVISED LEARNING



Computer Vision

NLP

● Drawing polygon bounding boxes● Labeling pages● Labeling documents

Complex annotation of passages, phrases, tables, line items, hierarchy nature of textual information

What should we feed as input to the algorithm?What to annotate?

]Built an in houseAnnotation System


Workflows supporthuge annotation jobs


Human and computation resources required?

Data Scientists

Engineers

● Data Scientists from Academia● Deep learning engineers● Research programme with Universities● Master Thesis sponsorship at omni:us

● Full stack engineers● Data Engineers● Devops

Leadership & Mentors

Cloud startup programmes

● Team leads with experience in AI● Identifying and convincing industry experts to mentor● Devops

● Credits to support memory and GPU training algorithms● Mentoring to scale operations



What are our Deadlines?

How to agile this?

Sprint Planning for Research

Quick turn around of POC

Engineer AI systems to run experiments in a systematic and automated way


RESULT



Successful AIdelivery


TECH STACK CHECK

GO LIVE OR GO HOME


Trained Models Predict

AI IN PRODUCTION

Human in the loop, fixes the errors and validates corrections

Train on the corrections, Continuous improvements


DO NOT IGNORE

Domain Knowledge is essential

Educate your customers on AI

Engineer end to end AI systems to solve business use case, not a dataset


PLATFORM

Training Platform Prediction Platform with human in the loop

Management Console of Infrastructure, Applications & Users


Training Platform


Annotation System

Ability to train and evaluate models

Mechanism and system to trigger training, retraining of evaluation and versioning of different types models, in a managed way across various infrastructures supporting CPU and GPU

System to define data models, annotate data, manage annotation jobs, audit the annotated data and version control the datasets]Console connecting

the two together



Async API for Ingestion

Data PipelinesRobust data pipelines connecting the services with providing capabilities of high throughput, reliability and retry mechanisms.

Rest API that supports asynchronous data upload capabilities ]Prediction console

connects all.

Prediction Platform with human in the loop

Validation UI

AI microservices

User interface to fix prediction errors

Scaling deep learning models as microservices


Management Console of Infrastructure, Applications & Users


Configuration management

Application logsMonitoring logs of applications and setting up dashboards for internal and external stakeholders

Central management of configuration of various systems, consoles and services ]Management and

monitoring console

User management

Infrastructure logs

Managing users and providing authentication and authorisation capabilities for services.

Monitoring infrastructure usage and patterns to setup alerts and notifications


TECH STACK CHECK


omni:us platform console |

Learnings

Learnings


● Very important for an entire organization to believe that AI can solve problems● Engineer AI products, do not believe that having just AI models are good enough● Agile for AI works, choose an interpretation that works for your team● Pay attention to details, domain knowledge and use case to be solved. ● Combination of multiple technologies have to be used to solve use case, not just one

hammer for all.● Do not try to “AI” everything, certain matured technologies are capable of solving

certain problems well. Use them wisely.● Believe in human in the loop, builds trust with business● Educate internal and external stakeholders around the possibilities and limitations

of AI.● Visualisation is power tool to understand and explain AI to everybody. Use them.● AI is no more a black box, it can fine tuned, managed and configured appropriately.● Automate your current processes as much as possible, this gives more room for

research.

document digitization - qcon.ai · @nischalhp | document digitization | qconai sfo 2019 human and...

Documents