automating programming via concept mining, probabilistic reasoning over semantic knowledge base of...

Automating programming via concept mining, probabilistic reasoning over

semantic knowledge base of SE domain

by Max Talanov

A lot of trivial tasks that could be automated Add/remove field Patronymic on Customer page

Add dropdown list on the form...

A lot of not so trivial solutions that should be reused but are not reused How-tos

Libraries...

Approximately 60% of developer's time in outsourcing is spent to solve this kind of problems

</ Problem

Template based code generators: IDEs

Visual studio. IDEA. ...

Template based generators Maven

Archetypes

CASE Tools: Rational Rose

ArgoUML

</ Current Solutions

Once generated solution is hardly maintainable and require to allocate significant amount of money for farther support. Developer generated the solution based on DB structure,

then added some functionality in it, then customer wants to change the DB structure.

Developer has to regenerate and merge his further changes

This is done for only one reason – generator does not understand what it's doing.

</ They cannot solve problem

2005 MIT media lab published the article “FEASIBILITY STUDIES FOR

PROGRAMMING IN NATURAL LANGUAGE”.

Metafor is the program that creates the sceleton of the Python classes based on

shallow English descripton.

</ MIT Metafor

Input: shallow English description

Output: scaffolding Python classes.

Metafor utilised natural language processor Montylingua, common sence KB ConceptNet, programmatic interpreter.

</ MIT Metafor

System should:

Operate with changes to be applied not the static structure of the target application (How-to).

Use domain knowledge model to map the inbound requirements into acceptance criteria to be used to create the solution.

Use both trained data and generation to create the solution.

Use several abstraction layers of the target application.

</ Key Ideas

System has to operate with knowledge.

System has to understand what it is doing Architecture of the target application.

Methods to change the architecture.

Domain specific information.

Requirements for changes.

System has to understand the human operator Communicate in natural language.

</ Requirements

Knowledge base.

Linguistic component.

Perceiving component.

Solution generator.

Communication component.

</ Key Components

LinguisticReq PerceivingSolution

generator

Communicator

Updatedapp

Request

KB

</ Collaboration Diagram

Is the main storage of the data to be used by the system.

KB is RDF storage with OWL data.

KB is used to store the semantic information: Target application architecture

Domain specific knowledge (How-tos)

Common sense information

Predicates generated based on requirements text

Acceptance criteria for generated solution

...

</ Knowledge Base

Human expert specifies the requirements, linguistuc component generates set of predicates for further processing.

Inbound: CR, bug report or FRS according to SE standard SPICE.

Outbound: set of predicates.

Stanford Parser creates the set of predicates that are treated as inbound knowledge.

LinguisticRequirements

</ Linguistic Component

Perceiving module maps inbound predicates to the model (domain model) in knowledge base, using trained data and stochastic search generation. In case of failure invokes Communicator to generate clarification request.

Inbound: set of predicates and domain knowledge model.

Outbound: predicates mapped to domain knowledge model.

Linguistic Perceiving Solution generatorPredicates Updated

model

</ Perceiving Component

Selects or generates the solution for the specified acceptance criteria in updated model. Provides the updated application with the confirmation request to human expert.

Inbound: acceptance criteria in KB

Outbound: solution in actual code.

UpdatedappPerceiving Solution

generatorUpdated

model

</ Solution GeneratorComponents

Reasoner. Reasoner interface.

Genetic generator. Solution checker.

Trainer. Associator.

Generalizer.

Analogy detector.

Target language translator.

</ Solution GeneratorComponents

</ Solution GeneratorActivities

Analogy Detector:retrieves solution for similar acceptancecriteria

Solution checker: returns solution assessment

Genetic generator:generates new solution

Communicator:sends confirmation request to human expert.

Communicator: analyses reply of human expert

Trainer:run

Solutuon found?

Solution ok?

Solution ok?

yes

yes

yes

no

no

no

For the purpose of making a logical inference for axepted alternatives within environment of possible contradictions and several probable variants, we decided to use probabilistic reasoner NARS.

NARS main features: Deduction

Induction

Analogy

...

</ Reasoner

Inbound: Acceptance criteria.

Outbound: Solution in form of How-tos sequences.

Generator is capable of creation of the sequences of How-tos according to the acceptance criteria. Inference is produced by NARS.

This could be interpreted as human imagination mechanism.

</ Genetic Generator

Is devoted to logically infer the percentage of how good is generated solution according to acceptance criteria.

Solution checker mainly relies on NARS probabilistic mechanisms, but collects all proper information from KB to be processed by reasoner.

</ Solution Checker

Machine learning component, is used to detect associations and infer generic associations of inbound acceptance criteria and approved solutions.

Analogy detector is used to retrieve previously learned associations that could be used for specified acceptance criteria.

</ Trainer

Translates knowledge representation of architecture in actual files in target language, based on previously described syntax in KB.

</ Target LanguageTranslator

Component is dedicated to generate requests to human expert and analyse replyes of the expert.

Perceiving Solution generator

CommunicatorRequest

</ Communicator

</ Feedback Loops

LinguisticReq Perceiving Solution generator

Communicator

Updatedapp

Request

KB

</ Current Implementation

Acceptancecriteria

Solution generator

Communicator

Updatedapp

Request

KB

Add evolution mechanism.

Add inbound information analysis.

Add architectural analysis.

Add self optimization and self improvement.

Extend perceiving algorithm to use encyclopaedic resources to extend domain knowledge.

</ Future Plans

Metafor:http://web.media.mit.edu/~lieber/Publications/Feasibility-Nat-Lang-Prog.pdf

Maven:http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

Stanford Parser:http://nlp.stanford.edu/software/lex-parser.shtml

Open NARS:http://code.google.com/p/open-nars/

Menta: http://code.google.com/p/menta/

</ References