Concept mining for programming automation

Problem ➲ A lot of trivial tasks that could be automated – Add field Patronim on Customer page. – Remove field Patronim from Customer page. – Add drop down list on the form. –... ➲ A lot of not so trivial solutions that should be reused are not reused – How-tos. – Libraries. –...

Current solutions ➲ Template based code generators: – IDEs Visual studio. IDEA. – Maven Archetypes.

They can not solve the problem ➲ Once generated solution is hardly maintainable  Developer generated the solution based on DB structure, then added some functionality in it, then customer wants to change the DB structure.  Developer has to regenerate and merge his further changes ➲ This is done for only one reason generator does not understand what's it's doing.

MIT Metofor 2005 MIT media lab published the feasibility test for Metafor the program that creates the sceleton of the Python classes based on shallow English descripton

MIT Metafor Input: shallow English description Output: scaffolding Python class. Metafor utilised natural language processor Montylingua, common sence KB ConceptNet, programmatic interpreter.

Requirements ➲ System has to operate with knowledge. ➲ System has to understand what it is doing  Architecture of the target application.  Methods to change the architecture.  Domain specific information.  Requirements for changes. ➲ System has to understand the human operator  Communicate in natural language.

Key ideas ➲ Operate with changes to be applied over current architecture not the static structure of classes. ➲ Use domain knowledge model to map the inbound requirements into acceptance criteria to be used to create the solution. ➲ Use both trained data and generation to create the solution. ➲ Use several abstraction layers of the target application.

Key components ➲ Knowledge base. ➲ Linguistic. ➲ Perceiving. ➲ Solution generator. ➲ Communication.

Collaboration diagram Linguistic Req PerceivingSolution generator Communicator Updated app Request KB

Knowledge base Is the main storage of the data to be used by the system. KB is RDF storage with OWL data. OWL(web ontology language) W3C recommendation. KB is used to store the semantic information: Target application architecture Domain specific knowledge (How-tos) Common sense information Predicates generated based on text Acceptance criteria for generated solution...

Linguistuc component Human expert specifies the requirements, linguistuc component generates set of predicates for further processing. Inbound: is CR, bug report or FRS according to SE standard SPICE. Outbound: set of predicates. Stanford Parser creates the set of predicates that are treated as inbound konwledge. Linguistic Requirements

Perceiving component Perceiving module maps inbound predicates to the model (domain model) in knowledge base, using trained data and stochastic search generation. In case of failure invokes Communicator to generate clarification request. Inbound: set of predicates and domain knowledge model. Outbound: predicates mapped to domain knowledge model. LinguisticPerceivingSolution generator PredicatesUpdated model

Solution generator component Selects or generates the solution for the specified acceptance criteria in updated model. Provides the updated application with the confirmation request to human expert. Inbound: acceptance criteria in KB Outbound: solution in actual code. Updated app PerceivingSolution generator Updated model

Solution generator components ➲ Reasoner.  Reasoner interface. ➲ Genetic generator.  Solution checker. ➲ Trainer.  Associator.  Generalizer.  Analogy detector. ➲ Target language translator.

Solution generator activities AnalogyDetector: retrieves solution for similar acceptance criteria Solution found? Solution checker: returns solution assessment Genetic generator: generates new solution Communicator: sends confirmation request to human expert. Communicator: analyses reply of human expert Solution ok? Trainer: run Solution ok?

Reasoner For the purposes of logical inference in the environment of possible contradictions and several probable variants, we decided to use probabilistic reasoner NARS. NARS main feautures: – Deduction – Induction – Analogy –...

Genetic generator Inbound: Acceptance criteria. Outbound: Solution in form of How-tos sequences. Generator is capable of creation of the sequences of How-tos according to the acceptance criteria. Inference is produced by NARS. This could be interpreted as human imagination mechanism.

Solution checker Is devoted to logically infer the percentage of how good is generated solution according to acceptance criteria. Solution checker mainly relies on NARS probabilistic mechanisms, but collects all proper information from KB to be processed by reasoner.

Trainer Machine learning component, is used to detect associations and infer generic associations of inbound acceptance criteria and approved solutions. Analogy detector is used to retrieve previously learned associations that could be used for specified acceptance criteria.

Target language translator Translates knowledge representation of architecture in actual files in target language, based on previously described syntax in KB.

Communicator Component is dedicated to generate requests to human expert and analyse replyes of the expert. PerceivingSolution generator Communicator Request

Feedback loops Linguistic Req PerceivingSolution generator Communicator Updated app Request KB

Current implementation Acceptance criteria Solution generator Communicator Updated app Request KB

Future plans ➲ Add evolution mechanism. ➲ Add inbound information analysis. ➲ Add architectural analysis. ➲ Add self optimisation and self improvement. ➲ Extend perceiving algorithm to use encyclopedia resources to extend domain knowledge.

References Metafor: Maven: Stanford Parser: Open NARS: