Clients/Faculty Advisors Dr. Eric Bartlett May01-14 Team Members David Herrick Brian Kerhin Chris Kirk Ayush Sharma Incremental Learning With Neural Networks February 1, 2001 Introduction

Problem Statement (Ayush) Design Objectives (Ayush) Intended Users (Ayush) End-Product Description (Brian) Assumptions/Limitations (Brian) Project Risks & Concerns (Brian) Technical Approach (Brian) System Overview (Dave) Technical Design (Dave) Evaluation of Project Success (Chris) Possible Future Work (Chris) Human/Financial Budget (Chris) Lessons Learned (Chris) Closing Summary (Chris) Overview

Real neurons are decision making cells in the brain Software neurons function similarly, to make decisions in software The brain is a network of “Real neurons” making decisions in parallel ANNs accomplish the same goal using “Software neurons” ANNs are used to interpolate nonlinear systems that are very complex Predict trends in the Stock Market Predict trends in power use through out the year Predict trends in global whether patterns ANN Description =

Unlike humans, who incrementally learn information as it is introduced, ANNs learn all at once. An existing ANN cannot adapt to dynamically changing system (i.e. CATASTROPHIC FORGETTING). Hence, as a system changes, a new ANN must be created from scratch. For many real- life applications of ANNs, it is impractical to regularly create replacement ANNs. Problem Statement

Create a software design document Create an incrementally learning ANN Create a GUI for incrementally learning ANN Apply to a power load problem and compare to traditional ANNs Design Objectives

Hardware Intel Gateway PCs in Adaptive Computing Laboratory Software Microsoft Windows NT4.0 Microsoft Visual C++ Microsoft Visual Basic Hardware and Software (Supplies) Operation and Construction Environment

Adaptive Computing Laboratory Dr Eric Bartlett Research Assistants Other Neural Network Programmers Intended Users

End-Product Description Incrementally learning ANN to model dynamic nonlinear systems Learning System can learn new data without forgetting Has goodness measures GUI operated, via command line

User should have a basic knowledge of neural networks Data file will be tab delimited following Dr Bartlett’s specified file format Assumptions

Set time period to produce code and documents Number of developers is not based on commercial properties of the software Computationally intensive technique of problem solving Limitations

Major accidents can (and have) cripple team members Finishing on time Incremental learning may not improve goodness measures or error results Project Risks & Concerns Project Risks and Concerns

Technical Approach (Design Alternatives) Technical Approach One strong neural network with augmenting neural nets Several augmenting neural nets Update a single augmenting neural net Produce result based on outputs average Produce result based on output sums Multiple weak neural nets average results

Incremental Learning System System Overview

Technical Design

Technical Design (cont.) Technical Design

Incremental Learning System System Overview

1 st Semester Milestones Project Plan (Fully Met) Project Poster (Fully Met) Design Report (Fully Met) Hardware Requirements (Fully Met) Software Requirements (Fully Met) Software Design (Fully Met) Evaluation of Project Success

2 nd Semester Milestones Software Implementation (Partially Met) 100% of algorithm designed 10% of algorithm implemented Final Implementation (Not Met) Final Report (Not Met) Presentation for Industrial Review Panel (Partially Met) 80% of Presentation Completed Evaluation of Project Success (cont.)

Explore alternative learning styles More research tools in the field of artificial intelligence Recommendations for Further Work Possible Future Work

Human Budget Personnel Estimated Total (1 st + 2 nd Semesters ) Actual (to date) Ayush 125 hours 86 hours Brian 115 hours 73 hours Chris 140 hours 91 hours Dave 130 hours 88 hours TOTAL 510 hours 334 hours

Financial Budget ItemsEstimatedActual Poster$50$35 Hardware$0$0 Software$0$0 Books$100$0 Total$150$35

Lessons Learned Establish two weekly meetings Progress meeting with faculty advisors Development meeting with team members Wounded team members don’t improve group efficiency Keep faculty advisors well-informed of progress and seek feedback Plan to complete milestones ahead of schedule Balance workload among team members Lessons Learned

Closing Summary New way of looking at neural networks Overcomes limitations of traditional neural nets Can be greatly reused and built upon Will further the field of Artificial Intelligence Closing Summary

Questions?