Presentation is loading. Please wait.

Presentation is loading. Please wait.

Interactive Evolutionary Computation Hideyuki TAKAGI Kyushu University

Similar presentations


Presentation on theme: "Interactive Evolutionary Computation Hideyuki TAKAGI Kyushu University"— Presentation transcript:

1

2 Interactive Evolutionary Computation Hideyuki TAKAGI Kyushu University

3 Part 1: Humanized Computational Intelligence 1. Humanized Computational Intelligence 2. What is IEC? 3. IEC Applications 4. IEC research for Reducing IEC User Fatigue 5. IEC for Human Science 6. New IEC framework

4 Computational Intelligence in the 20th Century

5 Cooperation of Computational Intelligence EC KE FS NN Powerful cooperative technologies were developed in the end of 20th Century.

6 What Comes Next?

7 Two Different Human Capabilities

8 explicit knowledge experience preference computational intelligence optimization auto-design implicit knowledge 1st Generation: 2nd Generation: 3rd Generation: Importance of Human Factors with Computational Intelligence

9 Humanized technology ROBOTICS, CONTROL DATA MINING SIGNAL PROCESSING natural environment human environment acquired knowledge physical measurement qualifying the knowledge human perception location, speed, obstacles, … preference, subjective evaluation emotion … database mining IF … THEN … filter designS/N ratio

10 NN FS EC NN+FS GA+FS NN+FS+GA GP:Koza (1980s) NN-driven Fuzzy Reasoning (1988) Karr et.al (1989) CI to be a competitor of humans CI as human models CI for human one research direction is: Direction of Computational Intelligence Humanized Computational Intelligence Interactive Evolutionary Computation

11 Part 2: What is IEC? 1. Humanized Computational Intelligence 2. What is IEC? 3. IEC Applications 4. IEC research for Reducing IEC User Fatigue 5. IEC for Human Science 6. New IEC framework

12 What is IEC ? Target System evolutionary computation subjective evaluation (1) fewer population size and fewer generation numbers, (2) nevertheless, satisfactory solutions can be obtained, i.e. simple landscape, (3) discrete fitness in n-evaluation levels, and (4) relative fitness in general. Features

13 Searching spaces of IEC tasks are generally simple, because Any searching points that human operators cannot distinguish are same for human.

14 Statistics of IEC Papers

15 IEC Takagi Lab (1) 3-D CG lighting design support (2) montage image system (3) speech processing (4) image processing (5) hearing-aid/cochlear implant fitting (6) virtual reality in robot control (7) media database retrieval (joint1) virtual aquarium (joint2) geoscientific simulation (joint3) 3-D CG modeling education (joint3) fireworks animation design (joint4) mental disease diagnosis (1) input interface 1.1 discrete fitness value input method (2) display interface 2.1 prediction of user's evaluation char's 2.2 display for time-sequential tasks (3) acceleration of EC convergence 3.1 approximation of EC landscape 3.2 IDE/gravity, IDE/moving vector (4) active user intervention to EC search 4.1 on-line knowledge embedding 4.2 Visualized IEC (5) IEC user model (6) New IEC frameworks application-oriented interface research IEC for Human Science (1) measuring human mind (2) finding unknown knowledge (3) IEC with physiological responces

16 IEC Research Categories art education writing education games and therapy social system speech processing hearing aids fitting virtual reality database retrieval data mining image processing control & robotics internet food industry geophysics graphic art & CG animation 3-D CG lighting design music editorial design industrial design face image generation discrete fitness value input method prediction of fitness values interface for dynamic tasks acceleration of EC convergence combination of IEC and non-IEC active intervention Visualized IEC Hideyuki Takagi, "Interactive Evolutionary Computation: Fusion of the Capacities of EC Optimization and Human Evaluation,” Proceedings of the IEEE, vol.89, no.9, pp (Sept., 2001).

17 Part 3: IEC Applications 1. Humanized Computational Intelligence 2. What is IEC? 3. IEC Applications 4. IEC research for Reducing IEC User Fatigue 5. IEC for Human Science 6. New IEC framework Hideyuki Takagi, "Interactive Evolutionary Computation: Fusion of the Capacities of EC Optimization and Human Evaluation,” Proceedings of the IEEE vol.89, no.9, pp (Sept., 2001).

18 Part 5: IEC for Human Science CONTENTS 1. Artistic Applications CG, lighting design, CG animation, montage, music, industrial design 2. Engineering Applications signal processing, robotics, VR, media DB retrieval, data mining, intrusion detection 3. Others geo-science, coffee blending, game & education

19 System designed by Tatsuo UNEMI Interactive GP for Graphic Art

20 Breeding Forms on the Infinite Plane by W. Latham (1992)

21 Cheerful impressionGloomy impression Heroine movie starWicked movie star by HAND by Interactive GA IGA for 3D CG Lighting Design By K. Aoki and H. Takagi

22 Result of Subjective and Statistic Tests 1% level of significance 5% level of significance cheerful impression IGA-based lighting design looks effective for beginners. heroine impression villain impression gloomy impression

23 Expanding Lighting Environments Thanks to Increasing LED Performance and Decreasing LED Costs Lighting environments expands thanks to Increasing LED performance and decreasing LED costs. Increasing the # of LED lights makes lighting design difficult.

24 IEC-based Lighting Design

25 Education of Imagination and Creativity for 3D Shape It is difficult and time consuming for computer beginners (artist, non-technical student etc.) to acquire 3D modeling skill. Apply IEC to support CG skill and focus on educating imagination and creativity.

26 3D Model Example blend deform P2P2 P3P3 P1P1 P5P5 P4P4 P5 P4 P3P6P5 P4 P3P6 P1P2P1P2 IEC changes shape parameters based on user's imagination Nishino et al. (SMC1998, SMC1999)

27 IEC-base 3D Modeling Concept designer 3D shape image to create rate each shape :very good :good :not good Genetic Algorithm create new 3D geometries by inheriting highly rated shapes and simulating natural evolutionary processes like crossover and mutation display a set of new shapes Nishino, Takagi, and Utsumiya (2000)

28 Subjective Test Realistic Examination Creative Examination ferocious green pepper Quality Operability Manual >> IEC Manual > IEC Manual = IEC Manual < IEC Only 48 (8 parameters ×6 primitives) parameters are modified by GA

29 Internet version of IEC 3-D Modeling Education System car designers students artists IEC-based Modeling Interface CG System for Car Body Design CG System for Education CG System for Artwork Creation specify rates based on subjective preference explore CG parameters by evolutionary computation visualize 3D models by explored parameters network display generate d models

30 Evolution of Funny Animated Figures by Jeffrey Ventrella

31 CG Fireworks Animation Made by IEC

32 Montage Image by C. Caldwell and V.S. Johnston (1991) target face montage face three days later, 10 GA generations

33 by H. Takagi and K. Kishi (KES1999) Montage Image

34 IGA for Music by J. A. Biles (1994) GenJam: Jazz Solo Generator Rhythm Generator by D. Horowithz (1994) Sonomorphs: Melody Generator by G. L. Nelson (1993)

35 Industrial Design by H. Furuta

36 HTML Design EC font of letters, colors of letters and background, and etc... Looks nice ! by N. Monmarche et al.

37 Changing Colors and Fonts with IEC Making Layout with IEC Stability 1.0, Power 0.5, etc.... Vigor 1.0, etc... Completed Poster By T. Obata and M. Hagiwara Color Poster Design Colors and fonts are VIGOR, and layout is STABILITY. Colors and fonts are CLEAN, and layout is ELEGANT.

38 Q1Q2 R1 R2 EC Visitors who click the banner by R. Gatarsk Evolutionary Banner

39 Part 5: IEC for Human Science CONTENTS 1. Artistic Applications CG, lighting design, CG animation, montage, music, industrial design 2. Engineering Applications signal processing, robotics, VR, media DB retrieval, data mining, intrusion detection 3. Others geo-science, coffee blending, game & education

40 Why IEC-based Signal Processing? There are many cases that SP users are not SP experts but need to design SP filters. image filter hearing aid ? ? ? ? ? ? ? ? Solution is auditory-based SP and visual- based SP without any SP knowledge. IEC realizes this approach.

41 Recovering Distorted Speech Distorted Speech Watanabe and Takagi (SMC2005)

42 Experimental Result -- Method of Successive Categories -- distance scale Ideal Situation : GA filter design when ideal speech is given. Sheffe’s method of paired comparison showed that this difference is significant (p<0.01).

43 Experimental Result Sheffe’s Method of Paired Comparisons difference is significant (p<0.01)

44 How Distorted Speech was Recovered? Formant area was mainly recovered

45 Voice Conversion by Interactive EC by Y. Sato et al.

46 Example of Prosodic Modification by Y. Satoh, et al Original VoiceModified by IEC Text-to-SpeechIntelligible voice Text-to-Speechmale-like voice Intelligible voice real voicechild-like voice real voice

47 By T. Morita, S. Iba and M. Ishizuka IEC for Agent’s Voice Design EC evaluation Prosody control

48 Hearing Compensation is Difficult

49 cognitive level perceptual level sensory level preference final hearing input sounds measurable in part target goal Ideal Goal is Far

50 hearing aid Conventional EC hearing aid Proposed Tuning System based on How User hears Ohsaki and Takagi ( )

51 Evaluation evaluationIEC Fitting vs. audiologist fitting monosyllable articulation sound quality fitting timeIEC < audiologist One – Two Weeks Later (5 subjects) Six Months Later (4 subjects) evaluationIEC Fitting vs. audiologist fitting sound quality APHAB

52 Visualized IEC Fitting on a PDA Visualized EC: parameters in an n-D EC landscape are mapped on a 2-D space for visualization. IEC Fitting: IEC-based hearing aid fitting Hayashida and Takagi (IECON2000)

53 Image Enhancement using Interactive GP by R. Poli and S. Cagnoini Image enhancement filter evolves according to how processed images look well.

54 Echo-Cardiographic Image by R. Poli and S. Cagnoni (1997)

55 Non-linear Density Transformation by Using of GP linear density transformation non-linear density transformation by using GP ex.invert g(i,j) output input g(i,j)= G ( f ( i, j ) ) generated by GP output density input density o n f(i,j) n output density input density o n f(i,j) n

56 GP-Based Image Filter by Using Neighborhood Pixels Information input pixelsoutput pixels operator G g(i,j) = G(f) x y x y GP generates G( ) zoom in G=max[abs(f(i,j-1)),max{-0.41,0.79+sin(min(f(i+1,j-1),-0.03))-f(i+1,j+1)}]

57 Experiment of IGP-Based Edge Detection input image Laplacian filter 2nd generation conventional math-based filter IEC-based filter

58 high-pass filter6th generation Experiment of IGP-Based High Pass Filter Design input image conventional math-based filter IEC-based filter

59 x-axis pass filter input image Prewitt4th generation conventional math-based filterIEC-based filter IGP-Based X-component Extraction

60 6th generation Prewitt input image y-axis pass filter conventional math-based filter IEC-based filter IGP-Based Y-component Extraction

61 IEC-based Color Filter Design input image examples of coloring

62 Image Enhancement Filter Design by a Medical Doctor original ultrasonic image of a lymph node Enhanced Image after 12 IEC generations

63 Consumer Robots Business Has Been Launched Home robots will come soon.

64 Human Friendly Trajectory Control of a Robot Arm by N. Kubota, K. Watanabe and F. Kojim

65 by H. H. Lund, O. Miglimo, L. Pagliarini, A. Billard, and A. Ijspeert NN Controller for LEGO Jeep-Robot 1.Children want to make a robot avoiding obstacles. 2.Children cannot make a program of its controller but can choose better robot moving. 3.Let's evolve the robot controller according to the children's choice.

66

67 Interactive EC for Virtual Reality Arm wrestling: human vs. robot controller fuzzy control rules for VR IEC subjective evaluation for VR modifying rules Kamohara, Takagi, and Takeda (SMC2007)

68 IEC for Dental Education in VR How to realize the reality of VR force feedback? VR dental education software elianealhadeff.blogspot.com/2009/05/dental-serious-games-with-3d-touch.html

69 Media DB Retrieval & Media Converter My impression is … by H. Takagi et al.

70 IEC-based Image DB Retrieval by S.-B.- Cho, et al.

71 Interactive EC for Data Mining by T. Terano, Y. Ishino, et al.

72 Visual Data-mining Through 2-D Mapping by GP X = (x1-u) / (87.3 / x12)Y = (46.7 * x6) * ( ) for example, by Venturini

73 Intrusion Pattern Generations for Intrusion Detection System IDS How to evaluate an intrusion Detection system? Ja-Min KOO and Sung-Bae CHO (2004) virus illegal attack computer virus How to generate several new types of intrusion patterns for evaluation? virus

74 J. Budynek, E. Bonabeau, and B. Shargel (2005) script individuals gene pool Generating Intrusion Patterns for Intrusion Detection Systems

75 crossover mutation deletion J. Budynek, E. Bonabeau, and B. Shargel (2005) Generating Intrusion Patterns for Intrusion Detection Systems

76 Intrusion Pattern Generations for Intrusion Detection System Intrusion Detection System Ja-Min KOO and Sung-Bae CHO (2004) generated intrusion patterns + GA detection performance code evaluation 73% of generated executable intrusion patterns were different.

77 Part 5: IEC for Human Science CONTENTS 1. Artistic Applications CG, lighting design, CG animation, montage, music, industrial design 2. Engineering Applications signal processing, robotics, VR, media DB retrieval, data mining, intrusion detection 3. Others geo-science, coffee blending, game & education

78 Geological Modeling Based on Interactive EC

79 Difficulty many variables –strength, depth, stress, etc. no numerical target –Numerical similarity may not mean qualitative similarity, such as wrong fault inclination, wrong depth extent. –Help of geologists is necessary. computational optimisation with human judgement –Computational optimisation methods to search material parameters are required besides geologist’s judgement.

80 sketch drawn by a geologist geological forward model EC evaluation based on geological knowledge and experience material parameters IEC-based Modelling of Extension of the Earth’s Crust Wijns, Moresi, Boschetti, and Takagi (SMC2001)

81 IEC-based Modelling of Subduction of Oceanic Crust animations geological forward model EC evaluation based on geological knowledge and experience material parameters Strength of continental/oceanic crust Convergence rate Depth of sedimentary wedge

82 Mocha Kilimanjaro Colombia ?% ?% ?% Target Brended Coffee IEC Fitness Value Mixture Ratio Evolving Blends of Coffee by M. Hardy

83 Simulated Breeding for Composition Support System by K. Kuriyama, T. Terano, and M. Numao Two sequences of four scenes are chosen as parents. Two better sequences are chosen as parents. Composition and comparison with similar composition.

84 Part 4: IEC research for Reducing IEC User Fatigue 1. Humanized Computational Intelligence 2. What is IEC? 3. IEC Applications 4. IEC research for Reducing IEC User Fatigue 5. IEC for Human Science 6. New IEC framework

85 Approaches for Reducing IEC User Fatigue (1) input interface 1.1 discrete fitness value input method (2) display interface 2.1 prediction of user's evaluation char's 2.2 display for time-sequential tasks (3) acceleration of EC convergence 3.1 approximation of EC landscape 3.2 IDE/gravity, IDE/moving vector (4) active user intervention to EC search 4.1 on-line knowledge embedding 4.2 Visualized IEC (5) IEC user model (6) New IEC frameworks

86 Part 5: IEC for Human Science 1. Humanized Computational Intelligence 2. What is IEC? 3. IEC Applications 4. IEC research for Reducing IEC User Fatigue 5. IEC for Human Science 6. New IEC framework

87 Part 5: IEC for Human Science CONTENTS 1. measuring dynamic range of happy-sad scale 2. finding unknown psychophysiological facts 3. extended IEC: IEC with physiological responses

88 Psychotherapy / Diagnostics IEC-based CG Lighting Design mental patient (schizophrenics) sad happy sadhappy emotion range of mentally healthy persons emotion range of the patient psychotherapist psychiatrist Umm, still his range is narrower. Takagi, Takahashi, and Aoki (SMC2004)

89 PM PK NH NS NY NK happy impression NN PT happy scale

90 NY NK NN NS NH PM PK PT sad impression sad scale

91 Discussion obtain the order scale of "happy - sad" expression range from psychological scales of happy and sad obtained through a subjective test NS NN NK PM NY PT NH PK 1st 5th 2nd 3rd 4th 6th 7th 8th sadhappy sadhappy

92 Part 5: IEC for Human Science CONTENTS 1. measuring dynamic range of happy-sad scale 2. finding unknown psychophysiological facts 3. extended IEC: IEC with physiological responses

93 Cochlear Implants Fitting P. Legrand, C. Bourgeois-Republique, et al. conventional fitting E1E4E5E2E3E6E7E9E8E13E14E electric channels intensity IEC fitting evaluation fitting 45% 91.5% (IEC fitting) 48.5% (practitioner’s result)

94 IEC Breaked Common Sense? Setting all electrodes but E1,E7, E9 under the liminary value gives 82.1% Adding electrodes E4 and E15 gives 58.8% !!! More electrodes  better audition! E 1E 2E 3E 4E 5E 6E 7E 8E 9E 13E 14E 15 Only 3 electrodes (E1, E7, E9) have 3 a significant C-T range, but the best 91.5%! Slide made by Prof. Pierre Collet

95 Obtained Knowledge from Analysis of IEC Fitting Loudness charact. tuned by speech (IEC fitting) ≠ fitting using speech1 w/wo noise fitting using speech2 w/wo noise fitting using speechN w/wo noise ・・・・ fitting using music w/wo noise ≠ differences are small (1) (2) The best hearing characteristics of human seems to depend on sounds. EC hearing aid Loudness charact. tuned by noise or pure tones (conventional fitting) M. Ohsaki and H. Takagi (1999)

96 Part 5: IEC for Human Science CONTENTS 1. measuring dynamic range of happy-sad scale 2. finding unknown psychophysiological facts 3. extended IEC: IEC with physiological responses

97 Emotion Enhancement Filter Example Application physiological responses control physical features Emotion Control with 3 Steps IEC with Physiology H. takagi (2004)

98 Step 3: Conventional IEC Based on Psychological Feedback EC image filters Conventional subjective evaluation IEC = system optimization technique based on human subjective feedback

99 Step 3: Extended IEC Based on Physiological Feedback Step 3: Extended IEC Based on Physiological Feedback EC image filters Proposal - physiological measurement given target physiological conditions

100 Part 6: New IEC framework 1. Humanized Computational Intelligence 2. What is IEC? 3. IEC Applications 4. IEC research for Reducing IEC User Fatigue 5. IEC for Human Science 6. New IEC framework

101 Part 6: New IEC framework CONTENTS 1. IEC + EMO 2. Interactive PSO and interactive DE 3. paired comparison-based IEC

102 Multi-Objective Optimization with Subjective Object(s) I'm looking for an apartment that is (1) wider, (2) less expensive, and (3) closer to a station. width tenant width distance to st. tenant distance to st. Then, how to find an apartment (4) with great view (5) in a quite area?

103 Multi-Objective Optimization: MEMS Design joint research with UC Berkeley MEMS (Micro Electronic Mechanical Systems) for Sensors, Robotics, Communications, Biotechnology, Energy Generation Multi-objective optimization for given specification: receiving frequency, strength, and others. We want to use domain knowledge for circuit design.

104 Raffi Kamalian et al. (2004) EMO: four objects: 1.Resonant freq = 100kHz 2.K_vertical = 1 N/m 3.K_lateral = 100 N/m 4.Area = minimized IEC: embedding domain knowledge for MEMS circuit design IEC + EMO for MEMS Design

105 Subjective Test + Statistic Test for EMO+IEC vs. EMO Subjective Test + Statistic Test for EMO+IEC vs. EMO EMO+IEC > EMO (99% significant by Wilcoxon matched-pairs signed-ranks test.)

106 IEC + EMO for Architectural Design

107 106 Algorithm for Generating Layout

108 107 IEC +EMO for Room Layout -- Algorithm for Generating Layout -- IEC +EMO for Room Layout -- Algorithm for Generating Layout -- Makoto Inoue et al. (2008) wall

109 Layout Algorithm objective 1 objective 2 EC EMO IEC IEC +EMO for Room Layout 1.room sizes 2.room shapes 3.moving line to any rooms without passing through other private rooms 4.window sizes requested by architectural laws 5.total water pile length 6.total wall length

110 Part 6: New IEC framework CONTENTS 1. IEC + EMO 2. Interactive PSO and interactive DE 3. paired comparison-based IEC

111 132 Particle Swarm Optimization

112 GAPSO F1○ F2○ F3○ F4○ F5○ F6 ≒ F7○ F8○ Comparison of Performance GA vs. POS and IGA vs. IPOS (○ : faster ,≒: equivalent ) complexity high low IGAIPSO ○ ○ ○ ○ ○ ○ ○ ○

113 levels 10-levels 100-levels 1,000-levels quantization noise in fitness Tolerance of PSO for Quantization Noise in Fitness fewer more

114 Interactive PSO levels Experimental Result for F1 Interactive GA F2 ~ F8 has similar tendency

115 IGA IPSO IPSO with proposed methods generations fitness After Implementing Four Methods for Increasing Noise Tolerance Nakano and Takagi (CEC2009)

116 Part 6: New IEC framework CONTENTS 1. IEC + EMO 2. Interactive PSO and interactive DE 3. paired comparison-based IEC

117 Pair Comparison: Tournament IEC Ordinary IEC

118 Paired Comparison-based Interactive Differential Evolution Paired Comparison-based Interactive Differential Evolution individuals target vector trial vector target vector parameter vector 1 parameter vector 2 base vector mutant vector F Takagi and Mitsuyasu (patent) Takagi and Palles(CEC2009)

119 IGA vs. Paired Comparison-based IDE GA HA 1 HA 2 HA 1 DE ON HA 5HA 6 HA 9HA 10HA 11HA 12 HA 13HA 14HA 15HA 16 HA 3 HA 8 HA 2 HA 7 HA 4

120 CONCLUSION One of IEC research directions is Humanized Computational Intelligence, and IEC is one of tools for this direction. IEC has high applicability in wide variety of areas. IEC research includes research reducing IEC user fatigue and research for human science besides major IEC research, IEC applications. We also introduced recent new IEC research on extending IEC algorithms and frameworks.

121 Further Information IEC tutorial & survey paper –Hideyuki Takagi, "Interactive Evolutionary Computation: Fusion of the Capacities of EC Optimization and Human Evaluation,” Proceedings of the IEEE vol.89, no.9, pp (Sept., 2001). –Its draft paper is downloadable from the below URL. IEC slides are downloadable from the below URL. Personal Contact –http://www.design.kyushu-u.ac.jp/~takagi


Download ppt "Interactive Evolutionary Computation Hideyuki TAKAGI Kyushu University"

Similar presentations


Ads by Google