1. Artificial Immune Systems: A New Computational Intelligence Approach
New Trends in Intelligent Information Processing
and Web Mining.
Zakopane, Poland, June 2-5, 2003
Jonathan Timmis
Computing Laboratory
University of Kent
CT2 7NF. UK.

2. Novel paradigms are proposed and
accepted not necessarily for being faithful
to their sources of inspiration, but for
being useful and feasible

3. What do I want to achieve?
Give you a taster of what AIS is all about
Define an AIS
Why do we find the immune system useful?
Explain what AIS are
Show you where they are being used
Some high level case studies
Comments for the future
I won’t
Talk about all areas of AIS and applications
Talk too much about how AIS relate to other bioinspired ideas (although I will mention it)
Go into too much detail: this is an introduction

4. Outline What are AIS? Useful immunology Thinking about AIS
Application Areas and Case Studies
The Future

5. Why the Immune System? Recognition Robustness Feature extraction
Anomaly detection
Noise tolerance
Robustness
Feature extraction
Diversity
Reinforcement learning
Memory; Dynamically changing coverage
Distributed
Multi-layered
Adaptive
·        Uniqueness: each individual possesses its own immune system, with its particular vulnerabilities and capabilities;
·        Diversity: there is a large amount of types of elements (cells, molecules, proteins, etc.) that altogether perform the same role of protecting the body from malefic invaders. Additionally, there are different fronts of defense, like innate and adaptive immunity;
·        Disposability (robustness): no single component of the natural immune system is essential for its functioning. Cell death is usually balanced by cell production;
·        Autonomy: the immune system does not require outside management or maintenance. It autonomously classifies and eliminates pathogens, and it repairs itself by replacing damaged cells;
·        Multilayered: multiple layers of different mechanisms are combined to provide high overall security, as summarized in Figure 2.5 (Section 2.3);
·        No secure layer: any cell of the human body can be attacked by the immune system, including those of the immune system itself;
·        Recognition of foreigners: the (harmful) molecules that are not native to the body are recognized and eliminated by the immune system;
·        Anomaly detection: the immune system can detect and react to pathogens that the body has never encountered before;
·        Dynamically changing coverage: as the immune system can not maintain a set of cells and molecules large enough to detect all pathogens, it makes a trade-off between space and time. It maintains a circulating pool of lymphocytes that is constantly being changed through cell death, production and reproduction;
·        Distributability: the immune cells, molecules and organs are distributed all over the body and, most importantly, are not subject to any centralized control;
·        Imperfect detection (noise tolerance): an absolute recognition of the pathogens is not required, hence the system is flexible;
·        Reinforcement learning and memory: the immune system can “learn” the structures of pathogens. It retains the ability to recognize previously seen pathogens through immune memory, so that future responses to the same pathogens are faster and stronger; and
·        An arms race: the vertebrate immune system replicates cells to deal with replicating pathogens, otherwise the pathogens would quickly overwhelm the immune defenses.

6. A Definition
AIS are adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to complex problem domains

7. Some History
Developed from the field of theoretical immunology in the mid 1980’s.
Suggested we ‘might look’ at the IS
1990 – Bersini first use of immune algos to solve problems
Forrest et al – Computer Security mid 1990’s
Hunt et al, mid 1990’s – Machine learning

8. Scope of AIS:
Computer Security(Forrest’94’96’98, Kephart’94, Lamont’98’01,02, Dasgupta’99’01, Bentley’00’01,02)
Anomaly Detection (Dasgupta’96’01’02)
Fault Diagnosis (Ishida’92’93, Ishiguro’94)
Data Mining & Retrieval (Hunt’95’96, Timmis’99’01, ’02)
Pattern Recognition (Forrest’93, Gibert’94, de Castro ’02)
Adaptive Control (Bersini’91)

9. Scope of AIS (Cont……): Job shop Scheduling (Hart’98, ’01, ’02)
Chemical Pattern Recognition (Dasgupta’99)
Robotics (Ishiguro’96’97,Singh’01)
Optimization (DeCastro’99,Endo’98, de Castro ’02)
Web Mining (Nasaroui’02)
Fault Tolerance (Tyrrell, ’01, ’02, Timmis ’02)
Autonomous Systems (Varela’92,Ishiguro’96)
Engineering Design Optimization (Hajela’96 ’98, Nunes’00)
And so on …

10. Outline What are AIS? Useful immunology Thinking about AIS
Application Areas and Case Studies
The Future

11. Role of the Immune System
Protect our bodies from pathogen and viruses
Primary immune response
Launch a response to invading pathogens
Secondary immune response
Remember past encounters
Faster response the second time around

12. How does it work: A simplistic view

13. Immune cells There are two primarily types of lymphocytes:
B-lymphocytes (B cells)
T-lymphocytes (T cells)
Others types include macrophages, phagocytic cells, cytokines, etc.

14. Self/Non-Self Recognition
Immune system needs to be able to differentiate between self and non-self cells
Antigenic encounters may result in cell death, therefore
Some kind of positive selection
Some element of negative selection

15. Antigen
Substances capable of starting a specific immune response commonly are referred to as antigens
This includes some pathogens such as viruses, bacteria, fungi etc .

16. Immune Pattern Recognition
The immune recognition is based on the complementarity between the binding region of the receptor and a portion of the antigen called epitope.
Antibodies present a single type of receptor, antigens might present several epitopes.
This means that each antibody can recognize a single antigen

17. Clonal Selection

18. Main Properties of Clonal Selection (Burnet, 1978)
Elimination of self antigens
Proliferation and differentiation on contact of mature lymphocytes with antigen
Restriction of one pattern to one differentiated cell and retention of that pattern by clonal descendants;
Generation of new random genetic changes, subsequently expressed as diverse antibody patterns by a form of accelerated somatic mutation

19. Immune Network Theory Idiotypic network (Jerne, 1974)
B cells co-stimulate each other
Treat each other a bit like antigens
Creates an immunological memory
Mention Bersinis' principles

20. Reinforcement Learning and Immune Memory
Repeated exposure to an antigen throughout a lifetime
Primary, secondary immune responses
Remembers encounters
No need to start from scratch
Memory cells
Continuous learning

21. Learning (2)

22. Immune System: Summary
Define host (body cells) from external entities.
When an entity is recognized as foreign (or dangerous)- activate several defense mechanisms leading to its destruction (or neutralization).
Subsequent exposure to similar entity results in rapid immune response.
Overall behavior of the immune system is an emergent property of many local interactions.
So it is useful?

23. Outline What are AIS? Useful immunology Thinking about AIS
Application Areas and Case Studies
The Future

24. Artificial Immune Systems
AIS are adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to complex problem domains

25. This Section
General Framework for describing and constructing AIS models
A short review of where AIS are used today
Can not cover them all, far too many
Also we are not experts in all application areas !
Where are AIS headed?

26. What do want from a Framework?
In a computational world we work with representations and processes. Therefore, we need:
To be able to describe immune system components
Be able to describe their interactions
Quite high level abstractions
Capture general purpose processes that can be applied to various areas

27. General Framework for AIS
Immune Algorithms
Affinity Measures
Representation
Application Domain

28. Representation – Shape Space
Describe the general shape of a molecule
Describe interactions between molecules
Degree of binding between molecules

29. Representation Vectors Ab = Ab1, Ab2, ..., AbL
Ag = Ag1, Ag2, ..., AgL
Real-valued shape-space
Integer shape-space
Binary shape-space
Symbolic shape-space
·        Real-valued shape-space: the attribute strings are real-valued vectors;
·        Integer shape-space: the attribute strings are composed of integer values;
·        Hamming shape-space: composed of attribute strings built out of a finite alphabet of length k;
·        Symbolic shape-space: usually composed of different types of attribute strings where at least one of them is symbolic, such as a ‘name’, a ‘color’, etc.
Assume the general case in which an antibody molecule is represented by the set of coordinates Ab = Ab1, Ab2, ..., AbL, and an antigen is given by Ag = Ag1, Ag2, ..., AgL, where boldface letters correspond to a string.

30. Define their Interaction
Define the term Affinity
Distance measures such as Hamming, Manhattan etc. etc.
Affinity Threshold

31. Basic Immune Models and Algorithms
Negative Selection Algorithms
Clonal Selection Algorithm
Immune Network Models
Somatic Hypermutation

32. Negative Selection (NS) Algorithms
Forrest 1994: Idea taken from the negative selection of T-cells in the thymus
Applied initially to computer security
Split into two parts:
Censoring
Monitoring

33. Clonal Selection Algorithm (de Castro & von Zuben, 2001)
1. Initialisation: Randomly initialise a population (P)
2. Antigenic Presentation: for each pattern in Ag, do:
2.1 Antigenic binding: determine affinity to each P’
2.2 Affinity maturation: select n highest affinity from P and clone and mutate prop. to affinity with Ag, then add new mutants to P
3. Metadynamics:
3.1 select highest affinity P to form part of M
3.2 replace n number of random new ones
4. Cycle: repeat 2 and 3 until stopping criteria
1.      Randomly initialize a population of individuals (P);
2.      For each pattern of S, present it to the population P and determine its affinity with each element of the population P;
3.      Select n1 highest affinity elements of P and generate copies of these individuals proportionally to their affinity with the antigen. The higher the affinity, the higher the number of copies, and vice-versa;
4.      Mutate all these copies with a rate proportional to their affinity with the input pattern: the higher the affinity, the smaller the mutation rate, and vice-versa;
5.      Add these mutated individuals to the population P and re-select n2 of these maturated (optimized) individuals to be kept as the memory M of the system;
6.      Replace a number n3 of individuals with low affinity by (randomly generated) new ones;
7.      Repeat Steps 2 to 6 until a certain stopping criterion is met.

34. Discrete Immune Network Models (Timmis & Neal, 2001)
Initialisation: create an initial network from a sub-section of the antigens
Antigenic presentation: for each antigenic pattern, do:
2.1 Clonal selection and network interactions: for each network cell,
determine its stimulation level (based on antigenic and network interaction)
2.2 Metadynamics: eliminate network cells with a low stimulation
2.3 Clonal Expansion: select the most stimulated network cells and
reproduce them proportionally to their stimulation
2.4 Somatic hypermutation: mutate each clone
2.5 Network construction: select mutated clones and integrate
3. Cycle: Repeat step 2 until termination condition is met
Initialise the immune network (P)
For each pattern in Ag
Determine affinity to each P’
Calculate network interaction
Allocate resources to the strongest members of P
Remove weakest P
EndFor
If termination condition met
exit
else
Clone and mutate each P (based on probability a)
Integrate new mutants into P based on affinity
Repeat

35. Somatic Hypermutation
Mutation rate in proportion to affinity
Very controlled mutation in the natural immune system
Trade-off between the normalized antibody affinity D* and its mutation rate ,

36. Case Study: Data Mining

37. Data mining: Problem description
More benchmark problem in this case
Assume a set of labelled vectors
Classification

38. AIRS: (Artificial Immune Recognition System) Watkins 2003
Clonal Selection
Based initially on immune networks, though found this did not work
Resource allocation
Somatic hypermutation
Eventually
Antibody/antigen binding

39. AIRS: Mapping from IS to AIS
Antibody Feature Vector
Recognition Combination of feature Ball vector and vector class
Antigens Training Data
Immune Memory Memory cells—set of mutated ARBs

40. Classification
Stimulation of an ARB is based not only on its affinity to an antigen but also on its class when compared to the class of an antigen
Allocation of resources to the ARBs also takes into account the ARBs’ classifications when compared to the class of the antigen
Memory cell hyper-mutation and replacement is based primarily on classification and secondarily on affinity
Sparse in AIS literature
Not as straight forward as initially suspected

41. AIRS Algorithm Data normalization and initialization
Memory cell identification and ARB generation
Competition for resources in the development of a candidate memory cell
Potential introduction of the candidate memory cell into the set of established memory cells

42. AIRS: Performance Evaluation
Fisher’s Iris Data Set
Pima Indians Diabetes
Data Set
Ionosphere Data Set
Sonar Data Set
Iris: 3 way classification problem; 150 data items; 5XCV; avg. 3 times; 4 features
Ionosphere: 2-way classification, good & bad radar returns; 34 features; 200 in training, 151 test set
Diabetes: 2-way class, has diabetes or not; 10XCV; 8 features; 768 instances total
Sonar: 2-way class; 13XCV; 60 features; 16 instances in each test set

43. Classification Accuracy
Important to maintain accuracy
AIRS1: Accuracy
AIRS2: Accuracy
Iris
96.7
96.0
Ionosphere
94.9
95.6
Diabetes
74.1
74.2
Sonar
84.0
84.9

44. Features No need to know best architecture to get good results
Default settings within a few percent of the best it can get
User-adjustable parameters optimize performance for a given problem set
Generalization and data reduction

45. aiNET: Artificial Immune Network for Data Mining

46. Problem description More benchmark problem in this case
Assume a set of unlabelled vectors
We can ask the questions:
Is there a large amount of redundancy?
Are there any groups or subgroups intrinsic to the data?
What is the structural or spatial distribution?

47. aiNET: Immune principles employed
B-cells (antibodies)
Antigens
Antibody/antigen binding
Clonal selection process
Immune network theory
Combined with statistical analysis tools

48. Data mining: Immune Network Algorithm
1. Initialization: create an initial random population of network antibodies;
2. Antigenic presentation: for each antigenic pattern, do:
2.1 Clonal selection and expansion:
2.2 Affinity maturation:
2.3 Clonal interactions:
2.4 Clonal suppression:
2.5 Metadynamics:
2.6 Network construction:
3. Network interactions:
4. Network suppression:
5. Diversity:
6. Cycle: repeat Steps 2 to 4 until a pre-specified number of iterations is reached.
Initialization: create an initial random population of network antibodies;
2. Antigenic presentation: for each antigenic pattern, do:
2.1 Clonal selection and expansion: for each network element, determine its affinity with the antigen presented. Select a number of high affinity elements and reproduce;
2.2 Affinity maturation: mutate each clone inversely proportional to affinity. Re-select a number of highest affinity clones and place them into a clonal memory set;
2.3 Clonal interactions: determine the network interactions (affinity) among all the elements of the clonal memory set;
2.4 Clonal suppression: eliminate those memory clones whose affinity is less than a pre-specified threshold;
2.5 Metadynamics: eliminate all memory clones whose affinity with the antigen is less than a pre-defined threshold;
2.6 Network construction: incorporate the remaining clones of the clonal memory with all network antibodies;
3. Network interactions: determine the affinity (degree of similarity) between each pair of network antibodies;
4. Network suppression: eliminate all network antibodies whose affinity is less than a pre-specified threshold;
5. Diversity: introduce a number of new randomly generated antibodies into the network;
6. Cycle: repeat Steps 2 to 4 until a pre-specified number of iterations is reached.

49. Data mining: Mapping from IS to aiNET
Immune System
aiNET
B-cell (antibody)
Internal data vector
Antigen
Training data vector
Binding
Calculation of Euclidean distance
Cell cloning
Duplication of internal data vectors
Somatic hypermutation
Affinity proportional mutation
Immune network
Network of internal data vectors
Metadynamics
Removal and creation of internal data vectors

50. Data mining: Clustering (aiNet)
Limited visualisation
Interpret via MST or dendrogram
Compression rate of 81%
Successfully identifies the clusters
Training Pattern
Result immune network

51. Data mining:Hierarchical Clustering (aiNET)

52. Other Interesting Applications
Immune Network for continuous learning (Neal 2002)
Track moving data over time
Maintains clusters in absence of patterns
Useful for dynamic environments
Continuous Classification
classification of interesting/non-interesting s
Changing profile of the user
Maintain classification accuracy
Comparable to Naïve Bayes

53. New Trends Danger Theory Could this be useful for Web Mining?
Not self/non-self but Danger/Non-Danger
Immune response is initiated in the tissues. Danger Zone.
This makes it context dependant
Could this be useful for Web Mining?

54. Summary Immune metaphors
Covered much, but there is much work not covered (so apologies to anyone for missing theirs)
Immune metaphors
Antibodies and their interactions
Immune learning and memory
Self/non-self
Negative selection
Application of immune metaphors

55. The Future Rapidly emerging field Much work is very diverse
Framework helps a little
More formal approach required?
Wide possible application domains
What is it that makes the immune system unique?
More work with immunologists
Theories such as Danger theory, Self-Assertion may have something to say to AIS

56. The Future (2)
ARTIST: A Network for Artificial Immune Systems (EPSRC funded network)
Work towards:
A theoretical foundation for AIS as a new CI
Extraction of accurate metaphors
Immune System Modelling
Application of AIS
Train PhD students
Fund workshops/meetings
Coordinate and Disseminate UK based AIS research (links to Europe)

57. The Future (hopefully)
IT IS: Information Technology Inspired by the Immune System
FP 6 IP:
16 institutions across Europe
Create a European Library of immune algorithms
Theoretical analysis of AIS
Application of AIS
Autonomous boat
Immunoinformatics
Web Mining
Modelling of Immune System

58. AIS Resources: Books
Artificial Immune Systems and Their Applications by Dipankar Dasgupta (Editor) Springer Verlag, January 1999.
Artificial Immune Systems: A New Computational Intelligence Approach by Leandro N. de Castro, Jonathan Timmis, Springer Verlag, November 2002.
Immunocomputing: Principles and Applications by Alexander O. Tarakanov, Victor A. Skormin, Svetlana P. Sokolova, Springer Verlag, April 2003.

59. AIS Related Events in 2003:
Special Session on Artificial Immune Systems at the Congress on Evolutionary Computation (CEC), December 8-12, 2003, Canberra, Australia.
Special Session on Immunity-Based Systems at Seventh International Conference on Knowledge-Based Intelligent Information  & Engineering Systems (KES), September 3-5, 2003, University of Oxford, UK.  
Second International Conference on Artificial Immune Systems (ICARIS), September 1-3, 2003, Napier University, Edinburgh, UK.
 Tutorial on Artificial Immune Systems at 1st Multidisciplinary International Conference on Scheduling: Theory and Applications (MISTA), 12 August 2003, The University of Nottingham, UK.
 Tutorial on Immunological Computation at International Joint Conference on Artificial Intelligence (IJCAI), August 10, 2003, Acapulco, Mexico.
 Special Track on Artificial Immune Systems at Genetic and Evolutionary Computation Conference (GECCO), Chicago, USA, July 12-16, 2003