Adaptive Resonance Theory (ART)

Adaptive Resonance Theory (ART) networks perform completely unsupervised learning.

Their competitive learning algorithm is similar to the first (unsupervised) phase of CPN learning.

However, ART networks are able to grow additional neurons if a new input cannot be categorized appropriately with the existing neurons.

A vigilance parameter r determines the tolerance of this matching process.

A greater value of r leads to more, smaller clusters (= input samples associated with the same winner neuron).

ART networks consist of an input layer and an output layer.

We will only discuss ART-1 networks, which receive binary input vectors.

Bottom-up weights are used to determine output-layer candidates that may best match the current input.

Top-down weights represent the “prototype” for the cluster defined by each output neuron.

A close match between input and prototype is necessary for categorizing the input.

Finding this match can require multiple signal exchanges between the two layers in both directions until “resonance” is established or a new neuron is added.

ART networks tackle the stability-

plasticity dilemma:

Plasticity: They can always adapt to 

unknown inputs (by creating a new 

cluster with a new weight vector) if the 

given input cannot be classified by 

existing clusters.

Stability: Existing clusters are not 

deleted by the introduction of new 

inputs (new clusters will just be 

created 

in addition to the old ones).

Problem: Clusters are of fixed size, 

depending on r.

ART Example Computation

For this example, let us assume that we have an ART-1 network with 7 input neurons (n = 7) and initially one output neuron (n = 1).

Our input vectors are

{(1, 1, 0, 0, 0, 0, 1),
 (0, 0, 1, 1, 1, 1, 0),
 (1, 0, 1, 1, 1, 1, 0),
 (0, 0, 0, 1, 1, 1, 0),
 (1, 1, 0, 1, 1, 1, 0)}

and the vigilance parameter r = 0.7.

Initially, all top-down weights are set to tl,1(0) = 1, and all bottom-up weights are set to b1,l(0) = 1/8.