CmSc 310 Artificial Intelligence


Chapter 7: Machine Learning: An Overview of Neural Networks


The AI methods studied so far process symbols - characters.
Neural Networks (NN) follow a different approach - they process numeric information.

  1. Characteristics
    • A large number of very simple neuron-like processing elements.
    • A large number of weighed connections between the elements. The weights encode the knowledge of the network.
    • Highly parallel, distributed control.
    • An emphasis on learning internal representations automatically.

  2. Architecture
  3. An NN is built out of many simple processing elements.

    Each element has

    • some numeric inputs with weights,
    • a function that computes the output

    The links can be

    • Excitatory - positive weight
    • Inhibitory - negative weight

    Basic computational neuron

    The elements are organized in layers:

    • Input layer - nodes that accept input
    • Output layer - nodes that deliver output - the result of the network processing
    • Hidden layers - intermediate layers, optional, used for better learning.

    2.1. Feed-forward networks

    Signals to travel one way only: from input to output. There is no feedback (loops) i.e. the output of any layer does not affect that same layer. They are extensively used in pattern recognition.

    2.2. Feedback (Recurrent) networks

    Signals travel in both directions by introducing loops in the network. Feedback networks are dynamic; their 'state' is changing continuously until they reach an equilibrium point. They remain at the equilibrium point until the input changes and a new equilibrium needs to be found. Feedback architectures are also referred to as interactive or recurrent.

  4. How does it work?
  5. NN act as classifiers.
    Given a set of input values, the task is to recognize the object.
    This is performed through learning.

    Learning phase:

    • For a given set of examples (training set) we know the correct outputs
    • The values are input to the network and the actual output is compared with the desired output
    • If there are differences, the weights are modified and the process is repeated until the output for the training set is correct, or some time limit is exceeded.

    Back-propagation - a method for adjusting the weights in the learning phase, used with recurrent networks.

    The number of nodes for serious tasks - hundreds, thousands
    The number of test examples - 200 - 300 is usually enough.

  6. Summary
  7. Advantage: NN are robust, even if there are missing inputs, or unpredicted values of inputs, the result still would be correct in most of the cases.

    Problems with NN:

    1. How to specify the inputs.
    2. Two approaches:
      an input corresponds to an object - local NN
      an input corresponds to a "microfeature" of the object: distributed NN

    3. We don't know what's happening inside the network - this makes us suspicious.
    4. It is difficult to employ reasoning techniques based on some logic
    5. Hybrid networks - combining symbolic networks and NN to implement reasoning and learning abilities with a robust behavior

    Tutorial on NN:
    http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html

    Example: page 159


Back to Contents page
Created by Lydia Sinapova