neural networks
An Introduction to Neural Networks
Vincent Cheung
Kevin Cannons
Signal & Data Compression Laboratory
Electrical & Computer Engineering
University of Manitoba
Winnipeg, Manitoba, Canada
Advisor: Dr. W. Kinsner
May 27, 2002
Neural Networks
Outline
● Fundamentals
● Classes
● Design and Verification
● Results and Discussion
● Conclusion
Cheung/Cannons
1
Classes
Fundamentals
Neural Networks
What Are Artificial Neural Networks?
● An extremely simplified model of the brain
● Essentially a function approximator
Transforms inputs into outputs to the best of its ability
Design
►
Results
Inputs
Cheung/Cannons
Inputs
Outputs
NN
Outputs
2
What Are Artificial Neural Networks?
● Composed of many “neurons” that co-operate to perform the desired function
Results
Design
Classes
Fundamentals
Neural Networks
Cheung/Cannons
3
Classes
Fundamentals
Neural Networks
What Are They Used For?
● Classification
►
Pattern recognition, feature extraction, image matching ● Noise Reduction
Design
►
Recognize patterns in the inputs and produce noiseless outputs
Results
● Prediction
Cheung/Cannons
►
Extrapolation based on historical data
4
Classes
Fundamentals
Neural Networks
Why Use Neural Networks?
● Ability to learn
►
►
NN’s figure out how to perform their function on their own
Determine their function based only upon sample inputs
►
i.e. produce reasonable outputs for inputs it has not been taught how to deal with
Results
Design
● Ability to generalize
Cheung/Cannons
5
How Do Neural Networks Work?
● The output of a neuron is a function of the weighted sum of the inputs plus a bias i1 w1 w2 i2 w3 Neuron i3 Design
Classes
Fundamentals
Neural Networks
Output = f(i1w1 + i2w2 + i3w3 + bias)
Results
bias
● The function of the entire neural network is simply the computation of the
Vincent Cheung
Kevin Cannons
Signal & Data Compression Laboratory
Electrical & Computer Engineering
University of Manitoba
Winnipeg, Manitoba, Canada
Advisor: Dr. W. Kinsner
May 27, 2002
Neural Networks
Outline
● Fundamentals
● Classes
● Design and Verification
● Results and Discussion
● Conclusion
Cheung/Cannons
1
Classes
Fundamentals
Neural Networks
What Are Artificial Neural Networks?
● An extremely simplified model of the brain
● Essentially a function approximator
Transforms inputs into outputs to the best of its ability
Design
►
Results
Inputs
Cheung/Cannons
Inputs
Outputs
NN
Outputs
2
What Are Artificial Neural Networks?
● Composed of many “neurons” that co-operate to perform the desired function
Results
Design
Classes
Fundamentals
Neural Networks
Cheung/Cannons
3
Classes
Fundamentals
Neural Networks
What Are They Used For?
● Classification
►
Pattern recognition, feature extraction, image matching ● Noise Reduction
Design
►
Recognize patterns in the inputs and produce noiseless outputs
Results
● Prediction
Cheung/Cannons
►
Extrapolation based on historical data
4
Classes
Fundamentals
Neural Networks
Why Use Neural Networks?
● Ability to learn
►
►
NN’s figure out how to perform their function on their own
Determine their function based only upon sample inputs
►
i.e. produce reasonable outputs for inputs it has not been taught how to deal with
Results
Design
● Ability to generalize
Cheung/Cannons
5
How Do Neural Networks Work?
● The output of a neuron is a function of the weighted sum of the inputs plus a bias i1 w1 w2 i2 w3 Neuron i3 Design
Classes
Fundamentals
Neural Networks
Output = f(i1w1 + i2w2 + i3w3 + bias)
Results
bias
● The function of the entire neural network is simply the computation of the
References: [AbDo99] H. Abdi, D. Valentin, B. Edelman, Neural Networks, Thousand Oaks, CA: SAGE Publication Inc., 1999. [Hayk94] S. Haykin, Neural Networks, New York, NY: Nacmillan College Publishing Company, Inc., 1994. [Mast93] T. Masters, Practial Neural Network Recipes in C++, Toronto, ON: Academic Press, Inc., 1993. [Scha97] R. Schalkoff, Artificial Neural Networks, Toronto, ON: the McGraw-Hill Companies, Inc., 1997. [WeKu91] S. M. Weiss and C. A. Kulikowski, Computer Systems That Learn, San Mateo, CA: Morgan Kaufmann Publishers, Inc., 1991. [Wass89] P. D. Wasserman, Neural Computing: Theory and Practice, New York, NY: Van Nostrand Reinhold, 1989.