Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction and ANN Structure.
- Comparison of biological and artificial neurons.
- The structural model of an ANN.
- Activation functions employed in ANNs.
- Common categories of network architectures.
Mathematical Foundations and Learning mechanisms.
- Review of vector and matrix algebra.
- State-space concepts.
- Optimization principles.
- Error-correction learning.
- Memory-based learning.
- Hebbian learning.
- Competitive learning.
Single layer perceptrons.
- Perceptron structure and learning processes.
- Introduction to pattern classifiers and Bayes' classifiers.
- Using perceptrons as pattern classifiers.
- Perceptron convergence.
- Limitations inherent to perceptrons.
Feedforward ANN.
- Architecture of Multi-layer feedforward networks.
- The Back propagation algorithm.
- Training and convergence in Back propagation.
- Functional approximation via back propagation.
- Practical and design considerations for back propagation learning.
Radial Basis Function Networks.
- Pattern separability and interpolation.
- Theory of Regularization.
- Regularization in the context of RBF networks.
- Design and training of RBF networks.
- Approximation characteristics of RBFs.
Competitive Learning and Self organizing ANN.
- General clustering methods.
- Learning Vector Quantization (LVQ).
- Competitive learning algorithms and architectures.
- Self-organizing feature maps.
- Characteristics of feature maps.
Fuzzy Neural Networks.
- Neuro-fuzzy systems.
- Foundations of fuzzy sets and logic.
- Design of fuzzy systems.
- Design of fuzzy ANNs.
Applications
- Discussion of various Neural Network application examples, including their benefits and challenges.
DAY -2 MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets – consistent case
- Guarantees for finite hypothesis sets – inconsistent case
- Generalities
- Deterministic vs. Stochastic scenarios
- Bayes error noise
- Estimation and approximation errors
- Model selection
- Rademeacher Complexity and VC – Dimension
- Bias - Variance tradeoff
- Regularisation
- Over-fitting
- Validation
- Support Vector Machines
- Kriging (Gaussian Process regression)
- PCA and Kernel PCA
- Self Organisation Maps (SOM)
- Kernel induced vector space
- Mercer Kernels and Kernel - induced similarity metrics
- Reinforcement Learning
DAY 3 - DEEP LEARNING
Content taught in relation to topics covered on Day 1 and Day 2
- Logistic and Softmax Regression
- Sparse Autoencoders
- Vectorization, PCA and Whitening
- Self-Taught Learning
- Deep Networks
- Linear Decoders
- Convolution and Pooling
- Sparse Coding
- Independent Component Analysis
- Canonical Correlation Analysis
- Demos and Applications
Requirements
A solid grasp of mathematics.
Strong foundational knowledge of basic statistics.
While basic programming skills are not mandatory, they are highly advisable.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
