MATH3431: Machine Learning and Neural Networks
Preface
Extra reading
1
Preliminaries
1.1
Data matrices
1.1.1
Summaries of variability
1.1.2
Linear combinations
1.1.3
Eigen decomposition
1.2
Distances between individuals
1.2.1
Euclidean distance
1.2.2
Pearson distance
1.2.3
Mahalanobis distance
1.2.4
Manhattan distance
1.2.5
Distances for categorical data
1.3
Linear regression
Aside: deriving the least squares estimator
1.3.1
Using different distance metrics
1.3.2
Partitioning of variance
1.4
Linear discriminant analysis
2
Fundamental concepts
2.1
Types of learning
2.2
Training basics
2.3
Model performance
2.3.1
Classification performance
2.3.2
Regression performance
2.3.3
Overfitting
2.3.4
Model selection criteria
2.4
Cross-validation
2.5
Missing or few data
3
Supervised Learning — Classification
3.1
Historical notes
3.2
Running examples
3.3
\(k\)
-nearest neighbours
3.3.1
Basic algorithm
3.4
Naive Bayes classifier
3.4.1
Missing data in naive Bayes
3.5
Decision trees
3.5.1
Missing data in decision trees
3.5.2
Complexity parameter
3.6
Perceptron
3.7
Balancing classes
4
Supervised Learning — Regression
4.1
Historical notes
4.2
Running example
4.3
Regularised Regression
4.3.1
Types of Regularisation
4.3.2
Mathematical Foundation
4.3.3
Elastic Net Regression
4.3.4
Hyperparameter Selection
Example
Example
4.4
\(k\)
-nearest neighbours
4.5
Decision trees
4.6
Multivariate Adaptive Regression Splines
5
Ensemble methods
5.1
Weak and strong learners
5.2
Stacking
5.2.1
Stacking of classifiers
5.3
Bagging
5.3.1
Bootstrapping
5.3.2
Random forests
5.4
Boosting
5.4.1
AdaBoost
5.4.2
XGBoost
5.5
Error-Correcting Output Codes
5.6
Other ensemble methods
6
Model Interpretation
6.1
Variable importance
6.1.1
Permutation-based variable importance
6.1.2
Decision tree structure
6.2
Main effect visualisations
6.2.1
Main effects
6.2.2
Partial dependence plots
6.2.3
Accumulated local effects
7
Unsupervised learning
7.1
Traditional statistical approaches
7.1.1
Clustering
7.1.2
Dimension reduction
7.2
Modern unsupervised learning techniques
7.2.1
Clustering with Density-Based Spatial Clustering of Applications with Noise (DBSCAN)
7.2.2
Dimensionality Reduction with t-SNE
7.3
Other ideas in unsupervised learning
References
Published with bookdown. Copyright Durham University
MATH3431: Machine learning and neural networks
References