01What Is Machine Learning?
A computer program is said to learn from experience E with respect to a task T and performance measure P if its performance at T, as measured by P, improves with experience E.
This definition, from Tom Mitchell's 1997 textbook, is useful because it forces three questions: what exactly is the task, what data provides the experience, and how will success be measured? Most machine learning projects that fail do so because one of those three was poorly defined.
In practice, a learning algorithm searches a family of candidate models for the one that minimizes a loss function on training data — for example, the mean squared difference between predicted and actual values.
02Supervised Learning
In supervised learning, each training example has an input and a known correct output (a label). Classification predicts a category (is this email spam?); regression predicts a number (what load will this beam carry?). Common methods include linear and logistic regression, decision trees, random forests, gradient-boosted trees, support vector machines and neural networks.
03Unsupervised Learning
Unsupervised learning finds structure in data without labels. Clustering (such as k-means) groups similar examples; dimensionality reduction (such as principal component analysis) finds a smaller set of variables that preserves most of the variation; anomaly detection flags unusual examples. Results are harder to evaluate because there is no ground truth, so domain experts must judge whether discovered structure is meaningful.
04Reinforcement Learning
In reinforcement learning, an agent interacts with an environment, receives rewards, and learns a policy that maximizes cumulative reward over time. It is suited to sequential decision problems such as control and game playing. It typically needs very large amounts of interaction, which is why training often happens in simulation, and poorly specified rewards can lead agents to exploit loopholes rather than solve the intended problem.
| Paradigm | Data | Goal | Example |
|---|---|---|---|
| Supervised | Labelled input–output pairs | Predict labels for new inputs | Classifying defects in inspection images |
| Unsupervised | Unlabelled inputs | Discover structure | Grouping sensor readings by operating mode |
| Reinforcement | Rewards from interaction | Learn a decision policy | Tuning a controller in simulation |
05Training and Testing Data
Data are split into three parts. The training set fits model parameters; the validation set is used to choose hyperparameters and compare models; the test set is held back and used once, at the end, to estimate real-world performance. When data are limited, k-fold cross-validation rotates which portion is held out.
Leakage happens when information from the test data influences training — for example, normalizing all data before splitting, or allowing records from the same patient or machine to appear in both sets. It produces impressive but false results. For time-series data, always split by time, training on the past and testing on the future.
06Feature Engineering
Features are the input variables a model sees. Feature engineering transforms raw data into informative inputs: scaling numeric values, encoding categories, extracting frequency components from vibration signals, or computing ratios that domain knowledge says matter. Deep learning reduces the need for hand-crafted features in images, audio and text, but for tabular engineering data, thoughtful features often matter more than the choice of algorithm.
07Model Evaluation
For a binary classifier, outcomes on the test set fall into four groups:
| Metric | Formula | Answers the question |
|---|---|---|
| Accuracy | (TP + TN) / all | What fraction of predictions are correct? |
| Precision | TP / (TP + FP) | When the model says positive, how often is it right? |
| Recall (sensitivity) | TP / (TP + FN) | What fraction of real positives does it find? |
| F1 score | 2 · P · R / (P + R) | Harmonic mean balancing precision and recall |
| ROC AUC | Area under true-positive vs false-positive rate curve | How well does it rank positives above negatives across thresholds? |
| MAE / RMSE (regression) | Mean absolute error / root mean squared error | How far off are numeric predictions? RMSE penalizes large errors more. |
Accuracy is misleading on imbalanced data. If 1% of parts are defective, a model that always predicts “not defective” is 99% accurate and completely useless; its recall is zero. Choose metrics that reflect the real costs of each type of error.
08Overfitting and Generalization
A model overfits when it learns noise and peculiarities of the training data, performing well there but poorly on new data. It underfits when it is too simple to capture real patterns. The gap between training and validation performance is the main diagnostic.
Remedies for overfitting include collecting more data, using simpler models, regularization (penalizing large parameter values), dropout in neural networks, early stopping and data augmentation. The underlying idea is the bias–variance trade-off: reducing a model's sensitivity to the particular training sample usually means accepting some systematic error.
09Deep Learning Applications
Deep learning uses neural networks with many layers trained by gradient descent and backpropagation. Convolutional networks process images; recurrent networks and, more recently, transformers process sequences. Applications include image recognition, speech recognition, medical image analysis and surrogate models that approximate expensive physics simulations. Deep models usually need large datasets and considerable computation, and their internal reasoning is difficult to interpret.
10Natural Language Processing Applications
NLP applications of machine learning include text classification, sentiment analysis, information extraction from technical documents, machine translation and question answering. Modern systems represent text as learned vectors (embeddings) that place related words and passages near each other. Evaluation is difficult: automatic metrics for generated text correlate imperfectly with human judgement, so careful human evaluation remains important. See AI research for the broader context of NLP.
11Machine Learning in Engineering
- Predictive maintenance: estimating remaining useful life from vibration, temperature and usage data.
- Quality inspection: detecting defects in manufactured parts from images.
- Surrogate modelling: replacing a slow simulation with a fast learned approximation for design exploration.
- Control and optimization: tuning process settings in energy systems and manufacturing.
In safety-relevant engineering, a learned model should be validated like any other engineering component, with documented operating limits. See computational science for verification and validation.
12Bias, Fairness and Model Limitations
Models learn whatever patterns their data contain, including historical bias and gaps in coverage. A model trained mostly on one population, machine type or operating condition may perform worse elsewhere. Fairness can be measured in several ways — for example, equal error rates across groups or equal positive prediction rates — and these definitions can conflict, so the choice must be justified for each application.
Other limitations include distribution shift (the world changes after training), spurious correlations (the model relies on an irrelevant cue that happened to correlate with the label), and poor calibration (confidence scores that do not match real probabilities).
13Practical Research Considerations
- Start with a simple baseline; a complex model must beat it to justify itself.
- Fix and report random seeds, and report mean and variation across several runs.
- Document data sources, preprocessing, splits, hyperparameter search and compute used.
- Analyse errors qualitatively, not only aggregate scores.
- Release code and, where permitted, data so that others can reproduce the work.
The research paper guidelines cover reproducibility in more depth.
Further reading
- Mitchell, T. M. (1997). Machine Learning. McGraw-Hill.
- Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
- Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.