ArchiveCSE 2022 was held on December 17–18, 2022. It is a past event: submissions and registration are closed. This website is an independent educational and historical resource.

Data • Models • Evaluation • Generalization

Machine Learning Applications: Models, Data and Research

Machine learning fits models to data instead of programming rules by hand. This guide explains how models learn, how to measure whether they work, and why most failures come from data and evaluation rather than algorithms.

Last reviewed: · Editorial team, cse2022.org

01What Is Machine Learning?

A computer program is said to learn from experience E with respect to a task T and performance measure P if its performance at T, as measured by P, improves with experience E.

This definition, from Tom Mitchell's 1997 textbook, is useful because it forces three questions: what exactly is the task, what data provides the experience, and how will success be measured? Most machine learning projects that fail do so because one of those three was poorly defined.

In practice, a learning algorithm searches a family of candidate models for the one that minimizes a loss function on training data — for example, the mean squared difference between predicted and actual values.

02Supervised Learning

In supervised learning, each training example has an input and a known correct output (a label). Classification predicts a category (is this email spam?); regression predicts a number (what load will this beam carry?). Common methods include linear and logistic regression, decision trees, random forests, gradient-boosted trees, support vector machines and neural networks.

03Unsupervised Learning

Unsupervised learning finds structure in data without labels. Clustering (such as k-means) groups similar examples; dimensionality reduction (such as principal component analysis) finds a smaller set of variables that preserves most of the variation; anomaly detection flags unusual examples. Results are harder to evaluate because there is no ground truth, so domain experts must judge whether discovered structure is meaningful.

04Reinforcement Learning

In reinforcement learning, an agent interacts with an environment, receives rewards, and learns a policy that maximizes cumulative reward over time. It is suited to sequential decision problems such as control and game playing. It typically needs very large amounts of interaction, which is why training often happens in simulation, and poorly specified rewards can lead agents to exploit loopholes rather than solve the intended problem.

The three main learning paradigms
ParadigmDataGoalExample
SupervisedLabelled input–output pairsPredict labels for new inputsClassifying defects in inspection images
UnsupervisedUnlabelled inputsDiscover structureGrouping sensor readings by operating mode
ReinforcementRewards from interactionLearn a decision policyTuning a controller in simulation

05Training and Testing Data

Data are split into three parts. The training set fits model parameters; the validation set is used to choose hyperparameters and compare models; the test set is held back and used once, at the end, to estimate real-world performance. When data are limited, k-fold cross-validation rotates which portion is held out.

Data leakage

Leakage happens when information from the test data influences training — for example, normalizing all data before splitting, or allowing records from the same patient or machine to appear in both sets. It produces impressive but false results. For time-series data, always split by time, training on the past and testing on the future.

06Feature Engineering

Features are the input variables a model sees. Feature engineering transforms raw data into informative inputs: scaling numeric values, encoding categories, extracting frequency components from vibration signals, or computing ratios that domain knowledge says matter. Deep learning reduces the need for hand-crafted features in images, audio and text, but for tabular engineering data, thoughtful features often matter more than the choice of algorithm.

07Model Evaluation

For a binary classifier, outcomes on the test set fall into four groups:

Predicted positive
Predicted negative
Actual positive
True positive (TP)
False negative (FN)
Actual negative
False positive (FP)
True negative (TN)
The confusion matrix. Every classification metric is computed from these four counts.
Common evaluation metrics
MetricFormulaAnswers the question
Accuracy(TP + TN) / allWhat fraction of predictions are correct?
PrecisionTP / (TP + FP)When the model says positive, how often is it right?
Recall (sensitivity)TP / (TP + FN)What fraction of real positives does it find?
F1 score2 · P · R / (P + R)Harmonic mean balancing precision and recall
ROC AUCArea under true-positive vs false-positive rate curveHow well does it rank positives above negatives across thresholds?
MAE / RMSE (regression)Mean absolute error / root mean squared errorHow far off are numeric predictions? RMSE penalizes large errors more.

Accuracy is misleading on imbalanced data. If 1% of parts are defective, a model that always predicts “not defective” is 99% accurate and completely useless; its recall is zero. Choose metrics that reflect the real costs of each type of error.

08Overfitting and Generalization

A model overfits when it learns noise and peculiarities of the training data, performing well there but poorly on new data. It underfits when it is too simple to capture real patterns. The gap between training and validation performance is the main diagnostic.

Remedies for overfitting include collecting more data, using simpler models, regularization (penalizing large parameter values), dropout in neural networks, early stopping and data augmentation. The underlying idea is the bias–variance trade-off: reducing a model's sensitivity to the particular training sample usually means accepting some systematic error.

09Deep Learning Applications

Deep learning uses neural networks with many layers trained by gradient descent and backpropagation. Convolutional networks process images; recurrent networks and, more recently, transformers process sequences. Applications include image recognition, speech recognition, medical image analysis and surrogate models that approximate expensive physics simulations. Deep models usually need large datasets and considerable computation, and their internal reasoning is difficult to interpret.

10Natural Language Processing Applications

NLP applications of machine learning include text classification, sentiment analysis, information extraction from technical documents, machine translation and question answering. Modern systems represent text as learned vectors (embeddings) that place related words and passages near each other. Evaluation is difficult: automatic metrics for generated text correlate imperfectly with human judgement, so careful human evaluation remains important. See AI research for the broader context of NLP.

11Machine Learning in Engineering

  • Predictive maintenance: estimating remaining useful life from vibration, temperature and usage data.
  • Quality inspection: detecting defects in manufactured parts from images.
  • Surrogate modelling: replacing a slow simulation with a fast learned approximation for design exploration.
  • Control and optimization: tuning process settings in energy systems and manufacturing.

In safety-relevant engineering, a learned model should be validated like any other engineering component, with documented operating limits. See computational science for verification and validation.

12Bias, Fairness and Model Limitations

Models learn whatever patterns their data contain, including historical bias and gaps in coverage. A model trained mostly on one population, machine type or operating condition may perform worse elsewhere. Fairness can be measured in several ways — for example, equal error rates across groups or equal positive prediction rates — and these definitions can conflict, so the choice must be justified for each application.

Other limitations include distribution shift (the world changes after training), spurious correlations (the model relies on an irrelevant cue that happened to correlate with the label), and poor calibration (confidence scores that do not match real probabilities).

13Practical Research Considerations

  • Start with a simple baseline; a complex model must beat it to justify itself.
  • Fix and report random seeds, and report mean and variation across several runs.
  • Document data sources, preprocessing, splits, hyperparameter search and compute used.
  • Analyse errors qualitatively, not only aggregate scores.
  • Release code and, where permitted, data so that others can reproduce the work.

The research paper guidelines cover reproducibility in more depth.

Further reading

  • Mitchell, T. M. (1997). Machine Learning. McGraw-Hill.
  • Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
  • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.
  • Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.