"If the system exhibits a structure which can be represented by a mathematical equivalent, called a mathematical model, and if the objective can be also so quantified, then some computational method may be evolved for choosing the best schedule of actions among alternatives. Such use of mathematical models is termed mathematical programming." (George B Dantzig, "Linear Programming and Extensions", 1963)
"An artificial neural network (or simply a neural network) is a biologically inspired computational model that consists of processing elements (neurons) and connections between them, as well as of training and recall algorithms." (Nikola K Kasabov, "Foundations of Neural Networks, Fuzzy Systems, and Knowledge Engineering", 1996)
"The fundamental, indivisible unit of information is the bit. The fundamental, indivisible unit of digital computation is the transformation of a bit between its two possible forms of existence: as structure (memory) or as sequence (code). This is what a Turing Machine does when reading a mark (or the absence of a mark) on a square of tape, changing its state of mind accordingly, and making (or erasing) a mark somewhere else." (George B Dyson, "Turing's Cathedral: The Origins of the Digital Universe", 2012)
"When datasets are small, a parametric model may perform well because the strong assumptions made by the model - if correct - can help the model to avoid overfitting. However, as the size of the dataset grows, particularly if the decision boundary between the classes is very complex, it may make more sense to allow the data to inform the predictions more directly. Obviously the computational costs associated with nonparametric models and large datasets cannot be ignored. However, support vector machines are an example of a nonparametric model that, to a large extent, avoids this problem. As such, support vector machines are often a good choice in complex domains with lots of data." (John D Kelleher et al, "Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies", 2015)
"[...] adding nonlinear features to the representation of our data can make linear models much more powerful. However, often we don’t know which features to add, and adding many features (like all possible interactions in a 100-dimensional feature space) might make computation very expensive. Luckily, there is a clever mathematical trick that allows us to learn a classifier in a higher-dimensional space without actually computing the new, possibly very large representation. This is known as the kernel trick, and it works by directly computing the distance (more precisely, the scalar products) of the data points for the expanded feature representation, without ever actually computing the expansion." (Andreas C Müller & Sarah Guido, "Introduction to Machine Learning with Python: A Guide for Data Scientists", 2017)
"In practice, increasing the size of your model will eventually cause you to run into computational problems because training very large models is slow. You might also exhaust your ability to acquire more training data. [...] Increasing the model size generally reduces bias, but it might also increase variance and the risk of overfitting. However, this overfitting problem usually arises only when you are not using regularization. If you include a well-designed regularization method, then you can usually safely increase the size of the model without increasing overfitting." (Andrew Ng, "Machine Learning Yearning", 2018)
"Machine learning is often associated with the automation of decision making, but in practice, the process of constructing a predictive model generally requires a human in the loop. While computers are good at fast, accurate numerical computation, humans are instinctively and instantly able to identify patterns. The bridge between these two necessary skill sets lies in visualization - the precise and accurate rendering of data by a computer in visual terms and the immediate assignation of meaning to that data by humans." (Benjamin Bengfort et al, "Applied Text Analysis with Python: Enabling Language-Aware Data Products with Machine Learning", 2018)
"The higher the dimension, in other words, the higher the number of possible interactions, and the more disproportionally difficult it is to understand the macro from the micro, the general from the simple units. This disproportionate increase of computational demands is called the curse of dimensionality." (Nassim N Taleb, "Skin in the Game: Hidden Asymmetries in Daily Life", 2018)
"Feature selection is the first critical step in model optimization. It's the art of identifying which features in your dataset contribute most significantly to the outcome you're trying to predict. This not only helps in enhancing the model's accuracy but also in reducing computational complexity, leading to more efficient models. [...] While feature selection is about cherry-picking the most useful features, feature engineering is about creating new features that increase the predictive strength of the model. This is where creativity and domain knowledge come into play, especially in financial data, where market sentiment, economic indicators, and other external factors can influence market movements." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)
"Genetic programming represents a frontier in feature engineering, allowing for the automated creation of new features through the application of evolutionary algorithms. By combining existing features in non-linear and complex ways, genetic programming can uncover hidden relationships in the data that were not apparent through manual exploration. This technique, while computationally intensive, holds the promise of discovering novel predictors that can enhance the performance of trading algorithms." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)
"Data efficiency in LLMs is about maximizing the quality of learning from the available data while minimizing the required dataset size and computational resources. Large datasets are costly to process, and redundant or noisy data can negatively impact model performance. Therefore, data efficiency techniques aim to achieve high model accuracy and generalization with a reduced or optimized dataset. This process includes filtering data for quality, reducing redundancy, and applying sampling techniques to emphasize high-value samples." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)
