"Little experience is sufficient to show that the traditional machinery of statistical processes is wholly unsuited to the needs of practical research. Not only does it take a cannon to shoot a sparrow, but it misses the sparrow! The elaborate mechanism built on the theory of infinitely large samples is not accurate enough for simple laboratory data. Only by systematically tackling small sample problems on their metrics does it seem possible to apply accurate tests to practical data." (Ronald A Fisher,Statistical Methods for Research Workers", 1925),
"The degree of confirmation assigned to any given hypothesis is sensitive to properties of the entire belief system [...] simplicity, plausibility, and conservatism are properties that theories have in virtue of their relation to the whole structure of scientific beliefs taken collectively. A measure of conservatism or simplicity would be a metric over global properties of belief systems." (Jerry Fodor,Modularity of Mind", 1983)
"No subjective metric can escape strategic gaming [...] The possibility of mischief is bottomless. Fighting ratings is fruitless, as they satisfy a very human need. If one scheme is beaten down, another will take its place and wear its flaws. Big Data just deepens the danger. The more complex the rating formulas, the more numerous the opportunities there are to dress up the numbers. The larger the data sets, the harder it is to audit them." (Kaiser Fung,Numbersense: How To Use Big Data To Your Advantage", 2013)
"The urge to tinker with a formula is a hunger that keeps coming back. Tinkering almost always leads to more complexity. The more complicated the metric, the harder it is for users to learn how to affect the metric, and the less likely it is to improve it." (Kaiser Fung," Numbersense: How To Use Big Data To Your Advantage", 2013)
"Precision and recall are ways of monitoring the power of the machine learning implementation. Precision is a metric that monitors the percentage of true positives. […] Recall is the ratio of true positives to true positive plus false negatives." (Matthew Kirk,"Thoughtful Machine Learning", 2015)
"Just as with aggregated data, an average is a summary statistic that can tell you something about the data - but it is only one metric, and oftentimes a deceiving one at that. By taking all of the data and boiling it down to one value, an average" (and other summary statistics) may imply that all of the underlying data is the same, even when it’s not." (John H Johnson & Mike Gluck,Everydata: The misinformation hidden in the little data you consume every day", 2016)
"Roughly stated, the No Free Lunch theorem states that in the lack of prior knowledge" (i.e. inductive bias) on average all predictive algorithms that search for the minimum classification error" (or extremum over any risk metric) have identical performance according to any measure." (N D Lewis,Deep Learning Made Easy with R: A Gentle Introduction for Data Science", 2016)
"Variables which follow symmetric, bell-shaped distributions tend to be nice as features in models. They show substantial variation, so they can be used to discriminate between things, but not over such a wide range that outliers are overwhelming." (Steven S Skiena,The Data Science Design Manual", 2017)
"Dimensionality reduction is a way of reducing a large number of different measures into a smaller set of metrics. The intent is that the reduced metrics are a simpler description of the complex space that retains most of the meaning." (Danyel Fisher & Miriah Meyer,Making Data Visual", 2018)
"Statistical metrics can show us facts and trends that would be impossible to see in any other way, but often they’re used as a substitute for relevant experience, by managers or politicians without specific expertise or a close-up view." (Tim Harford,The Data Detective: Ten easy rules to make sense of statistics", 2020)
"Non-linear associations are also quantifiable. Even linear regression can be used to model some non-linear relationships. This is possible because linear regression has to be linear in parameters, not necessarily in the data. More complex relationships can be quantified using entropy-based metrics such as mutual information. Linear models can also handle interaction terms. We talk about interaction when the model’s output depends on a multiplicative relationship between two or more variables." (Aleksander Molak,Causal Inference and Discovery in Python", 2023)
"Defining an appropriate distance metric from the context ofthe problem is a most important decision. For example, if your variables are all numeric, and on the same scale, then Euclidean distance might be best. If your variables are categorical, you might need to use something like Hamming distance." (Dianne Cook & Ursula Laa, "Interactively Exploring High-Dimensional Data and Models in R", 2026)
"When the ratio of the largest to smallest observation is large you should question whether the data are being analyzed in the right metric" (transformation)." (George E P Box)
No comments:
Post a Comment