"We can consider three broad classes of statistical pitfalls. The first involves sources of bias. These are conditions or circumstances which affect the external validity of statistical results. The second category is errors in methodology, which can lead to inaccurate or invalid results. The third class of problems concerns interpretation of results, or how statistical results are applied (or misapplied) to real world issues." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995)
"There are two problems with sampling - one obvious, and the other more subtle. The obvious problem is sample size. Samples tend to be much smaller than their populations. [...] Obviously, it is possible to question results based on small samples. The smaller the sample, the less confidence we have that the sample accurately reflects the population. However, large samples aren't necessarily good samples. This leads to the second issue: the representativeness of a sample is actually far more important than sample size. A good sample accurately reflects (or 'represents') the population." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)
"Given the important role that correlation plays in structural equation modeling, we need to understand the factors that affect establishing relationships among multivariable data points. The key factors are the level of measurement, restriction of range in data values (variability, skewness, kurtosis), missing data, nonlinearity, outliers, correction for attenuation, and issues related to sampling variation, confidence intervals, effect size, significance, sample size, and power." (Randall E Schumacker & Richard G Lomax, "A Beginner’s Guide to Structural Equation Modeling" 3rd Ed., 2010)
"There are several key issues in the field of statistics that impact our analyses once data have been imported into a software program. These data issues are commonly referred to as the measurement scale of variables, restriction in the range of data, missing data values, outliers, linearity, and nonnormality." (Randall E Schumacker & Richard G Lomax, "A Beginner’s Guide to Structural Equation Modeling" 3rd Ed., 2010)
"Machine learning is well suited for the unpredictable future, because most algorithms learn from new information. But as new information is found, it can also come in unstable forms, and new issues can arise that weren’t thought of before. We don’t know what we don’t know. When processing new information, it’s sometimes hard to tell whether our model is working." (Matthew Kirk, "Thoughtful Machine Learning", 2015)
"Hypotheses build the foundation for data analytics. Develop alternative hypotheses to explain the issue at hand. These hypotheses will guide your data collection." (Shonna D Watters et al, "The Practical Guide for HR Analytics: Using data to inform, transform, and empower HR decisions", 2019)
"A widely adopted technique for solving the data sparsity issue and the cold-start problem is based on graph representation, navigation, and processing. Graph navigation methods (like the pathfinding example [...]) and graph algorithms (such as PageRank) are applied to fill some gaps and create a denser representation of [a] dataset." (Alessandro Negro, "Graph-Powered Machine Learning", 2021)
"It is often said that data scientists and data analysts spend only 20% of their time doing data analysis work, with 80% consumed by data 'issues'. The bulk of their time is spent finding, evaluating, understanding, and preparing data before analysis can begin. A data catalog inverts this principle by enabling data analysts and data scientists to spend 20% of their time looking for data and 80% performing analysis." (Fadi Maali & Jason Lim, "Implementing a Modern Data Catalog to Power Data Intelligence: Make Trustworthy Data Central to Your Organization", 2022)
"Over 80% of models are never operationalized because the efforts involved in deploying them are enormous and the models are deployed and found to produce drift or fairness issues that outweigh the benefits." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)
"A common pitfall in ML delivery is the horizontal slicing of work, where we sequentially deliver functional layers of a technical solution - e.g., data lake, ML platform, ML models, UX interfaces - from the bottom-up. This is a risky delivery approach because customers can only experience the product and provide valuable feedback after months and even years of significant engineering investment. In addition, horizontal slicing naturally leads to late integration issues when horizontal slices come together, increasing the risk of release delays." (David Tan et al,"Effective Machine Learning Teams: Best Practices for ML Practitioners", 2024)
"Observability extends the concept of reliability. It’s about having the ability to monitor the health and performance of the data product. By using tools to track various metrics like response times and error rates, organizations can proactively manage the data product’s health. This proactive management plays a crucial role in maintaining the product’s reliability, as it allows for the early identification and resolution of potential issues before they escalate." (Jean-Georges Perrin & Eric Broda, "Implementing Data Mesh: Principles and Practice to Design, Build, and Implement Data Mesh", 2024)
"Driven by a fear of missing out (FOMO), many companies have launched AI initiatives. But many failed. For example, customer service chatbots have been introduced to handle inquiries, reduce costs and improve efficiency. But chatbots failed to grasp the complexity of customer issues; they created frustration rather than solving problems. Customers were stuck in repetitive loops, being asked the same questions, and unable to reach a human when needed. Instead of enhancing the customer experience, they delivered spikes in customer complaints instead." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)
No comments:
Post a Comment