SQL Troubles

30 May 2023

Graphical Representation: Understanding (Just the Quotes)

"Charts and graphs are a method of organizing information for a unique purpose. The purpose may be to inform, to persuade, to obtain a clear understanding of certain facts, or to focus information and attention on a particular problem. The information contained in charts and graphs must, obviously, be relevant to the purpose. For decision-making purposes. information must be focused clearly on the issue or issues requiring attention. The need is not simply for 'information', but for structured information, clearly presented and narrowed to fit a distinctive decision-making context. An advantage of having a 'formula' or 'model' appropriate to a given situation is that the formula indicates what kind of information is needed to obtain a solution or answer to a specific problem." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"If two or more data paths ate to appear on the graph. it is essential that these lines be labeled clearly, or at least a reference should be provided for the reader to make the necessary identifications. While clarity seems to be a most obvious goal. graphs with inadequate or confusing labeling do appear in publications, The user should not find identification of data paths troublesome or subject to misunderstanding. The designer normally should place no more than three data paths on the graph to prevent confusion - particularly if the data paths intersect at one or more points on the Cartesian plane." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"Learning to make graphs involves two things: (l) the techniques of plotting statistics, which might be called the artist's job; and (2) understanding the statistics. When you know how to work out graphs, all kinds of statistics will probably become more interesting to you." (Dyno Lowenstein, "Graphs", 1976)

"Understanding is accomplished through: (a) the use of relative size of the shapes used in the graphic; (b) the positioning of the graphic-line forms; (c) shading; (d) the use of scales of measurement; and (e) the use of words to label the forms in the graphic. In addition. in order for a person to attach meaning to a graphic it must also be simple, clear, and appropriate." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"We would wish ‘numerate’ to imply the possession of two attributes. The first of these is an ‘at-homeness’ with numbers and an ability to make use of mathematical skills which enable an individual to cope with the practical mathematical demands of his everyday life. The second is ability to have some appreciation and understanding of information which is presented in mathematical terms, for instance in graphs, charts or tables or by reference to percentage increase or decrease." (Cockcroft Committee, "Mathematics Counts: A Report into the Teaching of Mathematics in Schools", 1982)

"Graphs can present internal accounting data effectively. Because One of the main functions of the accountant is to communicate accounting information to users. accountants should use graphs, at least to the extent that they clarify the presentation of accounting data. present the data fairly, and enhance management's ability to make a more informed decision. It has been argued that the human brain can absorb and understand images more easily than words and numbers, and, therefore, graphs may be better communicative devices than written reports or tabular statements." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"Understandability implies that the graph will mean something to the audience. If the presentation has little meaning to the audience, it has little value. Understandability is the difference between data and information. Data are facts. Information is facts that mean something and make a difference to whoever receives them. Graphic presentation enhances understanding in a number of ways. Many people find that the visual comparison and contrast of information permit relationships to be grasped more easily. Relationships that had been obscure become clear and provide new insights." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"The effective communication of information in visual form, whether it be text, tables, graphs, charts or diagrams, requires an understanding of those factors which determine the 'legibility', 'readability' and 'comprehensibility', of the information being presented. By legibility we mean: can the data be clearly seen and easily read? By readability we mean: is the information set out in a logical way so that its structure is clear and it can be easily scanned? By comprehensibility we mean: does the data make sense to the audience for whom it is intended? Is the presentation appropriate for their previous knowledge, their present information needs and their information processing capacities?" (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"In part, graphing data needs to be iterative because we often do not know what to expect of the data; a graph can help discover unknown aspects of the data, and once the unknown is known, we frequently find ourselves formulating a new question about the data. Even when we understand the data and are graphing them for presentation, a graph will look different from what we had expected; our mind's eye frequently does not do a good job of predicting what our actual eyes will see." (William S Cleveland, "The Elements of Graphing Data", 1985)

"A chart is a bridge between you and your readers. It reveals your skills at comprehending the source information, at mastering presentation methods and at producing the design. Its success depends a great deal on your readers ' understanding of what you are saying, and how you are saying it. Consider how they will use your chart. Will they want to find out from it more information about the subject? Will they just want a quick impression of the data? Or will they use it as a source for their own analysis? Charts rely upon a visual language which both you and your readers must understand." (Bruce Robertson, "How to Draw Charts & Diagrams", 1988)

"Charts and diagrams are the visual presentation of information. Since text and tables of information require close study to obtain the more general impressions of the subject, charts can be used to present readily understandable, easily digestible and, above all, memorable solutions." (Bruce Robertson, "How to Draw Charts & Diagrams", 1988)

"Charts offer opportunities to distort information, to misinform. An old adage can be extended to read: 'There are lies, damned lies, statistics and charts'. Our visual impressions are often more memorable than our understanding of the facts they describe. [...] Never let your design enthusiasms overrule your judgement of the truth." (Bruce Robertson, "How to Draw Charts & Diagrams", 1988)

"Confusion and clutter are failures of design, not attributes of information. And so the point is to find design strategies that reveal detail and complexity - rather than to fault the data for an excess of complication. Or, worse, to fault viewers for a lack of understanding. Among the most powerful devices for reducing noise and enriching the content of displays is the technique of layering and separation, visually stratifying various aspects of the data." (Edward R Tufte, "Envisioning Information", 1990)

"When analyzing data it is many times advantageous to generate a variety of graphs using the same data. This is true whether there is little or lots of data. Reasons for this are: (1) Frequently, all aspects of a group of data can not be displayed on a single graph. (2) Multiple graphs generally result in a more in-depth understanding of the information. (3) Different aspects of the same data often become apparent. (4) Some types of graphs cause certain features of the data to stand out better (5) Some people relate better to one type of graph than another." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"Most dashboards fail to communicate efficiently and effectively, not because of inadequate technology (at least not primarily), but because of poorly designed implementations. No matter how great the technology, a dashboard's success as a medium of communication is a product of design, a result of a display that speaks clearly and immediately. Dashboards can tap into the tremendous power of visual perception to communicate, but only if those who implement them understand visual perception and apply that understanding through design principles and practices that are aligned with the way people see and think." (Stephen Few, "Information Dashboard Design", 2006)

"The purpose of an evidence presentation is to assist thinking. Thus presentations should be constructed so as to assist with the fundamental intellectual tasks in reasoning about evidence: describing the data, making multivariate comparisons, understanding causality, integrating a diversity of evidence, and documenting the analysis. Thus the Grand Principle of analytical design: 'The principles of analytical design are derived from the principles of analytical thinking.' Cognitive tasks are turned into principles of evidence presentation and design." (Edward R Tufte, "Beautiful Evidence", 2006)

"A viewer’s eye must be guided to 'read' the elements in a logical order. The design of an exploratory graphic needs to allow for the additional component of discovery - guiding the viewer to first understand the overall concept and then engage her to further explore the supporting information." (Felice C Frankel & Angela H DePace, "Visual Strategies", 2012)

"Context (information that lends to better understanding the who, what, when, where, and why of your data) can make the data clearer for readers and point them in the right direction. At the least, it can remind you what a graph is about when you come back to it a few months later. […] Context helps readers relate to and understand the data in a visualization better. It provides a sense of scale and strengthens the connection between abstract geometry and colors to the real world." (Nathan Yau, "Data Points: Visualization That Means Something", 2013)

"A good chart can tell a story about the data, helping you understand relationships among data so you can make better decisions. The wrong chart can make a royal mess out of even the best data set." (John H Johnson & Mike Gluck, "Everydata: The misinformation hidden in the little data you consume every day", 2016)

"A scatterplot reveals the strength and shape of the relationship between a pair of variables. A scatterplot represents the two variables by axes drawn at right angles to each other, showing the observations as a cloud of points, each point located according to its values on the two variables. Various lines can be added to the plot to help guide our search for understanding." (Forrest W Young et al, "Visual Statistics: Seeing data with dynamic interactive graphics", 2016)

"One of the main problems with the visual approach to statistical data analysis is that it is too easy to generate too many plots: We can easily become totally overwhelmed by the shear number and variety of graphics that we can generate. In a sense, we have been too successful in our goal of making it easy for the user: Many, many plots can be generated, so many that it becomes impossible to understand our data." (Forrest W Young et al, "Visual Statistics: Seeing data with dynamic interactive graphics", 2016)

"As a first principle, any visualization should convey its information quickly and easily, and with minimal scope for misunderstanding. Unnecessary visual clutter makes more work for the reader’s brain to do, slows down the understanding (at which point they may give up) and may even allow some incorrect interpretations to creep in." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

"Well-designed data graphics provide readers with deeper and more nuanced perspectives, while promoting the use of quantitative information in understanding the world and making decisions." (Carl T Bergstrom & Jevin D West, "Calling Bullshit: The Art of Skepticism in a Data-Driven World", 2020)

Graphical Representation: Meaning (Just the Quotes)

"It is desirable in all chart work to have certain conventions by which colors would be understood to have certain definite meanings. Thus, following railroad practice, red could generally be used in chart work to indicate dangerous or unfavorable conditions, and green to indicate commended features or favorable conditions. Where neither commendation nor adverse criticism is intended, colors such as blue, yellow, brown, etc., could be used." (Willard C Brinton, "Graphic Methods for Presenting Facts", 1919)

"Graphic forms help us to perform and influence two critical functions of the mind: the gathering of information and the processing of that information. Graphs and charts are ways to increase the effectiveness and the efficiency of transmitting information in a way that enhances the reader's ability to process that information. Graphics are tools to help give meaning to information because they go beyond the provision of information and show relationships, trends, and comparisons. They help to distinguish which numbers and which ideas are more important than others in a presentation." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"The more complex the shape of any object. the more difficult it is to perceive it. The nature of thought based on the visual apprehension of objective forms suggests, therefore, the necessity to keep all graphics as simple as possible. Otherwise, their meaning will be lost or ambiguous, and the ability to convey the intended information and to persuade will be inhibited." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"There are two kinds of misrepresentation. In one. the numerical data do not agree with the data in the graph, or certain relevant data are omitted. This kind of misleading presentation. while perhaps hard to determine, clearly is wrong and can be avoided. In the second kind of misrepresentation, the meaning of the data is different to the preparer and to the user." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"There is a technical difference between a bar chart and a histogram in that the number represented is proportional to the length of bar in the former and the area in the latter. This matters if non-uniform binning is used. Bar charts can be used for qualitative or quantitative data, whereas histograms can only be used for quantitative data, as no meaning can be attached to the width of the bins if the data are qualitative." (Roger J Barlow, "Statistics: A guide to the use of statistical methods in the physical sciences", 1989)

"The more clues to meaning that are supplied elsewhere, the less the need for cluttersome scales." (Eric Meyer, "Designing Infographics", 1997)

"[...] the form of a technological object must depend on the tasks it should help with. This is one of the most important principles to remember when dealing with infographics and visualizations: The form should be constrained by the functions of your presentation. There may be more than one form a data set can adopt so that readers can perform operations with it and extract meanings, but the data cannot adopt any form. Choosing visual shapes to encode information should not be based on aesthetics and personal tastes alone." (Alberto Cairo, "The Functional Art", 2011)

"To keep accuracy and efficiency of your diagrams appealing to a potential audience, explicitly describe the encoding principles we used. Titles, labels, and legends are the most common ways to define the meaning of the diagram and its elements." (Vasily Pantyukhin, "Principles of Design Diagramming", 2015)

"Essentially, magnitude is the size of the effect. It’s a way to determine if the results are meaningful. Without magnitude, it’s hard to get a sense of how much something matters. […] the magnitude of an effect can change, depending on the relationship." (John H Johnson & Mike Gluck, "Everydata: The misinformation hidden in the little data you consume every day", 2016)

"Are your insights based on data that is accurate and reliable? Trustworthy data is correct or valid, free from significant defects and gaps. The trustworthiness of your data begins with the proper collection, processing, and maintenance of the data at its source. However, the reliability of your numbers can also be influenced by how they are handled during the analysis process. Clean data can inadvertently lose its integrity and true meaning depending on how it is analyzed and interpreted." (Brent Dykes, "Effective Data Storytelling: How to Drive Change with Data, Narrative and Visuals", 2019)

"Before you can even consider creating a data story, you must have a meaningful insight to share. One of the essential attributes of a data story is a central or main insight. Without a main point, your data story will lack purpose, direction, and cohesion. A central insight is the unifying theme (telos appeal) that ties your various findings together and guides your audience to a focal point or climax for your data story. However, when you have an increasing amount of data at your disposal, insights can be elusive. The noise from irrelevant and peripheral data can interfere with your ability to pinpoint the important signals hidden within its core." (Brent Dykes, "Effective Data Storytelling: How to Drive Change with Data, Narrative and Visuals", 2019)

"When narrative is coupled with data, it helps to explain to your audience what’s happening in the data and why a particular insight is important. Ample context and commentary are often needed to fully appreciate an analysis finding. The narrative element adds structure to the data and helps to guide the audience through the meaning of what’s being shared." (Brent Dykes, "Effective Data Storytelling: How to Drive Change with Data, Narrative and Visuals", 2019)

29 May 2023

Graphical Representation: Line Graphs (Just the Quotes)

"Data should not be forced into an uncomfortable or improper mold. For example, data that is appropriate for line graphs is not usually appropriate for circle charts and in any case not without some arithmetic transformation. Only graphs that are designed to fit the data can be used profitably." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"The numerous design possibilities include several varieties of line graphs that are geared to particular types of problems. The design of a graph should be adapted to the type of data being structured. The data might be percentages, index numbers, frequency distributions, probability distributions, rates of change, numbers of dollars, and so on. Consequently, the designer must be prepared to structure his graph accordingly." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"While circle charts are not likely to present especially new or creative ideas, they do help the user to visualize relationships. The relationships depicted by circle charts do not tend to be very complex, in contrast to those of some line graphs. Normally, the circle chart is used to portray a common type of relationship (namely. part-to-total) in an attractive manner and to expedite the message transfer from designer to user." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"There are several uses for which the line graph is particularly relevant. One is for a series of data covering a long period of time. Another is for comparing several series on the same graph. A third is for emphasizing the movement of data rather than the amount of the data. It also can be used with two scales on the vertical axis, one on the right and another on the left, allowing different series to use different scales, and it can be used to present trends and forecasts." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"In the case of graphs, the number of lines which can be included on any one illustration will depend largely on how close the lines are and how often they cross one another. Three or four is likely to be the maximum acceptable number. In some instances, there may be an argument for using several graphs with one line each as opposed to one graph with multiple lines. It has been shown that these two arrangements are equally satisfactory if the user wishes to read off the value of specific points; if, however, he wishes to compare the lines, than the single multi-line graph is superior." (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"A connected graph is appropriate when the time series is smooth, so that perceiving individual values is not important. A vertical line graph is appropriate when it is important to see individual values, when we need to see short-term fluctuations, and when the time series has a large number of values; the use of vertical lines allows us to pack the series tightly along the horizontal axis. The vertical line graph, however, usually works best when the vertical lines emanate from a horizontal line through the center of the data and when there are no long-term trends in the data." (William S Cleveland, "The Elements of Graphing Data", 1985)

"A bar graph typically presents either averages or frequencies. It is relatively simple to present raw data (in the form of dot plots or box plots). Such plots provide much more information. and they are closer to the original data. If the bar graph categories are linked in some way - for example, doses of treatments - then a line graph will be much more informative. Very complicated bar graphs containing adjacent bars are very difficult to grasp. If the bar graph represents frequencies. and the abscissa values can be ordered, then a line graph will be much more informative and will have substantially reduced chart junk." (Gerald van Belle, "Statistical Rules of Thumb", 2002)

Graphical Representation: Relations (Just the Quotes)

"Logging skewed variables also helps to reveal the patterns in the data. […] the rescaling of the variables by taking logarithms reduces the nonlinearity in the relationship and removes much of the clutter resulting from the skewed distributions on both variables; in short, the transformation helps clarify the relationship between the two variables. It also […] leads to a theoretically meaningful regression coefficient." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)

"If you want to dramatize comparisons in relation to the whole. use a pie chart. If you want to add coherence to the narrative, the pie chart also helps because it depicts a whole. If your main interest is in stressing the relationship of one factor to another, use bar charts. If you wish to achieve all these effects. you can use either type of chart. and decide on the basis of which one is more aesthetically or pictorially interesting." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"In order to be easily understood, a display of information must have a logical structure which is appropriate for the user's knowledge and needs, and this structure must be clearly represented visually. In order to indicate structure, it is necessary to be able to eemphasiz, divide and relate items of information. Visual emphasis can be used to indicate a hierarchical relationship between items of information, as in the case of systems of headings and subheadings for example. Visual separation of items can be used to indicate that they are different in kind or are unrelated functionally, and similarly a visual relationship between items will imply that they are of a similar kind or bear some functional relation to one another. This kind of visual 'coding' helps the reader to appreciate the extent and nature of the relationship between items of information, and to adopt an appropriate scanning strategy." (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"Statistics are numerical statements of facts in any department of inquiry, placed in relation to each other; statistical methods are devices for abbreviating and classifying the statements and making clear the relations." (Arthur L Bowley, "An Elementary Manual of Statistics", 1934)

"Scatter charts show the relationships between information, plotted as points on a grid. These groupings can portray general features of the source data, and are useful for showing where correlationships occur frequently. Some scatter charts connect points of equal value to produce areas within the grid which consist of similar features." (Bruce Robertson, "How to Draw Charts & Diagrams", 1988)

"Area graphs are generally not used to convey specific values. Instead, they are most frequently used to show trends and relationships, to identify and/or add emphasis to specific information by virtue of the boldness of the shading or color, or to show parts-of-the-whole." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"A good graph displays relationships and structures that are difficult to detect by merely looking at the data." (Gerald van Belle, "Statistical Rules of Thumb", 2002)

"The content and context of the numerical data determines the most appropriate mode of presentation. A few numbers can be listed, many numbers require a table. Relationships among numbers can be displayed by statistics. However, statistics, of necessity, are summary quantities so they cannot fully display the relationships, so a graph can be used to demonstrate them visually. The attractiveness of the form of the presentation is determined by word layout, data structure, and design." (Gerald van Belle, "Statistical Rules of Thumb", 2002)

"Three key aspects of presenting high dimensional data are: rendering, manipulation, and linking. Rendering determines what is to be plotted, manipulation determines the structure of the relationships, and linking determines what information will be shared between plots or sections of the graph." (Gerald van Belle, "Statistical Rules of Thumb", 2002)

"Color can tell us where to look, what to compare and contrast, and it can give us a visual scale of measure. Because color can be so effective, it is often used for multiple purposes in the same graphic - which can create graphics that are dazzling but difficult to interpret. Separating the roles that color can play makes it easier to apply color specifically for encouraging different kinds of visual thinking. [...] Choose colors to draw attention, to label, to show relationships (compare and contrast), or to indicate a visual scale of measure." (Felice C Frankel & Angela H DePace, "Visual Strategies", 2012)

Anker V Andersen - Collected Quotes

"An economic justification for computer graphics is that the organization spends an enormous amount of money on data processing, often providing managers with too many reports, too many data, and an overload of information. The report output has to be condensed into a more usable form. The computer graph essentially is the data represented in a structured pictorial form. The role of the graph is to provide meaningful reports. To the extent that it does. it can be justified." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"Graphs are used to meet the need to condense all the available information into a more usable quantity. The selection process of combining and condensing will inevitably produce a less than complete study and will lead the user in certain directions, producing a potential for misleading." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"Reliability is highly valued by accountants and has been defined as 'the faithfulness with which it (information) represents what it purports to represent'. The reason reliability is so important is that an essential characteristic of an accounting report is its acceptance, and if a report is considered to be misleading or superfluous, it and future reports will be disregarded." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"The bar graph and the column graph are popular because they are simple and easy to read. These are the most versatile of the graph forms. They can be used to display time series, to display the relationship between two items, to make a comparison among several items, and to make a comparison between parts and the whole (total). They do not appear to be as 'statistical', which is an advantage to those people who have negative attitudes toward statistics. The column graph shows values over time, and the bar graph shows values at a point in time. bar graph compares different items as of a specific time (not over time)." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"The scales used are important; contracting or expanding the vertical or horizontal scales will change the visual picture. The trend lines need enough grid lines to obviate difficulty in reading the results properly. One must be careful in the use of cross-hatching and shading, both of which can create illusions. Horizontal rulings tend to reduce the appearance. while vertical lines enlarge it. In summary, graphs must be reliable, and reliability depends not only on what is presented but also on how it is presented." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

24 May 2023

Strategic Management: Principles (Just the Quotes)

"It is superfluous to suppose that what can be accounted for by a few principles has been produced by many." (Thomas Aquinas, "Summa Theologica", cca. 1266-1273)

"Men are often led into errors by the love of simplicity, which disposes us to reduce things to few principles, and to conceive a greater simplicity in nature than there really is." (Thomas Reid, "Essays on the Intellectual Powers of Man", 1785)

"Amid the pressure of great events, a general principle gives no help [in decision making]." (Georg W Hegel, "Lectures on the Philosophy of History", 1837)

"Failure to succeed greatly in management usually occurs not so much from lack of knowledge of the important principles of the science of management as from failure to apply them. Most of the principles of successful management are old, and many of them have received sufficient publicity to be well known, but managers are curiously prone to look upon managerial success as a personal attribute that is slightly dependent on principles or laws." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"We make a difference between general and special principles of strategy. The general principles originate directly from the aim and the nature of Chess, and therefore they are constantly in force. It is, for instance, a general principle which goes without saying, that one has to procure the greatest possible freedom of action for one's men. The special principles apply only if the position shows certain peculiarities on account of which a special line of strategy has to be followed." (Dr. Max Euwe, "Strategy & Tactices in chess", 1937)

"It is always more easy to discover and proclaim general principles than it is to apply them." (Winston Churchill, "The Second World War: The gathering storm", 1948)

"The essence of managership is the achievement of coordination among people. Coordination is a complex concept, including principles by which harmonious enterprise activity can be accomplished and the many techniques for achieving the greatest synchronized effort." (Harold Koontz & Cyril O Donnell, "Principles of Management", 1955)

"Principles are the territory. Values are maps. When we value correct principles, we have truth - a knowledge of things as they are." (Stephen R Covey, "The 7 Habits of Highly Effective People", 1989)

"It is possible to learn strategic flexibility [...] however, that it is difficult to teach it. It is not a matter of learning a few readily grasped general principles, but of learning a lot of small, 'local' rules, each of which is applicable in a limited area. The point is not to learn how to drive a steamroller with which one can flatten all problems in the same way, but to learn the adroitness of a puppeteer, who at one time holds many strings in his hands and who is able to adapt his movements to the given circumstances in the most sophisticated ways." (Dietrich Dörner, "The Logic of Failure", Philosophical Transactions of the Royal Society of London (B), 1990)

"Trust is the glue of life. It's the most essential ingredient in effective communication. It's the foundational principle that holds all relationships - marriages, families, and organizations of every kind - together." (Stephen Covey, "First Things First", 1994)

"Values are social norms - they're personal, emotional, subjective, and arguable. All of us have values. [...] The question you must ask yourself is, Are your values based upon principles? In the last analysis, principles are natural laws - they're impersonal, factual, objective and self-evident. Consequences are governed by principles and behavior is governed by values; therefore, value principles." (Stephen R Covey, "The 8th Habit: From Effectiveness to Greatness", 2004)

"For values or guiding principles to be truly effective they have to be verbs. It's not 'integrity'," it's 'always do the right thing'. It's not 'innovation', it's 'look at the problem from a different angle'. Articulating our values as verbs gives us a clear idea - we have a clear idea of how to act in any situation." (Simon Sinek, "Start With Why: How Great Leaders Inspire Everyone to Take Action", 2009)

21 May 2023

Allan C Haskell - Collected Quotes

"All charts should be given a comprehensive title, that there may be no doubt as to what data the chart is intended to cover. This is especially necessary in connection with separate exhibits used for illustrating statements in special reports, as when the charts are not given distinct titles they are valueless as records when detached from the reports." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"An exact knowledge of conditions, and consequent timely application of praise or of constructive criticism, is one of the chief forces of the executive in securing satisfactory results. Undeserved criticism is unjust and destroys-initiative, while unmerited praise tends to render the executive ridiculous in the eyes of his subordinates; both are detrimental to discipline and weaken the power of the executive." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"At the time a chart is drawn explanatory notes should be used unsparingly and all curves or diagrams should be so keyed or lettered that there will be no question as to just what each represents when the chart is referred to later. When it is not necessary to make blue-prints, different colored inks or water-color paints may be used in a drawing-pen to distinguish one curve from another [...]" (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Every business has its own particular sort of rat holes, through which its profits are carried piecemeal, and in quantities hardly noticeable at the time, but which aggregate thousands every year. The best way to plug these sources of loss is by accumulating data in regard to them and then keeping this data prominently before the executive." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"For any manager to utilize graphic methods for visualizing the vital facts of his business, in the first place it must be impressed upon his that the method will produce the results for him and then he must know how to get up a chart correctly, and last, but far from least, he must know what the essential facts of his business are. Charts, in themselves, mean little and like many another force for the accomplishment of good, if misdirected, may result unprofitably." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Generally speaking, the plotting of a curve consists of graphically representing numbers and equations by the relation of points and lines with reference to other given lines or to a given point." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Graphic methods convey to the mind a more comprehensive grasp of essential features than do written reports, because one can naturally gather interesting details from a picture in far less time than from a written description. Further than this, the examination of a picture allows one to make deductions of his own, while in the case of a written description the reader must, to a great degree, accept the conclusions of the author." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Graphic representation by means of charts depends upon the super-position of special lines or curves upon base lines drawn or ruled in a standard manner. For the economic construction of these charts as well as their correct use it is necessary that the standard rulings be correctly designed." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"In working through graphics one has, however, to be exceedingly cautious in certain particulars, for instance, when a set of figures, dynamical or financial, are available they are, so long as they are tabulated, instinctively taken merely at their face value. When plotted, however, there is a temptation to extrapolation which is well nigh irresistible to the untrained mind. Sometimes the process can be safely employed, but it requires a rather comprehensive knowledge of the facts that lie back of the data to tell when to go ahead and when to stop." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"It is not possible to lay down any hard and fast rules for determining what chart is the best for any given problem. Ordinarily that one is the best which will produce the quickest and clearest results. but unfortunately it is not always possible to construct the clearest one in the least time. Experience is the best guide. Generally speaking, a rectilinear chart is best adapted for equations of the first degree, logarithmic for those other than the first degree and not containing over two variables, and alignment charts where there are three or more variables. However, nearly every person becomes more or less familiar with one type of chart and prefers to adhere to the use of that type because he does not care to take the time and trouble to find out how to use the others. It is best to know what the possibilities of all types are and to be governed accordingly when selecting one or the other for presenting or working out certain data." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Many people imagine that graphic charts cannot be understood except by expert mathematicians who have devoted years of study to the subject. This is a mistaken idea, and if instead of passing over charts as if they were something beyond their comprehension more people would make an effort to read them, much valuable time would be saved. It is true that some charts covering technical data are difficult even for an expert mathematician to understand, but this is more often the fault of the person preparing the charts than of the system." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Nothing is so illuminating as a set of properly proportioned diagrams. [...] In addition to the significance of graphics in analytical work, it is likewise a valuable aid to the memory. A picture is manifestly more readily retained in mind than a description of the same subject, no matter how vividly it may have been expressed. A pictorial or diagrammatic illustration usually produces a firmer and more lasting impression than any composition of words or tabulation of figures, however well they may be arranged or set forth." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"One of the most valuable uses to which charts may be put is that of analysis - the analytical study of facts which many times do not adhere to any known mathematical law, but which are nevertheless capable of being reduced to a workable form." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Since a table is a collection of certain sets of data, a chart with one curve representing each set of data can be made to take the place of the table. Wherever a chart can be plotted by straight lines, the speed of this is infinitely greater than making out a table, and where the curvilinear law is known, or can be approximated by the use of the empiric law, the speed is but little less." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"The best-known function of charts is for demonstration purposes, to show up facts. When so presented they do not require a trained mind for their appreciation, since the spatial sense through the optic nerve is among the commonest of the human attributes." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"The practice of drawing several curves on the same sheet is not to be commended except in cases where the curves will not intersect. A crowded chart on which the curves frequently intersect resembles a Chinese puzzle more than a graphic record, and a report submitted in figures is to be preferred to a chart of this kind. Even when the curves do not intersect, they should be made in different colors in order that they may be readily distinguished, one from the other." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"The problem of the executive, then - once his organization is perfected - is to secure live data covering the exact conditions of the business at all times. These data should be arranged so as to give him all the facts, subordinated according to their relative bearing upon net earnings, and do so with the least demand upon his time. Furthermore, these facts must be so exhibited that the general laws underlying the business may be easily and accurately deduced and standards of accomplishment set which will be a continual incentive to greater accomplishment." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"The zero of the scale should appear on every chart, and should shown by a heavy line carried across the sheet. If this is not done the reader may assume the bottom of the sheet to be zero and so be mis- led. The scale should be graduated from zero to a little over the maximum figure to be plotted on the charts, so that there will be a space between the highest peak on the curve and the top of the chart." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Under certain conditions, however, the ordinary form of graphic chart is slightly misleading. It will be conceded that its true function is to portray comparative fluctuations. This result is practically secured when the factors or quantities compared are nearly of the same value or volume, but analysis will show that this is not accomplished when the amounts compared differ greatly in value or volume. [...] The same criticism applies to charts which employ or more scales for various curve. If the different scale are in proper proportion, the result is the same as with one scale, but when two or more scales are used which are not proportional an indication may be given with respect to comparative fluctuations which is absolutely false." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"When dealing with very large quantities it is not always practicable to use a scale which starts at zero, and is carried up by even steps to a figure representing the highest peak on the curve. Such a chart would either be too large for convenient handling, or else the scale would have to be condensed so that only very large fluctuations would be indicated on the curve. In a ease of this kind the best practice is to start the at zero, and just above this point draw a wavy line across the sheet to indicate that the scale is broken at this point. This line can be very easily drawn with an ordinary serrated edge ruler as used by many accountants. The scale starts again on the upper side of the wavy line at a figure a little lower than the lowest point on the curve, and is carried up by even steps to a figure a little above the highest point to be shown on the curve." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"Wherever unusual peaks or valleys occur on a curve it is a good plan to mark these points with a small figure inside a circle. This figure should refer to a note on the back of the chart explaining the reason for the unusual condition. It is not always sufficient to show that a certain item is unusually high or low; the executive will want to know why it is that way." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

"With the ordinary scale, fluctuations in large factors are very noticeable, while relatively greater fluctuations in smaller factors are barely apparent. The semi-logarithmic scale permits the graphic representation of changes in every quantity on the same basis, without respect to the magnitude of the quantity itself. At the same time, it shows the actual value by reference to the numbers in the scale column. By indicating both absolute and relative value and changes to one scale, it combines the advantages of both the natural and percentage scale, without the disadvantages of either." (Allan C Haskell, "How to Make and Use Graphic Charts", 1919)

Robert Lefferts - Collected Quotes

"A flow chart is a graphic method to show pictorially how a series of activities, procedures. operations. events. ideas, or other factors are related to each other. It shows the sequence, cycle. or flow of these factors and how they are connected in a series of steps from beginning to end." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"A good graphic must give the impression that its various parts all belong together. They must be arranged in such a way that the illustration looks like a single entity. A good graphic chart should be more than just the sum of its individual lines, shapes, and shades. It should be more than the individual bars in a bar chart, more than the pieces of a pie chart, more than the boxes in a flow chart. Unity requires the establishment of coherent relationships among the component parts of the drawing. These relationships can be depicted in a very direct manner through the use of connecting lines that serve to connect shapes." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"A graphic is an illustration that, like a painting or drawing, depicts certain images on a flat surface. The graphic depends on the use of lines and shapes or symbols to represent numbers and ideas and show comparisons, trends, and relationships. The success of the graphic depends on the extent to which this representation is transmitted in a clear and interesting manner." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"A pie chart is comprised of a circle that is divided into segments by straight lines within the circle. The circle represents the total or whole amount. Each segment or wedge of the circle represents the proportion that a particular factor is of the total or whole amount. Thus, a pie chart in its entirety always represents whole amounts of either 100% or a total absolute number, such as 100 cents or 5,000 people. All of the segments of the pie when taken together (that is, in the aggregate) must add up to the total." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"An organization chart is a graphic device that uses pictorial methods to show qualitative information about an organization. [...] The organization chart can be used to show one or more of three things: (1) What the various staff positions in the organization are, how they are structurally related to each other and the span of control and chain of command within the organization. (2) What the different units of the organization are and how they are arranged and related to each other. (3) What the various functions are within the organization and how they are organized and related." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"For most line charts the maximum number of plotted lines should not exceed five; three or fewer is the ideal number. When multiple plotted lines are shown each line should be differentiated by using (a) a different type of line and/or (b) different plotting marks, if shown, and (c) clearly differentiated labeling." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"Graphic charts are ways of presenting quantitative as well as qualitative information in an efficient and effective visual form. Numbers and ideas presented graphically are often more easily understood. remembered. and integrated than when they are presented in narrative or tabular form. Descriptions. trends. relationships, and comparisons can be made more apparent. Less time is required to present and comprehend information when graphic methods are employed. As the old truism states, 'One picture is worth a thousand words.'" (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"It should be noted that graphics for the purpose of clarity should not be a substitute for words and numbers in the narrative text. The graphics presentation is used to supplement the narrative; otherwise. there wouldn't be anything to clarify." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"Some believe that the vertical bar should be used when comparing similar items for different time periods and the horizontal bar for comparing different items for the same time period. However, most people find the vertical-bar format easier to prepare and read. and a more effective way to show most types of comparisons." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"The bar or column chart is the easiest type of graphic to prepare and use in reports. It employs a simple form: four straight lines that are joined to construct a rectangle or oblong box. When the box is shown horizontally it is called a bar; when it is shown vertically it is called a column. [...] The bar chart is an effective way to show comparisons between or among two or more items. It has the added advantage of being easily understood by readers who have little or no background in statistics and who are not accustomed to reading complex tables or charts." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"The preparation of well-designed graphics is both an art and a skill. There are many different ways to go about the task, and readers are urged to develop their own approaches. Graphics can be creative and fun. At the same time, they require a degree of orderly and systematic work." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"To see is to reason. Thus, the use of visual forms of communication has great potential for influencing what a person thinks. Graphic presentation is always much more than a way to present just facts or information. Rather, it is a way to influence thought, and, as such, graphics can be a powerful mode of persuasion." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"Unlike some art forms. good graphics should be as concrete. geometrical, and representational as possible. A rectangle should be drawn as a rectangle, leaving nothing to the reader's imagination about what you are trying to portray. The various lines and shapes used in a graphic chart should be arranged so that it appears to be balanced. This balance is a result of the placement of shapes and lines in an orderly fashion." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

15 May 2023

Data Science: Fitting (Just the Quotes)

"It may often happen that an inefficient statistic is accurate enough to answer the particular questions at issue. There is however, one limitation to the legitimate use of inefficient statistics which should be noted in advance. If we are to make accurate tests of goodness of fit, the methods of fitting employed must not introduce errors of fitting comparable to the errors of random sampling; when this requirement is investigated, it appears that when tests of goodness of fit are required, the statistics employed in fitting must be not only consistent, but must be of 100 percent efficiency. This is a very serious limitation to the use of inefficient statistics, since in the examination of any body of data it is desirable to be able at any time to test the validity of one or more of the provisional assumptions which have been made." (Sir Ronald A Fisher, "Statistical Methods for Research Workers", 1925)

"[…] fitting lines to relationships between variables is often a useful and powerful method of summarizing a set of data. Regression analysis fits naturally with the development of causal explanations, simply because the research worker must, at a minimum, know what he or she is seeking to explain." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)

"Fitting lines to relationships between variables is the major tool of data analysis. Fitted lines often effectively summarize the data and, by doing so, help communicate the analytic results to others. Estimating a fitted line is also the first step in squeezing further information from the data." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)

"Fitting data means finding mathematical descriptions of structure in the data. An additive shift is a structural property of univariate data in which distributions differ only in location and not in spread or shape. […] The process of identifying a structure in data and then fitting the structure to produce residuals that have the same distribution lies at the heart of statistical analysis. Such homogeneous residuals can be pooled, which increases the power of the description of the variation in the data." (William S Cleveland, "Visualizing Data", 1993)

"Many good things happen when data distributions are well approximated by the normal. First, the question of whether the shifts among the distributions are additive becomes the question of whether the distributions have the same standard deviation; if so, the shifts are additive. […] A second good happening is that methods of fitting and methods of probabilistic inference, to be taken up shortly, are typically simple and on well understood ground. […] A third good thing is that the description of the data distribution is more parsimonious." (William S Cleveland, "Visualizing Data", 1993)

"Many of the applications of visualization in this book give the impression that data analysis consists of an orderly progression of exploratory graphs, fitting, and visualization of fits and residuals. Coherence of discussion and limited space necessitate a presentation that appears to imply this. Real life is usually quite different. There are blind alleys. There are mistaken actions. There are effects missed until the very end when some visualization saves the day. And worse, there is the possibility of the nearly unmentionable: missed effects." (William S Cleveland, "Visualizing Data", 1993)

"Time-series forecasting is essentially a form of extrapolation in that it involves fitting a model to a set of data and then using that model outside the range of data to which it has been fitted. Extrapolation is rightly regarded with disfavour in other statistical areas, such as regression analysis. However, when forecasting the future of a time series, extrapolation is unavoidable." (Chris Chatfield, "Time-Series Forecasting" 2nd Ed, 2000)

"An oft-repeated rule of thumb in any sort of statistical model fitting is 'you can't fit a model with more parameters than data points'. This idea appears to be as wide-spread as it is incorrect. On the contrary, if you construct your models carefully, you can fit models with more parameters than datapoints [...]. A model with more parameters than datapoints is known as an under-determined system, and it's a common misperception that such a model cannot be solved in any circumstance. [...] this misconception, which I like to call the 'model complexity myth' [...] is not true in general, it is true in the specific case of simple linear models, which perhaps explains why the myth is so pervasive." (Jake Vanderplas, "The Model Complexity Myth", 2015)

"Supervised learning, or function approximation, is simply fitting data to a function of any variety. […] Unsupervised learning involves figuring out what makes the data special. […] Reinforcement learning involves figuring out how to play a multistage game with rewards and payoffs. Think of it as the algorithms that optimize the life of something." (Matthew Kirk, "Thoughtful Machine Learning", 2015)

"Often when people relate essentially the same variable in two different groups, or at two different times, they see this same phenomenon - the tendency of the response variable to be closer to the mean than the predicted value. Unfortunately, people try to interpret this by thinking that the performance of those far from the mean is deteriorating, but it’s just a mathematical fact about the correlation. So, today we try to be less judgmental about this phenomenon and we call it regression to the mean. We managed to get rid of the term 'mediocrity', but the name regression stuck as a name for the whole least squares fitting procedure - and that’s where we get the term regression line." (Richard D De Veaux et al, "Stats: Data and Models", 2016)

"Just as they did thirty years ago, machine learning programs (including those with deep neural networks) operate almost entirely in an associational mode. They are driven by a stream of observations to which they attempt to fit a function, in much the same way that a statistician tries to fit a line to a collection of points. Deep neural networks have added many more layers to the complexity of the fitted function, but raw data still drives the fitting process. They continue to improve in accuracy as more data are fitted, but they do not benefit from the 'super-evolutionary speedup'." (Judea Pearl & Dana Mackenzie, "The Book of Why: The new science of cause and effect", 2018)

"Random forests are essentially an ensemble of trees. They use many short trees, fitted to multiple samples of the data, and the predictions are averaged for each observation. This helps to get around a problem that trees, and many other machine learning techniques, are not guaranteed to find optimal models, in the way that linear regression is. They do a very challenging job of fitting non-linear predictions over many variables, even sometimes when there are more variables than there are observations. To do that, they have to employ 'greedy algorithms', which find a reasonably good model but not necessarily the very best model possible." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

Graphical Representation: Aesthetics (Just the Quotes)

"Good design looks right. It is simple (clear and uncomplicated). Good design is also elegant, and does not look contrived. A map should be aesthetically pleasing, thought provoking, and communicative." (Arthur H Robinson, "Elements of Cartography", 1953)

"The practice of framing an illustration with a drawn rectangle is not recommended. This kind of typographic detailing should never be added purely for aesthetic reasons or for decoration. A simple, purely functional drawing will automatically be aesthetically pleasing. Unnecessary lines usually reduce both legibility and attractiveness." (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"The main goal of data visualization is its ability to visualize data, communicating information clearly and effectively. It doesn’t mean that data visualization needs to look boring to be functional or extremely sophisticated to look beautiful. To convey ideas effectively, both aesthetic form and functionality need to go hand in hand, providing insights into a rather sparse and complex dataset by communicating its key aspects in a more intuitive way. Yet designers often tend to discard the balance between design and function, creating gorgeous data visualizations which fail to serve its main purpose - communicate information." (Vitaly Friedman, "Data Visualization and Infographics", Smashing Magazine, 2008)

"Data art is characterized by a lack of structured narrative and absence of any visual analysis capability. Instead, the motivation is much more about creating an artifact, an aesthetic representation or perhaps a technical/technique demonstration. At the extreme end, a design may be more guided by the idea of fun or playfulness or maybe the creation of ornamentation." (Andy Kirk, "Data Visualization: A successful design process", 2012)

"Good visualization is a winding process that requires statistics and design knowledge. Without the former, the visualization becomes an exercise only in illustration and aesthetics, and without the latter, one of only analyses. On their own, these are fine skills, but they make for incomplete data graphics. Having skills in both provides you with the luxury - which is growing into a necessity - to jump back and forth between data exploration and storytelling." (Nathan Yau, "Data Points: Visualization That Means Something", 2013)

"Visualization can be appreciated purely from an aesthetic point of view, but it’s most interesting when it’s about data that’s worth looking at. That’s why you start with data, explore it, and then show results rather than start with a visual and try to squeeze a dataset into it. It’s like trying to use a hammer to bang in a bunch of screws. […] Aesthetics isn’t just a shiny veneer that you slap on at the last minute. It represents the thought you put into a visualization, which is tightly coupled with clarity and affects interpretation." (Nathan Yau, "Data Points: Visualization That Means Something", 2013)

Data Science: Transformations (Just the Quotes)

"Logging size transforms the original skewed distribution into a more symmetrical one by pulling in the long right tail of the distribution toward the mean. The short left tail is, in addition, stretched. The shift toward symmetrical distribution produced by the log transform is not, of course, merely for convenience. Symmetrical distributions, especially those that resemble the normal distribution, fulfill statistical assumptions that form the basis of statistical significance testing in the regression model." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)

"The logarithmic transformation serves several purposes: (1) The resulting regression coefficients sometimes have a more useful theoretical interpretation compared to a regression based on unlogged variables. (2) Badly skewed distributions - in which many of the observations are clustered together combined with a few outlying values on the scale of measurement - are transformed by taking the logarithm of the measurements so that the clustered values are spread out and the large values pulled in more toward the middle of the distribution. (3) Some of the assumptions underlying the regression model and the associated significance tests are better met when the logarithm of the measured variables is taken." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)

"The logarithm is one of many transformations that we can apply to univariate measurements. The square root is another. Transformation is a critical tool for visualization or for any other mode of data analysis because it can substantially simplify the structure of a set of data. For example, transformation can remove skewness toward large values, and it can remove monotone increasing spread. And often, it is the logarithm that achieves this removal." (William S Cleveland, "Visualizing Data", 1993)

"Compound errors can begin with any of the standard sorts of bad statistics - a guess, a poor sample, an inadvertent transformation, perhaps confusion over the meaning of a complex statistic. People inevitably want to put statistics to use, to explore a number's implications. [...] The strengths and weaknesses of those original numbers should affect our confidence in the second-generation statistics." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"All forms of complex causation, and especially nonlinear transformations, admittedly stack the deck against prediction. Linear describes an outcome produced by one or more variables where the effect is additive. Any other interaction is nonlinear. This would include outcomes that involve step functions or phase transitions. The hard sciences routinely describe nonlinear phenomena. Making predictions about them becomes increasingly problematic when multiple variables are involved that have complex interactions. Some simple nonlinear systems can quickly become unpredictable when small variations in their inputs are introduced." (Richard N Lebow, "Forbidden Fruit: Counterfactuals and International Relations", 2010)

"Transforming data to measurements of a different kind can clarify and simplify hypotheses that have already been generated and can reveal patterns that would otherwise be hidden." (Forrest W Young et al, "Visual Statistics: Seeing data with dynamic interactive graphics", 2016)

"Feature generation (or engineering, as it is often called) is where the bulk of the time is spent in the machine learning process. As social science researchers or practitioners, you have spent a lot of time constructing features, using transformations, dummy variables, and interaction terms. All of that is still required and critical in the machine learning framework. One difference you will need to get comfortable with is that instead of carefully selecting a few predictors, machine learning systems tend to encourage the creation of lots of features and then empirically use holdout data to perform regularization and model selection. It is common to have models that are trained on thousands of features." (Rayid Ghani & Malte Schierholz, "Machine Learning", 2017)

"Data analysis and data mining are concerned with unsupervised pattern finding and structure determination in data sets. The data sets themselves are explicitly linked as a form of representation to an observational or otherwise empirical domain of interest. 'Structure' has long been understood as symmetry which can take many forms with respect to any transformation, including point, translational, rotational, and many others. Symmetries directly point to invariants, which pinpoint intrinsic properties of the data and of the background empirical domain of interest. As our data models change, so too do our perspectives on analysing data." (Fionn Murtagh, "Data Science Foundations: Geometry and Topology of Complex Hierarchic Systems and Big Data Analytics", 2018)

"Many statistical procedures perform more effectively on data that are normally distributed, or at least are symmetric and not excessively kurtotic (fat-tailed), and where the mean and variance are approximately constant. Observed time series frequently require some form of transformation before they exhibit these distributional properties, for in their 'raw' form they are often asymmetric." (Terence C Mills, "Applied Time Series Analysis: A practical guide to modeling and forecasting", 2019)

14 May 2023

Robert L Harris - Collected Quotes

"A coordinate is a number or value used to locate a point with respect to a reference point, line, or plane. Generally the reference is zero. […] The major function of coordinates is to provide a method for encoding information on charts, graphs, and maps in such a way that viewers can accurately decode the information after the graph or map has been generated." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"Although in most cases the actual value designated by a bar is determined by the location of the end of the bar, many people associate the length or area of the bar with its value. As long as the scale is linear, starts at zero, is continuous, and the bars are the same width, this presents no problem. When any of these conditions are changed, the potential exists that the graph will be misinterpreted." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"As a general rule, the fewer the time intervals used in the averaging process, the more closely the moving average curve resembles the curve of the actual data. Conversely, the greater the number of intervals, the smoother the moving average curve. […] Moving average curves tend to have a delayed reaction to changes." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"Grouped area graphs sometimes cause confusion because the viewer cannot determine whether the areas for the data series extend down to the zero axis. […] Grouped area graphs can handle negative values somewhat better than stacked area graphs but they still have the problem of all or portions of data curves being hidden by the data series towards the front." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"Standard quantile graphs offer certain advantages over cumulative percent frequency graphs. Among these advantages are ease of construction, actual data points are shown as opposed to summaries of class intervals, no decisions are required as to what the best size class interval might be, the same curve functions as a less-than and greater-than curve, and the actual maximum and minimum values are shown on the graph." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"Technically, there is no limit as to the number of data series that can be plotted on a single graph. Practically, if the number goes above three or four the graph becomes confusing." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

"When approximations are all that are needed, stacked area graphs are usually adequate. When accuracy is desired, this type of graph is generally not used, particularly when the values fluctuate significantly and/or the slopes of the curves are steep." (Robert L Harris, "Information Graphics: A Comprehensive Illustrated Reference", 1996)

09 May 2023

Graphical Representation: Axes (Just the Quotes)

"As a general rule, plotted points and graph lines should be given more 'weight' than the axes. In this way the 'meat' will be easily distinguishable from the 'bones'. Furthermore, an illustration composed of lines of unequal weights is always more attractive than one in which all the lines are of uniform thickness. It may not always be possible to emphasise the data in this way however. In a scattergram, for example, the more plotted points there are, the smaller they may need to be and this will give them a lighter appearance. Similarly, the more curves there are on a graph, the thinner the lines may need to be. In both cases, the axes may look better if they are drawn with a somewhat bolder line so that they are easily distinguishable from the data." (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"The frequency of labelled scale calibrations on the axes of a graph can significantly affect the accuracy with which it is interpreted. As little interpolation as possible should be required of the user, in order to minimise errors. If single units cannot be marked, it has been suggested that multiples of 2,5 or 10 should be used." (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"The plotted points on a graph should always be made to stand out well. They are, after all, the most important feature of a graph, since any lines linking them are nearly always a matter of conjecture. These lines should stop just short of the plotted points so that the latter are emphasised by the space surrounding them. Where a point happens to fall on an axis line, the axis should be broken for a short distance on either side of the point." (Linda Reynolds & Doig Simmonds, "Presentation of Data in Science" 4th Ed, 1984)

"An axis is the ruler that establishes regular intervals for measuring information. Because it is such a widely accepted convention, it is often taken for granted and its importance overlooked. Axes may emphasize, diminish, distort, simplify, or clutter the information. They must be used carefully and accurately." (Mary H Briscoe, "Preparing Scientific Illustrations: A guide to better posters, presentations, and publications" 2nd ed., 1995)

"Graphic misrepresentation is a frequent misuse in presentations to the nonprofessional. The granddaddy of all graphical offenses is to omit the zero on the vertical axis. As a consequence, the chart is often interpreted as if its bottom axis were zero, even though it may be far removed. This can lead to attention-getting headlines about 'a soar' or 'a dramatic rise (or fall)'. A modest, and possibly insignificant, change is amplified into a disastrous or inspirational trend." (Herbert F Spirer et al, "Misused Statistics" 2nd Ed, 1998)

"If you want to show the growth of numbers which tend to grow by percentages, plot them on a logarithmic vertical scale. When plotted against a logarithmic vertical axis, equal percentage changes take up equal distances on the vertical axis. Thus, a constant annual percentage rate of change will plot as a straight line. The vertical scale on a logarithmic chart does not start at zero, as it shows the ratio of values (in this case, land values), and dividing by zero is impossible." (Herbert F Spirer et al, "Misused Statistics" 2nd Ed, 1998)

"The visual representation of a scale - an axis with ticks - looks like a ladder. Scales are the types of functions we use to map varsets to dimensions. At first glance, it would seem that constructing a scale is simply a matter of selecting a range for our numbers and intervals to mark ticks. There is more involved, however. Scales measure the contents of a frame. They determine how we perceive the size, shape, and location of graphics. Choosing a scale (even a default decimal interval scale) requires us to think about what we are measuring and the meaning of our measurements. Ultimately, that choice determines how we interpret a graphic." (Leland Wilkinson, "The Grammar of Graphics" 2nd Ed., 2005)

"One of the easiest ways to display data badly is to display as little information as possible. This includes not labelling axes and titles adequately, and not giving units. In addition, information that is displayed can be obscured by including unnecessary and distracting details." (Jenny Freeman et al, "How to Display Data", 2008)

"Histograms are often mistaken for bar charts but there are important differences. Histograms show distribution through the frequency of quantitative values (y axis) against defined intervals of quantitative values(x axis). By contrast, bar charts facilitate comparison of categorical values. One of the distinguishing features of a histogram is the lack of gaps between the bars [...]" (Andy Kirk, "Data Visualization: A successful design process", 2012)

28 April 2023

Data Science: Standard Deviation (Just the Quotes)

"The bounds on the standard deviation are pretty crude but it is surprising how often the rule will pick up gross errors such as confusing the standard error and standard deviation, confusing the variance and the standard deviation, or reporting the mean in one scale and the standard deviation in another scale." (Gerald van Belle, "Statistical Rules of Thumb", 2002)

"Data often arrive in raw form, as long lists of numbers. In this case your job is to summarize the data in a way that captures its essence and conveys its meaning. This can be done numerically, with measures such as the average and standard deviation, or graphically. At other times you find data already in summarized form; in this case you must understand what the summary is telling, and what it is not telling, and then interpret the information for your readers or viewers." (Charles Livingston & Paul Voakes, "Working with Numbers and Statistics: A handbook for journalists", 2005)

"Roughly stated, the standard deviation gives the average of the differences between the numbers on the list and the mean of that list. If data are very spread out, the standard deviation will be large. If the data are concentrated near the mean, the standard deviation will be small." (Charles Livingston & Paul Voakes, "Working with Numbers and Statistics: A handbook for journalists", 2005)

"A feature shared by both the range and the interquartile range is that they are each calculated on the basis of just two values - the range uses the maximum and the minimum values, while the IQR uses the two quartiles. The standard deviation, on the other hand, has the distinction of using, directly, every value in the set as part of its calculation. In terms of representativeness, this is a great strength. But the chief drawback of the standard deviation is that, conceptually, it is harder to grasp than other more intuitive measures of spread." (Alan Graham, "Developing Thinking in Statistics", 2006)

"Numerical precision should be consistent throughout and summary statistics such as means and standard deviations should not have more than one extra decimal place (or significant digit) compared to the raw data. Spurious precision should be avoided although when certain measures are to be used for further calculations or when presenting the results of analyses, greater precision may sometimes be appropriate." (Jenny Freeman et al, "How to Display Data", 2008)

"Need to consider outliers as they can affect statistics such as means, standard deviations, and correlations. They can either be explained, deleted, or accommodated (using either robust statistics or obtaining additional data to fill-in). Can be detected by methods such as box plots, scatterplots, histograms or frequency distributions." (Randall E Schumacker & Richard G Lomax, "A Beginner’s Guide to Structural Equation Modeling" 3rd Ed., 2010)

"Outliers or influential data points can be defined as data values that are extreme or atypical on either the independent (X variables) or dependent (Y variables) variables or both. Outliers can occur as a result of observation errors, data entry errors, instrument errors based on layout or instructions, or actual extreme values from self-report data. Because outliers affect the mean, the standard deviation, and correlation coefficient values, they must be explained, deleted, or accommodated by using robust statistics." (Randall E Schumacker & Richard G Lomax, "A Beginner’s Guide to Structural Equation Modeling" 3rd Ed., 2010)

"Outliers make it very hard to give an intuitive interpretation of the mean, but in fact, the situation is even worse than that. For a real‐world distribution, there always is a mean (strictly speaking, you can define distributions with no mean, but they’re not realistic), and when we take the average of our data points, we are trying to estimate that mean. But when there are massive outliers, just a single data point is likely to dominate the value of the mean and standard deviation, so much more data is required to even estimate the mean, let alone make sense of it." (Field Cady, "The Data Science Handbook", 2017)

"Theoretically, the normal distribution is most famous because many distributions converge to it, if you sample from them enough times and average the results. This applies to the binomial distribution, Poisson distribution and pretty much any other distribution you’re likely to encounter (technically, any one for which the mean and standard deviation are finite)." (Field Cady, "The Data Science Handbook", 2017)

"With time series though, there is absolutely no substitute for plotting. The pertinent pattern might end up being a sharp spike followed by a gentle taper down. Or, maybe there are weird plateaus. There could be noisy spikes that have to be filtered out. A good way to look at it is this: means and standard deviations are based on the naïve assumption that data follows pretty bell curves, but there is no corresponding 'default' assumption for time series data (at least, not one that works well with any frequency), so you always have to look at the data to get a sense of what’s normal. [...] Along the lines of figuring out what patterns to expect, when you are exploring time series data, it is immensely useful to be able to zoom in and out." (Field Cady, "The Data Science Handbook", 2017)

"With skewed data, quantiles will reflect the skew, while adding standard deviations assumes symmetry in the distribution and can be misleading." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

27 April 2023

Data Science: Fuzziness (Just the Quotes)

"Today we preach that science is not science unless it is quantitative. We substitute correlation for causal studies, and physical equations for organic reasoning. Measurements and equations are supposed to sharpen thinking, but [...] they more often tend to make the thinking non-causal and fuzzy." (John R Platt, "Strong Inference", Science Vol. 146 (3641), 1964)

"Information that is only partially structured (and therefore contains some 'noise' is fuzzy, inconsistent, and indistinct. Such imperfect information may be regarded as having merit only if it represents an intermediate step in structuring the information into a final meaningful form. If the partially Structured information remains in fuzzy form, it will create a state of dissatisfaction in the mind of the originator and certainly in the mind of the recipient. The natural desire is to continue structuring until clarity, simplicity, precision, and definitiveness are obtained." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"Mental models are fuzzy, incomplete, and imprecisely stated. Furthermore, within a single individual, mental models change with time, even during the flow of a single conversation. The human mind assembles a few relationships to fit the context of a discussion. As debate shifts, so do the mental models. Even when only a single topic is being discussed, each participant in a conversation employs a different mental model to interpret the subject. Fundamental assumptions differ but are never brought into the open. […] A mental model may be correct in structure and assumptions but, even so, the human mind - either individually or as a group consensus - is apt to draw the wrong implications for the future." (Jay W Forrester, "Counterintuitive Behaviour of Social Systems", Technology Review, 1971)

"In general, complexity and precision bear an inverse relation to one another in the sense that, as the complexity of a problem increases, the possibility of analysing it in precise terms diminishes. Thus 'fuzzy thinking' may not be deplorable, after all, if it makes possible the solution of problems which are much too complex for precise analysis." (Lotfi A Zadeh, "Fuzzy languages and their relation to human intelligence", 1972)

"Fuzziness, then, is a concomitant of complexity. This implies that as the complexity of a task, or of a system for performing that task, exceeds a certain threshold, the system must necessarily become fuzzy in nature. Thus, with the rapid increase in the complexity of the information processing tasks which the computers are called upon to perform, we are reaching a point where computers will have to be designed for processing of information in fuzzy form. In fact, it is the capability to manipulate fuzzy concepts that distinguishes human intelligence from the machine intelligence of current generation computers. Without such capability we cannot build machines that can summarize written text, translate well from one natural language to another, or perform many other tasks that humans can do with ease because of their ability to manipulate fuzzy concepts." (Lotfi A Zadeh, "The Birth and Evolution of Fuzzy Logic", 1989)

"Probability theory is an ideal tool for formalizing uncertainty in situations where class frequencies are known or where evidence is based on outcomes of a sufficiently long series of independent random experiments. Possibility theory, on the other hand, is ideal for formalizing incomplete information expressed in terms of fuzzy propositions." (George Klir, "Fuzzy sets and fuzzy logic", 1995)

"[…] interval mathematics and fuzzy logic together can provide a promising alternative to mathematical modeling for many physical systems that are too vague or too complicated to be described by simple and crisp mathematical formulas or equations. When interval mathematics and fuzzy logic are employed, the interval of confidence and the fuzzy membership functions are used as approximation measures, leading to the so-called fuzzy systems modeling." (Guanrong Chen & Trung Tat Pham, "Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems", 2001)

"Fuzzy relations are developed by allowing the relationship between elements of two or more sets to take on an infinite number of degrees of relationship between the extremes of 'completely related' and 'not related', which are the only degrees of relationship possible in crisp relations. In this sense, fuzzy relations are to crisp relations as fuzzy sets are to crisp sets; crisp sets and relations are more constrained realizations of fuzzy sets and relations." (Timothy J Ross & W Jerry Parkinson, "Fuzzy Set Theory, Fuzzy Logic, and Fuzzy Systems", 2002)

"The vast majority of information that we have on most processes tends to be nonnumeric and nonalgorithmic. Most of the information is fuzzy and linguistic in form." (Timothy J Ross & W Jerry Parkinson, "Fuzzy Set Theory, Fuzzy Logic, and Fuzzy Systems", 2002)

"Each fuzzy set is uniquely defined by a membership function. […] There are two approaches to determining a membership function. The first approach is to use the knowledge of human experts. Because fuzzy sets are often used to formulate human knowledge, membership functions represent a part of human knowledge. Usually, this approach can only give a rough formula of the membership function and fine-tuning is required. The second approach is to use data collected from various sensors to determine the membership function. Specifically, we first specify the structure of membership function and then fine-tune the parameters of membership function based on the data." (Huaguang Zhang & Derong Liu, "Fuzzy Modeling and Fuzzy Control", 2006)

"Granular computing is a general computation theory for using granules such as subsets, classes, objects, clusters, and elements of a universe to build an efficient computational model for complex applications with huge amounts of data, information, and knowledge. Granulation of an object a leads to a collection of granules, with a granule being a clump of points (objects) drawn together by indiscernibility, similarity, proximity, or functionality. In human reasoning and concept formulation, the granules and the values of their attributes are fuzzy rather than crisp. In this perspective, fuzzy information granulation may be viewed as a mode of generalization, which can be applied to any concept, method, or theory." (Salvatore Greco et al, "Granular Computing and Data Mining for Ordered Data: The Dominance-Based Rough Set Approach", 2009)

"We use the term fuzzy logic to refer to all aspects of representing and manipulating knowledge that employ intermediary truth-values. This general, commonsense meaning of the term fuzzy logic encompasses, in particular, fuzzy sets, fuzzy relations, and formal deductive systems that admit intermediary truth-values, as well as the various methods based on them." (Radim Belohlavek & George J Klir, "Concepts and Fuzzy Logic", 2011)