"A good rule of thumb for deciding how long the analysis of the data actually will take is (1) to add up all the time for everything you can think of - editing the data, checking for errors, calculating various statistics, thinking about the results, going back to the data to try out a new idea, and (2) then multiply the estimate obtained in this first step by five.
"Almost all efforts at data analysis seek, at some point, to
generalize the results and extend the reach of the conclusions beyond a
particular set of data. The inferential leap may be from past experiences to future
ones, from a sample of a population to the whole population, or from a narrow
range of a variable to a wider range. The real difficulty is in deciding when
the extrapolation beyond the range of the variables is warranted and when it is
merely naive. As usual, it is largely a matter of substantive judgment - or, as
it is sometimes more delicately put, a matter of 'a priori nonstatistical
considerations'."
"[…] fitting lines to relationships between variables is often a useful and powerful method of summarizing a set of data. Regression analysis fits naturally with the development of causal explanations, simply because the research worker must, at a minimum, know what he or she is seeking to explain." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)
"Fitting lines to relationships between variables is the major tool of data analysis. Fitted lines often effectively summarize the data and, by doing so, help communicate the analytic results to others. Estimating a fitted line is also the first step in squeezing further information from the data.
"If two or more describing variables in an analysis are
highly intercorrelated, it will be difficult and perhaps impossible to assess accurately
their independent impacts on the response variable. As the association between
two or more describing variables grows stronger, it becomes more and more
difficult to tell one variable from the other. This problem, called 'multicollinearity' in the statistical jargon, sometimes causes
difficulties in the analysis of nonexperimental data. […] No statistical
technique can go very far to remedy the problem because the fault lies
basically with the data rather than the method of analysis. Multicollinearity
weakens inferences based on any statistical method - regression, path analysis,
causal modeling, or cross-tabulations (where the difficulty shows up as a lack
of deviant cases and as near-empty cells)."
"[…] it is not enough to say: 'There's error in the data and
therefore the study must be terribly dubious'. A good critic and data analyst
must do more: he or she must also show how the error in the measurement or the
analysis affects the inferences made on the basis of that data and analysis."
"Logging size transforms the original skewed distribution
into a more symmetrical one by pulling in the long right tail of the
distribution toward the mean. The short left tail is, in addition, stretched.
The shift toward symmetrical distribution produced by the log transform is not,
of course, merely for convenience. Symmetrical distributions, especially those
that resemble the normal distribution, fulfill statistical assumptions that
form the basis of statistical significance testing in the regression model."
"Logging skewed variables also helps to reveal the patterns
in the data. […] the rescaling of the variables by taking logarithms reduces
the nonlinearity in the relationship and removes much of the clutter resulting
from the skewed distributions on both variables; in short, the transformation
helps clarify the relationship between the two variables. It also […] leads to
a theoretically meaningful regression coefficient."
"Our inability to measure important factors does not mean
either that we should sweep those factors under the rug or that we should give
them all the weight in a decision. Some important factors in some problems can
be assessed quantitatively. And even though thoughtful and imaginative efforts
have sometimes turned the 'unmeasurable' into a useful number, some
important factors are simply not measurable. As always, every bit of the
investigator's ingenuity and good judgment must be brought into play. And,
whatever un- knowns may remain, the analysis of quantitative data nonetheless can
help us learn something about the world - even if it is not the whole story."
"Quantitative techniques will be more likely to illuminate if the data analyst is guided in methodological choices by a substantive understanding of the problem he or she is trying to learn about. Good procedures in data analysis involve techniques that help to (a) answer the substantive questions at hand, (b) squeeze all the relevant information out of the data, and (c) learn something new about the world." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)
"Random data contain no substantive effects; thus if the
analysis of the random data results in some sort of effect, then we know that
the analysis is producing that spurious effect, and we must be on the lookout
for such artifacts when the genuine data are analyzed."
"Sometimes clusters of variables tend to vary together in the
normal course of events, thereby rendering it difficult to discover the magnitude
of the independent effects of the different variables in the cluster. And yet
it may be most desirable, from a practical as well as scientific point of view,
to disentangle correlated describing variables in order to discover more
effective policies to improve conditions. Many economic indicators tend to move
together in response to underlying economic and political events."
"The problem of multicollinearity involves a lack of data, a
lack of information. […] Recognition of multicollinearity as a lack of
information has two important consequences: (1) In order to alleviate the problem,
it is necessary to collect more data - especially on the rarer combinations of
the describing variables. (2) No statistical technique can go very far to
remedy the problem because the fault lies basically with the data rather than
the method of analysis. Multicollinearity weakens inferences based on any
statistical method - regression, path analysis, causal modeling, or
cross-tabulations (where the difficulty shows up as a lack of deviant cases and
as near-empty cells)."
"Statistical techniques do not solve any of the common-sense difficulties about making causal inferences. Such techniques may help organize or arrange the data so that the numbers speak more clearly to the question of causality - but that is all statistical techniques can do. All the logical, theoretical, and empirical difficulties attendant to establishing a causal relationship persist no matter what type of statistical analysis is applied.
"The language of association and prediction is probably most often used because the evidence seems insufficient to justify a direct causal statement. A better practice is to state the causal hypothesis and then to present the evidence along with an assessment with respect to the causal hypothesis - instead of letting the quality of the data determine the language of the explanation.
"The logarithmic transformation serves several purposes: (1) The resulting regression coefficients sometimes have a more useful theoretical interpretation compared to a regression based on unlogged variables. (2) Badly skewed distributions - in which many of the observations are clustered together combined with a few outlying values on the scale of measurement - are transformed by taking the logarithm of the measurements so that the clustered values are spread out and the large values pulled in more toward the middle of the distribution. (3) Some of the assumptions underlying the regression model and the associated significance tests are better met when the logarithm of the measured variables is taken."
"The matching procedure often helps inform the reader what is going on in the data […] Matching has some defects, chiefly that it is difficult to do a very good job of matching in complex situations without a large number of cases. […] One limitation of matching, then, is that quite often the match is not very accurate. A second limitation is that if we want to control for more than one variable using matching procedures, the tables begin to have combinations of categories without any cases at all in them, and they become somewhat more difficult for the reader to understand.
"The use of statistical methods to analyze data does not make a study any more 'scientific', 'rigorous', or 'objective'. The purpose of quantitative analysis is not to sanctify a set of findings. Unfortunately, some studies, in the words of one critic, 'use statistics as a drunk uses a street lamp, for support rather than illumination'. Quantitative techniques will be more likely to illuminate if the data analyst is guided in methodological choices by a substantive understanding of the problem he or she is trying to learn about. Good procedures in data analysis involve techniques that help to (a) answer the substantive questions at hand, (b) squeeze all the relevant information out of the data, and (c) learn something new about the world." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)
"Typically, data analysis is messy, and little details clutter it. Not only confounding factors, but also deviant cases, minor problems in measurement, and ambiguous results lead to frustration and discouragement, so that more data are collected than analyzed. Neglecting or hiding the messy details of the data reduces the researcher's chances of discovering something new." (Edward R Tufte, "Data Analysis for Politics and Policy", 1974)
"An especially effective device for enhancing the explanatory
power of time-series displays is to add spatial dimensions to the design of the
graphic, so that the data are moving over space (in two or three dimensions) as
well as over time. […] Occasionally graphics are belligerently multivariate,
advertising the technique rather than the data."
"Clear, detailed, and thorough labeling should be used to defeat graphical distortion and ambiguity. Write out explanations of the data on the graphic itself. Label important events in the data." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"Each part of a graphic generates visual expectations about its other parts and, in the economy of graphical perception, these expectations often determine what the eye sees. Deception results from the incorrect extrapolation of visual expectations generated at one place on the graphic to other places." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"For many people the first word that comes to mind when they think about statistical charts is 'lie'. No doubt some graphics do distort the underlying data, making it hard for the viewer to learn the truth. But data graphics are no different from words in this regard, for any means of communication can be used to deceive. There is no reason to believe that graphics are especially vulnerable to exploitation by liars; in fact, most of us have pretty good graphical lie detectors that help us see right through frauds." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"Graphical competence demands three quite different skills:
the substantive, statistical, and artistic. Yet now most graphical work, particularly
at news publications, is under the direction of but a single expertise - the
artistic. Allowing artist-illustrators to control the design and content of
statistical graphics is almost like allowing typographers to control the
content, style, and editing of prose. Substantive and quantitative expertise
must also participate in the design of data graphics, at least if statistical
integrity and graphical sophistication are to be achieved."
" In time-series displays of money, deflated and standardized units of monetary measurement are nearly always better than nominal units."
"Nearly all those who produce graphics for mass publication
are trained exclusively in the fine arts and have had little experience with
the analysis of data. Such experiences are essential for achieving precision
and grace in the presence of statistics. [...] Those who get ahead are those who
beautified data, never mind statistical integrity."
"Relational graphics are essential to competent statistical analysis since they confront statements about cause and effect with evidence, showing how one variable affects another." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"The conditions under which many data graphics are produced -
the lack of substantive and quantitative skills of the illustrators, dislike of
quantitative evidence, and contempt for the intelligence of the
audience-guarantee graphic mediocrity. These conditions engender graphics that
(1) lie; (2) employ only the simplest designs, often unstandardized time-series
based on a small handful of data points; and (3) miss the real news actually in
the data."
"The interior decoration of graphics generates a lot of ink
that does not tell the viewer anything new. The purpose of decoration varies - to
make the graphic appear more scientific and precise, to enliven the display, to
give the designer an opportunity to exercise artistic skills. Regardless of its
cause, it is all non-data-ink or redundant data-ink, and it is often chartjunk."
"The number of information-carrying (variable) dimensions depicted should not exceed the number of dimensions in the data." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"[…] the only worse design than a pie chart is several of them, for then the viewer is asked to compare quantities located in spatial disarray both within and between pies. […] Given their low data-density and failure to order numbers along a visual dimension, pie charts should never be used." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"The problem with time-series is that the simple passage of
time is not a good explanatory variable: descriptive chronology is not causal
explanation. There are occasional exceptions, especially when there is a clear
mechanism that drives the Y-variable."
"The representation of numbers, as physically measured on the
surface of the graphic itself, should be directly proportional to the numerical
quantities represented."
"The time-series plot is the most frequently used form of
graphic design. With one dimension marching along to the regular rhythm of
seconds, minutes, hours, days, weeks, months, years, centuries, or millennia,
the natural ordering of the time scale gives this design a strength and
efficiency of interpretation found in no other graphic arrangement."
"Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all his sentences short, or that heavoid all detail and treat his subjects only in outline, but that every word tell." (Edward R Tufte, "The Visual Display of Quantitative Information", 1983)
"At the heart of quantitative reasoning is a single question: Compared to what? Small multiple designs, multivariate and data bountiful, answer directly by visually enforcing comparisons of changes, of the differences among objects, of the scope of alternatives. For a wide range of problems in data presentation, small multiples are the best design solution." (Edward R Tufte, "Envisioning Information", 1990)
"Confusion and clutter are failures of design, not attributes of information.
And so the point is to find design strategies that reveal detail and complexity - rather than to fault the data for an excess of complication. Or, worse, to
fault viewers for a lack of understanding. Among the most powerful devices for
reducing noise and enriching the content of displays is the technique of
layering and separation, visually stratifying various aspects of the data.
"Consider this unsavory exhibit at right – chockablock with
cliché and stereotype, coarse humor, and a content-empty third dimension. [...] Credibility vanishes in clouds of chartjunk; who would trust a chart that looks
like a video game?"
"Gray grids almost always work well and, with a delicate line, may promote more accurate data reading and reconstruction than a heavy grid. Dark grid lines are chartjunk. When a graphic serves as a look-up table (rare indeed), then a grid may help with reading and interpolation. But even then the grid should be muted relative to the data." (Edward R Tufte, "Envisioning Information", 1990)
"Information consists of differences that make a difference." (Edward R Tufte, "Envisioning Information", 1990)
"Lurking behind chartjunk is contempt both for information and for the audience. Chartjunk promoters imagine that numbers and details are boring, dull, and tedious, requiring ornament to enliven. Cosmetic decoration, which frequently distorts the data, will never salvage an underlying lack of content. If the numbers are boring, then you've got the wrong numbers.
"The ducks of information design are false escapes from flatland, adding pretend dimensions to impoverished data sets, merely fooling around with information.
"Visual displays rich with data are not only an appropriate and proper complement to human capabilities, but also such designs are frequently optimal. If the visual task is contrast, comparison, and choice - as so often it is - then the more relevant information within eyespan, the better. Vacant, low-density displays, the dreaded posterization of data spread over pages and pages, require viewers to rely on visual memory - a weak skill - to make a contrast, a comparison, a choice." (Edward R Tufte, "Envisioning Information", 1990)
No comments:
Post a Comment