Statistics Primer

@COL1:INTRODUCTION

THE DANGER OF SUCCESS IN MATH

DESCRIPTIVE STATISTICS

@COL2:Data Reduction

Measures of Association

Regression Analysis

@COL1:OTHER MULTIVARIATE TECHNIQUES

@COL2:Path Analysis

Time-Series Analysis

Factor Analysis

@COL1:INFERENTIAL STATISTICS

@COL2:Univariate Inferences

Tests of Statistical Significance

The Logic of Statistical Significance

Chi Square

@COL1:MAIN POINTS

KEY TERMS

REVIEW QUESTIONS AND EXERCISES

ADDITIONAL READINGS

@H1:INTRODUCTION

@GT-no-indent:It has been my experience over
the years that many students are intimidated by statistics. Sometimes statistics
makes them feel they’re

@BL:o A few clowns short of a circus

o Dumber than a box of hair

o A few feathers short of a duck

o All foam, no beer

o Missing a few buttons on their remote control

o A few beans short of a burrito

o As screwed up as a football bat

o About as sharp as a bowling ball

o About four cents short of a nickel

o Not running on full thrusters*

@FN:*Thanks to the many contributors to humor
lists on the Internet.

@GT:Many people are intimidated by quantitative
research because they feel uncomfortable with mathematics and statistics.
And indeed, many research reports are filled with unspecified computations.
The role of statistics in social research is often important, but it is
equally important to see this role in its proper perspective.

Empirical research is first and foremost a logical
rather than a mathematical operation. Mathematics is merely a convenient
and efficient language for accomplishing the logical operations inherent
in quantitative data analysis. Statistics is the applied branch of mathematics
especially appropriate to a variety of research analyses.

This discussion begins with an informal look
at one of the concerns many people have when they approach statistics.
I hope this exercise will make it easier to understand and feel comfortable
with the relatively simple statistics introduced in the remainder of the
discussion. We’ll be looking at two types of statistics: descriptive and
inferential. Descriptive statistics is a medium for describing data in
manageable forms. Inferential statistics, on the other hand, assists researchers
in drawing conclusions from their observations; typically, this involves
drawing conclusions about a population from the study of a sample drawn
from it.

@H1:THE DANGER OF SUCCESS IN MATH

@GT-no indent:Since I began teaching research
methods that include at least a small amount of statistics, I’ve been struck
by the large number of students who report that they are "simply no good
at math." Just as some people are reported to be inherently tone-deaf and
others unable to learn foreign languages, about 90 percent of my college
students have seemed to suffer from congenital math deficiency syndrome
(CMDS). Its most common symptoms are frustration, boredom, and drowsiness.
I’m delighted to report that I have finally uncovered a major cause of
the disease and have brewed up a cure. In the event that you may be a sufferer,
I’d like to share it with you before we delve into the statistics of social
research.

@GT:You may be familiar with the story of Typhoid
Mary, whose real name was Mary Mallon. Mary was a typhoid carrier who died
in 1938 in New York. Before her death, she worked as a household cook,
moving from household to household and causing ten outbreaks of typhoid
fever. Over 50 people caught the disease from her, and 3 of them died.

The congenital math deficiency syndrome has a
similar cause. After an exhaustive search, I’ve discovered the culprit,
whom I’ll call Mathematical Marvin, though he has used countless aliases.
If you suffer from CMDS, I suspect you’ve met him. Take a minute to recall
your years in high school. Remember the person your teachers and your classmates
regarded as a "mathematical genius." Getting A’s in all the math classes
was only part of it; often the math genius seemed to know math better than
the teachers did.

Now that you have that math genius in mind, let
me ask you a few questions. First, what was the person’s gender? I’d guess
he was probably male. Most of the students I’ve asked in class report that.
But let’s consider some other characteristics:

@NL1:1. How athletic was he?

2. Did he wear glasses?

3. How many parties did he get invited to during
high school?

4. If he was invited to parties, did anyone ever
talk to him?

5. How often did you find yourself envying the
math genius, wishing you could trade places with him?

@GT:I’ve been asking students (including some
in adult classes) these questions for several years, and the answers I’ve
gotten are amazing. Marvin is usually unathletic, often either very skinny
or overweight. He usually wears glasses, and he seems otherwise rather
delicate. During his high school years, he was invited to an average (mean)
of 1.2 parties, and nobody talked to him. His complexion was terrible.
Almost nobody ever wanted to change places with him; he was a social misfit,
to be pitied rather than envied.

As I’ve discussed Marvin with my students, it
has become increasingly clear that most of them have formed a subconscious
association between mathematical proficiency and Marvin’s unenviable characteristics.
Most have concluded that doing well in math and statistics would turn them
into social misfits, which they regard as too high a price to pay.

Everything I’ve said about Mathematical Marvin
represents a powerful stereotype that many people still seem to share,
but the fact is that it’s only a social stereotype, not a matter of biology.
Women can excel in mathematics; attractive people can calculate as accurately
as unattractive ones. Yet the stereotype exercises a powerful influence
on our behavior, as evidenced in the tragic examples of young women, for
example, pretending mathematical impotence in the belief that they will
be seen as less attractive if they’re gifted in that realm.

So if you’re one of those people who’s "just
no good at math," it’s possible you carry around a hidden fear that your
face will break out in pimples if you do well in statistics in this course.
If so, you’re going to be reading the rest of this discussion in a terrible
state: wanting to understand it at least until the next exam and, at the
same time, worrying that you may understand it too well and lose all your
friends.

There is no cause for concern. The level of statistics
contained in the rest of this discussion has been proved safe for humans.
There has not been a single documented case of pimples connected to understanding
lambda, gamma, chi square, or any of the other statistics discussed in
the pages that follow. In fact, this level of exposure has been found to
be beneficial to young social researchers.

By the way, uncovering Marvin can clear up a
lot of mysteries. It did for me. (In my high school class, he didn’t wear
glasses, but he squinted a lot.) In the first research methods book I wrote,
I presented three statistical computations and got one of them wrong. In
the first edition of this book, I got a different one wrong. Most embarrassing
of all, however, the first printing of the earlier book had a unique feature.
I thought it would be fun to write a computer program to generate my own
table of random numbers rather than reprinting one that someone else had
created. In doing that, I had the dubious honor of publishing the world’s
first table of random numbers that didn’t have any nines! It was not until
I tracked Marvin down that I discovered the source of my problems, and
statistics has been much more fun (and trouble-free) ever since. So enjoy.

@H1:DESCRIPTIVE STATISTICS

@GT-no indent:As I’ve already suggested, descriptive
statistics present quantitative descriptions in a manageable form. Sometimes
we want to describe single variables, and sometimes we want to describe
the associations that connect one variable with another. Let’s look at
some of the ways to do these things.

@H2:Data Reduction

@GT-no indent:Scientific research often involves
collecting large masses of data. Suppose we surveyed 2,000 people, asking
each of them 100 questions—not an unusually large study. We would then
have a staggering 200,000 answers! No one could possibly read all those
answers and reach any meaningful conclusion about them. Thus, much scientific
analysis involves the reduction of data from unmanageable details to manageable
summaries.

@GT:To begin our discussion, let’s look briefly
at the raw data matrix created by a quantitative research project. Table
17-1 presents a partial data matrix. Notice that each row in the matrix
represents a person (or other unit of analysis), each column represents
a variable, and each cell represents the coded attribute or value a given
person has on a given variable. The first column in Table 17-1 represents
a person’s gender. Let’s say a "1" represents male and a "2" represents
female. This means that persons 1 and 2 are male, person 3 is female, and
so forth.

**[Table 17-1 about here; pickup from 8e p. 407]**

In the case of age, person 1’s "3" might mean
30?39 years old, person 2’s "4" might mean 40-49. However age has been
coded, the code numbers shown in Table 17-1 describe each of the people
represented there.

Notice that the data have already been reduced
somewhat by the time a data matrix like this one has been created. If age
has been coded as suggested previously, the specific answer "33 years old"
has already been assigned to the category "30-39." The people responding
to our survey may have given us 60 or 70 different ages, but we have now
reduced them to 6 or 7 categories.

@H2:Measures of Association

@GT-no indent:The association between any two
variables can also be represented by a data matrix, this time produced
by the joint frequency distributions of the two variables. Table 17-2 presents
such a matrix. It provides all the information needed to determine the
nature and extent of the relationship between education and prejudice.

**[Table 17-2 about here; pickup from 8e p. 407]**

@GT:Notice, for example, that 23 people (1) have
no education and (2) scored high on prejudice; 77 people (1) had graduate
degrees and (2) scored low on prejudice.

Like the raw-data matrix in Table 17-1, this
matrix provides more information than can easily be comprehended. A careful
study of the table shows that as education increases from "None" to "Graduate
Degree," there is a general tendency for prejudice to decrease, but no
more than a general impression is possible. For a more precise summary
of the data matrix, we need one of several types of descriptive statistics.
Selecting the appropriate measure depends initially on the nature of the
two variables.

We’ll turn now to some of the options available
for summarizing the association between two variables. Each of these measures
of association is based on the same model—proportionate reduction of error
(PRE).

To see how this model works, let’s assume that
I asked you to guess respondents’ attributes on a given variable: for example,
whether they answered yes or no to a given questionnaire item. To assist
you, let’s first assume you know the overall distribution of responses
in the total sample—say, 60 percent said yes and 40 percent said no. You
would make the fewest errors in this process if you always guessed the
modal (most frequent) response: yes.

Second, let’s assume you also know the empirical
relationship between the first variable and some other variable: say, gender.
Now, each time I ask you to guess whether a respondent said yes or no,
I’ll tell you whether the respondent is a man or a woman. If the two variables
are related, you should make fewer errors the second time. It’s possible,
therefore, to compute the PRE by knowing the relationship between the two
variables: the greater the relationship, the greater the reduction of error.

This basic PRE model is modified slightly to
take account of different levels of measurement—nominal, ordinal, or interval.
The following sections will consider each level of measurement and present
one measure of association appropriate to each. Bear in mind, though, that
the three measures discussed are only an arbitrary selection from among
many appropriate measures.

@H3:Nominal Variables @GT:If the two variables
consist of nominal data (for example, gender, religious affiliation, race),
lambda (l) would be one appropriate measure. (Lambda is a letter in the
Greek alphabet corresponding to l in our alphabet. Greek letters are used
for many concepts in statistics, which perhaps helps to account for the
number of people who say of statistics, "It’s all Greek to me.") Lambda
is based on your ability to guess values on one of the variables: the PRE
achieved through knowledge of values on the other variable.

Imagine this situation. I tell you that a room
contains 100 people and I would like you to guess the gender of each person,
one at a time. If half are men and half women, you will probably be right
half the time and wrong half the time. But suppose I tell you each person’s
occupation before you guess that person’s gender.

What gender would you guess if I said the person
was a truck driver? Probably you would be wise to guess "male"; although
there are now plenty of women truck drivers, most are still men. If I said
the next person was a nurse, you’d probably be wisest to guess "female,"
following the same logic. While you would still make errors in guessing
genders, you would clearly do better than you would if you didn’t know
their occupations. The extent to which you did better (the proportionate
reduction of error) would be an indicator of the association that exists
between gender and occupation.

Here’s another simple hypothetical example that
illustrates the logic and method of lambda. Table 17-3 presents hypothetical
data relating gender to employment status. Overall, we note that 1,100
people are employed, and 900 are not employed. If you were to predict whether
people were employed, knowing only the overall distribution on that variable,
you would always predict "employed," since that would result in fewer errors
than always predicting "not employed." Nevertheless, this strategy would
result in 900 errors out of 2,000 predictions.

**[Table 17-3 about here; pickup from 4e p. 408]**

Let’s suppose that you had access to the data
in Table 17-3 and that you were told each person’s gender before making
your prediction of employment status. Your strategy would change in that
case. For every man, you would predict "employed," and for every woman,
you would predict "not employed." In this instance, you would make 300
errors—the 100 men who were not employed and the 200 employed women—or
600 fewer errors than you would make without knowing the person’s gender.

Lambda, then, represents the reduction in errors
as a proportion of the errors that would have been made on the basis of
the overall distribution. In this hypothetical example, lambda would equal
.67; that is, 600 fewer errors divided by the 900 total errors based on
employment status alone. In this fashion, lambda measures the statistical
association between gender and employment status.

If gender and employment status were statistically
independent, we would find the same distribution of employment status for
men and women. In this case, knowing each person’s gender would not affect
the number of errors made in predicting employment status, and the resulting
lambda would be zero. If, on the other hand, all men were employed and
none of the women were employed, by knowing gender you would avoid errors
in predicting employment status. You would make 900 fewer errors (out of
900), so lambda would be 1.0—representing a perfect statistical association.

Lambda is only one of several measures of association
appropriate to the analysis of two nominal variables. You could look at
any statistics textbook for a discussion of other appropriate measures.

@H3:Ordinal Variables @GT:If the variables being
related are ordinal (for example, social class, religiosity, alienation),
gamma (g) is one appropriate measure of association. Like lambda, gamma
is based on our ability to guess values on one variable by knowing values
on another. However, whereas lambda is based on guessing exact values,
gamma is based on guessing the ordinal arrangement of values. For any given
pair of cases, we guess that their ordinal ranking on one variable will
correspond (positively or negatively) to their ordinal ranking on the other.

Let’s say we have a group of elementary students.
It’s reasonable to assume that there is a relationship between their ages
and their heights. We can test this by comparing every pair of students:
Sam and Mary, Sam and Fred, Mary and Fred, and so forth. Then we ignore
all the pairs in which the students are the same age and/or the same height.
We then classify each of the remaining pairs (those who differ in both
age and height) into one of two categories: those in which the older child
is also the taller ("same" pairs) and those in which the older child is
the shorter ("opposite" pairs). So, if Sam is older and taller than Mary,
the Sam-Mary pair is counted as a "same." If Sam is older but shorter than
Mary, then that pair is an "opposite." (If they’re the same age and/or
same height, we ignore them.)

To determine whether age and height are related
to one another, we compare the number of same and opposite pairs. If the
same pairs outnumber the opposite pairs, we can conclude that there is
a positive association between the two variables—as one increases, the
other increases. If there are more opposites than sames, we can conclude
that the relationship is negative. If there are about as many sames as
opposites, we can conclude that age and height are not related to each
another, that they’re independent of each other.

Here’s a social science example to illustrate
the simple calculations involved in gamma. Let’s say you suspect that religiosity
is positively related to political conservatism, and if Person A is more
religious than Person B, you guess that A is also more conservative than
B. Gamma is the proportion of paired comparisons that fits this pattern.

Table 17-4 presents hypothetical data relating
social class to prejudice. The general nature of the relationship between
these two variables is that as social class increases, prejudice decreases.
There is a negative association between social class and prejudice.

**[Table 17-4 about here; pickup from8e p. 409]**

Gamma is computed from two quantities: (1) the
number of pairs having the same ranking on the two variables and (2) the
number of pairs having the opposite ranking on the two variables. The pairs
having the same ranking are computed as follows. The frequency of each
cell in the table is multiplied by the sum of all cells appearing below
and to the right of it—with all these products being summed. In Table 17-4,
the number of pairs with the same ranking would be 200(900 + 300 + 400
+ 100) + 500(300 + 100) + 400(400 + 100) + 900(100), or 340,000 + 200,000
+ 200,000 + 90,000 = 830,000.

The pairs having the opposite ranking on the
two variables are computed as follows: The frequency of each cell in the
table is multiplied by the sum of all cells appearing below and to the
left of it—with all these products being summed. In Table 17-4, the numbers
of pairs with opposite rankings would be 700(500 + 800 + 900 + 300) + 400(800
+ 300) + 400(500 + 800) + 900(800), or 1,750,000 + 440,000 + 520,000 +
720,000 = 3,430,000. Gamma is computed from the numbers of same-ranked
pairs and opposite-ranked pairs as follows: **[Set as in 8e p. 410]**

same ? opposite

gamma = ---------------------

same + opposite

In our example, gamma equals (830,000 ? 3,430,000)
divided by (830,000 + 3,430,000), or ?.61. The negative sign in this answer
indicates the negative association suggested by the initial inspection
of the table. Social class and prejudice, in this hypothetical example,
are negatively associated with each other. The numerical figure for gamma
indicates that 61 percent more of the pairs examined had the opposite ranking
than the same ranking.

Note that whereas values of lambda vary from
0 to 1, values of gamma vary from ?1 to +1, representing the direction
as well as the magnitude of the association. Because nominal variables
have no ordinal structure, it makes no sense to speak of the direction
of the relationship. (A negative lambda would indicate that you made more
errors in predicting values on one variable while knowing values on the
second than you made in ignorance of the second, and that’s not logically
possible.)

Table 17-5 is an example of the use of gamma
in social research. To study the extent to which widows sanctified their
deceased husbands, Helena Znaniecki Lopata (1981) administered a questionnaire
to a probability sample of 301 widows. In part, the questionnaire asked
the respondents to characterize their deceased husbands in terms of the
following semantic differentiation scale: **[set as in 8e p. 410]**

@T-1:Characteristic

@T-2:Positive Extreme Negative Extreme

@TB:Good 1 2 3 4 5 6 7 Bad

Useful 1 2 3 4 5 6 7 Useless

Honest 1 2 3 4 5 6 7 Dishonest

Superior 1 2 3 4 5 6 7 Inferior

Kind 1 2 3 4 5 6 7 Cruel

Friendly 1 2 3 4 5 6 7 Unfriendly

Warm 1 2 3 4 5 6 7 Cold

**[Table 17-5 about here; pickup from 8e p. 410]**

@GT:Respondents were asked to describe their
deceased spouses by circling a number for each pair of opposing characteristics.
Notice that the series of numbers connecting each pair of characteristics
is an ordinal measure.

Next, Lopata wanted to discover the extent to
which the several measures were related to each other. Appropriately, she
chose gamma as the measure of association. Table 17-5 shows how she presented
the results of her investigation.

The format presented in Table 17-5 is called
a correlation matrix. For each pair of measures, Lopata has calculated
the gamma. Good and Useful, for example, are related to each other by a
gamma equal to .79. The matrix is a convenient way of presenting the intercorrelations
among several variables, and you’ll find it frequently in the research
literature. In this case, we see that all the variables are quite strongly
related to each other, though some pairs are more strongly related than
others.

Gamma is only one of several measures of association
appropriate to ordinal variables. Again, any introductory statistics textbook
will give you a more comprehensive treatment of this subject.

@H3:Interval or Ratio Variables @GT:If interval
or ratio variables (for example, age, income, grade point average, and
so forth) are being associated, one appropriate measure of association
is Pearson’s product-moment correlation (r). The derivation and computation
of this measure of association are complex enough to lie outside the scope
of this book, so I’ll make only a few general comments here.

Like both gamma and lambda, r is based on guessing
the value of one variable by knowing the other. For continuous interval
or ratio variables, however, it is unlikely that you could predict the
precise value of the variable. But on the other hand, predicting only the
ordinal arrangement of values on the two variables would not take advantage
of the greater amount of information conveyed by an interval or ratio variable.
In a sense, r reflects how closely you can guess the value of one variable
through your knowledge of the value of the other.

To understand the logic of r, consider the way
you might hypothetically guess values that particular cases have on a given
variable. With nominal variables, we’ve seen that you might always guess
the modal value. But for interval or ratio data, you would minimize your
errors by always guessing the mean value of the variable. Although this
practice produces few if any perfect guesses, the extent of your errors
will be minimized. Imagine the task of guessing peoples’ incomes and how
much better you would do if you knew how many years of education they had
as well as the mean incomes for people with 0, 1, 2 (and so forth) years
of education.

In the computation of lambda, we noted the number
of errors produced by always guessing the modal value. In the case of r,
errors are measured in terms of the sum of the squared differences between
the actual value and the mean. This sum is called the total variation.

To understand this concept, we must expand the
scope of our examination. Let’s look at the logic of regression analysis
and discuss to correlation within that context.

@H2:Regression Analysis

@GT-no indent:At several points in this text,
I have referred to the general formula for describing the association between
two variables: Y = f(X). This formula is read "Y is a function of X," meaning
that values of Y can be explained in terms of variations in the values
of X. Stated more strongly, we might say that X causes Y, so the value
of X determines the value of Y. Regression analysis is a method of determining
the specific function relating Y to X. There are several forms of regression
analysis, depending on the complexity of the relationships being studied.
Let’s begin with the simplest.

@H3:Linear Regression @GT:The regression model
can be seen most clearly in the case of a linear regression analysis, where
there is a perfect linear association between two variables. Figure 17-1
is a scattergram presenting in graphic form the values of X and Y as produced
by a hypothetical study. It shows that for the four cases in our study,
the values of X and Y are identical in each instance. The case with a value
of 1 on X also has a value of 1 on Y, and so forth. The relationship between
the two variables in this instance is described by the equation Y = X;
this is called the regression equation. Because all four points lie on
a straight line, we could superimpose that line over the points; this is
the regression line.

**[Figure 17-1 about here; pickup from 8e p.
411]**

@FN:Figure 17-1 @FT:Simple Scattergram of Values
of X and Y

@GT:The linear regression model has important
descriptive uses. The regression line offers a graphic picture of the association
between X and Y, and the regression equation is an efficient form for summarizing
that association. The regression model has inferential value as well. To
the extent that the regression equation correctly describes the general
association between the two variables, it may be used to predict other
sets of values. If, for example, we know that a new case has a value of
3.5 on X, we can predict the value of 3.5 on Y as well.

In practice, of course, studies are seldom limited
to four cases, and the associations between variables are seldom as clear
as the one presented in Figure 17-1.

A somewhat more realistic example is presented
in Figure 17-2, representing a hypothetical relationship between population
and crime rate in small- to medium-sized cities. Each dot in the scattergram
is a city, and its placement reflects that city’s population and its crime
rate. As was the case in our previous example, the values of Y (crime rates)
generally correspond to those of X (populations), and as values of X increase,
so do values of Y. However, the association is not nearly as clear as it
is in Figure 17-1.

**[Figure 17-2 about here; pickup from 8e p.
412]**

@FN:Figure 17-2 @FT:A Scattergram of the Values
of Two Variables with Regression Line Added (Hypothetical)

@GT:In Figure 17-2 we can’t superimpose a straight
line that will pass through all the points in the scattergram. But we can
draw an approximate line showing the best possible linear representation
of the several points. I’ve drawn that line on the graph.

You may (or may not) recall from algebra that
any straight line on a graph can be represented by an equation of the form
Y = a + bX, where X and Y are values of the two variables. In this equation,
a equals the value of Y when X is 0, and b represents the slope of the
line. If we know the values of a and b, we can calculate an estimate of
Y for every value of X.

We can now say more formally that regression
analysis is a technique for establishing the regression equation representing
the geometric line that comes closest to the distribution of points on
a graph. The regression equation provides a mathematical description of
the relationship between the variables, and it allows us to infer values
of Y when we have values of X. Recalling Figure 17-2, we could estimate
crime rates of cities if we knew their populations.

To improve your guessing, you construct a regression
line, stated in the form of a regression equation that permits the estimation
of values on one variable from values on the other. The general format
for this equation is Y ’ = a + b(X), where a and b are computed values,
X is a given value on one variable, and Y ’ is the estimated value on the
other. **[Y ’ is Y prime, as in 4e p. 413]**The values of a and b are computed
to minimize the differences between actual values of Y and the corresponding
estimates (Y ’) based on the known value of X. The sum of squared differences
between actual and estimated values of Y is called the unexplained variation
because it represents errors that still exist even when estimates are based
on known values of X.

The explained variation is the difference between
the total variation and the unexplained variation. Dividing the explained
variation by the total variation produces a measure of the proportionate
reduction of error corresponding to the similar quantity in the computation
of lambda. In the present case, this quantity is the correlation squared:
r2. Thus, if r = .7, then r2 = .49, meaning that about half the variation
has been explained. In practice, we compute r rather than r2, because the
product-moment correlation can take either a positive or a negative sign,
depending on the direction of the relationship between the two variables.
(Computing r2 and taking a square root would always produce a positive
quantity.) You can consult any standard statistics textbook for the method
of computing r, although I anticipate that most readers using this measure
will have access to computer programs designed for this function.

Unfortunately—or perhaps fortunately—social life
is so complex that the simple linear regression model often does not sufficiently
represent the state of affairs. It’s possible, using percentage tables,
to analyze more than two variables. As the number of variables increases,
such tables become increasingly complicated and hard to read. But the regression
model offers a useful alternative in such cases.

@H3:Multiple Regression @GT:Very often, social
researchers find that a given dependent variable is affected simultaneously
by several independent variables. Multiple regression analysis provides
a means of analyzing such situations. This was the case when Beverly Yerg
(1981) set about studying teacher effectiveness in physical education.
She stated her expectations in the form of a multiple regression equation:

**[Set as in 8e p. 413; see addition of comma
and "where" after first equation]**

@EX:F = b0 + b1I + b2X1 + b3X2 + b4X3 + b5X4
+ e, where

F = Final pupil-performance score

I = Initial pupil-performance score

X1 = Composite of guiding and supporting practice

X2 = Composite of teacher mastery of content

X3 = Composite of providing specific, task-related
feedback

X4 = Composite of clear, concise task presentation

b = Regression weight

e = Residual

@EXS:(ADAPTED FROM YERG 1981:42)

@GT:Notice that in place of the single X variable
in a linear regression, there are several X’s, and there are also several
b’s instead of just one. Also, Yerg has chosen to represent a as b0 in
this equation but with the same meaning as discussed previously. Finally,
the equation ends with a residual factor (e), which represents the variance
in Y that is not accounted for by the X variables analyzed.

Beginning with this equation, Yerg calculated
the values of the several b’s to show the relative contributions of the
several independent variables in determining final student-performance
scores. She also calculated the multiple-correlation coefficient as an
indicator of the extent to which all six variables predict the final scores.
This follows the same logic as the simple bivariate correlation discussed
earlier, and it is traditionally reported as a capital R. In this case,
R = .877, meaning that 77 percent of the variance (.8772= .77) in final
scores is explained by the six variables acting in concert.

@H3:Partial Regression @GT:In exploring the elaboration
model in your textbook, we paid special attention to the relationship between
two variables when a third test variable was held constant. Thus, we might
examine the effect of education on prejudice with age held constant, testing
the independent effect of education. To do so, we would compute the tabular
relationship between education and prejudice separately for each age group.

Partial regression analysis is based on this
same logical model. The equation summarizing the relationship between variables
is computed on the basis of the test variables remaining constant. As in
the case of the elaboration model, the result may then be compared with
the uncontrolled relationship between the two variables to clarify further
the overall relationship.

@H3:Curvilinear Regression @GT:Up to now, we
have been discussing the association among variables as represented by
a straight line. The regression model is even more general than our discussion
thus far has implied.

You may already know that curvilinear functions,
as well as linear ones, can be represented by equations. For example, the
equation X2 + Y2 = 25 describes a circle with a radius of 5. Raising variables
to powers greater than 1 has the effect of producing curves rather than
straight lines. And in the real world there is no reason to assume that
the relationship among every set of variables will be linear. In some cases,
then, curvilinear regression analysis can provide a better understanding
of empirical relationships than can any linear model.

Recall, however, that a regression line serves
two functions. It describes a set of empirical observations, and it provides
a general model for making inferences about the relationship between two
variables in the general population that the observations represent. A
very complex equation might produce an erratic line that would indeed pass
through every individual point. In this sense, it would perfectly describe
the empirical observations. There would be no guarantee, however, that
such a line could adequately predict new observations or that it in any
meaningful way represented the relationship between the two variables in
general. Thus, it would have little or no inferential value.

Earlier in this book, we discussed the need for
balancing detail and utility in data reduction. Ultimately, researchers
attempt to provide the most faithful, yet also the simplest, representation
of their data. This practice also applies to regression analysis. Data
should be presented in the simplest fashion (thus, linear regressions are
most frequently used) that best describes the actual data. Curvilinear
regression analysis adds a new option to the researcher in this regard,
but it does not solve the problems altogether. Nothing does that.

@H3:Cautions in Regression Analysis @GT:The use
of regression analysis for statistical inferences is based on the same
assumptions made for correlational analysis: simple random sampling, the
absence of nonsampling errors, and continuous interval data. Because social
scientific research seldom completely satisfies these assumptions, you
should use caution in assessing the results in regression analyses.

Also, regression lines—linear or curvilinear—can
be useful for interpolation (estimating cases lying between those observed),
but they are less trustworthy when used for extrapolation (estimating cases
that lie beyond the range of observations). This limitation on extrapolations
is important in two ways. First, you are likely to come across regression
equations that seem to make illogical predictions. An equation linking
population and crimes, for example, might seem to suggest that small towns
with, say, a population of 1,000 should produce 123 crimes a year. This
failure in predictive ability does not disqualify the equation but dramatizes
that its applicability is limited to a particular range of population sizes.
Second, researchers sometimes overstep this limitation, drawing inferences
that lie outside their range of observation, and you’d be right in criticizing
them for that.

The preceding sections have introduced some of
the techniques for measuring associations among variables at different
levels of measurement. Matters become slightly more complex when the two
variables represent different levels of measurement. Though we aren’t going
to pursue this issue in this textbook, the box "Measures of Association
and Levels of Measurement," by Peter Nardi, may be a useful resource if
you ever have to address such situations.

**[Box "Measures of Association and Levels of
Measurement" about here; pickup from 8e p. 415]**

@H1:OTHER MULTIVARIATE TECHNIQUES

@GT-no indent:For the most part, this book has
focused on rather rudimentary forms of data manipulation, such as the use
of contingency tables and percentages. The elaboration model of analysis
was presented in this form, as were the statistical techniques presented
so far in this discussion.

@GT:This section of the discussion presents a
cook’s tour of three other multivariate techniques from the logical perspective
of the elaboration model. This discussion isn’t intended to teach you how
to use these techniques but rather to present sufficient information so
that you can understand them if you run across them in a research report.
The three methods of analysis that we’ll examine—path analysis, time-series
analysis, and factor analysis—are only a few of the many multivariate techniques
used by social scientists.

@H2:Path Analysis

@GT-no indent:Path analysis is a causal model
for understanding relationships between variables. Though based on regression
analysis, it can provide a more useful graphic picture of relationships
among several variables than other means can. Path analysis assumes that
the values of one variable are caused by the values of another, so it is
essential to distinguish independent and dependent variables. This requirement
is not unique to path analysis, of course, but path analysis provides a
unique way of displaying explanatory results for interpretation.

Recall for a moment one of the ways I represented
the elaboration model in your textbook. Here’s how we might diagram the
logic of interpretation: **[Set the following as in 8e p. 416; lc "v"]**

Independent variable Æ Intervening variable
Æ Dependent variable

The logic of this presentation is that an independent
variable has an impact on an intervening variable, which, in turn, has
an impact on a dependent variable. The path analyst constructs similar
patterns of relationships among variables, but the typical path diagram
contains many more variables than shown in this diagram.

Besides diagramming a network of relationships
among variables, path analysis also shows the strengths of those several
relationships. The strengths of relationships are calculated from a regression
analysis that produces numbers analogous to the partial relationships in
the elaboration model. These path coefficients, as they are called, represent
the strengths of the relationships between pairs of variables, with the
effects of all other variables in the model held constant.

The analysis in Figure 17-3, for example, focuses
on the religious causes of anti-Semitism among Christian church members.
The variables in the diagram are, from left to right, (1) orthodoxy, or
the extent to which the subjects accept conventional beliefs about God,
Jesus, biblical miracles, and so forth; (2) particularism, the belief that
one’s religion is the "only true faith"; (3) acceptance of the view that
the Jews crucified Jesus; (4) religious hostility toward contemporary Jews,
such as believing that God is punishing them or that they will suffer damnation
unless they convert to Christianity; and (5) secular anti-Semitism, such
as believing that Jews cheat in business, are disloyal to their country,
and so forth.

**[Figure 17-3 about here; pickup from 8e p.
416]**

@FN:Figure 17-3 @FT:Diagramming the Religious
Sources of Anti-Semitism

@GT:To start with, the researchers who conducted
this analysis proposed that secular anti-Semitism was produced by moving
through the five variables: Orthodoxy caused particularism, which caused
the view of the historical Jews as crucifiers, which caused religious hostility
toward contemporary Jews, which resulted, finally, in secular anti-Semitism.

The path diagram tells a different story. The
researchers found, for example, that belief in the historical role of Jews
as the crucifiers of Jesus doesn’t seem to matter in the process that generates
anti-Semitism. And, although particularism is a part of one process resulting
in secular anti-Semitism, the diagram also shows that anti-Semitism is
created more directly by orthodoxy and religious hostility. Orthodoxy produces
religious hostility even without particularism, and religious hostility
generates secular hostility in any event.

One last comment on path analysis is in order.
Although it is an excellent way of handling complex causal chains and networks
of variables, path analysis itself does not tell the causal order of the
variables. Nor was the path diagram generated by computer. The researcher
decided the structure of relationships among the variables and used computer
analysis merely to calculate the path coefficients that apply to such a
structure.

@H2:Time-Series Analysis

@GT-no indent:The various forms of regression
analysis are often used to examine time-series data, representing changes
in one or more variables over time. As I’m sure you know, U.S. crime rates
have generally increased over the years. A time-series analysis of crime
rates could express the long-term trend in a regression format and provide
a way of testing explanations for the trend—such as population growth or
economic fluctuations—and could permit forecasting of future crime rates.

@GT:In a simple illustration, Figure 17-4 graphs
the larceny rates of a hypothetical city over time. Each dot on the graph
represents the number of larcenies reported to police during the year indicated.

**[Figure 17-4 about here; pickup from 8e p.
417]**

@FN:Figure 17-4 @FT:The Larceny Rates over Time
in a Hypothetical City

@GT:Suppose we feel that larceny is partly a
function of overpopulation. You might reason that crowding would lead to
psychological stress and frustration, resulting in increased crimes of
many sorts. Recalling the discussion of regression analysis, we could create
a regression equation representing the relationship between larceny and
population density—using the actual figures for each variable, with years
as the units of analysis. Having created the best-fitting regression equation,
we could then calculate a larceny rate for each year, based on that year’s
population density rate. For the sake of simplicity, let’s assume that
the city’s population size (and hence density) has been steadily increasing.
This would lead us to predict a steadily increasing larceny rate as well.
These regression estimates are represented by the dashed regression line
in Figure 17-4.

Time-series relationships are often more complex
than this simple illustration suggests. For one thing, there can be more
than one causal variable. For example, we might find that unemployment
rates also had a powerful impact on larceny. We might develop an equation
to predict larceny on the basis of both of these causal variables. As a
result, the predictions might not fall along a simple, straight line. Whereas
population density was increasing steadily in the first model, unemployment
rates rise and fall. As a consequence, our predictions of the larceny rate
would similarly go up and down.

Pursuing the relationship between larceny and
unemployment rates, we might reason that people do not begin stealing as
soon as they become unemployed. Typically, they might first exhaust their
savings, borrow from friends, and keep hoping for work. Larceny would be
a last resort.

Time-lagged regression analysis could be used
to address this more complex case. Thus, we might create a regression equation
that predicted a given year’s larceny rate based, in part, on the previous
year’s unemployment rate or perhaps on an average of the two years’ unemployment
rates. The possibilities are endless.

If you think about it, a great many causal relationships
are likely to involve a time lag. Historically, many of the world’s poor
countries have maintained their populations by matching high death rates
with equally high birthrates. It has been observed repeatedly, moreover,
that when a society’s death rate is drastically reduced—through improved
medical care, public sanitation, and improved agriculture, for example—that
society’s birthrate drops sometime later on, but with an intervening period
of rapid population growth. Or, to take a very different example, a crackdown
on speeding on a state’s highways is likely to reduce the average speed
of cars. Again, however, the causal relationship would undoubtedly involve
a time lag—days, weeks, or months, perhaps—as motorists began to realize
the seriousness of the crackdown.

In all such cases, the regression equations generated
might take many forms. In any event, the criterion for judging success
or failure is the extent to which the researcher can account for the actual
values observed for the dependent variable.

@H2:Factor Analysis

@GT-no indent: Factor analysis is a unique approach
to multivariate analysis. Its statistical basis is complex enough and different
enough from the foregoing discussions to suggest a general discussion here.

@GT:Factor analysis is a complex algebraic method
used to discover patterns among the variations in values of several variables.
This is done essentially through the generation of artificial dimensions
(factors) that correlate highly with several of the real variables and
that are independent of one another. A computer must be used to perform
this complex operation.

Let’s suppose that a data file contains several
indicators of subjects’ prejudice. Each item should provide some indication
of prejudice, but none will give a perfect indication. All of these items,
moreover, should be highly intercorrelated empirically. In a factor analysis
of the data, the researcher would create an artificial dimension that would
be highly correlated with each of the items measuring prejudice. Each subject
would essentially receive a value on that artificial dimension, and the
value assigned would be a good predictor of the observed attributes on
each item.

Suppose now that the same study provided several
indicators of subjects’ mathematical ability. It’s likely that the factor
analysis would also generate an artificial dimension highly correlated
with each of those items.

The output of a factor analysis program consists
of columns representing the several factors (artificial dimensions) generated
from the observed relations among variables plus the correlations between
each variable and each factor—called the factor loadings.

In the preceding example, it’s likely that one
factor would more or less represent prejudice, and another would more or
less represent mathematical ability. Data items measuring prejudice would
have high loadings on (correlations with) the prejudice factor and low
loadings on the mathematical ability factor. Data items measuring mathematical
ability would have just the opposite pattern.

In practice, factor analysis does not proceed
in this fashion. Rather, the variables are input to the program, and the
program outputs a series of factors with appropriate factor loadings. The
analyst must then determine the meaning of a given factor on the basis
of those variables that load highly on it. The program’s generation of
factors, however, has no reference to the meaning of variables, only to
their empirical associations. Two criteria are taken into account: (1)
a factor must explain a relatively large portion of the variance found
in the study variables, and (2) every factor must be more or less independent
of every other factor.

Here’s an example of the use of factor analysis.
Many social researchers have studied the problem of delinquency. If you
look deeply into the problem, however, you’ll discover that there are many
different types of delinquents. In a survey of high school students in
a small Wyoming town, Morris Forslund (1980) set out to create a typology
of delinquency. His questionnaire asked students to report whether they
had committed a variety of delinquent acts. He then submitted their responses
to factor analysis. The results are shown in Table 17-6.

**[Table 17-6 about here; pickup from 8e p. 419]**

As you can see in this table, the various delinquent
acts are listed on the left. The numbers shown in the body of the table
are the factor loadings on the four factors constructed in the analysis.
You’ll notice that after examining the dimensions, or factors, Forslund
labeled them. I’ve bracketed the items on each factor that led to his choice
of labels. Forslund summarized the results as follows:

@EX:For the total sample four fairly distinct
patterns of delinquent acts are apparent. In order of variance explained,
they have been labeled: 1) Property Offenses, including both vandalism
and theft; 2) Incorrigibility; 3) Drugs/Truancy; and 4) Fighting. It is
interesting, and perhaps surprising, to find both vandalism and theft appear
together in the same factor. It would seem that those high school students
who engage in property offenses tend to be involved in both vandalism and
theft. It is also interesting to note that drugs, alcohol and truancy fall
in the same factor.

@EXS:(1980:4)

@GT:Having determined this overall pattern, Forslund
reran the factor analysis separately for boys and for girls. Essentially
the same patterns emerged in both cases.

This example shows that factor analysis is an
efficient method of discovering predominant patterns among a large number
of variables. Instead of being forced to compare countless correlations—simple,
partial, and multiple—to discover those patterns, researchers can use factor
analysis for this task. Incidentally, this is a good example of a helpful
use of computers.

Factor analysis also presents data in a form
that can be interpreted by the reader or researcher. For a given factor,
the reader can easily discover the variables loading highly on it, thus
noting clusters of variables. Or, the reader can easily discover which
factors a given variable is or is not loaded highly on.

But factor analysis also has disadvantages. First,
as noted previously, factors are generated without any regard to substantive
meaning. Often researchers will find factors producing very high loadings
for a group of substantively disparate variables. They might find, for
example, that prejudice and religiosity have high positive loadings on
a given factor, with education having an equally high negative loading.
Surely the three variables are highly correlated, but what does the factor
represent in the real world? All too often, inexperienced researchers will
be led into naming such factors as "religio-prejudicial lack of education"
or something similarly nonsensical.

Second, factor analysis is often criticized on
basic philosophical grounds. Recall that to be useful, a hypothesis must
be disprovable. If the researcher cannot specify the conditions under which
the hypothesis would be disproved, the hypothesis is in reality either
a tautology or useless. In a sense, factor analysis suffers this defect.
No matter what data are input, factor analysis produces a solution in the
form of factors. Thus, if the researcher were asking, "Are there any patterns
among these variables?" the answer always would be yes. This fact must
also be taken into account in evaluating the results of factor analysis.
The generation of factors by no means ensures meaning.

My personal view of factor analysis is the same
as that for other complex modes of analysis. It can be an extremely useful
tool for the social science researcher. Its use should be encouraged whenever
such activity may assist researchers in understanding a body of data. As
in all cases, however, such tools are only tools and never magical solutions.

Let me reiterate that the analytical techniques
we’ve touched on are only a few of the many techniques commonly used by
social scientists. As you pursue your studies, you may very well want to
study this subject in more depth later.

@H1:INFERENTIAL STATISTICS

@GT-no indent:Many, if not most, social scientific
research projects involve the examination of data collected from a sample
drawn from a larger population. A sample of people may be interviewed in
a survey; a sample of divorce records may be coded and analyzed; a sample
of newspapers may be examined through content analysis. Researchers seldom
if ever study samples just to describe the samples per se; in most instances,
their ultimate purpose is to make assertions about the larger population
from which the sample has been selected. Frequently, then, you’ll wish
to interpret your univariate and multivariate sample findings as the basis
for inferences about some population.

@GT:This section examines inferential statistics—the
statistical measures used for making inferences from findings based on
sample observations to a larger population. We’ll begin with univariate
data and move to multivariate.

@H2:Univariate Inferences

@GT-no indent:Your textbook dealt with methods
of presenting univariate data. Each summary measure was intended as a method
of describing the sample studied. Now we’ll use such measures to make broader
assertions about a population. This section addresses two univariate measures:
percentages and means.

@GT:If 50 percent of a sample of people say they
had colds during the past year, 50 percent is also our best estimate of
the proportion of colds in the total population from which the sample was
drawn. (This estimate assumes a simple random sample, of course.) It’s
rather unlikely, however, that precisely 50 percent of the population had
colds during the year. If a rigorous sampling design for random selection
has been followed, however, we’ll be able to estimate the expected range
of error when the sample finding is applied to the population.

Your textbook's discussion of sampling theory,
covered the procedures for making such estimates, so I’ll only review them
here. In the case of a percentage, the quantity

**[Set p times q, over n, all in a square root
radical, as in 8e p. 421]**

where p is a proportion, q equals (1 ? p), and
n is the sample size, is called the standard error. As noted in your textbook,
this quantity is very important in the estimation of sampling error. We
may be 68 percent confident that the population figure falls within plus
or minus one standard error of the sample figure; we may be 95 percent
confident that it falls within plus or minus two standard errors; and we
may be 99.9 percent confident that it falls within plus or minus three
standard errors.

Any statement of sampling error, then, must contain
two essential components: the confidence level (for example, 95 percent)
and the confidence interval (for example, 62.5 percent). If 50 percent
of a sample of 1,600 people say they had colds during the year, we might
say we’re 95 percent confident that the population figure is between 47.5
percent and 52.5 percent.

In this example we’ve moved beyond simply describing
the sample into the realm of making estimates (inferences) about the larger
population. In doing so, we must take care in several ways.

First, the sample must be drawn from the population
about which inferences are being made. A sample taken from a telephone
directory cannot legitimately be the basis for statistical inferences about
the population of a city, but only about the population of telephone subscribers
with listed numbers.

Second, the inferential statistics assume several
things. To begin with, they assume simple random sampling, which is virtually
never the case in sample surveys. The statistics also assume sampling with
replacement, which is almost never done—but this is probably not a serious
problem. Although systematic sampling is used more frequently than random
sampling, it, too, probably presents no serious problem if done correctly.
Stratified sampling, because it improves representativeness, clearly presents
no problem. Cluster sampling does present a problem, however, because the
estimates of sampling error may be too small. Quite clearly, street-corner
sampling does not warrant the use of inferential statistics. Finally, this
standard error sampling technique assumes a 100-percent completion rate—that
is, that everyone in the sample completed the survey. This problem increases
in seriousness as the completion rate decreases.

Third, inferential statistics are addressed to
sampling error only, not nonsampling error such as coding errors or misunderstandings
of questions by respondents. Thus, although we might state correctly that
between 47.5 and 52.5 percent of the population (95 percent confidence)
would report having colds during the previous year, we couldn’t so confidently
guess the percentage who had actually had them. Because nonsampling errors
are probably larger than sampling errors in a respectable sample design,
we need to be especially cautious in generalizing from our sample findings
to the population.

@H2:Tests of Statistical Significance

@GT-no indent:There is no scientific answer to
the question of whether a given association between two variables is significant,
strong, important, interesting, or worth reporting. Perhaps the ultimate
test of significance rests with your ability to persuade your audience
(present and future) of the association’s significance. At the same time,
there is a body of inferential statistics to assist you in this regard
called parametric tests of significance. As the name suggests, parametric
statistics are those that make certain assumptions about the parameters
describing the population from which the sample is selected. They allow
us to determine the statistical significance of associations. "Statistical
significance" does not imply "importance" or "significance" in any general
sense. It refers simply to the likelihood that relationships observed in
a sample could be attributed to sampling error alone.

@GT:Although tests of statistical significance
are widely reported in social scientific literature, the logic underlying
them is rather subtle and often misunderstood. Tests of significance are
based on the same sampling logic discussed elsewhere in this book. To understand
that logic, let’s return for a moment to the concept of sampling error
in regard to univariate data.

Recall that a sample statistic normally provides
the best single estimate of the corresponding population parameter, but
the statistic and the parameter seldom correspond precisely. Thus, we report
the probability that the parameter falls within a certain range (confidence
interval). The degree of uncertainty within that range is due to normal
sampling error. The corollary of such a statement is, of course, that it
is improbable that the parameter would fall outside the specified range
only as a result of sampling error. Thus, if we estimate that a parameter
(99.9 percent confidence) lies between 45 percent and 55 percent, we say
by implication that it is extremely improbable that the parameter is actually,
say, 90 percent if our only error of estimation is due to normal sampling.
This is the basic logic behind tests of statistical significance.

@H2:The Logic of Statistical Significance

@GT-no indent:I think I can illustrate the logic
of statistical significance best in a series of diagrams representing the
selection of samples from a population. Here are the elements in the logic:

@NL1:1. Assumptions regarding the independence
of two variables in the population study

2. Assumptions regarding the representativeness
of samples selected through conventional probability sampling procedures

3. The observed joint distribution of sample
elements in terms of the two variables

@GT:Figure 17-5 represents a hypothetical population
of 256 people; half are women, half are men. The diagram also indicates
how each person feels about women enjoying equality to men. In the diagram,
those favoring equality have open circles, those opposing it have their
circles filled in.

**[Figure 17-5 about here; pickup from 8e p.
423]**

@FN:Figure 17-5 @FT:A Hypothetical Population
of Men and Women Who Either Favor or Oppose Sexual Equality

@GT:The question we’ll be investigating is whether
there is any relationship between gender and feelings about equality for
men and women. More specifically, we’ll see if women are more likely to
favor equality than are men, since women would presumably benefit more
from it. Take a moment to look at Figure 17-5 and see what the answer to
this question is.

The illustration in the figure indicates no relationship
between gender and attitudes about equality. Exactly half of each group
favors equality and half opposes it. Recall the earlier discussion of proportionate
reduction of error. In this instance, knowing a person’s gender would not
reduce the "errors" we’d make in guessing his or her attitude toward equality.
The table at the bottom of Figure 17-5 provides a tabular view of what
you can observe in the graphic diagram.

Figure 17-6 represents the selection of a one-fourth
sample from the hypothetical population. In terms of the graphic illustration,
a "square" selection from the center of the population provides a representative
sample. Notice that our sample contains 16 of each type of person: Half
are men and half are women; half of each gender favors equality, and the
other half opposes it.

**[Figure 17-6 about here; pickup from 8e p.
424]**

@FN:Figure 17-6 @FT:A Representative Sample

@GT:The sample selected in Figure 17-6 would
allow us to draw accurate conclusions about the relationship between gender
and equality in the larger population. Following the sampling logic used
in the textbook, we’d note there was no relationship between gender and
equality in the sample; thus, we’d conclude there was similarly no relationship
in the larger population—since we’ve presumably selected a sample in accord
with the conventional rules of sampling.

Of course, real-life samples are seldom such
perfect reflections of the populations from which they are drawn. It would
not be unusual for us to have selected, say, one or two extra men who opposed
equality and a couple of extra women who favored it—even if there was no
relationship between the two variables in the population. Such minor variations
are part and parcel of probability sampling.

Figure 17-7, however, represents a sample that
falls far short of the mark in reflecting the larger population. Notice
it includes far too many supportive women and opposing men. As the table
shows, three-fourths of the women in the sample support equality, but only
one-fourth of the men do so. If we had selected this sample from a population
in which the two variables were unrelated to each other, we’d be sorely
misled by our sample.

**[Figure 17-7 about here; pickup from 8e p.
425]**

@FN:Figure 17-7 @FT:An Unrepresentative Sample

@GT:As you’ll recall, it’s unlikely that a properly
drawn probability sample would ever be as inaccurate as the one shown in
Figure 17-7. In fact, if we actually selected a sample that gave us the
results this one does, we’d look for a different explanation. Figure 17-8
illustrates the more likely situation.

**[Figure 17-8 about here; pickup from 8e p.
426]**

@FN:Figure 17-8 @FT:A Representative Sample from
a Population in Which the Variables Are Related

@GT:Notice that the sample selected in Figure
17-8 also shows a strong relationship between gender and equality. The
reason is quite different this time. We’ve selected a perfectly representative
sample, but we see that there is actually a strong relationship between
the two variables in the population at large. In this latest figure, women
are more likely to support equality than are men: That’s the case in the
population, and the sample reflects it.

In practice, of course, we never know what’s
so for the total population; that’s why we select samples. So if we selected
a sample and found the strong relationship presented in Figures 17-7 and
17-8, we’d need to decide whether that finding accurately reflected the
population or was simply a product of sampling error.

The fundamental logic of tests of statistical
significance, then, is this: Faced with any discrepancy between the assumed
independence of variables in a population and the observed distribution
of sample elements, we may explain that discrepancy in either of two ways:
(1) we may attribute it to an unrepresentative sample, or (2) we may reject
the assumption of independence. The logic and statistics associated with
probability sampling methods offer guidance about the varying probabilities
of varying degrees of unrepresentativeness (expressed as sampling error).
Most simply put, there is a high probability of a small degree of unrepresentativeness
and a low probability of a large degree of unrepresentativeness.

The statistical significance of a relationship
observed in a set of sample data, then, is always expressed in terms of
probabilities. "Significant at the .05 level (p £ .05)" simply means
that the probability that a relationship as strong as the observed one
can be attributed to sampling error alone is no more than 5 in 100. Put
somewhat differently, if two variables are independent of one another in
the population, and if 100 probability samples are selected from that population,
no more than 5 of those samples should provide a relationship as strong
as the one that has been observed.

There is, then, a corollary to confidence intervals
in tests of significance, which represents the probability of the measured
associations being due only to sampling error. This is called the level
of significance. Like confidence intervals, levels of significance are
derived from a logical model in which several samples are drawn from a
given population. In the present case, we assume that there is no association
between the variables in the population, and then we ask what proportion
of the samples drawn from that population would produce associations at
least as great as those measured in the empirical data. Three levels of
significance are frequently used in research reports: .05, .01, and .001.
These mean, respectively, that the chances of obtaining the measured association
as a result of sampling error are 5/100, 1/100, and 1/1,000.

Researchers who use tests of significance normally
follow one of two patterns. Some specify in advance the level of significance
they’ll regard as sufficient. If any measured association is statistically
significant at that level, they’ll regard it as representing a genuine
association between the two variables. In other words, they’re willing
to discount the possibility of its resulting from sampling error only.

Other researchers prefer to report the specific
level of significance for each association, disregarding the conventions
of .05, .01, and .001. Rather than reporting that a given association is
significant at the .05 level, they might report significance at the .023
level, indicating the chances of its having resulted from sampling error
as 23 out of 1,000.

@H2:Chi Square

@GT-no indent:Chi square (c2) is a frequently
used test of significance in social science. It is based on the null hypothesis:
the assumption that there is no relationship between the two variables
in the total population. Given the observed distribution of values on the
two separate variables, we compute the conjoint distribution that would
be expected if there were no relationship between the two variables. The
result of this operation is a set of expected frequencies for all the cells
in the contingency table. We then compare this expected distribution with
the distribution of cases actually found in the sample data, and we determine
the probability that the discovered discrepancy could have resulted from
sampling error alone. An example will illustrate this procedure.

Let’s assume we’re interested in the possible
relationship between church attendance and gender for the members of a
particular church. To test this relationship, we select a sample of 100
church members at random. We find that our sample is made up of 40 men
and 60 women and that 70 percent of our sample say they attended church
during the preceding week, whereas the remaining 30 percent say they did
not.

If there is no relationship between gender and
church attendance, then 70 percent of the men in the sample should have
attended church during the preceding week, and 30 percent should have stayed
away. Moreover, women should have attended in the same proportion. Table
17-7 (part I) shows that, based on this model, 28 men and 42 women would
have attended church, with 12 men and 18 women not attending.

**[Table 17-7 about here; pickup from 8e p. 428]**

Part II of Table 17-7 presents the observed attendance
for the hypothetical sample of 100 church members. Note that 20 of the
men report having attended church during the preceding week, and the remaining
20 say they did not. Among the women in the sample, 50 attended church
and 10 did not. Comparing the expected and observed frequencies (parts
I and II), we note that somewhat fewer men attended church than expected,
whereas somewhat more women attended than expected.

Chi square is computed as follows. For each cell
in the tables, the researcher (1) subtracts the expected frequency for
that cell from the observed frequency, (2) squares this quantity, and (3)
divides the squared difference by the expected frequency. This procedure
is carried out for each cell in the tables, and the several results are
added together. (Part III of Table 17-7 presents the cell-by-cell computations.)
The final sum is the value of chi square: 12.70 in the example.

This value is the overall discrepancy between
the observed conjoint distribution in the sample and the distribution we
would expect if the two variables were unrelated to each other. Of course,
the mere discovery of a discrepancy does not prove that the two variables
are related, since normal sampling error might produce discrepancies even
when there is no relationship in the total population. The magnitude of
the value of chi square, however, permits us to estimate the probability
of that having happened.

@H3:Degrees of Freedom @GT:To determine the statistical
significance of the observed relationship, we must use a standard set of
chi square values. This will require the computation of the degrees of
freedom, which refers to the possibilities for variation within a statistical
model. Suppose I challenge you to find three numbers whose mean is 11.
There is an infinite number of solutions to this problem: (11, 11, 11),
(10, 11, 12), (?11, 11, 33), etc. Now, suppose I require that one of the
numbers be 7. There would still be an infinite number of possibilities
for the other two numbers.

If I told you one number had to be 7 and another
10, there would be only one possible value for the third. If the average
of three numbers is 11, their sum must be 33. If two of the numbers total
17, the third must be 16. In this situation, we say there are two degrees
of freedom. Two of the numbers could have any values we choose, but once
they are specified, the third number is determined.

More generally, whenever we are examining the
mean of N values, we can see that the degrees of freedom is N ? 1. Thus
in the case of the mean of 23 values, we could make 22 of them anything
we liked, but the 23rd would then be determined.

A similar logic applies to bivariate tables,
such as those analyzed by chi square. Consider a table reporting the relationship
between two dichotomous variables: gender (men/women) and abortion attitude
(approve/disapprove). Notice that the table provides the marginal frequencies
of both variables.

@T-1:Abortion Attitude Men Women Total

@TB:Approve 500

Disapprove 500

Total 500 500 1,000

@GT:Despite the conveniently round numbers in
this hypothetical example, notice that there are numerous possibilities
for the cell frequencies. For example, it could be the case that all 500
men approve and all 500 women disapprove, or it could be just the reverse.
Or there could be 250 cases in each cell. Notice there are numerous other
possibilities.

Now the question is, How many cells could we
fill in pretty much as we choose before the remainder are determined by
the marginal frequencies? The answer is only one. If we know that 300 men
approved, for example, then 200 men would have had to disapprove, and the
distribution would need to be just the opposite for the women.

In this instance, then, we say the table has
one degree of freedom. Now, take a few minutes to construct a three-by-three
table. Assume you know the marginal frequencies for each variable, and
see if you can determine how many degrees of freedom it has.

For chi square, the degrees of freedom are computed
as follows: the number of rows in the table of observed frequencies, minus
1, is multiplied by the number of columns, minus 1. This may be written
as (r ? 1)(c ? 1). For a three-by-three table, then, there are four degrees
of freedom: (3 ? 1)(3 ? 1) = (2)(2) = 4.

In the example of gender and church attendance,
we have two rows and two columns (discounting the totals), so there is
one degree of freedom. Turning to a table of chi square values (see Appendix
F), we find that for one degree of freedom and random sampling from a population
in which there is no relationship between two variables, 10 percent of
the time we should expect a chi square of at least 2.7. Thus, if we selected
100 samples from such a population, we should expect about 10 of those
samples to produce chi squares equal to or greater than 2.7. Moreover,
we should expect chi square values of at least 6.6 in only 1 percent of
the samples and chi square values of 7.9 in only half a percent (.005)
of the samples. The higher the chi square value, the less probable it is
that the value could be attributed to sampling error alone.

In our example, the computed value of chi square
is 12.70. If there were no relationship between gender and church attendance
in the church member population and a large number of samples had been
selected and studied, then we would expect a chi square of this magnitude
in fewer than 1/10 of 1 percent (.001) of those samples. Thus, the probability
of obtaining a chi square of this magnitude is less than .001, if random
sampling has been used and there is no relationship in the population.
We report this finding by saying the relationship is statistically significant
at the .001 level. Because it is so improbable that the observed relationship
could have resulted from sampling error alone, we’re likely to reject the
null hypothesis and assume that there is a relationship between the two
variables in the population of church members.

Most measures of association can be tested for
statistical significance in a similar manner. Standard tables of values
permit us to determine whether a given association is statistically significant
and at what level. Any standard statistics textbook provides instructions
on the use of such tables.

@H3:Some Words of Caution @GT:Tests of significance
provide an objective yardstick that we can use to estimate the statistical
significance of associations between variables. They help us rule out associations
that may not represent genuine relationships in the population under study.
However, the researcher who uses or reads reports of significance tests
should remain wary of several dangers in their interpretation.

First, we have been discussing tests of statistical
significance; there are no objective tests of substantive significance.
Thus, we may be legitimately convinced that a given association is not
due to sampling error, but we may be in the position of asserting without
fear of contradiction that two variables are only slightly related to each
other. Recall that sampling error is an inverse function of sample size—the
larger the sample, the smaller the expected error. Thus, a correlation
of, say, .1 might very well be significant (at a given level) if discovered
in a large sample, whereas the same correlation between the same two variables
would not be significant if found in a smaller sample. This makes perfectly
good sense given the basic logic of tests of significance: In the larger
sample, there is less chance that the correlation could be simply the product
of sampling error. In both samples, however, it might represent an essentially
zero correlation.

The distinction between statistical and substantive
significance is perhaps best illustrated by those cases where there is
absolute certainty that observed differences cannot be a result of sampling
error. This would be the case when we observe an entire population. Suppose
we were able to learn the ages of every public official in the United States
and of every public official in Russia. For argument’s sake, let’s assume
further that the average age of U.S. officials was 45 years old compared
with, say, 46 for the Russian officials. Because we would have the ages
of all officials, there would be no question of sampling error. We would
know with certainty that the Russian officials were older than their U.S.
counterparts. At the same time, we would say that the difference was of
no substantive significance. We’d conclude, in fact, that they were essentially
the same age.

Second, lest you be misled by this hypothetical
example, realize that statistical significance should not be calculated
on relationships observed in data collected from whole populations. Remember,
tests of statistical significance measure the likelihood of relationships
between variables being only a product of sampling error; if there’s no
sampling, there’s no sampling error.

Third, tests of significance are based on the
same sampling assumptions we used in computing confidence intervals. To
the extent that these assumptions are not met by the actual sampling design,
the tests of significance are not strictly legitimate.

While we have examined statistical significance
here in the form of chi square, there are several other measures commonly
used by social scientists. Analysis of variance and t-tests are two examples
you may run across in your studies.

As is the case for most matters covered in this
book, I have a personal prejudice. In this instance, it is against tests
of significance. I don’t object to the statistical logic of those tests,
because the logic is sound. Rather, I’m concerned that such tests seem
to mislead more than they enlighten. My principal reservations are the
following:

@NL1:1. Tests of significance make sampling assumptions
that are virtually never satisfied by actual sampling designs.

2. They depend on the absence of nonsampling
errors, a questionable assumption in most actual empirical measurements.

3. In practice, they are too often applied to
measures of association that have been computed in violation of the assumptions
made by those measures (for example, product-moment correlations computed
from ordinal data).

4. Statistical significance is too easily misinterpreted
as "strength of association," or substantive significance.

@GT:These concerns are underscored by a recent
study (Sterling, Rosenbaum, and Weinkam 1995) examining the publication
policies of nine psychology and three medical journals. As the researchers
discovered, the journals were quite unlikely to publish articles that did
not report statistically significant correlations among variables. They
quote the following from a rejection letter:

@EX:Unfortunately, we are not able to publish
this manuscript. The manuscript is very well written and the study was
well documented. Unfortunately, the negative results translates into a
minimal contribution to the field. We encourage you to continue your work
in this area and we will be glad to consider additional manuscripts that
you may prepare in the future.

@EXS:(STERLING ET AL. 1995:109)

@GT:Let’s suppose a researcher conducts a scientifically
excellent study to determine whether X causes Y. The results indicate no
statistically significant correlation. That’s good to know. If we’re interested
in what causes cancer, war, or juvenile delinquency, it’s good to know
that a possible cause actually does not cause it. That knowledge would
free researchers to look elsewhere for causes.

As we’ve seen, however, such a study might very
well be rejected by journals. As such, other researchers would continue
testing whether X causes Y, not knowing that previous studies found no
causal relationship. This would produce many wasted studies, none of which
would see publication and draw a close to the analysis of X as a cause
of Y.

From what you’ve learned about probabilities,
however, you can understand that if enough studies are conducted, one will
eventually measure a statistically significant correlation between X and
Y. If there is absolutely no relationship between the two variables, we
would expect a correlation significant at the .05 level five times out
of a hundred, since that’s what the .05 level of significance means. If
a hundred studies were conducted, therefore, we could expect five to suggest
a causal relationship where there was actually none—and those five studies
would be published!

There are, then, serious problems inherent in
too much reliance on tests of statistical significance. At the same time
(perhaps paradoxically) I would suggest that tests of significance can
be a valuable asset to the researcher—useful tools for understanding data.
Although many of my comments suggest an extremely conservative approach
to tests of significance—that you should use them only when all assumptions
are met—my general perspective is just the reverse.

I encourage you to use any statistical technique—any
measure of association or test of significance—if it will help you understand
your data. If the computation of product-moment correlations among nominal
variables and the testing of statistical significance in the context of
uncontrolled sampling will meet this criterion, then I encourage such activities.
I say this in the spirit of what Hanan Selvin, another pioneer in developing
the elaboration model, referred to as "data-dredging techniques." Anything
goes, if it leads ultimately to the understanding of data and of the social
world under study.

The price of this radical freedom, however, is
the giving up of strict, statistical interpretations. You will not be able
to base the ultimate importance of your finding solely on a significant
correlation at the .05 level. Whatever the avenue of discovery, empirical
data must ultimately be presented in a legitimate manner, and their importance
must be argued logically.

@H1:MAIN POINTS

@BL:o Descriptive statistics are used to summarize
data under study. Some descriptive statistics summarize the distribution
of attributes on a single variable; others summarize the associations between
variables.

o Descriptive statistics summarizing the relationships
between variables are called measures of association.

o Many measures of association are based on a
proportionate reduction of error (PRE) model. This model is based on a
comparison of (1) the number of errors we would make in attempting to guess
the attributes of a given variable for each of the cases under study—if
we knew nothing but the distribution of attributes on that variable—and
(2) the number of errors we would make if we knew the joint distribution
overall and were told for each case the attribute of one variable each
time we were asked to guess the attribute of the other. These measures
include lambda (l), which is appropriate for the analysis of two nominal
variables; gamma (g), which is appropriate for the analysis of two ordinal
variables; and Pearson’s product-moment correlation (r), which is appropriate
for the analysis of two interval or ratio variables.

o Regression analysis represents the relationships
between variables in the form of equations, which can be used to predict
the values of a dependent variable on the basis of values of one or more
independent variables.

o Regression equations are computed on the basis
of a regression line: that geometric line representing, with the least
amount of discrepancy, the actual location of points in a scattergram.

o Types of regression analysis include linear
regression analysis, multiple regression analysis, partial regression analysis,
and curvilinear regression analysis.

o Other multivariate techniques include time-series
analysis, the study of processes occurring over time; path analysis, a
method of presenting graphically the networks of causal relationships among
several variables; and factor analysis, a method of discovering the general
dimensions represented by a collection of actual variables.

o Inferential statistics are used to estimate
the generalizability of findings arrived at through the analysis of a sample
to the larger population from which the sample has been selected. Some
inferential statistics estimate the single-variable characteristics of
the population; others—tests of statistical significance—estimate the relationships
between variables in the population.

o Inferences about some characteristic of a population
must indicate a confidence interval and a confidence level. Computations
of confidence levels and intervals are based on probability theory and
assume that conventional probability sampling techniques have been employed
in the study.

o Inferences about the generalizability to a
population of the associations discovered between variables in a sample
involve tests of statistical significance, which estimate the likelihood
that an association as large as the observed one could result from normal
sampling error if no such association exists between the variables in the
larger population. Tests of statistical significance are also based on
probability theory and assume that conventional probability sampling techniques
have been employed in the study.

o A frequently used test of statistical significance
in social science is chi square.

o The level of significance of an observed association
is reported in the form of the probability that the association could have
been produced merely by sampling error. To say that an association is significant
at the .05 level is to say that an association as large as the observed
one could not be expected to result from sampling error more than 5 times
out of 100.

o Social researchers tend to use a particular
set of levels of significance in connection with tests of statistical significance:
.05, .01, and .001. This is merely a convention, however.

o Statistical significance must not be confused
with substantive significance, the latter meaning that an observed association
is strong, important, meaningful, or worth writing home to your mother
about.

o Tests of statistical significance, strictly
speaking, make assumptions about data and methods that are almost never
satisfied completely by real social research. Despite this, the tests can
serve a useful function in the analysis and interpretation of data.

@H1:Key Terms

@UL:descriptive statistics

proportionate reduction of error (PRE)

regression analysis

linear regression analysis

multiple regression analysis

partial regression analysis

curvilinear regression analysis

path analysis

time-series analysis

factor analysis

inferential statistics

nonsampling error

statistical significance

tests of statistical significance

level of significance

null hypothesis

@H1:REVIEW QUESTIONS AND EXERCISES

@NL1:1. In your own words, explain the logic
of proportionate reduction of error (PRE) measures of associations.

2. In your own words, explain the purpose of
regression analyses.

3. In your own words, distinguish between measures
of association and tests of statistical significance.

4. Find a study that reports the statistical
significance of its findings and critique the clarity with which it is
reported.

5. Locate a study that uses factor analysis and
summarize the findings.

@H1:ADDITIONAL READINGS

@UL:Babbie, Earl, Fred Halley, and Jeanne Zaino.
2000. Adventures in Social Research. Newbury Park, CA: Pine Forge Press.
This book introduces the analysis of social research data through SPSS
for Windows. Several of the basic statistical techniques used by social
researchers are discussed and illustrated.

Blalock, Hubert M., Jr. 1979. Social Statistics.
New York: McGraw-Hill. Blalock’s textbook has been a standard for social
science students (and faculty) for decades. Tad Blalock’s death was a loss
to all social science.

Frankfort-Nachmias, Chava. 1997. Social Statistics
for a Diverse Society. Newbury Park, CA: Pine Forge Press. A comprehensive
textbook on social statistics that makes particularly good use of graphics
in presenting the logic of the many statistics commonly used by social
scientists.

Healey, Joseph F. 1999. Statistics: A Tool for
Social Research. Belmont, CA: Wadsworth. An effective introduction to social
statistics.

Mohr, Lawrence B. 1990. Understanding Significance
Testing. Newbury Park, CA: Sage. An excellent and comprehensive examination
of the topic: both the technical details of testing statistical significance
and the meaning of such tests.

@H1:Sociology Web Site

@GT-no indent:See the Wadsworth Sociology Resource
Center, Virtual Society, for additional links, Internet exercises by chapter,
quizzes by chapter, and Microcase-related materials:

http://www.sociology.wadsworth.com

@H1:InfoTrac College Edition

@H2:Search Word Summary

@GT-no indent:Go to the Wadsworth Sociology Resource
Center, Virtual Society, to find a list of search words for each chapter.
Using the search words, go to InfoTrac College Edition, an online library
of over 900 journals where you can do online research and find readings
related to your studies. To aid in your search and to gain useful tips,
see the Student Guide to InfoTrac College Edition on the Virtual Society
Web site:

http://www.sociology.wadsworth.com