Cinnabar and Alabaster

survey methodology:

• purpose: "to make valid generalizations about large groups of people by systematically gathering information from a small portion of the individuals in those groups" (457). (can have mult. purposes contained in the same survey if the same population can offer mult. sets of answers)


• research questions:


• descriptive: "researchers desire to compile facts, very much in the manner of a census taker, about the incidence and distribution of the phenomena under study"


• explanatory: "researchers are concerned with the ways that various phenomena relate to one another"; sample "categories of explanatory research questions":


• comparison within groups ("do members of a do more of x or y?")


• comparison between groups ("do members of a respond differently to x than members of y?")


• covariance among phenomena ("is an increase in x always accompanied by an increase in y?")


• cause and effect ("will changing x cause y to change?") (460)


• respondants: can be entire or samples of entire populations; if the sample-set isn't representative, its data lacks value. researchers have to


• define research population


• choose a sampling method


• probability sampling (every member of the pop. has a chance to be chosen, & that chance is mathematically known)


• random sampling


• stratified random sampling (random within identified & divided by subgroup or "stratum") (462-3)


• cluster sampling—groups by location, accessibility, etc.


• systematic sampling (taking every 10th in an ordered list of 1000 persons for a 100-person sample, etc.)


• nonprobability sampling:


• convenience sampling (use who's there)


• quota sampling (gather ppl by identified characteristic but also by accessability rather than random choice w/in the population) (465)


• true, mathematically thorough representativeness is hard to come by & isn't necessary for validity, but straying too far weakens results


• consider response rate: ppl not likely to respond aren't valuable, no matter how representative/random their inclusion might be (duh). ways to increase response rate:


• design survey forms to look easy


• start w/more interesting questions (demographics last)


• prepaid evelopes or otherwise account for easy return


• cover-letter to explain "social usefulness" (value) of study


• official stationary (466-7)


• determine optimal sample size, weighing analytically-viable data against cost (time, $, etc.)


• survey instrument:


• 4 categories of questions:


• attributes (how ppl describe their ed. level, age, etc.)


• behavior (how ppl describe what they do)


• beliefs (what ppl think is true)


• attitudes (how ppl feel about something)


• operational definitions (specified procedures to follow to measure whatever's being measured):


• "in the first step, researchers define as fully and precisely as possible the meaning of the abstraction they desire to study. what is meant, for instance, by 'the importance of writing'…"—includes dimensions s.a. "the extent to which writing is essential to performing a person's job and the effects of writing ability on a person's prospects for advancement" (470-1)


• "the second step…is to create the specific questions that will be asked to obtain measures along each dimension" & deciding whether they'll be closed or


• levels of measurement ("all [questions that might be asked in a survey] involve one of four basic methods of assigning numbers to phenomena"):


• nominal level: "responses are gathered in mutually exclusive and exhaustive categories, and each category is assigned a number" where "assignment of a particular number to a particular category is arbitrary"


• ordinal level: "respondants are asked to provide rank-order information" on some point-scale


• interval level: "the distance between one value and the next is the same as that between any other pair of adjacent values" but "there is no absolute zero" (i.e. "one could not say that someone with an IQ of 120 is twice as smart as a person with an IQ of 60"


• ratio level: has an absolute zero, & so "a person who writes 4 memos a day writes twice as many as a person who writes only 2" (472-3).


• measurement error—there's going to be some. account for it. some quarters from which it might hail:


• "errors that arise from the respondent's desire to mislead the researcher—for instance, in order to make a good impression"


• "errors that arise from the respondent's inability to remember accurately"


• "errors that arise from communication problems—for instance, when the respondent may misunderstand the question or the researcher may misunderstand the response" (475)


• reliability: usually defined operationally; re: stability or equivalence


• validity:


• face-validity (does it measure what it says it does)


• concurrent validity (does it match other concurrent data)


• predictive validity (compared to future performance etc.) (479)


• data analysis—needs to be appropriate to the data & questions asked of it


• descriptive statistics—analyze distributions of data: s.a. frequency distributions & percentages (480) but "used alone, they only summarize the responses from the sample; they do not provide a sound basis for drawing conclusions about the population from which the sample is drawn" (485)


• univariate analysis


• measures of central tendency


• mean (480)


• median


• mode


• measures of dispersion (481)


• range


• sample variance (spread)


• standard deviation (482)


• bivariate analysis—unlike the types of measures above, which are used only for "analysis involving only one variable," b.a. methods "concern the assiciation between the responses to two questions"


• used to link/describe links btw. dependent & independent variables


• highly interpretive/commonsense (483)


• rely on measures of association


• correlation coeficient


• multivariate analysis—examining the relationship btw. responses to 3 or more questions (484)


• inferential statistics: "can be used to make inferences about populations based on information gathered from samples"


• statistics—"the value obtained by performing some statistical procedure on data gathered from a sample"


• parameter—"the value that would be obtained by performing a statistical procedure gathered from every member of a population" (485)


• sampling error—the difference btw. a given statistic & the parameter that could have been established


• levels of confidence--%age of researchers' certainty that their statistical data is representative


• bivariate and multivariate analysis—leaps in connectivity are allowed here.


• null hypothesis—the base-guess preceding data-collection


• statistical significance—"the probability that it is a mistake the reject the null hypothesis"; "the probability that the difference found in the sample does not represent a real difference found in the population."  tells researchers "whether it is probable that a difference exists; it cannot tell researchers whether the difference is important" (488)


• substantive significance = whether the difference is important


• if you're doing survey work:


• "inferential statistics are essential to any survey intended to generalize beyond the sample to a larger population"


• "because inferential statistics constitute a very technical field, it is wise to enlist the advice of an expert"


• use your expert all along—from question-design on up (494)


• limitations of survey methodology


• generalizations only work if your sample is believably representative


• surveys only work "for studying phenomena about which people can report accurately"


• "researchers rigidly control the kinds of information that their subjects can provide about the phenomena under study; as a result, the subjects' responses may provide distored information about these phenomena (494).


• integration w/other methods: is a good idea.