Survey : Instrument Design

Instrument Design

One criticism of library surveys is that they are often poorly designed and administered (Busha and Harter 1980), resulting in data that is that is not very accurate, but that is energetically quoted and used to make important decisions. Surveys should be just as rigourously designed and administered as any other research method. Meyer (1998) has identified five preliminary steps that should be taken when embarking upon any research project: 1) choose a topic, 2) review the literature, 3) determine the research question, 4) develop a hypothesis, and 5) operationalization (i.e., figure out how to accurately measure the factors you wish to measure). For research using surveys, two additional considerations are of prime importance: representative sampling and question design. Much of the following information was taken from the book Research Methods in Librarianship: Techniques and Interpretation by Charles H. Busha and Stephen P. Harter.

Representative Sampling

A sample is representative when it is an accurate proportional representation of the population under study. If you want to study the attitudes of UT students regarding library services, it would not be enough to interview every 100th person who walked into the library. That technique would only measure the attitudes of UT students who use the library, not those who do not. In addition, it would only measure the attitudes of UT students who happened to use the library during the time you were collecting data. Therefore, the sample would not be very representative of UT students in general. In order to be a truly representative sample, every student at UT would have to have had an equal chance of being chosen to participate in the survey. This is called randomization.

If you stood in front of the student union and walked up to students, asking them questions, you still would not have a random sample. You would only be questioning students who happened to come to campus that day, and further, those that happened to walk past the student union. Those students who never walk that way would have had no chance of being questioned. In addition, you might unintentionally be biased as to who you question. You might unconsciously choose not to question students who look preoccupied or busy, or students who don't look like friendly people. This would invalidate your results, since your sample would not be randomly selected.

If you took a list of UT students, uploaded it onto a computer, then instructed the computer to randomly generate a list of 2 percent of all UT students, then your sample still might not be representative. What if, purely by chance, the computer did not include the correct proportion of seniors, or honors students, or graduate students? In order to further ensure that the sample is truly representative of the population, you might want to use a sampling technique called stratification. In order to stratify a population, you need to decide what sub-categories of the population might be statistically significant. For instance, graduate students as a group probably have different opinions than undergraduates regarding library usage, so they should be recognized as separate strata of the population. Once you have a list of the different strata, along with their respective percentages, you could instruct the computer to again randomly select students, this time taking care that a certain percentage are graduate students, a certain percentage are honors students, and a certain percentage are seniors. You would then come up with a more truly representative sample.