When to Use Surveys in Psychological Research

Building a better survey experience

When to Use Surveys in Psychological Research

Survey research is crucial to psychology and other behavioral and social sciences. At the crux of any line of research, whether in the laboratory, clinic or community, is reliable and valid measurement.

Surveys provide the opportunity to pilot, refine or otherwise evaluate measures in large samples of participants in a relatively low cost and time efficient manner. They can also be used to complement ongoing research projects.

The growth of crowdsourcing platforms, such as Amazon’s Mechanical Turk (mTurk), has made survey research all the more appealing for researchers who may have previously overlooked its advantages.

The first and most essential step for conducting survey research is building the survey.

Below are 10 tips for creating effective survey experiences that will not only be accessible to participants but also help generate informative data.

Much of this advice targets situations in which tight experimental control is lost (e.g., online or other remote sampling methods) and the weight of verifying study fidelity is placed on the survey design.

1. Seek out other surveys

Just as one of the best ways to hone your writing skills is to read others’ work, one of the best ways to develop effective surveys is to review others’ surveys.

Participating in survey research provides you a feeling for what it is to sit in your future participant’s seat and how different question formats may be particularly effective (or ineffective).

Similarly, you may be able to find surveys used in published research in the appendices or supplementary materials of those publications. These materials can help kick start your own survey creation and provide ideas for new survey formats.

2. Select validated measures (or validate yourself)

Selecting measures with demonstrated reliability and validity will save you a lot of heartache on the backend of data analysis and study write-up.

However, it is still important to verify that your factor structures and psychometrics hold in the sample that you collect.

Running a confirmatory factor analysis and evaluating internal consistency will generate information that will demonstrate to reviewers that you are appropriately handling the data and validating the strength of your measures.

If your measure has not yet demonstrated acceptable reliability and validity, do not fret, run your own reliability and validity study. Although methodological studies are not always considered the most glamorous, such research is critical for producing sound psychological science (and is often heavily cited).

3. Be mobile and tablet friendly

Participants sampled from outside the laboratory will often complete surveys on devices other than computers. Ensuring compatibility with smart phones or tablets will reduce burden and, in doing so, improve the quality of data collection and reduce the number of complaints from participants.

Many online survey platforms, such as Qualtrics or REDCap, include options that increase mobile compatibility and allow you to view what a question looks on a computer, phone or tablet.

If you don’t want participants completing surveys on mobile platforms, however, be sure to clearly state this in your instructions.

4. Act a plumber and use those pipes

Although a subtle alteration, piped text (i.e., displaying/utilizing responses that participants previously entered) can help improve the efficiency and quality of data collection by personalizing the survey experience.

For example, say that you are studying the effect of income reduction on decision-making. One way to do this is simply tell participants “imagine your typical income was cut in half.” However, with piped text you can first ask participants their income (e.g.

, $40,000/year) and then use this entry to create a personalized instruction “imagine your income was now $20,000/year.”

5. Use uncertainty as a tool

Uncertain or ambiguous responses are typically frowned upon during survey building. However, you can use uncertainty as a tool when evaluating information comprehension.

Including the response “I Don’t Know” or “Unsure” when testing participant’s knowledge about a topic will help discourage random guessing and improve the validity of your data. This is particularly important when the choices are otherwise dichotomous (e.g.

, “Yes/No” or “True/False”) and the lihood of a random guess being correct is high.

6. Use attention checks (but don’t go overboard)

One of the most common survey recommendations is to include attention checks. Attention checks, such as remembering a single digit number for later entry or selecting a particular response (e.g.

, requiring a “True” response to the prompt “My heart is beating”), can improve the quality of data collection by verifying participant attentiveness. However, the overuse of attention checks can also frustrate participants and, ultimately, diminish the benefits these checks otherwise present.

A balance between ensuring valid data and respecting participant’s time is therefore needed. One simple method for creating unobtrusive attention checks is to ask questions about stable characteristics across the survey (e.g., what is your age?) and use these answers to verify consistent responding.

These questions are often less bothersome for participants, but still address concerns about dishonest or inattentive responding.

7. Verify understanding

Instructions often include important details that are easily missed if participants skim over the text. One way to confirm understanding is to include instruction verification questions. Short questions that ask about what a participant is expected to do can help increase the quality of data collection.

For example, if participants are asked to think about standard alcoholic drinks for a series of survey items, including a question verifying that participants read and understand what a standard alcoholic drink is can improve the quality of data collected.

These questions will, by honing in on important details, also reduce the frustration participants may experience when not understanding a task. wise, if you are showing participants a video or vignette with differing conditions (e.g.

, one boss who is high in narcissism and one who is low), these questions can help verify whether participants were sufficiently exposed to and experienced the manipulation used.

8. Repetition, repetition, repetition…

Clear instructions are important, but repeating these instructions is equally important. Repeat instructions on each new page or within a page if a particularly long survey page is used.

Similarly, make sure that scale anchors are always visible.

Repeating instructions or scale anchors ensures that participants are not burdened by flipping through the survey or left otherwise confused when they forget the directions.

9. …and beta, beta, beta

One of the final steps in survey design is to beta test your survey multiple times. It is not enough to just run through the various questionnaires once and call it a day. Try to “break” your survey in order to identify any problems that may crop up during widespread dissemination.

The general axiom holds true that if something can be broken or disrupted in a task, a participant will find a way to do so. Be sure to also have friends, coworkers, research assistants or even pilot participants who are unfamiliar with the study complete the survey.

This can provide an adequate estimate of how long the survey may take as well as identify problems in the instructions that you may otherwise not recognize.

10. Respect participants

Ultimately, respect the time of your participants. Many of the suggestions noted above will help in the design of surveys that are easy (and maybe even fun) to complete. Quantitative researchers are often wary of open-ended, qualitative responses.

However, including a section for participants to express problems or concerns at the end of the survey is an effective way to recognize current problems and improve future research. Identifying participant grievances in this way helps sidestep the abundance of negative emails you may otherwise receive.

These sections also communicate to participants that you care about their experience and want to make it as straightforward as possible.

About the author

Justin C. Strickland is a representative on the APA Science Student Council. He is a fourth-year doctoral student at the University of Kentucky.

Источник: https://www.apa.org/science/about/psa/2017/12/survey-experience

7.1 Overview of Survey Research

When to Use Surveys in Psychological Research

  1. Define what survey research is, including its two important characteristics.
  2. Describe several different ways that survey research can be used and give some examples.

Survey research is a quantitative and qualitative method with two important characteristics.

First, the variables of interest are measured using self-reports (using questionnaires or interviews). In essence, survey researchers ask their participants (who are often called respondents in survey research) to report directly on their own thoughts, feelings, and behaviors. Second, considerable attention is paid to the issue of sampling.

In particular, survey researchers have a strong preference for large random samples because they provide the most accurate estimates of what is true in the population. In fact, survey research may be the only approach in psychology in which random sampling is routinely used. Beyond these two characteristics, almost anything goes in survey research.

Surveys can be long or short. They can be conducted in person, by telephone, through the mail, or over the Internet. They can be about voting intentions, consumer preferences, social attitudes, health, or anything else that it is possible to ask people about and receive meaningful answers.

 Although survey data are often analyzed using statistics, there are many questions that lend themselves to more qualitative analysis.

Most survey research is non-experimental. It is used to describe single variables (e.g., the percentage of voters who prefer one presidential candidate or another, the prevalence of schizophrenia in the general population) and also to assess statistical relationships between variables (e.g., the relationship between income and health).

But surveys can also be experimental. The study by Lerner and her colleagues is a good example. Their use of self-report measures and a large national sample identifies their work as survey research. But their manipulation of an independent variable (anger vs.

fear) to assess its effect on a dependent variable (risk judgments) also identifies their work as experimental.

History and Uses of Survey Research

Survey research may have its roots in English and American “social surveys” conducted around the turn of the 20th century by researchers and reformers who wanted to document the extent of social problems such as poverty (Converse, 1987).

By the 1930s, the US government was conducting surveys to document economic and social conditions in the country. The need to draw conclusions about the entire population helped spur advances in sampling procedures.

At about the same time, several researchers who had already made a name for themselves in market research, studying consumer preferences for American businesses, turned their attention to election polling. A watershed event was the presidential election of 1936 between Alf Landon and Franklin Roosevelt.

A magazine called Literary Digest conducted a survey by sending ballots (which were also subscription requests) to millions of Americans. this “straw poll,” the editors predicted that Landon would win in a landslide.

At the same time, the new pollsters were using scientific methods with much smaller samples to predict just the opposite—that Roosevelt would win in a landslide. In fact, one of them, George Gallup, publicly criticized the methods of Literary Digest before the election and all but guaranteed that his prediction would be correct.

And of course, it was. (We will consider the reasons that Gallup was right later in this chapter.) Interest in surveying around election times has led to several long-term projects, notably the Canadian Election Studies which has measured opinions of Canadian voters around federal elections since 1965.  Anyone can access the data and read about the results of the experiments in these studies (see http://ces-eec.arts.ubc.ca/)

From market research and election polling, survey research made its way into several academic fields, including political science, sociology, and public health—where it continues to be one of the primary approaches to collecting new data.

Beginning in the 1930s, psychologists made important advances in questionnaire design, including techniques that are still used today, such as the rt scale. (See “What Is a rt Scale?” in Section 7.2 “Constructing Survey Questionnaires”.

) Survey research has a strong historical association with the social psychological study of attitudes, stereotypes, and prejudice.

Early attitude researchers were also among the first psychologists to seek larger and more diverse samples than the convenience samples of university students that were routinely used in psychology (and still are).

Survey research continues to be important in psychology today. For example, survey data have been instrumental in estimating the prevalence of various mental disorders and identifying statistical relationships among those disorders and with various other factors.

The National Comorbidity Survey is a large-scale mental health survey conducted in the United States (see http://www.hcp.med.harvard.edu/ncs). In just one part of this survey, nearly 10,000 adults were given a structured mental health interview in their homes in 2002 and 2003. Table 7.

1 presents results on the lifetime prevalence of some anxiety, mood, and substance use disorders. (Lifetime prevalence is the percentage of the population that develops the problem sometime in their lifetime.

) Obviously, this kind of information can be of great use both to basic researchers seeking to understand the causes and correlates of mental disorders as well as to clinicians and policymakers who need to understand exactly how common these disorders are.

Table 7.1 Some Lifetime Prevalence Results From the National Comorbidity Survey
Lifetime prevalence*
Generalized anxiety disorder5.77.14.2
Obsessive-compulsive disorder2.33.11.6
Major depressive disorder16.920.213.2
Bipolar disorder4.44.54.3
Alcohol abuse13.27.519.6
Drug abuse8.04.811.6
*The lifetime prevalence of a disorder is the percentage of people in the population that develop that disorder at any time in their lives.

And as the opening example makes clear, survey research can even be used to conduct experiments to test specific hypotheses about causal relationships between variables.

Such studies, when conducted on large and diverse samples, can be a useful supplement to laboratory studies conducted on university students.

Although this approach is not a typical use of survey research, it certainly illustrates the flexibility of this method.

Key Takeaways

  • Survey research features the use of self-report measures on carefully selected samples. It is a flexible approach that can be used to study a wide variety of basic and applied research questions.
  • Survey research has its roots in applied social research, market research, and election polling. It has since become an important approach in many academic disciplines, including political science, sociology, public health, and, of course, psychology.


  1. Discussion: Think of a question that each of the following professionals might try to answer using survey research.
    1. a social psychologist
    2. an educational researcher
    3. a market researcher who works for a supermarket chain
    4. the mayor of a large city
    5. the head of a university police force

Источник: https://opentext.wsu.edu/carriecuttler/chapter/7-1-overview-of-survey-research/

Pros and Cons of Survey Research

When to Use Surveys in Psychological Research

Survey research, as with all methods of data collection, comes with both strengths and weaknesses. We’ll examine both in this section.

Researchers employing survey methods to collect data enjoy a number of benefits. First, surveys are an excellent way to gather lots of information from many people.

In my own study of older people’s experiences in the workplace, I was able to mail a written questionnaire to around 500 people who lived throughout the state of Maine at a cost of just over $1,000.

This cost included printing copies of my seven-page survey, printing a cover letter, addressing and stuffing envelopes, mailing the survey, and buying return postage for the survey. I realize that $1,000 is nothing to sneeze at. But just imagine what it might have cost to visit each of those people individually to interview them in person.

Consider the cost of gas to drive around the state, other travel costs, such as meals and lodging while on the road, and the cost of time to drive to and talk with each person individually. We could double, triple, or even quadruple our costs pretty quickly by opting for an in-person method of data collection over a mailed survey. Thus surveys are relatively cost effective.

Related to the benefit of cost effectiveness is a survey’s potential for generalizability.

Because surveys allow researchers to collect data from very large samples for a relatively low cost, survey methods lend themselves to probability sampling techniques, which we discussed in Chapter 7 «Sampling».

Of all the data-collection methods described in this text, survey research is probably the best method to use when one hopes to gain a representative picture of the attitudes and characteristics of a large group.

Survey research also tends to be a reliable method of inquiry. This is because surveys are standardizedThe same questions, phrased in the same way, are posed to all participants, consistent.

in that the same questions, phrased in exactly the same way, are posed to participants.

Other methods, such as qualitative interviewing, which we’ll learn about in Chapter 9 «Interviews: Qualitative and Quantitative Approaches», do not offer the same consistency that a quantitative survey offers. This is not to say that all surveys are always reliable.

A poorly phrased question can cause respondents to interpret its meaning differently, which can reduce that question’s reliability. Assuming well-constructed question and questionnaire design, one strength of survey methodology is its potential to produce reliable results.

The versatilityA feature of survey research meaning that many different people use surveys for a variety of purposes and in a variety of settings. of survey research is also an asset. Surveys are used by all kinds of people in all kinds of professions.

I repeat, surveys are used by all kinds of people in all kinds of professions. Is there a light bulb switching on in your head? I hope so. The versatility offered by survey research means that understanding how to construct and administer surveys is a useful skill to have for all kinds of jobs.

Lawyers might use surveys in their efforts to select juries, social service and other organizations (e.g.

, churches, clubs, fundraising groups, activist groups) use them to evaluate the effectiveness of their efforts, businesses use them to learn how to market their products, governments use them to understand community opinions and needs, and politicians and media outlets use surveys to understand their constituencies.

In sum, the following are benefits of survey research:

  1. Cost-effective
  2. Generalizable
  3. Reliable
  4. Versatile

As with all methods of data collection, survey research also comes with a few drawbacks.

First, while one might argue that surveys are flexible in the sense that we can ask any number of questions on any number of topics in them, the fact that the survey researcher is generally stuck with a single instrument for collecting data (the questionnaire), surveys are in many ways rather inflexible.

Let’s say you mail a survey out to 1,000 people and then discover, as responses start coming in, that your phrasing on a particular question seems to be confusing a number of respondents. At this stage, it’s too late for a do-over or to change the question for the respondents who haven’t yet returned their surveys.

When conducting in-depth interviews, on the other hand, a researcher can provide respondents further explanation if they’re confused by a question and can tweak their questions as they learn more about how respondents seem to understand them.

Validity can also be a problem with surveys. Survey questions are standardized; thus it can be difficult to ask anything other than very general questions that a broad range of people will understand.

Because of this, survey results may not be as valid as results obtained using methods of data collection that allow a researcher to more comprehensively examine whatever topic is being studied.

Let’s say, for example, that you want to learn something about voters’ willingness to elect an African American president, as in our opening example in this chapter.

General Social Survey respondents were asked, “If your party nominated an African American for president, would you vote for him if he were qualified for the job?” Respondents were then asked to respond either yes or no to the question. But what if someone’s opinion was more complex than could be answered with a simple yes or no? What if, for example, a person was willing to vote for an African American woman but not an African American man?

In sum, potential drawbacks to survey research include the following:

Surveys and Interviews

When to Use Surveys in Psychological Research

Interviews are a type of qualitative data in which the researcher asks questions to elicit facts or statements from the interviewee. Interviews used for research can take several forms:

  • Informal Interview: A more conversational type of interview, no questions are asked and the interviewee is allowed to talk freely.
  • General interview guide approach: Ensures that the same general areas of information are collected from each interviewee. Provides more focus than the conversational approach, but still allows a degree of freedom and adaptability in getting the information from the interviewee.
  • Standardized, open-ended interview: The same open-ended questions are asked to all interviewees. This approach facilitates faster interviews that can be more easily analyzed and compared.
  • Closed, fixed-response interview (Structured): All interviewees are asked the same questions and asked to choose answers from among the same set of alternatives. 


The survey method of data collection is a type of descriptive research, and is ly the most common of the major methods. Surveys have limited use for studying actual social behavior but are an excellent way to gain an understanding of an individual's attitude toward a matter.

Similar to an interview, a survey may use close-ended questions, open-ended questions, or a combination of the two.

«Closed-ended questions» are questions that limit the person taking the survey to choose from a set of responses.

Multiple choice, check all that apply, and ratings scale questions are all examples of closed-ended questions. «Open-ended questions» are simply questions that allow people to write in their own response.

Surveys are a highly versatile tool in psychology. Although a researcher may choose to only administer a survey to sample of individuals as their entire study, surveys are often used in experimental research as well.

For example, a researcher may assign one group of individuals to an experimental condition in which they are asked to focus on all the negative aspects of their week to induce a negative mood, while he assigns another group of people to a control group in which they read a book chapter. After the mood induction, he has both groups fill out a survey about their current emotions.

In this example, the mood induction condition is the independent (manipulated) variable, while participants' responses on the emotion survey is the dependent (measured) variable.

Advantages of Surveys

The benefits of this method include its low cost and its large sample size. Surveys are an efficient way of collecting information from a large sample and are easy to administer compared with an experiment.

Surveys are also an excellent way to measure a wide variety of unobservable data, such as stated preferences, traits, beliefs, behaviors, and factual information.

It is also relatively simple to use statistical techniques to determine validity, reliability, and statistical significance.

Surveys are flexible in the sense that a wide range of information can be collected. Since surveys are a standardized measure, they are relatively free from several types of errors. Only questions of interest to the researcher are asked, codified, and analyzed. Survey research is also a very affordable option for gathering a large amount of data.

Disadvantages of Surveys

The major issue with this method is its accuracy: since surveys depend on subjects' motivation, honesty, memory, and ability to respond, they are very susceptible to bias.

There can be discrepancies between respondents' stated opinions and their actual opinions that lead to fundamental inaccuracies in the data.

If a participant expects that one answer is more socially acceptable than another, he may be more motivated to report the more acceptable answer than an honest one. 

When designing a survey, a researcher must be wary of the wording, format, and sequencing of the questions, all of which can influence how a participant will respond. In particular, a researcher should be concerned with the reliability of their survey. «Reliability» concerns the degree to which the survey questions are ly to yield consistent results each time.

A survey is said to have high reliability if it produces similar results each time. For example, a reliable measure of emotion is one that measures emotion the same way each time it is used. However, for a survey to be useful, it needs to be not only reliable, but valid.

If a measure is has high «validity», this means that it is in fact measuring the concept it was designed to measure (in this case, emotion). It is important to note that a survey can be reliable, but not valid (and vise versa).

For example, just because our emotion survey is reliable, and provides us with consistent results each time we administer it, does not necessarily mean it is measuring the aspects of emotion we want it to. In this case, our emotion survey is reliable, but not necessarily valid.

Structured surveys, particularly those with closed-ended questions, may have low validity when researching affective variables. Survey samples tend to be self-selected since the the respondents must choose to complete the survey. Surveys are not appropriate for studying complex social phenomena since they do not give a full sense of these processes. 

Key Elements of a Successful Survey or Interview

While survey research is one of the most common types of psychological study, it can be difficult to create a survey that is free of bias and that reliably measures the factors it aims to capture. A researcher must have a strong understanding of the basics before they can create a valid survey from scratch.

Surveys must be carefully worded and include appropriate response formats. The way a question is written can confuse a participant or bias their response, and poorly framed or ambiguous questions will ly result in meaningless responses with very little value.

Questions should be clear, address only one topic at a time, and avoid leading the respondent to a specific answer (in other words, a question should not suggest the correct response in how it is worded).

When designing a survey, it is important to understand your audience and use words they will understand and make sure your survey is not too long for them to easily complete.

While survey research is one of the most common types of psychological study, it can be difficult to create a survey that is free of bias and that reliably measures the factors it aims to capture. A researcher must have a strong understanding of the basics before they can create a valid survey from scratch.

Types of Data Gathered in Surveys and Interviews

Surveys may measure either qualitative or quantitative data.

Qualitative data are the result of categorizing or describing attributes of a population such as hair color, blood type, or ethnic group. Qualitative data are usually described by words or letters.

This type of data does not lend itself to mathematical analysis, but bar graphs and pie charts tend to demonstrate this type of data well.

Quantitative data are always numbers. Quantitative data are the result of counting or measuring attributes of a population, such as money, pulse rate, weight, or populations.

This type of data may be either discrete (meaning they take on only certain numerical values, such as the number of phone calls you receive per day or the number of jeans you own—you might have 2 or 3 pairs of jeans, but you cannot have 2.

5 pairs) or continuous (data that are the result of measurements such as weight, height, or amount of blood donated). Discrete data use whole numbers, while continuous data utilize decimals and fractions.

Источник: http://kolibri.teacherinabox.org.au/modules/en-boundless/www.boundless.com/psychology/textbooks/boundless-psychology-textbook/researching-psychology-2/methods-of-collecting-data-28/surveys-and-interviews-129-12664/index.html

Добавить комментарий

;-) :| :x :twisted: :smile: :shock: :sad: :roll: :razz: :oops: :o :mrgreen: :lol: :idea: :grin: :evil: :cry: :cool: :arrow: :???: :?: :!: