What Makes a Good Survey?
Sampling
Sample, sample, sample. The goal of any good survey is to find out what people think to inform a decision or many future decisions. For that reason, you should plan to talk to a lot of people and a diverse range of people. A healthy sample size is critical to getting usable information. If you do not get many opinions, one person (or a few people) can skew your results.
So what is a healthy sample? Refer to the chart in Survey Monkey’s article on sampling. As a rule of thumb, if your target population is 5,000 or greater, surveying 100 people will give you a margin of error of about 10%. That means any result you have could be as much as 10% higher or lower in any direction (with a high degree of confidence). If you survey 100 incoming college students about their food preferences and 15% of them say they like greek yogurt, your actual range is 5% to 25% given your margin of error.
When it comes to prospective students, most of our audiences are larger than 5,000. There are millions of students who apply to college each year, and even if you narrow to a specific discipline, you are likely still looking at tens of thousands of applicants.
When you want to make a survey, prepare to get a sample that meets your needs. If you want to be reasonably certain and specific with your results, you might need to survey over 100 people. If you want a subgroup opinion (perhaps students from California or Latinx students), you need a larger sample or a plan to survey those individuals specifically.
Survey Questions
Good survey questions are difficult to write. You don’t want to lead the audience in one direction or another, and you want to avoid influencing future answers on the survey. For instance, if you ask someone their opinion on the University of Illinois Urbana-Champaign and then ask what their favorite universities are, it’s more likely that a person is already thinking about UIUC, so it will be overrepresented in your survey on people’s favorite universities.
Think about your primary survey goal. If your primary goal is to assess student opinion on departmental visit experiences, you want to set up your survey in a way that focuses on making sure you get the best answers (without leading or priming your survey participants) to that question above others.
It’s also a good idea to ask the same questions at different moments to measure or track change over time. For instance, the OUA accept/decline survey asks students what was most important to making their admissions choice every year. That way, if the importance of cost in admissions decisions changes from year to year, we can track that change.
There’s an excellent post from Survey Monkey on basic question writing do’s and don’ts that’s worth your time. This covers things like offering balanced response choices, making a diverse question set, and more.
What Surveys Already Exist?
We don’t want to duplicate efforts. While we encourage anyone with questions to get out there and survey, we also don’t want to overask our audiences.
Each year, OUA sends out a large, comprehensive accept/decline survey. The accept/decline survey asks a variety of questions to students who either accepted or declined their offers of admission to the University of Illinois Urbana-Champaign. Students respond to questions about how they found the university, why they either accepted or declined their offer, and much more. This is also our most robust survey, allowing it to be filtered for academic or demographic subgroups while still having a large enough sample to provide relevant data.
OUA also sends a variety of visit program surveys to evaluate the success of various programming types, as well as one-off special topic surveys as needed. We have surveyed on program names, why students love UIUC, online video viewing habits, and other topics.
Questions?
Reach out to us for help and advice in surveying the prospective student audience. Be sure to share any research done with us on the R&Y Network listserv! Survey responses often benefit many units.