The Assessment Diaries: 5 Questions to Ask Before Creating a Survey

Desalina Allen

A post by NACE Guest Blogger, Desalina Allen, Senior Assistant Director at NYU Wasserman Center for Career Development
Twitter: @DesalinaAllen
LinkedIn: www.linkedin.com/in/desalina

In addition to some of the methods I’ve already mentioned surveys can be a great way to collect both quantitative and qualitative information from students, employers and other key career services stakeholders. There are definitely questions you should ask yourself before deciding that a survey is the right collection method, but I’ll save those for another post.

For now, let’s assume you are dead set on surveying and you just don’t want to end up like this guy:

Image courtesy of GifBin.com

Here are five questions to ask yourself before you start designing and distributing your survey:

What information do I absolutely need to collect? Consider whether you already have access to accurate information on students like major, department and graduation date before asking these questions in your survey.  If you do, you can ask for a student ID and match up the two sets of information.  Many of the online survey software platforms also allow you to upload a list of survey recipients and send each one a customized hyperlink so you don’t need to collect name and contact information. When we survey, we rarely ask for school, major or grad date because we often have this information updated via our Career Services Management System and/or registrar records.  Two or three fewer questions, now that’s exciting.

What is your population? When you review your results or write your report, what is the group that you are trying to describe?  Will it be students who attended a resume seminar (more specifically: a resume seminar on December 13 or any resume seminar throughout the year)? Is it all juniors, or only juniors who have completed summer internships?  Having a clear understanding of  your population, will help you answer the next question which is:

How many responses do I need? Depending on your survey method, budget and population size you may not get responses from everyone.  This is OK – statistics allows you to describe your population without having data from everyone. This chart is really helpful – find the approximate size of your population on the far left column and then find the corresponding number of responses necessary to describe that population.  For example if you are trying to describe a population of 25,000 undergraduate students, you may only need between 700 and 10,000 responses – depending on how certain you want assumptions to be.  You should also be sure that there is not a difference in the group that did and did not respond to your survey.  For example, if all of your responses came from people who attended a particular event, your results may be skewed as these people may differ from the total population.  Finally, do some benchmarking and check past reports to get an idea about the response rate that is considered reasonable.  In the example above, a 40 percent response rate (10,000/25,000) may be acceptable for a student satisfaction survey but not for your annual first destination survey.

How will I collect this information?  Websites like SurveyMonkey offer free accounts and many institutions have licenses for software such as Qualtrics (my platform of choice). Of course there is always the old fashioned paper and pencil method, which is still a very effective way to collect information. Career Service professionals may also check to see if their existing Career Services Manager system offers surveying features (Symplicity’s NACElink system offers this as an add-on).

Will multiple methods be required to achieve the desired number of responses? Using one method of surveying may not be enough to achieve your target response rate or get the information you need.  Consider using a combination of paper forms, online surveying, phone surveying, in-person interviews, and even online research. My fellow NACE guest blogger, Kevin Grubb, mentioned that the new NACE position statement on first destination surveys will now use the term “knowledge rate” instead of response rate as we often collect information from faculty, employers, and even LinkedIn research to gather information about our students career outcomes.

What do you think? Add your thoughts in the comments section!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s