You’ll find this blog by Bless Vaidian—and more from the NACE Blog Team—at http://community.naceweb.org/blogs/bless-vaidian/2017/10/09/data-collection-toward-a-100-percent-knowledge-rate
That’s my official meditation for today at the NACE conference. This morning, I attended a session hosted by the NACE First Destination Task Force where we discussed what’s been happening at the association and beyond with our increasingly critical surveys about where our graduates go after they leave our institutions. With national attention being paid to these data and the numbers in the spotlight more often than ever, there’s no doubt this is a hot topic for career services attendees at the conference. Here’s a breakdown of the session and some commentary by one of your faithful bloggers.
NACE has already released a position statement about these First Destinations surveys in July 2012, and we kicked off the session with a review of the principles laid out in this statement. The short version of that is:
- Post graduate success is the mission of entire institution, not just career services
- All graduates of institutions should be tracked in these surveys
- Career services should have central role in collecting this information
- Outcomes should be inclusive, not just about immediate employment
- Human subject & institutional research protocols should be observed when collecting information
- Data may come from various reliable sources
- Data collection should be on-going, with the final collection efforts completed by 6-9 months from graduation
- Data should be reported in aggregate and should protect individual confidentiality
- Outcome data should consider: response rates, academic program breakdown of data, job titles, employers, salary data, further academic study (what program and what institution)
The NACE Task Force is working on a version of a standardized first destination survey which can be used by all institutions. The Task Force’s plan is to have all institutions be using this survey for the graduating class of 2014. So, with that in mind, the Task Force needed to do quite a bit more beyond what has been set forth in the position statement. Namely:
- There would need to be a core set of questions to be asked universally and consistently
- There would need to be establish definitions for standard measures (i.e. defining what “full-time employment” really means)
- There would need to be an agreed upon appropriate time frame for data collection
- There would need to be suggested response rate requirements to ensure that the data reported is statistically valid and reliable
This is all no small order. What about entrepreneurs? What about graduates in the summer, the fall, or schools on different academic calendars? How can we standardize all of this? Questions about the intricacies of this are abundant, and rightfully so.
The Task Force was ready to share a bit about where they are in the process, so here’s what was learned.
New Language for First Destination Surveys
- Perhaps we can lay the “p” word to rest? The suggestion is to call it “career outcomes” rather than “placement.”
- Recognizing that information about post graduate career outcomes comes from various sources (not just our surveys), the suggestion is to consider “knowledge rates” rather than “response rates.” For instance, say a faculty member or employer lets a career services office know a student was hired and reports job title & employer information. That’s knowledge, not a “response.”
- When the data collection period ends, we can “close the books.” Ongoing data collection can and should happen after graduation, and the profession should consider counting early, mid and later in academic year graduates (not just traditional “Spring” grads). However, knowing that spring graduation is the largest for a majority of institutions, we can consider closing the books six months after that date, which is approximately December 1. NACE would consider reaching out for information by the end of December, and then could share aggregate data in January to legislators, those involved in public policy, and those in trends reporting.
Suggestions for type and amount of information to collect
- The Task Force suggested a knowledge rate range between 65% and 85%. This is to serve as an initial guidepost for us, and should help us find a workable range that is achievable, valid, and reliable. Over time as we develop this, the suggested knowledge rate range may increase
- The outcome measures to be provided include information such as (this is not the whole picture here): percentage of graduates employed full-time, those pursuing further study, those still seeking employment, and those not seeking employment. While information should be collected for graduate and undergraduate students, there should also be separate information for the undergraduate and graduate levels as well
- For the employment category, examples of information to collect include: job title, employer, salary (both base salary & guaranteed first year compensation, which includes signing bonuses)
- For the further study category, the name of the academic program and institution name should be collected
- If a student is working and pursuing further study, it is suggested that the data be categorized by the graduate’s primary pursuit.
A few more dimensions the Task Force is considering:
- A way to measure a graduate’s satisfaction with their outcome? Meaning: is this where they wanted to be?
- For those who are reported as being employed full-time, is the employment related to their degree?
- For now, the further study category is intended for those who are pursuing a graduate degree. What about other types of study? Certification programs? Those who want to earn another undergraduate degree?
Suffice it to say, there are still many questions about this process yet to be answered. But, I think I can safely say there is agreement that this is important work which needs doing. It’s a challenge, no doubt. Life doesn’t fit into defined categories easily, and so it follows that neither does one’s career plans. At a time when many want to know, “is college worth it?,” these first destination data points can be key indicators of a piece of the puzzle that is an answer to that question.