Data Collection Toward a 100 Percent Knowledge Rate

BlessVaiBless Vaidian, Director, Career Counseling for Pace Career Services – Westchester, and Founder, Career Transitions Guide
Linkedin: https://www.linkedin.com/in/blessvaidian
Twitter: https://twitter.com/BlessCareers
Blog: http://careertransitionsguide.com

Is a 100 percent knowledge rate possible with a first-destination survey? That’s to be determined each year and with each effort. Due-diligence requires universities to extend maximum effort to try to achieve a 100 percent knowledge rate for all our students. The task of collecting and reporting data is a huge undertaking trusted to many career offices. Whether you are trying to meet the NACE deadline for data collection or your own office deadline, creating a systematic approach and incorporating “best practices” into your labor makes capturing career outcomes more manageable.

Lay the Foundation

Its essential to be able to analyze data with ease, as well as know ahead of time what questions to include in your outreach attempts to students. Follow the suggestions outlined by NACE in your database fields and match it to your first destination surveys. Bring in your school’s technology department to help create the database, as well as the electronic surveys that capture the responses fed into it. Once that’s done, a time line for when, where, and how you will collect data can be drawn out. Cap and Gown surveys, employer surveys, surveys to the campus community, classroom visits, social media searches, follow-up student surveys, calls and e-mails have to be systematically laid out on a timeline. Learn assessment best practices by attending conferences and events to know how others are capturing information. Make sure you use the NACE links on the topic and talk to Ed Koc, NACE’s Director of Research, Public Policy, and Legislative Affairs or his great team if you have questions. Koc is offering a webinar on the first-destination initiative in early January for NACE members. A solid foundation and plan of action will serve you well in the long run.

Designate a Point Person

If the college community knows that career outcome information has to be sent to a designated individual within their school, then more outcomes can be captured. Often university staff members possess career outcome information and never pass it onto career services. The human resources and admissions departments within your school may have first-destination information on numerous students who were hired or went onto graduate school at your institution. The designated point person should monitor the first destination survey numbers, solicit information from university sources consistently, and create a strategy for follow-up with graduates. It takes many people, numerous efforts, and even call-centers to capture data for bigger schools. But designate an expert to manage the whole process, set the timeline, and be the “face” of the initiative in order to drive the results.

It’s Not a Career Services Issue, It’s a University Issue

Helping students find opportunities and creating a path for successful outcomes is not just a career services goal. Higher education is a partnership of many units working collaboratively to ensure retention and capture every student’s career outcome. Long before first-destination surveys go out, building relationships with the campus community is where data collection really starts for career services. Meetings with the university community to build bridges, foster relationships, and outline the process is crucial. Students share career outcome information with professors, academic advisers, financial aid representatives, leaders of student organizations, and college staff. These sources become vital in the collection process and have to be included in the journey.

Keep the Community Vested

It is essential to make survey efforts and progress visible to the campus community. Every dean, faculty member, and university staff  member should know what the career office does. Career outcome and knowledge rate information should be displayed in infographics, charts, and reports on a regular basis with college partners. If others understand what goes on behind the scenes and where the numbers are, then they will be more apt to assist with first-destination information. It also keeps departments interested and looking forward to the next update.

Mandate Attendance 

Universities that promote, encourage, or even mandate attendance at career service events and one-to-one meetings with a career counselor can create more successful outcomes. Students that have worked with career offices feel more comfortable sharing career outcomes, and should be told that post-graduate follow-up will take place after graduation. Career services also helps students find pre-professional experience through internships that build resumes and lead to full-time offers. They offer networking opportunities with employers and alumni that have job leads every semester. Increased student engagement with career centers increases the “knowledge rate,” and also increases “outcomes.” Its a simple formula.

Multiple out-reach efforts to capture information throughout the year are made to graduating seniors, college partners, and employers to track career outcomes. I would love to hear your school’s best practices and ideas to reach that “100 percent knowledge rate.” Wishing each of you success in reaching your university’s goal and capturing outcomes. 

The Assessment Diaries: Beyond Satisfaction Follow-Up

Desalina Allen

Desalina Allen, Senior Assistant Director at NYU Wasserman Center for Career Development
Twitter: @DesalinaAllen
LinkedIn: www.linkedin.com/in/desalina

The results are in!  I recently shared assessment plans for our Dining for Success etiquette dinner.  We moved away from a satisfaction survey to a pre-dinner and post-dinner skills assessment for the first time and, as I shared in my previous post, I was a little nervous about the results.  Here is what we found:

Section One:  Understanding Formal Place Settings

Let’s face it.  We could all use a refresher on how not to steal your future boss’ bread plate, and our students were no different.  Before and after the event they were asked to identify each plate, cup, and piece of silverware in this photo:

Then, at the beginning of the event we had all utensils, plates, and glasses piled in the center of the table and asked each student to organize their place setting. We noticed a bit of uncertainty during the activity and our employer volunteers stepped in often to help, which tells us that students were not completely clear about formal place settings.

This experience conflicts with what we found via the assessment. We didn’t see much of a difference between pre and post results. In fact, most students correctly identified the items (with #6 dinner fork and #5 salad fork being confused just a few times).  We did see a slight drop in the number of blank responses, which could be interpreted to mean that students felt more certain about formal place settings after the event.

Section Two:  Appropriate vs. Inappropriate Table Topics

Students were asked to list three appropriate topics to discuss at mealtime interviews or networking event as well as three topics to avoid.  During the event, we provided employer volunteers with a list of suggestions and encouraged them to supplement based on their experience.

On the pre and post surveys, students were instructed to leave questions blank if they did not know the answer. Comparing responses revealed a significant increase in the number of students who answered these questions after the event.  We also noticed that a wider variety of more detailed topics were listed in the post surveys.  For example, students most often listed “career,” “food,” and “hobbies” in the pre-dinner survey, while post-dinner survey responses included things like “the professional’s background,” “the industry,” “new projects,” and “current events.”

Section Three: Ordering Food

While guests were only offered two entrèe options, employer volunteers were encouraged to share basic guidelines regarding how and what to order during sit-down dinners or interviews.  Almost all of the pre survey responses revolved around not ordering food that is too messy or difficult to eat.  Post survey results again provided more breadth and detail.  Student mentioned avoiding “smelly” food, considering price, and following the lead of the interviewer/host.  One student even suggested not ordering meat if your host is a vegetarian…discuss!

Section Four: Following Up

How should students follow up with an individual after a networking event or meal time interview?  Turns out, most students already understood the basics (insert career counselor sigh of relief here).  On the pre-event survey, many students responded that you should send a follow up thank you via e-mail (or in some cases, USPS), however after the event students included details like “within 24-48 hours” and mentioned LinkedIn for the first time.  

What we learned

Overall, we were happy with the improvements we saw between the pre and post-event surveys.  And, of course, we found that 97 percent of students were satisfied with the event!  Here are a few key takeaways and thoughts regarding the survey for next year’s event:

  • The table setting question may not have accurately measured students’ level of comfort with formal dining before and after the event.  The way the image was laid out may have been too simple.  For future surveys, we are considering having students draw a diagram or place items around a plate to more accurately reflect our table setting activity.

  • Students understand the basics regarding discussion topics, ordering, and following up after events, but the activities and discussions gave them a more broad and anecdotal understanding of how to navigate during mealtime events and interviews.

  • We will consider measuring different skills/content areas each year.  Our event also included activities revolving around introducing yourself and handling sticky situations that were not assessed in the pre- or post-event surveys.  It would be interesting to see how students’ understanding of these topics changed as a result of the event.

Fixated on “First Destinations”

kevin grubbA post by NACE Guest Blogger, Kevin Grubb.
Assistant Director at Villanova University’s Career Center.
Twitter: @kevincgrubb
LinkedIn: http://www.linkedin.com/in/kevingrubb
Blog: “social @ edu”.

 That’s my official meditation for today at the NACE conference.  This morning, I attended a session hosted by the NACE First Destination Task Force where we discussed what’s been happening at the association and beyond with our increasingly critical surveys about where our graduates go after they leave our institutions.  With national attention being paid to these data and the numbers in the spotlight more often than ever, there’s no doubt this is a hot topic for career services attendees at the conference.  Here’s a breakdown of the session and some commentary by one of your faithful bloggers.

NACE has already released a position statement about these First Destinations surveys in July 2012, and we kicked off the session with a review of the principles laid out in this statement.  The short version of that is:

  • Post graduate success is the mission of entire institution, not just career services
  • All graduates of institutions should be tracked in these surveys
  • Career services should have central role in collecting this information
  • Outcomes should be inclusive, not just about immediate employment
  • Human subject & institutional research protocols should be observed when collecting information
  • Data may come from various reliable sources
  • Data collection should be on-going, with the final collection efforts completed by 6-9 months from graduation
  • Data should be reported in aggregate and should protect individual confidentiality
  • Outcome data should consider: response rates, academic program breakdown of data, job titles, employers, salary data, further academic study (what program and what institution)

The NACE Task Force is working on a version of a standardized first destination survey which can be used by all institutions.  The Task Force’s plan is to have all institutions be using this survey for the graduating class of 2014.  So, with that in mind, the Task Force needed to do quite a bit more beyond what has been set forth in the position statement.  Namely:

  • There would need to be a core set of questions to be asked universally and consistently
  • There would need to be establish definitions for standard measures (i.e. defining what “full-time employment” really means)
  • There would need to be an agreed upon appropriate time frame for data collection
  • There would need to be suggested response rate requirements to ensure that the data reported is statistically valid and reliable

This is all no small order.  What about entrepreneurs?  What about graduates in the summer, the fall, or schools on different academic calendars?  How can we standardize all of this?  Questions about the intricacies of this are abundant, and rightfully so.

The Task Force was ready to share a bit about where they are in the process, so here’s what was learned.

New Language for First Destination Surveys

  • Perhaps we can lay the “p” word to rest?  The suggestion is to call it “career outcomes” rather than “placement.”
  • Recognizing that information about post graduate career outcomes comes from various sources (not just our surveys), the suggestion is to consider “knowledge rates” rather than “response rates.”  For instance, say a faculty member or employer lets a career services office know a student was hired and reports job title & employer information.  That’s knowledge, not a “response.”
  • When the data collection period ends, we can “close the books.”  Ongoing data collection can and should happen after graduation, and the profession should consider counting early, mid and later in academic year graduates (not just traditional “Spring” grads).  However, knowing that spring graduation is the largest for a majority of institutions, we can consider closing the books six months after that date, which is approximately December 1.  NACE would consider reaching out for information by the end of December, and then could share aggregate data in January to legislators, those involved in public policy, and those in trends reporting.

Suggestions for type and amount of information to collect

  • The Task Force suggested a knowledge rate range between 65% and 85%.  This is to serve as an initial guidepost for us, and should help us find a workable range that is achievable, valid, and reliable.  Over time as we develop this, the suggested knowledge rate range may increase
  • The outcome measures to be provided include information such as (this is not the whole picture here): percentage of graduates employed full-time, those pursuing further study, those still seeking employment, and those not seeking employment.  While information should be collected for graduate and undergraduate students, there should also be separate information for the undergraduate and graduate levels as well
  • For the employment category, examples of information to collect include: job title, employer, salary (both base salary & guaranteed first year compensation, which includes signing bonuses)
  • For the further study category, the name of the academic program and institution name should be collected
  • If a student is working and pursuing further study, it is suggested that the data be categorized by the graduate’s primary pursuit.

A few more dimensions the Task Force is considering:

  • A way to measure a graduate’s satisfaction with their outcome?  Meaning: is this where they wanted to be?
  • For those who are reported as being employed full-time, is the employment related to their degree?
  • For now, the further study category is intended for those who are pursuing a graduate degree.  What about other types of study?  Certification programs?  Those who want to earn another undergraduate degree?

Suffice it to say, there are still many questions about this process yet to be answered.  But, I think I can safely say there is agreement that this is important work which needs doing.  It’s a challenge, no doubt.  Life doesn’t fit into defined categories easily, and so it follows that neither does one’s career plans.  At a time when many want to know, “is college worth it?,” these first destination data points can be key indicators of a piece of the puzzle that is an answer to that question.