Kevin Gin:
And there’s a paper that came out a good number of years ago actually. And the name of the paper is weigh pig, feed, pig, weigh pig and weigh. W E I G H not w a Y. And the whole premise of that paper is that a pig never fattens because you weigh it, right? Because you put something on a scale doesn’t mean that it gains weight or loses weight, right? Just in the same way that in student affairs or an academic affairs or an institutional improvement, just because you do an assessment doesn’t mean anything improves, right? It’s all about the feed. It’s all about the action. It’s all about closing the loop, does what you do actually come back and have an impact or not.

Glenn DeGuzman:
Hello, welcome to Student Affairs NOW. I’m your host, Dr. Glenn DeGuzman. You know, I thought it would be fun to talk with three student affairs professionals who just love assessment, and who’ve been very successful in what they do. And I am just very fortunate, lucky to have three people who I have known at different parts of my 25 year career in student affairs, who are in my opinion, subject matter experts and assessment. So today we are going to explore what they think about the role of assessment and what it plays in our field of student affairs and just gain tips, advice, insight for student affairs professionals who are getting into it, just kind a lay of the land. So Student Affairs NOW, as you all know, is the premier podcast and online learning community for thousands of us who work in alongside or adjacent to the field of higher education and student affairs, we release new episodes every week on Wednesdays.

Glenn DeGuzman:
Find out details about this episode or browser archives at studentaffairsnow.com. Again, we want to make sure that we send gratitude to our sponsors. And this episode today is sponsored by Simplicity, a true partner, Simplicity, support, all aspects of student life with technology platforms that empower institutions to make data driven decisions. This episode does also sponsored by Vector Solutions, formally EverFi for you old for you, some of you old school folks, the trusted partner for 2000 plus colleges and universities, Vector Solutions is the standard care for student safety, wellbeing and inclusion. Again, my name is Glenn DeGuzman. I’m the associate Dean of students and director of residential life at University of California, Berkeley. I usually him his pronouns. I’m recording this episode from Livermore, California, which is the ancestral home of the unseen territory of the Aloni peoples. So let’s meet our panelists. I’d like to welcome to the show Dr. Jaqui Thomas, Go Bears. Dr. Kevin Gin, Go Pioneers and Dr. Jason Simon, Go Eagles. And they’ll probably do a quick intro with who they are, but welcome to Student Affairs NOW, podcast. So let’s get the audience to know all of you a little bit better. So the question that we always kick off our podcast is you could into introduce yourself and tell the audience a little bit more about you. All, you know, you have all a deep professional track record in assessment. Just give an overview of who you are, and how you got into assessment. Let’s start with the, the eldest of the bunch. Jason, how are you?

Jason Simon:
Wow. Now I know I’ve been in higher ed way too long. I’m great. Thanks for the opportunity to be here today, to share a perspective and you know, it’s so fantastic to just be able to have the space, to be able to talk about big issues affecting the field of student affairs and higher education. In general, I feel like you know, the role of higher education today is more important than ever. And so love, love the opportunity to be here. I actually have 29 years of experience in higher education. I’ve I started out originally, it’s a student activities advisor at good old Wichita State University home of the Shockers, nothing inspires fear, like an angry little bundle of wheat for my soccer fans online. And then through various career paths found my way to the University of California, Berkeley, where I met yourself and, and Kevin at the time working in the alumni association in advancement and, and advancement was really where sort of, I started to recognize the impact of data on decision making.

Jason Simon:
It’s very quantifiable as a field. And so when I knew I wanted to pursue my, my doctorate in higher ed here at the University of north Texas I reached out to, the vice president of student affairs at the University of North Texas and said, Hey, I’m coming with a lot of data experience. Can you help find me a grad assistantship in assessment. And that’s where I got my start here at the University of North Texas, an assessment. And from there finished my coursework completed the dissertation became affiliate faculty member, and then went on to become director of assessment at the institution, did that for a number of years. And that’s where my career then took the turn from student affairs. I was plucked into institutional research and effectiveness after an accreditation opportunity. And in my current role now I’m associate vice president for data analytics and institutional research. And I report directly to our VP of planning and chief of staff to the president and my team, and I help to leverage data, to inform decision making. And we certainly do use assessment, traditional assessment methodology, but we also increasingly are using analytic predictive analytics, machine learning and data science.

Glenn DeGuzman:
Thanks, Jason, welcome to the show. I’m realizing this is gonna have a thread. There is a a UC Berkeley thread. That’s gonna kind of, weave me through this introduction Kevin, a little bit about yourself.

Kevin Gin:
Yeah. Thanks Glenn. For the opportunity to, to be here today with you also, hello everyone, Kevin, Gin, him, he, him his pronouns. I’m currently the senior strategic partner for institutional initiatives and assessment at Cal State East Bay. I got my start off as an undergraduate student at UC Berkeley where I met Glen and then I eventually met Jason at the alumni association. And so that started my delve into student affairs. I went to Colorado State University of the SAHE program for my masters in higher education. And then I got my doctorate in higher education at Boston College as well. My introduction to assessment really actually was through Glen. I’m gonna put on the spot here. And I remember the very first time that we were talking about graduate programs and I was thinking about you know, core competencies and really some of the skills that you want to gain.

Kevin Gin:
One of the things that Glenn pointed me to was the CAS standards. And for any of you all out there who don’t know what the CAS standards are, I highly recommend you take a look at it. We’re gonna probably talk a little bit about cast standards in our conversation today as well. But that was the first time I actually thought about what is assessment, what are standards? What are ways that we could take a look at our effectiveness? As I worked in student affairs, I worked in student activities, student leadership, residence life and housing. I eventually moved up into institutional effectiveness and research and do an assessment, an accreditation at the institution level. I’ve coordinated two self-study reaff affirms for institutions now. And I currently serve on the subcommittee task force as well here in California. So I’ll bring a perspective. That’s about student affairs. That’s about academic affairs and teaching and a high level about institutional accreditation and really looking forward to sharing the time here with Jaqui and Jason as well.

Glenn DeGuzman:
Thanks, Kevin. And thanks for that. Shout out, I apparently now owe you a beverage of your choice when you’re in town. So I see, I see where you’re going earlier, this podcast, and then Jaqui. Hi, Jaqui. Welcome to the show.

Jaqui Thomas:
Hey, so hi everyone. Thanks again, Glenn, for the invite. I’m Jaqui Thomas, I use she, her pronouns. I’m currently the assistant director of assessment and academic success in the residential life department at UC Berkeley. I I originally started my college career at UC Berkeley. So I did undergrad at UC Berkeley. Stayed in the bay area for grad school, got a degree in counseling, and then realized that school counseling was not gonna be the way to go during the financial crisis. So I went and got a doctorate in higher education and started working in higher education. And that the doctoral program is exactly where assessment started for me. So I was pretty much committed to doing a quantitative dissertation because of a comment that a recent grad who was volunteering and came back to an orientation for new grad students in the doc program at Cal Lutheran University.

Jaqui Thomas:
And he basically said, if you want to finish your dissertation with the least amount of pain possible, you can front load the work and do your quantitative, your dissertation using quantitative methods. And that was all I needed to hear. I was committed at that point and I was ready to struggle a little bit with the quant class. I think like a lot people, the numbers did scare me just a little bit, cuz it had just been so long since I had taken math or done statistics or anything. But I was ready to go through it and I did not expect to fall in love with that IBM SPSS manual. And I know I probably sound like a big nerd, but that is exactly what happened. For people who don’t know what SPSS is, it’s a statistical analysis software.

Jaqui Thomas:
And when I was reading it, it just made perfect sense. Everything was explained really well and just being able to understand and interpret numbers and to be able to understand how to make recommendations off of those numbers made me feel really powerful as a student affairs professional. So I, I loved it and didn’t didn’t really want to stop using it. So when I eventually moved into student housing, I’ve been in student housing for about eight years now. I was in a position that didn’t require a doctorate that did not require me to do assessment. But I did feel like I had those skills and I didn’t want them to get old or dry or stale. So I asked to attend an assessment conference. It’s the assessment Institute that ACPA puts on. And once I got back, I basically took over departmental assessment and the director of the department I was in at the time was happy to hand it over.

Jaqui Thomas:
I put it together a presentation for how I was approaching assessment at the departmental level and presented it to others in the student affairs division at the campus I was at at the time. And then I saw a position at UC Berkeley, which is the position I am currently holding now where I can basically do everything in departmental assessment. I absolutely love this job. And the flexibility that it is me outside of this particular position, I am doing my own research and I presented at two conferences two assessment conferences for both national and international audiences. And I also am an emerging scholar practitioner reviewer with the journal of student affairs inquiry, which is the student affairs assessment journal. So that’s me.

Glenn DeGuzman:
I’m so glad you love your job. It must be your supervisor. I hear your supervisors really cool. No, but I, you know, it’s funny I can do a quick, quick sort of like theme I’m theme have a theme here, I just running across this group, so have an affiliation to use a and you somehow end up with an assessment perspective and that includes myself. So shout out to UC Berkeley anyways, let’s let’s thank you for the formal introduction. Let’s get the audience to get to know you a little bit, you know, informally a little bit, sort of like a lightning round. Let’s be quick with this one, but I’d like to know for each of you who was integral in your journey to get into assessment. Now it could be a person maybe there was a, I mean you referenced a book earlier, Jaqui, but maybe a another book or was there a conference speaker who just captured you and just said, I’m gonna go down this pathway. Does anyone want to kick it up?

Jason Simon:
Yeah. I, you know, I mentioned earlier that I worked with Witchita State for a few years and what a lot of people may not know is that’s the last institution that John Shu worked at while he was writing his, his seminal book assessment in student affairs. And so for me to be able to get the opportunity to be in the same division as him, I mean, he certainly planted a, a strong foundation for, for taking that book and applying it in some new ways. And so for me, definitely the, the ability to be with someone who I considered a thought leader and assessment at the time,

Glenn DeGuzman:
For sure John Shu. Wow.

Kevin Gin:
Yeah, I’ll hop in here too. So one of the things, and this is maybe cheating the question, but when I talk to student affairs or when I kind of do an introduction to assessment I always like to give people resources. I always like to give people a frame of reference or something to look up ahead of time. So the national Institute of learning outcomes, assessment, release occasional papers every once in a while. And there’s a paper that came out a good number of years ago actually. And the name of the paper is weigh pig, feed, pig, weigh pig and weigh. W E I G H not w a Y. And the whole premise of that paper is that a pig never fattens because you weigh it, right? Because you put something on a scale doesn’t mean that it gains weight or loses weight, right?

Kevin Gin:
Just in the same way that in student affairs or an academic affairs or an institutional improvement, just because you do an assessment doesn’t mean anything improves, right? It’s all about the feed. It’s all about the action. It’s all about closing the loop, does what you do actually come back and have an impact or not. So knowing that the audience, sometimes, you know, when we talk about assessment, especially in student affairs, we sometimes think about, well, I don’t like numbers. I don’t know how to do statistical analyses. I really don’t know how to have access to data, but conceptually, I think we all know this, right? Like student success doesn’t improve just because you did a survey or looked at analysis student success happens because you can make sense out of things and ask proper questions and close the loop. So I’ll just use that as a way that I kind of tease out some of the introductions to assessment, but that for me was one of the papers when I read it. And that’s from James Madison University as well. So shout up to anyone at the JMU campus over there, good job on that paper and having had huge impact in the ways that I frame assessment.

Glenn DeGuzman:
Wonderful.

Jaqui Thomas:
So I would say that there were two faculty members at California Lutheran university that were really integral to my development and my just interest in being an assessment. First was Dr. Dennis Sheridan, who I believe now is at Pacific University. And the other is Terese who came in at the end of my academic career there. But I was trying to fit my doctorate at that point, when she came in, she was the most knowledgeable quantitative person on the campus or in the program at that time or in the faculty at that time. And while I saw a lot of my classmates getting the opportunity to hire people, to do their stats for them, she figured out that I could do the stats myself pretty much just told me, like, I’m not gonna let you do it. I just want to see how far you get. And I was able to do it. I was able to complete all of my stats on my own for my dissertation. So just, I think that experience was really integral for me.

Glenn DeGuzman:
Wonderful. Thanks for sharing all that. Now let’s start diving into some of this topic of just being data driven, right? So Kevin, I’m gonna direct this question for you to start and I can Jason, please add, we always say that it’s essential to make data informed decisions and it’s really the best way to support and determine how to support our students. And I sometimes wonder if that really is the case, because there seems to be a lot of other factors and variables. Maybe you could say this is all data, but, you know, but that go into decision making, whether it’s a budget challenge, or maybe the, the current campus climate, maybe bandwidth of staff, maybe just how decision makers lead through the philosophy. Kevin, I’m gonna ask you, what have you observed over your career that is true to you and maybe what makes this not always the case.

Kevin Gin:
Yeah. So it’s interesting. You mentioned that Glenn, cuz it is it really the case that we always lead with ever evidence? No, it’s not, but should it be a absolutely, but and why should it be? I think one of the things that when we are in student affairs that we don’t realize is that we had first asked the question, why do assessment to begin with right? Like who is there anyone that forces us to do assessment? What do we use it for? Why is there such a big, why is there a big priority on finding evidence and measuring effectiveness? Well, if you really want to know, it starts at the department of ed and it goes down to the regional accreditors and it starts with the standards of institutional effectiveness. And how do you ensure that you have a high quality education for the students that you provide?

Kevin Gin:
If you look at any of the standards in any of the regional accreditors, there’s actually something in there that says your resource allocation should be rooted in some sort of data, right? Like you can’t just arbitrarily put money into places or you shouldn’t be, it’s not a practice. So I think that’s where I start to begin with is that, you know, individuals who say that if I’m gonna have a leadership style, that’s gonna be rooted in some sort of bias or here’s what an agenda that I have. And I want to, you know, really invest in those resources, but not others. Then you’re making a mistake. Even if you listen to the Michelle Cooper who currently works in the department of ed right now, the deputy under secretary for education, she has talked a lot about accountability for funding, the coronavirus relief package, the millions of dollars that were given to institutions she’s asking.

Kevin Gin:
So how did you know that money was effective? How did you allocate it in a way where, you know, that it actually had an impact on the students that you served? And if you can’t actually give a rationale for that? Well, that’s an issue and a regional creditors are now asking. So with the money and the budget that you have with all the financial restraints that institutions are having, how do you make these decisions? So it’s not even a question of, here’s a philosophy that I have, but higher education is increasingly coming under scrutiny over how do you allocate your resources? What decisions that you make. And it all goes back into the assessments that you have using things like the CAS standards, which standardize everything for student affairs, it’s laid out for there for you. It’s a best practice. It’s something you see in every graduate program.

Kevin Gin:
I’m sure we’ll talk a little bit about that later on as well, but gone are the days that you can frivolously spend and gone are the days you can hide behind the fact. Well, it’s just, I have a feeling this works. I’m sure Jaqui and Jason will talk a lot about, you know, down the analytical space on the statistical space. What do we know? That’s factually true. That’s grounded in something that will help student success, help advance equity, help us meet meaningful measures and retention, graduation, reducing academic probation. There’s a whole a number of metrics we could take a look at, but I’ll just stop there to see if Jason or Jaqui want to chime in as well.

Glenn DeGuzman:
And I’m sure they do. Jaqui, why don’t you go and add?

Jaqui Thomas:
Yeah. So I do see that decisions are not always made based on data and in some cases it’s not necessarily a bad thing. So I, I think that a good leader has a three to five year plan or vision for their area. And maybe it’s a different amount of years. Maybe it’s only one or two, maybe it’s 10, right. But good leader is going to have a plan for a number of years. And they’re gonna make a lot of decisions based on what aligns with that vision was what doesn’t align with that vision. So if you present someone who is in a leadership position with data that would steer actions, decisions, funding, resources in another direction, one, it needs to be real substantive if it’s going to be something that derails the vision overall. But I will say that those visions are only as good as the data that they’re founded on. So I think that as long as the vision is founded in some substantive data, that it’s, whether you take on the advisement of future sets of data is, is a up to, is up to the leader and all of the other things that they have to think about when they’re making decisions.

Jason Simon:
No, I think it’s as much as it is about data informed decision making. I also think it’s about a culture where data is valued, appreciated, and respected. And as someone who has had to develop, triangulated studies for executives to make multimillion dollar policy and program decisions you know, to me, it’s really the question of, is it really the leader not choosing to use the data, or is it the folks who furnish the leader with the data, not doing a thorough enough job to be able to present them with something that they can trust. And I think that’s where this is often time viewed as a disconnect in the field of student affairs assessment, where, you know, why aren’t they using the data that I’m producing? Well, maybe it’s just what we’re producing. Isn’t really meeting the need now strategically.

Jason Simon:
And so I think, you know, if we think about data and assessment as a continuum, you know, reporting assessment, indirect evidence, it happened in the past. You can’t fix it. It’s already too late versus more mature forms of utilizing data and assessment to get real time to Kevin’s point, academic alerting predicting who’s gonna be at risk or who’s not involved being able to look at real time ID cards, swipe data across a division of student affairs and identify within the first three weeks, which residents have never left their room and are not engaging anywhere on campus except in their classes. I mean, it really requires I think a fundamental, potentially new perspective on assessment in student affairs. And I loved Kevin Love Kevin’s accreditation, cuz he and I are cut from the same cloth on that as someone who also had accreditation responsibility it’s no longer a privilege.

Jason Simon:
It it’s an expectation. And I think we as a field is we’re really bound to the need to do a better job on the data side. The last thing I’ll say around this topic is I, I fundamentally don’t understand two kinds of, of, of practitioners and apologies if this is, if this is, you know a bit off, off topic, but one is someone who just asks for assessment for the sake of assessment. Hey, can you run some data for me without being able to answer the question that I pose back? What are you gonna do with it? If you don’t have the answer of what you’re gonna do with it, then no, we’re not gonna engage in the assessment. And then the second group of folks would be those that assume that the students they serve today in their student affairs capacity are exactly the same as the kind of student they serve with the same program they took out of the box five years ago, this notion of not keeping up with the changing student demographics and for us as an institution with 42,000 students now in a newly minted, Hispanic serving institution and also an R1 research institution, the students that we’re serving today and the data that we’re generating today are fundamentally different than the students and data that we do even just three years ago.

Glenn DeGuzman:
Jason, I’m gonna stay with you. And then we’ll see if Kevin and Jaqui, you want to add, but, and I want to be I want to kind of build off some of the things that you’re alluding to. Now. I know all, all of you, I’ve known you, I’ve known you all. So I know that you’ve all done department assessment. I also know that all of you have been of division or campus level involvement, institutional effectiveness. So what are some of the observations that you see when working with other student affairs professionals that do these type of assessments and more specifically, what do you think are some of the challenges or obstacles that campuses have?

Jason Simon:
I think, well, without a doubt, first of all, it’s really resources. You know, typically I think with, I don’t understand why as data tools, systems and applications become more and more advanced and the skill sets needed to be able to interpret and utilize those tools become more advanced even as the vendors proliferate in the space. Why most divisions of student affairs are still satisfied with one FTE and maybe a grad. And so, especially for really large diverse institutions of higher education with multiple departments. My first, my first observation would be, you know, to ask the vice president, you know, what is your level of financial commitment and how do we translate and change the view of assessment as a cost center into the view of an investment center where we can start to generate new returns, obviously given the economic value of retaining one single student, if you have a student affairs assessment professional, that’s partnering broadly with colleagues across campus, can demonstrate the true value of student affairs in that equation of helping to engage students to, to help them stay.

Jason Simon:
How do you put a price on that? I mean, I could put a price on it because we’ve done that study here at UNT, but, but certainly I think it’s important to note. The other thing that I would say is an observation is that a lot of assessments today require a village and it the, it’s my favorite favorite African proverb. And I wish I knew the exact attribution of it, but I, I don’t which is said, if you want to run fast, run alone, if you want to run far run as a group. And so to me, I feel like that should take the place as well. So if we’re gonna launch a national survey of student engagement as a campuswide assessment, that’s gotta be across partners from all of the different divisions. There needs to be buy in for that assessment, if we’re gonna do.

Jason Simon:
And I’m just throwing out some examples, if we’re gonna do the EDUcause ECAR survey on student and their use of technology, we definitely want to make sure that we have a partnership across the entire perspective. Now can student affairs lead in that at space? Oh yeah. On our campus, they definitely do. I’ve got an amazing director of student affairs assessment, Dr. Sarah Ube. She’s very involved on, on the, the student affairs assessment list serve. And so, you know, there’s, there’s a strong connection point now between the division of student affairs and, and the work that we do in data and analytics, institutional research, it doesn’t hurt that I used to do her job and totally get what she’s trying to do. The last thing that I would say about department-wide or institutional-wide assessments is don’t be afraid to really ask yourself what’s not being measured on campus right now. The, our quick knee-jerk response is always to try to invest in a nationally norm survey. Why? Because it gives us comparable benchmarks to other institutions. But what if the unique situation that your campus is facing, doesn’t have to necessarily require a normed data set. Why not innovate partner develop something locally to be able to help your campus. And I think we were gonna have a question later on when we start talking about equity and diversity, I’ll give you a really good example of that.

Glenn DeGuzman:
That’s that’s wonderful.

Kevin Gin:
Would you mind if I jump in your Glenn? So I think I kind of played off what Jason was talking about of, you know, it really kind of takes a village and it takes an institution to make assessment work. A as I think about student affairs and I think about my own experiences and I’m gonna own some of this as well when I entered the field and working with student affairs today, some of the observations that I have is that student affairs still really doesn’t understand assessment and there have been dozens of papers. I mean, you go to any peer reviewed journal out. And they’ll talk about how student affairs still is yet to understand assessment. And it’s, it’s true when you start thinking about many of you who are listening to this podcast, I wonder how many of you know, how to calculate a institution’s retention rate?

Kevin Gin:
Cause I always get that question. What’s our retention rate, what’s our graduation rate. And then I’ll ask, well, what do you want? You want the four year of the six year? Do you want your freshman? Or do you want your transfer? What, which ones you want? Do you want this aggregated by PE eligibility first generation? Do you want it by, you know what equity measures are there? What gaps are there? And they’ll say, well, no, I just want to know how many students we’re retaining and I’ll ask, but what does that mean? So I think we need to fundamentally go back to our graduate programs and to our training and to our mentorship and to the ways that we supervise the ways that we understand and can admit maybe I don’t underst and how these reports are or how these measures are calculated. If do you know what iPads is?

Kevin Gin:
I always ask, you know, do you ever go to and take a look? And they look at me like what’s. And it’s like, okay, well, let’s talk about what is. And then we get into the instrumental role. So Jason, I’m gonna go off of what you said. You said NESSE and then this, this prompt is something in my mind. How many of you all who do NESSE on your campuses actually know the question that are asked in NESSE? Because it’s usually my experience has been, we do the N survey and the student affairs division says, let’s do the survey. And then they’ll say, well, what did you find? And it’s like, well, what, what, what questions were you interested in? What were the themes? Or we’ll do the Bessie. I’m sure most campuses out there right now do the NESSE, the beginning college student survey.

Kevin Gin:
And so we have a profile of what the students mindsets are coming into the institution yet. How many of you know, what questions are there? What about the national clearing house, right. Do we know where students go when they depart the institution? We can answer that question as well. So I think the first thing that you need to think about when it comes to student affairs is even, yes, we’re limited by our resources. Absolutely. But if you have resources and don’t have the, the basic data literacy to understand, you know, how to actually formulate questions and understand what assessment is, and I’ll throw another one out there, what’s the difference between direct and indirect evidence? I think that’s a key one as well. If we were to say out there, you know, I’ll give everyone, you know, five minutes to brainstorm everything that direct and everything that’s indirect. Could we actually have a balanced list or not? So these are the conversations that I like to have with student affairs. And I really encourage emerging student affairs professionals and directors, and mentors and people working in assessment to really ask the questions, because it’s gonna be important for you to actually have that grasp of the competencies of assessment before you even move into doing it.

Jason Simon:
And Kevin, I would say also causation and correlation.

Kevin Gin:
Yep.

Jason Simon:
A lot of times in student affairs assessment, we, we say, oh, the student that used service X, their retention rate is 87%. The student didn’t use service X, their retention rate is 80%. Well, how do, how do we know? And so in traditional assessment, that becomes very difficult. When you get into the space of where we’re working, where we’re building artificial intelligence models, and we’re actually literally looking into 8 million student affairs records, and I can actually tell you the impact on our, on our, on our data science area, under the curve for, for, for interpretation, that’s a whole different story of where we’re able to say, no, this is the impact on retention. So I, I agree with you, da data literacy is fundamental, but right now, unfortunately, and we’ll probably talk about this later on the grad prep question right now, we’re, mostly focused on methodology. We’re not focused on, does the methodology mean, I’m sorry, Jaqui, you probably have something to say too on this one.

Jaqui Thomas:
Yeah. So I wanted to go a little bit deeper into like the effects of the under resourcing of student affairs assessment or just, right. So, so, so you have your, your student, well, you have your student affairs assessment folks, right? And we are out there and we’re trying to assess as much as we can and give people the best data we can. And then you have your student affairs professionals who are not assessment folks, but are probably interested in assessment cuz they know it’s important and they want to use it. Right. So I think when you start thinking about a division, a level or an institutional level, even under resourcing assessment means that the people who are not student affairs assessment professionals are not necessarily given the time to learn, to do assessment. Well, which is basically what you said. And then they, you know, they keep advancing in their careers and then you have senior professionals who don’t know how to make data driven decisions.

Jaqui Thomas:
And then also when you have your student affairs assessment professionals that are trying to get people excited about assessment, and you’re trying to build your culture of assessment and you’re trying to build competencies around assessment. Sometimes you just have to go in and ask for data because you’re are trying to build the data. Data sets that people are gonna need to be able to make successful decisions in the future. And they don’t understand. And because they don’t understand and we haven’t taken the time to really build competencies around assessment. When you ask for that data, you don’t always get quality data or you, they wait till the last minute, give you the information and it’s not because, you know, they didn’t want to do it. Everyone’s busy on college campuses, but that affects data, this well, that that affects the data that we’re able to collect and the decisions we’re able to make around decisions around data in the future as well.

Jason Simon:
You know, Jaqui just caused me to think of something. You know, I, I, the other piece of this too, is trying to engender I don’t think we’re ever gonna engender a love of assessment in every single student affairs professional. Rather. I think we’re gonna have to make some use cases around why it’s valuable to them in their career. So first of all, I would say every institution needs to have some form of assessment team with representation from across each of the student affairs units. And even if they only meet once a semester or once a month, there should be some group that is brought together to help discuss the issues of assessment. Second, I would ask the question, when was the last time that assessment was an equal stakeholder in the budget process for the, for the division of student affairs is the, is the assessment office are at the table when budgets are decided, is actual assessment data utilized and provided to either leadership or a student service fee committee when the budgets are being allocated. I like to encourage our student services fee on our campus to ask every single person that comes forward with a, with an SSF budget request, to see a copy of their usage statistics from the involvement and engagement dashboard that we put out every year from card swipe data. So I feel like there’s ways that we can either help be nice and bring people to the water to make them drink. But sometimes you just have to throw folks into the pool and help ’em swim.

Glenn DeGuzman:
Yeah. You know, Jason, Jason, you brought a good point. Not everyone’s gonna nerd out with assessment in a, our field.

Jason Simon:
I don’t know why it’s such a good think, but OK.

Glenn DeGuzman:
This group, I think we all did

Glenn DeGuzman:
Minority here, but I want to dive and go deeper to Kevin, your response to that last question and you, you kinda and I’m, but I’m gonna have Jaqui respond to this to start, because I think about like my depth of understanding an assessment didn’t occur during my grad program. It occurred actually when it became a professional, because I started to see the value of how that can help influence and how I can utilize that to leverage and influence others. If I need a certain, if I wanted to go in a certain direction or a certain path where to get a certain decision be made. When I think of my grad preparation program, shout outs to Colorado State University, Go Rams, I know that I took a steps course. I had my practicum where I, I built program evaluations, maybe built something that looked like a student learning outcomes. I, you know, if I think back probably wasn’t a really good one. I, you know, but I, I guess the question for me, for you, Jaqui, is what advice would you give to not I’ll give you choices. Student Affairs professionals or maybe even faculty in grad prep programs. So we can start to maybe create a a professional population who maybe can nerd out and fall in love with assessment, or at least have a solid competency in this, because I think it’s, it is something that’s really critical in our field thoughts.

Jaqui Thomas:
Yeah. So I’ll start with faculty and I’m sure that Jason and Kevin are, have a lot of thoughts on the, the faculty and the grad students, but briefly faculty and grad students. I think that when you are looking at grad prep, grad prep programs, it’s important for students to see the outcomes of assessment. So yes, there’s an as you know, here’s, as here’s an assessment project and we’re gonna run these stats or we’re gonna do this qualitative research together as a class, because most of these classes have some type of experience based project, which is perfect, but to show students what the outcomes are, what decisions were made what additional funding was made, how retention rates or other student success metrics were affected is really important. I think for grad students getting familiar with the ACPA and NASPA competencies around assessment and evaluation is really important just to be familiar with what they are and how you can move from foundational to intermediate, to advanced but for new professionals this can be difficult.

Jaqui Thomas:
I think that when you’re a new professional and you’re not necessarily in a seat where you get to make a ton of decisions, what you can do is start where you are. So it’s really about being honest about what’s in your jurisdiction and not the necessarily like what’s in your boss’s jurisdiction or what’s in the, you know, director of the department’s jurisdiction and so on and so forth. Right. What, what decisions are in your jurisdiction and what data can you use to drive those decisions? You should talk about that in meetings. Talk about the data that you’re using make your decisions in meetings group meetings, staff meetings, with your supervisor. People appreciate things like that. People respect things when it comes from numbers, especially when everybody has access to the data source too. You can be a really good influence on others to use data to make their decisions. And then I would also say because new professionals are typically looking to move up and forward in their careers. So keep track of those outcomes of your data informed decisions. Don’t just make a bunch of data informed decisions and forget about ’em right. Keep track of what you do and talk about it in those interviews. Moving forward, because at this point, I think most student affairs positions are asking questions about assessment,

Glenn DeGuzman:
Jason, Kevin, what do you think,

Kevin Gin:
Jason, you hop in first?

Jason Simon:
Great. So thanks, Kevin. I would say recognize when we talk about grad prep programs that it’s not just masters. And so I think there’s a real mismatch right now at the doctoral level curriculum. I’ll start there and work back. I feel like most of the assessment courses that I’ve either I’ve been asked to come in into a guest lecture or team teach on it, it’s very much focused on survey methodology focused on qualitative focused group process. And, and those are really vitally important, but I’d like to take it to a slightly different level. I’d like to see more, especially graduate faculty programs, teaching about analytic literacy, teaching about data literacy, teaching about how to establish a culture of data informed decision making across any area of higher education. You know, it really concerns me when, when folks who are making decisions around, let’s say a really expensive technology implementation that’s supposed to solve all their problems in a box, don’t understand or know the right questions to ask the vendors or the developers, or even within their own campus.

Jason Simon:
The folks that put that together about things like equity, about things like assumptions in the algorithm, is it a black box? Is it, is there, is it our data? Do we have to lose our data? Like basic questions that I think today’s higher education administrator needs to take their level of assessment and data literacy up a little bit with regard to master’s degree students. I really feel like so much of the focus is just spent on teaching them how to do X, Y or Z or plug button a into plug and B and B C that instead we lose sight of what the real purpose is. And I would love to see more graduate curriculums, embed inquiry in all of the coursework, not just the one token student affairs assessment class in the cohort. I would love to see assessment be part of capstone experiences for master’s thesis. Just in different ways. I, I just feel like that we, I mean, let’s think about our own personal lives and the data that surrounds all of us right now. Do we live all of our lives because of surveys that we respond to, or are we living our lives of all the different forms of data and the internet of things and everything that’s coming through, our spheres of influence on a daily basis. Why do we keep limiting within higher education? The definition of assessment? I don’t get it.

Glenn DeGuzman:
Thank you. So this is we alluded to this in the very beginning of this podcast, and I think that diversity equity and inclusion is becoming very central to our work. And I know that campuses are now really trying to see how they can assess and measure and get a, and get a good understanding of their climate. So my question, and I’m gonna start with you, Kevin. How has assessment helped campuses and departments evolve over the past five, 15 years to better serve our students?

Kevin Gin:
Yeah it’s an interesting question, Glenn. It’s a big question. I think it varies. And so my, my, my response here is gonna vary a little bit as well. I think for, for the most part one of the things is we get back into what is the purpose of higher ed, right? There’s a democratic purpose to this. So if we’re not graduating students and studying them up for success, if they’re not learning and there’s no social mobility we’re fundamentally failing here, right? So we always have to keep equity at, at the forefront of everything we do here. Who’s learning, who’s not learning who has access, who doesn’t have access. And, and when we really start doing assessment, it requires us to disaggregate that’s the key word, right? You hear that disaggregate word all the time, but what does it really mean?

Kevin Gin:
Right. So when we have convers about assessment in student affairs or on campus, or in academic affairs we, we at least on the campuses that we’re at now at Cal East Bay under the direction of Dr. Fannie Young, she has asked us to say, what is the equity angle that we’re taking here? Cuz every sort of assessment needs to have equity embedded into it. So for example, we were taking a look at the students who are not passing the intro intro 101 course for lack of a better, you know term for it. And so we were taking a look at that and wouldn’t, you know almost three quarters at the student who are not passing that course were underrepresented minorities, which the CSU system cause URM black, Latinx and native American students, that’s a huge issue. That’s a gigantic issue.

Kevin Gin:
So what can we do there? Well, let’s take a look at who are these students? Right? So we took a look at the students at the NESSE, the being college student survey. And what we found is that the students who weren’t passing the introductory course, majority students had AP units. They were coming into the school with the majority of them, had a GPA over 3.0, the majority of them were in honor societies, 86% of them intended to graduate from Cal State East Bay, 99% of them expected that their first school courses were to have collaborative teamwork together to discuss career plans with their faculty and to also have study sessions outside of the, of the courses. So we start asking the questions, what is, how are we teaching our students? And are we being responsive to the needs of the students culturally and to their mindsets?

Kevin Gin:
And so we had to start thinking about who has access to higher ed who’s succeeding, who’s not succeeding. And then coming up with meaningful measures that make sense for us. So equity measures need to be meaningful, whatever measure we come up with, whatever outcome we have to really define what is student success? And that’s another key buzz where you hear student success all the time. What is student success really? Because it sounds good. Say, well I’m dedicated student success, but can you actually define what that means? Again, it goes back into, is it identifying students who are going on AP and reducing that, is it getting students to get engage on campus? And what is engagement? What are the touch points that constitute engagement? What, you know is a meaningful engagement? Is it retention? Is it graduation? And is it, when is it graduating? Is it within the, the two year, four year, six year? Or is it just getting into the end point? So I’ll stop there. Cause I know that Jason has been doing a lot of really progressive work with the data analytics side at UNT. But I’ll just talk to the audience that you first have to understand and define what is equity and what are you trying to measure here? Cuz it could be in housing. It could be, it could be community engagement. It could be in the curriculum. It could be anywhere.

Jason Simon:
Laura Jos. I think first of all, you have to not separate yourself from the data that you’re working with. You have to recognize if you have any implicit bias, you have to recognize if there’s a privilege of perspective or a way that you raise that influences how you view the work you do on your campus. You know, there’s, there’s a lot there to unpack. I get it. But what I, what I would say though is for us, it’s also a matter of really trying to overcome some significant hurdles as a field. If you look at the data, the way that it’s structured, gender is very black and white ethnicity is very locked in. So we’re talking about lots of lumping and aggregating. We are wholesale ignoring certain members of our society right now because we don’t have a way of capturing their information on a national or even in some cases at some state levels.

Jason Simon:
So I think when we start to talking about data and assessment and equity and assessment, we have to start there that there is going to be a need for individual units to do a little bit on their own locally because the data’s just not there nationally. And so for us, we now do a every semester we do a campus poll survey of campus climate and it’s a relatively short assessment, but we’re asking all the right questions. And now we have a longitudinal data set. We partner broadly across all of our divisions and we disaggregate that data by school and college. We disaggregate it by ethnicity. We disaggregated by sexual orientation and we pass it off to our employee resource groups. I mean, there’s lots of ways that we’re now using data more effectively from an equity lens. And clearly when you read the headlines around how certain artificial intelligence machine learning black box tools are really predicated and built upon some pretty racist slash not good things.

Jason Simon:
I think it requires all of us as student affairs practitioners and data professionals to take a step back and really ask the tough questions about what we’re thinking about when we’re seeing results, love what Kevin said about disaggregation, because it’s amazing once you, once you stop comparing within groups and you start comparing within just that group and looking at trends over time, what I’m trying to say there is. So instead of just comparing whites versus Asian Americans and putting them side by side on the graph and saying, why is one lagging behind the other? The that’s not, that’s just perpetuating lots of the, of the historic inequities that we’re dealing with. Rather we should be looking at within the trend, how are Asian Americans doing over time? Are they growing? Are they slipping? And then we can start to design conversations and initiatives to address that to me, I just feel like we need to complete reset. And this is it’s so funny to me, how all these questions keep coming back to rad prep in some cases, but there needs to be some real focus and conversation on, on this at the graduate level. And so for me, you know, you guys throughout your, your, your grad school. So I went to the university of Vermont and, and there now this is a really big deal and it’s a really big issue. And so I think we need to start there. It needs to needs to really propagate from the very beginning.

Glenn DeGuzman:
Thank you Jason, for that, you know, I’m starting to look at time. And so I want to kind of start sorta closing out this conversation as much as I don’t really want to, but I do want to hear from each one of you you know, you’ve all done so many different types of assessments and maybe if you take an interview approach and sort of, I can concise response, what assessment are you most proud of professionally and why? And it could be for a variety of different reasons. So it’s an open question. Jaqui, do you want to lead us off?

Jaqui Thomas:
Yeah, so I actually have two things. I want to highlight and I’ll do it in my, in my short amount of time I’m supposed to use, but first is a learning goals project that I took on with a, with an amazing grad student from Penn State. And we build learning goals for staff, for our, for our RAs as well as for our, our RDs. And we used job descriptions, CAS standards, the NASPA, a CPA competencies. And then when we we use all of that as well for the RAs, but we also included student development theory and everything is grounded in those things. So once we built the assessment we were able to pinpoint the specific tasks associated with all the major responsibilities in both positions for both positions to be successful. And we were, we asked them like, how knowledgeable are you when it comes to this particular task, right.

Jaqui Thomas:
This really important task for position. And then how confident are you when you carry it out? And then if they indicated that they weren’t knowledgeable or weren’t confident, then we asked them why. And we are giving that information back to supervisors and making recommendations about in general, you know, a lot of people are not confident about this thing. Maybe you should do something different in training. So I’m really excited about that particular assessment and just finished it the second year. So I’m looking at giving data back to supervisors real soon. And then the other thing is more of a project, an assessment project, not so much a like a survey or anything that I collected data with, but I’ve been building presentations for both new and season professionals on how to approach unit assessment or departmental assessment in particular how to make data informed decisions and things like that. And those are things that I’m I’m pretty proud of those cause I get good feedback and I see people using data afterward.

Glenn DeGuzman:
I definitely benefit from these. Kevin.

Kevin Gin:
Yeah. I’ll, I’ll take a a, a, a quick approach to this. So I, again, I’ll remind everyone that I started off in alumni affairs working at the alumni association, then in student affairs, I did housing residence life, student activity, student leadership. So don’t think that I’m any statistical guru over here. I don’t have really any stats training outside of the graduate courses I took. Yes, I could use SPSS. Yes. I know how to use Tableau. Yes. I know how to use power BI, but I’m not really that skilled in it in that way. So I just want to be able to tell you all that as we kind of get into this and what I’m gonna explain as my kind of most proud assessment moment, it was helping to coordinate reaffirmation of accreditation for two institutions. You, one of ’em at Boston College, one of at Holy Names University.

Kevin Gin:
Again, I’m gonna tell you I wasn’t trained in this. So I’m gonna say this to all of you all sitting out there right now, as student affairs professionals, you can do this. It’s not about if you can do the regressions in your head, it’s not about the statistical chops that you have. It’s about the questions that you answer, and it’s about the mindset that you have for continuous improvement. And can you actually understand what are the ways that we’re trying to increase student success and improve and, and ensure the quality of higher education and the degree and services that we offer. So those are two things that I’ll put out there that I think I was most proud of because I realize that under just the tutelage of someone who is willing to teach me Dr. Bob Newton over at Boston college he took me under his wing and taught me.

Kevin Gin:
And so all of us, I think if you reach out to any one of us on here, we’re willing to teach you. And even with my mentees right now, I teach them and I’m working right now with an acronym advisor who said, I’ve never done assessment before. It’s not that you’ve never done assessment before. It’s that you need to reframe the work that you’re doing in a way that actually is about continuous improvement and student success. So I’ll just put it out there that I think that any of us can do it. And, and I’m proud that I was able to have people show me along the way, including Glenn and Jason, on this call here.

Glenn DeGuzman:
I was gonna flip it around. I was gonna say, you know, I had a conversation with you many years ago and you told me about accreditation. I actually got my accreditation because of you telling me about that. So you influenced me, Kevin Jason, close us out with this question.

Jason Simon:
Yeah, I would say two, two or three, one would be you know, we were noticing that we were losing students, not only at those who were in academic difficulty, but also those who had very high GPAs. So we, we engaged in a really novel assessment approach of why I stay and what we did was beyond just the focus groups who were really inform informative, but we also ended it by giving them all sheets of paper and magazines. And we said, we want you to clip out the magazines and assemble collages that typify your experience at the institution. And then we look for themes within those collages. And that was just a fantastic visual way to present that forward to our executives because everyone is used to seeing tabular data. But when you suddenly see these really creative montages of the student experience, it really translated in a different way.

Jason Simon:
Then in more of a erect evidence kind of way, really looking at assessing students that were at risk for drop for non-payment of tuition and, and so really designing a specific analytic product that had to bring to lots of data together and lots of both direct and indirect evidence to be able to identify where we were at risk of losing someone. And, and what would the impact just basically extending them a hundred dollars short term loan mean so very two examples, right? One very direct one, very indirect. And, and then probably the last thing that I would say is really proud of the fact that that 10 years ago we started a student affairs assessment portrait symposium on our campus. And prior to COVID, we would, would get anywhere from 300 to 350 people from across campus. And it provided all of our student affairs assessment areas and opportunity to, to share with what they learned throughout the year on their assessment with members of the community. And we had faculty, we had associate deans, we had assistant deans, that program actually even won the NASPA gold medal for assessment several years back. So really proud of that element. So really fantastic conversation today. Thanks again, Glenn.

Glenn DeGuzman:
I love the, to that reminder listeners affairs now website contact information for these folks are there.

Glenn DeGuzman:
So please feel free to contact them. Cause I know that that they, they would love to share more about what they’re doing. So this is our final question. It’s our wrap up question. This podcast is called Student Affairs NOW. So please take about 30 seconds to a minute. If you could summarize what you’re pondering, what you’re questioning right now, maybe something you’re excited about, or maybe something that was said in today’s podcast that you reflect on. I thoughts and let’s we started with Jason, so let’s go reverse Jaqui your up first.

Jaqui Thomas:
Honestly, I, I think that conversations like this, I are really exciting for me meeting people like Kevin and Jason, who I hadn’t met before today, really exciting for me because it is very common for those of us who are in student affairs assessment to work in a little bit of a silo and to not necessarily all get the opportunity to talk to other people who are doing what we, what we do, and it’s great to be a subject matter expert. And I mean, I definitely take that with you know, a level of seriousness and I make sure I stay up to date on knowing like the latest and greatest happening in student affairs assessment. But it also means that I don’t necess really have someone to always bounce things off of. So I just, I really appreciate organizations like student affairs, assessment leaders and conversations like this. So thanks.

Glenn DeGuzman:
Thanks Jaqui. Kevin.

Kevin Gin:
Yeah, I don’t want to be a down to here, but there is something on my mind that does really, to assessment. I do want to call out the one thing here that we haven’t talked about assessing and that’s ourselves, the human capital part of it. And, and we all know right now on higher education in general, but student affairs, acutely aware of the burnout that’s happening and the departures that are occurring there. And so I would, this is a call out for those, do an assessment. Those who are institutional, those who are vice presidents, those who have a role in helping assess morale and climate assess the viability of your staff right now, you know, see what are the workloads, what are response times who’s departing the institution. I would venture the guess. If you start taking a look at the departures of the institution for people in these frontline critical roles that you’ll be very troubled at, who’s gonna be leaving here, take a look at resource allocations, and also the amount of work that is now being asked of, of your staff as well. So Glen, going back full circle to your first question about, so how are resources allocated and how decisions are being made. This is a call out to student affairs, assessment, individuals, and those who are doing institutional assessment and leaders. You need to assess your staff as well. We do assess students, but it’s about institutional effectiveness. Remember that institutional effectiveness and, and we’re all part of the institution. So I will leave it at that for a hanger.

Glenn DeGuzman:
Thanks, Kevin and Jason,

Jason Simon:
I’m struck by and really excited about the intersectionality between assessment and analytics. I feel like there’s a lot of potential there for, for student affairs practitioners to get more deeply entrenched into modern approaches, the data, data literacy and data culture, and maybe stop resurrecting, just the historic practice as of assessment task, but rather really look to the head to the future. I would say, you know, really if, if you’re even remotely interested in an assessment don’t just limit your own boundaries and horizons by just thinking it has to be X, Y, or Z really look broadly across the field. I could see a future date time when masters and doctoral programs have specific pracs for people who want to have experience in assessment, data analytics, modern predictive analytics as a whole new subfield of our institution, especially as the risk get higher and higher, and the price gets higher and higher for the cost of an education. And we don’t want to be settling our students with massive amounts of debt. We have to do a better job. So I think there’s a real social ethical and moral calling for assessment practitioners to jump into these conversations. And so I really believe that the future is bright, but we just have to shine a spotlight in ourselves and make it that way.

Glenn DeGuzman:
Thanks. And I want to thank all our guests Dr. Kevin Gin, Dr. Jaqui Thomas and Dr. Jason Simon for joining me on this podcast, but I also thank Nat. Who’s gonna take this, this recording and basically transcribe and get it prepared for airing. So thank you NaS for that. And again, I want to thank our sponsor simplicity as a global leader in student services, technology platforms with state of the art technology that empowers institutions to make datadriven decisions specific to their goals. A true partner to the institution, Simplicity supports all aspects of student life, including, but not limiting limited to career services and development student conduct and wellbeing, student success and accessibility services to learn more visit simplicity.com or connect with them on Facebook, Twitter, and LinkedIn. And I also want to thank Vector Solutions. How will your institution rise to reach today’s socially conscious generation?

Glenn DeGuzman:
These students report commitments to safety, wellbeing, and inclusion are as important as academic rigor when selecting a college it’s time to reimagine the work of student affairs as an investment, not an expense. For over 20 years vector solutions, which now includes the campus prevention network formerly EverFi. For again, for these old school folks, they have been a partner of choice for over 2000 college of universities and national organizations with nine efficacy studies behind our courses, you can trust and have full confidence that you are using the standard of care for student safety, wellbeing, and inclusion transform the future of your institution and the community you serve. Learn more at vectorsolutions.com/studentaffairsnow. To our audience and listeners. Thanks for joining us. I’m so happy that I got, this is kind of a reunion for me, so I’m happy to be hanging out with my, my people. If you’re listening today and you’re not already receiving our weekly newsletter, please visit our website at studentaffairsnow.com. Scroll to the bottom, the page to add your email to our MailChimp list. While you’re there, check out your archives. I’m just so proud of Student Affairs NOW we have been blowing up our download numbers are closing in on 40,000, if not, we’re over. So I’m Glenn DeGuzman. Thanks for listening, watching, wherever you go, go out, make it a good day. And we’ll see around. Bye everybody.

When you collapse an accordion item and save, it will automatically display collapsed in front end