Darby Roberts:
Well, I will say what I’m thinking about relates to this social justice assessment and really campus climate. So on our campus right now we finished data collection from undergraduate students, graduate students, staff, and faculty about campus climate and fantastic data collection. We already know inherently that there are some issues on our campus we’re predominantly white that a lot of tradition and culture and impact to think about. But as the assessment person, what I really ponder is how do we get that data into the hands of the people who can actually make change?

Heather Shea:
Hello and welcome to Student Affairs Now I’m your host, Heather Shea. Today we are discussing assessing student learning outcomes with four assessment experts. We’ll be sharing ways to streamline the process through frameworks, as well as learn what various campuses are doing to build a culture of assessment. Student Affairs Now is the premier podcast and learning community for thousands of us who work in alongside or adjacent to the field of higher education and student affairs. We hope you’ll find these conversations make a contribution to the field and aree restorative to the profession. We release new episodes every week on Wednesdays. Find us at studentaffairsnow.com. This episode is brought to you by Anthology is your goal to engage in effective assessment. Do you want to boost data fluency and empower staff with strategic data collection and then use these results for change no matter where your campus is in the assessment journey, Anthology formerly Campus Labs can help you figure out what’s next.

Heather Shea:
With a short survey, you’ll receive customized results and tailored recommendations to address your most immediate assessment needs. Learn more about how Anthologies products and expert consultation can empower your division with actionable data by visiting campuslabs.com/sa-now. This episode is also sponsored by Everfi. How will your institution rise to reach today’s socially conscious generation. These students rate commitments to safety, wellbeing, and inclusion, as important as academics and extracurriculars. It is time to reimagine the work of student affairs as an investment, not an expense. For over 20 years, Everfi has been a trusted partner for over 1500 colleges and universities with nine efficacy studies behind our courses. You will have confidence that you’ll be using the standard of care for student safety and wellbeing, with the results to prove it transform the future of your institution and the community you serve. And you can learn more at everfi.com/studentaffairsnow.

Heather Shea:
As I mentioned, I am your host, Heather Shea. My pronouns are she/her/hers, and I am broadcasting from East Lansing, Michigan near the campus of Michigan State University, where I work. MSU occupies the ancestral homelands of the Anishinaabeg – Three Fires Confederacy of Ojibwe, Odawa, and Potawatomi Peoples. I am so grateful for our four panelists today. Thank you so much for joining me on Student Affairs Now let’s get to our conversation as each of you introduce yourself, if you could tell us a little bit about your current role and your engagement with assessment, either broadly in the field or on your campus. And Nicole, I’m going to start you today.

Nicole Long:
Great. Hello everyone. My name is Nicole Long and my pronouns are she/her/hers, and currently, I am the executive director for planning and strategy at the University of Delaware, where I oversee our divisional assessment efforts in addition to strategic planning, as well as communications functions. In addition to that, I also serve as chair of the organization called student affairs assessment leaders, where I’ve had various roles in that group from being a committee member on the professional development committee to being a contributor to our about assessing or not about assessing about working in student affairs assessment and leading student affairs assessment and divisions, and also I’ve served as a member at large. And currently I serve as a liaison to the council for the advancement of standards before my role at the University of Delaware. I also worked at institutional research and institutional assessment along with a career as a student affairs practitioner. So the role I have now definitely is the perfect convergence of all of those experiences.

Heather Shea:
Awesome. Thanks so much for being here today, Nicole, and you’re at a great campus with a lot of good colleagues and friends. I appreciate you Dan, welcome to Student Affairs Now.

Dan Bureau:
Hey, thanks for having me. My name’s Dan Bureau, I use he/him/his pronouns. I’m coming to you from the ancestral homelands of the Chickasaw and the Quapaw and at the University of Memphis. In my role here, I am the assistant to the vice president for student academic success. That role places me as a project manager for a number of initiatives across the division. And my day-to-day work with assessment is a little bit limited now in terms of my professional role though, I have worked at the University of Memphis previously as our director of student affairs, learning and assessment. And also prior to that, I was at the center for post-secondary research for the NSSE survey back in my doctoral work. I’m also here as the president for the council for the advancement of standards, and look forward to talking with you all today about CA as well as our new.

Heather Shea:
Awesome. Thank you so much, Dan. Yes. President of CAS. Definitely an important thing to mention. Not last but not least, right. Darby and Patrick are the, are the two authors of the book that we’re going to be talking about extensively today. Darby, tell us a little bit about you.

Darby Roberts:
Sure. I’m Darby Roberts use pronouns, she/her/hers and I am the Director of Student Life Studies at Texas A&M University in the division of student affairs. And I have been in the department since 1998 and been director since 2013. So I’ve spent a fair amount of time working on student affairs assessment, and it’s a great career path

Darby Roberts:
For me. So I’m excited always to talk assessment and student learning. Part of what our department does is serve the division of student affairs, as well as recognized student organizations in any sort of assessment need that they have. So we ended up doing 250 to 300 projects per year for our constituents. So we’re a little bit busy all the time. In addition, we coordinate our program review process for our division of student affairs and spend a fair amount of time also doing training and development for our staff in the division to really help them up their assessment game, learn those skills that we find to be really essential. And then in my spare time, I also teach in our student affairs master’s program, teaching assessment in student affairs. So helping those grad students really develop those skills early on that they can take with them as new professionals. And then I also am co-chair of the NASPA assessment evaluation and research knowledge community. So all things assessment.

Heather Shea:
Is your middle initial a.

Darby Roberts:
it should be.

Heather Shea:
It should be thanks so much for being here, Darby and Patrick. Welcome to Student Affairs Now. Tell us about you.

Patrick Biddix:
Thank you very much. My name is Patrick Biddix. I am a professor at the university of Tennessee. I have a couple of odd different job titles that maybe as a good insight into what I, what I do nowadays. So I’m the interim department head. It was supposed to be a one-year job. And then with the hiring freezes and COVID and everything else, I agreed to take it on for a second. So fortunately we are in the process hiring now, so we’ll hopefully have a department head soon. So I do that I would like to say that’s my job, but that’s been my full-time job now for two years. as professor of higher ed, I teach various different courses to our student affairs in higher ed master’s students and, and work with them on their dissertations specific to assessment and research that’s tends to be the sequence of courses that I teach, but I also now do some policy courses as well coordinate our higher ed doctoral program.

Patrick Biddix:
And then I’m an associate the associate director for our post-secondary education research center, which has a lot of different areas of focus. So been at UT for 10 years as a faculty member, prior to that, I was at Valdosta State for six years. And prior to that, I was at Washington University in St. Louis as a student affairs professional. I still am very much involved in assessment work specifically at UT. So I like Darby. I teach our assessment course, but also do quite a bit of training for the division that our department, a lot of our students do. A lot of the assessment work masters and doctoral students do within the division of student affairs. And I’ll talk about this probably a little bit later. But I also coordinate our pulse surveys on campus. So it’s sort of a quick, one-time one answer based surveys for the division. Everything from, are you planning on coming back next term to a quick sense of belonging things we’re talking a minute and a half questions to our students? So that’s been my latest, I guess, assessment endeavor.

Heather Shea:
I imagine that has become a little bit more complicated to get responses. If you’re not walking down the hall in the student union or wherever you might’ve grabbed students to, to have those quick responses, we’ll certainly talk about COVID and assessment work today.

Patrick Biddix:
I would add there I to quote my friend, Melinda Matney and many of you know on, on here as well, she says, I like to be their distraction. So I tend to try to find when students are looking for a distraction and ask them their question, then

Heather Shea:
Probably the libraries, like the best place then right there, like give me some seconds to do something other than study. Yes. Oh, I love it. Well, as we mentioned, we are here today to talk about this book Frameworks for Assessing Learning and Development Outcomes. The Genesis of this episode is this book. We’re going to certainly talk about much more beyond that. But Patrick, why don’t you start with talking a little bit, this is the second edition of the faldo’s book as it’s affectionately known. How did this book come about and how does this book build upon the first edition?

Patrick Biddix:
Well, I, it really came about through a conversation with, with Gavin and Gavin president of at the time reached out and just said, Hey, we’re, you know, we’re thinking about doing a new version of this book. It’s been one of our best sellers and our most used books, which was what is connotation there was and our practitioners and our audience really likes it because it’s an off the shelf resource. And of course I was very familiar with the book have, have used it many times both as a practitioner. And then as a faculty member in sort of teaching out of it. And so my first thought was, Oh my gosh, what do you add to this book? In many ways it really codified what assessment was when it came out. I mean, it came out at a time when we were still, many of us were doing assessment work, but we weren’t necessarily calling it that. And certainly it offered frameworks that didn’t exist before. So I, I agreed with some caveats that I thought you know, we needed to have at least another author or two that could join that could join the, the effort with us. Unfortunately he had already done that legwork. So there was no, there was nothing else I had to do there. And he said, well, what do you know, Darby? I said, I’ll come on. Of course

Patrick Biddix:
We’ve worked together many times. And so that was a that was a blessing really to get to write with, with her. I’ll, I’ll share a story. I, I’ve shared a couple of different times though. My my tie to the text is, is kind of funny. The, when I came to the University of Tennessee I the previous person who was here was and Tarell, I literally sit in his office now Bailey 316. And when I got in there, you know, when you inherit somebody else’s office, there’s typically things that are kind of left there. And there were some shelves and some furniture and a chair, and this stack of bound books, there were 10 of the original FALDOS texts in there in print. And I held onto them. I mean, this was, you know, again, this was a while back.

Patrick Biddix:
I did not, no idea I would ever be, you know, getting a chance to work on the second edition of the book. And so when Gavin reached out, I went, wait, that’s the book that I have. I mean, so it was fun kind of moment of gosh, you know, what do I do with that? So I think the other thing I would add is that we, we really spent a lot of very intentional, early time, pre planning time before even any writing began on maybe more so, even than the writing on just thinking about what this could add, really dissecting the first version of the text, understanding what was there and what assessment was and what the context was at the time. And then what we thought that we could add in the spirit of still keeping the book as an off the shelf product.

Heather Shea:
I love that story about how the books were in the office. Cause it seems like that was like an early sign, right? Like that you are meant to do this thing. That’s funny. So Darby, if you think about this book and as you work, as you worked on it, and I know it talks about it in the beginning part but what audiences do you had in mind and as a faculty member, maybe talk a little bit about maybe how you use it in your classes or, or what role it plays in teaching, master students in student affairs.

Darby Roberts:
Now, when we put this together, I think one really important component is that anyone can pick it up and use it in a variety of ways, but I really see it as graduate students, new professionals people who are new to assessment, who haven’t been doing this maybe it’s been given to them as a volun-told, you’re now the assessment person for a department or the division.

Darby Roberts:
And it’s a great way to pick up something and look at it and say, okay, I get the gist of this. If we’re talking about student learning, what is that all about? And then for someone who might be at the division level of how do we frame assessment in a broader context, not just one department, one program, but how do we look at it from a broader perspective of things that naturally fit together that we might actually want to know more holistically about transition or diversity and inclusion, not just that one component of it. So I think we really framed it that way, a broad audience, but a lot of can I just pick this up and use it? And in my course, we talk about the importance of assessment in general and the importance of student learning in the co-curricular in student affairs and how we do it and how we do it well, so that the students in my class have competence and competence confidence to go forward and say, I can assess, I can assess the student learning in my program.

Darby Roberts:
I can figure out how to do this in multiple ways, because one it’s about continuous improvement. So we need to know, we need to assess, we need to be able to make change. And then that also helps in a broader sense for student affairs to be transparent, to be accountable, to really provide evidence that there is learning, taking place in the co-curricular and that we’ve done a quality job to figure out how to do that and what it means, and then what it means for us moving forward. So really, you can look at it from very individual here’s my program here are the students I work with. I want to understand the learning, but also how we look at it as a profession and a field to say we’re contributing to learning and higher education. So trying to get that across to our master’s students, this is really important. It’s, you know, some of them come in and say, Oh, but I’m here to, to work with students. And I, that’s my passion and that’s what I want to do. And I’m like, that’s fantastic. And this is an important skill to have, and it’s going to help you as you work with students, as you work in your department, as you work in a division. So really encapsulating the theory and the practice together.

Heather Shea:
I love that. And I think that is so needed, right? I think it’s an older version of student affairs, right? That we’re at planning activities, but they were just for food fun and festivities, not for, you know, seeing our role as important educators on our campus and that by planning some activities, students are gaining these skills. And so we need to help understand what those are and then document understand and learn from that as well. So Dan, I know this is one of several resources that CAS provides broadly to, within the CAS universe. We kind of mentioned earlier can you talk a little bit about how this compliments and, and then also of course, you know, where people can find out about ways to access CAS resources?

Dan Bureau:
Yeah. There’s a lot of resources out there for CAS users, but I can make one about Darby’s

Dan Bureau:
Response there too. I think the book is really important because I think people underestimate just how assessment professionals in student affairs have to become functional area generalists. And they have to understand a little bit about everything across the division and the book highlights some good literature that you might use in order to better understand functional areas that you didn’t necessarily come up through. So that’s one more thing I’d want to add, but in terms of what CAS has out there, you know, we’ve spent the last 10 years evaluating kind of how we contribute to the space of higher education. And we realized that while we’ve had these standards for awhile and we’re known for self assessment, we haven’t really been known for the development of programs and services period. Right? You can start a whole new women’s program as you’re doing right now, how there at Michigan state, you can go and do that using our standards.

Dan Bureau:
So we have resources for that. We also have come into our own kind of understanding that we are one of the leaders in terms of talking about student learning and development in the co-curriculum. And as we know, student learning is now a primary outcome versus a nice by-product of what we do in student affairs, as you’ve already mentioned. So CAS knows we had to do that work there. So really we had to create resources to help our users. And as we were thinking about that, and also remember too, that we’re consortium of 42 professional associations. So we have to serve diverse users across diverse functional areas as well. So, as we were thinking about that, as we were thinking about revising our 10th edition, which came out in 2019, we did a total overhaul of that, of that book. And I think people often think that in addition, there’s just a few changes here or there, and you have to have the eighth edition and the ninth edition to the CAS standards on good.

Dan Bureau:
The 10th edition is totally different. We totally revised our general standards. A lot of the content isn’t in there is different. And every book we’ve actually at least revised well, half of our standards. So 22 sets of standards in that book are, are revised during that timeframe. And our next book will come out in 2023. So there’s still time to buy the 10th edition right now because those standards will be relevant for awhile to come. Our standards and sags. You can buy it, our website, our book, you can buy the website. I do want to highlight a new set of standards. We just released. Thanks to the leadership within the ACPA commission for indigenous student affairs. We’re really excited about that. And I know many campuses are trying to look about how they can create programs to better serve their native students.

Dan Bureau:
And it would be important to maybe look at those standards. I think as you’re creating those programs, CAS also has what’s called the functional area. I’m sorry, the cross-functional frameworks and those, we have three of those right now. And what we realized that our functional area standards have a level of relevance that’s unique to the department, right? So whether it be multicultural affairs or residence life, fraternity and sorority life, et cetera, you need those departments standards. But there are some functions that we’re looking at that are more about creating an institutional ethos, if you will. And so we created these cross-functional frameworks, which really are an opportunity to bring together a team across the university. And we focus on those areas of first-year experience, behavioral intervention and advancing health and wellbeing. We also have a number of resource papers that are our functional area resource papers and diverse areas, such as learning assistance programs, women, and gender services.

Dan Bureau:
And we’ll have more of those to come in the future. We introduce a subscription option a few years ago. So for one low price, you can get access to all the materials you want for over one year. And that includes all the discounts were that we would offer as well. And we do have about 30 or 40 institutions that have taken advantage of that simply because they want to have all the access to the materials so they can help their departments best use the standards. The last couple of things I talk about. So those are some of the things that for a charge go to our CAS store and you can get those, but on our website, we also have a number of resources for graduate students, for faculty, for practitioners that are using the standards. We do a lot at conferences, functional area, as well as ACPA and NASPA to promote how to use casts.

Dan Bureau:
And I’m happy to, if anyone wants to contact me, set up a time to talk more about how you can implement that on your campus. But yeah, there’s a lot out there that can help our users better implement the standards from the creation of a program all the way through a thorough self-assessment and program review.

Heather Shea:
Yeah, I remember in 2019 when those not new document or the new book was coming out, cause I was, I was on the phone with Gavin. I was leading a review of new student orientation at Michigan State. And we were like, we just need the new self-assessment guy because things were pretty drastically different from the ninth to the 10th. Addition I noticed tags were also updated and so absolutely have a direct use of each of those resources that you mentioned. I also just wanted to pick up on one other thing that I noted as you were talking about the cross-functional standards and how the second half of this book really does kind of pick up on that same theme and use those, those broader areas to identify their frameworks for assessing student learning outcomes.

Heather Shea:
So that was how all these things are interconnected and related. Very good. So I want to turn to, Oh, go ahead, Patrick.

Patrick Biddix:
Yeah, I, I think that’s a great point. I wanted to jump in there for a second because Dan really outlined the resources that are incredibly helpful for us also as even on the faculty side and thinking about assessing programs, the challenge we had with this book, and I have a note in here to that when we were thinking about organizing it, do you organize around all these standards? We’d have 50 chapters. Do you, you know, how do you, how do you sort of put that thing together? And I think I showed you all earlier. Like I just for fun printed off all this stuff. I just early, a lot of the debate for that second half, you’re mentioning how there was really what fits, where and how do we write this in a way that’s specific enough to Dan’s point for the specialized generalist that is the student affairs practitioner right now.

Patrick Biddix:
Right. So, you know, where do you put, do you put mental health with physical health? If you do that, what do you call that? Do you put, you know, how do you, how do you put those folks together, but also not leave somebody out? And so one specific example, I remember as we ended up veterans within our transition service chapter at one point, and then we moved that to identity in diversity and inclusion at another point, because that felt more like an identity chapter, then it transitions chapter. So challenging the book early on but also to bring it back to an Darby’s point, we really wanted to be able to, you know, if at MichiganState it’s called orientation and it UT it’s called new student transition and somewhere else it’s called something else. And it’s implemented even with parent programming, the book would work still is a frame.

Heather Shea:
Yeah, I love it. That’s great. I also liked how one of you, I think it was Dan or Patrick mentioned that you really, as a person or maybe Darby, I dunno mentioned that as a, as a student affairs professional, who’s working in assessment, you need to be a generalist and the functional area expert at everything. So Nicole, that’s you on your campus and broadly. So I wanna turn to kind of some of the basic issues that come out when we’re trying to develop a culture of assessment within a division of student affairs or focus on just the, just the task of even writing really good assessable learning outcomes. Can you talk from your lens as a director, executive director of planning and strategy about how your campus is engaged in assessing learning outcomes and, and some of the challenges that have arisen as you tried to infuse this into your work?

Nicole Long:
I think the specialist generalist is spot on. And one thing I love about student affairs assessment is personally as a lifelong learner, I get to kind of dabble and all things, student affairs. And I absolutely love that about the work that I do. But I think also, as I mentioned previously, with my experience of having some work experiences and different student affairs units, whether it’s student conduct, I worked in fraternity and sorority advising as well as diversity programs. So it’s like I said, this, this work is really exciting for me because of how much I get to learn about all things, student affairs. So in my role as the executive director of planning and strategy I shouldn’t feel like I need to take a walk down memory lane and it’s really apropos because today is my sixth year anniversary at the University of Delaware. So it’s kinda been fun to reflect on the, the journey so far. You know, as a division, you know, we had

Nicole Long:
Divisional student learning outcomes when I came, they weren’t overly used necessarily. And I think that can be common at many institutions that you say we have learning outcomes, but they just kind of sit there and no one’s really doing anything with it. You know, so there was some difficulty, I think, getting traction on a divisional level. And we did some work around how can we reinvent breaths and life and to division-wide student learning outcomes to provide areas with some focus around student learning. And we really pulled upon many resources such as the casts, learning domains, learning reconsidered, AAC, and U we really looked at really the buffet of learning outcomes. And it’s so important. I mean, there’s so much overlap among all of these different resources. And so it was important for us to, you know, look at well what makes sense for our campus.

Nicole Long:
And that’s something I think that’s really important when you’re writing learning outcomes is that context is extremely important in your learning outcomes, I think should reflect your priorities, your mission. You should, it should be very clear about, you know, who you are as an institution who you are as a division, who you are as a department and in terms of what you’ve identified as learning you know, even though divisionally getting traction, you know, has been difficult. But we’ve made pretty large strides, I would say in the past few years you know, even at the divisional level, if that’s difficult independently departments were doing the work. So that’s not to say that at the department level that there weren’t units engaged in the work, our residence life and housing program has had a strong focus on student learning.

Nicole Long:
We also saw this, we see this quite a bit in our leadership program at the University of Delaware as well. A really robust assessment assessment process that they have around student learning and some, you know, well-developed rubrics that they use for the different opportunities that students participate in. We also have seen some of this demonstrated too, in terms of some of our learning related to health and wellbeing as well. So it’s not to say that departments are not, are not doing these things on their own, but just broadly. I think getting everyone up to speed as has been a focus in terms of the work that I do, I think something else in terms of engaging in student learning outcomes assessment is in support and have some structure built in to have some accountability around actually assessing your student learning.

Nicole Long:
So one of the interesting things about the context from where I work is that, and, you know, we weren’t a campus that did annual reports. That just wasn’t something that was an expectation. The, the folks who were doing annual reports consistently were often the units that needed it for accreditation, like our student health center or counseling center. And even though the focus is not necessarily student learning. And so we’ve made some strides in thinking about how do we build structures to ensure that departments have identified student learning outcomes, and also that they’re assessing around their student learning outcomes. So this is an opportunity to provide data to support that learning actually is occurring in, in your department. And I think that’s, that’s so important to have those structures in place because otherwise, you know, people may, you know, left to their own devices, may not focus on student learning and they’re focused only on satisfaction.

Nicole Long:
So that’s kind of, I feel like in a nutshell of round kind of, you know, where we are in terms of just broad divisionally, but also the work that’s happened across departments. It’s the focus for a number of years has been around individual units, focusing on student learning for just their, their program in a sense before we move on, I do want to touch on something too that with assessing learning, that’s really been important when I talk about context is thinking about attending to the shifts that need to happen. So for example, I looked at some of the work our office of student conduct has done around shifting from, you know, thinking about ethical development in this purest sense, but really focusing the focus on restorative justice. And so that learning is very different and thinking about like learning how to repair harm versus just self-reflecting on me as an individual and how I’m affected.

Nicole Long:
And so, you know, so we see shifts like that happening in student affairs, and we can’t just retain the same ethical development learning outcomes that may have existed. I think similarly, we see that particularly rated related to diversity equity and inclusion right now, too. You know, we have had folks assess around learning related to cultural competency, but now there’s a call of, well, how do we assess around anti-racism? So, you know, those are the types of things that, you know, in assessment, we have to keep the pulse on, and those are very real for our campuses. You know, if you would’ve said two years ago, we’d be talking about anti-racism or racism in general, and thinking about what learning looks like around that we would just probably be still focused on Oh, diversity learning, you know, something of nature, but those are the real conversations that we’re having in real time right now. And what does learning look like to teach content to students to be anti-racist versus just an appreciation for culture?

Heather Shea:
Yeah, yeah, yeah. I really appreciate you bringing that in. And I think one of the things I’ve I’ve noted or, and also heard you say is that you can’t just take one size fits all approach and put the exact same learning outcomes over top of every unit. And then you also can’t necessarily continue to replicate those learning outcomes for every single group of students, because times shift, and obviously the cultural context that we’re living in right now is demanding a different set of, of key components of understanding. Yeah. Thank you for mentioning that. And I, I wanted to also pick up on one other question related to your role, because I know as a, as a, as a person leading assessment initiatives on your campus, it sounds like you have departmental support and a lot of engaged leaders. How do you go about building that?

Heather Shea:
Buy-In because, you know, I think to be honest at Michigan state, I do think it falls into the the important, but not urgent category, right? Like all the other things that are happening you know, rise to the level. And then pretty soon it’s like, well, we’ll get to the assessment in the summertime. Well, then students are gone and you can’t actually ask them about the learning that they engaged in, in February. Right. so how do you get that Buy-In from assessment leaders to promote that culture at Delaware after six years, right? Yeah. So,

Nicole Long:
You know, it’s interesting. I think, you know, we’ve done some pretty unconventional, I think things around organizing assessment, but also too, I think as a division, we’ve been pretty unconventional in a way. There’s still some artifacts around just our division of student life, because for there were a number of years, we, we, we’re not a division, you know, we came, you know, we had departments reporting all over the place. And so, you know, you still see some artifacts around that. So I think even though we are a traditional kind of residential campus, there are still some pretty unconventional things. And one of the unconventional, you know, things that happened shortly after my arrival was just sunsetting our assessment committee. It’s okay to sunset it, like and bring it back and in the way that we need it to be. It’s not to say that the work that happened with that group wasn’t meaningful or useful or any of that, but, you know, thinking about the directions we needed to go in, we needed to organize ourselves and also to help instill some of that buy-in to becoming more of a representative group rather than just, you know, a few folks who are, you know, hand-picked to participate.

Nicole Long:
So that was really important with buy-in is knowing that every unit could have a place at the table around assessment. Some of the work that we’ve done recently around particularly around student learning assessment is the development of departmental student learning outcomes. And as I mentioned before, we have more structures now to support the need to do some of this work, that even if, you know, there’s not a value seen in, you know, just doing it on the day to day, you know, you still need to show up, you know, at the division level, in terms of some of our reporting structures through, in our reporting. But one of the, the newer initiatives that we embarked on to help get all of the units up to speed around having departmental learning outcomes, some had them some didn’t some have been actively working on them was to really provide our assessment council with the agency to be able to support that initiative.

Nicole Long:
So it was really helpful because all of the members of the council it’s representative of the whole division where they went through really extensive training on learning how to write student learning outcomes. And that was actually really exciting because they were doing that as a group. It was really doing it as a community, not me going into one departmental meeting or one person’s office and saying, this is how you do this, but we’re actually co-learning together. And that was really important for us to do. And through that process of, you know, learning how to write learning outcomes, they were also learning how to provide feedback to their colleagues on their learning outcomes. And so it really helped to, I think, create some trust among the different individuals in the, in the groups. So they work in teams to provide outcomes and we have consultation meetings with the unit director and whoever else they would like to designate to provide some feedback on the learning outcomes that they’ve created.

Nicole Long:
Does it mean that the unit needs to you know, take all the feedback and implement it? No, but we’re having a conversation and it really the one one of the, I always do a follow-up just to, you know, hear, well, how did that go for you and your unit? And one of the, you know, consistent things I’ve heard is it helped people gain some clarity. Particularly in thinking about student learning is not about just having the learning outcome, but do you have the experiences to facilitate the learning and support the learning to actually happen? And so so oftentimes the conversations are just learning about what people do. And again, you know, having the conversation around is your learning outcomes reflect your context. You know, if you are like, I, like, I use that example of moving toward a restorative justice model, do your learning outcomes, reflect that.

Nicole Long:
And so those are the kinds of conversations we’re having. And I, it really helps break down the wall of, you know, this is like my stuff, and these are my outcomes, but no, we should all kind of be aware of what your learning priorities are, you know, how can we support one another and doing that. And I really think that, you know, finding more opportunities to have people come together is so important. It can’t just be me saying, you need to do this. This is how to do it. Because you know, I’m one person and yeah, I can’t do it all. So it’s really important to have other champions throughout the division to assist.

Heather Shea:
That’s great. That’s great. Darby, Patrick, I’d love to hear responses from either of you as authors about the role that you see this book playing, and then also picking up on Nicole’s point about the shifting of assessment practice around the focus of diversity equity and inclusion. I know this book does pick up on Gavin prior work with Anne Lunquist around assessment for social deficit. So, you know, real life applicability, and then a little bit of benefit then for social justice. If I don’t know who wants to start not framing it to either of you. So whoever wants to start.

Darby Roberts:
I’ll start and then turn it over to Patrick. I think one of the advantages to this book is that it gives some good practical advice. It gives some tips, it gives some structure. So people that are new or don’t have confidence in what they’re doing.

Darby Roberts:
Here’s a place to start, and it really helps demystify assessment. Some folks don’t know what it is they’re fearful of it. And this is kind of a welcome into assessment. And here’s a start in how you can do it. And I think the other thing is it brings that connectivity or examples. So each of the chapters later in the book also provides examples in terms of case studies. And so it, it says, you know, learning assessment, isn’t one thing you could be developing rubrics, you could be using a national assessment, you could be doing exit interviews. And in this context, you, you know, if you’re at a religiously affiliated institution, you may have some core values or beliefs that you want to instill with the students. And so those case studies kind of can give you examples of how this works or ideas of, Oh, it doesn’t have to always be the same thing.

Darby Roberts:
It’s not always a survey that students fill out to say that they’ve learned something. And I think that key piece too, as we move forward is really how do we frame that in terms of social justice? And what does that mean? How are we dis-aggregating data as an initial piece of, do we even know who our students are or the experiences they’re having, because they’re not all having the same experience and then thinking, okay, well, where does that take us? And so I’ll turn it over to Patrick to add on that and expand on where we had that concept in the book.

Patrick Biddix:
Yeah. Thank you. I one of the things I want to emphasize on that section is we were really building the book out is we want it again, back to the core value of this book being off the shelf, what are the key components to off the shelf? And as, as you look at that first half of the book, which is really on the foundations of the assessment, that particular chapter gives you kind of a step by step process of, okay, I’ve met, I’ve never done this. Like what is step one? And I kind of had this, this moment of clarity within doing that, that I think many of us have had and have arrived at over the last couple of years, which is diversity, equity, inclusion work is not something you assess separately because it’s there it’s, if it permeates all that we do.

Patrick Biddix:
So for, if we’re looking at that as sort of a separate thing, then that’s maybe not the best approach to this, either on planning, implementation or on you know when we analyze our data. So for example, if I’m doing an assessment of learning outcomes from a fraternity members after a leadership program, there’s a component of learning in there. That’s not that’s not devoid of, of social justice based concepts in diversity and inclusion based concepts. So it doesn’t always have to be so separate. And I think that was, that was a moment of clarity for me as I really dug into the social justice assessment writing. So that’s kind of one half. The other part of this is we are still as a field kind of defining what that means in the same way, how to operationalize social justice based assessment in the same way that we, that Tarell was doing that when he wrote, you know, fell those one we were still trying to codify what it means as a field.

Patrick Biddix:
There are people doing excellent work in this area. You had Dan and, and not being well vantage to you had Gavin and and on Brian Burke, there are others that are really advancing this social justice based assessment work. And some of them, I think they all kind of started in different areas and it starting to merge together and coalesced. So we wanted to make sure we were inclusive of the different perspectives, but also very direct and okay, here is exactly how you do it because that’s who our audience was. And we landed on a couple of core concepts. The first one is very introspective and I think one of the most critical, you know, what qualitative research gets a lot of pushback because of it being somewhat subjective, right? Like I’m coming up with the questions I’m developing and I’m, I’m, they’re rating the direction of this interview, but quantitative questions emerged from somewhere too.

Patrick Biddix:
And if we really think about where those questions come from and what we’re asking and who we’re asking, there’s very much a positive, the approach there as well. And it’s not as, as, as objective as we’d like to believe. And so I would, we really wanted to emphasize that piece first, and that comes from Brian Burke’s work of as a white researcher researching black students to say, okay, who am I coming into this? And what do I bring that’s unique, but also what can’t, I understand because of, of identity. So those are that that introspection is an important part, or at least it was, as I was thinking about the social justice space assessment, and then, okay, how do we implement? Where are we going? How are we finding people no longer? Do we accept the excuse in assessment that, well, we just didn’t have a lot of people respond or this particular group didn’t respond well, there’s a reason they didn’t respond. And it’s because we’re not doing our due diligence and understanding where to meet people where they are, right. Or understand the history of that. Go ahead, Dan, your, your jacket.

Dan Bureau:
It was 1996 when the student learning imperative came out in student affairs, it takes so long for things to gain traction, right? And you’re talking to, or people here who are published in other areas around this idea of student learning, it’s gone back like Darby when was going to learning is not a sprint book, was that 2012, 2012, 2013 years ago. Right. And so it’s amazing how these things have not necessarily sustain the traction. We need them to sustain. And it’s amazing. It took as long as it did for us to intersect assessment and social justice. And I’m really grateful to Gavin and Ann for that conversation they had at NASPA back in like 2016. Right. But, you know, the reality is, is that I think student affairs assessment professionals were afraid of going there because of their positionality, as well as the fact that it’s not easily quantified.

Dan Bureau:
And I think we’ve spent a lot of time in the space of trying to quantify the experience where the qualitative stories really get undervalued.

Heather Shea:
Yeah, yeah. Yeah. I really recommend folks who are interested in unpacking the social justice based assessment, work of Gavin, and to look up some of their, and I think they’re working on a book actually, which I think will help coalesce as, as Patrick mentioned, all of the different lines of thought around this, because I think both on the practical level, as real, as well as the philosophical level, like we have, it’s a mindset shift. And the one model that they keep showing that I’ve seen them show several times at different presentations is kind of the Venn diagram, like here’s DEI work and here’s assessment work like, well, at what point do those two areas overlap and intersect. Having DEI served at Michigan State

Heather Shea:
On a DEI steering committee that was looking at student success and composition in particular. It’s really interesting how, you know, neither of those two areas are necessarily talking to one another, at least on my campus, but through some of those lenses are able to integrate yeah, I would really recommend folks to check out that episode as well. It was a really fascinating conversation.

Patrick Biddix:
I would add one other piece too, to what Dan said earlier, if you really want some like a good student affairs history lesson, we cited it in the book a couple of times, but go back to the student personnel point of view and the original, the 1937 read past the preamble into how they tell us we should be treating the field as a science of learning and how we should measure and evaluate and how it’s qualitative as well as quantitative. Like it’s all there. And that’s what eight years ago at this point. So, I mean, it ties back together to Dan’s point as well.

Heather Shea:
Yeah. Yeah. Well, and I think it’s the mindset, the mind set shift to see ourselves as educators. Right. which I mentioned earlier, which is I think a key component to this. So I want to talk just briefly before we close out today about kind of how we’ve responded broadly and in our kind of current moment of being in the middle of a global pandemic and what kinds of challenges that has added to our plates as assessment leaders. Nicole, to return to you for just a moment. I know you are the chair of the student affairs assessment leaders organization. Can you talk a little bit about, you know, not only challenges with writing, learning outcomes, but broadly how assessment leaders are thinking about or considering COVID issues on campus through their work? I guess it’s asking questions, but I won’t make that assumption.

Nicole Long:
Yeah, no. I mean, there’s been a lot of conversation about how assessment leaders have had to respond and pivot very, very quickly. Starting from, you know, at the start of the pandemic assessment, didn’t seem like a priority in the way that it was. You know, many people were making decisions to not participate in some of the large commercial surveys that they would run on a three-year cycle on our campus. You know, we did that for two projects as well at our campus, and I know many colleagues reported the same thing, and there’s been some challenge of thinking about the re-entry. So when do we, you know return to some of the planned assessment work that the institution would, or division department would typically engage and many people have at this point you know, whether it be program review, I mean, I heard about campuses doing program review over the summer, which I was like, wow, like they returned very quickly.

Nicole Long:
But that’s not everyone’s campus and their ability to do that. I, I think a major challenge in the response is the need for, we need data, like right now, like yesterday to take really quick decisions. And I think one of the things that the pandemic highlights is particularly in this area, that many divisions departments are under-resourced in terms of assessment. So you know, everyone thinks they need a dashboard. It’s like, well, do you have the personnel to support, you know you know, that kind of endeavor. So there are these really practical things that have arisen. Definitely many colleagues have talked about the pulse surveys that they’re doing on their campuses regularly to learn more about the student experience, to be able to make decisions on our campus. Our residence life and housing department does a Tell Us Tuesdays.

Nicole Long:
So that’s something that they instituted recently to get some real time immediate feedback about student experiences so that they can respond and attend to the student needs needs. So I think really the, just the need for like data now is something so it’s, it’s odd. It’s like it’s, you know, obviously been disruptive from whatever plan or survey activity or assessment activity you had, but there’s still plenty of assessment activity that has to be done in the pandemic is what I’ve heard. And also just even what I’ve personally experienced there was you know, at the start of the pandemic, everyone was trying to come up with a COVID survey. I mean, it just felt like listserv flooded with them. And, you know, I started putting together my own spreadsheet of all the COVID surveys and to help our campus identify what, what should we participate in. So and that happened very, very quickly. I was like, wow, there are some really quick survey writers out there.

Heather Shea:
Yeah. Well, and I think, you know, in terms of my own engagement with students and in my office, like one of the things that’s been really worked well is our ability to collect data right there, because they’re right there on zoom, right. We can send a poll or do some kind of did we meet our learning outcomes in the session and they can respond. It seems like at least in some of that functional work, it’s, it’s become somewhat easier. But challenging on, on other levels. So we are out of time, it’s always a quick hour here on Student Affairs Now, and we always conclude our podcast with asking the question about what you are pondering questioning or troubling now

Darby Roberts:
Either as a result of this conversation or just generally. And so Dan, why don’t we start with you closing us out with final thoughts?

Dan Bureau:
Sure. I will say that Student Affairs Now has given me lots upon during the last eight months. I’m a big fan and a recent podcast that you all recent episode you all did about ideal worker norms and the book for creating sustainable careers in student affairs is right there in my head right now, right. It’s after NASPA and after ACPA and listening to the stories of graduate students and new professionals and expectations about work. I, I want to figure out what is the agency that grad prep programs, professional associations institutions, as well as the students and professionals themselves have in creating a better work environment.

Heather Shea:
Yeah. That episode totally rocked my world. So absolutely recommend. And the, and the book, I have a copy on my shelf. Chris Wren is my mentor and on my campus, she brought me a copy and a mask one day. She’s like, so great. Thanks so much for sharing that. Darby, what about you? What are your, what are you whether you’re troublinh or thinking about now?

Darby Roberts:
Well, I will say what I’m thinking about relates to this social justice assessment and really campus climate. So on our campus right now we finished data collection from undergraduate students, graduate students, staff, and faculty about campus climate and fantastic data collection. We already know inherently that there are some issues on our campus we’re predominantly white that a lot of tradition and culture and impact to think about. But as the assessment person, what I really ponder is how do we get that data into the hands of the people who can actually make change?

Darby Roberts:
So I can tell you what the data means. I can analyze it, I can interpret it, I can present it. I can do a lot of things, but what I can’t do is go out and actually do the change. And so really working with my colleagues and thinking about, okay, how do we do that? And not only just, okay, we’re going to tell you what the campus climate is, but how are we promoting, encouraging supporting the program services, initiatives, whatever the case may be to actually make change, because that impacts student success. It impacts retention. It impacts sense of belonging. You know, and COVID aside of how that’s changed our world is we have to think about this and we have to do something because it’s no longer acceptable to just collect the data, tell people what it is is that’s, that’s not going to get us anywhere.

Darby Roberts:
And so I think it’s so much about how we use the data, how we empower people to make change and doing that at a huge institution to me is, is daunting. But I think we have people who can make a difference. We just need to work really hard to get that information out there and have some accountability and some imagination and innovation. So that’s what I’m thinking about.

Heather Shea:
That is a great point. I I agree, cause those surveys have a shelf life to write the reports that you’ve spent all the time writing, you know, in a couple of years that will not be relevant information anymore. So it’s actionable today. Nicole, what about you? What are you pondering or thinking about now?

Nicole Long:
Something that has been top of mind for me particularly is around racial equity and inclusion and institutional responses to it. I had a conversation with a colleague just a few days ago about, is this going to be a flash in the pan? Like other things, is this going, you know, is our focus in student affairs? Is this going to be the new sustainability? Like, you know, there seemed like a moment where it was all sustainability and now it’s kinda like we still talk about it, but it’s not like top priority necessarily. So, you know, I think about that and I think about that in relation to even some of the assessment work that we’re doing on our campus specifically related to campus culture, like the campus culture and environment that students are experiencing and doing some, you know, data analysis around that. So, you know, I just think is, is this just the hot topic? And particularly with assessment, is this the data that people want to hear? Well, people want to hear the results findings in a couple years from now and not just my campus, but just broadly in general, like, you know, is, is that something that, you know, we’ll still have the same priority level?

Heather Shea:
Well, that’s a really good point. Thank you so much, Patrick, what are your final thoughts or what are you pondering?

Patrick Biddix:
Oh, the super nerd of the group. I am very much a methodologist. I have always treated research methods and assessment as a puzzle that I really love solving for a long time back with my dissertation, I’ve played around with social network analysis and I am continuing to try to find applications in student affairs for how we draw on connectivity and relationships and networks, both to enable our work in the field. But also as I think it’s really finding some legs in DEI work. Now, as we continue to explore ways to support our Latin X community growing in Tennessee, what does it mean to have a community? What does the community look like? How does a community lift itself up and support each other? And what, how do we map a network and then utilize that network in a very intentional way? And so as I transition out of the, the current role I’m in and began to really think more specifically about networks and network analysis, I hope to build some tools to enable us to be able to do some work around understanding how the power of networks more so than we can right now.

Heather Shea:
I think you just pitched the new episode for Student Affairs Now I’d love to talk more about that. Very good, very good reference books. I love to hear more. That sounds absolutely fascinating. I’ve read some of your work on that. So thank you all of you so much for spending time with me today on Student Affairs Now, thanks also to our sponsors, Anthology and Everfi for those of you who are not currently subscribers, you can receive our newsletter in your inbox every Wednesday, by going to our website, studentaffairsnow.com, scrolling to the bottom and putting your email address in. If you listen to podcasts on iTunes, you can subscribe there. You can invite others to subscribe. And I know this is a little cheesy, but leaving a five star review really helps us keep conversations like this accessible and open so we can reach more folks and build a learning community is really what it’s about. Again, I’m Heather Shea, thanks for making thanks for all the fabulous guests and for everybody who’s watching or listening, make it a great week, everyone.

Books

 Frameworks for Assessing Learning and Development Outcomes