On February 21, 2024, Dr. Bianca Montrosse-Moorhead (Associate Professor in RMME Programs) joined MERL Tech’s Natural Language Processing Community of Practice (NLP-CoP) to discuss the impact of Artificial Intelligence (AI) on future evaluation practice. This exciting panel also included other experts in the field, including Sarah Mason, Izzy Thornton, Tarek Azzam, Sahiti Bhaskara, and Blake Beckmann. The session covered a wide variety of topics, ranging from the birth of AI to consulting challenges and AI competencies. Click here for more information on the talk and to register for part two of this discussion series!

RMME is growing! We are currently seeking qualified instructors to teach graduate-level courses in Research Methods, Educational Measurement, Quantitative Methods, and Evaluation. Connecticut residents are especially encouraged to apply! Interested candidates should apply here.

UConn’s Research Methods, Measurement, & Evaluation (RMME) Programs are excited to celebrate our newest graduates from RMME Master’s degree program and RMME’s Graduate Certificate in Program Evaluation program! We cannot wait to see all of the many ways you will make us proud, as a new RMME graduate! Congratulations, all, from the Research Methods, Measurement, & Evaluation Community!

The Research Methods, Measurement, & Evaluation (RMME) program is proud to recognize this year’s recipient of the 2023 RMME Book Award: Erbin Huang! Congratulations on this fantastic accomplishment, from the Research Methods, Measurement, & Evaluation Community!

RMME Master’s program alumna, Dr. Merve Sarac, delivers a presentation with Professor James Wollack at the 2023 Conference on Test Security. Congratulations on this outstanding work, from the Research Methods, Measurement, & Evaluation Community!

Dr. D. Betsy McCoach (RMME Faculty member, Discussant), Amanda Sutter (RMME PhD Student, Presenter), Marcus Harris (RMME PhD Student, Presenter), Claudia Ventura (RMME PhD Student, Presenter), Faeze Safari (RMME PhD Student, Presenter), & Kirsten Reyna (RMME PhD Student, Presenter) present a symposium on “The Future of Educational Measurement” at NERA 2023. Congratulations on this excellent symposium, from the Research Methods, Measurement, & Evaluation Community!

Symposium Chair/Discussant: D. Betsy McCoach

Symposium Presenters: Amanda Sutter, Marcus Harris, Claudia Ventura, Faeze Safari, & Kirsten Reyna

Symposium Abstract: Emergent measurement scholars provide their perspective on current issues and future directions in educational measurement. The four presentations focus on 4 critical areas: 1.) Equity and social justice, 2.) The context in which we operate, 3.) The rise of artificial intelligence, and 4.) Graduate training in educational measurement.

Symposium Presentations:

1) Context Matters
Amanda Sutter, Marcus Harris

2) Equity and Social Justice Issues and Values in Measurement
Claudia Ventura, Amanda Sutter

3) Exploring the Challenges and Potential of Artificial Intelligence in Educational Measurement
Faeze Safari, Kirsten Reyna

4) Transforming Graduate School Training: Advancing Measurement and Open Science
Marcus Harris, Kirsten Reyna

Dr. Brenna Butler, RMME instructor, presents at the AEA 2023 Conference. Dr. Butler shared two presentations at AEA 2023, including “Getting down to the essentials: How to effectively measure need on a needs assessment survey using a gap analysis approach” and “How can one-on-one client interactions be measured as organization-wide impacts?” Congratulations on these fantastic presentations, from the Research Methods, Measurement, & Evaluation Community!

Author and Presenter: Brenna Butler

Presentation Title: Getting down to the essentials: How to effectively measure need on a needs assessment survey using a gap analysis approach

Abstract: Measuring need through a needs assessment survey is often not easy, given the many different methods that are often used to define what a “need” is. In this Ignite presentation, participants will learn about one concrete way of measuring participants’ needs through a gap analysis, which measures both participants’ present state and their desired state (Watkins et al., 2012). Participants will be provided examples of survey questions to measure these states in a valid manner that is housed in a framework of a systems thinking approach, meaning that participants’ present and desired states under investigation are done so with personal, social, and societal influences in consideration (Arther & McMahon, 2005). The strengths and weaknesses of measuring need through this approach compared to other approaches will be briefly described. Participants will leave this Ignite presentation with one contextual framework to apply to their own needs assessment surveys along with a template of survey questions that can be structured to measure needs effectively.

Presenters: Brenna Butler, Michael Hamel, Matthew Spindler, Malinda Suprise

Presentation Title: Hidden stories: How can one-on-one client interactions be measured as organization-wide impacts?

Abstract: In large organizations, such as Cooperative Extensions, impacts that derive from one-on-one interactions with clientele are often missed in “hidden stories” due to a lack of effective data tracking measures present within the organization. This session will describe the methodology used to create a survey tool that captures the outputs and outcomes of educator-clientele interactions at Penn State Extension, focusing on supporting data utility at multiple levels (i.e., educator, supervisor, and leadership council). How the online survey built a foundation in the organization for data analytics supported through consistent and structured data input and storage processes will be described. Those data input and storage processes will facilitate future data analysis for complex decision-making throughout the organization’s evaluative cycles of activities and programs. The importance of stakeholder involvement in the development process of this survey as a form of boundary-making will be discussed in relation to maximizing the utility of data collection throughout the organization. Data collection is inherently a form of boundary-making that determines which elements of a situation should be included in the informational picture constructed of a context and which elements should be excluded (Schwandt, 2018). Boundary-making was used in this instrument development process to utilize the important role of stakeholder involvement in defining what information should be collected and curated (Archibald, 2020). Participants should leave this demonstration with the knowledge and tools to employ a similar methodology at their own organization to track individual interactions using a structure that allows for data aggregation at an organizational level.