Dr. Brenna Butler, RMME instructor, presents at the AEA 2023 Conference. Dr. Butler shared two presentations at AEA 2023, including “Getting down to the essentials: How to effectively measure need on a needs assessment survey using a gap analysis approach” and “How can one-on-one client interactions be measured as organization-wide impacts?” Congratulations on these fantastic presentations, from the Research Methods, Measurement, & Evaluation Community!
Author and Presenter: Brenna Butler
Presentation Title: Getting down to the essentials: How to effectively measure need on a needs assessment survey using a gap analysis approach
Abstract: Measuring need through a needs assessment survey is often not easy, given the many different methods that are often used to define what a “need” is. In this Ignite presentation, participants will learn about one concrete way of measuring participants’ needs through a gap analysis, which measures both participants’ present state and their desired state (Watkins et al., 2012). Participants will be provided examples of survey questions to measure these states in a valid manner that is housed in a framework of a systems thinking approach, meaning that participants’ present and desired states under investigation are done so with personal, social, and societal influences in consideration (Arther & McMahon, 2005). The strengths and weaknesses of measuring need through this approach compared to other approaches will be briefly described. Participants will leave this Ignite presentation with one contextual framework to apply to their own needs assessment surveys along with a template of survey questions that can be structured to measure needs effectively.
Presenters: Brenna Butler, Michael Hamel, Matthew Spindler, Malinda Suprise
Presentation Title: Hidden stories: How can one-on-one client interactions be measured as organization-wide impacts?
Abstract: In large organizations, such as Cooperative Extensions, impacts that derive from one-on-one interactions with clientele are often missed in “hidden stories” due to a lack of effective data tracking measures present within the organization. This session will describe the methodology used to create a survey tool that captures the outputs and outcomes of educator-clientele interactions at Penn State Extension, focusing on supporting data utility at multiple levels (i.e., educator, supervisor, and leadership council). How the online survey built a foundation in the organization for data analytics supported through consistent and structured data input and storage processes will be described. Those data input and storage processes will facilitate future data analysis for complex decision-making throughout the organization’s evaluative cycles of activities and programs. The importance of stakeholder involvement in the development process of this survey as a form of boundary-making will be discussed in relation to maximizing the utility of data collection throughout the organization. Data collection is inherently a form of boundary-making that determines which elements of a situation should be included in the informational picture constructed of a context and which elements should be excluded (Schwandt, 2018). Boundary-making was used in this instrument development process to utilize the important role of stakeholder involvement in defining what information should be collected and curated (Archibald, 2020). Participants should leave this demonstration with the knowledge and tools to employ a similar methodology at their own organization to track individual interactions using a structure that allows for data aggregation at an organizational level.