EvaluateUR Modifications Fit Community College Student & Faculty Needs

by

Jill K. Singer, EvaluateUR Lead Director

Jill K. Singer, a State University of New York (SUNY) Distinguished Teaching Professor in the Geosciences Department at Buffalo State University, currently leads two Advanced Technological Education (ATE) projects that are developing new variations of the EvaluateUR Method.

The EvaluateUR Method measures student learning by providing statistically reliable assessments of student growth in a wide variety of outcome categories, including disciplinary and employability skills. A unique feature of the EvaluateUR Method is the growth of student academic self-awareness, often called metacognition, that the method fosters.

The EvaluateUR Method is implemented online and serves as a tool for students, helping them build that awareness and use it to improve their learning and problem-solving skills. The website resources include a series of short exercises designed to foster metacognitive practice.

The two current ATE projects examine the learning outcomes of aspiring technicians whose community and technical college experiences include independent or course-based research, internships, and academic competitions.

“The idea is to help the students really take the lead and responsibility about what they're doing and how they're doing it,” Singer said. The ATE-supported expansions of the EvaluateUR Method are designed to address students’ various experiential learning opportunities and the time constraints of community and technical college faculty who are preparing technicians for advanced technology careers.

A New Approach to Evaluate Student Learning Outcomes Resulting from Participation in Remotely Operated Vehicle Competitions (Award Abstract 1932929) has developed E-Compete for students participating in engineering design competitions. While initially developed to support MATE’s remotely operated vehicle (ROV) competitions, it has been expanded to support other collegiate competitions including the Society of Automotive Engineers Baja competition. In a Journal of Advanced Technological Education article, Singer and five co-authors reported “a primary benefit of the method is that it encourages students to become more aware of what learning strategies they employ to analyze and solve problems.”

Extending the EvaluateUR Method to Expand the Community of Users project (Award Abstract 2325397) is introducing new options that add flexibility to the method and eliminate barriers that in the past have limited two-year college faculty from using the EvaluateUR Method. (Read more about this project in the full ATE Impacts Blog post.)

This spring the project is launching a community forum where faculty can post questions about the EvaluateUR Method and receive advice from other educators who are using it. Singer encourages two-year college faculty who are interested in using the EvaluateUR Method to explore the program’s website and email her at [email protected].  

--

Pilot Testing of New EvaluateUR-CURE (E-CURE) Options at Whatcom Community College

Faculty at Whatcom Community College, which has 10-week quarters, have been helping to pilot test some new options in E-CURE, one of the variants of the EvaluateUR Method designed for course-based undergraduate research courses (CUREs) in geology and engineering. One of the options involves only the students scoring the assessments and another option is designed for smaller scale or shorter duration research projects.

The Whatcom faculty members are assessing their students on the outcomes that best align with their course goals from the options listed on EvaluateUR and EvaluateUR-CURE Outcome Components. They can add course-specific options if they choose, as well. The participating students’ research projects incorporate both disciplinary knowledge and employability skills. Singer uses the term “employability skills” rather than “soft skills” to reflect the high value that employers place on workplace skills such as teamwork, communication, and problem-solving.

There is no charge for using E-CURE and faculty may request a course dashboard by completing a short online form. The project team also provides onboarding resources as well as general and technical support. 

New Approach for Evaluating Internships

The project is also developing a variant for students participating in internships. This new variant recognizes that while industry employers welcome having interns from community colleges, they are sometimes uncomfortable assessing the students’ knowledge and skills using EvaluateUR-Internship’s online assessments.

Singer described how E-Internship, the latest addition to the Evaluate-UR Method, is being pilot tested at Hudson Valley Community College and Portland Community College.

Before starting their internships, students participating in the project answer a set of open-ended questions that encourage them to think about their internship experiences. Students then score themselves on the set of outcomes that the internship coordinator has identified. After submitting their responses, the students receive an automated email that lists things for them to think about as they embark on their internships. It also prompts them to schedule a meeting with their internship supervisor to talk about internship outcomes and workplace expectations. After that meeting, the students return to the Evaluate-UR online system to answer a few reflective questions. This process is repeated at the mid-point and at the end of the internship. 

According to Singer, E-Internship helps students think about what they are learning and how information from their coursework is applied to address real-world problems. The design encourages students to have periodic conversations with their supervisor, but the built-in prompts prepare them for these conversations. The students are also encouraged to talk to other people they interact with during their internships.

The Origins of EvaluateUR

Singer began work on EvaluateUR more than 20 years ago when, as director of the Undergraduate Research Office at Buffalo State, she wanted to document the impact of the college’s investment in the students conducting research. She and Daniel Weiler developed the methodology that grew into EvaluateUR.

Successful use of EvaluateUR at Buffalo State led to grants for scaling its use at other higher education institutions and involved close collaboration with the Science Education Resource Center (SERC) at Carleton College where the EvaluateUR method is hosted. Sean Fox, the technical director at SERC, is responsible for developing the code that underpins the EvaluateUR method.

Articles about EvaluateUR include a chapter in The Cambridge Handbook of Undergraduate Research doi:10.1017/9781108869508.021 and an article in the Journal of Engineering Technology.

» Read More or Comment

How a Demographic Cliff Could Reshape Community Colleges

by

A black and white photo of the back of a student, raising their hand

The Demographic Cliff and Its Impact on Community Colleges
The United States is experiencing a significant decline in the number of college-aged students, a phenomenon known as the "demographic cliff." This shift, driven by falling birth rates since the Great Recession, is expected to result in a 13% drop in high school graduates by 2041—nearly half a million fewer annually, according to the Western Interstate Commission for Higher Education (WICHE). For community colleges, this presents a dual challenge of reduced enrollment and financial strain, as many institutions rely on a steady flow of traditional students to sustain their operations.

Adapting to a Shrinking Pool of Students
Community colleges, however, have an opportunity to adapt by reaching out to nontraditional student populations. This includes working adults and Hispanic students, the latter of whom are expected to represent a growing proportion of high school graduates in the coming decades. By focusing on these groups, community colleges can expand their role in providing access to education while maintaining enrollment levels.

» Read More or Comment

EPCC Project Combines Adversarial Thinking With Peer-Led Team Learning, Hackathons & Workshops

by

Ivan Alonso, EPCC graduate and peer leader, speaks during a plenary session at the 2024 ATE PI Conference.

Ivan Alonso is an undergraduate who has been involved in multiple facets of ongoing research thanks to his role as a peer leader in the computer science program at El Paso Community College (EPCC).

EPCC’s Developing Computational Adversarial Thinking project (National Science Foundation Award Abstract #2300378), led by Computer Science Professor Christian Servin, explores what happens when adversarial thinking is infused in three introductory computer courses. The project is also studying the impact of experiential learning opportunities on student recruitment and retention.

“Adversarial thinking is a way to think ahead or to prevent potential attacks or capabilities that can happen to your code,” Alonso explained of the combination of cybersecurity skills in software engineering courses during the student poster session at the 2024 ATE Principal Investigators’ Conference in Washington, D.C. A peer leader for one year and then leader of the peer leaders another year, Alonso’s scientific poster included displays of data from several cohorts of EPCC students.

“So we did a kind of an experiment where we had two groups of students. One group that took the courses, just normal courses without the adversarial concept, and then this group that took the courses with the adversarial-thinking concept and the statistics showed that students that took the courses with the adversarial concept implemented more secure programs,” Alonso said, pointing to graphs on his poster.

Peer-led help sessions have been used by EPCC’s computer science faculty since 2013. The Peer-Led Team Learning model started at the City University of New York and has been used in various science, technology, engineering, and math programs across the country.

EPCC faculty and others have found that students have been willing to ask questions of peer leaders that they have been shy about posing to faculty in class or during office hours. At EPCC peer leaders meet in person or online with up to 15 students at a time to work on activities that reinforce concepts taught by the instructor. Participation in these sessions counts toward 10% of students’ grades in the three introductory computer science courses. 

With the ATE grant, EPCC’s computer science program has gained new industry partners who are providing information about real-world challenges. Servin, who is the project’s principal investigator, has incorporated this information in activities for the peer-led team learning sessions as well as for hackathons and workshops that are being supported by the ATE grant. (The grant pays for half of the peer leaders’ wages; half of their wages are paid by the college’s Student Technology Services program.)

Servin said he has found that experiential learning activities “can help the students to get motivated and also increase their sense of belonging.”

» Read More or Comment

 

ATE Impacts is also a book! Copies are available upon request or at the annual ATE PI Conference in Washington, DC.

ATE Impacts also has a video series, that tells the stories of students, educators, administrators, and industry partners who have had their lives positively impacted by the work of the ATE program.
Have an ATE story to tell?
Email us at [email protected]
Creative Commons License The ATE Impacts blog is licensed under a Creative Commons Attribution-ShareAlike 4.0 license. You are free to share, copy, distribute, transmit, and adapt this work, provided you attribute it to the Internet Scout Research Group. If you alter this work, you may distribute your altered version only under a similar license.