Program Evaluation

Vince Commented on Group 1 and Group 2

Program Evaluation

Troy Rector

and

Vincent Stults

Ball State University

November 9th, 2014

EDAC 634

Roles: Vince - intro, Josh Arthur's profile and review, and reflected on review in the conclusion

           Troy - joint conclusion, Dr. Marianne Wokeck's profile and review, and reflected on review in the conclusion 

Introduction

What follows is a program evaluation regarding our program design that was completed for our last month's assignment. Our program design was based upon transformational learning theory.  It was based on the idea that transformational learning can occur when student’s personal, structural, historical, and programmatic experiences directly contrast with their prior thoughts and conceptions (Kiely, 2005). We chose to use cultural immersion as the trigger to lead learners into an understanding that is not simply informational, but that significantly changes what they know, moving them through the transformational learning cycle of "experience, critical reflection, rational discourse, and action" (Merriam et al 2007, p 134). We chose adult learners at Ivy Tech Community College to be our participants, inviting them on a learning journey to seek answers to their own learning quest, about themselves, their social context, or society, through facilitating learning opportunities in a cross-cultural context (the Arab community of Dearborn Michigan) with the goal of seeing students become better equipped to be leaders in a global society. 

We sent this program design to two educators who we believed could provide critical feedback on our project design and who had the breadth of knowledge and experience to effectively evaluate the scope and purpose of our design.

Josh Arthur's Profile

For the past 12 years, Josh Arthur has been engaged in initiating and building local community development and educational programs in East Central Indiana. He is currently the Regional Director, ASAP at IVY Tech, for East Central Indiana, which is a newly formed program to assist students in finishing their associate degree in one year as an alternative to the two year traditional path. "He has a passion for holistic student success and is committed to the academic, personal, formational, and vocational growth of each student" (Muncie Free Press, 2014). 

Josh Arthur's Review

Arthur's first response focused on the intended target participants and the scope of the program. He commented that, "As an educator at Ivy Tech Community College, this formational trip fills a large gap in the needs of Ivy Tech students—both those pursuing liberal arts and vocational trades.  Regarding the personal benefit to students, he added that "These types of learning experiences are beneficial personally, professionally, civically, and financially." Followed by reflection on his own personal experience, “In fact, I had very similar educational experiences organized by my undergraduate professors, and I cannot overstate their importance to my own development—even 15 years later."

In his review of the design, he offered a couple of suggestions for improvement.  First he recommended that it would be good to include students, "behind the scenes of the design" exposing them more to the "reasons" of its development. Secondly,  he encouraged us to build into the design, "an additional outcome for students to connect other courses' class work to the experience" and to provide them a means to explore how the experience impacted them "holistically—personally, politically, relationally, financially, academically, family of origin, civically, artistically, architecturally, nutritionally, etc."

He appreciated several design factors, such as "the use of evaluative tools for gauging student outcomes" and "using direct teaching methods to inform students of the total process." He further added that, "The project is well organized in such a way as to garner the students' trust in the leaders and the experience." He also thought the min-ethnography was useful process.  Overall he added that, "A project like this creates a needed bridge to development that current academic foci on data and information have lost." 

Dr. Marianne Wokeck Profile

Dr. Marianne Wokeck is a Chancellor’s Professor of History at Indiana University Purdue University Indianapolis (IUPUI).  She is also the director of the Institute for American Thought located at IUPUI.  In addition, she is an associate dean for academic affairs within the School of Liberal Arts. 

 

Dr. Wokeck’s education began in her native Germany where she was educated at Staatsexamen Hamburg University, graduating in 1973.  She went on to receive her Ph.D. from Temple University in 1982.   She has received numerous academic accolades in her career to include Chancellor's Professor in 2009, the Alwin S. Bynum Award for Excellence in Academic Mentoring 1996, the Oustanding Academic Advisor in years 1993, 1997,and 2005, the IU Teaching Excellence Recognition Award 1997, National Endowment for the Humanities grants (Biographical Dictionary of Early Pennsylvania Legislators, 1986-91,  The Works of George Santayana, 2003-20066), a Senior Fulbright Scholar (Germany) 1997-98, and Outstanding Female Faculty 2003, 2004

Dr. Wokeck’s publications include Trade in Strangers: The Beginnings of Mass Migration to North America(Penn State University Press 1999), (editor and author) Lawmaking and Legislators in Pennsylvania, vol. 1 (University of Pennsylvania Press 1991), (editor) The Papers of William Penn vols. 3-4 (University of Pennsylvania Press 1986, 1987), numerous articles in scholarly books and journals, and (editor with others) The Letters of George Santayana, volume 5 in 8 books of The Works of George Santayana (MIT Press). 

Dr. Marianne Wokeck’s Review

Dr. Wokeck began her review by stating that she did not know the particulars of the assignment but she found the current order awkward, leaving the reader with little guidance. She stated that we may want to consider rearranging our materials (since the proposal is not paginated, it is difficult to make reference to particular incidences).

Dr. Wokeck analyzed the introduction and concluded that it should start out with your goal and be clear in the audience for whom you write as well the following critiques:

• proposing an experiential learning experience in response to needs articulated by adult learners, who are employed by corporations with connections to and interest in Arabic and/or Muslim partners and clients 
• Creating an experiential learning experience that uses a cycle of instruction, information gathering, readings, discussion, activities, and reflection that, ideally, is transformational 
• Using intensive cultural immersion as the method for creating that experiential learning experience 
• State the objectives clearly 
• State the learning outcomes clearly 
• Indicate the evaluation methods and measures

Dr. Wokeck’s suggestion is to state the goals clearly and then provide the references from our literature review that are pertinent for each point. In her opinion, the experience of our literature review are secondary and can go into a footnote or appendix (preceding the bibliography, for example).

It was also her opinion that we are much more explicit and expansive in our interest in devising a cultural immersion experience than in sketching out how that cultural immersion would actually work. For example she asked, why do you choose certain places, foods, representatives? How do you vet them for the appropriateness of achieving the learning outcomes? As someone who has overseen comparable oversees programs and cultural immersion experiences I would want to know more about those considerations and selections. 

She elaborated further about the literature cited and said that much of the literature that we cited is jargon-porne, those professional habits have made it into our own words. She went on to say that unless our audience are only other educators, that kind of jargon-laden language detracts from what we want to convey. 

Conclusion and Reflection 

Arthur's qualifications in community development and education, and his current role at IVY Tech make him an ideal candidate for evaluation of this program. His response was encouraging, balanced, and helpful. Arthur's affirmation of our intended focus on students at IVY Tech, demonstrated that our program would be valuable for students in that context. He also revealed important design considerations. Each recommendation for improvement would provide a more meaningful experience for the student participants. The challenge to including students in the reasoning behind the design process would need to be integrated into our pre-field preparations as a part of the students first meeting and might also include additional pre-reading materials.  Creating an additional learning outcome to connect the experience to their previous classes and for evaluating the experiences impact on their lives in a more "holistic" manner could prove challenging, but would add benefit to the students and their advisory teams.

Dr. Wokeck’s credentials and professionalism speak for themselves.  She has reviewed, counseled, and conducted performance reviews on other professors.  I not only value her opinion and comments, but fully understand that she does not “sugar coat” anything she analyzes.  She gives a firm yet fair evaluation to anything she is asked.  

We fully intend to take her recommendation on the details of the culture immersions.  In her professional opinion, we didn’t elaborate on how the cultural immersion would actually work and with some detailed instructions, guidance, and explanation, this can be accomplished.  We also intend to use her comments to further “professionalize” our project.  What can be viewed as jargon or normal language use, can be “professionalized” and “cleaned up” to reflect the education level as well as adult educators that we’ve become.  

Both reviewers came from different backgrounds but brought valuable and helpful reviews for our project.  Both of them were fair yet firm and didn’t appease us or the project.  They were hard on us when they needed to be yet professional and gave us suggestions and improvement ideas alongside their criticism.  This professional review will aide us greatly as we move forward with this project.  We can see clearly some of the errors and mistakes that can be corrected, and we can move forward to creating an adult education program that is both professional yet informative.

References

Kiely, R. (2005). Transformative international service learning. Academic Exchange Quarterly, 9(1), 275-281.

Merriam, S. B., Caffarella, R. S., & Baumgartner, L. M. (20007). Learning in adulthood: A comprehensive guide. John Wiley & Sons.

Muncie Free Press. (2014, January 24). Ivy Tech Launching One Year Associate Degree Program. Retrieved November 9, 2014, from http://www.munciefreepress.com/node/30167