# data[].attributes.body
jsp325@drexel.edu

November 7, 2016
EDUC 804 Program Evaluation in Organizations
Dr. Craig Bach
Gallaudet University Department Evaluation Plan

# data[].attributes.body
I think your padlet looks great! I love how you addressed the question of why you are evaluating this and will for sure add a section like this to my padlet! I also loved how you added a section to list possible evaluation issues which is also something that I am thinking of incorporating into my padlet. I am intrigued by  Gallaudet University and its purpose. I had never heard of it before prior to you Jennifer so thank you for introducing me to it!

# data[].attributes.body
You demonstrate a high degree of understanding our reading assignments by pulling in the various  steps in evaluation planning. I like how you included the "what, why and how" at the top of the padlet.  The one thing I did not notice was your logic model.  I am not sure if you are finding this to be a challenge or not, but how to display the "postings" in some kind of order that makes sense to the reader.  I wish padlet had a few more options to accessorize our presentation and help direct the reader so it flows better....
Since I am not familiar with the education system....but jut thinking of healthcare policy/procedures we have best practices or evidence based practices that we research to see what other organizations are doing.  Is this something that you could do to assist in making sure you operations policies and procedures are in line with other academic departments are doing in comparable universities?
Great job!!!

# data[].attributes.body
Merriam-Webster (2016) Definition of OPERATION. Available at: http://www.merriam-webster.com/dictionary/operation (Accessed: 3 November 2016).

Merriam-Webster (2016) Definition of QUALITY. Available at: http://www.merriam-webster.com/dictionary/quality (Accessed: 29 October 2016).

Russ-Eft, D. and Preskill, H. (2009) Evaluation in organizations: A systematic approach to enhancing learning, performance, and change - 2nd edition. 2nd edn. New York:Basic Books.

Stitt-Bergh, M., Rickards, W.H. and Bertrand Jones, T. (2016) ‘Beyond the rhetoric: Evaluation practices in higher education’, New Directions for Evaluation, 2016(151), pp.123–132. doi: 10.1002/ev.20203.

The Trustees of Indiana University (2016) National Survey on Student Engagement. Available at: http://nsse.indiana.edu/pdf/survey_instruments/2016/NSSE_2016-US_English.pdf (Accessed: 30 October 2016).

# data[].attributes.body
Before analysis begins, a matrix will be developed to track each key evaluation question, the various forms of data to be collected and how the two intersect.

I will confess at this point to being a bit overwhelmed by the different forms of analysis and what the best approach for analysis is for this project. It has occurred to me that collecting the department policies and procedures as well as the mission statement might be better preliminary steps in the process before collecting and analyzing any of the other data. It may be that analyzing these documents will provide a framework from which the remaining data can be evaluated, but that their evaluation will not specifically answer any of the key evaluation questions. Hopefully further reading in the next few weeks will help clarify this for me.

The faculty and chair surveys data will be distributed electronically through either BlackBoard or Survey Monkey. The results will be evaluated for ways in which the responses answer key questions #2 and #3.

Results from the National Survey on Student Engagement will be analyzed separately from the rest of the quantitative data. It is our hope that Gallaudet will allow us access to the raw data so that the students can be separated out by department.

# data[].attributes.headline
Difficulty using results to make improvements

# data[].attributes.subject
Difficulty using results to make improvements

# data[].attributes.body
In a large institution, it can be difficult to get things done. There is the possibility that the administration will have difficulty implementing any changes to address the needs identified by this evaluation, which will prove frustrating for those that participated in it as well as for the evaluators.

# data[].attributes.body
There is a potential that the quantitative data will prove to be excessive. In that case, the evaluation plan will have to be adjusted to identify which archival data is essential to answering the key evaluation questions.

# data[].attributes.body
Stitt-Bergh, Rickards and Bertrand Jones (2016) describe faculty resistance in this way: “In general, faculty are not willing to participate in evaluation for evaluation’s sake or only to satisfy external compliance requirements, but they are willing to have useful, productive conversations about student learning...Faculty participation in assessment is crucial.”

To combat this resistance, evaluators will hold an initial kick-off meeting before any evaluation activities begin. This meeting will be casual and informational to help stakeholders feel comfortable with the process and with how the data will be used. The hope is that if they understand how and why the data will be used (to improve their department and not to be punitive in any way) they will be more engaged in participating in the evaluation.

# data[].attributes.body
Faculty Resistance

Too much data

Difficulty using evaluation results to improve departments

# data[].attributes.body
Two surveys will be designed as part of this evaluation plan.

The first will be a survey for faculty in order to assess their level of engagement and satisfaction with the department. Questions will be designed to evaluate the level of communication and support the faculty member feels from department leadership, as well as their communication and interactions with their student advisees.

The second survey will be for the department chairs and program coordinators. A few of the larger departments have a chair as well as a program coordinator and both will be asked to complete this survey. This survey will include all of the questions from the faculty survey in addition to more specific questions related to some of the tasks that only chairs/program coordinators are responsible for.

# data[].attributes.body
Department Documents

Policies & Procedures

Unit Effectiveness Plan

Mission

Department Records

Employee Records

Budget records

Course Data

Number of Courses

Enrollment

Ratio Student/Faculty

Ratio Advisees/Faculty

Employee Data

Number of Faculty

Full-time vs. Part-time

Demographics of Faculty

Student Data

Number of majors/minors

Average GPA

Number of Withdrawals

National Survey on Student Engagement results

# data[].attributes.headline
Wait, what does "quality of operations" even mean?

# data[].attributes.subject
Wait, what does "quality of operations" even mean?

# data[].attributes.body
Quality:
A high level of value or excellence.

Operations:
Performance of a practical work or of something involving the practical application of principles or processes.

For an academic department then, quality of operations means how well is the department run on a day-to-day basis. Is administrative paperwork turned in on time, are meetings held regularly and with some degree of organization, are students given guidance in completing required paperwork, are faculty supported administratively in travel requests as well as promotion, tenure and reappointment processes?

# data[].attributes.body
In what ways can the quality of operations within a department/program be improved through training or additional resources?

# data[].attributes.body
To what extent do students and faculty feel supported and encouraged by the culture and community of their department/program?

# data[].attributes.body
In order to answer the key evaluation questions, my evaluation plan uses mixed methods, both qualitative and quantitative.

# data[].attributes.body
How well a university department operates has a big impact on the sense of community and satisfaction level of everyone involved with the department - students, faculty, and staff. Having a better understanding of the quality of a department's operations will help identify improvements that can be made, the satisfaction level and ultimately the retention and academic success of students.

# data[].attributes.body
This is a plan for evaluating the quality of operations in academic departments and programs in the College of Arts and Sciences at Gallaudet University.

# data[].attributes.body
Gallaudet University, located in Washington, D.C., was established by an act of Congress in 1864 and was signed into law by President Lincoln. It was established to serve the educational needs of deaf and hard of hearing individuals (Gallaudet University, 2015, p.25).

Gallaudet is considered to be an integral part of the deaf community - especially regarding employment opportunities, new research, deaf history, deaf culture and American Sign Language. It is a bilingual learning environment, using both American Sign Language and English, providing full access for all students.

# data[].attributes.body
To what extent do the operations and activities of the identified department/program meet its stated goals and outcomes?

# data[].attributes.body
(a) What are the important details about the program you are focusing on?; (b) What are the key focus questions for the evaluation?; (c) How do your data collection and analysis strategies reflect your key questions?; and (d) what are some of your concerns about your plan?  The voice and/or video application will allow you to create a link that you then easily post to your Padlet Evaluation Plan Wall. Your voice narrative should not go beyond 5 minutes. Make sure you introduce yourself (name and topic) and then explain your map.