Evaluation Plan by Alicia Lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir Overview en-us 2017-11-10 14:41:32 UTC 2024-10-13 13:54:48 UTC hello@padlet.com Program Description alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205706816 The Career Exploratory program is a dual effort between the Career Magnet School (CMS), and the Franklin County Career and Technology Center (FCCTC), that shares the building with CMS. The Career Exploratory program is for current 9th graders attending CMS and any incoming 10th and 11th graders to CMS, who are not already accepted into FCCTC. This program allows students to choose six of the shop programs available at FCCTC (ex. Carpentry, Pre-Engineering, Electronics, etc.) to observe and participate in to determine whether it is a shop they are interested in attending. Students observe that shop for one period (40 minutes), for three consecutive days. While there, they interact with students already in the shop to complete activities/tasks assigned in that shop. They do this six different times to observe the six shops they requested at the beginning of the program. After students complete each round of observations, they are required to complete a survey in regards to their experiences in that shop. These surveys are used to identify areas of improvement for each specific shop, and the program in general. Upon completion of the program, students earn half an elective credit and may apply and interview for a position in the shop of their choice.  This program is not mandatory for students, nor is it a requirement to apply and be accepted to the Franklin County Career and Technology Center.]]> 2017-11-10 14:47:34 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205706816 Rationale alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205707662 According to Patton (2008), “Program evaluation is the systematic collection of information about the activities, characteristics, and results of programs to make judgements about the program, improve or further develop program effectiveness, inform decisions about future programming, and/or increase understanding.” (as cited in Russ-Efft & Preskill, 2009, p. 4). The Career Exploratory program is in its sixth year of operation and has not had many changes made to the program, and minimal data has been collected. Since the initial launch of this program the only changes that have been made are to the number of shops that students observe and the items on the survey given at the end of each observation. Data has been collected on the shops students initially observed and those chosen as their career path, however, this data is limited and does not include interviews of program participants. ]]> 2017-11-10 14:49:22 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205707662 Why create a logic model? alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205708796 Creating a logic model allows one to better understand the Career Exploratory Program, as well as, its resources, activities, and goals. According to Russ-Eft and Preskill (2009), “By developing a logic model of the program to be evaluated, the evaluator and stakeholders gain an understanding of the evaluand’s background and the evaluation purpose.” (p. 223). A logic model, provides a visual representation of the program, and allows stakeholders, those with a vested interest in the program, to determine areas in need of evaluation and the purpose for that evaluation.]]> 2017-11-10 14:51:54 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205708796 Career Exploratory Logic Model alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205709581 2017-11-10 14:53:36 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205709581 Program Goals alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205711631 The major goal of this program is to provide the opportunity for students to see first-hand what the shop entails and the skills they will be required to know and learn, in order to allow students to select a shop that will maximize their chances of success in the career field of their choice. Unfortunately, the number of students who think they are interested in a specific shop, until they see first-hand what that shop really entails, is higher than one might assume. Another goal of this program is to ensure that students feel comfortable about their shop choices, which in turn eliminates or at least lowers the number of students who switch or drop shops after acceptance into FCCTC. Switching and dropping shops can cost the district money. A final goal of the program is to allow students currently attending the shop, the chance to practice their skills and teach others those skills. ]]> 2017-11-10 14:57:42 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205711631 Key Evaluation Questions alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205711865 After taking into consideration the program description and the rationale or purpose for evaluating the program, one can then determine the key evaluation questions that will drive the evaluation. Based upon the information provided previously, the questions below would be the key evaluation questions for this particular evaluation.
1.     How effective do the current and past participants feel that this program was in helping them choose a shop?
2.     What areas of this program are in need of improvement in order to make the program more effective?
]]> 2017-11-10 14:58:14 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205711865 Why collect data? alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205713446 Evaluating this program and collecting data within that evaluation would allow for improvements to be made. First an evaluation would allow for data to be collected on program effectiveness in aiding students in selecting a shop that will assist them in future career paths. This could be completed in the form of surveys and interviews to determine how past participants felt about the program’s effectiveness, as well as current participants. Second, evaluating this program would allow data to be collected on areas of the program in need of improvement. Again, interviews would be beneficial in allowing all students, past and current, to provide feedback as to any areas that are in need of improvement. ]]> 2017-11-10 15:01:59 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205713446 Survey alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205714444 The first data collection method would be to provide an online survey, via email, to both the current participants of the program, as well as, the past participants of this program who are still attending both the Career Magnet School and the Franklin County Career and Technology Center. The first two questions will distinguish current participant from past participant. The survey questions three thru five would allow data to be collected on whether current and past students ended up applying to the same shops they had originally intended on applying to, or whether after observing the shops, changed their minds and applied to other shops. Then questions six thru nine would allow data to be collected on how effective the time allotted in each shop was, and any areas of the program that need improvement. Finally, the last two questions would be a way to determine the students who are willing to participate in the one-on-one interview and provide names for a random sampling to complete that second data collection.

Survey
The purpose of this survey is to collect data on the effectiveness of the Career Exploratory Program and to determine areas in need of improvement. Data collected will be utilized to make necessary changes to the program.

Directions: Please circle your choice.
1.     Are you planning on applying to Franklin County Career and Technology Center for the 2018-2019 school year?
a.     Yes
b.     No
2.     Are you currently attending Franklin County Career and Technology Center?
a.     Yes
b.     No
3.     Prior to completing the Career Exploratory Program which three of these programs were you planning on applying to? Please rank your three choices in order of preference from 1st to 3rd by placing that number next to it in the list below.
a.     HVAC
b.     Carpentry
c.     Building Construction Trades (BCT)
d.     Electrical Occupations
e.     Plumbing
f.      Allied Health
g.     Dental Assisting
h.     Medical Assisting
i.      Veterinary Assisting
j.      Cosmetology
k.     Culinary Arts
l.      Graphic Communications
m.   Landscaping & Horticulture
n.     Marketing/Web Design
o.     Mechatronics
p.     Electronics
q.     Engineering Technology
r.      Information Systems and Technology
s.     Computer Integrated Manufacturing
t.      Welding
u.     Agriculture Mechanics
v.     Automotive Collision Repair
w.   Automotive Technology
x.     Diesel Technology
4.     Which 6 shops did you observe while participating in the Career Exploratory Program?
5.     Upon completing the Career Exploratory Program, which of three these programs did you apply for? Please rank your three choices in order of preference from 1st thru 3rd by placing that number next to it in the list below.  
a.     HVAC
b.     Carpentry
c.     Building Construction Trades (BCT)
d.     Electrical Occupations
e.     Plumbing
f.      Allied Health
g.     Dental Assisting
h.     Medical Assisting
i.      Veterinary Assisting
j.      Cosmetology
k.     Culinary Arts
l.      Graphic Communications
m.   Landscaping & Horticulture
n.     Marketing/Web Design
o.     Mechatronics
p.     Electronics
q.     Engineering Technology
r.      Information Systems and Technology
s.     Computer Integrated Manufacturing
t.      Welding
u.     Agriculture Mechanics
v.     Automotive Collision Repair
w.   Automotive Technology
x.     Diesel Technology
6.     Do you feel the time allotted in each shop to observe was enough? 
a.     Yes
b.     No
7.     If you answered No to the question above why not?
8.     Were there any areas of the program that you feel could be improved upon?
     a. Yes
  b. No
9.     If you chose Yes for Question #6, what areas do you feel need improvement?
10.  What grade are you currently in?
a.     9th
b.     10th
c.     11th 
11.  Would you be willing to be interviewed one-on-one for further questions?
a.     Yes
b.     No
If you answered Yes to the above question please provide your name ______________________
]]> 2017-11-10 15:04:45 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205714444 One-On-One Interviews alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205714649 A second data collection method that should be utilized in this evaluation to gather more specific information on the areas of the program that need improvement would be to conduct one-on-one interviews. The reason this would be more effective than a focus group is that people, especially teenagers, are more apt to share their true feelings when there are not others around. The one-on-one interviews could be done with current and past participants, as well as current instructors and administrators. Observations of the program in action in the various shops, would also be beneficial in collecting data on areas in need of improvement.]]> 2017-11-10 15:05:17 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205714649 Stakeholders alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205715103 According to Russ-Eft and Preskill, it is important when conducting an evaluation to consider the stakeholders, or in other words, determine who has a vested interest in the evaluation and the program being evaluated (2009). When examining the Career Exploratory Program there are numerous stakeholders due to there being more than one organization involved in this program. The stakeholders from the Career Magnet School (CMS), would be the current and past participants of the Career Exploratory Program, as well as the administrators for CMS. The stakeholders from the Franklin County Career and Technology Center (FCCTC) would be the shop instructors, the FCCTC administration, and the Program Coordinator. One could also argue, that since these students eventually become employees in the community, that the community itself, including the local businesses, would also be stakeholders. ]]> 2017-11-10 15:06:24 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/205715103 Data Analysis alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/206025240 By utilizing an online survey form, such as Survey Monkey, the data can be collected and tallied quite easily. That would allow for more time to be spent analyzing the patterns that emerge within and between the current and the past participants in regard to program effectiveness and needs. The results could be displayed in a table, and key areas could be turned in to pie or bar graphs for a more visual representation of that specific data. Survey data could also be utilized in creating the one-on-one interview questions, in order to be able to drill down further into what was or was not effective about the program and the areas in need of improvement.

According to Taylor-Powell & Renner, when analyzing narrative data, such as that from the one-on-one interviews, it is important to look for emerging themes or patterns in the data provided in the narratives, especially that which relates to the key evaluation questions (2003). For this program, that data would include whether students selected the same shops after the program to apply to that they had originally planned on, as well as whether the time allotted in shops was sufficient and areas they felt were in need of improvement.]]> 2017-11-12 19:47:05 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/206025240 References alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/206027665 Russ-Eft, D. and Preskill, H. (2009). Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance and Change. New York, NY: Perseus Books.     

Taylor-Powell, E. & Renner, M.   (2003). Analyzing qualitative data. [Supplemental material]. Program Development and Evaluation, p.2.]]> 2017-11-12 20:05:20 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/206027665 Narrative alicia_lippert https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/206031555 2017-11-12 20:36:07 UTC https://padlet.com/alicia_lippert/jmyjz5n4f3ir/wish/206031555