Evaluation of ATI Pharmacology 3.0 by Donna

PROGRAM DESCRIPTION:

dtdarcy — 2015-10-31 01:18:42 UTC

The program under evaluation is a computer-assisted learning program, known as Assessment Technologies Institute (ATI®) NCLEX-RN® Success Package, used at our university as a supplement to the course content, textbooks and quizzes/exams. One component of this package is known as Pharmacology Made Easy 3.0® used exclusively in NURS 315 Pharmacology for Nursing. Students spend 2-3 hours per week on assigned tutorials, post-tests, practice assessments, and the end-of-semester proctored assessment. The faculty has access to all student activities and grades, and are provided trended data benchmarked against participating schools. My evaluation would focus on student perception of value of this program in terms of time and financial resources and assistance in achieving personal academic goals.

PURPOSE OF THE EVALUATION:

dtdarcy — 2015-10-31 01:20:00 UTC

This evaluation will determine the effectiveness of the Assessment Technologies Institute (ATI®) Pharmacology 3.0 Tutorial program in the nursing program at one suburban university on Long Island. This evaluation will determine if the program is functioning as intended and what can be learned and applied as a result of this this evaluation. It will evaluate student satisfaction with the program in terms of content, outcomes and effectiveness in assisting students to pass NURS 315 Pharmacology in Nursing, NCLEX-RN licensing exam on first attempt.

KEY QUESTIONS:

dtdarcy — 2015-10-31 01:21:50 UTC

1. To what
extent do students feel ATI Pharmacology 3.0 supports and reinforces
pharmacology class material?

2. To what extent do students feel
their participation in ATI® Pharmacology 3.0 contributes to their success on
NCLEX-RN?

3. Do students who participated in ATI®
Pharmacology 3.0 score better on Pharmacology Proctored Assessment than
students who do not participate in this program?

Data collection for this evaluation will include:

dtdarcy — 2015-10-31 01:52:12 UTC

Survey graduates of the nursing program to determine: the extent of students feel ATI Pharmacology 3.0 supports and reinforces pharmacology class; Focus groups to determine:how students feel ATI Pharmacology 3.0 improves their knowledge of pharmacologic agents Review ATI™ benchmarking reports to determine if students who participated in ATI® Pharmacology 3.0 score better on Pharmacology Proctored Assessment than students who do not participate in this program

Concerns about the plan and/or executing it:

dtdarcy — 2015-10-31 02:01:54 UTC

In terms of the surveys, the intent is to send surveys to graduates of the nursing program who completed the Pharmacology 3.0 package. The first concern is survey return. The program is small, so a representative sample of graduates would be smaller still. There is a concern of validity of a small N and the results’ ability to be applied to larger populations. There is a concern about modifying existing instruments, thereby compromising their validity and reliability. Good survey questions are difficult to develop and might not have the same meaning to all respondents. Testing of the instrument might be required, delaying data collection. There are concerns regarding the use of focus groups revolving around the time required to complete the process and graduates’ willingness to participate. Difficulty in scheduling groups and expense in training are other concerns. Focus groups would require all interviewees to be trained to conduct the groups and transcribe the data in the same manner. Regarding the availability to receive, review and analyze the ATI® benchmark data, the representative from ATI®, there is assurance that this type of data is available for review. The nurse researcher explained that access to comparative results for schools that do and do not participate in Pharmacology 3.0 is available. The evaluator needs to be sure the nursing programs are comparable in terms of degree offered (baccalaureate program) with a free-standing pharmacology course. There is a need to control for other variables as well, such as size and location of program and how and when the Pharmacology Proctored Assessment is offered.

dtdarcy — 2015-10-31 13:32:57 UTC

Survey Instrument:

dtdarcy — 2015-10-31 13:44:31 UTC

National League for Nursing (2004). Student Satisfaction and Self-Confidence in Learning in Descriptions of Available Instruments. Retrieved from http://www.nln.org/docs/default-source/default-document-library/instrument-2_satisfaction-and-self-confidence-in-learning.pdf?sfvrsn=0 Instructions: This questionnaire is a series of statements about your personal attitudes about the instruction you receive during your simulation activity. Each item represents a statement about your attitude toward your satisfaction with learning and self-confidence in obtaining the instruction you need. There are no right or wrong answers. You will probably agree with some of the statements and disagree with others. Please indicate your own personal feelings about each statement below by marking the numbers that best describe your attitude or beliefs. Please be truthful and describe your attitude as it really is, not what you would like for it to be. This is anonymous with the results being compiled as a group, not individually.

dtdarcy — 2015-10-31 13:47:08 UTC

Satisfaction with Current Learning 5 Strongly Agree 4 Agree 3 Undecided 2 Disagree 1 Strongly Disagree

1.I am confident that this [simulation] covered critical content necessary for the mastery of medical surgical curriculum.

Vocaroo link:

dtdarcy — 2015-10-31 14:23:24 UTC

http://vocaroo.com/i/s1dzwBw96cij

dtdarcy — 2015-11-02 22:54:04 UTC

Donna,

2015-11-04 23:18:35 UTC

Nice job here and like your Logic Model and detail. Like my padlet I was a bit wordy and crowded but maybe yours is a little crowded but don't really have an issue with it,

Ernie

Jennifer H

2015-11-05 16:26:52 UTC

If you figure out an answer to increase you survey completion rate, I would love to hear it. That is something my own organization is currently trying to figure out for our virtual workshops. We thought of doing some type of incentive, but there is a worry about the anonymous nature being maintained to keep in tact the integrity of feedback answers. So, that it one idea to consider, but it would hav to be balance on the importance of keeping your feedback anonymous.