~Week 4: Section 3~

Section 3:  Evaluating, Implementing, and Managing Instructional Programs and Projects

Chapter 10 discusses evaluation in instructional design and provides you with two evaluation models, the CIPP and Kirkpatrick models for evaluation.  Search for at least two other models used for evaluation and summarize these models.  Describe how you would use them to evaluate your instruction.

The two models I have chosen are the Input, Process, Output (IPO) Model and the Training Validation System (TVS) Model.

The following is a chart that compares and clarifies these two models with the two above mentioned, Kirkpatrick and CIPP.

Kirkpatrick (1959)	CIPP Model (1987)	IPO Model (1990)	TVS Model (1994)
1. Reaction: to gather data on participants reactions at the end of a training program	1. Context: obtaining information about the situation to decide on educational needs and to establish program objectives	1. Input: evaluation of system performance indicators such as trainee qualifications, availability of materials, appropriateness of training, etc.	1. Situation: collecting pre-training data to ascertain current levels of performance within the organization and defining a desirable level of future performance
2. Learning: to assess whether the learning objectives for the program are met	2. Input: identifying educational strategies most likely to achieve the desired result	2. Process: embraces planning, design, development, and delivery of training programs	2. Intervention: identifying the reason for the existence of the gap between the present and desirable performance to find out if training is the solution to the problem
3. Behavior: to assess whether job performance changes as a result of training	3. Process: assessing the implementation of the educational program	3. Output: Gathering data resulting from the training interventions	3. Impact: evaluating the difference between the pre- and post-training data
4. Results: to assess costs vs. benefits of training programs, i.e., organizational impact in terms of reduced costs, improved quality of work, increased quantity of work, etc.	4. Product: gathering information regarding the results of the educational intervention to interpret its worth and merit	4. Outcomes: longer-term results associated with improvement in the corporation’s bottom line- its profitability, competitiveness, etc.	4. Value: measuring differences in quality, productivity, service, or sales, all of which can be expressed in terms of dollars

IPO Model

I can primarily see the benefits of the IPO Model in terms of instructional technology and web-based instruction.  With consideration to a third grade classroom, the IPO Model would bring the evaluation from a abstract level to a more specific level.  The four levels include paradigm, theory, model, and measurement.  Compared to the Kirkpatrick model which is goal-oriented, the IPO has emphasis focuses primarily on the paradigm (basically meaning “rules for the game” ) level. 

In a lesson conducted in the computer lab to introduce students to a computer program called Study Island, the CPO Model could be utilized to facilitate learning.  First students would go take the pretest.  This pretest determines the level at which they are functioning mathematically.  Then students would answer ten questions at their level.  Upon completion of this the teacher would stop the students and discuss the current happenings in regard to the program.  Students who successfully mastered their ten questions have an option to play a quick 90 second game.  Those without mastery go back to more instructional question.  Interesting dynamics happen here, students realize that if they don’t actually “try” they aren’t rewarded with the fun game.  The instructor can begin to observe who really is struggling in what areas and who needs extra motivation to be successful.  Wow!  This is very advantageous for the instructor.  With this evaluation of learning there are many variables between input and output with consideration of how students react to the program as well as attitudinal alterations which leads to behavioral changes.  Ultimately this model provides a guideline for instructors/evaluators to advance towards implementing an instructional program in an abstract manner throughout an input-process-output manner. 

TVS Model*

The first step of this model is to collect pre-training data to ascertain current levels of performance within the organization and determining an advantageous level in future performances.  I see this in relation to Garland ISD curriculum assessments (CA) which are given by teachers to their students upon teaching specific skills related to the curriculum assessment.  These CA’s are monitored by campus administration as well as district administration to determine how students will perform on state mandated tests at the culmination of the year. 

For teachers, this leads directly to the next step of intervention by looking at the first round of scores provided through their CA’s and determining why they present a gap in present and desirable performance and how to solve this problem.   Before the next round of CA’s are conducted, the teacher must determine what methods and strategies will be utilized to effectively produce desirable results on the next CA’s.  This might mean extra tutoring, presenting a skill in a different learning technique, consistent reviewing of/building upon skills.  Regardless, the idea is that when the next CA is administered, and evaluation is conducted to determine the impact (3^rd step of TVS) of the tools implemented to produce desirable results.   The final step of TVS is value.  This means measuring differences in quality.  Ideally, when the teacher compares current scores to previous scores a positive difference should be relevant.

*This model is ideal and directed towards business world in many ways but I can see the benefits within classroom instructional evaluations as well.

         

Think about a technological innovation within your social system that was recently introduced.  Any innovation has what is known as perceived attributes-relative advantage, compatibility, complexity, trialability, and observability.  Describe these attributes of the innovation and which of the attributes assisted and/or hindered adoption of the innovation.  For example, was it not compatible with the existing beliefs of the system or was it overly complex to use or understand.

Over the past four to five years, my district has implemented an online curriculum for teachers to originally use as a guideline for instruction.  Now it has become the fundamental norm as (the above mentioned CA’s) curriculum assessments are aligned with it to monitor teacher/student success.  Initially, I along with many of my peers had my reservations with the mindset, “I know how to teach my students. There is no need for the district to give me guidelines as the state already does.”  With this frame of mind I would glance at the online curriculum and meet the necessary demands but ultimately not use it as effectively as it was intended.  At first it the online curriculum appeared overly complex and hard to understand with it’s difficult usability.  However improvements have been made and trainings have been provided to make it more user-friendly.

Now, the district has catered most instruction to the online curriculum with TEKS alignment, extensions, and much more.  Two years ago, teachers were told that implementing the online curriculum is not optional but a requirement.  Therefore, begrudgingly, I climbed over the fence to the side I saw as “not so green”.  Over the past few years, I have utilized the online curriculum effectively through observations of student successes in relation to curriculum assessments and skill mastery.  Today, I see it as beneficial to the teachers in my district as it is mandatory.  I also believe it provides distinct benefits to the first-year teacher as it lays all lessons out for the teacher.  Am I on the online curriculum bandwagon?  The answer for me is that I am half-way on board and mostly because it’s mandatory.  I can see the perceived attributes while understanding it’s the district’s intent and purposes.

Chapter 12 & 13 focus on project management and how to manage projects when resources are scarce.  You have been assigned to develop a series of professional development sessions focusing on technology use in the classroom for teachers.  How will you use Situational Leadership to facilitate this project?

Situational Leadership

I would apply this in direct relation to the above mentioned topic of Garland ISD’s online curriculum for teacher’s struggling with its effective utilization.

Phase 1: Assuming the teachers present for the training would be unsure, my leadership style would be confident, directive, detailed, and overseeing while maintaining an attitude that is non-threatening, self-righteous, or demanding.  As the teachers become more comfortable with simple skills addressed in navigating the online curriculum, we would move to…

Phase 2:  At this point I would move from being more directive of an explanatory nature.  This would include clarification of decisions and rewarding improvements in direction and knowledge.  Once I felt certain that teachers were developing an appreciation and understanding of the online curriculum, I would continue to promote autonomy and plan my move to the next phase.

Phase 3:  In this phase, my role would go from leadership to ensuring teachers are being rewarded for effort and production in a self-evolved success method.  I would no longer be directing instruction as now the teachers would be learning tricks and tips for utilizing the online curriculum to meet their individual needs as a teacher and their student demands as well.

Phase 4:  I would transition from director/instructor to monitor.  Mainly in this stage I would facilitate the teacher’s knowledge of the online curriculum as the demand necessitated.  Primary focus should lie within me that what is important is (with each phase) is how I communicated my goal for learners in designing and producing an exemplary product which in this case was effective usage of the online curriculum.