MIT 530 - Evaluation and Change in Instructional Development
 |
Context
“AECT New Definition Problem” was generated within the context of MIT 530: Evaluation and Change in Instructional Development as a partial requirement of the course. This project was supervised by Dr. Sue-Jen Chen, during the spring semester of 2009. The project consisted of completing a needs assessment to collect data that provided information that the project team utilized to list possible reccommendations to solve the problem.
Conditions
This project was an extension of a needs assessment plan that was generated at the beginning of the course. The project began when Dr. Sue Jen Chen, a
full-time MIT program professor, expressed concern that the MIT faculty and staff want to make sure they are
adequately preparing graduates for the work force. MIT faculty were concerned that when AECT redefined the field of instructional technology in 2008, the curriculum at UNCW would no longer be able to prepare students in the program with necessary skills and knowledge needed to acquire a job after graduation. Therefore, the project aimed to answer the question: Are the current competencies geared towards preparing students
for employment after graduation?
Scope
Working collaboratively with three other MIT classmates, this project involved: identifying a real-life performance problem, designing an evaluation tool that was both reliable and valid, analyzing extant data and interview information, and then proposing recommendations based on the findings. To complete this project, Allison Rossett's model was chosen due to Rossett’s focus on purpose-based training needs assessment and because it helped our project team to determine the following: optimal performance/knowledge regarding the new AECT definition, actual
performance/knowledge regarding the current MIT competencies at UNCW, and
feelings/attitudes of the stakeholders regarding the problem. Possible causes of the current problem were then discovered after interpreting the results of the data and opinions. Based on the data and findings, recommendations were outlined and the needs assessment report was generated and communicated to project stakeholders.
Role
For this project, I worked with three other MIT students to design, develop, conduct, and write up a needs assessment. Throughout this project, I assumed the roles of researcher, data analyst, and instructional designer.
Reflection
This was my first experience completing the following: developing an evaluation tool (survey), reviewing the data, and interpreting the findings. Although I do not have a statistical background, completing this project made it possible for me to get hands-on experience interpreting data findings to determine the need. Having this experience has made me feel more comfortable with the needs assessment process. After completing this project, I realized that I was disappointed that I did not get a chance to generate a change management plan for this project.
"Summative Evaluation Plan: Rapid Online Course Development"
Context
“Summative Evaluation Plan: Rapid Online Course Development” project was generated within the context of MIT 530: Evaluation and Change in Instructional Development as a partial requirement of the course. This project was supervised by Dr. Sue-Jen Chen, during the spring semester of 2009. The project stepped me through the process of implementing a self-instrucitonal module and then conducting a summative evaluation.
Conditions
This summative evaluation plan was completed to evaluate a self-instructional module that was designed in a different MIT program course. In order to complete the summative evaluation plan, analysis of the exit surveys for reactions to the module, data from the pre- and post- tests, and the assessment book questions and checklists were
analyzed to determine the learning results. After completing the analysis and interpretation of the data, a written summative evaluation report was generated to relay pertinent information regarding the effectiveness of the self-instructional module.
Scope
This project allowed me the chance to utilize Kirkpatrick's Four Levels of Evaluation model to determine the effectiveness of the self-instructional module. Using the model, I was able to guide the design of my summative evaluation plan. For this summative evaluation plan, the focus is on the self-instructional
module that informs the learner on how to create an assignment, an assessment, and an electronic
calendar using Blackboard Vista. This summative evaluation involved the analysis of the exit surveys and each participant’s responses was analyzed using
descriptive statistics. Observations were conducted to ensure learners’ successful transfer of
knowledge and skills to the work environment. The results concluded, helped to determine the
impact the self-instructional module has on the UNCW organization as they are preparing for the
future of online courses.
After collecting, analyzing, and interpreting the data the summative evaluation report was written
detailing the findings.
Role
For this project, I worked independently as instructional designer and developer.
Reflection
This was my first experience completing a summative evaluation. This project helped me see the difference between formative and summative evaluations. This project provided the chance to utilize Kirkpatrick's Four Levels of Evaluation model. This model was new to me and I can see how it will come in handy in the future.