This in depth evaluation demonstrates my ability to assess the adequacy of student learning and evaluate the effective implementation of educational technologies.
Part 4. Evaluation
4a. Evaluation Plan
In order for a project to be successful evaluation must be woven throughout. An effective evaluation plan will provide the designers and instructors with the information necessary to adjust and improve the project. The evaluation plan for this project is described here and uses Kirkpatrick’s Four-Level Evaluation Model found in Larson and Lockee (2014).
Level I: Learner Reaction
During the seminar, the instructor will be available to the learners and take note of any questions they ask about the material. The instructor should also take note of how the learners are reacting to the content in order to improve instructional materials for future seminars. After the seminar is completed, a survey will be provided in order to measure their reaction to the material covered in the seminar. Suggested questions include:
- Did the instructional materials (videos) help you in learning the content better then if you learned it on your own?
- In your opinion was the ASL used in the videos clear and easy to follow?
- Did the videos cover the content adequately? If not, what do you feel could use more attention?
- Did you use the Learner Objective checklist? Was it helpful to your learning process?
- Were you able to get your questions answered by the seminar instructor?
- Were you able to use this newly learned content in your classroom with ease? If not, what would assist you in doing this?
Level II: Learning
The learning that takes place in this seminar is best assessed using performance tasks due to it’s pass or fail nature. And also due to this pass or fail nature, there are no numerical comparisons of pre and post seminar learning. Therefore, the learning that took place during this seminar will be evaluated using a survey method that requires participants to use the skills they learned in the seminar. The survey should be sent via a Google Doc where participants will transfer it to their personal drive and answer the questions using the comment feature. Suggested questions for this survey include:
- Did you successfully transfer this document to your personal Google Drive?
- What are the ways you can share this document with others?
- Have you used Google Drive in your classroom?
- How is your Google Drive organized? How are your folders labeled?
- If I receive a link to a Google project from a student, where can I find it in Google Drive?
Level III: Behavior
To evaluate how the skills learned in the seminar are being transferred to the participants day-to-day tasks, a survey will be provided. The suggested questions given here are to be sent to the participants of the seminar, but another idea is to survey the students of the seminar participants and ask about the teacher’s confidence level with Google Drive. Some suggested questions for the survey sent to the participants of the seminar include:
How often do you use Google Drive after the seminar compared to before?
- Do you feel confident sending and receiving shared documents?
- During your day-to-day tasks, how often do you turn to Google Drive for work with documents, sheets, and presentations?
Level IV: Results
Come time for the next American Sign Language Teacher Association’s Fall seminar, the overall results of the previous seminar can be adequately evaluated. During this seminar tasks involving Google Drive’s share and collaborate features can be utilized. Judging weather or not the seminar was a worthwhile option can be aided by the following questions:
Did the teachers successfully transfer a shared project to their personal Google Drive?
- Did the teachers successfully transfer a personal project into a shared folder?
- Do the teachers collaborate via comments and chat?
- Are the teachers Google Drives organized?
- Do the teachers seem confident with Google Drive?
- Are the teachers asking a lot of questions related to Google Drive navigation and collaboration?
4b. Expert Review
There are two Subject Matter Experts working on this project. The first is a Deaf woman by the name of Davina Snow, the head of the American Sign Language department at Boise State University. She was to review the use of American Sign Language in the learning materials and received the project on April 29, 2015 with returned comments on April 30, 2015. The second Subject Matter Expert reviewing the project is Reggie Walters, a PhD candidate in the Geoscience department of Boise State University. He is an avid Google Drive user and recommended throughout the department when it comes to technology tools and questions. Mr. Walters received the project and survey on April 3rd, 2015 and returned comments on April 20, 2015.
4c. Evaluation Survey
The survey found here: http://goo.gl/forms/CD40NGbzOA includes the following survey questions sent to Mr. Walters to assist in his evaluation of the project:
- How can the goal of this project better meet the needs of novice Google Drive users?
- Which objectives adequately guide instruction toward the goal and what are some other objectives that could better assist this process?
- What steps, if any, are missing in the instructional videos?
- What skills, if any, are missing from the learner objective checklist that you feel novice Google Drive learners should know?
- What, if anything, is unclear in the instructor written guide and how can it be adjusted to improve the quality of the seminar?
- Please describe your overall impression of the project and any changes you feel should be made.
4d. Expert Review Results
The survey results returned by Mr. Walters were mainly positive with a few suggestions related to content. In his opinion, the goal of the project is clear and adequately touches on the needs of novice learners’ as do the objectives. When it came to the instructional videos, both Mr. Walters and Mrs. Snow had the same recommendation: a more professional background and appearance. Mr. Walters proceeded to recommend adding some common mistakes into the videos to show the learners what to avoid doing. Besides the videos, Mr. Walters was overall impressed with the project as the instructor guide, videos, objectives, and overall goal relate nicely and are easy to follow hitting all commonly asked questions related to Google Drive.
4e. Comments on Change
Two major changes were suggested and are in the process of being addressed. The first is the appearance of the videos. The American Sign Language use is clear but the background and appearance will be addressed. The second change that will be added to the project is the inclusion of common mistakes; accidentally deleting a shared document, what happens when you move an item out of a shared folder, and how to counteract these mistakes.