AECT Poster Proposal– Word Document
Boise State University
Title: Usability Evaluation of an Online Video Feedback Software Within an American Sign Language Context
Short Description –
A poster presentation outlining an evaluation of the contextual video feedback tool: GoReact. The presenter reports findings on the usability, efficiency, effectiveness, and impact of the tool when providing contextual feedback on student created videos demonstrating American Sign Language acquisition. Information presented is especially useful to attendees employing the use of student created videos to demonstrate knowledge but is also applicable to those looking for a creative way to make and use instructional videos.
Session type: Poster
Category of Session: Research Proposal
AECT Divisions, Councils and National Affiliated Organizations: Emerging Learning Technologies (usability testing, evaluation of instructional multimedia)
Video Assessment, distance education, technology integration
An American Sign Language (ASL) department at a mid sized northwest university will begin employing the use of GoReact, a contextual video feedback tool, with two courses in the Spring 2016 semester. The usability, efficiency, effectiveness, and impact of the tool will be evaluated and results presented in this poster session. Information presented on this tool is of great interest to any teacher or instructor that employs the use of student created videos to demonstrate knowledge and performance, long-distance or face-to-face. The evaluation will focus on the use of this tool for providing effective and timely feedback but it should be noted here that there are many creative applications of this emerging technology that can reach a wide range of teachers and subjects.
The recent increase in online video has sparked many studies on its use and impact in education (Artz, Bernstein, & Arithi, 2013; Cleary & Bates, 2014; Moonaghi et al, 2012; Tuong, Larsen, & Armstrong, 2014; Tsur & Beck, 2014). The use of video in the classroom is thought by many to have a major impact on education (Harvey, 2015), but this impact has not yet been felt due to factors, including but not limited to, video assignments lacking the functionality to provide students with immediate and effective feedback (Foertsch et al., 2002, McKinney & Page, 2009 and O’Bannon et al., 2011). The ever pressing issue of collecting and providing feedback on instructional and student created videos has drawn attention to the lack of video assessment software available to teachers. This issue can be especially pressing in American Sign Language classrooms as the nature of the language tends to require use of multiple student created video assignments. One common method of assessing student created videos is to have students upload their video to YouTube where teachers review and grade it online, then provide written feedback separate from the video itself (Alpay & Gulati, 2010). Providing effective feedback is difficult enough (Crook, 2012), while providing effective feedback on student created videos is nearly impossible. Trying to write out feedback on a rubric or in an email is lengthy and ineffective as students rarely go back and watch their video while reviewing written comments. This could be detrimental to the progression of student learning as the significance of effective feedback has been noted in multiple studies (Alpay & Gulati, 2010; Crook, 2012; Stigler, Geller, & Givvin, 2015). One emerging technology, GoReact, has been developed to alleviate this feedback issue. However, few formal evaluations on its effectiveness have been done.
GoReact is a cloud-based software that allows teachers and students the ability to upload and share videos. Instructors are able to access student videos and provide immediate contextual feedback into the video via audio, video, or written comments. GoReact saves all uploaded videos, is easily linked with an LMS, provides an integrated rubric and grading system, and keeps track of group analytics. This makes it easy for teachers to see how often students review their feedback all the while tracking student progress toward performance outcomes. GoReact can not only be used to provide feedback but can also be used to create instructional videos by uploading a video and adding teacher comments or questions. It can then be the location of a streaming lecture where students have a real-time discussion in a chat style side panel. Although GoReact provides many features, this evaluation will focus merely on its effectiveness as a feedback tool in regards to student created videos.
The data collected for this evaluation will be designed to measure progress toward the following goals:
- High student and teacher perceived ease of use (usability)
- Reduced grading time for teachers (efficiency)
- More effective feedback for students / decrease in repeat student error (effectiveness)
- Increase in student language skill and confidence (Impact)
The data will be collected from four courses, two that are using GoReact and two that are not. Data will be collected through interviews with teachers and students, pre and post semester surveys, formative surveys throughout the semester, and analytics available through GoReact.
The presentation will include a poster summarizing evaluation results as well as an iPad available for a hands on learning experience with the tool. Attendees will be able to explore the tool and it’s capabilities while simultaneously reviewing common questions regarding employment of this new technology in their classrooms. The presenter will provide student and teacher perspectives on the usability of the software including time spent navigating and learning the software, time spent providing and reviewing feedback, positive and negative features of the software, perceived improvement in language skill and confidence, and overall satisfaction. Graphs to be included on the poster are as follows; a comparison of the amount of time teachers spend providing feedback with GoReact versus without, a comparison of the number of times students review feedback via GoReact versus re-watching their YouTube videos, and a display of the increase in student language skill and confidence with and without GoReact feedback.
This presentation will provide language teachers with a research driven evaluation of the video feedback tool GoReact. The evaluation will not only add to the body of knowledge on best practice video assessment but can also be used to determine if this tool, or another like it, can provide teachers with reduced grading time and more effective feedback to improve student learning and confidence with second language use. Not only language teachers will benefit from this information as any teacher that utilizes student created video can use this tool evaluation for more efficient and effective assessment.
With video becoming more popular as an instructional tool as well as an avenue for student demonstration of learning, an effective video assessment tool is a necessity. GoReact is one option and this evaluation will shed light on its usability, efficiency, effectiveness, and impact in the context of providing feedback on student created ASL videos.
Alpay, E., & Gulati, S. (2010). Student-led podcasting for engineering education. European Journal of Engineering Education, 35(4), 415-427.
Artz, P., Bernstein, R., & Arithi, P. (2013, October). Streaming Media for Each Student in Every Class: Interdepartmental Best Practices for Accessible, Legal, Affordable, and Effective Video Delivery to Online Students. In World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (Vol. 2013, No. 1, pp. 575-580)
Cleary, C. E., & Bates, A. (2014). Online video in education: acquisition, value & ROI.
Crook, A., Mauchline, A., Maw, S., Lawson, C., Drinkwater, R., Lundqvist, K., … & Park, J. (2012). The use of video technology for providing feedback to students: Can it enhance the feedback experience for staff and students?.Computers & Education, 58(1), 386-396.
Foertsch, G.A. Moses, J.C. Strikwerda, M.J. Litzkow. (2002). Reversing the lecture/homework paradigm using eTeach® web-based streaming video software. Journal of Engineering Education, 91 (3) (2002), pp. 267–274
Kay, R. H. (2012). Exploring the use of video podcasts in education: A comprehensive review of the literature. Computers in Human Behavior,28(3), 820-831.
McKinney, K. Page. (2009). Podcasts and video streaming: Useful tools to facilitate learning of pathophysiology in under graduate nurse education?. Nurse Education in Practice, 9 (6) (2009), pp. 372–376 http://dx.doi.org.libproxy.boisestate.edu/10.1016/j.nepr.2008.11.003
Moonaghi, H. K., Hasanzadeh, F., Shamsoddini, S., Emamimoghadam, Z., & Ebrahimzadeh, S. (2012). A comparison of face to face and video-based education on attitude related to diet and fluids: Adherence in hemodialysis patients. Iranian journal of nursing and midwifery research, 17(5), 360.
O’Bannon, J.K. Lubke, J.L. Beard, V.G. Britt. (2011). Using podcasts to replace lecture: Effects on student achievement. Computers & Education, 57 (3) (2011), pp. 1885–1892
Stigler, J. W., Geller, E. H., & Givvin, K. B. (2015). Zaption: A Platform to Support Teaching, and Learning about Teaching, with Video. Journal of e-Learning and Knowledge Society, 11(2)
Tuong, W., Larsen, E. R., & Armstrong, A. W. (2014). Videos to influence: a systematic review of effectiveness of video-based education in modifying health behaviors. Journal of behavioral medicine, 37(2), 218-233.
Tsur, M., & Beck, J. (2015). The State Of Video in Education. Global Learn,2015(1), 201-203.