Understanding the Webinar Skills Assessment Results
After completing the evaluation form, there are three levels of effectiveness, based on different groups of LEARNERS or AUDIENCE TYPES. There is also a consistency deviation that shows the difference across audience types, and there is an overall Speaker Index.
This webinar version of the skills assessment uses 19 of the 21 skill categories found on the STANDARD Evaluation Form because the presenter is not seen by the audience, only heard. Thus, certain visible elements including those related to the category of body language and the room setup environment do not apply.
AUDIENCE TYPES
When comparing groups of student learners with groups of professional learners, research indicates that each group places significance differently on each skill category. This assessment summary accounts for different learning styles and separates those styles into three audience types: Students, Professionals and a combination of both.
A STUDENT learner is defined as one who requires knowledge in order to participate in a desired activity. This group not only includes traditional “students”, but it is extended to also include residents, trainees, and anyone who needs to immediately apply learned concepts. So the term “student” is used in this report.
A professional is also a “learner” but in a slightly different sense.
A PROFESSIONAL learner is defined as one who desires knowledge in order to add to an existing activity. Professionals include those who incorporate learned concepts over a period of time. Like students, they are learning, but they are using the knowledge to augment an existing practice or job function over a longer period.
Audiences can be a mix of student learners and professional learners, and this combination is basically an “average” of the two groups. At times, such as in video recorded sessions, you are “simulating” the presence of an audience by performing in a closed studio or set, and the eventual viewing public is likely to be a mix of both student and professional learners.
MEASUREMENTS
At the very end of the form, key statistics appear.
The LEVEL OF EFFECTIVENESS is a measure of how clearly the elements of skill were demonstrated in a thin-slice of behavior at a particular moment in time. The measurements may vary across audience types, depending on the roles played (moderator, commentator, presenter, etc.), the related challenges and opportunities observed, as well as the number of observations made.
The CONSISTENCY DEVIATION shows the extent to which the skills are performed differently across different groups. The lower the deviation, the more consistent the skills appear to different audiences. A deviation of no more than 5% is preferable, especially for a mixed audience of professionals and students.
The overall SPEAKER INDEX is expressed a number not a percent. Think of the Speaker Index as the extent to which the current level of talent (skill) is being used to satisfy any type of audience. The index is a product of the various levels of effectiveness (for specific audiences) and the consistency deviation. The greater the consistency across audiences (that is — a lower deviation), the closer the level of effectiveness for a mixed group is to the overall Speaker Index.
A Speaker Index of 70 or greater is preferable.
NOTE: The overall speaker index may NOT be the best result to examine. If the typical audience type is “Professional” learners, then that value provides a more accurate measure of effectiveness, rather than the overall speaker index.
SELF-IMPROVEMENT PROCESS
To maximize effectiveness, review the evaluation form and note any opportunities that are NOT checked. Where possible, consider adding those yet-to-be-used skill elements in order to maximize efficiency.
For example, suppose that on the evaluation form, in the category of Interaction, the box “Repeats the question or the comment as a part of the response” is not checked. This indicates that the opportunity for the skill has yet to be used. If the presentation includes interaction, consider implementing this skill.
A copy of the published research, from which this assessment is derived, is available if you wish to review the support details. In 2010, the underlying research supporting the calculations used on this form was formulated into a COMPLETE SET of interactive assessment tools and published on the MedEdPortal, a peer-reviewed repository for application-based publications and programs. The entire set of PRESENTATION SKILLS ASSESSMENT TOOLS is available on the MedEdPortal (Publication ID 7930).
To develop any of these skills to a higher level, consider the book: A Guide to Better Teaching (Jahangiri, Mucciolo). Although geared toward educators, the book covers all of these skills in depth, and the advice is applicable to anyone who presents information.