Table of Contents, Master Presenters Resources Page Previous Section, Master Presenters Next Section, Master Presenters

Westside Toastmasters is located in Los Angeles and Santa Monica, California

Maximizing Salient Feedback

Oh, that God would give us the gifts to be able to see ourselves as others see us.

—Robert Burns, Scottish poet

Salient feedback is feedback that is so personally meaningful that we actually change our behavior. The problem is that even though we live in a feedback-rich world, most of us do not harvest the feedback necessary for total quality improvement. We will present 12 methods to ensure that you are getting the feedback necessary to put you squarely on the road to becoming a Master Presenter.

There are many techniques that will help you get the feedback you need from those who have seen you present. A good evaluation form given right after the presentation will give you some indication of how well you presented. If you use the same evaluation form over time, you can gauge progress and the effect of changing an element of what you present or how you present.

There are almost as many types of presentation evaluations as there are presenters, however, many presentations are not evaluated at all. Presenters who fail to evaluate their presentations miss out on valuable feedback. We have found, as have the Master_Presenters that we interviewed, that even presenters who have a great deal of natural talent will eventually present less well than their less talented counterparts who have sought out and benefited from constructive feedback.

We favor using scaled items (to help measure between presentations) and open-ended items to get a deeper sense of how the participants reacted to the presentation and to individual differences. Two things to keep in mind when you use these types of evaluations: You can't please everyone, and extreme scores can bias the ratings.

The 12 techniques to increase the amount of salient feedback you receive are:

  1. The 3 × 3 Feedback Form.

  2. The Presentation Evaluation Form.

  3. The "A Penny for Your Thoughts" Evaluation.

  4. The Post-it Note Evaluation.

  5. The Daily Evaluation Form.

  6. Highly Focused Feedback.

  7. Audio/Video Feedback.

  8. The Component Parts Analysis.

  9. The Instant Component Analysis.

  10. Be Vigilant for Opportunities to Maximize Feedback.

  11. Seek Feedback from Spouse, Children, and Significant Other.

  12. The Results Achieved Over Time Evaluation.

1 The 3 × 3 Feedback Form

The 3 × 3 Feedback Form is designed to help you get more systematic feedback on what you do well as a presenter, in addition to providing targets for improvement. It solicits feedback in threes: three things done well and three areas in which to improve. Research has proven that we tend to be poor observers of our own behavior and that we become much more accurate when we have a systematic method of data collection. In addition, we get much more accurate feedback by asking at least three different people to rate us. There are a number of criteria to consider when choosing the people who will respond to the form. You need to choose someone who is both free to and capable of giving you honest, direct, and straightforward feedback.

There are three reasons for starting with positive feedback: 1) It is important to be acknowledged for what we do well—no one in any of our training sessions has admitted that they suffered from too much positive feedback; 2) Behavior that is acknowledged and reinforced tends to occur more frequently; and 3) Positive feedback is often instrumental in helping us develop the focused motivation necessary to work harder in areas where we want to improve.


Name______________________

Please list three specific things I do well as a presenter. For example, "Pat is a good presenter," is not specific. "Pat uses creative and unexpected visuals to anchor her points both aurally and visually," is specific.

  1. ______________________________________________

  2. ______________________________________________

  3. ______________________________________________

Please list three specific targets for improvement. For example, "Paul needs to add more impact to his presentation," is not specific. "Paul needs to develop high-impact introductions and test his introduction with a focus group," is specific.

  1. ______________________________________________

  2. ______________________________________________

  3. ______________________________________________



Figure 7-1: THE 3 × 3 FEEDBACK FORM

Lastly, an alternative method of data collection is to ask a neutral third party to collect the data for you and then present you with a summary of the data in such a way that no specific respondent could be recognized.

2 The Presentation Evaluation Form

The most common evaluation is an evaluation given immediately after the presentation. The evaluation will vary in length and level of complexity depending on the extent and intricacy of the presentation. We like evaluations that are both quantitative and qualitative. Quantitative evaluations allow you to see if you are making progress over time. Qualitative evaluations allow you to get information that is more idiosyncratic about the content of the presentation, what your audience enjoyed, what they learned, and specific targets for improvement. An example of this type of evaluation is presented in Figure 7-2 below.


Please rate this presentation on the following seven scales:

The goals of the presentation were:

Unclear

     

Clear


1

2

3

4

5

6

7

The presentation was:

Disorganized

    

Organized


1

2

3

4

5

6

7

The presenter was:

Poorly prepared

   

Well prepared


1

2

3

4

5

6

7

The presentation was:

Dull

     

Lively


1

2

3

4

5

6

7

The presenter was:

Boring

     

Dynamic


1

2

3

4

5

6

7

I found the presenter (Difficult/Easy) to interact with:

Difficult

     

Easy


1

2

3

4

5

6

7

My overall evaluation of the presentation was:

Poor

     

Excellent


1

2

3

4

5

6

7

Please list three specific things you enjoyed about this presentation:

  1. __________________________________________________

  2. __________________________________________________

  3. __________________________________________________

Please list three specific things you learned from this presentation:

  1. __________________________________________________

  2. __________________________________________________

  3. __________________________________________________

Please list any suggestion you have for improving this presentation.

________________________________________________________

________________________________________________________

What other presentations could we offer to support your continuing professional development, follow-up on this program, or apply what you have learned?

________________________________________________________

Thank You



Figure 7-2: PRESENTATION EVALUATION FORM

3 The "A Penny for Your Thoughts" Evaluation

Each participant is given five pennies. At the end of the presentation, or possibly at the end of the first day of a multiple-day presentation, the participants can rate the value they derived from the presentation by putting a portion of their five pennies into a jar. For example, if they got relatively little from the first day, they would put in one penny. If they received a great deal of value, they could put in four or five pennies, and so on.

The following day, the instructor can either report the total number of pennies he or she collected in the jar, or work it out proportionally out of 100. As an example, assume there are 15 participants, each with five pennies, and at the end of the day there are 60 pennies in the jar. The instructor could say that on day one, the participants received 80-percent value (60 out of 75). Participants find this a fun activity. It also gives the instructor a good use for his or her "leftover pennies." The only drawback from this form of evaluation is that it does not tell the instructor what is working and what is not, but it can be used to start a conversation on feedback, for what worked well and what needs improving.

4 The Post-it Note Evaluation

The Post-it Note evaluation can be done by itself or in conjunction with the "A Penny for Your Thoughts" evaluation. For the Post-it Note evaluation, you will need two different colored Post-it Notes no smaller than 3 × 3 inches. Let's assume that the first color is green. Each participant is asked to write down on the note up to three things that worked well in the speaker's presentation. If the second color is yellow, the participants would write down up to three suggestions for improvement on the yellow Post-it Notes. On the way out, each participant can put his or her notes in a designated area such as on a flip chart. This evaluation gives specific feedback on what worked and what needs improving.

5 The Daily Evaluation Form

If you are giving a multiple-day program, you can use a form that is similar to The Daily Evaluation Form found in Figure 7-3.


  1. List several points/items you found useful today.

    ________________________________________________________

    ________________________________________________________

  2. List any areas that require clarification or need to be reviewed.

    ________________________________________________________

    ________________________________________________________

  3. Additional comments:

    ________________________________________________________

    ________________________________________________________



Figure 7-3: DAILY EVALUATION FORM

These evaluations can be used to start off the next day's session as a review, by highlighting items that the participants thought were particularly important, and by discussing those items that needed further clarification.

6 Highly Focused Feedback

An excellent way to get highly focused feedback is to do an e-mail survey where you ask only one or two questions. For example, you can e-mail 10 to 20 people who have heard you speak and ask them to tell you what they perceive as the most unique aspect of your content. Or you can ask them to tell you the most unique aspect of your presentation style. The purpose of this exercise is to receive as highly focused and specific feedback as possible, and for that reason it is better to ask only one question at a time.

For example, Brad asked 10 people what they most liked about his presentation style and five factors came back. One of the most interesting was the feedback he received from Jonathan. Jonathan said that he was very impressed with the way Brad used analogies, that is making something more understandable by relating it to something that is already well understood. The example that Jonathan related was using a trim tab (Figure 7-4) to help people better understand the leverage that can be gained from better understanding one's negotiating style.

Click To expand
Figure 7-4

Brad: Although I realized that I used analogies, it wasn't until Jonathan pointed it out that I realized how powerful they were. I also vowed to improve my use analogies more consciously and to ask audience members for more specific feedback as to whether the analogy I used to help answer a specific question was helpful.

When soliciting highly focused feedback, it is easy to vary the question. If you use a lot of humor, you can ask them the most unique aspect of your humor, or what they most like about your voice, your use of props, transitions, or PowerPoint slides, and so on. The value of this question is that it focuses entirely on one aspect of your ability to present: the feedback is highly focused and decidedly useful.

In doing this exercise most people chose to ask the same questions, such as how dynamic, humorous, or memorable the presentation was. We encourage you to ask a large variety of different questions. For example, Master_Presenter Tom Stoyan directly solicited feedback as to his trustworthiness. The following statements are among the responses he received:

  • "Demonstrates confidence."

  • "Shows genuine concern and interest."

  • "Identifies and focuses on reducing our worry list."

  • "Demonstrates thoughtfulness."

  • "Demonstrates commitment."

  • "Looks for opportunities to empower us."

  • "Establishes and maintains a non-threatening environment."

  • "Looks for opportunities to provide and get feedback."

  • "Says what he is going to do and then does it."

  • "Tells stories that demonstrate and reinforce trust-building."

As you can see, it is easy to vary the question. If you use a lot of humor, you can ask your audience what is the most unique aspect of your humor, or what they most like about your voice, your use of props, transitions, or PowerPoint slides, etc. The value of this type of question is that it focuses entirely on one aspect of your ability to present, and therefore, the feedback is highly focused and decidedly useful.

EXERCISE 7-1

Design a question on which you would like to get highly focused feedback. Then list the people you can solicit this feedback from.


7 Audio/Video Feedback

Taped feedback is one of the best sources of feedback presenters can get. The problem is, no one likes the way they look or sound on tape. A frequent comment is, "But I don't sound like that!" Yes, you do. The tape is a more accurate indicator of how you sound to other people. Remember, the voice that you hear when you speak is how it sounds in your head. You are "hearing" the subtleties and nuances that your brain intended. The audience is not privy to those intentions, so all they hear is the sound waves that fall on their ears. Our advice is to tape yourself, grit your teeth, and listen to it. If you don't like what you hear on tape, there is a good chance that the audience may be thinking the same thing.

No matter what you think about how you look or sound, what you see and hear is honest and accurate. In addition, taping yourself affords these advantages:

  • You can play it for others to obtain their feedback and impressions.

  • You can try various presentation strategies and techniques and compare them.

  • You can capture not only what you say, but also how you say it.

  • You can immediately see if your body language is congruent with your verbal presentation.

Audio and videotaped feedback is important for all aspects of the presentation, and it is especially important for the beginnings and endings because they carry such weight in the presentation. Remember, 25 percent of the impact from any presentation is from the beginning, and 25 percent is from the ending.

Brad: One night, as I was preparing for a presentation, I asked my son to videotape the beginning, the ending, and one section from the middle that I was not happy with. Based on seeing the tape, the beginning was excellent. I saw clearly how to strengthen the ending. But I was still stuck on how to make that troublesome point in the middle work. I tried three variations before deciding on a fourth (which combined elements of the other three) before I was happy with it. As an added bonus, my son now has a much better understanding of what his Dad does for work and his younger sister has already volunteered for the next time I need a cameraperson.

If the cameraperson also knows a great deal about presentations and/or the content of your presentation, you can also benefit from his or her expertise. One point to remember is that you also benefit from feedback from people who don't know the content area, especially if you will have people attending the presentation who are not familiar with your topic or content. In other words, to increase the validity of this exercise, you should consider asking your "coach" to be as similar to the presentation audience as possible.

Another advantage of taping every presentation is that you can capture those special moments when something wonderful happens by surprise. Victor Borge, one of the great entertainers of all time recorded every single performance. Why? He said, "Because I never know when I might say something funny."

8 The Component Parts Analysis

The component parts analysis is an evaluation of the individual components that make up your presentation. The component parts analysis will help you get a good sense of what parts of the program are working well, what parts need to be improved, and what ones need to be eliminated. Examples of component parts are: exercises, activities, stories, and visual aids.

The beauty of a component analysis is that it will save you from erroneous assumptions. For example, Brad has been giving a presentation on dealing with difficult people for more than 10 years. In the presentation, he shows a film that he was getting tired of using and had planned to remove from his presentation. However, when he performed the component parts analysis, participants rated it as one of the most highly rated parts of the presentation so it remained as an integral part of the course.

What will be very helpful is how the components are rated. You may find that the participants in general do not rate one of your favorite exercises nearly as favorably as you thought they would and you may, in fact, make the presentation much more effective by eliminating that exercise and replacing it with something that the participants find much more effective. Conversely, you may find that a part of the presentation that you don't like or, as in Brad's case, have grown tired of, is rated as one of the most valuable aspects of the presentation.

Warning 

Our experience indicated that there is usually more discrepancy in how well or how poorly individual components are evaluated than there is when the participants evaluate the presentation as a whole. That is to say, the ratings of the components are usually significantly lower than the rating for the course as a whole. We believe this is because different components have a greater or lesser appeal to the participants than does the course as a whole—so don't be surprised or put off if the ratings for the components are lower than the ratings for the full program.

An example of a course component parts evaluation appears in Figure 7-5. Note that this is a components evaluation that was designed for our particular course. You can build your own form by removing our examples and substituting your own.


Please rate the following course components on their usefulness on the following scale, where 1 is not useful and 7 is very useful:

  1. Exercise 1-2: The characteristics of the best and worst teachers I had in school.

    1

    2

    3

    4

    5

    6

    7

  2. Understanding the four learning styles: theorists, reflectors, activists, and pragmatists.

    1

    2

    3

    4

    5

    6

    7

  3. Identifying your purpose and the film clips used to illustrate ITEM: to Inform, to Touch the emotions, to Entertain, and to Move to action.

    1

    2

    3

    4

    5

    6

    7

  4. Mastering one-minute talks.

    1

    2

    3

    4

    5

    6

    7

  5. Mind mapping using the TRAP model to reach each of the learning styles of your audience members.

    1

    2

    3

    4

    5

    6

    7

  6. Developing powerful beginnings.

    1

    2

    3

    4

    5

    6

    7

  7. The use of games to make it both fun and memorable.

    1

    2

    3

    4

    5

    6

    7

  8. The two-voice exercises: Moses Supposes and "This is the best presentation I ever attended" exercise.

    1

    2

    3

    4

    5

    6

    7

  9. Would you recommend changing any of the components? If yes, which ones? How would you recommend changing them?

    ________________________________________________________

  10. Do you have any additional suggestions regarding the components of the course or the way in which they are structured?

    ________________________________________________________

Thank You

_______________________________



Figure 7-5: COMPONENTS PARTS EVALUATION FORM

However, you don't have to wait for the end of the presentation to do a component analysis. You can do an instant component analysis.

9 Instant Component Analysis

There is an easy and fun way to get feedback on any of the components, new or old. We introduced a new exercise to a group of participants in our presentations course. The exercise was to develop a mind map for one section from this resource. Participants then developed a presentation based upon their mind map and applied the TRAP model (theorists, reflectors, activists, and pragmatists) to their presentation in order to make the presentation applicable to the learning style of the members of their audience. They were instructed to use a different colored marker for each learning style.

The beauty of using different colors is that the presenters get immediate feedback on how balanced their presentation is. For example, if green is the color that represents ways to involve the theorists, and there is hardly any green on the page, then the presenters know that they may need to develop more ways to involve the theorists in their audience. The participants then explained their mind map to the whole group. For example, one participant explained how he would make "the buddy system" attractive to theorists as well as to reflectors, activists, and pragmatists.

After completing this exercise, Brad suspected that his instructions were not clear enough nor was the time that had been allotted for the exercise sufficient. So he decided to do an Instant Component Analysis. He then asked each participant to fill out a yellow 3 × 3 inch Post-it Note on what worked in the exercise and to rate the exercise on a scale from 1 to 10 where 1 "didn't work" and 10 "worked very well." He also asked them to fill out a pink note on what would have made the exercise better. When they finished, they were asked to affix their Post-It-Notes on the flip chart paper that had "What Worked?" at the top and "What Would Make It Better?" written in the middle. After the exercise, they took a break. When the class started again, Brad reviewed the feedback. The average rating for the effectiveness of the exercise was 8.6. Typical comments of what worked and what would make it better are illustrated in Figure 7-6 starting on page 226.

What Worked?

Reading the assignment as homework and then designing the presentation as the first task in the morning.
Got everyone interested and involved.

How the map helps with actual presentation.

Combining the techniques forced us to understand rather than just recite the elements.

Moving from practice or theory into reality.

Implementing TRAP and mind mapping helps in organizing ideas in presenting.

Seeing how different teams approached the problem.

Seeing how to get ideas across visually.

Teamwork in preparation and relating brainstorming ideas to the TRAP definitions.

Team participation, hearing other people's views on the same topic.
Reinforced the TRAP model.

What Would Make It Better?

More specific instructions.
Add 10 minutes more prep time for the in-class exercise.

More time to prepare a suitable presentation, that is, the last thing you do on Day One is the mind map—first thing we do on Day Two is the developing and presenting the presentation.

More rehearsal on our part.

Using the techniques to solve an actual problem (case) would be more effective.

A second exercise to enforce what we have just done.
Although, I think this exercise worked well.

 

Figure 7-6

There is one more way to use an instant component analysis. If the presentation has 10 parts, and if you give the presentation frequently, you can evaluate a different component each time you give the presentation. By the time you have given the presentation 10 times, you will have evaluated each component. Both the quality of this feedback and the quantity are absolutely guaranteed to improve your presentation content and style of delivery. In summary, instant component analysis is one of the easiest, more immediate, and most fun ways to elicit that feedback. And as presenters, we learn how to make the presentation more interesting for both the participants and ourselves.

10 Be Vigilant For Opportunities to Maximize Feedback

There is a scene from The Sixth Sense in which Haley Joel Osment plays the role of Cole, an 8-year-old boy, and Bruce Willis plays the role of Malcolm Crowe, a renowned child psychologist. The scene opens with Cole standing in the entryway of his house and Malcolm sitting in the middle of the living room. Malcolm tries to get his young patient to open up to him by wisely asking Cole if he would like to play a game (games are the natural language of children). The essence of the game is that Malcolm will try to guess what Cole is thinking. If Malcolm guesses correctly, Cole will take a step closer, and if Malcolm guesses incorrectly, Cole will take a step back. If Malcolm guesses correctly enough, Cole will sit down and they will have a conversation, if he does not guess correctly enough and Cole reaches the front door of his home, both the game and the session are over.

One of the many things that is so intriguing about this scene is that Malcolm has set it up so that he gets immediate feedback as to the accuracy of his perceptions about Cole. Likewise, Master_Presenters are vigilant for opportunities and develop methods to maximize the feedback that they receive. This ability is clearly demonstrated in the following two examples.

For Master_Presenters there are no obstacles or excuses that stand between them and their goal of maximizing salient feedback. Brad was presenting a keynote at the Year 2000 Millennium Conference in Ottawa. He was the keynote speaker on the second day. Janet Lapp was the keynote speaker on the first day. Brad had arrived a day ahead of his presentation to get a good sense of the conference and to tie his remarks into both the conference in general and to Janet Lapp's comments in particular.

Brad: I was waiting to talk to Janet at the end of her presentation. There was a long line ahead of me, so I decided to listen to the questions and Janet's answers as I waited. Three things particularly impressed me: 1) the number of people who stood in line to speak to Janet; 2) the quality of her answers; and 3) how aggressively she asked for salient feedback at the end of the presentation. I used the word "aggressive" here in a very positive sense. I also heard the quality of the feedback that Janet received. Watching Janet reinforced my dictum that we live in a feedback-rich world. Most of us do not "harvest" the feedback that exists.

David: I have found audience feedback has risks as well as rewards. The rewards come in the form of affirmation that you connected in the manner in which you intended, coupled with legitimate suggestions of how to make your points better. Every Master_Presenter depends on this kind of feedback. This is how we grow.

Yet there are risks in processing feedback as well. Some people are simply poor listeners. Therefore, if you try to adjust your presentation based on the comment of someone who clearly misheard or misinterpreted you, you could end up trying to fix a problem that doesn't exist, except in the mind of one poor listener. This is another reason why taping yourself is so important. If an evaluation that says you said something, you can go to the tape to verify or refute the feedback in question.

EXERCISE 7-2

Write down three opportunities that exist for you to receive more feedback about your presentations.


11 Seek Feedback from Spouses, Children, and Significant Others

It was potentially one of the most important presentations of Brad's life. He was flying to Toronto to give a presentation to Meeting Professionals International. MPI is the world's largest association of meeting professionals and one of their primary responsibilities is to organize meetings and conventions. If he did a good job it would increase the likelihood that he would give more presentations and keynotes. To add more pressure, the potential publisher of his newest book was coming to hear him present. If he presented well, it would dramatically increase the likelihood that he would get the contract; if he did not present well, the contract was history.

Brad: I had already booked our family's annual Easter weekend retreat at Nova Scotia's beautiful White Point Beach Lodge. There were many wonderful activities that the children could engage in and there would be lots of time for me to refine my presentation, practice it, and record it. Although I really liked the introduction, exercises, the transitions, and the ending, there was a part in the middle that I just wasn't happy with.

I asked my children—Katie, age 10, and Andrew, age 13–to listen to parts of the presentation, which included the part I wasn't happy with. Surprisingly, they both thought it was quite good. However, when it came to the part in the middle, Katie suggested that I change the order of a few of the words and that I add some increased emphasis and vocal variety to the parts I was having the most trouble with. Her feedback was right on. This once again proves that we live in a feedback-rich world, but most of us, however, do not do enough to harvest the feedback that exists.

David: Seek advice from many; accept advice from few. As I went to various Toastmasters clubs to practice for a high profile speech contest, on two specific points I heard constant and unanimous criticism. I had learned that the quickest way to please no one is to try to please everyone, so with each rehearsal when criticism on those two points was raised, I thanked the evaluator for his comments and then promptly told myself, "This is not negotiable." I then turned to my most trusted advisor, my wife, Beth. She, knowing me better than anyone else, said, "Yes, this is who you are, and this is right for you." The lesson was clear: Turn to people who know you best for the best advice.

12 The Results Achieved Over Time Evaluation

It is all well and good to find out that the participants enjoyed the presentation and that all of the presentation's components worked. However, the acid test is whether the presentation had a long-term impact on the "bottom line," the corporate culture, or whatever the desired goal was of the presentation. This is the most important and the most difficult-to-measure form of feedback. But measure it we must if we are to objectively determine the ROI (return on investment) for doing the presentation.

One way to get some of this data is to survey the people who attended the presentation and ask them for tangible proof that the materials that were presented have, in fact, been put into practice and have made a positive difference to the attendees and/or to their organizations. To better determine the long-term effects of the presentation, conduct a survey three months, six months, or a year after the participants have attended the session. You can ask some general questions about what the participants remember and what they have been able to use, but you should also ask specific questions regarding how well the material presented has transferred to the participants' actual work setting. The following figure is an example of a post-presentation evaluation form.


Presentation:________________________________________________________

Presenter:__________________________________________________________

Date of Presentation:__________________________________________________

As a follow-up to your attending this presentation, please rate its long-term effectiveness:

The overall effectiveness of the presentation was:

Poor

     

Excellent


1

2

3

4

5

6

7

Please list one to three specific things you have been able to apply from the presentation session in your place of work and/or home life:

  1. __________________________________________________________

  2. __________________________________________________________

  3. __________________________________________________________

Please list one to three specific benefits in your place of work and/or home life that have been derived from your attending this presentation:

  1. __________________________________________________________

  2. __________________________________________________________

  3. __________________________________________________________

Please estimate the ROI (return on investment) from your attending this presentation:

0%

25%

50%

75%

100%

Please explain:

__________________________________________________________

__________________________________________________________

At this point, do you have any suggestions for improving the presentation or its effectiveness?

__________________________________________________________

__________________________________________________________

Thank You

____________________



Figure 7-7: SIX MONTHS TO ONE YEAR POST-PRESENTATION EVALUATION FORM

Table of Contents, Master Presenters Resources Page Previous Section, Master Presenters Next Section, Master Presenters