Who Should Decide How You Decide?

Each approach can be profitably used, depending on the subject matter and the targeted population. Per your example, if the target was an experienced supervisor, scenario 2 would work. If the target was a beginning supervisor with lots of work experience, scenarios 1 would work. And scenario 3 would work for those beginning supervisors who don’t have a lot of work experience.

BTW…where can I buy the Sleep at Work eye stickers?

Dude! Most excellent! Good deal. You still da man.

Great post Tom! I’m with you on making it more realistic with ambiguity by letting them make the decision then access more information after. But, if you’re going to go this route it’s even more important to include quality answer options. Like you mentioned, many learners will just guess away and click their way through until they make it to the end unless you have quality answer options that make the learner want to access more information.

Another option for timing of providing links to references/resources for further study that designers might consider:

As I learner, I would much prefer the option of being provided with these links to references/resource material in an emailed summary report at the end of the training. This allows me to review/study/reflect AFTER completion of the elearning course (this is particularly true for those courses that are “required” and for which I do not have unlimited time to complete).

I personally find it inconvenient/distracting when I’m taking an online course to ONLY be provided with the hot links for references/resources for further study DURING the course. I feel I get distracted off the e-learning course and sent into 50 million directions, rather than being able to complete the course.

Moreover, having it in an emailed report not only allows me to go back and review over time, but compiles the references for me to refer to later, bookmark, etc.

I could use some of those eye stickers so bad

I support a scenario-based approach that focuses more on learner decisions and use of information rather than pushing content on them. Nice post.

What I like about all 3 options is that the learner who already knows the content is able to move through the course quickly while the learner who is new to the material can explore for more information.

The use where I like options 1 and 3 (info before and info before/after) is where the information in the course isn’t used on frequent basis and the learner isn’t really expected to retain all of the information. For example, a course about a company policy like you used above.

a. Typical course: Boring course about the policy. Short quiz after asking you to regurgitate the information. Learner forgets the information in a few days.

b. Scenario approach: Scenarios right away. Learner is asked to use a resource (policy handbook) and find the answer. In a real
world setting, the learner doesn’t need to retain the info and knows where to go to solve the problem (You just need to be sure to let them know what that resource is and where they can find it).

I really like the second option, in which the learner must make a choice without too much hand holding before hand. I think this approach is most likely to stimulate a meaningful consideration of the problem. The “it’s okay to make a guess” prompt is a friendly little reminder that it isn’t the end of the world if you get it wrong. The eLearning environment feels like a safer place to “fail” (though that’s not the best word for getting a wrong answer), because there is no audience to observe the failure. In fact, I’m guessing that many eLearners give more thoughtful answers the next time around when confronted with this approach.

Giving too much away ahead of time takes away the learners opportunity to sort things out. Some feedback after a wrong answer is appropriate, including some insight as to why the “right” answer is the best choice.

Good thoughts.. I would like to provide more of this type of scenarios, but I still need to validate the learner is indeed absorbing the information. How could this concept be built with measurements to make sure that all users end up selecting the appropriate answer, either before or after completing the scenario, or does this still have to be done with a separate exam quiz at the end of the lesson, and if so, what are some suggestions for getting quality scenario questions into the exam.

We could debate the merits of each option you offered, but any of these choices is clearly preferable to the “typical course” approach that Bryan noted in his post. Sadly, I must admit I have created such courses in the past. My apologies to those who may have had to endure such drudgery. After reading your post, Tom, I’m going to give serious thought to applying Option 2 on the redesign of an existing mandatory CBT on our Information Security policies. The course is a real snoozer as it stands today–the kind of CBT where those eye stickers should be issued along with the course link. Thanks for sharing your insights, all!

Tom,

This post is valuable as a resource for me to share with SMEs and stakeholders. The next time I discuss this design option with them, I’ll take your post into our meeting.

I have built scenarios similar to what you describe, with success!

However….

Clients who are new to e-learning tend to feel more comfortable with the linear (book online) design. I’ll use your post to encourage them to go with more engaging e-learning for their learners.

I think they’ll see the difference.

Thanks!

@jenisecook

I disagree with your personal choice. With that choice you are making the common mistake that many instructional designers make. That is determining what learning options will be available based on your personal style and preferences. My answer is always to let the learner decide for themselves. That is what student-centered learning is all about, and plenty of research has indicated that student-centered learning is more effective.

I’ve been a manager/leader for over 25 years and I’ve seen two types of decision-makers: intuitive deciders and researcher deciders. Your statement that “many of us tend to make decisions and then learn from the consequences” may be true for yourself, an intuitive decider, but is not true for all of those research deciders. By taking away the option to do research or to learn prior to making a decision from those types of learners/deciders is the equivalent of telling a blind person “too bad you can’t see the pictures on the screen, just make an informed guess based on your experience of pictures that were described to you in the past.”

I would hypothesize that those who are intuitive deciders won’t bother to click to get more information prior to making the decision anyway. They will naturally answer the question first based on their past experience and knowledge, and then see if they are right. So, by offering more options for the “research decider” you are not at all disadvantaging the intuitive decider.

Finally, if you are concerned about simulating real-life decisions that do not allow time for research, then you need to provide a timed scenario that simulates that instance. Your example is not one where a manager must make a quick decision. The employee has been engaging in the behavior for some time and the manager is just now getting to it. There is certainly time for some research if the manager wishes to learn before making a decision.

An example of a time-based scenario would be: You just received a phone call about a chemical spill in the science lab, you have one minute in which to deal with the problem. Rank the following twenty items into the top ten things you need to do right now.

Then time it and force the learner to answer within the one minute time. Then provide a means for the learner to debrief the scenario through an online discussion as he/she would in their normal work environment.

Good Luck with your course.

“I’ve done a fair share that are just check marks on some [manager’s] annual performance plan with little consideration to the course’s effectiveness.”

Tom, those “e-learning modules” are what I call glorified commercials. I’ve made many of them, too. It’s nice to know I’m not the only one.

Hi Tom. This is a great, thought-provoking post.

Generally speaking, I have found it important to create pauses for reflection several times throughout a course. In reflecting, a decision or choice may or may not be clear for the learner. Above and beyond learning styles, clarity is affected by attention span and distractions. What would cause a learner to “research” may not be attributed to a single learning style.

In any event, since we all learn differently I always incorporate supporting resources (documents, spreadsheets, intranet, internet, etc) and ensure they are accessible at any time throughout the course (via attachments, menus). I will refer to “additional information” and point to the menus. I will sometimes also use the “additional information” as a way to familiarize learners with where and how to find this information outside the module after training is completed. Overall, I find that incorporating supporting resources helps to ensure that the course contains core learning only while not limiting access to supporting information.

Highlighting specific supporting resources at decision/reflection points will be appreciated by any learner who needs additional info to make a decision or to further digest/reinforce their understanding. I use “appreciate” because it is a simple yet thoughtful touch to bring information to the learner in a subtle, non-mandatory fashion (even if they don’t need it) rather than pointing them someplace else to find the information.

tom: speaking to the “check marks on a performance plan” (likened to “butts in the seat” for classroom courses]remark: Measuring the transferred learning in on the job settings would be an ideal way to determine the effectiveness of an elearning course-is the being done by anyone out there? how are or can these measurements be accurately made? especially in a technological setting rather than soft-skills? Any ideas? Desires? thanks!

Tom, great as usual.

However, no offence but… the little grey ‘bean’ man picture at the title? Almost turned me off. It’s not like you at all. I nearly skipped past the article on the main page subconsciously thinking it was an ad, or something old.

Just being honest. Your images are usually a heck of a lot more engaging.

p.s. love the eye stickers, wonderful idea for using at a conference.

I prefer option 3. Let the learner decide how he learns best — Exploring additional content before making the decision and/or exploring feedback after making the decision.

Great post Tom and an excellent example of allowing the user to determine how they will choose to move through the course. Like others I’ve created my fair share of “electronic books” that are geared more toward simply putting out information without concern for changes in performance.

One thing I try to do with my testing though is to put the student into a decision scenario where they much choose a course of action vice a simple yes/no or mulitple choice with immediate feedback. Rather what I’m trying to do is to create a realistic activity where their decision lead to a different decision situation where their previous decision drives the next choice. The choices presented to the student must have a final output and their decisions right or wrong can actually lead to one of three conclusions, the best outcome, the worst or one that while not fully correct will address the original situation they were trying to solve successfully. This type of testing does two things for me, it gages not only the students ability to assess the situation but also tests their ability to apply their knowledge in order to obtain a successful outcome. When I use this approach I also find that my students take the assessments a couple of times to find out what happens when they make other choices.

Thanks again for your great insights.

Great post!

I had a similar situation during one of my course development and used option 1 for a few scenarios and option 2 for some.

This is a great post!

I feel that there isn’t a one size fits all answer to your question.

In the example provided, it looks as though you are encouraging the learner to only review the policy to make a decision.

I think a couple of better questions are: “What do you want your learners to do before addressing the situation?” and “What are the ramifications of the decision you make?” If it’s the black and white policy, that’s one thing. But maybe other times, the additional information might be “AH-HA” moments, like his spouse just lost their job and now he’s working two just to make ends meet for his family, or maybe you talk to another manager to get their opinion.

Not that it negates the company policy, but also provides additional information to why trying to help him is a better answer than the other two (although the other two in this scenario are obvious…lol).

So, I would suggest that during storyboarding, we look at where different types of information would be pertinent (and realistic) and provide it then. This way we encourage the learner to make an educated decision, not just one that we guide them to.

Tom, great post. I agree with Maggie about the two types of learners. We call this the “satisfying approach” and “optimizing approach”. In that case, I think Option 3 would naturally be the best choice, because it addresses both types, and gives the flexibility to the learner to choose when to review the information, if at all.

Some really great comments on this post especially around providing reference and feedback.

One of the more memorable scenes from Karate Kid was when Miagi had Daniel do some chores. Daniel didn’t have context for the exercise. It wasn’t until he was about to quit that Miagi showed Daniel how his painting transferred to fighting.

I always felt that was an awesome example of teaching while holding a little back until the right time.

How would such delayed feedback and reference work in elearning? If it could, how and where would we include feedback and reference?

Great, thought-provoking blog. The second option of creating tension is by far my favorite, especially where it reinforces knowledge that the majority of participants have already been exposed to in the past but may have forgotten (very common with company policies!). However, do I favor this approach because I am a confident risk-taker and I believe in the value of learning from mistakes? Possibly.
My personal view is that (as a stereotype) traditional, educational practices have trained us to believe that there is only one correct answer – the teacher’s! People are anxious about “guessing wrong” and we all remember the other kids laughing when we said something different to the expected!

Instructional design can teach to a new pedagogy… let’s throw out “wrong answers” and just call them alternatives (possibly dead-end alternatives but still not wrong) and whilst we’re at it let’s throw out “the teacher is always right” and that sharing ideas or learning from peers should be labelled “cheating”!

@Maggie: Whoah! Are you really suggesting that if a research learner makes a mistake they NEVER learn from that?!

As for Tom’s decision making style, do you really believe that you have enough info to determine what it is? Aren’t you making a gross assumption? Having been a reader of this blog for nearly two years now – my guess is that he leans the other way. Which is why he provides 3 ways to use this technique not just one.

The truth is learners will sometimes come to our classes with various levels of knowledge, experience, and skill.

By employing this technqiue, you can move your learner from ‘the known to the unknown’ – allowing the learner to make a choice of what they want to do based on their pre-exisinting knowledge and their learning preference. Which is what, ironically, you are advocating in your response.

This approach is also beneficial as an informal pre-assessment to help learners identify what additional knowledge / skills that need to build up.

If you consider the Honey & Mumford Learning types (Activist, Reflector, Theorist & Pragmatist – and their various combinations) – everybody has different preferences and learning needs.

Some want to do everything – they are naturally curious and what to read more info. Others just want to jump in and see how they go, some just want the facts.

I think it’s also important to note that Tom is providing you with ideas – not a prescription. Sometimes people who read this blog really get it wrong – Tom is just showing a technique you can use. It’s not about the content of the example but about the technique used and his intention is stimulate ideas in us creative types.

@LShumpe: I approach eLearning assessment in the way that I would assess face-to-face. In Australia, it is considered best practice to have a minimum of 3 Level 1 assessment methods: to assess the theory, to assess practical application of the learning and to see if they know what to do when a curve ball is thrown at them. And with vocational learning – you should also consider pre-assement and Level 2 assessments (eg. Re-assessment 4 weeks after they’ve done the course, supervisor feedback, QA results, stats etc).

If you really want to get into measuring the effectiveness and have the time you can always go for a Level 3 assessment (compare groups who have done the eLearning against those who haven’t and compare the results).

@Tom: I have used a similar technique where I asked participants to rate their existing knowledge. For example, if they chose ‘This is completely new to me’ – they did the whole section. If they chose ‘I consider myself and expert’ – then it jumped them
to summary page with all the topics listed and they had the option to only complete a particular section (or all if they wanted to). If they said they had some knowledge then it jumped them to the same page as the expert learner.

But I really like how you used this concept – I think I might be borrowing yet again!!

Cheers Jen

The point is that Tom is suggesting building courseware which is learner centred and which meets their individual needs. Giv

@David Anderson

You just moved up three notches on my elearning hero scale with your use of a Karate Kid reference.

Love it.

I like the second choice…Kind of intriguing..will certainly grab learner’s attention..

Tom I appreciate your posts! I am currently in grad school working towards becoming an instructional designer, and I have already learned so much from your blog. You always provide me with food for thought. Thanks for sharing your wisdom!

To answer your question, I believe option 1 would be best in most situations. Typically managers will have access to reference materials before making a decision in the real world. Option 1 will promote managers who will make informed decisions. Since the user does not have access to the information after making a decision, they will be more likely to take advantage of it before making a decision.

If you are creating training for managers in a fast paced, or life or death environment, then I believe option 2 would be best. If a manager needs to make split second decisions with confidence, then it makes sense to only provide an option for more information after they have made a decision. This way they learn to make decisions quickly and with confidence.

Option 2 provides the learner with the ability to rely on episodic memories and knowledge to make a decision. Making a wrong decision in a private environment has fewer consequences. Failing and learning from mistakes can be a great teaching tool because the manager can learn from a negative response. The more one learns from failing, the less likely they are to fail the next time and this can boost their confidence. Being able to take risks in protected eLearning environment encourages the non-confident learner to take more risks. This can be supplemented by comments made in a past blog (http://www.articulate.com/rapid-elearning/why-e-learning-is-so-effective/). “Elearning lets you fail without fear.” Option 2 encourages exploration and testing of ideas. “Worst case, you can always start over. Something you can’t always do in class. “

[…] Who Should Decide How You Decide? […]

Thanks for the timely post Tom! I am just now struggling with the same questions for a scenario-based review course I am designing. I think I am going with option 1 – but in the future when I am less of a scaredy cat, I will try option 2. Also, I am enjoying the Miyagi/Karate Kid comments. Wax on, wax off!

I see no harm in requiring the individual to make a decision. There is no harm in placing them in an environment requiring tough choices to be made.

You mentioned the possibility that “a confident learner could skip the information and go right to making decisions.” I would imagine this option would be used quite often, especially if a particular course is mandated. This is something we encounter in my organization. There is a purpose for the training and it is highly beneficial to the individuals, but if they discover short cuts allowing the course to be completed faster, they will exploit it. In this case, it is not just the “confident learner” skipping the information. What about the realization that “a confidant learner” may opt to skip the information, resulting in the possibility of making a rash decision?

Confident or not, I like the option of allowing the individual access to the information. The military encourages the use of all available resources, yet sometimes you must move forward with the information on hand. This is why I personally like Option 2. I spent time in the military and situations requiring individuals (leaders or subordinates) to make tough decisions build character. There are too many times when that individual cannot wait for additional information and must act with the level of knowledge known. Hopefully the individual is “thick skinned,” therefore, allowing for something to be learned from his/her decision.

I would be curious to see statistical data compiled from the training of those individuals who:
• Ignored the information, regardless of where it was available to them.
• Utilized the information

I feel there is a lot to be gained from it. For instance, if many people answered incorrectly and did not view the additional information, it could indicate the necessity of formal training to occur. We recently had a situation similar to this. An individual failed to pass the training and retraining situation resulting in mandatory, formal one-on-one training.

All the three options are okay but each will be apt depending on factors such target audience, content of programme in terms of stress level required for learners to understand the principle or idea being taught, time alloted etc. I will favour each option after a well researched or informed learning situation is done.
This is indeed an incisive scenario for me as a trainer in a bank.
Thank you so much.

On the one hand, the decision must take into account who your end-users are and the circumstances surronding their enrollment in the course. On the other hand, I think I would generally go with Option 3 (provide information before and after the choice). I especially prefer this option if the LMS documents not only what choice was ultimately made, but also at what point in the process (before or after the decision was made) each user selected the option to seek additional help (or whether sought additional help at all).

I don’t think a report like this should necessarily be used as a means to determine what the best approach is. Instead, I think it would be useful to determine what trends there are with respect to user tendencies and whether or not there is a critical mass that prefers one approach over another. I also think it would be interesting to determine if there are certain characteristics that typically lead to each approach.

Great points. I like the idea of having the material before and after. Michael Allen’s work also mentions delaying feedback in some cases or using other vehicles to provide the feedback. Example: A thermometer based on medication given. This tries to provide feedback as real as possible.

However, in these cases, I suspect there is more time and budget than one might expect for rapid e-Learning. Although the customer usually wnats it fast regardless.

Thanks… John

It looks like there may be new supporting evidence for option 2. It’s summed up in Quinnovator’s latest blog post:
http://blog.learnlets.com/?p=1494

Basically, the article that’s referenced says that allowing learners to make mistakes supports learning.

As a learner who is starting a Master’s in Instructional Design, I prefer option #3. Is there anything more annoying than going through an e-course, being forced to take an assessment quiz and then not knowing why a particular response is considered incorrect? Isn’t the point of e-learning (and traditional learning) to assess and, hopefully, correct our assumptions about processes?