The Importance of Findability for Learners

Dec 16, 2016 by

How do you feel when you go to find information on a website, and you just can’t find it? This happens to me all the time when I want to find out what some new ed-tech wonder product does, and I visit the website and can’t see any screenshots, any descriptions, or any videos of the product in action. I find it incredibly frustrating and this story generally ends by me giving up on even signing up for a trial. The same thing happens to students when they go to find information and it is buried in a non-sensical place.

As everyone finishes a semester, and prepares documents and course shells for the next, it seems a good time to share this article, The Impact of Findability on Student Motivation, Self-Efficacy, and Perceptions of Online Course QualityWhile the research targeted online courses, many face-to-face courses are now accompanied by a myriad of resources that live in an LMS course shell and I think there are also implications for findability in course packets and syllabi as well.

For this article, one of the researchers, Dr. David Robins, User Experience Design professor at Kent State University, has presented the study in a webinar format available on YouTube. Their research question: What happens when students have trouble finding components of a course?


 

The researchers took two courses that were well-designed and passed Quality Matters standards, and then “broke” them in terms of findability. The broken courses still technically passed QM standards, but the components were harder to find. Students were asked to perform scenario-based tasks in the online courses.

Sidenote: If you’ve never seen a standard software usability test, here’s a nice “findability fail reel” for a mobile website with questionable usability.

I don’t think anyone will be surprised to find that poor findability correlated with decreased self-efficacy and decreased motivation. However, there was an interesting set of actionable findings regarding navigation and visual design that came from researchers watching participants attempt to navigate the courses. Consider looking for these types of things in your course or syllabus and then improving them:

  • navigation items that are not grouped into logical categories
  • poor labeling (e.g. using the file name instead of a true description)
  • poor categorization (e.g. placing an exam review under “Course Documents” instead of in the section labeled “Prepare for the Exam”)
  • deeply buried content (e.g. syllabus is buried four levels deep)

This article also got me thinking about whether the most important items of a syllabus might be presented in a more 21st-century-friendly manner. There is a whole rabbit hole of syllabi created as infographics on the Interwebs.

Probably your university is still going to want an old-fashioned text version, but maybe students could use more visual infographics for what I would consider the top-5 syllabus items of interest to students:

  • How is this course graded?
  • What are tests like?
  • Are there any projects or papers?
  • Do I have to attend class?
  • Is there group work?

As well as the additional syllabus items that instructors want them to know:

  • What are you going to learn?
  • Why should you care about what you are going to learn in this class?
  • How strict is this instructor on deadlines?
  • What is considered good/bad behavior in this class?
  • What are the instructor’s pet peeves? (come on, that’s a real thing and whole chapters of your syllabi get devoted to these issues)

Challenge: Take a fresh look at your syllabus and/or course shell. Assume that you do have findability issues and look for them. If you don’t think you have them, had over the questions above to a friend or family member and see how long it takes them to find the key components. Revise and improve the findability of important components to lower student frustration for the next semester.

Note: A weekly bite of learning design and a challenge goes out every week. If you’d like to have it delivered to your inbox, sign up at Weekly Teaching Challenge.

Reference:

Simunich, B., Robins, D. B., & Kelly, V. (2015). The Impact of Findability on Student Motivation, Self-Efficacy, and Perceptions of Online Course Quality. American Journal of Distance Education, 29(3), 174-185.


Possibly Related Posts:


read more

Learners Need to Focus on Errors

Dec 9, 2016 by

Let’s move on to the excellent article It’s Not How Much; It’s How: Characteristics of Practice Behavior and Retention of Performance Skills, by Duke et al. (2009), which is another dive into analyzing what leads to good retention of learning in music education.  Just to be different, I’ll start with the conclusion, and then circle around to the study construction.

“The results showed that the strategies employed during practice were more determinative of performance quality at retention than was how much or how long the pianists practiced”

17 students (advanced piano performance students) were given a 3-measure passage from a difficult Concerto to learn. The students each followed the following protocol for practice:

  • 2-min warmup of their choice
  • As much time as they need with the passage, a metronome, a pencil, and a piano, to lean the passage well and play it confidently at the target tempo.
  • A 24-hour break away from the music, with a promise not to practice it at all

The next day, the students had the same 2-minute warmup period (asking that they do not warm up with the practiced passage). Then they were asked to play straight through the 3 measures at the target tempo 15 times without stopping (these are the retention trials).

After recording numerical data about the practice sessions and retention trials, the researchers watched the recorded practice sessions to make detailed observations about the techniques each participant used to practice. Then they ranked the retention trials of the 17 participants, examining tone, character, and expressiveness of musical performance.

Here are the practice variables that were NOT related significantly to the retention trial rankings:

  • total time practiced
  • total number of times through the passage during practice
  • total number of correct and near-correct times through the passage

Here are the practice variables that were significantly related to the retention trial rankings:

  • number of complete incorrect performance trials (positive correlation)
  • percentage of all complete trials that were correct (negative correlation)
  • percentage of complete trials that were correct and near-correct (negative correlation)

Three of the participants were ranked as clearly superior to the other 14 students. Examining the practice sessions from these three students yielded a set of 8 strategies (see p. 317 of the article for the full list), a few of which are summarized here :

  • early in practicing, the playing was hands-together
  • when errors appeared, they were addressed immediately
  • the precise location and source of each error was identified accurately, rehearsed, and corrected
  • the tempo was varied
  • target passages were repeated until stabilized (no more errors)

While the other 14 players would employ some of the 8 strategies, it was only the top 3 that used all of them. In other words, the top-performing practicers had very good metacognition skills. They could accurately identify their mistakes, sometimes anticipating them in advance, and knew what kinds of actions to take to work in the direction of correcting the mistakes and avoiding them in the future.

One of my key takeaways from this article was a quote from the literature review, which I think is much more widely applicable to many subjects (not just music):

“making practice assignments in terms of time practiced instead of goals accomplished remains one of the most curious and stubbornly persistent traditions in music pedagogy”

In mathematics, we assign lists of problems, and students are to do all the problems regardless of their mastery over time. I’m wondering if we shouldn’t give the students a suggested list to practice from, with goals about what they should learn, and tell them to practice until they are reasonably sure they have accomplished the goals. I might go so far as to ask them to do the assignment in pen, crossing out and correcting mistakes as they go, so they can see the progress towards mastery of the topic.

I’m afraid that our relentless focus on time or quantity practiced might be handicapping students’ metacognitive abilities. Do our students actually know when they have mastered a topic? Do they transparently remember the mistakes they make after erasing them and overwriting them? I’m reminded of the Trevor Ragan video about Blocked and Random Practice that says that practice is going to be ugly. I think practice that produces learning IS going to be ugly, and somehow we need to help students be okay with this.

Challenge: Think about how to structure student out-of-class practice so that the focus is not on time or quantity, but on the error analysis and thoughtful correction. Try to make the whole purpose of the assignment around accurately identifying mistakes and working towards mastery of the material rather than “checking off” the to-do items on a list.

Note: A weekly bite of learning and challenge goes out every week. If you’d like to have it delivered to your inbox, sign up at Weekly Teaching Challenge.

Reference:

Duke, R. A., Simmons, A. L., & Cash, C. D. (2009). It’s not how much; it’s how characteristics of practice behavior and retention of performance skills. Journal of Research in Music Education, 56(4), 310-321.

Possibly Related Posts:


read more

Learning Math is Not a Spectator Sport

Dec 4, 2016 by

In November, I gave the keynote at the American Mathematical Association of Two-Year Colleges (AMATYC) Conference in Denver.

I have given versions of this talk that are not specific for mathematics, but I don’t have recordings of those. I promise that the math in this talk is not inaccessible and is used more for examples than a framework for the talk. In other words, don’t let the word “math” scare you away. The alternate version of the talk is “Learning is Not a Spectator Sport.”

Three triangles surrounding a central triangle with the letters C, I, and D
The first half of the video is the awards ceremony, so I’ve directed the embed link below to begin when the keynote actually begins at 45:48 (direct link to video on YouTube beginning at the keynote is here).


The talk emphasizes the importance of interaction, and as such, this talk has a lot of audience interaction in it near the beginning, so you may want to jump through some of that interaction as you watch (between 51:30 and 1:02:00).

At the end of the keynote, audience members are invited to participate in a Weekly Teaching Challenge to continue exploring the ideas and research in the talk. You’re invited too. Just sign up!

 

Possibly Related Posts:


read more

Why high contextual interference?

Dec 2, 2016 by

This week I followed a hunch and, with the help of a friend who is a music educator, dug into some additional research around this idea of blocked and random practice. In music there are a few goals to achieve with any passage of music:
  • can you play a passage accurately by itself?
  • can you play the passage in the larger context of the piece?
  • can you play the passage to tempo?
  • can you play the passage with the right expression?

Think about these goals in your own subject area and see if you can find a similar set of goals. For example, here are some potential goals for solving a math problem:

  • can you find the correct solution?
  • can you solve the problem in an elegant way?
  • can you prove your solution is correct?
  • can someone else understand your solution?

The first research paper I looked at was When Repetition Isn’t the Best Practice Strategy (2001), by Laura A. Stambaugh. A short summary is available here, though the original paper is a bit harder to get ahold of. I’ll elaborate a bit on the summary with the relevant points to our study of learning design.

Students were asked to practice three passages (denoted below in three colors) in either blocked- or random-formatpractice sessions. The three practice sessions were covered on three different days (denoted 1, 2, and 3 in the diagram below). The performance during the last three trials of each practice session were used as the baseline measure of comparison for the retention measures.

music-research
In this experiment, practicing “randomly” meant practicing the same three passages in either At the end of the three sessions, there were no performance differences between the two groups. However, when tested for retention, the blocked-practice students’ performance began to slow to the level of early practice in the trials. While the accuracy of the two groups of students was still the same, the random-practice students could now play the passages faster than the blocked-practice students. Stambaugh also tested transferability of skills, but did not find any statistically significant differences from this experiment. One other variable that Stambaugh thought to test was attitude towards practice depending on the research treatment (maybe students will really dislike random practice or blocked practice?). Here too, there were no statistically significant differences in attitude towards practice between the two student groups.

One of the reasons I find this article interesting is that it discusses the idea of contextual interference, the amount of cognitive disruption the learner experiences during practice with multiple tasks. When the learner has to redirect attention as the tasks change, this results in a high degree of contextual interference. When the tasks don’t change much (blocked practice), the brain can go into a sort of “autopilot” and stop paying attention. At this point, there may not be much point to practicing more on that day. Practicing the same things on a different day would have positive effect (that’s spaced repetition).

 

Possibly Related Posts:


read more

Recorded Webinar: Teaching Math in 2020

Nov 30, 2016 by

Just realized I never shared this webinar video from 2014 (you know, back when 2020 still seemed pretty far away).

What Does Teaching Math look like in 2020?

With every new iteration of technology, we create a generation of students whose primary media “language” for learning and interacting with the world is different than the one before it. In the last 5 years, technologies like free online videos, personalized learning software, and mobile devices, have been chipping away at the corners of education and traditional teaching. Technology-enhanced learning is here to stay, and it will alter the face of education, like it or not. This webinar is your guide to navigating and thriving in this new world.

Possibly Related Posts:


read more

Strategies for Escaping the Echo Chamber

Nov 29, 2016 by

After the 2016 U.S. election ended, we all got a rude awakening to just how bad the echo chambers and filter bubbles have gotten. Our social networks are streams of one-sided news stories. News outlets are increasingly biased. Viral fake news spreads like wildfire and the implications are scary.

In light of recent events, I decided to make a conscious effort to escape from the echo chambers and redesign my consumption of news to try to get a better sense for what is a reflection of true reality and what is not.

filter-bubbles-and-truth

In case you find yourself in the same situation and would like to try to escape your echo chamber, here are some of my strategies:

1. De-newsed Facebook: One by one, I removed every single news source I encountered from Facebook using the “Hide” feature in the upper right hand corner of shared posts. This means I removed news sources I considered biased as well as those I consider legit. I will no longer allow Facebook to decide what news I see and what news I do not see. I just don’t want to see any news on Facebook. Period. On the bright side, Facebook has returned to a state where I actually do see pictures of friends, rants about “new math” and cute cat photos – which is a nice haven from what it had become. This removed my largest echo chamber source of news.

In case you don’t know how to turn off feeds from sites in Facebook, here’s what that looks like:

hide-in-facebook

2. One reliable news source: I read a single reliable news source each morning (for me that is the New York Times. In particular, I read using the “Today’s Paper” app, which delivers the paper in a very similar fashion to the printed version, without ads. This is important because if you just visit a news site, you still fall victim to following only the stories you’re likely to click on instead of seeing a list of all the stories of the day. While the NY Times might not be your choice for most unbiased news, I figure this is balanced by living in conservative Utah. Don’t judge. Also see #3.

3. Highly-biased contrast bumpers: After reading my daily dose of news, I  go to Blue Feed Red Feed, a project from the Washington Post. This provides the most extreme liberal and conservative “viral” posts on a variety of topics. While seeing some of these stories makes me a bit ill, I find it better than seeing delivery from friends and family on Facebook (with additional commentary from them that makes me even more ill). What I’m starting to find is that often both “sides” often tell an incomplete story, the part of the story that makes them look good or the other side look bad. One day last week, the blue feed had stories like “Trump has only lobbyists in his cabinet” and the red feed provided stories like “Trump fires all lobbyists from his cabinet.” Reality was somewhere between those two stories – Trump did seem to have developed a cabinet with a lot of lobbyists AND he then fired most of them.

“Truth” is somewhere in between the bumpers of the most liberal stories and the most conservative stories. But knowing what the bumpers are on these extremes helps me rationally evaluate the stories I see throughout the day and also helps me with #4.

4. Avoidance of clear click-bait: Armed with the most atrociously viral clickbait of the day (from step #3) I can more clearly avoid stories I see on Twitter and banner links to stories that are the viral nonsense of the day. Clicking on this stuff means the world will just produce more of it. Making these sites profitable to advertisers is how they stay in business. If I had not read some of the viral headlines on Blue Feed Red Feed, I would be more susceptible to clicking on stories that are obviously designed to goad one side or the other.

5. No TV News: TV news is designed to be entertainment. So if you’re watching TV News, I’d really consider stopping all together. You don’t realize how bad it is until you remove yourself for a while. When I am forced to watch it in airports, I find it hard to believe that it provides any real value other than making us all terrified of each other (see #6).

6. Actually talk to people who are not like you: I am making a more active effort (see Lean in to the Discomfort) to try to talk to other people who are not “just like me” more: At the gym, in the grocery store, and while out on walks. I am trying to seek out those who are from other generations, other educational backgrounds, other religions, etc. and engage in conversation. We have got to put down our echo-chamber-filled devices and apps and get to know the people in our physical proximity again.

Those are my strategies, what are yours? And if you don’t think you’re in an echo chamber … well, how do you know you’re not?

Possibly Related Posts:


read more

Why Random Practice is Important

Nov 28, 2016 by

As educators, we often find ourselves in the uncomfortable position of trying to explain why students don’t seem to have learned what we know we’ve taught them. Economics instructors ask math instructors, “How come these students who have taken College Algebra still don’t understand slope?” Science teachers ask English instructors, “How come students still don’t understand basic grammar rules when they write in my science class?” The key here is to understand that students aren’t learning skills in a way that helps them to transfer the skills to new situations – the learners have compartmentalized the skill to a particular domain and it doesn’t get sufficient escape velocity due to lack of random or varied practice.

In sports, there has been some eloquent research showing that random practice leads to more transferrable and long-lasting skills than blocked practice. It’s worth taking a short dive into this research area.

shea-and-morgan-research

The gains shown in blocked practice erode when we look at longer timelines. Random practice provides short-term gains AND holds up in the long-term.

Watch the 16-min video “Motor Learning: Blocked vs Random Practice” by Trevor Ragan. He does a lovely job of walking through some of the motor learning research that very eloquently shows that “random practice” is more effective for transference and long-term retention than “blocked practice.” This is basically the same concept as massed vs varied practice discussed in cognitive science.

If you’re interested in reading the research that Ragan touches on in the video, you can find some of it in these papers:

Shea, J. B., & Morgan, R. L. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5(2), 179.

Hall, K. G., Domingues, D. A., & Cavazos, R. (1994). Contextual interference effects with skilled baseball players. Perceptual and motor skills, 78(3), 835-841.

In education we are really good at having students practice the “Do” of the “Read, Play, Do” process that Ragan describes in the video. “Do” skills are orderly and easy to monitor and assess. How can we shift to the messier strategy of having students practice all three parts of the process? For students you teach, what is the equivalent to practicing basketball shots from a variety of distances with different blockers around them?

Weekly Teaching Challenge: Consider all the topics you teach next week and design one new activity that focuses on “random” practice instead of “blocked” practice.

If you’d like the weekly teaching challenge delivered to your inbox each Friday, sign up to receive the Challenge here.

Possibly Related Posts:


read more

Lean in to the Discomfort

Nov 22, 2016 by

When you walk in to a room full of people, choose someone who seems the most different from you (on the surface) or the person that seems the “scariest” to start a conversation with. Start there. Lean in to the Discomfort of having that awkward first conversation. In all likelihood, you DO have something in common with the person – seek to find it. If you always start with the conversation that you perceive to be the most difficult one to have, you will, over time, reduce your own fear of talking to strangers.

Diagram showing 10% of the iceberg above the surface

90% of an iceberg is unseen beneath the surface of the water

About 90% of an iceberg is unseen beneath the surface of the water and likewise, the majority of who we are as a person is beneath what you can see or hear on the surface. Consider all the characteristics of a person that you can’t “see”:

  • what sports do they like
  • what do they do for a living
  • what is their family like
  • what do they do for fun
  • where do they live
  • what do they do when they hang out with friends
  • what are their favorite foods

Whether at a conference or having Thanksgiving Dinner with far-flung relatives, there is so much to learn from the people that surround us. But we have to actually engage them in conversations and interact. If we want our students to interact more to learn, we need to get better at modeling this willingness to have conversations with strangers or those who differ from us.

When I walk into a room and feel the first uncomfortable feelings of having to mingle or that I will be alone in a room full of people, I remember the phrase “Lean in to the Discomfort,” take a deep breath, and begin the first conversation.

Possibly Related Posts:


read more

AMATYC Keynote Notes: Durable Learning

Nov 22, 2016 by

In the 2016 AMATYC keynote, I covered three main themes:

  1. Interaction & Impasse
  2. Challenge & Curiosity
  3. Durable Learning (this post)

Three triangles surrounding a central triangle with the letters C, I, and D

Here are references and resources for Durable Learning:

What is durable learning? The learning design practices that make learning “stick” over the long-term. These include (but are not limited to) spaced repetition, knowledge retrieval, interleaving, and varied practice.

A really good book on the subject of durable learning is “Make It Stick” by Brown, Roediger, and McDaniel.

We also took a dive into some cognitive science and again, there is a fantastic, easy-to-read book I recommend “Cognitive Development and Learning in Instructional Contexts” by James Brynes.

We explored the idea of a schema – a mental representation of what all instances of something have in common (plural is schemata). In particular, schemata help you to categorize your experiences,  they help you remember what you are experiencing, they help you to comprehend what you are experiencing, and are important in developing the ability to problem solve.

Visual representation (with no numbers) of distribution - shown as a set of arcs

A schema for distribution

When confronted with a new situation, learners try to run a schema they already have. This leads to all sorts of interesting misconceptions.

not-distribution

By engaging the learner in varied practice, we hope to modify the existing schema.

No numbers representation of distribution with visual arcs and plus-minus signs to hold the spaces

A better mental schema for distribution because the spaces are now held by plus-minus signs

To help the learners refine schema, we can abandon massed practice for varied practice. In massed practice, the learner does nothing but activate the exact same schema over and over. In varied practice, the learner has to distinguish between different schemata in order to successfully complete the practice.

massed-practice-and-varied-practice

There is a lengthier talk I gave on cognitive science in the context of algebra called “Algebra is Weightlifting for the Brain” (not the world’s best recording, but you’ll hear more about the ideas of Information Processing Theory and see plenty of math examples).

We didn’t quite get to interleaving in the talk, but we will cover that during the teaching challenge.

What is the Teaching Challenge?

For the next year, I will send you a teaching challenge every week to help us, together, change the way students learn and engage. The challenge will be delivered each week by email and will include:

  1. Something to learn or ponder
  2. Best practices shared by participants in previous challenges
  3. A new challenge

Sign up for the teaching challenge here. All are welcome.

Possibly Related Posts:


read more

AMATYC Keynote Notes: Challenge and Curiosity

Nov 21, 2016 by

In the 2016 AMATYC keynote, I covered three main themes:

  1. Interaction & Impasse (last post)
  2. Challenge & Curiosity (this post)
  3. Durable Learning

Here are references and resources for Challenge & Curiosity:

First, I have to point you to one of my favorite books on the subject, A Theory of Fun for Game Design, by Raph Koster.

Quote from Game Design: “How do I get somebody to learn something that is long and difficult and takes a lot of commitment, but get them to learn it well?” – James Gee

How do players learn a game? 

  • They give it a try
  • They push at boundaries
  • They try over and over
  • They seek patterns

It looks something like this:

Shows web of many nodes and branches coming off a person, with bridges between branches and potential paths to expand knowledge.

How does a player learn a game?

How do we teach students?

  • We tell them what we’re going to tell them.
  • We tell them.
  • We tell them what we told them.
  • We have them practice repetitively.

It looks something like this:

Very few linear paths branching out from the person at the center. Few nodes and few places to expand on knowledge.

How do we teach students?

Reference: Productive Failure in Mathematical Problem Solving

There’s a much wider body of research on productive failure worth reading.

Video: Playing to Learn Math

Resource: Good Questions from Cornell

Resource: Classroom Voting Questions from Carroll College

Design more activities that let the student figure out the mathematical puzzle, instead of providing all the secrets yourself.

Shows the graph of a rational function with vertical asymptote at x=5 and horizontal asymptote at y=2.

Explain the differences in the graphs: The student is given five rational functions to graph, each function looks only slightly different mathematically but produces very different results.

Possibly Related Posts:


read more