Measuring Teaching and Learning in Mathematics

Nov 15, 2010 by

This weekend at AMATYC I presented this Prezi presentation on How can we measure teaching and learning in math? My husband was kind enough to act as the videographer for the presentation, and so I can also share the video presentation with you today.

I think the video should add quite a bit of context to the presentation, so I hope you’ll take the time to watch it.  What I propose (at the end) is a research solution that would help all of the math instructors in the country (who want to) participate in one massive data collection and data mining project to determine what actually works to improve learning outcomes.

If you have any suggestions for where to go from here, I’d be happy to hear them.

Possibly Related Posts:


read more

Student Conceptions of Mathematics

Jan 5, 2010 by

sotl_blog_button1

Do you ever get the feeling that you’re not making any real progress with your students?  Sure, they pass tests and progress through the courses (well, most of them), but have you ever just had the uneasy feeling that they really don’t get what math is all about?  Suppose you were to ask your students the following question:

Think about the math that you’ve done so far.  What do you think mathematics is?

What do you think they would tell you?

Well, in 1994, a research group from Australia did ask 300 first-year university students this question (Crawford, Gordon, Nicholas, and Prosser).  They identified patterns in the responses, and classified all 300 responses into categories of conceptions in order to explore the relationships between (a) conceptions of mathematics, (b) approaches to learning mathematics, and (c) achievement.

If you want all the gory details about the study, you’ll have to track down the journal article, but let me attempt to summarize their findings.

Students’ conceptions of mathematics fell into one of five categories (Crawford et al., 1994, p. 335):

  1. Math is numbers, rules, and formulas.
  2. Math is numbers, rules, and formulas which can be applied to solve problems.
  3. Math is a complex logical system; a way of thinking.
  4. Math is a complex logical system which can be used to solve complex problems.
  5. Math is a complex logical system which can be used to solve complex problems and provides new insights used for understanding the world.

The first two categories represent a student view of mathematics that is termed “fragmented” while the last three categories present a more “cohesive” view of mathematics.  Note that the terms fragmented and cohesive are well-used throughout the international body research.  The categories above, as you may have noticed, form a hierarchical list, with each one building on the one above it.

Here’s where this study gets (even more) interesting.  The researchers also asked students about how they studied (changing the language slightly here to “americanize it” a bit):

Think about some math you understood really well. How did you go about studying that?  (It may help you to compare how you studied this with something you feel you didn’t fully understand.) How do you usually go about learning math?

Again, the researchers went through a meticulous process of categorization and came up with five categories (Crawford et al., 1994, p. 337):

  1. Learning by rote memorization, with an intention to reproduce knowledge and procedures.
  2. Learning by doing lots of examples, with an intention to reproduce knowledge and procedures.
  3. Learning by doing lots of examples with an intention of gaining a relational understanding of the theory and concepts.
  4. Learning by doing difficult problems, with an intention of gaining a relational understanding of the entire theory, and seeing its relationship with existing knowledge.
  5. Learning with the intention of gaining a relational understanding of the theory and looking for situations where the theory will apply.

Again, these five categories were grouped, this time according to intention, into two general categories: reproduction and understanding.  In the first two approaches to learning math, students simply try to reproduce the math using rote memorization and by doing lots of examples.  In the last three categories, students do try to understand the math, by doing examples, by doing difficult problems, and by applying theory.  Other researchers in this community have seen similar results on both general surveys of student learning and on subject-specific surveys and have termed this to be surface approach and deep approach to learning (see Marton, 1988).

Still reading?  Good.  Remember my first question? Do you ever get the feeling that you’re not making any real progress with your students? Let’s answer that now.

Here’s how the conceptions of math and the approaches to learning math correlated in this study (Crawford et al., p. 341):

conceptions_of_math

Did you catch that?  Look at how strongly conception and approach correlates.  It’s probably what you’ve always suspected, deep down inside, … but there’s the cold, hard, proof.

Of course, all this is not so meaningful unless there’s a correlation with achievement.  At the end of their first year, the students’ final exam scores were compared to the conceptions and approaches to mathematics (again, for technical details, get the article). The researchers made two statistically significant findings:

  1. Students with a cohesive conception of math tended to achieve at a higher level (p < .05).
  2. Students with a deep approach to learning math tended to achieve at a higher level (p < .01).

Okay, so where does this leave us?  Well, we don’t have causation, only correlation (at least, that’s all we have from the 1994 study).  However, Crawford, Gordon, Nicholas, and Prosser were nice enough to use their research to develop a survey inventory that we can use to measure students’ conceptions of mathematics (1998).  The 19-item inventory (5-point Likert scales) has been thoroughly tested for validity and reliability, and can be found in their 1998 paper, University mathematics students’ conceptions of Mathematics (p. 91).

Suppose you want to try something innovative in your math class, but you don’t know how to tell if it works.  You could, at the very least, try to measure a positive change on your students’ conceptions of math (there are other ways to measure the approach to learning, but this is already a long blog post and you’ll have to either wait for another week, or view my presentation How can we measure teaching and learning in math?).

measure_tlm

To give the Conceptions of Mathematics Questionnaire (CMQ) would take approximately 10 minutes of class time (you should ask for permission from Michael Prosser before you launch into any potentially publishable research).  This would give a baseline of whether students’ conceptions are fragmented or cohesive.   If you were to give the survey again, at the end of the semester, you would be able to see if there is any significant gain in cohesive conceptions (or loss of fragmented conceptions).

So, I have permission (I met Michael Prosser this summer when he was at a conference in the U.S.), and I’m going to use this in all my classes starting next week.  I figure that when I start to see major differences on those pre and post-semester CMQ inventories, that I’m doing something right.  If I’m not seeing any gain in cohesive understanding of mathematics, then I’m going to keep changing my instructional practices until I do.

Papers that you will want to find (and read!):

Crawford, K., Gordon, S., Nicholas, J., & Prosser, M. (1994). Conceptions of Mathematics and how it is learned: The perspectives of students entering University. Learning and Instruction, 4, 331-345.

Crawford, K., Gordon, S., Nicholas, J., & Prosser, M. (1998, March). University mathematics students’ conceptions of Mathematics. Studies in Higher Education, 23, 87-94.

Possibly Related Posts:


read more

How can we measure Teaching and Learning in Math?

Oct 12, 2009 by

Last week I prepared a new presentation for the MichMATYC conference based partially on the literature review for my dissertation.  In my dissertation I am studying instructors, but in this talk I addressed both the instructor and the student side.  It was also the first presentation I’ve built using Prezi, and it was interesting to re-think presentation design using a new tool.  Of course, the presentation misses something without the accompanying verbal descriptions, but there is enough information on here that you can begin to understand the problem (we don’t actually know much) and the solution (common language, common measurement tools).

mtl_developingconceptions

There are also a few new cartoons/illustrations in this presentation.  I’ve started just paying for a couple of illustrations per presentation to help viewers to understand (and mostly to remember) difficult concepts.  Just to give you a rough idea in the time involved to create something like this, I spent about 18 hours on the Prezi build (which doesn’t even begin to account for the time spent doing the research).

How can we Measure Teaching and Learning in Math?
 

Possibly Related Posts:


read more

Mathematics Instructional Practices

Apr 30, 2009 by

What follows are ten mathematics instructional practices (MIPs) are meant to capture the nuances of the majority of mathematics instruction for the first two years of college mathematics.

  1. Lecture
  2. Collaborative Lecture
  3. Cooperative Learning
  4. Inquiry-based Learning
  5. Emphasis on Application Problems
  6. Emphasis on Project-based Learning
  7. Emphasis on Multiple Representations
  8. Emphasis on Communication Skills
  9. Mastery Learning
  10. Emphasis on Formative Assessment

These MIPs are not part of my Ph.D. dissertation research, they are just something I needed to do in order to focus my literature review on a narrow and clearly defined set of instructional practices for mathematics. Thank you to everyone who gave me feedback in the last two days – it was vital to developing a comprehensive list of practices.

I am permanently posting the list (with descriptions and illustrative examples) under the Resources tab on this website (go here). You can also print the full list and research notes from the feedback surveys in the last two days (here). I only ask that if you use these as part of a study, paper, or presentation, that you use the proper citation (or if it is digital, link back to the site).

Please note that the use of technology for instruction is not its own category on the MIP list (ironic, I know). However, technology is used as part of the implementation of various instructional strategies. Emphasis on technology itself is not an instructional strategy that can be easily summarized and is left to future exploration. The practices in the MIP are also meant to describe the nuances of teaching face-to-face (and not online). The nuances of mathematics instructional practices for online courses will be left for future research. It is important to first establish a baseline for MIP that occurs in face-to-face instruction.

Remember, no instructor will fall distinctly into any one category. All of us will use a mix of each of these practices, to varying degrees. In the final instrument to measure instructional practice, you would be asked to what extent you use each strategy in your courses (at a specific level of mathematics) and you would respond on a scale. Likewise, many activities that you do in the classroom may span more than one category of instruction. The intent of the categories is to lend description to different nuances of instructional practice in mathematics.

If you would like to publish this list as part of a scholarly work, or you have suggestions for where to seek publishing of the MIPs and the development of the list, please let me know. I hope it will be useful as a way to focus the language of the research in mathematics instructional practices with a common language.

Possibly Related Posts:


read more