UPDATE: I’ve added a sample text to the bottom of this post for those who want to try a test.
On my FutureBible blog I’ve been looking at the subject of reader fluency as a measure of comprehension. This is based two articles published in the journal of the International Reading Association:
- Fluency: Bridge Between Decoding and Reading Comprehension (citation) (pdf)
- Oral Reading Fluency Norms: A Valuable Assessment Tool for Reading Teachers (citation) (pdf)
It’s quite difficult to measure how well someone understands a text. You can ask content questions such as, “Where did the people build the tower?” and “How did God stop the people from building the tower?” but these types of questions often focus on details while people, when they are listening to a story like the Tower of Babel, are often focusing on the story and its main point.
According to research done on fluency, there has been shown to be a correlation between how easily a person can read a text and how well they understand it. If that were true, then we might be able to easily test comprehension simply by testing fluency.
I’ve performed one such test and published the preliminary results. My wife and I took turns reading different versions of the Tower of Babel story in Portuguese. Each text was actually a combination of two different translations. What’s interesting about the test is that I actually made more errors reading the supposedly easier text, O Livro. O Livro was meant to be an equivalent of The Living Bible in Portuguese.
There are three possible reasons for this:
- O Livro is more difficult.
- My text started with O Livro and ended with Almeida so I “warmed up” over time.
- I’m unfamiliar with O Livro but have used Almeida for years.
As you can see it’s really tricky to definitively prove your results one way or the other, especially with such a small sample. The focus of my study was on speakers of a second language. But this kind of test would work equally well on testing speakers of a text that is written in their mother tongue. You might consider putting together a test like this and trying it out on your family or friends. You can make note of errors by following along as the person reads and underlining where they falter or make errors. Or if you’re feeling high tech you can record the readings and then study them later.
See my post for details on the study and please let me know if you try this test.
UPDATE: Here’s a text for you to use: ORF test for English Bible versions
- Print two copies.
- Fold one copy in half.
- Ask a volunteer to read either version 1 or version 2.
- While they read, underline places where the reader makes a mistake or stutters.
Calculate the number of errors per paragraph.
I’ve tried to pick an unfamiliar text and also split the text evenly. Let me know if I’ve made any mistakes!
Translation ABC is Version, 1 paragraph 1 plus Version 2, paragraph 2
Translation XYZ is Version 2, paragraph 2 and Version 2, paragraph 1
I chose these two translations because they are the most common versions printed in South Africa.
Here are results for the four readers I tested:
|Reader||ABC errors||XYZ errors|
Please let us know your results if you try this test on anyone!