Recently, I’ve been analyzing a batch of learning data and have a question I really want to confirm: Do people who answer more questions also have higher accuracy rates?
This question seems easy to answer by just putting two columns of numbers together. But some people answered dozens of times, while others answered thousands; some had a strong foundation, while others were repeatedly practicing weak areas. Looking directly at the averages can easily lead to oversimplification.
At this point, we need an appropriate tool: Spearman correlation analysis (Spearman’s rank correlation analysis, Spearman’s rank correlation analysis).
The word “rank” might sound intimidating, but you can simply understand it as ordering (the English original is “rank”, meaning ordering).
For Example
| Student | Number of Answers | Accuracy Rate |
|---|---|---|
| Xiao Wang | 20 times | 50% |
| Xiao Li | 50 times | 60% |
| Xiao Zhang | 100 times | 70% |
| Xiao Chen | 300 times | 80% |
Sorted by number of answers from fewest to most:
Xiao Wang → Xiao Li → Xiao Zhang → Xiao Chen.
Sorted by accuracy rate from lowest to highest, it is still:
Xiao Wang → Xiao Li → Xiao Zhang → Xiao Chen.
The two rankings are completely consistent. In this example, the Spearman correlation coefficient is +1. If the order were completely reversed, with the person answering the most having the lowest accuracy, the result would be −1. In reality, things are rarely this neat, so the calculated value usually falls between these two extremes.
How to Use It?
The Spearman correlation coefficient (Spearman’s rank correlation coefficient) is typically denoted as ρ, or sometimes rₛ, ranging from −1 to +1.
| Result | Interpretation |
|---|---|
| Close to +1 | The two rankings generally move in the same direction: when one is higher, the other tends to be higher too. |
| Close to 0 | No clear pattern of ranking in the same or opposite directions is observed. |
| Close to −1 | The two rankings generally move in opposite directions: when one is higher, the other tends to be lower. |
Why Do This?
Let’s look back at the previous example. If we change Xiao Chen’s number of answers from 300 to 3,000, as long as the other data remains unchanged, he will still rank last. Since neither ranking changes, the Spearman result remains unchanged.
Therefore, it focuses more on whether the order is consistent, rather than giving disproportionate weight to individuals based on the magnitude of their values due to extreme outliers.
Pearson correlation analysis, which often appears alongside Spearman, focuses more on the linear relationship between the two sets of raw numerical values, i.e., whether they change roughly along a straight line. Spearman, however, first checks if the rankings change in the same direction, without requiring the raw values to fall on a straight line.
However, switching to rankings means losing some information: the difference between 20 and 50 answers versus the difference between 100 and 3,000 answers cannot be directly reflected in the ranking intervals.
Does Being Close to 0 Mean “No Relationship”?
What Spearman excels at detecting is monotonic relationships: when one variable increases, the other generally gets higher or lower overall.
But some relationships are not monotonic. For instance, an indicator might rise and then fall as another indicator increases, forming an inverted U-shape. Such a relationship is obvious, but Spearman might yield a value close to 0 because the opposing trends cancel each other out.
The Most Important Caution: Correlation Does Not Imply Causation
Suppose we indeed find a positive correlation between the number of answers and accuracy rate.
We can only say: In this dataset, those who answered more questions generally had higher accuracy rates.
We cannot directly claim: Because they answered more questions, their accuracy rate was higher.
There may be other explanations: Students with stronger foundations are more willing to do more practice problems; different students solved problems of varying difficulty; repeatedly solving familiar problems might inflate cumulative accuracy rates.
Conversely, if the correlation coefficient is low, we cannot conclude that practice has no effect. Some students started with weak foundations and improved from 20% to 50% after extensive practice, but compared to those who could already achieve 80%, their accuracy rate remains lower. Comparing different individuals mixed together might obscure individual progress.
Therefore, “Do people who practice more tend to answer more accurately?” and “Can more practice help the same person improve?” are two different questions. The latter requires before-and-after comparisons, ensuring that conditions like problem difficulty and source are comparable.
Summary
Spearman helps us see if two rankings change together; it can reveal associations but does not automatically tell us why.
Reference: SciPy Official Documentation: spearmanr