Regression to the Mean

Imagine a church planter. If he presents the Gospel to 100 people, what are the odds that 60 of them will become disciples? It might seem a fairly impossible question to answer, so let’s consider a different question: Dr. Daniel Kahneman suggested to one class that they imagine taking a coin, holding it in the hand, and ‘flipping’ it (like you’d flip a coin) toward a target. Imagine doing this 100 times. What are the odds that 60 of the ‘flips’ will be within two inches of the target?
The two questions are similar in that part of the result is due to skill (communicating the Gospel, aiming the coin) and part to forces beyond your control (cultural issues, state of the person’s heart, the work of the Holy Spirit; or, atmospheric pressure, physics, gravity, and other factors). Some would call the factors skill vs. chance.
You might argue there’s no such thing as ‘chance’ in the presentation of the Gospel; all we’re doing is letting the word stand in for the aggregate of all the unseen, uncontrolled realities. When the Gospel is presented some accept and some walk away, and we don’t know why. Looking at any given Gospel-recipient, we might think the chance of his becoming a disciple would therefore be, from our perspective, 50/50. But remember the skill portion.
If you share the Gospel with him in a language he can’t understand, the probability of his accepting it is pretty near zero. If you have a skilled communicator who can present the Gospel in a way that avoids any basic misunderstandings, his chance of becoming a disciple might be greater than 50%. (Or it might still be significantly less, depending on cultural factors.)
It’s hard to know from a few ‘samples’ (a few instances of presenting the Gospel) because of the chance portion. This is where we come to the statistical idea of regression to the mean. Granted, I’m not a Ph.D. in statistics, but here’s the basics and how we can apply it.
Let’s go back to the example of flipping a coin at the target. Imagine making an initial run of 10 flips, and you get 2 within a few inches. Does this mean you’re terrible? Not necessarily. Make another run of 10. This time, let’s say you got 8. A quick learner? Not necessarily. Make another run of 10. This time, 5. Another time, 6. Another time, 4. Another, 5 again. What’s going on?
Part of the result is skill, and part is chance . Factors in your location—maybe air flow, maybe other people, maybe distractions, who knows—are affecting your shots. The more shots you make, however, the more these small chances are being ‘averaged’ out. The ‘mean’ or average of all these runs is indicative of the portion due to skill (which, from that limited data, looks about 4.8).
This is important in many life skills—church planting among them—because we often give something a few tries and decide we’re not very good at it and give up. The problem is, it’s very possible chance factors were against us on those tries.
To find out just how good (or bad) you are requires multiple tries. Those ‘samples’ will also give you data on weaknesses, which can lead to specific suggestions for self-improvement.
This is also an important principle when considering an evangelistic method. You might try something—for example, tract distribution—and hear reports of a dozen who became believers. You might be quite excited about this (and well you should; the angels rejoice over one sinner).
However, we might ask a couple of questions: twelve out of how many tracts distributed? how does that ratio compare to other evangelistic methods? Again, multiple samples are required, and to best test the method you’d need to have the same workers try both.
All of this may seem like a terrible exercise in smothering the joy of Gospel-sharing with quantification, numbers, measurement and spreadsheets—but evangelism isn’t just a joy, it’s a responsibility. By measuring multiple ‘sample sets’ of Gospel-sharing, we can get a better understanding of what is working in the place, and what isn’t.
How might we set about doing some of this?
1. Bigger sample sizes. The typical set really should have 30 to 50 samples in it to be a good sample. Consider adopting a way of sharing the Gospel (or perhaps just a ‘living out loud’ statement) with 5 to 10 people each day for a week. As you get a chance, record the response to each sharing: was it seeking (+2), friendly (+1), neutral/uninterested (0), disinterested (-1), hostile (-2)? Actually recording this (you don’t need their names and such; “guy at bus stop” might be enough) will help you be accountable (did I hit 5 today?) and see the real pattern (that one lady was hostile, but most people were either neutral or friendly).
2. Multiple samples. Do this over 3 to 4 weeks and you’ll have a fairly good view of the mean or average of the number of times a Gospel-offering finds an interested listener. You’ll also see the vagaries of ‘chance’ (I use the term loosely)—you might see many friendlies one day, and many unfriendly people another, but what’s the average situation?
3. Now, begin to change. Remember the skill vs. chance mix. Is there something about your skill set that you can improve, something about the method of presentation that you can change, which might result in an increase in positive responses? You might imagine this change or another, but the only real answer is to try something and measure it. Again, measure for a week or more. Get a good sample to control for chance .
Is this too much? In many places we’re looking at enormous populations who need to hear the Gospel. Can we afford to waste time on methods that are very ineffective? Can we afford to lose people who think they’re bad witnesses just because they had a bad first week? Can we afford to avoid evaluating our methods just because they’re exciting and we’re seeing a little fruit? Let’s measure—and let’s learn to measure well.