Challenges of measuring well-being

Photo by FreelySky

Measuring well-being is really hard. That's probably the first thing to underline, and for a long time we thought it was basically impossible; we especially thought that it was impossible to measure subjective well-being — that's how people think about how their own lives are going. In the last 100 years or so, there have been a lot of breakthroughs. Still, there are many, many things to break through; we still have a lot of obstacles and challenges to face, but we're starting to have a more sophisticated understanding of how we can measure both objective and subjective well-being. This has inspired a big push to take these measures into consideration in public policy in particular, and to make them more publicly available so people can think about their own lives in terms of some of these issues.

Gaps in using GDP as a measurement

Let’s cover a few of these measures to start off with. One of the first places to start is with objective measures of well-being. Often we think, we focus too much on GDP and we should focus on well-being; but GDP is, of course, one way of measuring well-being. Income is very important to well-being. If you don't have enough money to pay for your basic needs and also to pay for kind of fairly basic luxuries like an occasional holiday or entertainment and these kinds of things, then it's going to be a struggle to be well, so we should take income into consideration.

With GDP, of course, you always see that the figures are published – ‘this country's GDP has grown by so and so much’ – but actually, the way the sausage is made is quite concerning. A lot of these measures that we take for granted have their own problems just as much as subjective measures might. With income, we often ask people in social surveys, how much money did you make this year? And they can't quite remember. They don't remember how many benefits payments they got. They don't remember how much they spent in tax. It's hard to get a very clean estimate.

Similarly, when we're calculating the income of the nation, that nation's gross domestic product, there are a lot of adjustments we need to make. We need to consider inflation; we need to consider things that are very hard to monetise, like the digital economy; and we also, if we're making comparisons between countries, need to make adjustments for the cost of living in those countries, exchange rates and these kinds of things. At each stage of these adjustments, we lose some degree of precision. So just as a kind of preamble to when we think about measuring well-being, particularly in a public policy context, it's worth acknowledging that a lot of the existing measures of things that we think of as very objective and very precisely measured are actually fraught with their own problems.

Life satisfaction scales

If we turn to measuring psychological phenomena that we think might be important to people's well-being, the most widespread and I think most integrated into public policy measure is life satisfaction scales. These are very simple instruments where we just ask people: all things considered, how satisfied are you with your life on a scale from 0 to 10, and there's now wide-ranging international surveys that have been done on an annual basis that look at this life satisfaction. There's also, within countries, longitudinal studies where we follow the same people over time and measure their life satisfaction year-on-year to see how it changes. This research program has thrown up a lot of really interesting findings. For example, it seems that middle age is actually where we're least satisfied with life and that as we get older, we become more satisfied with life. It seems that income has a very strong effect on life satisfaction up to the point where your basic needs and maybe a bit beyond that are met. After that, however, while it definitely has an effect, that effect really starts to become smaller and smaller as you become richer. So then there's a question of: maybe we should stop worrying about income quite so much once we're wealthy and focus on other factors. Then we start to think about richer notions or more complex notions of psychological well-being.

Measuring the unmeasurable

Photo by Andrey_Popov

So how can we measure things like whether people have good relationships with the people in their community? How can we measure whether they think they're a good person? How can we measure whether they have meaning and purpose in their life and this kind of stuff? That becomes a little bit more difficult, but we've also developed tools for that. There's a very well-established survey now for measuring meaning and purpose in people's lives, measuring whether they have the ability to achieve their goals, whether they can apply their strengths every day. These kinds of surveys typically take this Likert scale form: “To what extent do you agree with the following statement: ‘I get to use my strengths every day’.”? You can say one, which means that you don't agree with it, or five, which means you strongly agree with this statement. When we give people a set of 48 of these kinds of questions, we start to get quite a complex sense for their psychological well-being.

The challenge for public policy with these kinds of surveys is twofold. One is, a 48-item questionnaire? That's a lot of questions. It's very expensive to administer a survey like that, especially if you want to get a representative sample of your country's population. So even somewhere like Australia, which has a relatively small population for an OECD nation, we want to get maybe 10,000 to 14,000 people.

Problems with long lists of questions

We want to give them these 48 questions, but we also want to ask them about their access to green spaces. We want to ask them about their income, their health, what their kids are doing, what tax benefits they're receiving; over time these surveys start to take four or five hours to complete, and then no one wants to do it, and it's very hard to get it going. So, we have this tension between having enough questions to get a complete and a rich understanding of people's well-being, but also having short enough surveys that people actually want to do them and we can administer them to a lot of people.

The second tension is, how can we make interpersonal or intercountry comparisons when we have these very large lists of questions? In the past, the 20th century, a lot of our comparisons were just GDP. In the Cold War, it was: is Russia richer or is Britain richer? If Britain or the US is growing faster and making more money, then the free market democratic system of capitalism must be better than socialism and communism in Russia, and that was a fairly simple way of thinking about it. You can make that comparison because you just have the one index, just GDP.

The weighting problem

You have many different things that you care about in public policy, and you want to be able to create one index, one number that summarises all these different things that you're interested in, and that allows you to compare one policy against another, one area against another, one country against another. We might want to assess, should we be spending money on making child care more accessible or should we be spending money on providing roads?

In making that comparison, if we have 50 items that we need to consider, some items would be better with the roads and some items will be better with the child care; how do we decide which items we should prioritise? This is the weighting problem and, ideally, we'd want to get all of these 50 items into one measure that we can compare across. The reason why income is so nice, and why we've historically used income for these kinds of cost-benefit analyses, these kinds of interpersonal inter-group comparisons, is because income is basically worth the same amount to everybody.

If I save $10, then I can go and spend that $10 on whatever it is that I care about. Someone else, that $10 is worth the same amount to them, even though they might spend it on something completely different. Thus, the nice thing about income is that, in a sense, it includes people's own weighting in how they spend their money. If we look at the impact of policy changes, or the impact of infrastructure building, for example, on how people spend their money, then we can get a sense for the overall impact on well-being, on what people value from these policies.

What can’t be measured by income

The problem with income, of course, is that there are a lot of things that people value that are very hard to measure with income: it's very hard to measure your engagement with your community; it’s very hard to measure whether you think that you're living a virtuous life; it's very hard to measure whether you think the state of the nation is good, whether the country is moving in the right direction, this sort of stuff. So again, we have this tension now. We want to think about more things that can't be measured with income, but we have to try to get them into an index or we have to think about how we're going to weight these problems. So how do we do that? One solution is to not stress so much about cost-benefit analysis.

Maybe we just don't need to use this mechanism for a lot of our policy-making decisions. One option would be to have more direct democracy or more community engagement in decision-making and try to let the citizens decide in a political process what we should prioritise.

Another option that's really popular and that has been really spearheaded by the OECD in particular, is to just collect a lot of data on all these different aspects of well-being, create a website or some other platform that people can access easily, and then let people decide the weights.

The Better Life Index

Photo by Gil C

There's a really cool tool called the Better Life Index that the OECD has where they have a large set of variables measured at a different geographical scale: by city, by country, by region and so on. You can go on and adjust the weights and say, what I really care about is green spaces, or what I really care about is the availability of entertainment and art galleries and this kind of stuff. Based on what you weight and the scores that each of these regions or countries have on these particular items, the platform will tell you the ranking of these different places.

Through that, you could potentially make migration choices. You might think, what I really care about is sunny weather. Even though a lot of other things seem really great in Norway, it's really cold and dark there for most of the year, so I'm not going to move to Norway. I'm instead going to move to Australia, New Zealand, Thailand or one of these sunnier places. New Zealand actually has a lot of rain, so I probably wouldn't suggest that.

No good answer

This is one way of doing it. We can have these indexes where citizens put their own values on. The problem with that is that again, in the policy-making space, a lot of statisticians don't have time to survey the citizens about what they think. Sometimes we should get the citizens involved, but sometimes the citizens want the statistical agency to just produce a report that says, if we build a light rail, it's going to create this much money; and if we build a bus network, it's going to create this much money. Which one do you prefer? Then that cost-benefit analysis is an input into the public debate and the public deliberation about that policy, and that seems reasonable.

Then again, we're back in this problem of, how does that statistical agency think about well-being? How do they weight these different considerations? And I'm afraid at this point, there's just no really good answer to that. But there's a lot of research going on trying to figure out what seems to have the biggest impact on people's well-being and trying to develop a parsimonious set of indicators that really capture a lot of the variation in people's well-being, what they value and that sort of thing, and with that parsimonious set of indicators, maybe the weighting problem won't be quite so difficult.

Discover more about

Measuring well-being

Fabian, M. (2019). Scale norming undermines the use of life satisfaction scale data for welfare analysis. SocArXiv.

Alexandrova, A. (2017). A Philosophy for the Science of Well-Being. Oxford University Press.

How to take account of wellbeing in our policy-making decisions