Methodology for calculating inflation in Russia - the nuances of the Rosstat methodology
Rosstat was repeatedly criticized for the quality of the data provided, and at the end of 2018, its long-term leader Alexander Surinov was dismissed. The opinions of experts were divided - it is not clear whether the former head of Rosstat prepared statistics too bad or too good. One of the traditional stumbling blocks is the inflation rate, which has reached record low levels in recent years. The authorities set long-term goals for this indicator, and the population looks at price tags in stores and never stop to be surprised such statistics. In this article, we examine whether the claims are justified, where exactly Rosstat is mistaken in calculating inflation and how significant this error is.
What is inflation and how is it considered?
But at the beginning there is a bit of theory. Inflation is the average price growth rate in the economy. Inflation is usually measured by means of consumer price indices, which show the change in the value of a certain typical set of goods in the economy. To clarify the general logic, we give the following (simplified) formula:
Iij = (P1i * w1 + P2i * w2 + ... Pxi * wx) / (P1j * w1 + P2j * w2 + ... + Pxj * wx)
In this formula, I is the total inflation rate. The numerator shows the prices of goods of the current year (year i), the denominator shows the prices of the same goods in the previous (base) year (year j). As weights use the share of the cost of this product in the total consumption. The weights for each product are the same in both the numerator and the denominator. Although the real structure of people's consumption is changing, such a prerequisite is needed to ensure data comparability - since the task is to understand how prices have changed, not consumer preferences of the population. Weights are almost always taken on the basis of consumption data in the base year - this approach is called the Laspeyres index , which is used by almost all statistical services in the world to measure consumer inflation.
At first glance, the mathematical apparatus for calculations is quite simple and straightforward. However, there is a lot of the nuances. For example, what was the price of eggs this year, and what was the previous? If we go to the store, we will see many varieties of eggs, many brands, even for the same brand, the price will vary in different places. And in general, what is considered a commodity - in principle, eggs or eggs of grade S-1? Or another question - on what list of goods does the index need to be calculated? Which products to consider when calculating the inflation index, and which ones not? We probably take the eggs for sure - is it necessary to include in the calculation a dog’s paste, which citizen Ivanov regularly buys for his pet? And even if we take, with what weight should I take it into account? Suddenly, egg prices will rise sharply, and they will drop on paste — doesn’t this mean that the country has zero inflation?
There are a lot of questions like what, where and what to count, so the methodology for calculating inflation is usually a multi-page document.
Rosstat considers price indices (the same P in the formula) in 271 settlements:
- calculations are carried out in at least two cities of each subject of the federation (but not more than in four)
- prices are considered by commodity items (e.g. tea, coffee, eggs, etc.)
- in each city for each commodity position Rosstat records no less than 5 prices, after which the average is calculated
- further, for each commodity item, the average price for the subject of the federation is considered - in proportion to the population in those cities where measurements were taken.
- averaging the prices of commodity items by federal subjects for the calculation of all-Russian price indices is proportional to the consumer spending of the federal subject on a specific product
Surveillance is carried out in trade and service organizations, as well as in clothing, mixed and food markets, both in stationary trading establishments and in objects of mobile trading (tents, kiosks, etc.). To put it simply, Rosstat employees go to multi-format trade institutions and photograph price tags according to the approved list of commodity items.
Weighting coefficients in the Laspeyres formula are defined as the share of expenses for a specific commodity item in the total expenses of the population. The cost structure is determined on the basis of a sample survey of households and, in essence, represents a consumer basket of an average family. The calculation of the consumer price index includes those commodity items whose share exceeds 0.1% in the total expenses of all respondents.
But with the seemingly logical approach, the result is an indicator, the value of which many continue to distrust. Let us see why this happens and what is wrong in the methodology of Rosstat.
Romir index steadily exceeds the inflation rate published by Rosstat
Price data, unlike, for example, industrial or agricultural production, do not constitute any secret knowledge. If Rosstat collects information on production volumes mainly from a small number of large enterprises through specialized reporting forms, then anyone can find information on prices in the public domain. Therefore, as it might seem, Rosstat can’t cheat significantly here - any researcher, if desired, can calculate the consumer price index independently, using his own methodology or using the methodology of the Rosstat. Something similar is regularly undertaken by the Romir international research bureau, which also publishes the values of its deflator indicator on a monthly basis.
Romir has a large own panel of respondents - these are people who agreed to install special software on their phone that allows fixing purchases made by a person in the context of commodity items. The company then sells this data to marketers of large companies, but also uses it for collateral purposes - to calculate and publish its alternative index deflator.
Surprisingly, inflation, according to Romir’s data, is consistently higher than the figures published by the official authorities:
Romir inflation is usually higher
An important note - the indicator that Romir calculates is called the deflator. This means that the weight of the goods in the formula given at the beginning of the article is calculated based on the consumption structure in the current year (data on purchases of respondent participants made today). Rosstat, unlike an analytical agency, takes as a basis a basket of past periods. Roughly speaking, Romir’s analysts make the assumption that people in the past period consumed exactly the same as in this one. Rosstat, on the contrary, believes that people this year consumed the same as in the past. This might seem like an insignificant detail if the substitution effect did not exist.
Imagine that it is customary in your family to buy fruit regularly. Naturally, the proportion in which you consume apples and pears varies from year to year - fruits in general are important to you as a source of vitamins, and to a lesser extent their specific types. Now suppose, for any reason, the price of pears increased by 30%, and the price of apples - only 5%. You are likely to redistribute your consumption - increase the proportion of apples in your consumption. If we calculate inflation by the classical consumer price index (which Rosstat uses), then we will take the basket of last year as a basis - with a high share of pears. As a result, their contribution to the calculation of the inflation index will be quite high. But you have adapted (most likely even intuitively), redistributed your consumption - therefore, your personal inflation will be lower, and without loss of quality of life - as these products to one degree or another replace each other. As a result, Rosstat inflation should be on average higher than inflation determined using the deflator coefficient - the latter, on the contrary, underestimates inflation, assuming that you bought a lot of apples last year, so the pear price increase almost hit you.
But in Russia there is an amazing statistical paradox - contrary to theory, the coefficient-deflator stably exceeds the consumer price index.
Of course, we know even worse how Romir considers his index - the research company does not disclose its algorithms, unlike Rosstat, on whose website a multi-page methodology is available. We must either believe them or disbelieve, decomposing the calculation into atoms will not possible. But, on the other hand, Romir takes into account actually made purchases, while the statistical service only takes into account price tags in stores and population polls about the structure of expenses. By the way, the structure of expenses is further specified by Rosstat with the help of special adjustments - for example, it is believed that people during their surveys underestimate their expenses on alcohol, and Rosstat manually corrects this figure, which also leads to an additional measurement error. We will understand further.
Valuation is based on the old consumer basket
Starting in 2013, Rosstat changed the methodology for calculating the consumer price index. Now the weights in the Laspeyres formula are calculated on the basis of consumption for two displaced years (for 2013 - the period from the 4th quarter of 2010 to the 3rd quarter of 2012). According to the old method, only one offset year was to be used - that is, for 2013, weights would be calculated according to consumption data for the period from the 4th quarter of 2011 to the 3rd quarter of 2012. On the one hand, it would seem that the longer the calculation period, the more accurately weights are obtained, but on the other ...
The farther away from the current period the base one is, the more ignored are the effects of substitution in consumption, the higher is inflation.
In other words, contrary to our expectations, Rosstat is manipulating inflation towards overstatement. According to V.A. Bessonov assessment , this effect is 0.2%.
Rosstat formula does not take into account rental prices
Unlike most countries, the Russian inflation index does not take into account housing prices. For a number of theoretical reasons, the price of buying a house does not include inflation in the calculation of inflation in the world, but instead of them, most countries take into account rental prices to measure inflation. Even if the family lives in their own house or apartment, they are “charged” rent when calculating, as if they were renting a house from themselves. This is generally logical - even if specifically now the family does not pay anything for housing, it was paid earlier (upon purchase). In addition, housing prices are an essential component of the overall well-being of the population. Such differences in methods strongly affect the calculation - in the West, the share of housing costs in the Laspeyres formula can reach up to 40% percent, which means that the inflation index is almost half determined.
Organization for Economic Co-operation and Development pointed out Rosstat to this problem. According to the Federal Statistical Service, in Russia it is extremely difficult to monitor housing rental prices. The liquid rental market exists only in a small number of large cities, and in small cities, rental housing as an economic phenomenon is almost invisible - people live in their own housing almost everywhere. In such circumstances, it is difficult to collect high-quality statistics on prices (the sample is too small) - which means that inflation data will not be reliable either. And although the arguments of Rosstat are logic enough, it is important to understand that housing prices usually do not change in the same way as prices for provincial goods.
During a crisis, house prices usually rise at a slower pace than other goods; during a period of economic growth - on the contrary, housing prices are rising faster than other goods
Indeed, after 2014, when Russia entered a phase of a sluggish crisis, housing rental rates began to fall even in Moscow. In the crisis phase, people choose more budget options, rent less rooms, more often decide to rent a house together. Demand is declining, and housing supply remains stable, which leads to a slowdown in price increases or even their decline. It is much more difficult for people to reduce the demand for basic foodstuffs and clothes; here prices usually rise at a slightly higher rate. Thus, in the crisis period (that is, now), the calculation of inflation without taking into account the cost of rental housing leads to a significant overestimation of numbers. In other words, we again come to the conclusion that the methodology of Rosstat overestimates, but does not underestimate, inflation.
This leads to an amazing conclusion - Rosstat could show even lower inflation if the corrections described above were in its methodology. But still, let's not rush to conclusions.
We don’t know how Rosstat considers grassroots prices
Despite the existence of a very detailed methodology, we have practically no opportunity to check how prices are considered at the primary (grassroots) level. For example, it is obvious that the price range can be two or more times for the headings “domestic automobile” and “lunch in a restaurant”. Prices for juice-containing drinks, nectars and juices can differ significantly depending on the percentage of natural juice they contain - Rosstat will take into account any of these drinks in one product category. Finally, a situation is typical for many products when the price per unit of product decreases as the volume of containers increases. For example, if you buy a small glass bottle of Coca-Cola, then in terms of liter you will pay much more than if you buy a large two-liter bottle. Rosstat considers the price in units of useful substance - depending on how much packaging was used in the calculation, the result can be very different.
As a rule, Rosstat considers a larger number of quotes for commodity positions with a wide range of prices.
In other words, the average juice price in the current period and in the previous one can be very different if the averaging at the grassroots level was carried out differently in the current and base periods. For example, in one period, a larger number of objects with a low price were included in the calculation, in another, a larger number of expensive objects. Considering that it is possible to make a mistake here in any direction, on a large sample these errors should level each other and not have a significant effect on the overall effect. Provided that at the time of recording the statistics there was no task to deliberately make a mistake in one direction or another - since it is impossible to verify this, fertile ground appears for the emergence of various conspiracy theories.
But in some situations, systematic errors may also occur. The most obvious case is the systematic inclusion in the sample of more domestic goods to the detriment of foreign ones. Since the ruble has tended to weaken in recent years, prices for imported products usually rise faster than for domestic ones. If Rosstat observers choose domestic goods for calculations, this will lead to an underestimation of inflation. For example, now it is not completely clear how Rosstat considers inflation for medicine drugs - here the effect of a possible substitution of domestic drugs can lead to an underestimation of the indicator.
Is Rosstat really purposefully underestimating prices in this way? Here we quote the site of the already mentioned Romir analytical agency: "The official inflation index provided by Rosstat is not only a statistical indicator, but also an economic and political indicator. Thus, official inflation rates can be considered objective with some limitations. " Inflation is one of the few statistical indicators that is really understandable and interesting to ordinary people. All other indices (GDP, industrial production, etc.) are interesting only for various researchers and statistical geeks. Since 2014, the Central Bank switched to inflation targeting - just the consumer price index is used as an indicator for targeting. The Ministry of Economic Development regularly publishes its inflation forecast. In other words, many people, both in the Government and the Central Bank, is directly affected by Rosstat statistics.
Keep this in mind, the decision to make Rosstat accountable the Ministry of Economic Development again is very confusing
Rosstat in general strongly resembles a tennis ball - in recent years it has either been transferred to the Ministry of Economic Development, then returned to direct subordination to the Government. This time, the decision to return to the Ministry of Economy was explained by the desire to increase the quality of statistics - supposedly, at the Government level, statistics were not a priority topic, and it was quite difficult to promote various initiatives to reform the service. But in this decision, something else is embarrassing - the Ministry of Economic Development is responsible for improving those economic indicators that Rosstat is measuring. When receiving the statistical service in its direct subordination, the ministry will acquire just the tool with which these indicators can be “improved" most easily. This means that trust in the statistical service and the data provided to it will become even less. It was not so much before - the Government, as a rule, is also interested in favorable macroeconomic statistics. This is about the same as entrusting the student himself marking his diploma work. It is useful to draw parallels with stock markets - a company that wants to place its shares on the stock exchange is required to publish its financial statements, certified by an independent auditor. This provides a certain level of confidence among market participants in the financial information presented. Here, in essence, information is prepared by someone who directly depends on people whose work is evaluated based on this information.
Despite all of the above, we are still not inclined to think that the reason for low inflation is in the targeted actions of Rosstat.
Rosstat, when calculating the price index, does not make adjustments for changes in product quality.
In essence, Rosstat considers what is sold in domestic stores. Therefore, if structural assortments occur in their assortment, then the Rosstat price index simply takes this into account. If we talk about the period starting in 2014, then, perhaps, there are three main trends:
- The victory of big retail chains (Pyaterochka, Magnit) over other forms of trading
- Substitution of import products by domestic
- Decreased quality of products sold
Each of these causes is global, the phenomena are interconnected, but it is necessary to explain how each of them affects the level of inflation.
Whereas at the end of the 2000 years and the beginning of the 2010 years, consumers had to go far for cheap prices - to large hypermarkets of international chains (Auchan, Metro, etc.), then starting from the middle of the 2010 year, small stores of big retail chains (Pyaterochka, Magnit) began to offer the same prices. As a result, retail sales were redistributed towards these stores. Network giants have reached unprecedented power - their bargaining power allows them to knock out significant discounts from food manufacturers. As a result, prices in chains grow on average more slowly than in other retail chains. The methodology of Rosstat as a whole should correctly process this effect - stores for calculation are selected in proportion to their share in total sales, which means that their "role" in the index should gradually increase and this is absolutely normal. By the way, the Pyaterochka index, which the chain of the same name considers for its commodity positions, does not show significant contradictions with the Rosstat calculations.
The same thing with the notorious import substitution. If buyers began to buy more domestic products, then their share in the index should increase proportionally. Consequently, the methodology correctly captures this effect, and does not overestimate inflation, even despite the fact that prices for foreign products grow on average faster — their share in the index is gradually decreasing, which levels the effect.
But with the decline in product quality, everything is not so clear. Products in chain must meet certain requirements - primarily in terms of price and shelf life. To achieve this, manufacturers are forced to use more preservatives and reduce the amount of expensive ingredients. We all know that almost all the confectionery products in Pyaterochka cannot do without the use of the notorious palm oil. Part of milk products is not even required to be stored in refrigerators during the sale. Famous “rubber” tomatoes are also a kind of “quality mark”, which are gradually replacing rapidly deteriorating and expensive tomatoes from network shelves. It is even easier to cheat sausage producers - the percentage and type of meat ingredients can be varied almost imperceptibly. Such examples can be cited quite a lot - in Russian retail today, as once in the Soviet years, starch and palm oil prevail.
In other words, when we measure the price in one year and the next, we do not find a significant difference - it is not the price that inflates, but the product quality.
International methods prescribe hedonic corrections - take into account changes in quality when measuring price differences. Similar corrections are made by , for example, the US Bureau of Statistics. The Organization for Economic Cooperation and Development-one of the main trendsetters in the field of international price statistics - adviced to Rosstat to make such adjustments . Rosstat promised to include this in its long-term plan. According to the Rosstat, the adjustment for quality is a too labor-intensive procedure, the implementation of such adjustments will lead to a significant increase in the costs of the statistical agency and is unlikely to be agreed by the Government.
It is important to add here that quality adjustments should not be confused with the substitution effect described above. It refers to the fact that buyers voluntarily choose one product instead of another under conditions when prices change with different dynamics - it is assumed that the goods are equivalent for the consumer (we assume that the consumer does not care what is pears or apples) . Here we are talking about the fact that, firstly, the goods are not full substitutes for each other (palm chocolate instead of chocolate), and secondly, the basket of the base year is used to calculate inflation - therefore, you cannot be sure that the consumer agreed to a substitution - perhaps he just switched to a good chocolate bar from another manufacturer, now he simply has to pay more for it.
Total - in the context of a decline in the quality of goods, the consumer price index underestimates inflation, in contrast to the deflator coefficient, which repels the current year’s consumption structure. This may explain why the Romir index is systematically lower than inflation, according to Rosstat. And the older the base year basket, the stronger this effect.
Rosstat does not consider rural price statistics
Perhaps, it is worth mentioning another methodological omission - Rosstat conducts its observation exclusively in urban areas. The rural population (26% of the total Russian population) is not taken into account in the calculations, which means that changes in prices in towns and villages are not covered by official statistics. The reason is the same - according to the Federal State Statistics Service, it is too expensive, and the products are still being brought to the village from the cities, which means that prices there will change at about the same rate. To this problem, Rosstat also pointed out by the OECD - in many countries of the world prices in rural areas are covered by statistics.
In the absence of any information, we cannot reliably state how much inflation in rural areas is higher than in cities, and how significantly this affects the indicator. But we can assume that the price of goods in small shops also includes fuel, the prices of which are also rising, which means that a peculiar double inflation arises for the villagers. Also, it should be borne in mind that the population in villages in Russia is gradually decreasing - as a result, the number of trading establishments is also decreasing. In these conditions, the economies of scale should work the other way around - the fixed costs at the stores are the same, and the number of customers is getting smaller and smaller. As a result, these costs are laid down in prices - to this is added the increasing effect of monopolization of the market. Therefore, we can assume that rural inflation is higher on average - which means that this can lead to an underestimation of official inflation.
We pay attention only to price increases
Despite the problems described above, it’s not only Rosstat who is to blame. We, as people, are so arranged that we remember first of all the price increase and practically do not pay attention when the price does not rise or fall. Statistical offices are criticized in all countries of the world - even where the methodology is almost perfect and does not contain the flaws described above. Indeed, in recent years, prices for some socially important goods have grown at a faster rate than the average price level. A classic illustration is the price of gasoline, the dynamics of which are shown in the graph below:
Dynamics of prices for gasoline 92 in Russia according to Rosstat
Back in 2009, a liter of 92nd gasoline cost less than 18 rubles - since then the price has more than doubled. Particularly noteworthy is the fact that in some years prices rose especially rapidly - in 2010 and 2018 this happened against the backdrop of a jump in oil prices. As a rule, in such situations, the price tag changes dramatically in a matter of months - and this is better remembered by people than creeping changes. Similarly, over the past years, prices for housing and living services have risen - they were affected by the introduction in 2014 of an additional item of expenditure - the cost of overhaul. In 2014, household appliances also rose sharply in price against the backdrop of the devaluation of the ruble. Finally, transportation also usually rises discretely - and people remember this well. Also, many of us sometimes take seasonal fluctuations in the prices of fruits and vegetables for price increases.
People also do not always understand the differences between statistical indicators. Rosstat conducts weekly monitoring of prices for the largest food products, in addition to the usual consumer price index discussed above, it calculates the so-called basic one (for a limited set of goods, goods that are subject to seasonal fluctuations - vegetables, gasoline are excluded from the calculation). Also, indices can be considered in annual terms, compared with the comparable month last year, relative to December last year, etc. There are also producer price indices, a construction price index - of course, for the average citizen all this merges into a single index.
Do not forget that Rosstat inflation should not be equal to your personal inflation by default
As a rule, the higher your income, the less you spend money on food and the more - on non-food goods and services. Poor and rich Russians have a different consumer basket - Rosstat in its calculations is based on a certain average set of consumer goods. If prices rise mainly for essential goods, and prices for services stagnate (as can often be in a crisis), the poor will feel higher inflation than the rich, since the share of these goods in their total consumption is higher. And they will think that the statistical agency deliberately deceives them. In fact, their personal inflation is simply greater than the average inflation in the country.
It is clear who is blamed. So what to do?
In general, in the era of the Internet and computer technology, the methods of Rosstat look, to put it mildly, slightly outdated. Yes, the calculation methodology itself is generally close to international recommendations, scientifically sounds, but the quality of the source data continues to raise a large number of questions. Shopping trips, questioning of sales staff, photographing of price tags - the administrative methods of statistical monitoring has not changed much since the beginning of the 1990s. As a result, the same Romir, simply by installing the software on the respondents' mobile devices, can use no less transactions for calculation than a government agency with territorial offices throughout the country.
The obvious way seems to use the data of the tax inspectorate - after installing online cash desks throughout the country, the Federal Tax Service of Russia collects simply huge amounts of information.
In general, there is already talk of this. True, there are several problems. The first and, as it seems, so far the main one is the complexity of interagency cooperation. The tax is not ready to share "their" data with another agency. Another important problem - the tax far from always collects statistics on sales of goods in the context of commodity items. Very often it’s just information about the amount of the broken check, and the introduction of cash systems that can beat checks by type of product will greatly increase the overhead of small businesses. Yes, information can be taken on a selective basis from cash registers of large networks - but in this case a deliberately unrepresentative sample is obtained (this is simply state analogue of the Pyaterochka Index).
Another area of work may be the gradual introduction of quality amendments into the Rosstat methodology. Unfortunately, the Federal State Statistics Service is an extremely non-media body, which is at the forefront of media attention. Rosstat itself, however, has never been seen in attempts to change this tendency. All previous changes in the methodology of price statistics (for example, a change in the period for calculating weight coefficients, even earlier there was a transition from arithmetic average to geometric average when calculating price indices at the grassroots level) - they were practically not discussed even in the expert community, they were not announced publicly. It’s just that at a certain point, Rosstat posted new Methodology on its website. Finally, Rosstat calmly swallows reproaches against the quality of the information provided. And these reproaches were not only pouring from the "expert" community or anonymous bloggers - both the Minister of Economic Development and the Minister of Finance spoke about their claims openly. Rosstat is not trying to explain, for example, in the business press, what and how he considers and why it receives such results (and what will happen if it would be calculated differently). They simply say, that the methodology is scientifically based, consistent with the practice of the UN and other countries, and those who do not agree - read for yourself and understand. But for some reason no one runs to read and understand himself, but prefer to believe alternative metrics - which in themselves are also not without flaws.
Therefore, the problem of Rosstat is sometimes not even how it calculates, but how he presents what he counts.
Why do some countries, cities or companies achieve economic success, while others are in poverty? Why is economic growth so weak and income inequality constantly increasing? How can new technologies change the global economic landscape? On this site we look at the most important global trends from the perspective of an economist.
