The case for accessible audit data: tracking the effectiveness of Indian governance schemes

Governance is a major problem affecting outcomes as diverse as economic growth, the environment, and security, all which impact people’s wellbeing. Big scams make headline news but it is rare for media outlets to follow up on investigations years down the line. In order to design anti-corruption policies, we first need to understand the determinants of corruption; we argue that audit data can offer a rich source of data. The data should be made available in a way that does not violate any privacy norms, however even sensitive data may become accessible once it has been archived. 

In Brazil, a country comparable to India on the development index, the federal government transfers funds to municipalities for local development. The federal government however has limited oversight on the use of those funds. In 2003, the Brazilian federal government introduced a groundbreaking anti-corruption initiative implemented by an independent federal agency called the Controladoria-Geral del Uniao (CGU). The agency is the SAI in Brazil, and was instituted as late as 2003. Their main remit is to conduct a (public) random lottery which selects municipalities for relatively short but intense performance audits. The results of the audits are also publicly available on their website. This data has been the subject of a large number of research studies¹, and the findings of these studies show (causally) that past audits have a deterrent effect, the deterrent being stronger when there is a clear link to judicial punishment. They are also able to show the kinds of activities that respond most to the threat of an audit. Most importantly these reports are released publicly. A review of the literature on audits is available in our working paper (Afridi, Dhillon, Roy Chaudhuri, 2020).

The Comptroller and Auditor General (CAG) of India is a constitutional authority which performs the role of the “Supreme Audit Institution of India.” Among its many functions as the preeminent audit authority of India, the CAG periodically carries out audits of different federal government schemes. The CAG carried out an all-India audit of PMGSY (a large rural road building programme) in 2015-16 and of MNREGS (a large rural employment guarantee programme) in 2012-13. The findings of these exercises were brought out in the form of reports². 

For example in MNREGS, an earlier performance audit covering the period from 2007-2012 showed how prescribed records were not kept by up to 54% of the audited GPs and the uploading of data on MIS (Management Information System) by states was often incomplete and inconsistent with records on the ground. Additionally, “job cards were not issued to 12,455 households in six states, photographs on 4.33 lakhs job cards were not found to be pasted in seven states. Non-payment or underpayment of wages of Rs 36.97 crore was noticed in 14 states. There were several cases of delayed payment of wages for which no compensation was paid…” Works with a total expenditure of Rs 4070.76 crore remained incomplete despite already substantial delays. (CAG Audit Report 2013).

Similarly, in the case of PMGSY, the performance audit for the period between 2010-15 showed discrepancies such as:

• unconnected roads were shown to be connected 
• some eligible habitations were not included
• irregularities in the award of contracts to ineligible contractors or without a proper tendering process
• delays
• missing roads
• inspections were not carried out 

In many cases, these discrepancies led to substantial losses to the state exchequer.  The CAG carries out performance audits on these centrally sponsored schemes at the behest of the ministry of rural development. 

We argue that there can be a substantial benefit to the Ministry of Rural Development and the Government of India if the audit reports are presented in a form that can be used by scholars and the audit office itself to analyse patterns of discrepancies and how such discrepancies  may be prevented.  We list below some of the key issues that we faced while trying to extract data from the CAG reports.

• Save for the aforementioned reports, the CAG does not release its findings in more accessible forms, e.g. excel spreadsheets or as a csv file. 
• Most of the data is presented only at a highly aggregated level – such as the state – making any serious empirical exercise using the CAG audit findings very difficult. 
• Although the reports describe the process through which units are chosen to be audited, some of the details are not clear. This makes it difficult to use program rules on the choice of units to audit, to aid in econometric identification (i.e. to be able to establish causality rather than just correlations). For example:

1. The PMGSY audit states that each state was divided into a number of geographically contiguous regions and then 25% of the districts from each region (subject to minimum of two) were selected using Probability Proportional to Size Without Replacement (PPSWOR) method on the basis of size of expenditure under PMGSY during the last five years. However, the report does not identify the exact composition of the regions of the states in the first stage. Neither is the report clear about which exact expenditure variable is being used during the selection of districts by PPSWROR. This is confusing since the official PMGSY outcomes data lists several expenditure variables.
2. The description of selection of audit units for the MNREGS audit also suffers from similar lack of clarity. What is the strata that the states were initially divided into?
3. In the MNREGS audit, gram panchayats³ were selected from blocks based on PPSWOR. However, the variable on the basis of which PPSWOR has been carried out is not mentioned. 
4. Even when there is useful data in the report, it is not often presented in a useful manner. For example in the MNREGS audit, each state was divided into strata,
5. districts were chosen from each strata, then blocks from each selected district, gram panchayats from each selected block and then works and beneficiaries from each selected gram panchayat. The report lists out the districts, blocks and gram panchayats chosen for each state. However, the way they are listed, it makes it difficult to decipher which gram panchayats were chosen from which blocks and similarly which blocks were chosen from which districts. Given gram panchayats belonging to different blocks often have the same names, it makes it impossible to arrive at a consistent matching of the selected district/block/gram panchayat (GP).

We would recommend that the CAG presents the audits in a more accessible form and at a more granular level, taking into account the trade off between privacy and transparency.

• For the PMGSY it means that the data is presented at the package (group of works put to tender in one lot) level 
• For MNREGS it means that the data is presented at the GP level.  Aggregating it to this level implies that beneficiary details need not be revealed.
• Although we have discussed only the PMGSY and NREGS audits here, this applies to all CAG audits equally. 
• A good model to follow would be how the central government puts out program data on  PMGSY⁴ and MNREGS⁵ outcomes online.
• Note however, both of these portals would benefit from having a codebook describing the variables in the data. 
• The data presented in the portal should be presented in such a way that makes it possible to merge with other public data such as the Census. 

If such a portal can be created and maintained and can serve as a repository for all CAG data, it would serve as a rich resource for social science researchers working in this space, allowing for a more comprehensive analysis of India’s audit infrastructure.

Overall, we believe the effort involved in maintaining a database that is easily accessible for analysis would be low and the benefits would be high, and would prove to be a great public good for scholars as well as the policy community.

[1] e.g. Ferraz and Finan, 2008; Ferraz and Finan (2011); Colonelli and Prem (2017), Lichand et al (2016)

[2] The PMGSY is a rural road construction program, started in 2000, which aims to provide all rural habitations with all-weather road access. MNREGS is a workfare scheme which was started in 2006 in a few districts and subsequently expanded to cover the entire country in 2008. The scheme guarantees one hundred days of employment annually to every rural household who demands such work. The scheme envisages using unskilled manual labour to build essential rural infrastructure.

[3] A gram panchayat is a unit of local governance in rural areas. These were selected from blocks based on PPSWOR. However the variable on the basis of which PPSWOR has been carried out is not mentioned. 

[4] http://omms.nic.in/

[5] https://nregarep2.nic.in/netnrega/dynamic2/dynamicreport_new4.aspx

References

Afridi, F., A. Dhillon, A. Roy Chaudhuri and D. Kaur, 2020 “  Efficacy of Top down audits and Community Monitoring: A Review” , Working Paper, King’s College.

Colonnelli, E. and Prem, M. (2017). Corruption and firms: evidence from randomized audits in brazil. Working Paper. Available at SSRN 2931602.

Ferraz, C. and Finan, F. (2008). Exposing corrupt politicians: the effects of brazil’s publicly released audits on electoral outcomes. The Quarterly journal of economics, 123(2):703–745. 

Ferraz, C. and Finan, F. (2011). Electoral accountability and corruption: Evidence from the audits of local governments. American Economic Review, 101(4):1274–1311.