A general practitioner’s perspective on the guest blog by Dr Kevin Denny

Dr William Behan, co-author of “Does Eliminating Fees at Point of Access Affect Irish General Practice Attendance Rates in the Under 6 Years Old Population? A Cross Sectional Study at Six General Practices”

You can download a .pdf of this blog post with references here. 

Most of the state sponsored; CSO or university department generated statistics on general practice utilisation since 2001 have been based on surveys employing 1 year recollection. Dr. Denny uses Growing Up in Ireland (GUI) the largest database available for the purpose of determining the marginal effect of granting private under 6s patients medical cards. Intuitively this makes sense……or does it when the potential biases of that particular survey data are explored?

We really have to examine biases in statistics collection to determine what is more likely.

First of all, the Growing Up in Ireland (GUI) data is based on a questionnaire employing  9 month or 1 year recollection : The relevant question is: “Since <baby> was born, how many times have you seen, or talked on the telephone with any of the following about <baby’s> physical health

A general practitioner (GP), or family physician

A public health nurse or practice nurse ”[ii]

Review of the national and international data on surveys of this type suggests that this question automatically provokes a 30-40% degradation of recollection due to the time period of recall demanded in the survey (with the GMS patients’ disproportionately under-recollecting attendances).

Its raw data is subject to 3 other significant universally recognised biases:

1. proxy recollection which has the effect of further reducing recollection by 20%.

2. practice nurse consultations are recorded as public health nurse consultations, therefor under reflecting general practice attendance rates.

3. the salience (registered importance in the memory of the survey respondent) of the clinical consultation which appears to be less important for public patients due to a possible combination of the lack of fee associated with the consultation and the increased likelihood it will be perceived by the patient themselves as trivial compared to a private patients perception of their own consultations

There are two government figures on the suggested increase in attendance rate by new medical card patients; 0.4 and 0.9 per annum. I have not seen the method of calculating the 0.4 figure but I have seen the paper supporting the 0.9 figure. It is from GUI data which is subject to all the above biases and it miscalculated the number of new GMS patients by 60,000 (affecting the numerator) and has 2,800 more GPs in the denominator, which is larger than the government figure of 2,415.

The IMJ 2014 paper suggests an extra number of consultations of 750,000 brought on by introducing free GP access to the whole population.

Considering that

• it is an audit and is not subject to many of the biases the GUI survey suffers from,
• most of the biases affecting it are downward rather than upward with the exception of the relative difference in private/GMS attendance rates,
• the extra demands of routine care in the new contract which will have a greater effect on current private than GMS patients,
• the UK data which indicates attendance rates in this age group are over twice what GUI predicts for Ireland with minimal differences between the income groups and
• NUI Galway data looking at the under 5 year old age group from the 2006 Lifeways study[iii] which has figures of overall GMS attendance rates that are remarkably similar to the IMJ 2014 data (but also has higher private attendance rates that cannot clearly be explained),
• the population studied in the 2014 IMJ paper was the same as the 2013 IMJ paper by the same authors and it predicted previous and subsequent Northern Ireland GP visiting rates with remarkable accuracy

it appears that the IMJ 2014 paper should be the one used for considering future workload as its figures are taken from real world data and not fancy mathematical analysis of heavily biased data.

It is important that the data used to promote national health policy changes is not only convincing, but it must also be reliable. It appears that both the GUI data and the government sponsored analysis of it is not reliable enough to have any role in the debate about future health policy changes.

Where do these figures come from that suggest such a large degradation of recollection versus audit of GP records?

The OECD annual publication Health as a Glance states “estimates from administrative sources tend to be higher than those from surveys because of problems with recall and non-response rates”. The 2005 edition of OECD Health at a Glance reviewed GP consultation rates in 27 OECD countries. For 8 of the countries it had both administrative and survey data. Consultation rates measured by assessment of records were found to be 28% higher than rates assessed by survey (6.63 vs 5.17). [iv]

Eurostat, the European statistical office and the WHO clearly highlight the 1960s and 1970s work of Charles F Cannell and colleagues which identifies the degradation of recollection that occurs with time and the importance of salience of the questions in surveys.[v]

“45. There are three problems that health accountants must consider when contemplating the use of household surveys as the primary data source for estimating household expenditures:

i.               sampling error in surveys;

ii.              biases arising from non-sampling errors; and

iii.            lack of annual repetition of most household surveys.

Of these, the second is typically under-appreciated, and is behind the most important errors that can

arise when estimating household spending.”

“51. These non-sampling biases can be large (see Box 1), and tend to show the following patterns:

(i) The number of events forgotten increases proportionately with the length of the recall period.

(ii) Events with less salience or impact on the individual are more likely to be forgotten.

(iii) Proxy respondents tend to report 20% fewer events.”[vi]

There are other international papers that suggest that there is minimal degradation of recollection with time with the exception of the elderly and frequent attenders. However, on close examination of these there is often a significant difference between how the survey question is asked: survey questions that are longer and more detailed are more likely to provoke a reliable response than shorter survey questions such as used in GUI. Other papers don’t correlate patient recollection with general practice records, but compare recollection with proxy records such as state insurance payment records which will be missing telephone consultation activity, consultations in which a charge is not applied or when the receipt from the general practitioner is lost by the patient. Some surveys correlate issues other than general practice utilisation, or only correlate routine general practice utilisation with records and do not include acute episodic care.

But, the detractors from this concept of memory loss with time will all state, where is the recent evidence supporting this unreliability of survey data depending on the period of recollection rather than using Charles Cannell figures? We actually do not need to go very far to find it but we just need to travel back to 2001 when Ireland produced two surveys on GP attendance rates.

The 2001 CSO National Quarterly Household Survey was performed on a population of 44,844 and used a 2 week recollection period for GP consultations.[vii]  Living in Ireland Survey (LIIS) formed the Irish component of the European Community Household Panel (ECHP): an EU-wide project, co-ordinated by Eurostat, the statistical office of the European Union.  The final LIIS was performed also in 2001 on a population of 6,521 adults and employed a 1 year recollection of GP consultations. The recollection figures for GP utilisation rates were36% lower in the LIIS 1 year recollection than the CSO 2 weeks recollection, with public patients forgetting significantly more than private patients.

(These are 2 separate samples reflective of the national population)

The question asked by LIIS 2001 was “During the last 12 months, about how many times have you consulted with each of the following either here in your own home or in their surgery or offices? Please include only consultations made on your own behalf and exclude those made on behalf of children or other persons …

– Your family doctor, GP (including home visits by the doctor)“[viii]

The CSO 2001 question was “DOCTOR: Did you consult with your GP or family doctor in the last 2 weeks?                                1.              Yes                                              2.              No”[ix]

(Note that this provides potential for under-recording attendance as multiple consultations in the last 2 weeks will only be reflected as one.)

So we have an Irish survey asking similar questions about GP utilisation but employing different recollection periods which seems to be the most plausible reason for the significantly different responses.

Come forward to 2006 and see what international best practice was in healthcare surveys. The European Health Interview Survey (EHIS) was developed between 2003 and 2006, and the first wave of it was implemented during the period 2006-2009 under a gentlemen’s agreement by 19 different countries.It requested reports of consultations with GPs during the previous 4 weeks.[x]

Irish results from the 2007 CSO National Quarterly Household Survey were not included in the EHIS wave 1 report due to the information collected not being comparable, particularly due to the different recall period. [xi]

Official Irish Adult GP attendance rate figures are based on the 2007 (sample population 21,253)[xii] and 2010(population 15,673)[xiii] CSO National Quarterly Household Surveys. They employed the same 1 year recollection of consultations as the 2001 Living In Ireland Survey. (Sample size 6,521)

The 2007 and 2010 CSO surveys show a much lower GP attendance rate for both GMS and private patients compared to the larger 2001 CSO survey (44,844 population) which was based on 2 weeks recollection, but almost identical consultation rates to the 2001 LIIS which was also based on a 1 year recollection. The 2007 and 2010 CSO surveys highlighted a reduced GP attendance rate for GMS patients as they get progressively older over the age of 54 y.o in 2007 and over the age of 64 y.o. in 2010 which is at odds with previous research. The 2007 CSO methodology was changed between the original pilot survey in early 2007 which demanded 2 week recollection to a 1 year recollection despite the common practice of the European Statistics Office.[xiv] The 2007 survey showed GMS patients attendance rates peaking in the 45-54 y.o. age group then tapering off with progressive age, and yet it did not generate a review of its methodology. These CSO surveys predict a much lower current number than the figure of approximately 24 million consultations occur in general practice annually (15.4 million GMS consultations and 8.8 million private consultations) predicted in the 2013 paper I co-authored.[xv] That was based on a population of 20,706 adults from 6 different general practices spread though out the country reflecting the national urban/rural device and national deprivation mix. They used the same practice management software the same way so their data could be considered comparable. That cohort was more reflective of the national age/sex/GMS demographics than both the 2007 and 2010 CSO surveys.

Extrapolating our findings to a national population of 4.59 million (2013 CSO), suggested that were private patients to attend at the same rate as GPVC patients, then one might anticipate an increase in general practice workload of 4.4 million consultations per year, which is well in excess of the figures currently used in planning of Universal Primary Care.[xvi] We suggested Irish GPs would need to provide 28 million consultations of work practices remained the same and there was universal free GP access. Extrapolation of RCGPNI Presidents 2015 figure for GP attendances in Northern Ireland to the Republic population suggests an expected attendance rate of 31 million consultations.

A report to North South Inter-Parliamentary Association in September 2013 States: “At least 350 GP practices are in operation in Northern Ireland.The average number of patients per practice is 5,374, (around 1,631 patients per GP). Patients consult their practice on average 6.5 times per year.This rate is 100% higher – (twice as much) as patients who see their GP in the Republic of Ireland.”  (based on Information obtained from BMA Northern Ireland, response dated 11.6.13 )

But poorer under 6s will need more healthcare than richer under 6s.

An interesting figure from GUI page 127 is the relative difference between GMS (public) and private patients usage of routine, free public health nurse check-ups and general practice nurse immunisation visits of 2.84 for GMS patients and 2.66 for private patients, which could indicate that if free routine health care is made available to the whole population, all sections of the community will take advantage of it. GUI itself indicates that the infants from higher social classes are more likely to receive their six-month vaccine on time.[xvii] A comparison of breast and cervical cancer screening rates depending on socio-economic group indicates better uptake of free routine care by the richer and healthier members of our community when it is available.[xviii] These figures just confirm the “Inverse Care Law” devised by Julian Tudor Harte in 1971. If universal care is made available to the population the richer in society will benefit from it disproportionately, particularly if healthcare distribution is left open to market forces.[xix]

But general practice does not just offer routine care but also acute episodic care, and surely the more deprived sections of the population will have greater need of this because of the greater rates of illness and lower life expectancy in these communities?

The British Medical Journal published a review of 1 million East London patients measuring general practice complex consultation usage based on income quintiles (lowest 20%, 20-40% incomes, etc) and made a very interesting discovery. In deprived areas attendance rates are much greater in the 30-70 year old age group, but remarkably similar attendance rates occur otherwise. It appears to suggest a concept that over a lifetime, poorer people develop chronic diseases at a younger age, engage with general practice more at a younger age and use up the same allotment of healthcare activity over a lifetime as richer people do but their lives are shorter so they consume it more quickly. The attendance rates over the age of 70 are similar probably due to a type of survival of the fittest in the most deprived areas.  This concept has never been considered in Ireland. [xx]

Minister Alex White stated in the Oireachtas that “research carried out for the Department in 2013 indicates that fee-paying children under 6 years of age have an annual GP visit rate of 2.7, whereas, medical card/GP visit card holding children in the same age cohort have an annual visitation rate of 3.1.”[i] This would not be consistent with 2008/09 UK QRESEARCH (audit of 4.3 million patients involving 21.7 million clinical consultations in the  UK) which predicts an attendance rate of 6.8 for the same age group for delivering a very similar service.[ii]^,[iii]  Applying longitudinal techniques to analyse data that follows the behaviour of individuals over time is very sophisticated and appears to be a good strategy, but it appears from the GUI results that is only works if it is applied to audit of clinical records rather than survey data.

The more recent Irish Government projected figures for extra GP consultations as a result of universal free general practice access appear to come from another analysis of GUI data. Its figures were presented at the 2014 annual meeting of the Association of University Departments of General Practice in Ireland (AUDGPI). It was stated at this conference “Based on an estimated population of 360,000 under sixes served by 2800 GPs, with 60% of these getting a medical card, an increase of 77 consultations per GP per year was estimate”. This figure supposes a new under 6s medical card figure of 216,000.

This is a common calculation mistake made by assuming that the national percentage of medical card patients is evenly spread over the whole population, forgetting that the over 65 year olds and particularly the over 70s population are disproportionately recipients of medical cards, resulting in the under 65 year old population having a much lower medical card eligibility that the national average.

My colleagues and I in 2014 in our paper on under 6s attendances stated that our “findings are based on a sample of over 1900 children and a total population comparable to Ireland’s national population in terms of proportion aged between 1 and 6 (7% versus 8% nationally) and GMS eligibility (34% versus 33% nationally)^” referencing Population and Migration Estimates (Accessed 22 February 2014) and the Primary Care Reimbursement Service STATISTICAL ANALYSIS OF CLAIMS AND PAYMENTS 2012.  

We calculated that the valid number of potential new under 6s patients would be the total April 2013 CSO estimation of the population of 432,320 and the total number of GMS patients were 173,780. This would leave 258,540 under 6s as non-GMS patients. However, a few percent of GMS entitled patients don’t have a valid GMS card at any one time so we decided to round down our original figure to 250,000 to give a more realistic number for the calculation of projected attendance rates.

IMJ 2014 data as it is missing the out of hours and telephone consultations in Table 2. This was as a result of a comment made in response to the IMJ  2013 paper at AUDGPI 2014, when it was suggested that as the out of hours/telephone data came from one practice, it should be separately represented. This is appearing to cause some confusion with the 2014 IMJ audit as it is being presented in the public as private patients attending twice per year and GMS patients five times per year when the out of hours and telephone consultations should be added to give figures of 2.7 and 5.8 for total private and GMS consultations.

Because the 2014 IMJ paper is an audit, it is not exposed to many of the non-sampling errors that affect the GUI survey.  ie. Errors that cause results that deviate from the true values in the population being surveyed. There is a potential sampling error due to the size of the doctor and patient population audited only representing about 0.6% of the national populations. However it does appear to reflect reliably CSO/PCRS data in terms of demographics so it is an accepted cost of ensuring that the data generated is reliable. Another downward bias in its figures would be as a result of its data collection which is retrospective and therefor only patients who are a year old or more can be included in the sample. It is true that private patients may have non-practice consultations elsewhere more than GMS patients increasing further the downward bias on the figures which would affect private more than GMS patients, but this is not something that can be measured by audit in Ireland. The the rounding down of both the national population and difference in attendance rate figure between GMS and private patients when making the final extra consultation figure of 750,000 as estimated by our paper is subject to further downward bias. Also, this study was retrospective and the new Under 6s contract potentially demands extra routine attendances at 2 and 5 years old and also the under 6s demographics have changed since the 2012/2013 population data was employed in the 2014 paper with Minister Varadkar claiming that there are not a potential 280,000 extra GMS patients. Both these factors will cause an increase in activity.

Considering the Irish data on routine care and UK data on all care; deprivation appears to affect the under 6 attendance rates much less than would intuitively be expected. There is also the potential that  in the short term, there might be a release of pent up demand that has developed between the announcement of free GP care and its actual delivery, resulting in a spike of attendances from new medical card patients after its initial introduction. Also, that with time, free GP care might induce a cultural change that promotes excess attendances. However we might not see the coalescing of more deprived and affluent patients attendance rates figures for a long time.

Considering all these biases with most of them causing a downward bias on the IMJ 2014 data and only the relative difference in attendance rates having an upward bias on it, it is very likely that the overall increased number of attendances of 750,000 as suggested by the IMJ 2014 paper is possibly on the conservative side.

Not only is the GUI data inconsistent with UK data (suggesting Irish patients attend their GP less than half the number of times UK patients do), GP visiting for private 9 month olds of 3.2 would also be totally inconsistent with the Health Protection Surveillance Reporting of childhood vaccination rates on 92% in this cohort, indicating most children have had 3 attendances for 2,4 and 6 month vaccination on top of possible free 2 week and 6 week baby checks through the Maternity and Infant Care Scheme. This results in most children having at least 3 but some up to 5 free routine GP attendances before any activity associated with illness is considered. [iv]^,[v]The GUI survey also claims 80% of 9 month olds visited their GP in the last 9 months but over 92% must have visited for at minimum one of their vaccinations.

Government commissioned GUI data analysis projects the figure of 77 extra consultations per GP per year as a result of universal free under 6 GP access. This is based on the number of GPs being 2,800 despite the government claiming that it only offered 2,413 contracts. (It is acknowledged that 2,800 figure is a more realistic assessment of the whole time equivalent GPs in clinical practice but that is another story)[vi] Government GUI analysis suggests that the population of new medical cards to be 60% of the rounded figure of 360,000 (clearly only the under 5, or 0-4 year old cohort and not the under 6 or 0-5 year old cohort) or 216,000 when the IMJ 2014 data, clearly calculates the population of potential new medical card patients to be a potential 258,540 and rounded it down to 250,000.

The government modelling suffers from massive underestimation of the under 6s population and over-estimation number of contracts being offered

Dr Kevin Denny uses mathematical modeling of GUI data and a reasonable assumption of 270,000 extra patients and 2,500 whole time equivalent GPs.

The problem with the GUI data caused by its 9 month or 1 year proxy recollection and the expected degradation of recollection of GMS more than private consultations that have not been corrected for, the assignment of practice nurse consultations to Public Health Nurse activity, its inconsistency with reliable UK data and multiple Irish audits including HSE audits of vaccine uptake. It clearly has too many non-sampling biases to be considered reliable for the purpose of projecting future workload.

I accept that Dr. Denny’s figures are a very sophisticated mathematical modelling of GUI data. The problem is that it does not relate in any way to real world of general practice activity. As the saying goes: junk in; junk out.

There are always going to be biases in any study, but non-sampling errors caused by asking the wrong question and degradation of recollection have a much greater effect that is consistently reproduced as is evident in surveys requesting one year recollection such as LIIS 2001, CSO 2007, 2010 and also the large-scale, nationally representative, longitudinal study on ageing in IrelandThe Irish LongituDinal Study on Ageing (TILDA) data when it is compared with 2001 CSO, IMJ 2013 which incorporates 2010 NUI Galway, 2006 Lifeways and QRESEARCH data.[vii]

Considering the BMJ 2014;349:g6814 article on how deprivation has minimal effect on children and young adults GP attendance in a universal free general practice system, and all the obvious methodological problems with the evidence used to support official figures for expected under 6 attendance rates, it is quite clear that it would be unwise to use government GUI analysis as that basis of informing significant changes in the national health system. Any minor methodological questions and potential biases that affect the IMJ 2013 and 2014 papers appear to cancel each other out particularly when the results are compared to other Irish and UK audits, or the 2001 CSO survey which only requires 2 week recollection.