The impact of the PRTLI

Thanks Richard. Overall I share your view that this evaluation was a wasted opportunity (and I say that as someone who had to figure out what to put into the forms!!).

I can only discuss what we (UCD Geary Inst) claimed and indeed didn’t claim.

From the outset it remains very very disappointing that we do not wish to examine the academic output in these exercises, as you note. I am very pleased to note the reference to Jim Heckman whose role at UCD indeed was funded by PRTLI (and of course is now funded by his own ERC grant!), but disappointed at how that was basically it as far as the Institute goes – good as this is, I think we have more to shout about (and dare I say it more than many other PRTLI investments!). To use accounting parlance, the HEA seem to not wish to value the ‘goodwill’ or the intangible asset.

As an economic evaluation it seems amazing not to see reference to the elephant in the room which is the missing counterfactual of what would have happened without PRTLI. That again is a hard thing to do but it does need some effort by evaluators. Simply listing how many papers are produced before and after PRTLI is, as you know, meaningless.

Also as an economic evaluation some calculation of the net costs would have been sensible but this perhaps would work against the Sciences where capital costs outweigh the recurrent (mainly human capital) costs. For Geary the vast bulk of what we received in two PRTLI rounds (1 & 3) was spent on hiring folks who of course pay taxes, spend in the economy etc etc. So the cost per job is high in the evaluation document but perhaps higher than it should have appeared given it includes capital spend and it not net of payroll costs recaptured (incidentally we did an exercise of this nature – mainly because I am married to an accountant! – so it says a lot for our influence).

The evaluation makes light also of the ability to use PRTLI funding to leverage additional (research) funding. Again this may be because the vast bulk of science funding outside of PRTLI is also exchequer funded (SFI/HRB etc) – for Geary the ratio of non exchequer to exchequer was 2.5 to 1 over the PRTLI3 period, including the PRTLI grant (ok ok – cue lots of comments that EU funding is indirectly exchequer but I am taking the goal when I can get it). And again the bulk of that money was spent on (a) uni overheads, and (b) salaries which are spent in the economy, payroll taxes etc etc.

I think PRTLI was a good thing but mainly for forcing some upping of the game and, in our case, bringing on some wonderful smart young folks who have gone on to bigger, better things in academic life and outside. I think these exercises miss that point completely which is a pity.

But upping the game may not be a good enough reason to continue the Programme – that needs debate. Also at some point PRTLI perhaps became a beast that had to be fed by subsequent rounds of….PRTLI (a point I hope not to be true about Geary in that we went as fast as possible for big EU/US/Philanthropic funding). Moreover PRTLI has – as the conclusions hints – become a major tool in the strategic planning toolbox of the HEA in that phrases like ‘national priorities’ and a requirements for collaboration have popped up in funding calls which, for better or worse, is somewhat unusual.

Unfortunately we are still not getting the answer to the question of the ‘knowledge economy’ debate. Your colleagues at the ESRI are far more attuned to that literature than I am, and this evaluation has nothing approximating a model of that nature. If data is missing we need to start gathering it properly. Personally I think all that an exercise like PRTLI can do is create an environment for this sort of research led economy stuff. It can’t be responsible for it. That is the folly of these evaluation exercises.

@Richard

“It is good that government agencies are increasingly open to such evaluation.”

If by such evalaution Richard means in house evaluations then I doubt if it is any good at all, as indeed his subsequent strictures and those of others suggest. Indeed, getting agencies to hire independent consultants would not be much better. Consultants quickly sense which side of the bread is buttered. In fact such a large evaluation can only usefully done by a totally independent and sceptically inclined entity. The CAG might fit that bill. Better yet would be an institution located abroad.

Ah, Diogenes, welcome. We have waited some time for you to join us on this board. Your strictures based on ‘he who pays the piper calls the tune’ are timely, but, being the original ‘cosmopolitan’ I’m surprised you reckon foreign consultants would tell the truth.

It is very simple economics which most economists here seem to ignore. There is no supply of genuinely independent, objective consulting services in these areas because there is no demand offering a range of prices for different levels of quantity and quality. But there is no shortage of demand for, or supply of, consulting services that present the answer those commissioning the work require.

In theory, the Oireachtas should be the body that commissions these independent assessments because it, ultimately, authorises all public expenditure. However, it has allowed itself to be deprived of the powers and resources to commission these assessments. And even if it had the necessary powers and resources, the ‘tyranny of faction’ exercised by the governing faction(s) would ensure that the scope and terms of reference of any assessment would prevent the consultants from doing their job properly. And, in any event, they would be left in no doubt about the specific bounds around their expected results.

All these assessments, policy impact assessments and regulatory impact assessments conducted or commissioned by the ‘government machine’ are just a mixture of some facts, estimates and bullshit in varying combinations depending on the answer that is required. It is time-consuming and totally futile to attempt to disentangle the components.

And, because some externally unquantifiable amount of bullshit is mixed in, the entire output is bullshit.