New bibliometric tool

Scholarometer is a new tool to rank academics. It uses crowdsourcing to disentangle people with common names, and to attribute people to disciplines and subdisciplines. It has a widget to display your results on your homepage. And it uses the h_f index, which allows for the comparison of people across disciplines. Paul Krugman beats Stephen Jay Gould.

39 replies on “New bibliometric tool”

Is this quantity over quality? Can we not just use our own judgment? Why not read what people have to say and then assessing it using one’s own knowledge, experience and intellect? Surely only dullards and the ignorant need resort to “what’s hot and what’s not” indices howsoever produced?

For property it’s location location location.
For professionals it’s reputation reputation reputation.
For academics, is it reputation, reference and publication?

@Richard

You can’t measure quality in a number. You can only make a subjective anlaysis (using my preferred method) and then assign it an imaginary and somewhat arbitrary measure.

How good looking was she? The foxy minx was 7.632761 out of 10.

@Richard Tol

Does this discount citations where you cite yourself. I presume this happens a lot as people develop on their work.

In fact, I am fairly sure that you linked to a paper of yours that had seven or so citations all to other papers authored by you. I can’t find it but I did find “The Social Cost of Carbon: Trends, Outliers and Catastrophes” which cites 12 of your own papers and “The Economic Effects of Climate Change” which cites 14 of your own papers.

Like I said, it would be strange if one did not cite one’s own papers if one works in a particular area. In fact, of the economics papers and books I have read the authors always cite themselves.

If there is no discount for self citation then the system might need to be looked at. 🙂

@zhou
It seems from the FAQ that there is a lot more work planned. It also explains how the index or weight is computed, and it appears to be reliant on data from Google Scholar.

@zhou: to paraphrase Woody Allen, self-citation is citing someone whose work you really respect

@Brian Lenihan

If anyone suggests that you need to pay more attention to the academics, read this thread.

@ simpleton

Ranking systems like this one or ideas are handy for getting an idea of who/what is influencing economics most. This site covers economics, so ranking systems for economic papers are relevant.

@MauriceOC
I think all other economics blogs other than this one have, over the last couple of years, carried several threads on the crisis of relevance facing academic economics. Self referential ranking rhymes with…

@simpleton

Everybody likes to know how they are performing and how they rank. Academics don’t face the electorate and they don’t have revenue. A bit of naval gazing is inevitable.

BTW, I expect Brian Lenihan to top the poll when his own assessment comes around.

Well said simpleton. I think if we had less interest in our egos and more in our nation we would never have the problems we have

@zhou
You make a fair point, I guess I have done the opposite. But I worry about academic priorities. Surely you can find something more important to discuss in a public forum? Spreads blowing out again? Leaked budget details on the front page of Itimes? But i acknowledge it’s your ball so you can play with it anyway you wish.

@simpleton
This thread will not make headlines.

Besides the pecking order of Ireland-based economists (which is of great interest to the contributors of this blog), our dear leaders continue to pour money into universities in the hope of creating academic excellence that would in turn lead to commercial success.

Scholarometrics helps to evaluate the return to this investment. The government investment has a strong disciplinary bias, so the h_f index would reveal whether this has resulted in a particularly strong improvement in the performance of the favoured disciplines.

@ richard tol
does the government not pour a lot of money into the ESRI?
How much did the bail out of the ESRI pension scheme cost?

@Richard Tol
“Besides the pecking order of Ireland-based economists (which is of great interest to the contributors of this blog), our dear leaders continue to pour money into universities in the hope of creating academic excellence that would in turn lead to commercial success.

Scholarometrics helps to evaluate the return to this investment.”

Has anyone, not connected with a vested interest ever measured return for investment in education? That russian economist from TCD was on Vincent Browne a week or two ago and he made an assertion that not a single job has ever been created from our recent attempts at funding science in uni

@Enda
I didn’t come close to even implying that. I’m arguing from the premise that this is a blog; one that addresses issues of topical interest around the general theme of ‘the Irish Economy’.
How you all rate and/or how good a job you are doing is terribly interesting I’m sure, but should be a matter mostly between you and your employer; important, but not sure if it belongs here, given other, slightly more pressing matters. IMHO.

@simpleton, fair enough. I think the website is more of a sandpit for “Irish economists” because many issues discussed (behavioural economics, Chinese trade flows) aren’t directly relevant to the Irish economy but are certainly economics. But point taken.

What if 30 academic papers say something like ‘Tol (2006) is fundamentally misconceived and can be ignored by all serious students of this subject’? Or ‘O’Rourke (2009) contains elementary mistakes that would disgrace an undergraduate’? Do these count as 30 citations?

(No offence, these are just the two academic names in this thread)

nothing wrong, as an author or a journal in self citation, if the work being cited is relevant.

@acknefton
The ESRI voluntarily subjects itself to peer-review and comparison.

Our pension fund was nationalized. I wish I could tell you that I am better of as a result.

@entropy
Gurdgiev may have dramatized it a bit, but I would not be surprised if the average return on public investment in R&D is indeed low. There has been no independent evaluation, so it is hard to tell. There have been a number of window dressing exercises, and even those do not impress.

@richard tol

Well, whatever your scholometric rating might be you are the first person in a long while who has spoken the plain truth and not the spin of academia and for that I thank you!

One of the problems why science funding is going nowhere is because we’ve aimed far too high in most cases and we haven’t stratified our students into sensible divisions- those few who are potentially world class, those who might not be world class but are entrepreneurial, those who would make great industrial scientists etc. Too many people with 1st class honours and a view of science that says getting published in nature is all that counts. Everyone is taught to turn up their nose at pharmaceutical, chemical,engineering type jobs but its only when we have people trained by working in industry will we see innovation emerge and a multiplicative effect in job creation result

@Richard

Does this tool include the number of weeks annualy actually worked by Academics ? If it does it probably explains the poor rankings for the Irish Universities.

@entropy

”Everyone is taught to turn up their nose at pharmaceutical, chemical,engineering type jobs but its only when we have people trained by working in industry will we see innovation emerge and a multiplicative effect in job creation result”

This is not a fair comment. Everone is not taught to turn their nose up at industrial jobs. The Chemical and Pharmaceutical Industry in Ireland has provided employment for graduates for years. PhD and other graduates have contributed extensively to the generation of IP within the companies where they are employed. Most scientists would be happy if the people they train had good careers in industry to go into.

see http://www.sciencecouncil.ie/media/asc091215_role_of_phds.pdf
for a report.

@PaulM
Yep, they have but a lot of our major industries in these sectors were established in the 1980s and 1990s when the outlook was more “down to earth” and when universities produced many excellent people to fill these jobs on very little funding, with very poor equipment to train students. My point is along the lines that we have now poured bucket loads of money into academia with very little recent return in terms of tangible jobs beyond people doing PhDs and Postdocs in the universities themselves…and this money is rapidly evaporating as we speak as its source was Bob the Builders tax take and property sales etc
I’ve heard from a college enterprise ireland representative that really the only plan is to continue this as a means of employing people indirectly with government funds and grants as researchers…..but would they be on the dole if their major employer, the university and allied research sector was gone due to reduced funds?
…put it another way, we focus our students on aspects of molecular biology research such as signal transduction, we dress this up as “cancer research”, perhaps we get the odd patent on a new chemical entity which we dub as a “potential drug cure ” funded by SFI researchers in a media release but are we really going to see factories opening on the basis of the discovery, however world class it may be? I can only think of a few successes , maybe Luke O’neill’s monoclonal antibody company, a few companies in micro and nanofluidics but not many people employed. Its all very sexy, its a dream come true for the Principal Investigator, but for the vast majority of students who will never make PI you see talent wasted on pie in the sky and a career that will end by 40 unless they find a job as a rep or maybe SFI employ them directly.
I was at a meeting of engineers last february hosted by an american trouble shooter hired by Cowan to advise on our knowledge economy and he pretty much made that very same point- that its all ideas and research but little by the way of end product manufacture and that Ireland was full of academics who travel to all the research meetings but none of the product design meetings ( massively attended by all the eastern european countries who lead the tables in actiual manufacturing)

@Ray
Two points for consideration:
(1) Who do you think will be more productive in general: someone working 14 hours a day, 6 days a week, 51 weeks a year, or someone working 8 hours a day, 5 days a week, 48 weeks a year? Economic history a pretty clear answer to that one.

More generally, people should be measured by their outputs not their inputs. Time worked is an input.

(2) You are probably thinking of summer as a long period of time off for academics. If you ask the academic economists on this blog, I think they might say that it’s probably the only real time each year that they get to do uninterrupted research.

@Ronan L

I would be happy that Academics worked 48 weeks a year and a 40 hour week but I am pretty certain this is not the case in Irish Universities and Colleges of Technology. I would be surprised that any Academic actually works 2,000 hours per annum in an Irish University or College of Technology. I worked in a University for 1 year years ago and saw at first hand how few hours they worked. My daughter worked in a College for the last 3 years and advises me that the situation has not changed. Looking at the Department sites for some of the Universities the research output of a lot of Academics is abysmal. If there is good input there should/will be good output.

My question to Richard Tol is whether weeks worked is a metric in the system,and, if it is, possibly may be affecting Irish University rankings in other systems.

@Ray
Instead of complaining about the presumed laziness of others, you could have clicked a few times and found out for yourself that neither nominal nor real hours worked are collected by this tool.

@ zhou

If there is no discount for self citation then the system might need to be looked at.

Are you suggesting that an unscrupulous bounder might attempt to game such systems? If so, shocking.

That russian economist from TCD was on Vincent Browne a week or two ago and he made an assertion that not a single job has ever been created from our recent attempts at funding science in uni

That particular Russian economist says a lot of things (among others, claiming that Obama is a communist and frequently comparing Ireland to Cuba). I would suggest that you ask people who actually know something in each area what spin-off and new technologies have emerged, as Gurdiev (if what you say is true) appears to have no interest in Havok and others even in his own university of TCD.

@Ray
In addition to Richard’s point above, I would add the following.

I have studied in and worked with people from an academic environment for most of the past decade and while there are lazy people, as there are in every sector (I’ve experience of academic, public, private and voluntary sectors), no other sector has inspired me by the percentage of people with amazing amounts of initiative, essentially entrepreneurs, who work hard, go off and get the funding they need to pay for important research and – if you’re obsessed about inputs rather than outputs – do actually work all the hours they can to do that research.

It is a bit unfair to compare Krugman to Gould. Gould after all is dead since 2002. Come back in 50 years and see which is remembered.

Besides, Krugman has not made it on to “The Simpsons” – yet, anyway. So that makes it evens.

I believe a lot of what Ronan and Richard say is true about dedicated academics in the universities, but there are real questions to be asked about the RTC’s (or “Institutes of Technology”, as they were needlessly re-branded with taxpayers’ money a few years ago). I remember PhD candidates in British universities in the ’90’s asking me if there really were such disneyland places as the RTC’s that they’d heard mention of, where the conditions were so great where no research was required, and they had incredibly long holidays. I believe the HEA investigated some of them a few years ago and all/many colleges didn’t even know what peer-review meant! Now, if we’re looking for savings….why not run these colleges for 52 weeks of the year (and the secondary schools too) with different student streams at different times of the year – we are paying their salary anyway (unless we stop that..) so why not keep them open and save money? The rest of us work all year, if we’re lucky enough to have a job…

Comments are closed.