jeki2011-blog

[jeki2011-blog]

Let’s move beyond the rhetoric: it’s time to change how we judge research

Stephen Curry

Declarations are bound to fall short. The 240-year-old United States Declaration of Independence holds it self-evident that “all men [sic] are created equal”, but equality remains a far-off dream for many Americans.

The San Francisco Declaration on Research Assessment (DORA) is much younger, but similarly idealistic. Conceived by a group of journal editors and publishers at a meeting of the American Society for Cell Biology (ASCB) in December 2012, it proclaims a pressing need to improve how scientific research is evaluated, and asks scientists, funders, institutions and publishers to forswear using journal impact factors (JIFs) to judge individual researchers.

DORA’s aim is a world in which the content of a research paper matters more than the impact factor of the journal in which it appears. Thousands of individuals and hundreds of research organizations now agree and have signed up. Momentum is building, particularly in the United Kingdom, where the number of university signatories has trebled in the past two years. This week, all seven UK research councils announced their support.

Impact factors were never meant to be a metric for individual papers, let alone individual people. They’re an average of the skewed distribution of citations accumulated by papers in a given journal over two years. Not only do these averages hide huge variations between papers in the same journal, but citations are imperfect measures of quality and influence. High-impact-factor journals may publish a lot of top-notch science, but we should not outsource evaluation of individual researchers and their outputs to seductive journal metrics.

Most agree that yoking career rewards to JIFs is distorting science. Yet the practice seems impossible to root out. In China, for example, many universities pay impact-factor-related bonuses, inspired by unwritten norms of the West. Scientists in parts of Eastern Europe cling to impact factors as a crude bulwark against cronyism. More worryingly, processes for JIF-free assessment have yet to gain credibility even at some institutions that have signed DORA. Stories percolate of research managers demanding high impact factors. Job and grant applicants feel that they can’t compete unless they publish in prominent journals. All are fearful of shrugging off the familiar harness.

So, DORA’s job now is to accelerate the change it called for. I feel the need for change whenever I meet postdocs. Their curiosity about the world and determination to improve it burns bright. But their desires to pursue the most fascinating and most impactful questions are subverted by our systems of evaluation. As they apply for their first permanent positions, they are already calculating how to manoeuvre within the JIF-dependent managerialism of modern science.

There have been many calls for something better, including the Leiden Manifesto and the UK report ‘The Metric Tide’, both released in 2015. Like DORA, these have changed the tenor of discussions around researcher assessment and paved the way for change.

It is time to shift from making declarations to finding solutions. With the support of the ASCB, Cancer Research UK, the European Molecular Biology Organization, the biomedical funder the Wellcome Trust and the publishers the Company of Biologists, eLife, F1000, Hindawi and PLOS, DORA has hired a full-time community manager and revamped its steering committee, which I head. We are committed to getting on with the job.

Our goal is to discover and disseminate examples of good practice, and to boost the profile of assessment reform. We will do that at conferences and in online discussions; we will also establish regional nodes across the world, run by volunteers who will work to identify and address local issues.

This week, for example, DORA is participating in a workshop at which the Forum for Responsible Metrics — an expert group established following the release of ‘The Metric Tide’ — will present results of the first UK-wide survey of research assessment. This will bring broader exposure to what universities are thinking and doing, and put the spotlight on instances of good and bad practice.

We have to get beyond complaining, to find robust, efficient and bias-free assessment methods. Right now, there are few compelling options. I favour concise one- or two-page ‘bio-sketches’, similar to those rolled out in 2016 by the University Medical Centre Utrecht in the Netherlands. These let researchers summarize their most important research contributions, plus mentoring, societal engagement and other valuable activities. This approach could have flaws. Perhaps it gives too much leeway for ‘spin.’ But, as scientists, surely we can agree that it’s worth doing the experiment to properly evaluate evaluation.

This is hard stuff: we need frank discussions that grind through details, with researchers themselves, to find out what works and to forestall problems. We need to be mindful of the damage wrought to the careers of women and minorities by bias in peer review and in subjective evaluations. And we need to join in with parallel moves towards open research, data and code sharing, and the proper recognition of scientific reproducibility.

Declarations such as DORA are important; credible alternatives to the status quo are more so. True success will mean every institution, everywhere in the world, bragging about the quality of their research-assessment procedures, rather than the size of their impact factors.

https://www.nature.com/articles/d41586-018-01642-w

[jeki2011-blog]

Dafam Rohan Hotel, Yogyakarta

[jeki2011-blog]

Data — from objects to assets

Sabina Leonelli

Data. The confusingly plural cornerstone of research. The grounding for a scientific understanding of the world. Lightning rods for the negotiation of political, social and economic interests.

Over the past 150 years, ideas have shifted drastically as to what counts as data, which data are reliable and who owns them. Once regarded as stable objects whose significance was determined by a handful of professional interpreters, data are now reusable goods. Their mettle depends on the extent to which they are mobilized across contexts and aggregated with others. Growing in volume, variety and value, data have come to drive the very process of discovery.

This explicit designation as assets has become possible only through a complex web of institutional, technological and economic developments. The history and consequences of how this web has been woven have repeatedly transformed research and its role in society.

Collecting commodities

Until the start of nineteenth century, efforts to collect facts and objects of study were spearheaded by visionary individuals, typically backed by wealthy patrons. Naturalists roamed the globe in search of biological specimens that were new to science. Court astronomers devised tools to observe new parts of the cosmos. The large quantities of data accumulated were systematized and analysed through simple and powerful models (think Kepler’s laws) and classification systems (such as that developed by botanist Carl Linnaeus). Thus was born the myth of the heroic theoretician, mining order from the chaos of observations. This individualistic view was tied to an understanding of data as fundamentally private — their scientific value residing in conceptual interpretation.

The nineteenth century marked a shift. Data, as we now recognize them, became institutionalized as social commodities. Their intellectual, financial and political worth arose from investments, requiring regulation and oversight. The botanical wonder cabinet that was Paris’s natural-history museum was reorganized as a world-leading, publicly accessible repository of objects of potential scientific value. By the 1850s, the natural-history museums of Berlin, London and New York City followed suit.

https://www.nature.com/articles/d41586-019-03062-w

[jeki2011-blog]

What are the barriers to growth in the halal industry?

[jeki2011-blog]

Developing social impact requires the research agenda to move beyond conventional academic boundaries.

Peter van Bergeijk 

The Dutch Senate recently passed a new Standard Evaluation Protocol (SEP). The SEP highlights the importance of social impact for research. The new Protocol was developed by the KNAW (Royal Netherlands Academy of Arts and Sciences), VSNU (Association of Dutch universities) and NWO (Dutch Science Council) and is to be used to evaluate academic research from 2015-2021. Based on recent discussions and publications, Peter A.G. van Bergeijk, Shyamika Jayasundara-Smits, and Linda Johnson discuss some of the implications of the Protocol’s new criterion, using Development Studies as an example.

The Dutch Standard Evaluation Protocol (SEP) adds new elements to well established research evaluation criteria. As before, excellent science will continue to be scrutinized as to its scientific merit and influence, if only because high impact is a disaster if the underlying research is bad. But academic quality (and its twin, academic productivity) is no longer the ultimate factor to be considered. Commercial valorization of knowledge and societal impact were added to the list of areas in which scientists must demonstrate their contribution.

Open question

This is an important change in particular for our own field, development studies. Development Studies aims at impacting society, in particular the transformation of society and the economy to the benefit of vulnerable groups. Activist-type research that is characteristic of many of the knowledge building activities in development studies, involves two-way communication with societal movements, local actors and other stakeholders. For a long time researchers of development policies have seen social impact as an ethical and moral responsibility of scientific research towards societies that increasingly realize that science can have positive and negative consequences on everyday life. Therefore ‘science with’ and ‘science for’ society is not just an option – it is a necessity in Development Studies.

https://blogs.lse.ac.uk/impactofsocialsciences/2014/11/20/developing-social-impact/

[jeki2011-blog]

[jeki2011-blog]

What scientists can expect when dabbling in science writing

Brittney G. Borowiec

I’m a zoologist, freelance science writer and assistant editor of Massive Science. My portfolio includes blogs, popular science and science education outlets, and even some children’s books. It took me a long time (and still does) to get my work out there into the world.

Breaking into science writing is hard. You might shoot an e-mail off to a magazine such as Popular Science, a television network such as CNN or your favourite blog, offering your services to write a piece. Days and then weeks might pass without a response.

But as grant awarders and other scientific institutions push for more public engagement with research, science communication seems to be a growing field.

There are many excellent resources for scientists pursuing science writing (see ‘Help and advice for budding science writers’). They were invaluable in teaching me the basics of pitching, interviews and storytelling. However, most resources lack a frank discussion of how hard it is to get started. This leaves would-be science writers blind-sided by the inevitable bumps in the road to publishing in the popular press.

https://www.nature.com/articles/d41586-019-01753-y

[jeki2011-blog]

Workshop in Grand Zuri Hotel, Yogyakarta

[jeki2011-blog]

Economic Theory Needs a Major Overhaul

Sergio Focardi

Current mainstream economic theory needs a major overhaul. Why? Because it seems to ignore that modern advanced economies are evolutionary complex systems. Both economic theory and political economics seem to believe that modern economies are big physical systems whose output can be precisely quantified in aggregate. But this belief is far from reality as modern economies change and innovate at a fast pace; they are more akin to biological systems. One of the most important aggregates is real GDP. Long-term economic growth is measured as percentage change of real GDP, expansions and recessions are also measured as changes of real GDP. Strictly related to real GDP is inflation, which is a closely watched global parameter. Even small changes in GDP or inflation provoke strong reactions.

“Real” GDP that isn’t so real

Unfortunately, both concepts, real GDP and inflation, are inadequate to describe economies whose products and services keep on changing continuously and rapidly and whose complex structure also keeps on changing together with the web of symbolic values of goods and services. Real GDP is not a measure of the quantity produced by a country. Instead, it is an abstract term obtained discounting nominal GDP, that is the market value of economic output, by an inflation index.

https://www.socialsciencespace.com/2019/10/opinion-economic-theory-needs-a-major-overhaul/

[jeki2011-blog]

We are pleased to announce that the Jurnal Ekonomi dan Keuangan Islam (JEKI) has been accredited by Ministry of Research, Technology and Higher Education of the Republic Indonesia with Decree No. 10/EKPT/2019. JEKI was accepted at the S3 level in SINTA (Science and Technology Index). Articles published in the journal will be more visible and will most likely receive a higher citation.

[jeki2011-blog]

Dafam Rohan Hotel, Yogjakarta

[jeki2011-blog]

[jeki2011-blog]

Halal Tourism: Attracting tourists from the Middle East