Clinical Practice & Epidemiology in Mental Health




ISSN: 1745-0179 ― Volume 15, 2019
REPLY

Cost-effectiveness of US National Institute of Health and European Union FP7 Projects on Active Ageing and Elderly Quality of Life-Author's reply



Mauro Giovanni Carta1, *, Michela Atzeni1, Alessandra Perra1, Quirico Mela1, Martina Piras2, Giorgia Testa2, Germano Orrù3, Iskren Kirilov2
1 Department of Medical Sciences and Public Health, University of Cagliari, Cagliari, Italy
2 Innovation Sciences and Technologies, University of Cagliari, Cagliari, Italy
3 Department of Surgical Sciences, University of Cagliari, Cagliari, Italy

Abstract

Background:

The use of bibliometric analysis to assess scientific productivity and impact is particularly relevant for EU funding programs. The objective of the present study is to assess the impact on scientific literature by focusing specifically on the cost-effectiveness of FP7 and NHI projects in the fields of AA and QoL, respectively.

Methods:

Twenty projects were randomly selected from the CORDIS database in accordance with the following criteria: funded by the FP7; accepted from 1st January 2007 to 31st December 2012; concluded by 31st August 2017;

For each project selected, we determined: number of publications in Scopus and Google databases attributable to the project; number of papers published in Q1 quartile of the SCIMAGO rank; number of citations found in Scopus and Scholar Google; amount of funds allocated.

Results:

The study has confirmed the results of the previous one, namely that the number of publications and the number of citations per project on active ageing are similar in projects funded by the NHI in the United States and those funded by the FP7 in Europe. However, when it comes to cost-effectiveness, it results that European projects have a cost ten times higher than the Americans ones.

Conclusion:

Our study shows lower cost-effectiveness of FP7-European projects than the American-NIH on active aging. The results of this research, albeit with the limits already outlined, will have to be taken into consideration in the evaluative research of the future.



Article Information


Identifiers and Pagination:

Year: 2019
Volume: 15
First Page: 10
Last Page: 14
Publisher Id: CPEMH-15-10
DOI: 10.2174/1745017901915010010

Article History:

Received Date: 10/11/2018
Acceptance Date: 15/12/2018
Electronic publication date: 23/1/2019
Collection year: 2019

Article Metrics:

CrossRef Citations:
0

Total Statistics:

Full-Text HTML Views: 153
Abstract HTML Views: 143
PDF Downloads: 81
ePub Downloads: 65
Total Views/Downloads: 442

Unique Statistics:

Full-Text HTML Views: 99
Abstract HTML Views: 80
PDF Downloads: 52
ePub Downloads: 45
Total Views/Downloads: 276
Geographical View

© 2019 Carta et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: https://creativecommons.org/licenses/by/4.0/legalcode. This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


* Address correspondence to this author at the Department of Medical Sciences and Public Health, University of Cagliari, Italy; Tel: + (39) 335 499994; Fax: + (39) 070 6093498; E-mail: mgcarta@tiscalli.it




1. INTRODUCTION

A study recently evaluated the impact on scientific literature of a sample of projects funded by the 7th European Union’s Framework Program for Research and Innovation (FP7) on Active Ageing (AA) and elderly quality of life (QoL) in comparison to the same number of projects funded by the USA’s National Institute of Health (NHI) [1Kirilov I, Atzeni M, Perra A, Moro D, Carta MG. Active Aging and Elderly’s Quality of Life: Comparing the Impact on Literature of Projects Funded by the European Union and USA. Clin Pract Epidemiol Ment Health 2018; 14: 1-5.[http://dx.doi.org/10.2174/1745017901814010001] [PMID: 29492097] ]. The results show that the European and American projects have a similar impact in terms of a number of published articles and number of citations by published papers per single project.

This study was part of a broader one aiming to set up a methodology that allows measuring the impact of European projects funded by Horizon 2020 [2European Commission https://ec.europa.eu/programmes/ horizon2020/] and the subsequent EU framework programs for research and innovation. The use of bibliometric analysis to assess scientific productivity and impact is particularly relevant for the EU funding programs, in particular, in relation to the promotion of so-called open science (especially, open access to publications and scientific data) and also in relation to the importance they have in the market created innovation.

The choice of the scientific domain of AA and QoL is motivated by the fact that it has been one of the priority biomedical areas of FP7 supported with a total budget of €115 million [3European Commission https:// ec.europa.eu/ research/ fp7/ index_en.cfm ? pg = sitemap # themes].

The number of Europeans aged over 65 is expected to nearly double, from 85 million in 2008 to 151 million by 2060, and the number of those over 80 is expected to rise from 22 to 61 million in the same period. Research furthering lifelong health, active ageing and well-being will, therefore, be a cornerstone in the successful adaptation of societies to this demographic change [4Bramanti A, D’Aloja E, Cabras F, et al. The Elderly and the City: Lack of Knowledge on Violence Perception and Consequences on Daily Life. Clin Pract Epidemiol Ment Health 2018; 14: 46-52.[http://dx.doi.org/10.2174/1745017901814010046] [PMID: 29541151] ].

A recent commentary on the previous study acknowledged interest in the results presented while highlighting the need to introduce costs of the evaluated projects as an independent variable in future researches [5Sancassiani Federica, Petretto Donatella Rita, Romano Ferdinando, Preti Antonio. Exploring physical and psychosocial well-being and self-awareness as a new frontier in active aging. Clinical Practice & Epidemiology in Mental Health 2018; 14: 294-95.]. The commentary suggests “The Horizon / FP7 calls to finance projects on average larger than NIH. If there were any disparities in the funds collected, this would not show an equal impact between the two shores of the Atlantic. This hypothesis must be verified” [5Sancassiani Federica, Petretto Donatella Rita, Romano Ferdinando, Preti Antonio. Exploring physical and psychosocial well-being and self-awareness as a new frontier in active aging. Clinical Practice & Epidemiology in Mental Health 2018; 14: 294-95.].

The commentary also introduced some elements that would suggest a greater impact on US publications. The authors state that the US is the world leader in terms of publications. They also recall that none of the 20 top scientific journals in Scimago ranking is European, and also the fact that, among the first 100 scientific journals of the SCIMAGO ranking, only 12 are European [6SCIMAGO https://www.scimagojr.com]. Moreover, recently published evidence suggests that in some countries of the European Union, the performance of scientific researchers does not have the same value for academic careers [7Carta MG. Why has scientific productivity increased in Italy? Lancet 2015; 386(10009): 2143-4.[http://dx.doi.org/10.1016/S0140-6736(15)01053-3] [PMID: 266389 64] , 8Preti A. Italian Abilitazione Scientifica Nazionale. Lancet 2017; 389(10071): 803-4.[http://dx.doi.org/10.1016/S0140-6736(17)30513-5] [PMID: 282481 77] ]. In the field of psychosocial interventions, an analysis of some systematic reviews indicates a greater impact of American literature compared to that of Europe [9Mura G, Carta MG. Physical activity in depressed elderly. A systematic review. Clin Pract Epidemiol Ment Health 2013; 9: 125-35.[http://dx.doi.org/10.2174/1745017901309010125] [PMID: 24009640] -12Agabio R, Trincas G, Floris F, Mura G, Sancassiani F, Angermeyer MC. A Systematic Review of School-Based Alcohol and other Drug Prevention Programs. Clin Pract Epidemiol Ment Health 2015 Feb 26; 11(Suppl 1 M6): 102-2.[http://dx.doi.org/doi: 10.2174/1745017901511010102.] ].

The objective of the present study is to assess the impact on scientific literature and cost-effectiveness of FP7 and NHI projects in the fields of AA and QoL, respectively.

2. METHODS

2.1. Design

Observational design.

2.2. Study Sample

Twenty projects were randomly selected from the CORDIS database in accordance with the following criteria:

  1. Funded by the FP7;
  2. Accepted from 1th January 2007 to 31th December 2012.
  3. Concluded by 31th August 2017.

As keywords for the search, we used “active ageing” and “ageing and quality of life”.

With the same criteria and keywords for the search, twenty projects funded by the US NHI were selected. A block was built for each EU project which included all eligible NHI projects funded ± 1-year with the matched EU project. Then 20 US projects were randomly selected, one from each block. The selected projects were automatically excluded when extracted.

2.3. Variables, Search for Publications and Measure of Citations

For each selected project, we determined:

  1. The number of publications found in the Scopus and Google databases attributable to the project, by introducing the project title, the keywords of the project, the name of the principal investigator into the two databases (Scopus and Scholar), and then verifying the correct link between each found paper and the project;
  2. Using the same method and criteria, we calculated the number of papers published in journals classified in the Q1 quartile of the SCIMAGO rank for each of the two groups;
  3. For each publication and for each project, we calculated the number of citations found in Scopus and Scholar Google databases;
  4. For each project, we found the amount of funds allocated.

The search of articles and relative citations referred to August 2017 (for producing data comparable with the previous research).

The search of papers was carried out using the name of the principal investigator, the names of the official collaborators of the project and the title of the project. Furthermore, all principal investigators were contacted by e-mail requesting information about any ongoing publications or other publications in a journal indexed in Scopus or Scholar that we may not have found.

The search for publications and citations was carried out in the blind by two independent researchers as well as verification of the correspondence of each publication to a project. A third researcher was invited to decide, in case of disagreement.

2.4. Data Analysis

The comparison will be processed using the one-way Analysis Of Variance (ANOVA) as the variables were measured on numeric scales. Nominal data were analyzed by means of the chi-square test.

2.5. Ethics

The study was conducted according to the principles of the Helsinki Declaration. The ethics committee of the Azienda Mista Ospedaliero Universitaria di Cagliari approved the study. All results of this study will be granted open access.

3. RESULTS

Table 1 shows the results concerning all indicators in the two samples (as the mean and standard deviation in the overall sample); Table 2 shows the results of comparisons. The two groups were homogeneous in all the indicators considered in the previous research (mean number per project of articles published in journals indexed in Scopus; mean number of articles per project published in journals ranked in the first quartile of quality (Q1) in Scimago; mean number of citations per project in journals indexed in the Scopus; mean number of citations per project in journals indexed in Google) with any difference between European and US projects.

Table 1A
Projects of US NIH.


Table 1B
Projects of EU.


Table 2
Comparisons Between EU And US Projects.


The projects for which the publications showed no citations in the journals indexed in the Scopus depository were 7 (35.0%) in the European sample and 2 (10.0%) in the American sample; this difference reaches the limit of statistical significance (χ2 = 0.59; p=0.05). The projects for which their publications did not show any citations in the journals indexed in the Google depository were 5 (20.0%) in the European sample and 2 (10.0%) in the American sample; this difference did not reach statistical significance (χ2 = 0.15; p=0.21).

The new indicator introduced the average budget per project showed a marked difference between the two groups. The mean budget of each project funded by the UE was 3,785,513 dollars against 315,813 dollars of each project funded by the NIH.

If we consider the total of 104 publications of the NIH project, the total of 114 publications of EU/7WP EU and the total cost of the 20 projects (NIH = $3,472,149; EU/7WP EU = Euro 44,543,630), the cost per publication is 33,380 dollars in the US and 390,733 euros in the UE).

4. DISCUSSION

The study confirms the results of the previous one, namely the fact that the number of publications and the number of citations per project on active ageing are similar between projects funded by the NHI in the United States and those funded by the FP7 in Europe. However, when it comes to cost-effectiveness, it results that European projects have a cost ten times higher than the Americans ones.

Owing to the size of the sample, these results should be taken with caution and considered mainly as a stimulus to go further into this research, including other indicators, and also extending it to other priority scientific areas.

Interpretation of results must be done in the light of the specificities of each program. Thus, the American programs appear to be more basic research-oriented, while the European ones are more focused on applied research and turning scientific results into innovative marketable products and services [13European Commission - Commission Staff Working Document. Horizon 2020 Annual Monitoring Report 2015 2016., 14Matthews H. The 7th EU research framework programme. Nanotechnol Percept 2005; 1: 99-105.]. On the other hand, without the presence of solid results, the European Medicines Agency (EMA) in Europe or the Food and Drug Administration (FDA) in the US [15https://www.ema.europa.eu/documents/report/annex-1-detailed-information-conditional-marketing-authorisations_en.pdf2018.] cannot approve innovative treatments or technologies. Therefore, the objective of supporting market competitiveness is strictly linked, if it does not coincide, with the aim of publishing a lot and with a strong impact on literature (ie attracting many citations). But in addition to needing a minimum of published evidence to file for patents or market, the more treatment is supported by scientifically robust published evidence, the more competitive it will be.

However, the modus operandi and rules of participation in both programmes are different. In the case of EU programs, the standard implementation mode is collaborative research that requires participants from at least three different countries, among EU Member states and associated countries. Moreover, the EU programs are not intended to replace national ones, but rather to support the achievement of political objectives, mainly completion of the European research area inspired by the European single market, in which researchers, scientific knowledge and technology circulate freely (as stated in Title XIX of the Treaty on the Functioning of the European Union).

In terms of expenditure on research and innovation (R&I), Europe still lags behind its main competitors. Thus, in 2015, in accordance with the OECD, Europe spent 1.96% of its GDP against 2.07% in China and 2.69% in the US. In the case of China, there has been a 60% growth in R&I expenditure from 2000 to 2015, while in Europe, the growth was only 15% [16Kneipp SM, Schwartz TA, Drevdahl DJ, et al. Trends in Health Disparities, Health Inequity, and Social Determinants of Health Research: A 17-Year Analysis of NINR, NCI, NHLBI, and NIMHD Funding. Nurs Res 2018; 67(3): 231-41.[http://dx.doi.org/10.1097/NNR.0000000000000278] [PMID: 296983 29] , 17 OECD Research and Development Statistics (RDS) - Retrieved September 2018 http:// www.oecd.org / innovation / inno / researchanddevelopmentstatisticsrds.htm]. It would be worth mentioning that achieving 3% of the EU's GDP invested in R&I is one of the targets of the Europe 2020.

Regarding scientific publications, the European Union, considered as the sum of its member states, maintains its excellence of production, both in terms of quality and quantity. Nevertheless, the European countries have been losing positions, in particular in some emerging scientific areas, including the biomedical. In 2017, only Germany, Italy and Poland maintained their SCIMAGO ranking for scientific production which they achieved 10 years ago (fourth, eighth and eighteenth), while others, such as France, Spain, the Netherland, Sweden, Belgium and Austria have stepped down in this ranking [1Kirilov I, Atzeni M, Perra A, Moro D, Carta MG. Active Aging and Elderly’s Quality of Life: Comparing the Impact on Literature of Projects Funded by the European Union and USA. Clin Pract Epidemiol Ment Health 2018; 14: 1-5.[http://dx.doi.org/10.2174/1745017901814010001] [PMID: 29492097] ]. In addition, the number of research centers in the world in the top 100 dropped from 14 in Europe to only 10 in 2018 [1Kirilov I, Atzeni M, Perra A, Moro D, Carta MG. Active Aging and Elderly’s Quality of Life: Comparing the Impact on Literature of Projects Funded by the European Union and USA. Clin Pract Epidemiol Ment Health 2018; 14: 1-5.[http://dx.doi.org/10.2174/1745017901814010001] [PMID: 29492097] ].

CONCLUSION

Our study shows lower cost-effectiveness of the FP7 European projects compared to the American NIH on active aging. This poorer outcome was found only in relation to cost of the researchers at the equal impact on scientific literature.

The differences between the EU and the US research and innovation landscape and policies may be a determinant on the difference in the costs of projects.

However, the results of this research, albeit with the limits already outlined, will have to be taken into consideration in the evaluative research of the future.

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGEMENTS

Declared none.

REFERENCES

[1] Kirilov I, Atzeni M, Perra A, Moro D, Carta MG. Active Aging and Elderly’s Quality of Life: Comparing the Impact on Literature of Projects Funded by the European Union and USA. Clin Pract Epidemiol Ment Health 2018; 14: 1-5.[http://dx.doi.org/10.2174/1745017901814010001] [PMID: 29492097]
[2] European Commission https://ec.europa.eu/programmes/ horizon2020/
[3] European Commission https:// ec.europa.eu/ research/ fp7/ index_en.cfm ? pg = sitemap # themes
[4] Bramanti A, D’Aloja E, Cabras F, et al. The Elderly and the City: Lack of Knowledge on Violence Perception and Consequences on Daily Life. Clin Pract Epidemiol Ment Health 2018; 14: 46-52.[http://dx.doi.org/10.2174/1745017901814010046] [PMID: 29541151]
[5] Sancassiani Federica, Petretto Donatella Rita, Romano Ferdinando, Preti Antonio. Exploring physical and psychosocial well-being and self-awareness as a new frontier in active aging. Clinical Practice & Epidemiology in Mental Health 2018; 14: 294-95.
[6] SCIMAGO https://www.scimagojr.com
[7] Carta MG. Why has scientific productivity increased in Italy? Lancet 2015; 386(10009): 2143-4.[http://dx.doi.org/10.1016/S0140-6736(15)01053-3] [PMID: 266389 64]
[8] Preti A. Italian Abilitazione Scientifica Nazionale. Lancet 2017; 389(10071): 803-4.[http://dx.doi.org/10.1016/S0140-6736(17)30513-5] [PMID: 282481 77]
[9] Mura G, Carta MG. Physical activity in depressed elderly. A systematic review. Clin Pract Epidemiol Ment Health 2013; 9: 125-35.[http://dx.doi.org/10.2174/1745017901309010125] [PMID: 24009640]
[10] Mura G, Sancassiani F, Machado S, Carta MG. Efficacy of exercise on depression: A systematic review. Int J Psychosoc Rehabil 2014; 18(2): 23-36.
[11] Sancassiani F, Pintus E, Holte A, et al. Enhancing the Emotional and Social Skills of the Youth to Promote their Wellbeing and Positive Development: A Systematic Review of Universal School-based Randomized Controlled Trials. Clin Pract Epidemiol Ment Health 2015 Feb 26; 11(Suppl 1 M2): 21-40.[http://dx.doi.org/doi: 10.2174/1745017901511010021.]
[12] Agabio R, Trincas G, Floris F, Mura G, Sancassiani F, Angermeyer MC. A Systematic Review of School-Based Alcohol and other Drug Prevention Programs. Clin Pract Epidemiol Ment Health 2015 Feb 26; 11(Suppl 1 M6): 102-2.[http://dx.doi.org/doi: 10.2174/1745017901511010102.]
[13] European Commission - Commission Staff Working Document. Horizon 2020 Annual Monitoring Report 2015 2016.
[14] Matthews H. The 7th EU research framework programme. Nanotechnol Percept 2005; 1: 99-105.
[15] https://www.ema.europa.eu/documents/report/annex-1-detailed-information-conditional-marketing-authorisations_en.pdf2018.
[16] Kneipp SM, Schwartz TA, Drevdahl DJ, et al. Trends in Health Disparities, Health Inequity, and Social Determinants of Health Research: A 17-Year Analysis of NINR, NCI, NHLBI, and NIMHD Funding. Nurs Res 2018; 67(3): 231-41.[http://dx.doi.org/10.1097/NNR.0000000000000278] [PMID: 296983 29]
[17] OECD Research and Development Statistics (RDS) - Retrieved September 2018 http:// www.oecd.org / innovation / inno / researchanddevelopmentstatisticsrds.htm

Endorsements



"Open access will revolutionize 21st century knowledge work and accelerate the diffusion of ideas and evidence that support just in time learning and the evolution of thinking in a number of disciplines."


Daniel Pesut
(Indiana University School of Nursing, USA)

"It is important that students and researchers from all over the world can have easy access to relevant, high-standard and timely scientific information. This is exactly what Open Access Journals provide and this is the reason why I support this endeavor."


Jacques Descotes
(Centre Antipoison-Centre de Pharmacovigilance, France)

"Publishing research articles is the key for future scientific progress. Open Access publishing is therefore of utmost importance for wider dissemination of information, and will help serving the best interest of the scientific community."


Patrice Talaga
(UCB S.A., Belgium)

"Open access journals are a novel concept in the medical literature. They offer accessible information to a wide variety of individuals, including physicians, medical students, clinical investigators, and the general public. They are an outstanding source of medical and scientific information."


Jeffrey M. Weinberg
(St. Luke's-Roosevelt Hospital Center, USA)

"Open access journals are extremely useful for graduate students, investigators and all other interested persons to read important scientific articles and subscribe scientific journals. Indeed, the research articles span a wide range of area and of high quality. This is specially a must for researchers belonging to institutions with limited library facility and funding to subscribe scientific journals."


Debomoy K. Lahiri
(Indiana University School of Medicine, USA)

"Open access journals represent a major break-through in publishing. They provide easy access to the latest research on a wide variety of issues. Relevant and timely articles are made available in a fraction of the time taken by more conventional publishers. Articles are of uniformly high quality and written by the world's leading authorities."


Robert Looney
(Naval Postgraduate School, USA)

"Open access journals have transformed the way scientific data is published and disseminated: particularly, whilst ensuring a high quality standard and transparency in the editorial process, they have increased the access to the scientific literature by those researchers that have limited library support or that are working on small budgets."


Richard Reithinger
(Westat, USA)

"Not only do open access journals greatly improve the access to high quality information for scientists in the developing world, it also provides extra exposure for our papers."


J. Ferwerda
(University of Oxford, UK)

"Open Access 'Chemistry' Journals allow the dissemination of knowledge at your finger tips without paying for the scientific content."


Sean L. Kitson
(Almac Sciences, Northern Ireland)

"In principle, all scientific journals should have open access, as should be science itself. Open access journals are very helpful for students, researchers and the general public including people from institutions which do not have library or cannot afford to subscribe scientific journals. The articles are high standard and cover a wide area."


Hubert Wolterbeek
(Delft University of Technology, The Netherlands)

"The widest possible diffusion of information is critical for the advancement of science. In this perspective, open access journals are instrumental in fostering researches and achievements."


Alessandro Laviano
(Sapienza - University of Rome, Italy)

"Open access journals are very useful for all scientists as they can have quick information in the different fields of science."


Philippe Hernigou
(Paris University, France)

"There are many scientists who can not afford the rather expensive subscriptions to scientific journals. Open access journals offer a good alternative for free access to good quality scientific information."


Fidel Toldrá
(Instituto de Agroquimica y Tecnologia de Alimentos, Spain)

"Open access journals have become a fundamental tool for students, researchers, patients and the general public. Many people from institutions which do not have library or cannot afford to subscribe scientific journals benefit of them on a daily basis. The articles are among the best and cover most scientific areas."


M. Bendandi
(University Clinic of Navarre, Spain)

"These journals provide researchers with a platform for rapid, open access scientific communication. The articles are of high quality and broad scope."


Peter Chiba
(University of Vienna, Austria)

"Open access journals are probably one of the most important contributions to promote and diffuse science worldwide."


Jaime Sampaio
(University of Trás-os-Montes e Alto Douro, Portugal)

"Open access journals make up a new and rather revolutionary way to scientific publication. This option opens several quite interesting possibilities to disseminate openly and freely new knowledge and even to facilitate interpersonal communication among scientists."


Eduardo A. Castro
(INIFTA, Argentina)

"Open access journals are freely available online throughout the world, for you to read, download, copy, distribute, and use. The articles published in the open access journals are high quality and cover a wide range of fields."


Kenji Hashimoto
(Chiba University, Japan)

"Open Access journals offer an innovative and efficient way of publication for academics and professionals in a wide range of disciplines. The papers published are of high quality after rigorous peer review and they are Indexed in: major international databases. I read Open Access journals to keep abreast of the recent development in my field of study."


Daniel Shek
(Chinese University of Hong Kong, Hong Kong)

"It is a modern trend for publishers to establish open access journals. Researchers, faculty members, and students will be greatly benefited by the new journals of Bentham Science Publishers Ltd. in this category."


Jih Ru Hwu
(National Central University, Taiwan)


Browse Contents



Webmaster Contact: info@benthamopen.net
Copyright © 2019 Bentham Open