12 May 2018, Workshop on Replicability and Reproducibility of Research Results in Science and Technology of Language (4REAL Workshop 2018), Miyazaki, Japan
Reproduction and replication of research results are at the heart of the validation of scientific knowledge and of the scientific endeavor. But despite their key importance, reproduction and replication have not been sufficiently encouraged given the prevailing procedures and priorities for the reviewing, selection and publication of research results. This workshop seeks to foster the discussion and the advancement on a topic that has been given insufficient attention in the research area of language processing tools and resources and that has been an important topic emerging in other scientific areas. The workshop will be collocated with LREC 2018 11th Language Resources and Evaluation Conference.
We are nviting submissions of articles that present cases, either with positive or negative results, of actual replication or reproduction exercises of previous published results in our area. We are interested also in articles discussing the challenges, the risk factors, the appropriate procedures, etc. specific to our area or that should be adopted, or adapted from other neighboring areas, including methodologies for monitoring, maintaining or improving citation of language resources and tools and to assess the importance of data citation for research integrity. This includes also of course the new risks raised by the replication articles themselves and their own integrity, in view of the preservation of the reputation of colleagues and works whose results are reported has having been replicated, etc.