recentpopularlog in

pierredv : sciencemag   7

Estimating the reproducibility of psychological science | Science, Aug 2015,
"Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. ... The claim that “we already know this” belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not."
reproducibility  replication  psychology  ScienceMag  scientific-method 
july 2018 by pierredv
AI researchers allege that machine learning is alchemy | Science | AAAS, May 2018
Via John Helm

"Speaking at an AI conference, Rahimi charged that machine learning algorithms, in which computers learn through trial and error, have become a form of "alchemy." Researchers, he said, do not know why some algorithms work and others don't, nor do they have rigorous criteria for choosing one AI architecture over another. Now, in a paper presented on 30 April at the International Conference on Learning Representations in Vancouver, Canada, Rahimi and his collaborators document examples of what they see as the alchemy problem and offer prescriptions for bolstering AI's rigor."

"The issue is distinct from AI's reproducibility problem... It also differs from the "black box" or "interpretability" problem in machine learning"

"Without deep understanding of the basic tools needed to build and train new algorithms, he says, researchers creating AIs resort to hearsay, like medieval alchemists."
ML  machine-learning  AI  ScienceMag 
may 2018 by pierredv
Artificial intelligence faces reproducibility crisis | Science
The booming field of artificial intelligence (AI) is grappling with a replication crisis, much like the ones that have afflicted psychology, medicine, and other fields over the past decade. Just because algorithms are based on code doesn't mean experiments are easily replicated. Far from it. Unpublished codes and a sensitivity to training conditions have made it difficult for AI researchers to reproduce many key results. That is leading to a new conscientiousness about research methods and publication protocols. Last week, at a meeting of the Association for the Advancement of Artificial Intelligence in New Orleans, Louisiana, reproducibility was on the agenda, with some teams diagnosing the problem—and one laying out tools to mitigate it."

"Odd Erik Gundersen, a computer scientist at the Norwegian University
of Science and Technology in Trondheim, reported the results of a survey of 400 algorithms presented in papers at two top AI conferences in the past few years. He found that only 6% of the presenters shared the algorithm’s code. Only a third shared the data they tested their algorithms on, and just half shared “pseudocode”—a limited summary of an algorithm."
ScienceMag  reproducibility  replication  AI 
february 2018 by pierredv
On the Origin of Ecological Structure -- Stokstad 326 (5949): 33 -- Science
Abstract: Ecologists have wrestled with understanding what dictates the kinds and proportions of organisms in communities ranging from meadows to montane forests. Competition, predation, disturbance, and other factors have a heavy hand, and new research is showing the influential role of evolution as well. But there is still no consensus on the relative importance of the various forces. Darwin and many later ecologists emphasized competition among species, but proponents of a controversial theory of biodiversity that assumes competition has no impact argue that immigration and other random demographic events can account for much of the apparent makeup of communities. As a result, ecologists have a long way to go to come up with formulas that predict how communities might arise and change. Yet the ability to make predictions is important for the restoration and management of ecosystems impacted by invasive species or climate change.
ecology  evolution  ScienceMag  complexity  toget 
october 2009 by pierredv
Assessing the Impact of Science Funding -- Lane 324 (5932): 1273 -- Science
Policy Forum piece by Julia Lane, economist at NSF; also covered in podcast 5 June 2009
"Science supporters were rightly excited by the passage of the American Reinvestment and Recovery Act (ARRA, i.e., the stimulus package). Headlines in Science (1) and Nature (2) rejoiced at the new value placed on science as a basis for economic growth and associated job creation. Indeed, federal investment was at least partly based on a belief that the result would be more competitive firms and more, and better, jobs—and soon! (3). That belief was bolstered by advocacy groups: For example, a report by the Information Technology and Innovation Foundation (ITIF) estimated that an additional $20 billion investment in research in the stimulus package would create 402,000 American jobs for 1 year."
science  policy  NSF  ScienceMag 
june 2009 by pierredv
COMPUTER SCIENCE: Beyond the Data Deluge -- Bell et al. 323 (5919): 1297 -- Science
"The demands of data-intensive science represent a challenge for diverse scientific communities. "
See also podcast
computing  ScienceMag 
march 2009 by pierredv

Copy this bookmark:

to read