Sharp criticism of controversial ancient-human claims tests eLife’s revamped peer-review model

Archaeologists in South Africa wowed viewers of a Netflix documentary — released last week — with stunning scenes of a cramped cave crammed with bone fossils that, they argue, are the remains of the earliest-known burial by humans or their extinct relatives.

But days earlier, four scientists who peer-reviewed the paper making those claims called the supporting evidence “inadequate”, in an assessment that sits alongside the paper in the open-access journal eLife¹. The studies are a high-profile test of eLife’s new publishing model, in which it no longer formally accepts papers, but instead publishes them alongside peer reviewers’ reports.

Vetted but not endorsed, neither accepted nor rejected, the headline-grabbing research on the quarter-of-a-million-year-old human relative Homo naledi occupies a liminal zone created by the collision of highly publicized science with changing models of publishing and peer review.

“I want to understand how the H. naledi fossils got there. They are very important fossils, and critical to understanding human evolution,” says Jamie Hodgkins, a palaeoarchaeologist at the University of Colorado Denver, who was one of the study’s four reviewers for eLife. However, “there just wasn’t any science in the paper ultimately”.

Lee Berger, a palaeoanthropologist based at the National Geographic Society in Washington DC, who co-led the research, says that his team stand by its research claims. The authors plan to redraft the paper, taking the reviewers’ comments on board.

Cave of bones

The claimed burials are in the Rising Star cave system near Johannesburg, South Africa. In 2015, a team co-led by Berger reported the discovery of some 1,500 bones and teeth from at least 15 individuals belonging to a new hominin species that they named Homo naledi².

Later dating showed that H. naledi lived relatively recently — between 335,000 and 241,000 years ago. This surprised researchers, given that many of its features, including its small brain, were more typically found in much earlier hominins³.

Berger’s team has previously hinted at the possibility that the Rising Star site represented a burial. They fleshed out those claims in a preprint last month¹, reporting further excavations of several individuals from two deep chambers.

The researchers say that differences between the composition of the soil surrounding the remains and that in the rest of the cave are a sign of active digging. Some of the bones, including those comprising the right foot, ankle and lower leg bones of one individual, were in the correct anatomical orientation, or articulation, suggesting that they had decomposed in place, another potential sign of intentional burial.

Being at least 240,000 years old, the H. naledi remains pre-date the oldest-known Homo sapiens burials by at least 100,000 years. Berger and his team argued that such advanced behaviour by a small-brained hominin should force a rethink about the capacities of other ancient human relatives — and of what sets H. sapiens apart from them. A second preprint reported scratches on the walls of Rising Star, which the researchers interpreted as intentional engravings by H. naledi⁴. Convincing evidence of such symbolic behaviour has previously been found only in H. sapiens, and Neanderthals (Homo neanderthalenis), another big-brained hominin.

Open reviews

Berger’s team initially submitted its findings to a leading journal (which he declined to name), but they were ultimately rejected after a review process that dragged on for around six months. “That was a little bit frustrating for us,” says Berger.

The authors had had a good experience publishing the initial descriptions and dating of H. naledi in eLife. So they decided to resubmit the results there under a publishing model that the journal rolled out earlier this year.

Papers submitted to eLife under this model must first be posted as preprints. Editors then decide whether to send them out for peer review (a large proportion of submissions are rejected). Studies that the journal agrees to consider are published online alongside the reviews and an ‘eLife assessment’ summarizing them. Authors can submit a paper for re-review to get a new assessment, or let the first — or any subsequent revision — stand as the version of record.

Berger’s team announced its discoveries at a press conference, coinciding with the release of the preprints on bioRxiv in early June. The team also mentioned that the findings were being reviewed at eLife. “We felt and feel these papers were very, very strong,” Berger says. “They meet and exceed what this community has published for burials of Homo sapiens.”

‘Inadequate science’

Many scientists were deeply sceptical of the evidence presented. The scattered bones bore little resemblance to those of more completely articulated skeletons from other archaeological sites in which intentional burial is clear, critics said. And the researchers did not make a convincing case that the wall scratchings were made by a hominin, and presented no evidence that they date to a period when H. naledi occupied the cave.

“I don’t see an anatomical connection, I don’t see a hole or a pit that has been intentionally dug,” says María Martinón-Torres, a palaeoanthropologist at the Spanish National Research Center for Human Evolution in Burgos, who co-authored an essay critiquing the H. naledi findings the day after their announcement. “These hypotheses have been sold with a very strong media campaign before the evidence was ready to support it.”

The papers’ peer reviews, posted on 12 July, come to much the same conclusion about the scientific evidence. After citing a litany of missing evidence, one reviewer wrote: “The manuscript in its current condition is deemed incomplete and inadequate, and should not be viewed as finalized scholarship.”

Berger says that his team is still taking in the reviews, and they plan to address some — but maybe not all — of the concerns in future versions. “We haven’t published our final paper yet.” He says that the team will stop seeking further review “when we feel that we have come as close to meeting the valid criticisms as we could”.

Hodgkins says that she agreed to review the burial paper because of the site’s importance. But now she’s not sure if the time spent reviewing was worth it — or whether she’ll volunteer to review any revisions. “They simply don’t care what our scientific questions are regarding their work.”

Sven Ouzman, an archaeologist and rock-art specialist at the University of West Australia in Perth, who reviewed the engravings paper for eLife⁵, says that “the possibility has been raised but the proof really isn’t there”. He worries that eLife’s publishing model has created a loophole that allows unsupported studies to stand. “It’s essentially up there and published, and they can say, ‘we have reviewed the reviewer’s comments, and we thank them for it. But we stand by our arguments,’” he says. “That’s sort of cheeky.”

A eLife spokesperson says that the journal is holding conversations with its editors about lessons learned from the H. naledi papers, but no changes to the model have been decided.

eLife senior editor George Perry, who oversaw the assessment of the H. naledi papers, stands by the decision to send them out for review. “We needed input from expert reviewers to be able to assess whether those interpretations are warranted,” says Perry, who is a biological anthropologist at the Pennsylvania State University in State College.

Perry would have preferred that Berger and his colleagues wait for those reviews before publicizing their work. But the discussions over what H. naledi did or didn’t do 250,000 years ago has underscored the value of peer review, he says, and served as a healthy reminder that science is an imperfect process. “Not every result is 100% correct.”