The sequencing results, published today in the journal Nature, also reveal Neanderthals, early modern humans and a sister group to Neanderthals, Denisovans, met and reproduced in the Late Pleistocene between 12,000 and 126,000 years ago.
Alan Cooper, a professor at the Australian Centre for Ancient DNA at the University of Adelaide, says the study "completely rewrites what we know about human evolutionary history".
"We now have a reasonably definitive picture of the mixing and matching of [hominin] groups through time," he says.
First author of the paper Kay Prufer, of the Max Planck Institute for Evolutionary Anthropology in Germany says the findings are based on DNA extracted from a toe bone found in the Siberian cave where the first Denisovan fossils were discovered in 2008.
The toe bone belonged to a Neanderthal woman who they estimate lived about 50,000 years ago, he says, adding that DNA analysis shows the woman's parents were very closely related.
"We conclude the parents of this Neanderthal individual were either half-siblings who had a mother in common, double first cousins, an uncle and a niece, an aunt and a nephew, a grandfather and a granddaughter, or a grandmother and a grandson," the researchers write.
Reason for extinction?
Prufer says the analysis shows this inbreeding was not a rare event.
"The parents were very closely related, but even if you ignore that [DNA analysis shows] … the past parents of the parents were related," he says.
Prufer says the inbreeding suggests the Neanderthal population was quite small or fragmented and this may have played into their demise.
"Of course if you have a small population size you begin to move into the danger zone [for extinction]," says Prufer.
Cooper, who was not involved in the study, agrees: "If you are breeding with your uncle, your population is on the way out. The fact this group has been doing it for a while suggests it was in decline."
Interbreeding across species
As part of the study, the international team also compared genomes of Neanderthals, Denisovans and modern-day humans.
Previous research has shown that while Neanderthals contributed to the genetic heritage of all modern populations outside Africa, Denisovans contributed exclusively to populations in Southeast Asia and Oceania.
However Prufer says the analysis reveals the picture may be more complex.
Their study shows gene flow from Neanderthals to Denisovans indicating interbreeding between the two groups.
The Denisovan DNA also contained genetic material from an "unknown archaic human that lived a million years ago", says Prufer. They should come and get some DNA from me; I'm an unknown archaic human, though not quite a million years old, sometimes I feel like it.
He says this "unknown archaic human" could be Homo erectus, but further analysis is needed to determine its origins. Could be an oragutan, or a wooly mammoth. Still looking for that missing link, I guess.
Ancient human relative's DNA puzzles scientists
Prufer says by comparing the genome of the various hominin groups, researchers will also be able to pinpoint the "defining changes" in genome that genetically make modern humans.
Their work suggests the proportion of Neanderthal-derived DNA in all people outside Africa is about 1.5 to 2.5 per cent.
Cooper says the findings show evolution works in a "complicated and messy fashion".
"And when you try and reconstruct evolutionary history by looking at modern genetic data you get it completely wrong," he says.
Ah! An honest scientist, willing to admit that almost everything they know is wrong and re-written by almost every significant new discovery.
While the work is "very convincing" Cooper says it is unlikely to be the final version of evolution.
"Five years ago we didn't even know of the existence of the Denisovans," he says.
"It is an incremental process [but] these discoveries are really changing how we think about human evolution."
Cooper says the sequencing of the genomes also opens up the possibility to "identify what bits of Neanderthal and Denisovan DNA survive in us and what they might be doing".