DNA will probably gain a better reputation among Mormons now that the Cohen modal haplotype has been found in Colombia, Brazil and Bolivia. (see this article in yesterday's Mormon Times.) But I've never had much faith in the possibility of proving that the people of the Americas descended from the Old World Israelites by DNA testing. The science of DNA continues to evolve, and thus far many theories coexist. There are several possibilities why DNA evidence to prove or disprove the Book of Mormon is not reliable.
Let's take Lehi's ancestry as our first example. We learn in the Book of Mormon that Lehi's biblical ancestors were Joseph and his son Manasseh. If you will recall, Manasseh is one of the ten lost tribes, who disappeared from the Biblical account after the Kingdom of Israel was totally destroyed, enslaved and exiled by ancient Assyria. We are not sure where the descendants of Manasseh ended up, and thus we are not sure where to find their DNA, to match it with Lehi's. Lehi's descendants would certainly have very different DNA than the modern-day Jews, who are descended from Judah, were taken into captivity and intermarried with many different peoples.
Thus far, DNA studies of American Indians have shown that they are related to populations of Asian heritage. But Michael Whiting, assistant professor of integrative biology at BYU believes that genetic drift and the Founder's Effect are two theories that can account for the loss of genetic markers within the "Lamanite" population. These two factors were probably at work over the last 1,600 years since Lehi and his family came to the American continent, he says. Genetic drift produces random changes in the frequency of traits in a population. The changes produced in any one generation by drift and natural selection are very small. But these differences accumulate with each subsequent generation and over time substantial changes occur. The Founder's Effect is a change in the gene pool of a colonizing population because it is founded by a limited number of individuals from a parent population.
Many Latter-day Saints believe there is much to suggest that Lehi and his family were absorbed into a larger group of people already living on the American continent. This is supported by the population increase which is described in the Book of Mormon itself. Scott R. Woodward, executive director of Sorenson Molecular Genealogy Foundation, has explained that when a small group of people intermarry into a larger population, the DNA markers that might identify their descendants could entirely disappear even though their genealogical descendants could number in the millions.
The state of DNA research is such that there are limited conclusions that can be made about human populations. Research teams who identified and sequenced all 20,000-25,000 genes as part of the historic Human Genome Project in 2000 declared that their studies showed that "race" was not a valid scientific concept. The genetic difference between two individuals of the same race can be greater than those between individuals of different races. Much controversy continues to be seen among genetic scientists on this issue. Scholars agree that cultural, ethical, social, and philosophical challenges are raised when DNA is relied upon to resolve questions of history and identity.
For many reasons, DNA studies and the BoM do not fit. There are too many things about the genetics of the Book of Mormon peoples that we just don't know. It remains doubtful that even the sealed portion of the scriptures would contain the things we need to know to make conclusions about racial issues. DNA studies are still in their infancy. While there are some areas in which it can be useful, I do not see Book of Mormon studies to be one of them.