Trace-Element Fingerprinting Doesn't Work...
Sunday, November 18, 2007
 
... at least, not for modern bullets using the FBI's techniques. At least that is the conclusion of metallurgists who reviewed the FBI's analytical and statistical procedures for what is called 'comparative bullet-lead analysis.' The idea is that FBI scientists analyze bullets from a crime scene and ones owned by a suspect and determine if the bullets match chemically. It is much like sourcing archaeological material. In the case of sourcing obsidian, the 'source' is, at least theoretically, rather straightforward: a particular volcano or flow. One must analyze samples from every volcanic flow in the region, establish a unique chemical pattern for each, and then match an artifact's composition to one of those volcanic flows using some type of data analysis. If chemical 'fingerprints' cannot be identified or multiple flows have identical trace-element patterns, sourcing cannot work. If obsidian from eastern Anatolian volcanos has the same 'trace-element fingerprint' as that from western Anatolian volcanos, any source assignments are useless, just as if many people shared the same set of fingerprints. This is basically the problem with the FBI's comparative bullet-lead analysis. FBI scientists had assumed that a particular box of bullets has a unique trace-element fingerprint and that, if a bullet recovered from a crime scene had the same pattern of trace-elements, it must have come from that box. In effect, they were doing a sourcing study, and they assumed that individual bullets could be exclusively assigned to particular boxes. The box was basically what they considered to be the 'source' for each bullet. One of the main conditions for sourcing is that inter-source variation is greater than intra-source variation. In other words, there must be greater trace-element variation between sources than within sources. Without this condition being fulfilled, it cannot work. And guess what researchers found when they put the FBI's assumptions to the test... Excerpts from the Washington Post: FBI's Forensic Test Full of Holes By John Solomon Hundreds of defendants sitting in prisons nationwide have been convicted with the help of an FBI forensic tool that was discarded more than two years ago, but the FBI lab has yet to take steps to alert the affected defendants or courts, even as the window for appealing convictions is closing, a joint investigation by the Washington Post and "60 Minutes" has found. The science, known as comparative bullet-lead analysis, was first used after President John F. Kennedy's assassination in 1963. The technique used chemistry to link crime-scene bullets to ones possessed by suspects on the theory that each batch of lead had a unique elemental makeup. In 2004, however, the nation's most prestigious scientific body concluded that variations in the manufacturing process rendered the FBI's testimony about the science "unreliable and potentially misleading". Specifically, the National Academy of Sciences said that decades of FBI statements to jurors linking a particular bullet to those found in a suspect's gun or cartridge box were so overstated that such testimony should be considered "misleading under federal rules of evidence". . . . . . the experts had made false matches based on a faulty statistical analysis of the elements contained in different lead samples, documents show. . . . . . the bureau told defense lawyers in a general letter dated Sept. 1, 2005, that although it was ending the technique, it "still firmly supports the scientific foundation of bullet lead analysis". . . Even the harshest critics concede that the FBI correctly measured the chemical elements of lead bullets. But the science academy found that the lab used faulty statistical calculations to declare that bullets matched even when the measurements differed slightly. FBI witnesses also overstated the significance of the matches. . . "Frankly, the letters that they sent them, you know, were minimizing the significance of the error in the first place," said defense lawyer Barry Scheck, whose nonprofit Innocence Project has helped free more than 200 wrongly convicted people. The letters said that "our science wasn't really inaccurate. Our interpretation was wrong. But the interpretation is everything". . . The FBI's bullet-lead analysis was created more than four decades ago to link suspects to crimes in cases in which bullets had fragmented to the point where traditional firearms tracing - based on gun-barrel groove markings - would not work. So FBI scientists used chemistry to try to find matches. Their assumption was that bullets made from the same batch of lead would have the same chemical composition. U.S. bullet-makers recycle lead from car batteries and melt it down in huge amounts, and it was believed that each batch would produce bullets sharing the same trace elements. . . In the early days, bullet fragments were subjected to neutron beams that would allow scientists to measure the presence and amounts of at least three chemical elements: antimony, arsenic and copper. If two bullets had similar measurements of those three elements - the FBI allowed for a small margin of error - they were declared a match. In 1996, the bureau switched to a new method called "inductively coupled plasma optical emission spectroscopy," in which scientists identified and measured seven trace elements in the bullets, adding the elements bismuth, cadmium, tin and silver. The goal was to increase the precision of the tests. But at the same time that it was measuring more elements, the FBI doubled the margin of error for declaring matches. . . The bureau conducted a study in 1991 that called bullet-lead analysis a "useful forensic tool" that produced "accurate" and "reproducible" matches. The study, however, raised two concerns. First, it found that bullets packaged 15 months apart - a span that assumed separate batches of lead - had the exact composition, potentially undercutting the theory that each batch was unique. Second, it found that bullets in a single box often had several different lead compositions. That finding, it cautioned, should have "significant impact on interpretation of results in forensic cases". . . They found that bullets made from the same batch did not always match, because subtle chemical changes occurred throughout the manufacturing process. Tobin bought bullets at several stores in Alaska and found that a large number of bullets with the same composition and manufacturing date were often sold in the same community, suggesting that it was wrong to assume that a bullet match could be narrowed to one suspect. "It hadn't been based at all on science but, rather, had been based on subjective belief," Tobin said in an interview. "Courts, and even practitioners, had been seduced by the sophistication of the analytical instrumentation for over three decades". . . To read the full article (which I highly recommend), please visit the Washington Post website: http://www.washingtonpost.com/wp-dyn/content/article/2007/11/17/AR2007111701681.htmlhttp://www.washingtonpost.com/wp-dyn/content/article/2007/11/17/AR2007111701681.htmlshapeimage_2_link_0
 
Main Page & CV Obsidian Sourcing Electron Microprobe Analysis in Archaeology Petroarchaeology Geo/Arch/Sci Blog Geoarchaeology Funding Everything Else../../Ellery%20Frahm/Ellery%20Frahm.html../../Obsidian/Obsidian%20Sourcing.html../../Microprobe/Electron%20Microprobe%20Analysis%20in%20Archaeology/Electron%20Microprobe%20Analysis%20in%20Archaeology.html../../Petroarchaeology/Petroarchaeology.html../../Geoarch/Geoarchaeology%20and%20Archaeological%20Geology/Geoarchaeology%20and%20Archaeological%20Geology.html../../Everything/Everything_Else/Everything_Else.htmlshapeimage_6_link_0shapeimage_6_link_1shapeimage_6_link_2shapeimage_6_link_3shapeimage_6_link_4shapeimage_6_link_5
GEO/ARCH/SCI BLOG GEOARCHAEOLOGY & ARCHAEOLOGICAL SCIENCE
Blog Author: Ellery Frahm