Example: EMPA and Coronado's Musket Balls - Part 2
Above: My data illustrate that even musket balls recovered from a single Coronado battlefield have significant compositional variations -- each color represents an individual musket ball sample, each little cube represents an individual analysis point on the sample, and 20 points were analyzed on each musket ball sample.
I should've known that, if any post here could cause controversy, it would be Can EMPA Identify Coronado's Musket Balls?. Why is that? Like I mentioned in my earlier post, finding the site of Chichilticale is some sort of Holy Grail for Coronado enthusiasts. Some of them are very passionate -- even fanatical -- about it. Some argue vehemently that Chichilticale is Site A while others claim that it is absolutely must be Site B. There are, however, some who critically weigh textual and archaeological evidence and are more objective.
I was unaware of this world of Coronado enthusiasts when I started this exploratory study. I was naive, and I didn't expect that my data would be interpreted by some to mean that Kuykendall is certainly the site of Chichilticale while others would use the same data to argue that Kuykendall is absolutely not Chichilticale. My results simply were not conclusive. When I started seeing my name popping up on Chichilticale webpages, I became concerned that some details and caveats about my work were missing from the discussion. I decided the easiest way to do this would be informally putting my description and interpretation of the analyses on this blog. I have learned, though, that further clarifications are needed and that I must better support my interpretations.
As you've probably guessed by now, my initial post did evoke comments and criticism from someone. I've given my responses to the person, but I realized that, if one reader had such impressions about the exploratory study and my blog, so too could others. Accordingly, I want to share this individual's comments (anonymously, of course) and my responses. I'll also share relevant data plots that support my interpretations.
Let's start with the first comment I received via email...
Reader Comment #1:
[What is] the difference between "non inconsistent with" and "consistent with"[?]
This question relates to the following paragraph from my original post:
What could be said at the end? Well, I agreed with the Coronado researchers that the musket balls from the place suspected to be Chichilticale are not inconsistent with musket balls from known Coronado sites. Yes, that is not saying too much: they're not inconsistent. If I was feeling particularly generous, I might even say that musket balls unearthed at the site are consistent with musket balls from known Coronado sites.
This is a good question because the answer is subtle -- too subtle. I realize that I should've been l more clear in my choice of words. The difference between "not inconsistent with" and "consistent with" is better understood using an intensifier. I won't use the word "significantly" because that carries a specific statistical meaning. Instead, I will use "markedly" to clarify my meaning. Consider the following statements:
Statement #1: Samples A & B are not markedly inconsistent with Samples C & D.
Statement #2: Samples A & B are markedly consistent with Samples C & D.
Statement #1 implies that there are no great differences between the samples. They're somewhat similar, but by no means are they identical or similar to the point of excluding other samples. In the case of the musket balls, it means they're all mostly lead with some trace-element impurities in common. On the other hand, Statement #2 implies that these samples are very similar, possibly to the point of making some sort of meaningful argument for their relation to each other. In the case of the musket balls, it could mean that they all have the same set of impurities in common at nearly the same concentrations. Statement #2 implies a greater similarity than #1.
I'm not confident enough in this exploratory study (i.e., a small number of samples, a lack of control samples, etc.) to state that the suspected Coronado musket balls are "markedly consistent with" the known ones. My phrase "If I was feeling particularly generous" was a joking way to say "If I were to lower my level of confidence."
Also keep in mind that I settled on the wording "not inconsistent with" on the phone with an individual who wanted to state that these musket balls were quite similar and consistent. I had to think of a phrase off the top of my head that suggested a similarity but sounded inconclusive and less than positive.
Therefore, to avoid further obfuscation, allow me reword my one-line assessment of the results: Those musket balls recovered from Kuykendall, a location suspected to be Chichilticale, are not markedly inconsistent with musket balls from known Coronado campsites and battlefields. There are currently insufficient data to state that the musket balls from the suspected Chichilticale are markedly consistent with those from known sites.
That's my assessment, bringing us to comments posted on a website...
Reader Comment #2:
Unfortunately the interpretation of lead shot by electron microprobe analysis conducted by a graduate student technician at a Minnesota lab is questionable... [T]here are numerous reasons to question the interpretation of the results and, therefore, the analysis of the samples is brought into question as well. The most obvious are that the technician... is clearly letting his personal feelings enter into his assessment rather than being objective.
I'm still trying to figure out that last statement -- I've come up with three possibilities:
(1) The individual believes that I do not wish the suspected site, Kuykendall, to be the true site of Chichilticale. To be perfectly blunt, I don't care. This isn't my region or time period, so I don't have a stake in this. I can't even pronounce Kuykendall or Chichilticale. Accordingly, I'm neither biased for nor against it.
(2) That statement refers to a few observations about the macabre subject matter. The fact is that the conquistadors' muskets were used as weapons of oppression and subjugation. I found it rather grim to think about how the samples were used. Most people, when visiting an old battlefield, regard it solemnly. The thoughts I expressed in the original post were basically an extension of that respect. If someone wants to fault me for that, fine. The notion, though, that this changed how I did my analyses or interpreted my data is preposterous.
(3) That statement relates to the fact that I provided my assessments and interpretations of the data, not just the data themselves. This meaning fits this individual's other comments, but first...
Reader Comment #3:
[He] did not understand that some of the samples were in fact control samples from actual known Coronado sites and battle fields. . .
Of course I knew that. Consider these excerpts from my original post:
Some of the researchers hope that musket balls from known Coronado camp sites can be utilized to identify suspected ones. In 2005, I was contacted by a group of such researchers interested in this, and I agreed to help... I did analyses of musket balls from known and suspected Coronado camp sites... They had musket balls from known Coronado campsites and hoped that there may be a recognizable signature... yadda, yadda, yadda...
My statement about no "control samples" refers to the fact that there were no musket balls known not to have been used by the Coronado expedition -- all of them were actual or suspected Coronado samples. There were no musket balls known to have come from other times and/or sources for comparison. I wrote:
The problem with this project is that I do not know what fraction of musket balls are consistent with those found at Coronado campsites. Are 10% of musket balls consistent with them? Or are 90% consistent with them? I wasn't provided with even one musket ball from a site not associated with Coronado's expedition. I had no samples from Portuguese explorers, Hernan Cortes, De Soto, Lewis and Clark, fur trappers, or anyone else. Without the proper stratigraphic information or dating, even the musket balls found (using metal detectors?) at known Coronado sites shouldn't be immediately assumed to be contemporaneous with Coronado. This boils down to some musket balls unearthed at one place are not terribly different from musket balls found at a few other places. Would these same musket balls be consistent with George Washington's ammunition? Perhaps.
My point is: Would, for instance, a Civil War musket ball have a consistent composition too? If it did, that'd certainly diminish the importance of any similarities between the known and suspected Coronado samples. But we can't say anything about that because no musket balls from the Civil War were available. It seems likely that the musket balls used by Cortes and all of the other conquistadors, Portuguese explorers, trappers and traders, settlers, Confederate soldiers, and Mexican revolutionaries could have similar compositions. It is impossible for us to rule out any of these other potential sources until one establishes compositional "fingerprints" for each of them. Any similarities are totally pointless unless one tests musket balls from other times and/or sources and can show that any similarities really are unique to Coronado musket balls, not just, say, all musket balls before 1800.
Reader Comment #4:
Moreover, rather than using established laboratory standards to describe what he found, he has imposed his subjective viewpoint on the data in a way that is not replicable, which makes it difficult to interpret. . .
Clearly this individual missed my intent with this page. I would've thought the "balls" joke at the very start of my post would clue most folks into the fact that this isn't my formal report or a paper that I am planning to submit to American Antiquity. As the research fellow in charge of the lab, I routinely put together formal reports for companies that make everything from implantable medical devices to aerospace steels. That is not what this site is. This is a blog where I informally discuss some of the archaeological samples I've investigated.
I write these posts about how electron microprobe analysis is useful in archaeology to teach about the technique and illustrate some of the "behind-the-scene" details that go into analyses. Analysts must adopt an appropriate analytical strategy and make choices about conditions, standards, and so on. Putting a sample in the electron microprobe and analyzing it isn't as straightforward as putting a frozen dinner in the microwave and heating it. Furthermore, although electron microprobe analysis is a versatile technique, it isn't the best tool for every sample or project. This website is where I can demonstrate the kinds of archaeological samples and questions to which electron microprobe analysis is well suited. I can also discuss the issues I encountered with the samples, and in the case of the musket balls, those issues included how I initially misidentified inclusions, how I realized there was a problem with that identification, and how I realized that the musket balls posed unique sample preparation challenges. Such details are usually left out of publications because they are considered irrelevant (or even embarrassing in the case of errors). I think that sharing such information illustrates some of the "nitty gritty" involved in analyses.
I get it, though... I understand what this individual wanted was something like the data in my post about Quantitative Chemical Analysis of Ancient Glass. Maybe you, dear reader, do too. Well, I will share similar data plots to support assessments that I made in my original post. This permits the group of Coronado researchers to share my raw data however they'd like, but I can still explain my interpretations here. I should point out, though, (and I will explain more later) that I consider these data initial, not fully publishable. Therefore, I'm not going to post the data for hundreds of analyses or show statistical tests because I do not want people focusing too much on the raw data. Instead, it is the relationships among the data which are important, not the individual numbers. To highlight on such relationships, I'll share three-axis graphs, which are more practical for discussing here.
That said, let's examine a few data plots based on my analyses...
For three locations, I had multiple fragments removed from the same musket ball, and I tested each of those pieces individually so that I could evaluate the variation within each musket ball.
Example #1 - Two subsamples from one musket ball
The data above represent the analyses of two different subsamples removed from a single musket ball -- each color represents a different subsample from the same musket ball. There are 20 analysis points per subsample. One will notice that the red and blue data points largely overlap. In fact, the mean concentrations for the two subsamples fall within the standards deviations. If we did the more sophisticated statistical analyses, we would find that the two data sets are not significantly different. This is an encouraging result. It would likely be problematic if subsamples from a single musket ball have different compositions. At best, large intra-sample variation may require larger sample sizes and/or more analysis points. At worst, it could sink this line of investigation.
Example #2 - Four subsamples from one musket ball
The data above represent the analyses of four different subsamples removed from a single musket ball. Each color represents a different subsample from one musket ball; there are 20 analyses per subsample. These data sets also largely overlap, and their mean concentrations fall within the standards deviations. Again we have subsamples that aren't significantly different. This is another encouraging result for this study.
Example #3 - Two subsamples from one musket ball
Here we see that this musket ball is less homogeneous. The two data sets have large spreads, but the trend is that silver, for example, has a higher concentration in the red subsample than in the blue one. This suggests the lead is somewhat heterogeneous, which isn't terribly surprising. It may be that this musket ball wasn't homogeneous when originally cast or that post-depositional conditions at this site caused some alteration.
Let's now look at the eight musket balls, all of them recovered from a single Coronado battlefield, that I was told "are interpreted to be from Coronado firearms" (I don't know whose interpretation that is):
Graphs: Eight musket balls from one known Coronado site
Interesting, yes? The different colors represent different musket balls. Each musket ball is separated from the others based upon these measured elements. The light-blue musket ball, for example, has more silver than the others. The green one has more antimony. The orange musket ball has the most sulfur. The rest of them -- dark blue, red, black, brown, and purple -- are also distinguished by their amounts of antimony, silver, sulfur, arsenic, and tin. Each musket ball has a different composition than the others. This supports what I wrote earlier:
One problem, though, is that I'd classify musket balls as "expedient tools." No one spends a great amount of time making a really nice musket ball. They are made by pouring molten lead into a mold, and after they're used once, musket balls are often lost and not reused. Musket balls don't even have to fit perfectly into the barrel -- they were usually wrapped in linen or paper in order to achieve a tight fit. More importantly, their composition really isn't that critical. A musket ball that is 99% lead is no more or less dangerous than one that is 97% lead. That's the reason modern bullet manufacturers use recycled lead from, for instance, old car batteries -- minor amounts of calcium or strontium don't, in practice, affect a bullet's performance. This is especially true for muskets, which, in Coronado's period, weren't accurate beyond 40 or 50 meters. We shouldn't expect that the musket-ball maker controlled their compositions closely. Each batch of musket balls could be extremely different.
Another possible complication would include the shear number and variety of musket balls carried by the Coronado expedition. As I mentioned above, there were over 300 Spaniards on the expedition. I'm guessing that there could easily have been over dozens, even hundreds, of muskets and pistols carried with them. Their guns may have had different calibers -- for instance, the British "Brown Bess" shot 3/4-inch ammunition. The expedition members might have had to haul their own ammunition, might have had a shared reserve, or even might have been responsible for making their own ammunition. Even a common stash may have included musket balls from multiple makers and/or batches. Would they have brought all their ammunition with them? Or would they have expended the musket balls brought with them and had to make more ammunition along the way? There could've easily been great variation in musket balls on the Coronado expedition. . .
We've established that these musket ball recovered at a single Coronado battlefield are all different, some more than others. It is possible that all of these musket balls were used by members of the Coronado expedition, but it might be that only a fraction actually date to the Coronado expedition. I commented earlier:
Without the proper stratigraphic information or dating, even the musket balls found (using metal detectors?) at known Coronado sites shouldn't be immediately assumed to be contemporaneous with Coronado.
The green musket ball, for instance, has the highest amount of antimony. As I noted previously, antimony has been used in shot since the 1800s, but did the Spanish use it in the sixteenth century? I have found no evidence they did. Antimony, though, can occur in lead ores at different concentrations, and different amounts of antimony could be left within the metallic lead smelted from such ores. Based on this one musket ball, it is not possible to know if antimony was deliberately added by a recent maker or was just a remnant impurity from the ore.
Arsenic has also been used to harden lead, a practice dating as far back as the Bronze Age, but arsenic isn't terribly useful in separating these eight musket balls. Silver, sulfur, and tin are more important, and all three elements often occur in lead ores. Therefore, the compositional variations of these musket balls suggest that all of them could have been made from different lead batches or a smaller number of heterogeneous batches.
Either way, this indicates that there might be no one "Coronado" signature or fingerprint.
Now let's examine all of the data plotted together -- those battlefield samples discussed above are all brown in these plots, and the other colors represent the other five known and suspected Coronado sites:
Graphs: All sixteen samples analyzed from six locations
What can be said based on these data? A main observation is that there is greater variation within that single known Coronado battlefield than there is among all of the other sites. The musket balls from the other locations fall near six from the known Coronado battlefield. Only the "green" location/sample overlaps to any notable degree with those six battlefield musket balls. The "blue" one, for instance, has less sulfur and tin.
What interpretations did I make based on these data?
(1) For the most part, subsamples taken from the same musket ball have the same mean composition when at least twenty measurements are taken on each. Only one musket ball seems to be slightly heterogeneous -- this may be a result of either its manufacture or some post-depositional alteration. Establishing that these musket balls are mostly homogeneous is encouraging -- this line of investigation might be possible.
(2) There is only one site for which there is more than one or two musket ball samples. That is the known Coronado battlefield. Each of the eight musket balls from the battlefield are chemically distinct. This suggests the expedition's ammunition had a variety of compositions -- there might not be a single "Coronado" signature. Given the number of expedition members and the "expedient" nature of musket balls, it isn't surprising.
(3) When the musket balls from other sites are graphed with those from the Coronado battlefield, there is a possible problem. The intra-site variation is greater than the inter-site variation. For sourcing geologic materials, it is a study-killer when the intra-source variation is greater than the inter-source variation. I don't think there is sufficient data to assert this project is doomed because of such variation, but caution is warranted.
(4) There are still no data for musket balls from other times and/or sources. Where would musket balls from Hernan Cortes and all the other conquistadors, Portuguese explorers, trappers and traders, European settlers, Confederate soldiers, or Mexican revolutionaries fall on the plots? A central tenant of sourcing geologic materials is that all likely sources must be represented in the database, otherwise the comparison is incomplete.
(5) One of the musket balls from the Coronado battlefield appears to be a good fit to one musket ball recovered at a suspected Coronado camp site. The significance of this remains unknown. It might be evidence that the location is indeed a camp for the Coronado expedition, or it might be a random similarity. This cannot be addressed unless (a) more work is done to establish the amount of variation in musket balls at individual sites and (b) analyses of musket balls from other times and sources are added to the compositional comparisons.
And now the last of the on-line comments...
Reader Comment #5:
[T]he tests [should] be rerun by a more reputable lab and a more experienced technician to ensure that the actual analysis is solid and consistent with accepted professional standards. . .
This comment does allow me to bring up an important point here: A different lab or a different analyst or a different analytical technique might generate somewhat different data. Why?
For starters, I consider the data to be initial, not fully publishable. This was an exploratory study. I'd never analyzed musket balls or any other lead-based artifact before, and I didn't have good information about what trace elements to expect. Therefore I chose an analytical strategy that would first determine what elements were present and at about what concentrations. Then I could set up quantitative measurements for any elements I detected. Such quantitative measurements require the analyst to make choices about operating conditions, calibration, standards, measurement time, selection of points to analyze, etc. These choices will affect the accuracy and precision of the results. It also is commonplace, when analyzing a new kind of sample for the first time, to have to experiment with different conditions, calibration, standards, and so forth to achieve optimal results. This was the case with the musket balls, and I'd make even further tweaks to the settings if I had time to do these analyses again.
For these reasons, I'll readily agree an electron microprobe user who is "more experienced" with lead artifacts would have me at an off-the-bat advantage. That individual would have had the opportunity to choose initial conditions and procedures, test and evaluate them, refine them, test again, and so on to collect optimal data. I've done this process numerous times for native copper and obsidian, for instance, to get the best possible data.
Above: This recent advertisement from Thermo Scientific (see the December 2007 issue of Microscopy and Analysis) would have people believe that X-ray microanalysis is this straightforward. The text of the ad starts: "With the Thermo Scientific NORAN SYSTEM SIX X-ray microanalysis system, all it takes is a single button press to go from samples to answers in an instant." Sorry, Thermo, it just doesn't (and shouldn't) work that way.
I'm satisfied, though, the precision was good enough, and the data are internally consistent because all the samples were analyzed using identical conditions. Therefore, the plots and the statistical tests should be valid. However, the accuracy isn't sufficient for these numbers to simply be copied into a database and then directly compared with data collected elsewhere. Additional work would be necessary to acquire more accurate results that could be added to a database and published. For instance, accuracy could be improved by adjusting several spectrometer settings. The results could also be improved by finding a way to put a better polish on the musket ball samples without introducing contamination -- something like an ion-beam slicer might be necessary. I didn't bother with these steps because the initial data were sufficiently precise for the purposes of this exploratory study.
Back to the comment in question: A different lab and analyst could easily produce different data but not necessarily because the lab has a "better reputation" and the analyst is "more experienced." Instead, it could be due to choices made by that analyst, different instrument settings, etc. I've also never claimed that the data have the best possible accuracy -- these data are simply "good enough" for this exploratory study. Geologists who use my lab could spend months studying a single sample (and some do), but most don't because such detail isn't necessary to address their research question. Analysts decide what data are "good enough." There are "accepted professional standards" for publishable data (e.g., good totals, stoichiometry adds up, etc), but such standards are not needed to answer every question (e.g., what elements are present and at what relative concentrations?).
Bottom line:
These quantitative data are initial but have enough precision to make some observations about the analyzed musket balls and the prospects for identifying a "Coronado" chemical signature based on trace elements. In my opinion, this exploratory study has detrimental flaws: too few samples, greater intra-site variation than inter-site variation, too little information about the manufacture and use of musket balls by the expedition or the conquistadors in general, and no analyses of musket balls from times and/or sources other than known or suspected Coronado sites. Consequently, I cannot claim anything conclusive. If someone states otherwise, that is their own interpretation of the data. With a lot more work, though, it is possible that this line of investigation could succeed.
* * *
Update - 6 May 2008:
I think I have figured out why the reader complained that. . .
. . . rather than using established laboratory standards to describe what he found, he has imposed his subjective viewpoint on the data in a way that is not replicable, which makes it difficult to interpret. . .
I finally got my hands on the paper published last year in the New Mexico Historical Review (Brasher, Nugent. 2007. The Chichilticale Camp of Francisco Vázquez de Coronado: The Search for the Red House. New Mexico Historical Review, vol. 82, no. 4, pp. 433-468). On pages 461 and 462, the paper states:
I arranged for a metallurgic analysis of the lead balls from Zuni and the one from Camp 23 June. Ellery E. Frahm conducted this examination at the University of Minnesota Electron Microprobe Laboratory in Minneapolis. Frahm provided the data that I used to generate graphics useful for non-parametric comparisons of the five lead-ball compositions.83 The lead ball from Camp 23 June (0.535 caliber) and Hawikku FS138 (0.48 caliber) correlate on individually measured metallurgic composition. On another front, I consulted flintlock rifle manufacturer G.L. Jones. He explained that lead balls are almost never pure lead since ball-makers throw about anything into the molten brew to add volume and that a handful of sand is a common additive. Jones described ridges and sprues on lead balls and related these features to the method of manufacturing and the amount of wear.
Considering the counsel of Jones and the data from Frahm, I interpreted that the Camp 23 June lead ball and the Hawikku FS158 lead ball correlate in many respects. The two balls are composed of the same relative amount of metals, the only difference being the quantity of sand tossed into the molten lead. Neither ball is store bought. Both balls were made in the field, used before being carried very far in a bag, and were made by ball-makers who needed to add some volume to their brew, likely as a result of lead shortage. The practice of making items in the field using expedient materials because of shortages fits in a setting such as the Coronado Expedition and offers encouragement the Camp 23 June will eventually produce artifacts diagnostic of the expedition.
83. Ellery E. Frahm, "Report on Lead Musket Balls" (unpublished paper, University of Minnesota Electron Microprobe Laboratory, Twin Cities, 2005)
The critical reader probably thought that my blog entry was the "Report on Lead Musket Balls" cited in the paper by Brasher. I never wrote a "Report on Lead Musket Balls," though. I sent Brasher one or two Excel spreadsheets of quantitative data, JPGs of electron images, and two email messages. We also talked on the phone. I didn't write a formal report, though, titled "Report on Lead Musket Balls" or anything like that.
A proper citation, therefore, would be: "Ellery Frahm, personal communication, telephone calls and emails."
My email messages did provide some of the interpretations I've shared here. For instance:
I finally had a chance to analyze the five musket ball samples you sent me. . . There is no clear pattern we can derive from even the minor elements (much less trace) in just these five samples. . . FS139 has a much higher Ag content than the others, even samples from the same site.
I suspect that these differences in composition, even from samples from the same site, represent different batches and the "expedient" nature of musket balls -- there is no need to be picky when one makes musket balls since some variation in composition will not affect its performance (i.e., it will still kill). I'm sure that there was a lot of recycling and reuse too. It is frustrating from a sourcing/authentication point of view, but it is interesting from a technology viewpoint.
It might be that, given a larger collection of samples, compositional ranges could be delineated and probable sources assigned to a assemblage of samples from a site. . .
At least it is explicit in the paper that the interpretations are those of Brasher. Now that I've shared the data plots on which I based my interpretations, others are free to inspect them and decide if they agree.
4/25/08
Electron Microprobe Analysis in Archaeology
Electron microprobe analysis (EMPA), also known as electron probe microanalysis (EPMA), is an analytical technique that combines scanning electron microscopy (SEM) and compositional analysis using x-ray spectrometry. The ability to determine structure and chemistry of samples makes EMPA very versatile. This is a dominant analytical technique in geology, but it is not as commonly used in archaeology despite similar materials in studied both fields. Here I will post about topics in EMPA, artifacts I have analyzed, archaeological studies that use EMPA, etc. If there is a topic you'd like to see posted here, please let me know.
Ellery Frahm
Doctoral Candidate, Archaeology
Research Fellow, Geology & Geophysics
University of Minnesota - Twin Cities
