The wear patterns of your jeans aren’t good forensic evidence, Ars Technica

Mismatched jeans –

Low reliability, high rate of false alarms in denim-matching forensics

( The “barcode “pattern of light and dark points along the seam of a pair of jeans. Is every pair of jeans like no other? According to the testimony of FBI forensic analysts , the patterns seen on denim are reliably unique and can be used to identify a suspect in surveillance footage.

The problem is, this technique has never been subjected to thorough scrutiny, and evidence acquired through it may not be as strong as it has been been claimed to be. A paper published in PNAS this week puts denim-pattern analysis through its paces, finding that it isn’t particularly good at matching up identical pairs of jeans — and may create a number of “false alarm” errors to boot.

Shoddy evidence

For some time, there have been rumblings about the reliability and quality of commonly-used forensic techniques. In 2019, the National Academy of Sciences published a weighty report observing that, apart from nuclear DNA analysis, “no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source. ”

The problems with forensic evidence — including fingerprint, bloodstain, and ballistics analysis — have terrible real-world consequences. According to the National Registry of Exonerations, nearly a quarter of wrongful convictions in the United States for the last (years can be attributed to flawed or misleading forensic evidence.

Computer scientists Sophie Nightingale and Hany Farid wanted to look at one technique in particular: photographic pattern analysis, which matches up the patterns of details on faces, hands, or clothing between suspects and crime-scene photographs. Jeans, for example, have a “barcode” pattern of dark and light splodges along their seams. Denim barcodes

These patterns have been used as central evidence to convict people, but is this kind of analysis reliable? That hasn’t been established. To test it out, Nightingale and Farid went out to buy 111 pairs of jeans from second-hand stores. They laid the jeans out flat on a hard surface, photographed the seams along the legs, and digitally traced the pattern of light and dark points along the seams. To bump up their sample, they had Amazon Turk workers supply images from another pairs, photographed using careful instructions.

Then, the researchers set about quantifying how different the patterns were across different pairs of jeans. Obviously, there’s a lot of randomness at play here — two pairs could be quite similar, just by chance, while another two pairs could be entirely different, also by chance. And most pairs would fall somewhere in the middle, with some degree of similarity. Based on these measurements, Nightingale and Farid worked out the range of similarity between the “barcode” patterns on different pairs of jeans.

The important question, of course, is whether these patterns can be used to determine whether two images show the same pair of jeans . So the researchers took pairs of jeans and took photos of each, using different cameras, in different lighting, and with different draping. What they found was that any given pair of photos could come back with a lot of similarities but could also come back with very different readings on the pattern. The range was broad — as Nightingale and Farid point out, soft fabric photographed in a bunch of different ways is going to have distortions that vary from one image to the next.

False alarms

So, if one pair of jeans can look noticeably different in different photos, is denim-pattern analysis actually a useful forensic technique ? The researchers used their measurements to estimate how often a true match would come up and how often their jeans would throw up a “false alarm” —a score that looked like a match even though the images actually came from two different pairs.

They found that the false alarm rate could be as high as one in a thousand. Given that the FBI has reported using photographic pattern analysis in hundreds of cases each year, that’s a meaningful possibility. The true match rate was also not great, at around to

percent, depending on factors like the length of the seam being analyzed .

This means that the technique of matching up jeans is likely to be pretty hit and miss — not catching actual similarities a lot of the time, and possibly throwing up a high rate of false alarms. And that’s under controlled experimental conditions using high-quality images and jeans laid out nice and flat, not grainy security footage showing jeans being worn. On the other hand, different features like damage, branding, and size could corroborate an analysis to improve the evidence one way or another.

There’s more work needed on whether jeans could be analyzed in a more reliable way using additional features — and also whether other pattern analysis —Like freckles on a face or patterns on other types of clothing — are similarly unreliable. But for now, write Nightingale and Farid, “identification based on denim jeans should be used with extreme caution, if at all.”

PNAS, 1582662423. DOI: . 1365 / pnas.