A critical point of view

Dog's nose

Failure to scale a promising idea is often the result of failing to evaluate the idea critically: interrogate both what’s wrong with the idea, and what could go wrong in expanding it. Let’s spend some time doing just that.

The Office for Science and Society (OSS)  at McGill University in Montreal seeks to bring critical thinking to scientific information. Their mission, ‘Separating Sense from Nonsense”, is timely and welcome.

In September 2020, Jonathan Jarry of OSS reviewed the promise of dog detection of diseases. The article, ‘Barking Up the Wrong Tree: The Trouble with Disease-Sniffing Dogs‘, asserts that although it’s factually accurate that dogs can smell the presence of disease, ‘there are too many obstacles to clear before we can unleash Fido in the laboratory.’

Although the article presents many objections in passing, the main obstacles presented by the article are:

  • Variability in dogs’ ability to detect disease through scent
  • Distractions when the dog is working a a real-life setting (in vivo)
  • Diabetes alert dogs don’t perform statistically better than chance
  • Training dogs is expensive, labor intensive and must be maintained
  • Dogs aren’t as reliable as machines: they can get sick and not be able to work; they can transmit disease to humans; what if a dog bites a human?

Let’s review these in more detail.

Variability in dogs’ ability to detect disease through scent

It’s true that not every dog is a candidate for scent work in a controlled, disciplined environment. Dogs who may have excellent scent perception and recognition may also have behaviors and traits that make them unsuitable for this work. It’s also true that this ability isn’t specific to breeds of dogs. Many scientific studies on dog detection have used multiple breeds of dogs because this ability has more to do with the individual dog than with its breed.

The OSS article doesn’t address the variability of scent detection in an individual dog who has been trained and given the responsibility to detect disease, but this type of reliability is the most essential. If a dog is used to detect a cancer or Covid, we must be able to rely on that dog’s results being consistent over time and with different samples. The scientific studies in this area have calculated the sensitivity and specificity of the results of individual dogs. Research has demonstrated that the results from a trained dog are highly reliable, even more reliable than the ‘gold standard’ lab tests currently being used for many cancers and the  RT-PCR test for Covid (at a fraction of the cost and time required).

What we can conclude from these facts is that scaling the use of dog detection for health won’t be limited to a specific breed, but will be limited by the smaller subset of individual dogs who are well suited for this work. We must learn how to train and maintain this skill with enough dogs to be useful for a society. This is indeed a critical need for scaling the solution beyond its current applications.

Distractions when dogs are working in vivo

The OSS article based this objection on a Dutch study that used a single dog, a two year old beagle. The dog was trained over two months to detect C Difficile infection in stool samples, then was tested for accuracy in samples by themselves (in vitro) and by exposure to patient wards (in vivo), sniffing the air around hospital patients. The research paper does mention in passing that one visit to a pediatric ward was eliminated from the findings due to the kids distracting the dog from doing its work. The paper reported high sensitivity and specificity results for the dog, albeit with the pediatric visit excluded.

Other in vivo studies have demonstrated that in vivo results for dog detection are superior to in vitro, so being able to provide in vivo screening would be critical to the successful use of large-scale applications. Finland used dogs to detect Covid-19 in September 2020, testing about 100 airline travelers per day and 2200 per month. Accuracy was reported to be almost 100% even five days before symptoms appeared.

A trial  in the Metro System of Medellin, which transports 1.5 million passengers daily, used three dogs to smell 550 passengers. This study did find that while the negative predictive value was high, the percentage of false positives increased substantially. The authors concluded:

“However, real-life conditions increased substantially the number of false positives, indicating the necessity of training a threshold for the limit of detection to discriminate environmental odoriferous contamination from infection.”

“Highly sensitive scent-detection of COVID-19 patients in vivo by trained dogs”, Omar Vesga et al.

Real life application will require overcoming the obstacle of distraction of scent. Distraction of human behavior is not so important: for decades, detection dogs and their handlers have been used in vivo at airports to detect drugs and explosives in baggage areas. Part of the training (for human handlers and dogs) requires maintaining a controlled environment despite the unpredictability of humans around them.

However, the distraction of scent is less controllable. The scent trained dog has to be able to discriminate the target scent from others that are not important. For instance, the Dutch research paper noted that one of its research limitations was in training and using the dog only in a hospital environment. Other research papers have noted that this can be problematic since hospitals have a unique set of smells. A dog trained only in this environment may be learning to include the ‘hospital smell’ along with the target smell, making the training useless in a non-hospital environment.

Training criteria must be established that address the issue of scent distraction for in vivo applications.

Diabetes alert dogs don’t perform statistically better than chance

The OSS article points out the well-documented issues with using diabetes alert dogs (also known as DADs). A paper (“Diabetes alert dogs: a narrative critical overview”, Giuseppe Lippi and Mario Plebani, 29 Sept 2018), which wasn’t cited by the OSS article, determined that DADs used not only their sense of smell but also their visual interpretation of human behavior to detect glycemic variations in the humans they assisted.  Lippi and Plebani also note the tactical issues that must be resolved to scale dog scent detection: standardization, certification, and so on.

These data suggest that we shouldn’t rely on dogs to perform visual medical diagnosis. This application isn’t being recommended by those working in cancer and infectious disease detection by dogs.

Training dogs is expensive, labor intensive and must be maintained

This challenge by the OSS paper is about the costs of dog detection. The word ‘expensive’ implies a comparison to some other means. As reported by The New York Times, scent training per dog is $16,000 in addition to the cost of the dog itself, exclusive of the salary and training costs for handlers. This becomes a big number when scaled to widespread use.

Is it expensive compared to the investments in R&D for medical screening? We can perform a comparison by using the costs charged for each type of screening (medical or canine). Estimates from those in dog detection are that a Covid sniff can cost as little as $2 per subject. The provider of the last PCR test I had (in December 2021) invoiced my insurance company $125.

Dogs aren’t as reliable as machines

It’s true that trained dogs aren’t able to work every single day. In addition to the sick days mentioned by the OSS article, one research study using un-spayed dogs found that a dog in heat was too distracted to perform professional scent work. Although spaying and neutering dogs would minimize distractions, any scalable solution must accommodate the desire of some owners to keep their dogs intact in order to produce offspring likely to be well-suited to scent work. It would be necessary to accept some limitations on working days.

It’s also true that anyone who’s working with dogs in scent detection must protect the health and safety of the dogs and trainers, and must screen out dogs who are aggressive. Almost every study I’ve read has addressed the protocols they used to protect all participants – canine and human – in the trials. This would be true for any medical screening procedure, whether dogs are used or not.

What about machines?

Common wisdom is that machines are more reliable than people, whether used to promote self-driving cars or fully automated warehouses.  We tend to think of machines as being easily scaled since they are manufactured. This is true if you consider only the constraints of capital and raw material availability. Less consideration is given to the dependence on humans in the production and use of machines. We could apply the same kind of critique employed by OSS to critique the use of machines:

  • Machines are devised by humans, and human knowledge has huge blind spots. The creators of social media algorithms didn’t foresee how these algorithms could be used for malevolence, and yet they were. Google’s image search software didn’t account for darker-than-white skin tones, leaving out much of humanity.
    • Dogs, by contrast, have an olfactory cognition apparatus that has evolved over millennia and has been tested by evolution, eliminating multitudes of design flaws. Do we think we can do better than this in even one lifetime, much less the investment cycle required of technology R&D?
  • Production and distribution of machines requires not just capital and raw materials but also labor and supply chains.
    • When these break down, will the technology be available where it’s needed?
    • Will spare parts and technical skills required to maintain and repair the machines be available when and where they are needed? If not, this machine-based solution would be unreliable.
  • Medical screening that uses machine technology also requires humans to process samples reliably and to interpret the results.
    • Sometimes humans have bad days, and their mistakes can cost lives.
    • Some humans are more reliable than others in interpreting lab results. This is true despite the fact that the variable humans are being trained by the same organizations.

Dogs sniffing
Two dogs sniffing

The false choice

No one working in dog detection advocates for using dogs exclusively to screen for disease. A machine that can replicate what a trained dog is able to do in scent detection would be terrific since it would facilitate bringing disease detection to more people more quickly.

Those who advocate for scent detection are only considering its use for screening, not diagnostics. Once a positive scent is detected, that subject must undergo diagnostic testing to confirm the presence of disease and gather much more data in order to treat the patient.

Many processes in standards, safety and service delivery must be created and replicated many times in order to provide this solution for widespread use. The fact that these processes aren’t in place doesn’t invalidate this particular screening tool. After development of the Covid home antigen tests, many processes had to be created in quality control, packaging, consumer communication, and distribution. Before those processes were defined, did their absence invalidate the concept?

In early disease detection, more is better. Using detection dogs at scale would make possible the early detection of cancers and infectious diseases. This would save lives.  

Why can’t we have both?

Leave a Reply

Your email address will not be published. Required fields are marked *