AI Models

Animal Detect uses over a dozen AI models, both open-source and our own specialised models, to give the best result on every image. This page explains how they work together in the Detection API and in the platform.

One ensemble, nothing to select

The models run as an ensemble. Each one is called when an image needs it, based on what is in the image and where it was taken. You get the best combined result without choosing a model, in the API and in the platform alike.

Processing before and after the models

Model outputs are only part of the answer. Every image also passes through several pre- and post-processing steps, including:

  • Pre-processing

    Every image is prepared for each model in several steps before it is classified.

  • Smoothing

    Results are smoothed across related detections, so one odd prediction does not stand alone.

  • Hierarchical classification

    When the evidence for a species is weak, the label moves up the taxonomy to genus or family instead of guessing.

  • Geofencing

    Labels are checked against the species known to live in the country or area of the image.

And many more steps that we keep improving.

Where the models run

Detection API

Every detection endpoint runs the ensemble on each image. Send country to turn on geofencing. Detect endpoint

Platform

The same ensemble runs on your cameras and uploads. Wildlife Monitoring adds our models for sex, age and behaviour, shown as suggestions you can confirm.

Open-source models we use

  • MegaDetector

    Detection

    The MegaDetector family finds the animals, people and vehicles in each image. We benchmark over 20 MegaDetector variations and aim to select the one that works best for each place automatically.

    MegaDetector project · MIT licence · github.com/agentmorris/MegaDetector

  • SpeciesNet

    Global species classification

    SpeciesNet is called as part of our larger ensemble and gives global coverage with over 2,000 species labels. We keep improving how its output is used, and the geofencing logic around it.

    Google · Apache 2.0 licence · github.com/google/cameratrapai

  • DeepFaune

    European species, on demand

    In Europe, DeepFaune is called on demand as part of the ensemble, either to validate other models' results or to help classify an animal. Its output is mapped to the same taxonomy used everywhere in Animal Detect, and a DeepFaune result is never used on its own.

    The DeepFaune initiative, CNRS · model weights CC BY-SA 4.0 · deepfaune.cnrs.fr · Rigoudy et al. 2023

The Animal Detect model zoo

Alongside the open-source models, we train our own: over a dozen models, both CNN and Vision Transformer (ViT) based. Some are specialised for species classification, others for an animal's sex, age and behaviour.

We respect the work done by the teams behind these open-source models and invite everyone to contribute to them.