Segmentation is a powerful tool that can isolate one or more animals in images or videos. In videos, it becomes essential for tracking behaviours and movements. Additionally, isolating the animal from its background allows for creative augmentation, mixing and matching different backgrounds with the segmented animal, scaled and positioned in various ways. This approach also helps in generating more training data for our models.
This weekend, I tested out several segmentation models for animals. Initially, I ran into issues with some out-of-the-box solutions – the models either tried to detect the animal on their own or required a single point selection, which often resulted in animals missing parts like half of their body or legs.
To overcome this, I developed a program that allows me to select multiple points on the animal as well as on the background. The result? Much more accurate segmentation! I utilised SAM2 for the segmentation along with some custom Python code to display images and handle the point selection.
Result:
