Point-to-Polygon transformation to enhance legacy data

IntroductionIn the data-intensive field of Computer Vision, especially applied to marine science, we have two options for collecting image data to train our models. We can either gather new data and annotate it from scratch or use annotation data from repositories, i.e. legacy data. The former option requires a lot of extra time and effort from experts, especially if they need to draw precise polygons around the object of interest. The latter depends on the quality of the existing annotation data, which may not have been created with Computer Vision in mind. This data, often consisting of basic point annotations, could potentially be useful for AI tasks, but it must be transformed to at least a polygon or a bounding box outlining the object. The availability of training images with valid annotations describing the extension of the shape of an object is crucial for developing advanced algorithms and models. This work introduces a new method of enhancing low-level annotation data, in particular point annotations, to make it usable for state-of-the-art Computer Vision tasks.MethodsBy repurposing the Segment Anything Model (SAM) from an interactive tool to an automatic conversion-based approach, we developed a method that transforms point annotations into machine-predicted polygons. We demonstrate its effectiveness by applying it to three different datasets, one for marine infrastructure and two for marine biology. The first consists of 384 point annotations, the second of 523, the third of 117.ResultsUsing the heuristics proposed in this paper on the first dataset, our method generates one effective mask for all starting point annotations, achieving a median IoU of 87.2%. On the second, our method generates effective mask annotations for 98.3% of the point annotations, achieving a median IoU of nearly 65%. This is an improvement on the baseline SAM results, where 18.4% of the annotations are successfully converted, with a median IoU of 50.4%. On the third dataset, our method converted 95% of point annotations compared to base SAM’s 79.1%, with comparable IoU.DiscussionWe introduce an efficient method, the Point-to-Polygon conversion, which can significantly accelerate the process of modernization of legacy datasets and simplifies the creation of new datasets with precise polygon annotations. The time saved by using this method to convert all point annotations in BIIGLE would be about 14,000 hours of manual work.