Segmentation model outputs as prompts to SAM
Some ideas on improving semantic segmentation with SAM
Use some pre-existing models like DeepLab and OneFormer as baseline semantic segmentation models. Get bounding box prompts by using semantic segmentation masks from the baseline models. Use the prompts to get SAM masks, these masks are not labelled well. Now use IoU score between both masks to label the SAM masks, and we end up with labels from the baseline models and masks from SAM. Test this hypothesis on around 100 images per dataset.