Just one thing: I think turning the default green into red when you select a box makes sense, to have a clear difference. But I think the red bounding box chip in the annotation item is very eye-catching / striking. They really stand out.
Maybe green chips (the vortanz bounding box green) could work as well for the annotation item? So red is only for the video image overlay to show which one is highlighted?