Inter IIT Tech Meet 14.0 · 2025
GeoNLI
Ask questions about a satellite image in plain language: captions, answers and the objects you asked about, outlined.
- Problem statementISRO, Inter IIT Tech Meet 14.0
- VQA accuracy83.3% on VRSBench
- TeamEight authors, IIT Bombay
Problem
One system for three tasks on remote-sensing images: describe the image, answer questions about it (yes or no, counts and open answers), and locate the objects a question refers to.
What we built
- Captioning and question answering with EarthMind, fine-tuned with extra weight on counting questions.
- Grounding by majority vote across seven segmentation models.
- A diffusion model that turns radar (SAR) images into optical ones, so the same models can read them.
- A web app: React front end, FastAPI back end.
Design choices
- An ensemble for grounding: the models fail in different places, and voting cancels single-model errors.
- Counting was the weakest skill, so the fine-tuning data was reweighted towards it.
Result
Visual question answering at 83.3% on VRSBench and 86.1% on RSVQA. Grounding mIoU of 0.649 for the ensemble against 0.590 for the best single model. Written up as a paper.