成人网

npj Artificial Intelligence 2025

AI system facilitates people with blindness and low vision in interpreting and experiencing unfamiliar environments

 

Haozhe Lin1#, Jiangtao Gong2#, Yu Wang1#, Jinsong Zhang3#, Bing Bai1, Yan Zhang2, Luyao Wang2, Chunyu Wei1, Yancheng Cao2, Kun Li3, Ruqi Huang4*, Guyue Zhou2,5*

1 Beijing National Research Center for Information Science and Technology, Tsinghua University  
2 Institute for AI Industry Research (AIR), Tsinghua University  
3 College of Intelligence and Computing, Tianjin University  
4 Tsinghua Shenzhen International Graduate School  
5 School of Vehicle and Mobility, Tsinghua University  

# Equal contribution   * Corresponding author

 

[Code] [Paper]

 

Abstract

Engaging with nature significantly enhances well-being, yet millions of individuals with blindness and low vision (BLV) are often excluded from these benefits due to constrained environmental perception. Here, we introduce VIPTour, an AI-driven system powered by the FocusFormer algorithm, which transforms complex scenes into structured, personalized graphs using tailored attention mechanisms and a BLV-in-the-Loop Adapter. Through intuitive, user-centered interaction, VIPTour facilitates active exploration and in-depth comprehension during dynamic sightseeing, while enabling accurate, long-lasting recollection and effective communication among BLV individuals post-journey. Extensive experiments demonstrate that VIPTour significantly enhances positive emotions and memory retention, with a 67.9% increase in positive emotional response, a 94.7% rise in arousal, a 772.73% improvement in cognitive mapping accuracy, and a 200% enhancement in long-term memory accuracy. These results underscore VIPTour’s ability to deliver an unparalleled, enjoyable, and memorable experience, promising profound benefits for the BLV community.


Method

 

 

Fig 1. Method overview.

 


Demo

 

 


Technical Paper

 


Citation

Haozhe Lin, Jiangtao Gong, Yu Wang, Jinsong Zhang, Bing Bai, Yan Zhang, Luyao Wang, Chenyu Wei, Yancheng Cao, Kun Li, Ruqi Huang, Guyue Zhou. "AI system facilitates people with blindness and low vision in interpreting and experiencing unfamiliar environments". npj Artificial Intelligence 2025.

 

@article{lin2025ai,
  author = {Haozhe Lin and Jiangtao Gong and Yu Wang and Jinsong Zhang and Bing Bai and Yan Zhang and Luyao Wang and Chenyu Wei and Yancheng Cao and Kun Li and Ruqi Huang and Guyue Zhou},
  title = {AI system facilitates people with blindness and low vision in interpreting and experiencing unfamiliar environments},
  booktitle = {npj Artificial Intelligence},
  volume={1},
  number={1},
  pages={1--13},
  year={2025},
}