A Method of Heuristic Human-LLM Collaborative Source Search

doi:10.16182/j.issn1004731x.joss.25-FZ0646E

Abstract

Abstract:

Traditional source search algorithms are prone to local optimization, and source search methods combining crowdsourcing and human-AI collaboration suffer from low cost-efficiency due to human intervention. In this study, we proposed a lightweight human-AI collaboration framework that utilized multi-modal large language models (MLLMs) to achieve visual-language conversion, combined chain-of-thought (CoT) reasoning to optimize decision-making, and constructed a heuristic strategy that incorporated probability distribution filtering and a balance between exploitation and exploration. The effectiveness of the framework was verified by experiments. The human-AI alignment heuristic strategy with large language model adaptation design provides a new idea to reduce manual dependency for source search task in complex scenes.

Key words: source search, human-AI collaboration, large language model, heuristic strategy, human-AI alignment

CLC Number:

TP391.9

Chen Yi, Qiu Sihang, Zhu Zhengqiu, Ji Yatai, Zhao Yong, Ju Rusheng. A Method of Heuristic Human-LLM Collaborative Source Search[J]. Journal of System Simulation, 2025, 37(12): 3112-3127.

Figures/Tables 9

Fig. 1

Table 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 2

Table 3

Table 4

References 30

[1]	Dellermann M Sc Dominik, Ebel Philipp, Söllner Matthias, et al. Hybrid Intelligence[J]. Business & Information Systems Engineering, 2019, 61(5): 637-643.
[2]	Guo Bin. On Intelligent IOT and Future Manufacturing Embracing the Era of Human-computer-things Swarm Intelligence Computing[J]. Frontiers, 2020(13): 32-42.
[3]	Zhao Yong, Zhu Zhengqiu, Chen Bin, et al. Leveraging Human-AI Collaboration in Crowd-powered Source Search: A Preliminary Study[J]. Journal of Social Computing, 2023, 4(2): 95-111.
[4]	Ristic Branko, Skvortsov Alex, Gunatilaka Ajith. A Study of Cognitive Strategies for an Autonomous Search[J]. Information Fusion, 2016, 28: 1-9.
[5]	Zhao Yong, Ji Yatai, Qiu Sihang, et al. A User Interface Design for Collaborations Between Humans and Intelligent Vehicles[C]//Web Engineering. Cham: Springer Nature Switzerland, 2024: 397-400.
[6]	Zhao Yong, Zhu Zhengqiu, Chen Bin, et al. Crowd-powered Source Searching in Complex Environments[C]//Computer Supported Cooperative Work and Social Computing. Singapore: Springer Nature Singapore, 2023: 201-215.
[7]	Chang Yupeng, Wang Xu, Wang Jindong, et al. A Survey on Evaluation of Large Language Models[J]. ACM Transactions on Intelligent Systems and Technology, 2024, 15(3): 39.
[8]	Perez E, Huang S, Song F, et al. Red Teaming Language Models with Language Models[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2022: 3419-3448.
[9]	Yejin Bang, Cahyawijaya S, Lee N, et al. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity[C]// Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics. Stroudsburg: ACL, 2023: 675-718.
[10]	Hutchinson M, Oh Hyondong, Chen Wenhua. A Review of Source Term Estimation Methods for Atmospheric Dispersion Events Using Static or Mobile Sensors[J]. Information Fusion, 2017, 36: 130-148.
[11]	Pang Shuo, Farrell J A. Chemical Plume Source Localization[J]. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2006, 36(5): 1068-1080.
[12]	Hutchinson M, Oh Hyondong, Chen Wenhua. Entrotaxis as a Strategy for Autonomous Search and Source Reconstruction in Turbulent Conditions[J]. Information Fusion, 2018, 42: 179-189.
[13]	Chen Xinxing, Huang Jian. Odor Source Localization Algorithms on Mobile Robots: A Review and Future Outlook[J]. Robotics and Autonomous Systems, 2019, 112: 123-136.
[14]	Zhao Yong, Chen Bin, Wang Xianghan, et al. A Deep Reinforcement Learning Based Searching Method for Source Localization[J]. Information Sciences, 2022, 588: 67-81.
[15]	Shi Yiwei, Wen Muning, Zhang Qi, et al. Autonomous Goal Detection and Cessation in Reinforcement Learning: A Case Study on Source Term Estimation[C]//The 39th Annual AAAI Conference on Artificial Intelligence. Palo Alto, CA, USA: AAAI Press, 2025: 738-745.
[16]	Tang Hongwei, Sun Wei, Yu Hongshan, et al. A Multirobot Target Searching Method Based on Bat Algorithm in Unknown Environments[J]. Expert Systems with Applications, 2020, 141: 112945.
[17]	Ojeda P, Monroy J, Gonzalez-Jimenez J. PSGSL: A Probabilistic Framework Integrating Semantic Scene Understanding and Gas Sensing for Gas Source Localization[EB/OL]. (2025-01-22) [2025-05-04]. .
[18]	Wei Qian, Sun Hongjun, Xu Yin, et al. Exploring the Application of Large Language Models Based AI Agents in Leakage Detection of Natural Gas Valve Chambers[J]. Energies, 2024, 17(22): 5633.
[19]	Hassan S, Wang Lingxiao, Mahmud K R. Integrating Vision and Olfaction via Multi-modal LLM for Robotic Odor Source Localization[J]. Sensors, 2024, 24(24): 7875.
[20]	Howe J. The Rise of Crowdsourcing[EB/OL]. (2006-06-01) [2025-05-04]. .
[21]	Yin Xicheng, Wang Hongwei, Wang Wei, et al. Task Recommendation in Crowdsourcing Systems: A Bibliometric Analysis[J]. Technology in Society, 2020, 63: 101337.
[22]	Liu Chengshuai, Li Wenzhong, Zhao Chenchen, et al. BK-SWMM Flood Simulation Framework Is Being Proposed for Urban Storm Flood Modeling Based on Uncertainty Parameter Crowdsourcing Data from a Single Functional Region[J]. Journal of Environmental Management, 2023, 344: 118482.
[23]	van Lent Michael, Fisher W, Mancuso M. An Explainable Artificial Intelligence System for Small-unit Tactical Behavior[C]//Proceedings of the 16th Conference on Innovative Applications of Artifical Intelligence. Palo Alto: AAAI Press, 2004: 900-907.
[24]	Gao Hui, Feng Jianhao, Xiao Yu, et al. A UAV-assisted Multi-task Allocation Method for Mobile Crowd Sensing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 3790-3804.
[25]	Wei J, Wang Xuezhi, Schuurmans D, et al. Chain-of-thought Prompting Elicits Reasoning in Large Language Models[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2022: 24824-24837.
[26]	Huang Hanyao, Zheng Ou, Wang Dongdong, et al. ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-modal Large Language Model[J]. International Journal of Oral Science, 2023, 15(1): 29.
[27]	Schumann Raphael, Zhu Wanrong, Feng Weixi, et al. VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View[C]//Proceedings of the Thirty-eighth AAAI Conference on Artificial Intelligence and Thirty-sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2024: 18924-18933.
[28]	Kambhampati S, Valmeekam K, Guan Lin, et al. Position: LLMs Can't Plan, but Can Help Planning in LLM-modulo Frameworks[C]//Proceedings of the 41st International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2024: 22895-22907.
[29]	Vergassola Massimo, Villermaux Emmanuel, Shraiman Boris I. 'Infotaxis' as a Strategy for Searching Without Gradients[J]. Nature, 2007, 445(7126): 406-409.
[30]	Zhu Zhengqiu, Ji Yatai, Huang Jiaheng, et al. AutoS$^2$earch: Unlocking the Reasoning Potential of Large Models for Web-based Source Search[EB/OL]. (2025-02-14) [2025-05-04]. .

Judgment type	Qwen-Max	DeepSeek-V3	GPT-4o
Coordinate point	9	25	9
Connectivity	97	97	97
Path length	11	20	21
Particle concentration	2	0	1

Group	Success rate/%	Average steps	Average interventions
Human aided	100	174.36	4.04
Baseline 1	78.5	154.04	/
Baseline 2	89	221.00	3.91
LLMs with heuristic strategy	94	181.89	2.67

Group	Success rate/%	Average steps	Average interventions
Complete model	94	181.89	2.67
Only LLMs	97	215.77	6.08
Only heuristic strategy	100	186.45	2.01

Metric	Statistical test used	Test statistic	p-value
Average steps	Kruskal-Wallis (non-parametric)	H = 56.02	< 0.001
Average interventions	One-way ANOVA (parametric)	F = 3.54	0.031