A delegation from National Instruments (NI) visited Peking University on May 8, 3023, led by Will Denman, Senior Vice President of NI, Honglin Zhu, General Manager of Chinese Institutional Business of NI, and Yong Qin, Account Manager of Northern China Institutional Business of NI. During their visit, they had a meeting with School of Electronics Prof. Xiang Cheng and his group around 6G wireless communication technology and its verification platform .
Mr. Will Denman, the Senior Vice President of NI, is in charge of NI's global sales and marketing business. Mr. Denman attaches great importance to the cooperation between NI and Prof. Cheng's group, and insisted on visiting the Pervasive Connectivity and Network Intelligence Lab of Peking University despite his full schedule in China. Prof. Cheng’s group is also the only university research group that Mr. Will Denman met with in China this time.
Both parties had an in-depth discussion on 6G wireless communication technology and further cooperate. Prof. Cheng analyzed the development trend of 6G wireless communication for the future. He expressed that 6G will expand and deepen on the basis of 5G to realize "smart connection of everything and digital twin". However, the mainstream communication-perception fusion technology is limited to the fusion of communication and single radio frequency perception, and is restricted to static or low-speed scenarios, which cannot support typical 6G application scenarios such as unmanned driving and intelligent logistics.
Therefore, Prof. Cheng and his group proposed the concept of “Synesthesia of Machines, SoM” for the first time in the world, i.e. to achieve the intelligent multi-modal sensing-communication integration, and to overcome the technical challenges of large communication bandwidth requirements, high perception capability requirements, and fast dynamic changes in 6G application scenarios. They also proposed three fundamental models of machine association perception: evocation model, enhancement model, and cooperation model. The relevant research results have been published in the Journal of Communications[1], a first-class academic journal sponsored by China Institute of Communications, and IEEE Internet of Things Journal, a SCI zone 1 journal of the Chinese Academy of Sciences (impact factor is 10.238) [2]. Besides, the latest research review on SoM has been submitted to IEEE Communications Surveys & Tutorials[3], which is the most authoritative review journal in the field of communications, it is representing the academic frontier and future directions in the field, with an impact factor of 33.84 in 2022, ranking the first place among all SCI journals of the IEEE. Prof. Cheng's group has launched the first Mixed Multi-Modal Sensing-Communication integration (M3SC) dataset, which uses software such as AirSim and WirelessInsite to collect multimodal sensing and wireless channel data to achieve deep fusion and accurate matching of physical space and electromagnetic space, covering multiple frequency bands (sub-6GHz and 28GHz millimeter wave), multiple antenna arrays, multiple weather, multiple time periods, multiple traffic densities, and other scenario conditions. More, with the support of NI equipment, a preliminary physical verification platform for the intelligent fusion of communication and multimodal sensing precisely aligned with simulation data was built to enable the injection of real measurement data. Prof. Cheng showed the preliminary results of dataset construction and platform building, and said that in the future, he would explore the research and application of higher frequency bands and richer scenarios.
In terms of building verification platform for intelligent multi-modal sensing-communication integration, Prof. Cheng said that the subsequent building plan would fully cover sub-6GHz, millimeter wave and terahertz bands, and the group will build indoor intelligent multi-modal sensing-communication integration verification platforms based on Universal Software Radio Peripheral (USRP) and PCI eXtensions for Instrumentation (PXI) system respectively. Meanwhile, with the joint support of USRP and PXI, an outdoor vehicle road collaborative verification platform based on communication and multimodal perception will be built. Prof. Cheng also indicated that based on the platform built, he will continue to explore the intrinsic mechanism of machine associative perception, use the three basic modes of machine associative perception as a guide, and adopt a model-enhanced data-driven scheme to design artificial neural networks to achieve and validate a credible and interpretable intelligent fusion between communication and multimodal perception.
Honglin Zhu introduced NI's strategic investment, product planning and market layout in the field of wireless communication technology, and extended NI's recognition and praise for the “SoM” and platform building plan of the Pervasive Connectivity and Network Intelligence Lab. Will Denman expressed NI's team in wireless communication technology and and intelligent vehicles has been working in the front line of research and development for a long time and has a deep technical accumulation. The team is willing to fully support the research and platform building work of PCNI Lab, and give full play to NI's advantages in test and measurement equipment manufacturing, large-scale verification platform building and other aspects to form a "strong alliance" with the Lab. NI will strengthen the cooperation between PCNI Lab and their own team in the verification platform construction and assist the group to build the world's first verification platform for SoM, and assist the Lab to make a major technological breakthrough in the field of intelligent multi-modal sensing-communication integration.
As discussed by both parties, the verification platform will be launched by the end of 2023 to the beginning of 2024, when both parties will jointly publish the relevant work results in the academic conferences in the field, the official website of Peking University School of Electronics and the website of PCNI Lab, and the official website of NI.
Next, the PCNI Lab of Peking University will continue to be focused in the fields of wireless communication, vehicle networking and networked intelligence, and will focus on key challenges by exploring the mechanism of machine connectivity and technology research, striving to be at the forefront of international scientific and technological development.
About National Instruments:
NI, known as National Instruments, was incorporated in 1976 and is involved various fields such as data acquisition and virtual instrumentation. NI serves more than 35,000 accounts in more than 40 countries and territories worldwide. It is an international leader in test and measurement systems, modular instrumentation and more. NI has been a deep participant in wireless communications research and innovation, working with top universities and research institutions around the world on key communications technologies.
About the Pervasive Connectivity and Network Intelligence Lab:
The Pervasive Connectivity and Network Intelligence (PCNI) Lab of Peking University is led by Prof. Xiang Cheng. The Lab consists of 5 professors/ associate professors/ assistant professors/ research professors plus about 20 postgraduate students. The research directions of the PCNI include model-enhanced data-driven smart networks and networked intelligence research, with focus on smart transportation and logistics scenarios, including wireless communication channel modeling and applications, intelligent multi-modal sensing-communication integration, B5G/6G smart vehicle networking and multi-intelligent body collaboration theory and applications.
Synesthesia of Machines (SoM):
Inspired by the phenomenon of human synesthesia (i.e., the phenomenon of mutual arousal, enhancement or cooperation between one sense and other senses of different categories due to the cross-activation of neurons in the brain neural network), a similar synesthesia phenomenon should also exist between cross-media multimodal information under 6G Everything Smart, i.e., machine synesthesia. Specifically, machine synesthesia refers to the process of using artificial neural networks to fully exploit the intrinsic association between multimodal information through synesthesia processing of multimodal environmental data to achieve enhanced environmental perception and enhanced communication and network functions.
References:
[1] X. Cheng, H. Zhang, Z. Yang, Z. Huang, L. Li, and A. Yu, “Integrated sensing and communications for internet of vehicles: current status and development trend," Journal on Communications, vol. 43, no. 8, pp.188- -202, 2022.
[2] X. Cheng, D. Duan, S. Gao, and L. Yang, “Integrated Sensing and Communications (ISAC) for Vehicular Communication Networks (VCN),” IEEE Internet of Things Journal, vol. 9, no. 23, pp. 23441–23451, Dec. 2022.
[3] X. Cheng et al., “Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines,” IEEE Communications Surveys & Tutorials, submitted for publication, 2023.
