On December 24, 2021, Associate Professor Han Yuexing’s research team held their annual year-end gathering at Jin Guan Lou in Shanghai. During the event, the professor shared his own educational experiences and provided valuable advice for our future development. Li Xiaolong, who had graduated and returned to join the gathering, shared his work experiences with everyone. The dinner concluded with mutual blessings and well wishes among the attendees.
The 5th Materials Genome Engineering Summit was held in Zhengzhou, Henan, from December 14th to 16th, 2021. On the afternoon of December 16th, Professor Han Yuexing delivered a presentation titled “Material Image Processing Methods Based on Computer Vision.” He described several methods for extracting information from material image data. Material data serves as the foundation of the Materials Genome Initiative and plays a crucial role in constructing various databases related to material research. It also aids in the understanding of the relationships between “microstructure,” “manufacturing processes,” and “macroscopic properties” for artificial intelligence. Material data primarily consists of computational data, experimental data, production data, and literature data, with a significant amount of images often included within these datasets. Extracting key information from these images is a primary objective of material image processing.
The presentation mainly described the use of computer vision methods, including machine learning and deep learning techniques, to extract and analyze various elements within complex material images. Three specific examples were discussed: 1. Recognition of DNA macromolecules in images of molecular robots: This example focused on using computer vision methods to identify and recognize DNA macromolecules in images of molecular robots. By applying machine learning algorithms, the system can automatically extract and analyze the structures and patterns of DNA molecules, contributing to the understanding and design of molecular machines. 2. Segmentation and recognition of Kikuchi patterns: The second example involved the segmentation and recognition of Kikuchi patterns. By employing image processing techniques and deep learning algorithms, the system can accurately identify and segment Kikuchi patterns, which are essential for analyzing crystal structures and orientations. 3. Material image segmentation using superpixels and improved DenseNet: The third example discussed material image segmentation using a combination of superpixels and an improved DenseNet method. By utilizing superpixels to group image regions and incorporating an enhanced DenseNet architecture, the system can effectively segment materials in images, providing valuable insights for material analysis and characterization. These examples demonstrate the application of computer vision techniques, including machine learning and deep learning, in extracting and analyzing important information from complex material images.
In addition, on the afternoon of December 16th, Professor Han Yuexing also chaired an academic conference in a parallel session.
Li Xiaolong is a remarkable individual who completed his undergraduate studies at Fujian Engineering College and was admitted to the School of Computer Science at Shanghai University in 2018. Shortly after starting his studies, he joined Professor Han Yuexing’s Image Research Group, focusing on computer vision-related topics. Under Professor Han’s guidance, he chose the topic of medical image segmentation. Over the course of three years, Li Xiaolong worked diligently and utilized various techniques, including deep learning, to achieve automated segmentation of liver and tumor CT images. His contributions have made significant strides in the development of intelligent healthcare. After graduation, Li Xiaolong joined United Imaging Healthcare Co., Ltd., where he continued to work in the field of medical imaging. In addition to his academic pursuits, Li Xiaolong has engaged in extensive extracurricular reading of literature and history. He also has a passion for physical fitness and has formed valuable relationships with mentors and friends. Throughout his three years at Shanghai University, Li Xiaolong has demonstrated tremendous growth. It is hoped that he will maintain his enthusiasm, continuously improve himself, and strive for further breakthroughs in his future endeavors.
Work during graduate studies
In order to assist doctors in making comprehensive assessments and plans for tumor patients, automated segmentation of the liver and tumors in CT images is performed using various deep learning techniques. This provides a data foundation for quantitative and qualitative diagnosis, facilitating clinical treatment. Automated segmentation of the liver and tumors using a combination of deep learning techniques allows doctors to obtain comprehensive evaluations and treatment plans for tumor patients. This automated approach alleviates the workload on doctors and improves the accuracy and efficiency of diagnosis. By utilizing deep learning algorithms, the system can learn and recognize the features of the liver and tumors in CT images, accurately segmenting them. Such segmentation results can be used for quantitative and qualitative assessments, aiding in clinical decision-making.
-
A boundary loss-based 2.5D fully convolutional network (FCN) is proposed to address the limitations of FCN, U-Net, and other deep learning methods in exploring spatial feature information in three-dimensional CT images. This novel approach effectively explores the spatial feature information in CT images while reducing the network parameters and computational resource consumption. The proposed method enhances the accuracy of liver and liver tumor segmentation.
-
To address the characteristics of medical images and the limited capability of common loss functions to optimize network exploration of boundary features, a new boundary loss function is designed. This novel loss function combines the distance, area, and boundary information of image contours, effectively optimizing deep learning networks to explore more image boundary and contour features.
-
To address the issue of encoding-decoding networks ignoring the correlation and dependency among local features, a segmentation framework based on dual-path attention is proposed by integrating 2D, 2.5D, and 3D networks with attention mechanisms. This framework incorporates dual-path self-attention modules, dense network blocks, residual network blocks, and dual-path network blocks. It comprises nine different network structures and is capable of effectively performing automated segmentation of liver and tumor CT images. The integration of these components allows the network to capture and leverage both local and global information, leading to improved segmentation accuracy.
Essay: Research on liver and tumor segmentation using multi-dimensional encoding-decoding networks
Our team has recently published a paper titled “A novel transfer learning for recognition of overlapping nano object” in the international journal “Neural Computing and Application” (IF: 5.606, Q2 in Chinese Academy of Sciences ranking). The School of Computer Engineering and Science at Shanghai University is listed as the first affiliation, with Associate Professor Han Yuexing as the first author and corresponding author. A significant contribution to this work was made by master’s students Liu Yuhong, Song Leilei, and Tong Lin.
Despite the rapid development of nanoscience and nanotechnology in various fields, the high cost associated with obtaining a sufficient number of nano-object samples remains a challenge, impeding the advancement of deep learning methods in the materials domain. To address this issue, we have designed a novel method for identifying nano-objects in atomic force microscopy (AFM) images. Firstly, we employ a LOG-based image denoising technique for preprocessing the images. Then, we propose two improved methods for segmenting overlapping objects based on the watershed algorithm. Finally, we establish a transfer learning-based convolutional neural network (CNN) recognition model. This model achieves excellent performance by pretraining on large-scale datasets of handwritten digits and letter shapes, enabling the identification of nano-objects in AFM images. The method proposed in this study effectively tackles the challenge of identifying nano-objects in AFM images, where the availability of small-sample nano-objects is limited.
Our team, in collaboration with Associate Professor Li Long’s team from the School of Artificial Intelligence, Shanghai University, has recently published a paper titled “A classification method for EEG motor imagery signals based on parallel convolutional neural network” in the international journal “Biomedical Signal Processing and Control” (IF: 2.954, Q2 in JCR and Chinese Academy of Sciences SCI journal ranking). The School of Computer Science at Shanghai University is listed as the first affiliation, with Associate Professor Han Yuexing as the first author and corresponding author.
Deep learning has been widely successful in various image classification tasks. However, its application to EEG (Electroencephalogram) motor imagery signal classification has been limited. In this study, we propose a preprocessing algorithm for representing EEG signals, followed by a parallel convolutional neural network (PCNN) architecture for classifying motor imagery signals. For the representation of raw EEG signals, a novel form of image is created to combine spatial filtering and frequency band extraction. By inputting the represented images into the PCNN, it stacks three unique sub-models together, aiming to optimize the classification performance. The proposed method achieves an average accuracy of 83.0±3.4% on the BCI Competition IV Dataset 2b, surpassing the compared methods by at least 5.2%. The average Kappa value of this method on Dataset 2b reaches 0.659±0.067, exhibiting at least a 20.5% improvement compared to the compared algorithms. These results demonstrate the effectiveness of this method in classifying motor imagery signals in EEG-based brain-computer interface (BCI) applications.
Our team, in collaboration with Professor Yang Jiong’s research group from the Institute of Materials Genome Engineering, has recently published a paper titled “Accelerating the Discovery of Cu−Sn−S Thermoelectric Compounds via High-Throughput Synthesis, Characterization, and Machine Learning-Assisted Image Analysis” in the international journal “Chemistry of Materials” (IF: 9.872, Q1 in JCR and Chinese Academy of Sciences SCI journal ranking). “Chemistry of Materials” is one of the most influential top-tier academic journals in the field of engineering, technology, and materials science. The paper is affiliated with Shanghai University as the first institution, with Sheng Ye, a PhD student from the Institute of Materials Genome Engineering, as the first author. Professor Yang Jiong, Professor Xi Jinyang from the Institute of Materials Genome Engineering, and Associate Professor Han Yuexing from the School of Computer Engineering and Science, Shanghai University, are the corresponding authors.
High-throughput (HTP) methods have emerged as powerful approaches for accelerating materials research and development. In this work, we demonstrate the capability of combining machine learning (ML) image segmentation techniques with HTP synthesis and characterization for the discovery of new thermoelectric compounds. First, cylindrical samples with nine different compositions were synthesized using an improved diffusion couple HTP synthesis method. Subsequently, scanning electron microscopy (SEM) characterization was performed on fixed fragments of each composition, resulting in the acquisition of 11 backscattered electron (BSE) images per fragment (99 images in total). To rapidly segment the different phases in the 99 BSE images, we propose two ML image segmentation strategies: a supervised active learning strategy utilizing a fully connected neural network and an unsupervised clustering strategy. The supervised strategy significantly reduces the time cost of image segmentation, achieving a classification accuracy of 0.9. This model is then used for automatic segmentation of a large batch of images. To further reduce the workload of manual annotation, active learning is introduced during the training process. Analyzing the energy-dispersive X-ray spectroscopy (EDS) characterization of the two major phases separated by the first strategy, we discovered a new compound, Cu7Sn3S10, which exhibits promising thermoelectric properties with a zT value exceeding 0.6. In contrast to the supervised strategy, the unsupervised strategy allows for the identification of compounds that may have been overlooked in the BSE images without manual annotation. Based on the results of the unsupervised strategy, we discovered an unreported compound, Cu1.6S. Overall, our work showcases an example of a fully HTP workflow, enabling automated analysis of HTP characterization and new compound identification.
Han Sifan: I obtained my bachelor's degree in Software Engineering from Northwest Minzu University and I am currently pursuing a master's degree in Software Engineering at Shanghai University. I am a person who loves to smile, enjoy life, appreciate simplicity, listen to music, and value freedom. I find joy in both tranquility and lively excitement. My history does not boast of dazzling achievements; instead, it is a testament to my perseverance and steady progress. I have served as a class monitor, taking responsibility for the academic progress of my classmates. I have led a team to participate in mathematical modeling competitions and achieved awards, understanding the power of teamwork. I was admitted to graduate school through recommendation, realizing that success does not come easily and requires humility and effort. In terms of personality, I am warm, friendly, lively, and optimistic. I have a spirit of initiative, team spirit, and strong hands-on abilities. I possess excellent coordination and communication skills, adaptability, quick response, and a positive and flexible attitude. I embrace innovation and approach life with sincerity, enthusiasm, and a positive outlook. I am conscientious, responsible, humble, and have a good sense of time management and adaptability. I actively engage in various social practices to enrich myself, unleash my talents, and explore my potential. I value teamwork and have a strong sense of collective consciousness. In my studies, I emphasize the integration of theory and practice. In the new learning environment, I will continue to work hard, enhance my abilities, embrace new challenges, and progress together with everyone.
Chen Zunlong: I obtained my bachelor's degree in Electronic Information Science and Technology from the School of Information Science and Engineering at Ocean University of China. Currently, I am pursuing a master's degree in Computer Science and Technology at Shanghai University. I have an outgoing personality, a passion for research, and excellent communication skills. I approach my studies and research with great rigor and attention to detail, often reflecting on my knowledge system. During my free time, I enjoy watching videos and browsing forums. I hope to improve myself during my graduate studies at Shanghai University and progress together with my professors and classmates on the path of researching computer technology.
Chi Rutin: I completed my undergraduate studies at the Department of Intelligent Science and Technology, School of Computer Science, Shanghai University. After graduation, I decided to pursue a master's degree in Computer Science and Technology within the same school, under the guidance of Professor Han Yuexing. I have a penchant for watching mystery movies, reading various books, jogging, and cooking. My personality is outgoing, optimistic, thoughtful, and I am always willing to take on new challenges. I hope to encounter a better version of myself on this new journey.
Wan Guanxin: I completed my undergraduate studies in Computer Science and Technology at Guilin University of Technology. Currently, I am pursuing a master's degree in Computer Science and Technology at Shanghai University. In terms of personality, I am outgoing and lively, and I easily get along well with others. I am good at communication, resilient, and possess a strong sense of responsibility. I can handle life and the difficulties encountered with a positive attitude. I have a strong adaptability to different environments, remain calm when faced with challenges, and have confidence in accomplishing various tasks effectively. In terms of hobbies, I enjoy reading books, listening to music, and watching suspenseful movies and TV shows as a way to relax. I also engage in activities such as playing badminton, table tennis, and running to keep myself physically fit. In my daily life, I approach tasks with seriousness and responsibility. I like to set small goals to motivate myself, and I maintain an optimistic and persistent attitude when faced with difficulties. My personal motto is "Success = Diligent study + Correct methods + Less empty talk." I hope to achieve my own version of success during my time at Shanghai University.
Our team, in collaboration with Professor Wu Jinbo’s research group from the Institute of Materials Genomics Engineering, has recently published a paper titled “Unclonable Micro-Texture with Clonable Micro-Shape towards Rapid, Convenient, and Low-Cost Fluorescent Anti-Counterfeiting Labels” in the international journal “Small” (IF: 11.459, top in Zone 1 according to JCR and Chinese Academy of Sciences SCI journal rankings). “Small” is one of the most influential top-tier academic journals in the field of engineering, technology, and materials science. In this publication, Shanghai University is listed as the primary affiliation, with Zhang Hongkun, a master’s student from the School of Computer Engineering and Science, and Lin Yuhong and Feng Jingyun, both master’s students from the Institute of Materials Genomics Engineering, being the co-first authors. Professor Wu Jinbo from the Institute of Materials Genomics Engineering and Associate Professor Han Yuexing from the School of Computer Engineering and Science are the corresponding co-authors. This work has received strong support from Academician Zhang Junyi, Professor Wen Weijia, and Professor Qi Yabing from the Okinawa Institute of Science and Technology Graduate University (OIST) in Japan.
Counterfeit products, ranging from high-end luxury goods to medical supplies, not only cause significant economic losses but also pose a significant threat to people’s health. The currently widely used anti-counterfeiting labels are based on deterministic production processes, which are simple to replicate for counterfeiters. Developing anti-counterfeiting labels with physical unclonable functions (PUFs) is a feasible solution. However, traditional PUF recognition techniques require matching against all images in a database, resulting in high production costs and slow identification speed.
To address the aforementioned issues, the research team utilized high-throughput parallel non-continuous dewetting technology to fabricate a multifunctional label with four layers of anti-counterfeiting capabilities. The first layer of anti-counterfeiting is achieved through the fluorescence of perovskite crystal films. These films exhibit unique fluorescence properties that can be used for authentication. The second layer of anti-counterfeiting involves macroscopic patterns composed of basic units, such as QR codes, which can carry information and serve as additional security features. The third layer of anti-counterfeiting is based on microscale basic units with diverse shapes. These units provide an additional level of security as their distinct shapes can be difficult to replicate. The fourth layer of anti-counterfeiting is achieved through the use of non-reproducible textures formed through self-assembly processes. These random textures create a unique and unclonable pattern, further enhancing the security of the label. By combining these four layers of anti-counterfeiting features, the research team has developed a label that offers enhanced security and counterfeit prevention across multiple levels.
Our main work involved designing and establishing a database that stores authentic thin film patterns, which includes shape and texture databases. For shape recognition, we employed the CNL (Control the Number of Landmarks) method, an improved Hough Transform method, and shape space theory. These techniques were utilized to accurately identify the shapes present in the labels. Subsequently, we utilized the GMS (Grid-based Motion Statistics) method to further recognize the textures of the labels. This method leverages grid-based analysis and motion statistics to effectively identify and differentiate the unique textures embedded in the labels. By combining the shape recognition techniques and texture identification using GMS, our research team has developed a comprehensive approach for the recognition and authentication of the labels based on their shape and texture characteristics.
In practical applications, anti-counterfeiting micro-patterns are captured using smartphones or portable microscopes and then fed into recognition software for matching and identification. The various contour shapes of the nanocrystal films are refined and classified, serving as “classification symbols” for image data recognition. By employing this “divide and conquer” strategy in the recognition process, the verification time can be reduced by more than 20 times. Combining high-throughput materials fabrication techniques with data-driven material applications, our approach achieves low reagent costs (2.1 × 10−4 USD), simple and fast authentication (total time of 12.17 seconds), and high coding capacity (2.1 × 10623). This integration of advanced technologies enables cost-effective and efficient anti-counterfeiting measures, providing a robust solution with high coding capacity and fast authentication for a wide range of applications.
The related work has received funding from various sources, including: 1. National Key Research and Development Program (2020YFB0704503, 2018YFB0704400, 2018YFB0704402): These programs are part of the national strategic research initiative that supports key research and development projects in China. 2. National Natural Science Foundation of China (21775101): This foundation provides funding for basic and applied research in various scientific disciplines. 3. 111 Project (D16002): The 111 Project is a national recruitment program that aims to attract and retain top-tier talent for research and innovation in Chinese universities. 4. Shanghai Municipal Science and Technology Commission Project (20ZR1419000): This project is funded by the Shanghai municipal government to support research and development activities in the city. The support from these funding sources has played a crucial role in facilitating the progress and success of the research work in developing innovative anti-counterfeiting technologies and materials.
Essay: https://onlinelibrary.wiley.com/doi/10.1002/smll.202100244
Congratulations to He Jiyuan, Bao Shengqi, Song Luoya, Chi Rutin, Shi Tian’an, and Xu Ying on graduating from their undergraduate program! They studied Computer Science and Technology at Shanghai University, and from their second year, they conducted research in the field of computer vision under the guidance of Professor Han Yanxing. Their graduation projects were also supervised by Professor Han. The four years of their undergraduate studies have passed quickly, and each of them has achieved something. They will continue to pursue their dreams with their ideals in mind.
Bao Shengqi is an optimistic and positive student, who is not afraid to challenge himself and overcome difficulties. During his four years of study at Shanghai University, he not only gained a wealth of knowledge but also formed meaningful connections with many great teachers and friends. We look forward to seeing Bao Shengqi continue to strive for excellence in the future and make continuous breakthroughs.
Song Luoya is a cheerful and outgoing student with a wide range of interests. While diligently studying, she often spends time in the library reading books and listening to music. During her free time, she enjoys activities like swimming and playing badminton to relax. The four years of undergraduate studies at Shanghai University have left Song Luoya with many wonderful memories. We hope that she continues to move forward and make progress in her future endeavors.
Chi Rutin is an enthusiastic and optimistic student who is adept at critical thinking and willing to take on new challenges. In her spare time, she enjoys reading books, jogging, and cooking. After completing her undergraduate studies, Chi Rutin will continue her academic journey as a master’s student in the Computer Science and Technology program at Shanghai University’s School of Computer Science. Under the guidance of Professor Han Yanxing, she hopes to discover a better version of herself in this new endeavor.
Xu Ying is a diligent and steady student who approaches tasks with dedication and maintains a calm demeanor. She enjoys making friends, traveling, exercising, and has a deep passion for life. In her master’s studies, she will pursue a graduate degree in the field of database research with a focus on software engineering at Zhejiang University. As Xu Ying continues her journey in computer science, we hope that she will soon find a field of study or work that she wholeheartedly devotes herself to, and continue to make progress in her academic and professional career.
He Jiyuan is a sincere and genuine individual who excels in communication and interpersonal skills. He is known for his honesty, enthusiasm, and strong sense of responsibility. We look forward to seeing He Jiyuan continue to improve and surpass himself in his future studies.
Shi Tian’an is an outgoing individual who approaches tasks diligently and has a keen interest in music and reading. After graduation, Shi Tian’an will pursue a master’s degree in Artificial Intelligence at the School of Computer Science at Fudan University. During his four years of undergraduate studies, Shi Tian’an has formed meaningful connections with many great teachers and friends. We hope that in the future, he will bravely strive for success and achieve personal growth on his life journey.
Once again, I wish all the students the best as they continue on their journey, breaking through challenges and forging ahead! I also hope that each of you will strive relentlessly to achieve your lofty life goals, seize the present moment, and seize your own lives. Through your unwavering efforts, may you create a bright future that belongs to both yourselves and our country. With determination, confidence, perseverance, and patience, may these four “hearts” always accompany you! May you use your wisdom to create a brilliant life of your own.
I am honored to receive recognition as the Outstanding Class Advisor in the first and second editions in 2019 and 2020, respectively. I attended the award ceremony on May 14, 2021. I will continue to strive for excellence in the future because the responsibilities of a teacher encompass both teaching and research, with other tasks serving as auxiliary to these two main aspects.
