Publications

2024

1. NeurIPS 2024

   MEQA: A Benchmark for Multi-hop Event-centric Question Answering with Explanations

   Ruosen Li , Zimu Wang, Son Quoc Tran, Lei Xia, and 1 more author

   The Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024), Dec 2024
2. PACLIC 2024

   Domain-specific Guided Summarization for Mental Health Posts

   Lu Qian , Yuqi Wang , Zimu Wang, Haiyang Zhang, and 3 more authors

   The 38th Pacific Asia Conference on Language, Information and Computation (PACLIC 2024), Dec 2024
3. UIC 2024

   Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection

   Jianfei He , Lilin Wang , Jiaying Wang , Zhenyu Liu, and 4 more authors

   2024 IEEE 21st International Conference on Ubiquitous Intelligence and Computing (UIC 2024), Dec 2024

   Abs

   Identifying offensive language is essential for maintaining safety and sustainability in the social media era. Though large language models (LLMs) have demonstrated encouraging potential in social media analytics, they lack thorough evaluation when in offensive language detection, particularly in multilingual environments. We for the first time evaluate multilingual offensive language detection of LLMs in three languages: English, Spanish, and German with three LLMs, GPT-3.5, Flan-T5, and Mistral, in both monolingual and multilingual settings. We further examine the impact of different prompt languages and augmented translation data for the task in non-English contexts. Furthermore, we discuss the impact of the inherent bias in LLMs and the datasets in the mispredictions related to sensitive topics.

4. UIC 2024

   Language-based Audio Retrieval with Co-Attention Networks

   Haoran Sun , Zimu Wang , Qiuyi Chen , Jianjun Chen, and 2 more authors

   2024 IEEE 21st International Conference on Ubiquitous Intelligence and Computing (UIC 2024), Dec 2024
5. ICONIP 2024

   MonoTCM: Semantic-Depth Fusion Transformer for Monocular 3D Object Detection with Token Clustering and Merging

   Changyu Zeng , Zimu Wang, Jimin Xiao, Anh Nguyen, and 3 more authors

   The 31st International Conference on Neural Information Processing (ICONIP 2024), Dec 2024
6. EMNLP 2024

   Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering

   Zimu Wang, Lei Xia , Wei Wang, and Xinya Du

   Findings of the Association for Computational Linguistics: EMNLP 2024 (EMNLP 2024 Findings), Nov 2024
7. INLG 2024

   MTSwitch: A Web-based System for Translation between Molecules and Texts

   Nijia Han , Zimu Wang , Yuqi Wang, Haiyang Zhang, and 2 more authors

   The 17th International Natural Language Generation Conference: System Demonstrations (INLG 2024 Demo), Sep 2024

   Abs HTML PDF

   We introduce MTSwitch, a web-based system for the bidirectional translation between molecules and texts, leveraging various large language models (LLMs). It supports two crucial tasks, including molecule captioning (explaining the properties of a molecule) and molecule generation (designing a molecule based on specific properties). To the best of our knowledge, MTSwitch is currently the first accessible system that allows users to translate between molecular representations and descriptive text contents. The system and a screencast can be found in https://github.com/hanninaa/MTSwitch.

8. WASSA 2024

   Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection

   Yuqi Wang , Zimu Wang, Nijia Han , Wei Wang^†, and 4 more authors

   The 14th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA@ACL 2024), Aug 2024

   Abs HTML PDF

   Emotion detection from text is a crucial task in understanding natural language with wide-ranging applications. Existing approaches for multilingual emotion detection from text face challenges with data scarcity across many languages and a lack of interpretability. We propose a novel method that leverages both monolingual and multilingual pre-trained language models to improve performance and interpretability. Our approach involves 1) training a high-performing English monolingual model in parallel with a multilingual model and 2) using knowledge distillation to transfer the emotion detection capabilities from the monolingual teacher to the multilingual student model. Experiments on a multilingual dataset demonstrate significant performance gains for refined multilingual models like XLM-RoBERTa and E5 after distillation. Furthermore, our approach enhances interpretability by enabling better identification of emotion-trigger words. Our work presents a promising direction for building accurate, robust and explainable multilingual emotion detection systems.

9. LKM 2024

   Knowledge Base-enhanced Multilingual Relation Extraction with Large Language Models

   Tong Chen, Procheta Sen , Zimu Wang, Zhengyong Jiang^†, and 1 more author

   The First International OpenKG Workshop on Large Knowledge-Enhanced Models (LKM@IJCAI 2024), Aug 2024
10. CCL 2024

    System Report for CCL24-Eval Task 8: Exploring Faithful and Informative Commonsense Reasoning and Moral Understanding in Children’s Stories

    Zimu Wang , Yuqi Wang, Nijia Han, Qi Chen, and 4 more authors

    The 23rd China National Conference on Computational Linguistics: Evaluations (CCL24-Eval), Jul 2024
11. CSCWD 2024

    Generating Valid and Natural Adversarial Examples with Large Language Models

    Zimu Wang , Wei Wang^†, Qi Chen, Qiufeng Wang, and 1 more author

    2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2024), May 2024

    Abs arXiv HTML

    Deep learning-based natural language processing (NLP) models, particularly pre-trained language models (PLMs), have been revealed to be vulnerable to adversarial attacks. However, the adversarial examples generated by many mainstream word-level adversarial attack models are neither valid nor natural, leading to the loss of semantic maintenance, grammaticality, and human imperceptibility. Based on the exceptional capacity of language understanding and generation of large language models (LLMs), we propose LLM-Attack, which aims at generating both valid and natural adversarial examples with LLMs. The method consists of two stages: word importance ranking (which searches for the most vulnerable words) and word synonym replacement (which substitutes them with their synonyms obtained from LLMs). Experimental results on the Movie Review (MR), IMDB, and Yelp Review Polarity datasets against the baseline adversarial attack models illustrate the effectiveness of LLM-Attack, and it outperforms the baselines in human and GPT-4 evaluation by a significant margin. The model can generate adversarial examples that are typically valid and natural, with the preservation of semantic meaning, grammaticality, and human imperceptibility.

12. EACL 2024

    FinBPM: A Framework for Portfolio Management-based Financial Investor Behavior Perception Model

    Zhilu Zhang, Procheta Sen^† , Zimu Wang, Ruoyu Sun, and 2 more authors

    The 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024), Mar 2024

    Abs HTML PDF

    The goal of portfolio management is to simultaneously maximize the accumulated return and also to control risk. In consecutive trading periods, portfolio manager needs to continuously adjust the portfolio weights based on the factors which can cause price fluctuation in the market. In the stock market, the factors affecting the stock price can be divided into two categories. The first is price fluctuations caused by irrational investment of the speculators. The second is endogenous value changes caused by operations of the company. In recent years, with the advancement of artificial intelligence technology, reinforcement learning (RL) algorithms have been increasingly employed by scholars to address financial problems, particularly in the area of portfolio management. However, the deep RL models proposed by these scholars in the past have focused more on analyzing the price changes caused by the investment behavior of speculators in response to technical indicators of actual stock prices. In this research, we introduce an RL-based framework called FinBPM, which takes both the factor pertaining to the impact on operations of the company and the factor of the irrational investment of the speculator into consideration. For our experimentation, we randomly selected twelve stocks from the Dow Jones Industrial Index to construct our portfolio. The experimental results reveal that, in comparison to conventional reinforcement learning methods, our approach with at least 13.26% increase over other methods compared. Additionally, it achieved the best Sharpe ratio of 2.77, effectively maximizing the return per unit of risk.

13. arXiv 2024

    Rethinking Human-like Translation Strategy: Integrating Drift-Diffusion Model with Large Language Models for Machine Translation

    Hongbin Na , Zimu Wang, Mieradilijiang Maimaiti^† , Tong Chen, and 3 more authors

    arXiv preprint arXiv:2402.10699, Feb 2024

    Abs arXiv

    Large language models (LLMs) have demonstrated promising potential in various downstream tasks, including machine translation. However, prior work on LLM-based machine translation has mainly focused on better utilizing training data, demonstrations, or pre-defined and universal knowledge to improve performance, with a lack of consideration of decision-making like human translators. In this paper, we incorporate Thinker with the Drift-Diffusion Model (Thinker-DDM) to address this issue. We then redefine the Drift-Diffusion process to emulate human translators’ dynamic decision-making under constrained resources. We conduct extensive experiments under the high-resource, low-resource, and commonsense translation settings using the WMT22 and CommonMT datasets, in which Thinker-DDM outperforms baselines in the first two scenarios. We also perform additional analysis and evaluation on commonsense translation to illustrate the high effectiveness and efficacy of the proposed method.

2023

1. EMNLP 2023

   OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding

   Hao Peng^* , Xiaozhi Wang^*, Feng Yao , Zimu Wang, and 4 more authors

   The 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP 2023 Demo), Dec 2023

   Abs arXiv HTML PDF

   Event understanding aims at understanding the content and relationship of events within texts, which covers multiple complicated information extraction tasks: event detection, event argument extraction, and event relation extraction. To facilitate related research and application, we present an event understanding toolkit OmniEvent, which features three desiderata: (1) Comprehensive. OmniEvent supports mainstream modeling paradigms of all the event understanding tasks and the processing of 15 widely-used English and Chinese datasets. (2) Fair. OmniEvent carefully handles the inconspicuous evaluation pitfalls reported in Peng et al. (2023), which ensures fair comparisons between different models. (3) Easy-to-use. OmniEvent is designed to be easily used by users with varying needs. We provide off-the-shelf models that can be directly deployed as web services. The modular framework also enables users to easily implement and evaluate new event understanding models with OmniEvent. The toolkit is publicly released along with the demonstration website and video.

2. BIBE 2023

   Attention-based Multimodal Bilinear Feature Fusion for Lung Cancer Survival Analysis

   Hongbin Na^* , Lilin Wang^*, Xinyao Zhuang, Jianfei He, and 3 more authors

   2023 IEEE 23rd International Conference on Bioinformatics and Bioengineering (BIBE 2023), Dec 2023

   Abs HTML

   Survival analysis (SA) is an essential task that aims to predict survival status and duration, determine individual and precise treatment strategies, and assess disease intensity and direction. However, the current research on multimodal SA has identified three unique challenges: inefficient cross-modal information integration, insufficient inter-modal key features, and noisy data. In this paper, we propose a novel SA framework, named Attention-based Multimodal Bilinear Feature Fusion (AMBF)-SA, to address the aforementioned challenges. Specifically, AMBF-SA first performs feature extraction with the off-the-shelf models on each modality separately, then fuses the features between multiple sources and modalities using our proposed AMBF method, and finally outputs the survival prediction by a multi-layer perception (MLP). Experimental results on the Non-small Cell Lung Cancer (NSCLC) Radiogenomics dataset demonstrate remark performance of AMBF-SA compared with the rest of the experimented models, including the models trained with single and combined modalities under the Mean Absolute Error (MAE) and the Concordance Index (C-index) evaluation metrics, indicating the usefulness of our proposed framework.

3. arXiv 2023

   When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks

   Hao Peng^* , Xiaozhi Wang^* , Jianhui Chen^* , Weikai Li, and 7 more authors

   arXiv preprint arXiv:2311.08993, Nov 2023

   Abs arXiv

   In-context learning (ICL) has become the default method for using large language models (LLMs), making the exploration of its limitations and understanding the underlying causes crucial. In this paper, we find that ICL falls short of handling specification-heavy tasks, which are tasks with complicated and extensive task specifications, requiring several hours for ordinary humans to master, such as traditional information extraction tasks. The performance of ICL on these tasks mostly cannot reach half of the state-of-the-art results. To explore the reasons behind this failure, we conduct comprehensive experiments on 18 specification-heavy tasks with various LLMs and identify three primary reasons: inability to specifically understand context, misalignment in task schema comprehension with humans, and inadequate long-text understanding ability. Furthermore, we demonstrate that through fine-tuning, LLMs can achieve decent performance on these tasks, indicating that the failure of ICL is not an inherent flaw of LLMs, but rather a drawback of existing alignment methods that renders LLMs incapable of handling complicated specification-heavy tasks via ICL. To substantiate this, we perform dedicated instruction tuning on LLMs for these tasks and observe a notable improvement. We hope the analyses in this paper could facilitate advancements in alignment methods enabling LLMs to meet more sophisticated human demands.

4. CCAI 2023

   Multi-level Adversarial Training for Stock Sentiment Prediction

   Zimu Wang, and Hong-Seng Gan

   2023 IEEE 3rd International Conference on Computer Communication and Artificial Intelligence (CCAI 2023), May 2023

   Abs HTML

   Stock sentiment prediction is a task to evaluate whether the investors are expecting or gaining a positive or negative return from a stock, which has a high correlation with investors’ sentiments towards the business. However, as the nature of social media, the textual information posted by ordinary people is usually noisy, inconsistent, and even grammatically incorrect, leading the model to generate unsatisfied predictions. In this paper, we improve the performance of stock sentiment prediction by applying and comparing adversarial training at multiple levels, including character, word, and sentence levels, with the utilization of three novel adversarial attack models: DeepWordBug, BAE, and Generative Adversarial Network (GAN). We also propose an effective pre-processing technique and a novel adversarial examples incorporation method to improve the prediction results. To make an objective evaluation, we select three backbone models: Embedding Bag, BERT, and RoBERTa-Twitter, and validate the models before and after adversarial training on the TweetFinSent dataset. Experimental results demonstrate remarkable improvements in the models after adversarial training, and the RoBERTa-Twitter model with word-level adversarial training performs optimally among the experimented models. We conclude that sentence-level and word-level adversarial training are the most appropriate for deep learning and pre-trained language models, respectively, and we further conduct ablation studies to highlight the usefulness of our data pre-processing and adversarial examples incorporation approaches and a case study to display the adversarial examples generated by the proposed adversarial attack models.

2022

1. EMNLP 2022

   MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction

   Xiaozhi Wang^* , Yulin Chen^*, Ning Ding, Hao Peng, and 8 more authors

   The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), Dec 2022

   Abs arXiv HTML PDF

   The diverse relationships among real-world events, including coreference, temporal, causal, and subevent relations, are fundamental to understanding natural languages. However, two drawbacks of existing datasets limit event relation extraction (ERE) tasks: (1) Small scale. Due to the annotation complexity, the data scale of existing datasets is limited, which cannot well train and evaluate data-hungry models. (2) Absence of unified annotation. Different types of event relations naturally interact with each other, but existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions. To address these issues, we construct a unified large-scale human-annotated ERE dataset MAVEN-ERE with improved annotation schemes. It contains 103,193 event coreference chains, 1,216,217 temporal relations, 57,992 causal relations, and 15,841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude. Experiments show that ERE on MAVEN-ERE is quite challenging, and considering relation interactions with joint learning can improve performances. The dataset and source codes can be obtained from https://github.com/THU-KEG/MAVEN-ERE.

2. ICICN 2022

   Generating Complex Questions from Knowledge Graphs with Query Graphs

   Zimu Wang

   2022 IEEE 10th International Conference on Information, Communication and Networks (ICICN 2022), Aug 2022

   Abs HTML

   Question Generation from Knowledge Graphs (KGQG) is a task that aims to generate natural language questions from subgraphs within the given Knowledge Graph. Previous research has discussed numerous KGQG approaches, generating simple and complex questions from triples and queries based on the template-based and Deep Learning-based models. However, the current KGQG research could be identified three unique challenges: determining how to represent the queries, mapping different query structures to different compositional structures in the natural language questions, and generating questions with high language variety. In this paper, we propose a novel framework to conduct the KGQG task, which consists of two stages: query graph construction and graph-to-question generation, to deal with the three identified challenges. Firstly, we propose a query graph structure that specifies the entities and relationships within a SPARQL query addition with the characteristic of each entity in order to explore an appropri-ate query graph representation. We then propose a Graph-to-Sequence (Graph2Seq) model that utilizes Gated Graph Neural Network (GGNN) to encode the constructed query graphs and an attention decoder with a Long Short-Term Memory (LSTM) network to generate natural language questions. To make an objective evaluation, we validate our framework on two compli-cated datasets designed for complex questioning and answering, namely LC-QuAD 2.0 and KQA Pro. Experimental results show a dramatic improvement in our framework compared with the Sequence-to-Sequence (Seq2Seq) baselines. The robustness of our graph structure and the Graph2Seq model in the KGQG tasks are all confirmed, accompanied by a case study to highlight the effectiveness of our proposed framework.