Peng Zhou

RWKV: Reinventing RNNs for the Transformer Era

2023 · arXiv (Cornell University)

Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence length. In contrast, recurrent neural networks (RNNs) exhibit linear scaling in memory and …

DeepRisk network: an AI-based tool for digital pathology signature and treatment responsiveness of gastric cancer using whole-slide images

2024 · Journal of Translational Medicine

Abstract Background Digital histopathology provides valuable information for clinical decision-making. We hypothesized that a deep risk network (DeepRisk) based on digital pathology signature (DPS) derived from whole-slide images could improve the prognostic value of the …

A Multi-Source Log Hidden Anomaly Detection Method Integrating Trans-Encoder and LSTM

2025 · IEEE Access

Identifying hidden anomalous behavior is a major challenge in anomaly detection, particularly in complex systems where anomalies are buried within massive log data and cannot be easily identified through simple patterns or rules. To address …

Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification

2016

Relation classification is an important semantic processing task in the field of natural language processing (NLP). State-ofthe-art systems still rely on lexical resources such as WordNet or NLP systems like dependency parser and named entity …

Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme

2017 · arXiv (Cornell University)

Joint extraction of entities and relations is an important task in information extraction. To tackle this problem, we firstly propose a novel tagging scheme that can convert the joint extraction task to a tagging problem. …

K-BERT: Enabling Language Representation with Knowledge Graph

2020 · Proceedings of the AAAI Conference on Artificial Intelligence

Pre-trained language representation models, such as BERT, capture a general language representation from large-scale corpora, but lack domain-specific knowledge. When reading a domain text, experts make inferences with relevant knowledge. For machines to achieve this …

FastBERT: a Self-distilling BERT with Adaptive Inference Time

2020

Pre-trained language models like BERT have proven to be highly performant. However, they are often computationally expensive in many practical scenarios, for such heavy models can hardly be readily implemented with limited resources. To improve …