Publications

2026

1. OmniFall: A Unified Staged-to-Wild Benchmark for Human Fall Detection

   David Schneider, Zdravko Marinov, Rafael Baur, Zeyun Zhong, Rodi Düger, and Rainer Stiefelhagen

   In European Conference on Computer Vision (ECCV), 2026

   arXiv Code

2. 

   Rethinking Expressivity and Efficiency in Test-Time Training

   Zeyun Zhong, Joya Chen, Manuel Martin, Frederik Diederichs, Juergen Gall, and Jürgen Beyerer

   2026

   Under Review

   Abs

   Test-Time Training (TTT) enables long-context processing via continuous weight updates during inference, but current methods struggle to balance the expressivity of per-token update dynamics with the hardware efficiency of chunk-wise approximations. We propose E²-TTT (Expressive and Efficient TTT) to bridge this gap. By deriving a closed-form state transition that exactly aggregates per-token momentum and decay coefficients within a chunk, E²-TTT enables fully parallelized chunk-level training while preserving the temporal structure of the update rule that prior chunk-wise methods discard. We validate E²-TTT by training models up to 1.3B parameters from scratch. Extensive experiments demonstrate that our method consistently outperforms previous TTT and hybrid attention baselines in language modeling and retrieval, while achieving significantly better extrapolation on the standard “Needle in a Haystack” test, maintaining > 90% accuracy on passkey retrieval at the training context length. Meanwhile, E²-TTT can match the training throughput of efficient chunk-wise methods, demonstrating that it effectively reconciles expressivity with efficiency.

3. 

   FlowNar: Scalable Streaming Narration for Long-Form Videos

   Zeyun Zhong, Manuel Martin, Chengzhi Wu, David Schneider, Frederik Diederichs, Juergen Gall, and 1 more author

   In International Conference on Machine Learning (ICML), 2026

   Abs arXiv Code Website

   Recent Large Multimodal Models (LMMs), primarily designed for offline settings, are ill-suited for the dynamic requirements of streaming video. While recent online adaptations improve real-time processing, they still face critical scalability challenges, with resource demands typically growing at least linearly with video duration. To overcome this bottleneck, we propose FlowNar, a novel framework for scalable streaming video narration. The core of FlowNar is a dynamic context management strategy for historical visual context removal, combined with our CLAM (Cross Linear Attentive Memory) module for streaming visual history retention, ensuring bounded visual memory usage and computational complexity, crucial for efficient streaming. We also introduce a realistic self-conditioned evaluation protocol and complementary evaluation metrics to assess streaming narration models under deployment-like conditions. Experiments on the Ego4D, EgoExo4D, and EpicKitchens100 datasets demonstrate that FlowNar substantially improves narration quality over strong baselines while being highly efficient, supporting processing of 10x longer videos and achieving 3x higher throughput (FPS).

4. 

   Scalable Video Action Anticipation with Cross Linear Attentive Memory

   Zeyun Zhong, Manuel Martin, David Schneider, David J Lerch, Chengzhi Wu, Frederik Diederichs, and 2 more authors

   In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2026

   Abs PDF Code

   Recent advances in action anticipation rely heavily on Transformer architectures to learn discriminative representations of the past observation, incurring high computational and memory overhead that limits their applicability to long videos. While temporal processors with linear complexity like RNNs and state-space models offer efficient alternatives, their sequential nature risks overlooking subtle cues in observed frames that could enhance future anticipation. We address this limitation with Cross Linear Attentive Memory (CLAM), a memory module that selectively retrieves complementary context cues from frame features. By reformulating linear attention to replace traditional cross attention, CLAM achieves linear computation complexity and constant memory usage relative to input length. Finally, by fusing the outputs of the temporal processor and CLAM, a non-autoregressive Transformer decoder generates future actions in one shot with high accuracy. Experiments on egocentric (EpicKitchens100 and Ego4D) and third-person (Thumos14) benchmarks demonstrate our model’s superior anticipation accuracy and scalability, processing longer sequences with significantly less latency growth than alternatives. Our approach also achieves promising results in online action detection.

5. IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly

   Di Wen, Zeyun Zhong, David Schneider, Manuel Zaremski, Linus Kunzmann, Yitian Shi, and 5 more authors

   arXiv preprint arXiv:2604.10409, 2026

   arXiv Code Website
6. Multi-modal Video Representation Alignment for Robust Self-supervised Driver Distraction Detection

   David J Lerch, Livien Majer, Zeyun Zhong, Manuel Martin, Frederik Diederichs, and Rainer Stiefelhagen

   In IEEE Intelligent Transportation Systems Conference (ITSC), 2026

   arXiv
7. IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory

   Weitong Kong, Di Wen, Kunyu Peng, David Schneider, Zeyun Zhong, Alexander Jaus, and 5 more authors

   In IEEE Systems, Man, and Cybernetics Society (SMC), 2026

   arXiv
8. IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning

   Qian Yin, Di Wen, Kunyu Peng, David Schneider, Zeyun Zhong, Alexander Jaus, and 5 more authors

   In IEEE Systems, Man, and Cybernetics Society (SMC), 2026

   arXiv
9. IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction

   Haoshen Zhang, Di Wen, Kunyu Peng, David Schneider, Zeyun Zhong, Alexander Jaus, and 5 more authors

   In IEEE Systems, Man, and Cybernetics Society (SMC), 2026

   arXiv

2025

1. SAMBLE: Shape-specific point cloud sampling for an optimal trade-off between local detail and global uniformity

   Chengzhi Wu, Yuxin Wan, Hao Fu, Julius Pfrommer, Zeyun Zhong, Junwei Zheng, and 2 more authors

   In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
2. Learning Robust Aligned Representations Across Multiple Visual Modalities in Human Action Recognition

   David Lerch, Bastian Rothenburger, Zeyun Zhong, Manuel Martin, Frederik Diederichs, and Rainer Stiefelhagen

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025

2024

1. HybridFormer: Bridging Local and Global Spatio-Temporal Dynamics for Efficient Skeleton-Based Action Recognition

   Zeyun Zhong, Tianrui Li, Manuel Martin, Mickael Cormier, Chengzhi Wu, Frederik Diederichs, and 1 more author

   In European Conference on Computer Vision (ECCV) Workshops, 2024

   Code

2. 

   Unsupervised 3D Skeleton-Based Action Recognition Using Cross-Attention With Conditioned Generation Capabilities

   David J Lerch*, Zeyun Zhong*, Manuel Martin, Michael Voit, and Jürgen Beyerer

   In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2024

   Abs Code

   Human action recognition plays a pivotal role in various real-world applications, including surveillance systems, robotics, and occupant monitoring in the car interior. With such a diverse range of domains, the demand for generalization becomes increasingly crucial. In this work, we propose a cross-attention-based encoder-decoder approach for unsupervised 3D skeleton-based action recognition. Specifically, our model takes a skeleton sequence as input for the encoder and further applies masking and noise to the original sequence for the decoder. By training the model to reconstruct the original skeleton sequence, it simultaneously learns to capture the underlying patterns of actions. Extensive experiments on NTU and NW-UCLA datasets demonstrate the state-of-the-art performance as well as the impressive generalizability of our proposed approach. Moreover, our experiments reveal that our approach is capable of generating conditioned skeleton sequences, offering the potential to enhance small datasets or generate samples of under-represented classes in imbalanced datasets

3. Activities that correlate with motion sickness in driving cars–an international online survey

   Frederik Diederichs, Amina Herrmanns, David Lerch, Zeyun Zhong, Daniela Piechnik, Lesley-Ann Mathis, and 5 more authors

   In International Conference on Human-Computer Interaction, 2024
4. Querymamba: A mamba-based encoder-decoder architecture with a statistical verb-noun interaction module for video action forecasting@ ego4d long-term action anticipation challenge 2024

   Zeyun Zhong, Manuel Martin, Frederik Diederichs, and Juergen Beyerer

   arXiv preprint arXiv:2407.04184, 2024

   Code
5. Rethinking attention module design for point cloud analysis

   Chengzhi Wu, Kaige Wang, Zeyun Zhong, Hao Fu, Junwei Zheng, Jiaming Zhang, and 2 more authors

   In International Conference on Pattern Recognition (ICPR), 2024
6. Synthact: towards generalizable human action recognition based on synthetic data

   David Schneider, Marco Keller, Zeyun Zhong, Kunyu Peng, Alina Roitberg, Jürgen Beyerer, and 1 more author

   In 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024

2023

1. 

   DiffAnt: Diffusion Models for Action Anticipation

   Zeyun Zhong, Chengzhi Wu, Manuel Martin, Michael Voit, Juergen Gall, and Jürgen Beyerer

   arXiv preprint arXiv:2311.15991, 2023

   Abs arXiv

   Anticipating future actions is inherently uncertain. Given an observed video segment containing ongoing actions, multiple subsequent actions can plausibly follow. This uncertainty becomes even larger when predicting far into the future. However, the majority of existing action anticipation models adhere to a deterministic approach, neglecting to account for future uncertainties. In this work, we rethink action anticipation from a generative view, employing diffusion models to capture different possible future actions. In this framework, future actions are iteratively generated from standard Gaussian noise in the latent space, conditioned on the observed video, and subsequently transitioned into the action space. Extensive experiments on four benchmark datasets, i.e., Breakfast, 50Salads, EpicKitchens, and EGTEA Gaze+, are performed and the proposed method achieves superior or comparable results to state-of-the-art methods, showing the effectiveness of a generative approach for action anticipation. Our code and trained models will be published on GitHub.

2. 

   A Survey on Deep Learning Techniques for Action Anticipation

   Zeyun Zhong, Manuel Martin, Michael Voit, Juergen Gall, and Jürgen Beyerer

   Under review at IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023

   Abs arXiv

   The ability to anticipate possible future human actions is essential for a wide range of applications, including autonomous driving and human-robot interaction. Consequently, numerous methods have been introduced for action anticipation in recent years, with deep learning-based approaches being particularly popular. In this work, we review the recent advances of action anticipation algorithms with a particular focus on daily-living scenarios. Additionally, we classify these methods according to their primary contributions and summarize them in tabular form, allowing readers to grasp the details at a glance. Furthermore, we delve into the common evaluation metrics and datasets used for action anticipation and provide future directions with systematical discussions.

3. 

   Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation

   Zeyun Zhong*, David Schneider*, Michael Voit, Rainer Stiefelhagen, and Jürgen Beyerer

   In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023

   Abs arXiv Code

   Although human action anticipation is a task which is inherently multi-modal, state-of-the-art methods on well known action anticipation datasets leverage this data by applying ensemble methods and averaging scores of unimodal anticipation networks. In this work we introduce transformer based modality fusion techniques, which unify multi-modal data at an early stage. Our Anticipative Feature Fusion Transformer (AFFT) proves to be superior to popular score fusion approaches and presents state-of-the-art results outperforming previous methods on EpicKitchens-100 and EGTEA Gaze+. Our model is easily extensible and allows for adding new modalities without architectural changes. Consequently, we extracted audio features on EpicKitchens-100 which we add to the set of commonly used features in the community.

2021

1. 

   Mixed probability models for aleatoric uncertainty estimation in the context of dense stereo matching

   Zeyun Zhong, and Max Mehltretter

   ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021

   Abs Code

   The ability to identify erroneous depth estimates is of fundamental interest. Information regarding the aleatoric uncertainty of depth estimates can be, for example, used to support the process of depth reconstruction itself. Consequently, various methods for the estimation of aleatoric uncertainty in the context of dense stereo matching have been presented in recent years, with deep learningbased approaches being particularly popular. Among these deep learning-based methods, probabilistic strategies are increasingly attracting interest, because the estimated uncertainty can be quantified in pixels or in metric units due to the consideration of real error distributions. However, existing probabilistic methods usually assume a unimodal distribution to describe the error distribution while simply neglecting cases in real-world scenarios that could violate this assumption. To overcome this limitation, we propose two novel mixed probability models consisting of Laplacian and Uniform distributions for the task of aleatoric uncertainty estimation. In this way, we explicitly address commonly challenging regions in the context of dense stereo matching and outlier measurements, respectively. To allow a fair comparison, we adapt a common neural network architecture to investigate the effects of the different uncertainty models. In an extensive evaluation using two datasets and two common dense stereo matching methods, the proposed methods demonstrate state-of-the-art accuracy.