Semantics-Aware Patch Encoding and Hierarchical Dependency Modeling for Long-Term Time Series Forecasting

Time series forecasting is a vital task with widespread applications.While recent advancements have adopted patching to enrich shortterm context, existing encoding methods often struggle to capture the diverse semantics within patches, resulting in semantic information loss and limited model performance.Moreover, most long-term dynamics modeling approaches rely on homogeneous architectures with fixed receptive fields, inevitably sacrificing either performance or efficiency.To address these challenges, we propose Mixture of Universals (MoU), a novel framework designed to prevent semantic loss during patch encoding and efficiently enhance long-term dynamics through a hybrid approach.Specifically, MoU is consist of two novel designs: Mixture of Feature Extractors (MoF) and Mixture of Architectures (MoA).MoF introduces a semantics-aware encoding mechanism that selectively activates the corresponding subextractor based on the semantic context of input patches, preserving diverse temporal patterns and mitigating information loss.MoA, on the other hand, hierarchically captures long-term dependency with progressively expanded receptive field, improving model performance while maintaining relatively low computational costs.We conducted extensive experiments on seven real-world datasets, and the results demonstrate the superiority of our model.Our Code is available at https://github.com/lunaaa95/mou/.