Does Memory Need Graphs? A Unified Framework and Empirical Analysis for Long-Term Dialog Memory

Graph structures are increasingly adopted in dialog memory systems, motivated by their success in retrieval-augmented generation and the associative nature of human memory. However, empirical findings on their effectiveness remain inconsistent, making it unclear which design choices truly matter. In this work, we present an experimental and system-oriented analysis of long-term dialog memory architectures. We formalize a unified framework that decomposes dialog memory systems into core components and supports both graph-based and non-graph approaches. Under this framework, we conduct controlled, stage-wise experiments on LongMemEval and HaluMem, comparing common design choices in memory representation, organization and maintenance, as well as indexing and retrieval. Our results show that underlying implementation details-often insufficiently specified in prior work-have a substantial impact on performance, and we identify stable, reliable strong baselines to support fair comparison and practical deployment. Code are available at https: //github.com/AvatarMemory/UnifiedMem