Lune

VLDB2025Top-tier venue

Goku: A Schemaless Time Series Database for Large Scale Monitoring at Pinterest

Monil Mukesh Sanghavi, Ming-May Hu, Zhenxiao Luo, Xiao Li, Kapil Bajaj

Abstract

Engineers rely heavily on observability tools to monitor their business and system metrics and set up alerting on it. A reliable and efficient monitoring system is very important for development velocity. In this paper, we introduce Goku, a time series database (TSDB) we built from the ground up at Pinterest. Over the years, we have studied user patterns and common requests to constantly evolve Goku to store and serve the use cases at Pinterest with high efficiency and reduced costs. At its core, Goku uses tiered storage to store new and frequently queried metrics data in memory while leveraging solid state drive (SSD) and hard disks (HDD) for older data. Goku aggregates metrics data at write time while also rolling up datapoints with lower time granularity to low latency to certain use cases. Goku also supports modifying configurations on metrics data like time to live (TTL), rollup granularity, backfilling capability, etc. Using multiple replicas and AWS S3 as backup, Goku is highly available and fault tolerant.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Related papers

  1. Heracles: An Efficient Storage Model And Data Flushing For Performance Monitoring TimeseriesZhiqi Wang, Jin Xue, Zili ShaoVLDB 2021 · 14 citations
  2. ForestTI: A Scalable Inverted-Index-Oriented Timeseries Management System with Flexible Memory EfficiencyZhiqi Wang, Zili ShaoSIGMOD 2023 · 2 citations
  3. An Efficient Cloud Storage Model with Compacted Metadata Management for Performance Monitoring Timeseries SystemsKai Zhang, Tianyu Wang, Zili ShaoFAST 2026
  4. Time Series Data Encoding for Efficient Storage: A Comparative Analysis in Apache IoTDBJinzhao Xiao, Yuxiang Huang, Changyu Hu, Shaoxu Song et al.VLDB 2022 · 37 citations
  5. STsCache: An Efficient Semantic Caching Scheme for Time-series Data Workloads Based on Hybrid StorageTao Kong, Hui Li, Yuxuan Zhao, Liping Li et al.VLDB 2025
Dusk over the sea between two cliffs drawn in fine vertical lines