SampleHST-X: a point and collective anomaly-aware trace sampling pipeline with approximate half space trees
File(s)
Published version
Author(s)
Type
Journal Article
Abstract
The storage requirement for distributed tracing can be reduced significantly by sam pling only the anomalous or interesting traces that occur rarely at runtime. In this
paper, we introduce an unsupervised sampling pipeline for distributed tracing that
ensures high sampling accuracy while reducing the storage requirement. The pro posed method, SampleHST-X, extends our recent work SampleHST. It operates
based on a budget which limits the percentage of traces to be sampled while adjust ing the storage quota of normal and anomalous traces depending on the size of this
budget. The sampling process relies on accurately defining clusters of normal and
anomalous traces by leveraging the distribution of mass scores, which characterize
the probability of observing different traces, obtained from a forest of Half Space
Trees (HST). In our experiments, using traces from a cloud data center, SampleHST
yields 2.3× to 9.5× better sampling performance. SampleHST-X further extends the
SampleHST approach by incorporating a novel class of Half Space Trees, namely
Approximate HST, that uses approximate counters to update the mass scores. These
counters significantly reduces the space requirement for HST while the sampling
performance remains similar. In addition to this extension, SampleHST-X includes
a Family of Graph Spectral Distances (FGSD) based trace characterization compo nent, which, in addition to point anomalies, enables it to sample traces with collec tive anomalies. For such traces, we observe that the SampleHST-X approach can
yield 1.2× to 19× better sampling performance.
paper, we introduce an unsupervised sampling pipeline for distributed tracing that
ensures high sampling accuracy while reducing the storage requirement. The pro posed method, SampleHST-X, extends our recent work SampleHST. It operates
based on a budget which limits the percentage of traces to be sampled while adjust ing the storage quota of normal and anomalous traces depending on the size of this
budget. The sampling process relies on accurately defining clusters of normal and
anomalous traces by leveraging the distribution of mass scores, which characterize
the probability of observing different traces, obtained from a forest of Half Space
Trees (HST). In our experiments, using traces from a cloud data center, SampleHST
yields 2.3× to 9.5× better sampling performance. SampleHST-X further extends the
SampleHST approach by incorporating a novel class of Half Space Trees, namely
Approximate HST, that uses approximate counters to update the mass scores. These
counters significantly reduces the space requirement for HST while the sampling
performance remains similar. In addition to this extension, SampleHST-X includes
a Family of Graph Spectral Distances (FGSD) based trace characterization compo nent, which, in addition to point anomalies, enables it to sample traces with collec tive anomalies. For such traces, we observe that the SampleHST-X approach can
yield 1.2× to 19× better sampling performance.
Date Issued
2024-04-16
Date Acceptance
2024-03-13
Citation
Journal of Network and Systems Management, 2024, 32 (3)
ISSN
1064-7570
Publisher
Springer
Journal / Book Title
Journal of Network and Systems Management
Volume
32
Issue
3
Copyright Statement
© The Author(s) 2024 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Subjects
Anomaly detection
Computer Science
Computer Science, Information Systems
Distributed tracing
Microservices
Sampling
Science & Technology
Technology
Telecommunications
Publication Status
Published
Article Number
44
Date Publish Online
2024-04-16