Visual Exploration of Time Series in R
© 2015 Anton Lebedevich
About Me
Anton Lebedevich
- back-end developer of scalable services
- debugging distributed performance problems
- started analyzing performance metrics and loved stats
blog about monitoring and time series mabrek.github.io
Agenda
- performance monitoring data
- SVD and PCA
- MDS and t-SNE
- Shiny: web apps in R
- time series libraries
Typical Load Test
simulated clients
Results
and 32000 more time series to find error causes
but they are silent or quite similar to each other
SVD
wikipedia.org/wiki/Singular_value_decomposition
$ \mathbf{X} = \mathbf{U}\mathbf{D}\mathbf{V}^* $
SVD U
SVD D
PCA
wikipedia.org/wiki/Principal_component_analysis
$ \mathbf{T} = \mathbf{X}\mathbf{V} $
$ \mathbf{T} = \mathbf{U}\mathbf{D} $
$ \mathbf{X} = \mathbf{U}\mathbf{D}\mathbf{V}^* $
SVD and PCA
- relatively fast fortran implementation (but still $ O(n^2) $)
- results make sense (most common shapes), but not always
- sensitive to outliers
- unclear how to scale data
MDS
wikipedia.org/wiki/Multidimensional_scaling
R function cmdscale
$ 1 - |corr(a, b)| $ used as distance between time series
there are many ways to do it (isoMDS, TSdist, ...)
MDS
MDS
MDS
MDS
MDS
MDS
MDS
t-SNE
wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding
t-SNE
t-SNE
t-SNE
t-SNE
t-SNE
t-SNE
shiny.rstudio.com
A web application framework for R
user doesn't see source code
there is no saved results, they are calculated on the fly
Libraries
- zoo, xts
- parallel
- ggplot2
- forecast
- strucchange
- tsne
- shiny
- dygraphs
Summary
- performance monitoring data is a lot of time series
- SVD and PCA help to find common shapes
- MDS and t-SNE allow to explore visually grouped data
- Shiny makes simple web demos
- R has good time series libraries
Q&A
Anton Lebedevich
mabrek@gmail.com
this presentation: