stumpyr - Demonstrating How to Run STUMPY in R

[seanlaw]

In reference to this sidebar conversation between @RamiKrispin and @seanlaw:

I'd like to discuss:

1. What is the general interest?
2. What needs to be done in order create a short tutorial (or some other output) that demonstrates how to use STUMPY in R using Reticulate (or some other combination)?
3. How much time/effort will be required?
4. Is it worth it?

There is absolutely no pressure and no rush but I wanted to capture this here.

[seanlaw]

@RamiKrispin I am a total n00b when it comes to R so I'd defer your expertise here and am ready to learn :)

[Steviey]

Very good idea!

[seanlaw]

@Steviey We'd also welcome new contributions if you have any experience with this!

[frankiethull]

@seanlaw did you all ever do this?

I'll reach out on the twitter/X post as well

[seanlaw]

@frankiethull That looks great! From what I can tell, one only needs a few lines of code if reticulate is used? Would this be a minimum reproducible example?

library(reticulate)
# py_install("stumpy", pip = TRUE)
stumpyr <- reticulate::import("stumpy")

m  <- as.integer(50)
df <- as.data.frame(matrix(runif(n=10000, min=0.0, max=1.0), nrow=10000))
mp <- stumpyr$stump(df$data, m)

Would you happen to know what you'd need to do to get a class working? I was thinking about this scrump example. It's not clear to me how one accesses class methods/attributes

[frankiethull]

minimum example would look like this:

stumpyr <- reticulate::import("stumpy")

m  <- as.integer(50)
df <- as.data.frame(matrix(runif(n=10000, min=0.0, max=1.0), nrow=10000))
mp <- stumpyr$stump(df$V1, m)
# OR
mp <- stumpyr$stump(df[,1], m)
# OR
mtx <- matrix(runif(n=10000, min=0.0, max=1.0), nrow=10000)
mp  <- stumpyr$stump(mtx[, 1], m)

df$data won't quite work in R. You can check in R using str() to see what the object structure is and type in R.

For the as.data.frame() this converts your column names to V1, V2, V3, etc. Weird R stuff, but you can reference the column names with a $ or could use [ , ] indexing where the first column is 1, not 0.

As for the class methods/attributes, I think you are referring to, I had to play with this for a second:

### class/method/attributes ----
numpyr  <- reticulate::import("numpy")
approx <- stumpyr$scrump(T_A = numpyr$array(mtx[,1]), m = m, percentage=0.01, pre_scrump=FALSE)

the booleans TRUE/FALSE work, the fact that scrump checks for shape, have to wrap the numeric vector with a numpy.array for the function to work.

These tests work for me. Personally, I'll probably write some custom functions and be ready to use this internally. I guess a stumpyr package would for the general R community would need a tidy.stumpyr wrapper package to help with a lot of these formats between R and Python.

Another example, the mp that is returned to R is a matrix where every value in the matrix is a list of length one. Which required grabbing each array, then unlist() to parse the results for use in R.

session info reference for other testers:

sessionInfo()
R version 4.3.0 (2023-04-21 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] BobRossColors_0.0.0.9000 dplyr_1.1.2 ggplot2_3.4.2 reticulate_1.30

loaded via a namespace (and not attached):
[1] Matrix_1.5-4 gtable_0.3.3 jsonlite_1.8.5 crayon_1.5.2 compiler_4.3.0 tidyselect_1.2.0
[7] Rcpp_1.0.10 JuliaCall_0.17.5 gridExtra_2.3 tidyr_1.3.0 scales_1.2.1 png_0.1-8
[13] lattice_0.21-8 here_1.0.1 R6_2.5.1 labeling_0.4.2 generics_0.1.3 knitr_1.43
[19] tibble_3.2.1 munsell_0.5.0 rprojroot_2.0.3 lubridate_1.9.2 pillar_1.9.0 rlang_1.1.1
[25] utf8_1.2.3 xfun_0.39 pkgload_1.3.2 timechange_0.2.0 cli_3.6.1 withr_2.5.0
[31] magrittr_2.0.3 grid_4.3.0 rstudioapi_0.14 rappdirs_0.3.3 lifecycle_1.0.3 vctrs_0.6.2
[37] glue_1.6.2 farver_2.1.1 fansi_1.0.4 colorspace_2.1-0 purrr_1.0.1 tools_4.3.0
[43] pkgconfig_2.0.3

[frankiethull]

@seanlaw -

Here's what I mean for a "tidy" wrapper function to handle the conversions for the user internally. This is how I plan on showing the examples to my team:

### tidy.stumpyr example ---

# lazy aliasing w/ conversions ---
tidy.stump <- function(T_A, m, ...){
stumpyr <- reticulate::import("stumpy")
  
        m <- as.integer(m) # handle conversion for R users
        
 mp <-  stumpyr$stump(T_A, m, ...)
 
 # handle issue where each val is a list of len 1:
 # return a tibble
 tidy_mp <- mp |> 
            as.data.frame() |> 
            dplyr::mutate(index = row_number()) |> 
            tidyr::unnest(cols = V1:V4) |> 
            select(-index) |>
            as_tibble()
 
 return(tidy_mp)
}

m  <- 50
df <- as.data.frame(matrix(runif(n=10000, min=0.0, max=1.0), nrow=10000))

mp <- tidy.stump(df[, 1], m)

[seanlaw]

Thanks for sharing @frankiethull. Are there any next steps that we need to take? Or do we think this is sufficient for others to discover?

[frankiethull]

@seanlaw - I think this python library is a great toolkit addition for anyone using R for data science or analysis.

I am wondering the best way to share the idea with others. Not sure if we should post an article or additional posts on social media or if this is sufficient? I did make an X post of the tutorial with little engagement, but there was also a large exodus of R programmers from Twitter in the last year. I'm open to other ideas to help R users discover this quick and easy. There is always the package route which might help..

Feel free to ping me if you want to test out more functions or have other thoughts or suggestions. I've been messing with {reticulate} for awhile and there are always nuances. A lot of R programmers avoid {reticulate} & Python, I'm trying to showcase the power of combining Python & R systems so I'm always open to collaborating.