-
Notifications
You must be signed in to change notification settings - Fork 14
/
README.Rmd
103 lines (80 loc) · 2.39 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# sparkxgb
<!-- badges: start -->
[![R-CMD-check](https://github.com/rstudio/sparkxgb/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/rstudio/sparkxgb/actions/workflows/R-CMD-check.yaml)
[![Spark Tests](https://github.com/rstudio/sparkxgb/actions/workflows/Tests.yaml/badge.svg)](https://github.com/rstudio/sparkxgb/actions/workflows/Tests.yaml)
[![Codecov test coverage](https://codecov.io/gh/rstudio/sparkxgb/branch/main/graph/badge.svg)](https://app.codecov.io/gh/rstudio/sparkxgb?branch=main)
[![CRAN status](https://www.r-pkg.org/badges/version/sparkxgb)](https://CRAN.R-project.org/package=sparkxgb)
<!-- badges: end -->
## Overview
**sparkxgb** is a [sparklyr](https://spark.posit.co/) extension that provides
an interface to [XGBoost](https://github.com/dmlc/xgboost) on Spark.
## Installation
```r
install.packages("sparkxgb")
```
### Development version
You can install the development version of `sparkxgb` with:
``` r
# install.packages("pak")
pak::pak("rstudio/sparkxgb")
```
## Example
**sparkxgb** supports the familiar formula interface for specifying models:
```{r, message = FALSE}
library(sparkxgb)
library(sparklyr)
library(dplyr)
sc <- spark_connect(master = "local")
iris_tbl <- sdf_copy_to(sc, iris)
xgb_model <- xgboost_classifier(
iris_tbl,
Species ~ .,
num_class = 3,
num_round = 50,
max_depth = 4
)
xgb_model %>%
ml_predict(iris_tbl) %>%
select(Species, predicted_label, starts_with("probability_")) %>%
glimpse()
```
It also provides a Pipelines API, which means you can use a `xgboost_classifier`
or `xgboost_regressor` in a pipeline as any `Estimator`, and do things like
hyperparameter tuning:
```{r}
pipeline <- ml_pipeline(sc) %>%
ft_r_formula(Species ~ .) %>%
xgboost_classifier(num_class = 3)
param_grid <- list(
xgboost = list(
max_depth = c(1, 5),
num_round = c(10, 50)
)
)
cv <- ml_cross_validator(
sc,
estimator = pipeline,
evaluator = ml_multiclass_classification_evaluator(
sc,
label_col = "label",
raw_prediction_col = "rawPrediction"
),
estimator_param_maps = param_grid
)
cv_model <- cv %>%
ml_fit(iris_tbl)
summary(cv_model)
spark_disconnect(sc)
```