Transform pandas dataframe with time series Index to format that is accepted by sktime

[till90]

Hi,

i don't get the right format. I get this error Message when try to make this

X = df_model[['CO2 [ppm]', 'Lufttemperatur [°C]', 'Luftfeuchte [%]']]
y = df_model['Rn-222 [Bq/m³]']
X_train, X_test, y_train, y_test = temporal_train_test_split(X_, y, test_size=0.2)
# Random Forest-Modell erstellen
model = ComposableTimeSeriesForestRegressor(n_estimators=100, random_state=42)
# Modell anpassen
model.fit(X_train, y_train)

TypeError: X must be in an sktime compatible format. Allowed scitypes for regressors are Panel mtypes, for instance a pandas.DataFrame with MultiIndex and last(-1) level an sktime compatible time index. Allowed compatible mtype format specifications are: ['nested_univ', 'numpy3D', 'numpyflat', 'pd-multiindex', 'pd-wide', 'pd-long', 'df-list', 'dask_panel'] . See the data format tutorial examples/AA_datatypes_and_datasets.ipynb. If you think the data is already in an sktime supported input format, run sktime.datatypes.check_raise(data, mtype) to diagnose the error, where mtype is the string of the type specification you want. Error message for checked mtypes, in format [mtype: message], as follows: [df-list: obj must be list of pd.DataFrame, found <class 'pandas.core.frame.DataFrame'>] [numpy3D: obj must be a numpy.ndarray, found <class 'pandas.core.frame.DataFrame'>] [pd-multiindex: obj must have a MultiIndex, found <class 'pandas.core.indexes.range.RangeIndex'>] [nested_univ: obj All columns must be object, found <class 'pandas.core.frame.DataFrame'>]

Please can you give me a hint how to transform data from the format of

check_raise(df_model, mtype='pd.DataFrame') says True.

I dont really get it what is the desired format for e.G. the model = ComposableTimeSeriesForestRegressor()

If i use datatypes.MTYPE_REGISTER it says 'pd.DataFrame' is a mtype.

Thanks for helping me :)

[fkiraly]

You are attempting time series regression, that is the learning task where you have, say, 142 time series with 142 labels, e.g., 142 temperature/pressure curves and 142 instances of "good", "bad", "other", etc. In sktime parlance, collections of time series like this (the 142) form a Panel.

What your screenshot shows is a single time series, which is Series abstract data type, and the pd.DataFrame format that you are complying with is a vlaid format to represent a single time series. However, for time series regression you need multiple series, and those need to be in a Panel format, e.g., pd-multiindex.

What I would suggest to do:

1. first, check whether your learning task is inded time series regression. That is, do you have labels that label entire time series, and do you have many time series, with one label per time series?
2. if yes, use check_raise to get your time series into the pd-multiindex format. This has a row multi-index, with index 0 being the index of time series, and index 1 being the time stamp (or another Panel format)

(if no, you need to identify your learning task first)

[ybenazouz]

I attempted creating a panel format for my multivariate time series data. It consists of a dataframe where each cell holds a series with the numerical time series values, making multiindexing unnecessary. Can this shape work in sktime, or is it too cumbersome? Should the time series be in one matrix all together instead of separately?

[fkiraly]

@ybenazouz, that sounds like the nested_univ format - using nested pandas is risky, see discussion here: #6415

We hence recommend to use one of the other formats, discussed above.

[ybenazouz]

Thanks so much for the quick reply.
My time series deal with varying sample frequencies between the different channels of a sample and varying recording times between samples. Another post suggested using masked NumPy arrays to deal with the problem.

Would you recommend using masked arrays in combination with a numpy3D or pd-multiindex structure?

[fkiraly]

masked arrays are currently not supported, pd-multiindex should work