[data interfacing] save profile data at the server-side (backend)

[kswang1029]

as of v4-stable, CARTA can only save profile data (spatial / spectral / histogram) as a text file at the client-side. This introduces a usability issue about having data in multiple places (ie client side and server side). Therefore it becomes less convenient to work with data and interface with other tools.

[markccchiang]

Does it need a corresponding frontend development?

[kswang1029]

Does it need a corresponding frontend development?

Yes CARTAvis/carta-frontend#2303

[markccchiang]

I guess we need to add a new CARTA::EventType that is sent from the frontend to the backend and requests spatial, spectral, or histogram data from the backend based on a specific file ID and region ID. So the frontend can use these data for further analysis. @kswang1029 is that correct?

[kswang1029]

I guess we need to add a new CARTA::EventType that is sent from the frontend to the backend and requests spatial, spectral, or histogram data from the backend based on a specific file ID and region ID. So the frontend can use these data for further analysis. @kswang1029 is that correct?

I imagine we need a message sent from the frontend to the backend on what data to be saved at the server side. Then the backend can generate the relevant data accordingly (or fetch from cache (preferred for performance reasons)) and save them directly. Then users can access those saved data by ssh-logging to the server where the CARTA backend runs.

[markccchiang]

I imagine we need a message sent from the frontend to the backend on what data to be saved on the server side. Then the backend can generate the relevant data accordingly (or fetch from cache (preferred for performance reasons)) and save them directly. Then users can access those saved data by ssh-logging into the server where the CARTA backend runs.

We may need to modify the Export data widget on the spatial/spectral/histogram view panel on the frontend and add a new option to export the data to the server side. The frontend can use the FILE_LIST_REQUEST command to browse server directories and users can choose where to save the data. Do we need an option that allows users to create a new folder on the server and save the data if they have permission? A new protobuf message sent from the frontend to the backend for exporting data may be like this:

message ExportData {
    // File directory
    string directory = 1;
    // File name
    string name = 2;
    // Comments (spatial/spectral/histogram profiles, x-label, y-label, region box coordinates, etc.)
    repeated string comments = 3;
    // Data (x, y)
    repeated string data = 5;
}

Although the backend has caches for profile raw data, users may want to save the data after smoothing (processed y-value), or with different coordinate values (processed x-value). So far these steps are done on the frontend (except for the histogram). This is a fast and straightforward way. It is also 100% consistent with the exported data to the client site. Although we may have a performance issue when the exported data is very large.