[PERF] Workflow Graph Loading Enhancement

[sfmskywalker]

The Problem

The current UI retrieves and loads the entire workflow graph from the server, which leads to:

• Large JSON payloads that are several megabytes in size.
• Increased download time and slower deserialization, affecting UI performance and user experience.

The Solution

New API Endpoint

• Endpoint Functionality: This endpoint will retrieve a parent node and its immediate children, instead of the entire graph.
• Input Parameter:
  • parentId: Identifies the parent node for which child nodes are to be fetched.
• Processing Steps:
  1. Retrieve the complete workflow graph.
  2. Identify and select the node based on parentId.
  3. Serialize the parent node and its immediate children into a JSON response.

UI Modifications

• Lazy Loading: Adjust the UI to support lazy loading, using the new API to dynamically fetch graph levels as needed.
• User Interaction: Update the UI to trigger API calls to fetch child node data only when the parent node is clicked.

Benefits

• Reduced Initial Load Time: The initial load time of the graph is significantly reduced by fetching smaller data chunks.
• Improved Performance: Smaller data payloads mean faster download and deserialization, enhancing responsiveness.
• Scalable and Efficient Data Handling: This approach reduces memory usage and client processing time by only loading necessary data.

Notes

The solution addresses performance issues related to sub-workflows (workflows as activities) and other complex activities like "ForEach" and "Container". It ensures efficient and scalable graph handling, regardless of the specific workflow structure or component type.

[glime-ai]

To implement the solution described in the problem statement, you would need to follow these steps:

1. Create a New API Endpoint:

   • Develop a new API endpoint in the backend that accepts a parentId as an input parameter.
   • The endpoint should be designed to retrieve only the parent node and its immediate children from the workflow graph.

2. Modify the Backend Logic:

   • Update the backend logic to handle the retrieval of a specific node and its immediate children based on the provided parentId.
   • Ensure that the backend can serialize the selected portion of the workflow graph into a JSON response.

3. Update the UI for Lazy Loading:

   • Adjust the UI components to support lazy loading of the workflow graph.
   • Implement logic to trigger API calls when a user interacts with a parent node, such as clicking on it to expand and view its children.

4. Optimize Data Handling:

   • Ensure that the UI only requests and processes the necessary data for the user's current context, reducing memory usage and improving performance.

5. Test the Solution:

   • Thoroughly test the new API endpoint and the updated UI to ensure that they work as expected.
   • Validate that the performance improvements are significant and that the user experience is enhanced.

By following these steps, you can achieve the benefits of reduced initial load time, improved performance, and scalable data handling for complex workflow graphs in the UI.

This comment was generated by Glime.