Skip to content

Organic set of software components that assists in implementing resilient distributed systems that excel in reconfigurability, monitorability, modularity, extensibility, and openness. It includes libraries of ready-to-use modules for Big Data data engineering, analytics, and visualization. It is designed with Reconfigurable Manufacturing in mind.

License

mircosoderi/State-of-the-art-Artifacts-for-Big-Data-Engineering-and-Analytics-as-a-Service

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

An organic set of software components to be used as building blocks for the implementation of resilient, monitorable, modular, extendable, open distributed systems with main applications in smart and reconfigurable manufacturing, and smart cities

This repository contains an organic set of containerized applications designed and developed from June 2021 onward in the Data Science Institute at University of Galway under the supervision of Prof. John Gerard Breslin. They are suitable for remote creation, configuration, and operation of complex AI-enabled distributed systems across heterogeneous devices and multiple geographical locations.

Get started

To get started, you will be guided through an extensively documented Postman collection, also available as a JSON file ready to be imported to your local Postman installation. You will build, configure, and exercise a simple system for real-time anomaly detection. In particular, you will be instructed to use the GettingStartedTest1 and GettingStartedTest2 text files for some guidance about how to see the system in operation. Docker is required. Exposing Docker daemon on tcp://localhost:2375 without TLS is also required; Windows users can do that from the General Settings of Docker Desktop, while Linux users may find this article of some help. Postman needs to be configured to accept self-signed certificates. Times of response observed running the collection on a HP 250 G8 laptop (Microsoft Windows 10 Home, Intel Core I5 1035G1 CPU @ 1GHz, quad-core, RAM 16 GB) are provided.

Learn more

Learn more about the NetworkFactory in M. Soderi, and J. G. Breslin, "A Service for Resilient Manufacturing," 2023 IEEE International Conference on Smart Computing (SMARTCOMP), 2023.. You can also have a look to the poster related to this paper.

Learn more about the NewmanNode in M. Soderi, and J. G. Breslin, "Synchronized Sub-Second Arbitrary Changes to Decoupled Components for Ultimate Resilience in Cross-Platform Geo-Distributed Smart Factories," 2023 IEEE International Conference on Smart Computing (SMARTCOMP), 2023., then enjoy a proof-of-concept of the Postman Collection Run as a Service envisioned in the paper by running the NewmanNode4CustomFDS Postman collection, which relies on the NewmanCustomFDSv4 Postman collection, run in the NewmanCustomFDSv4NewmanDemo Postman environment. You can also run the NewmanCustomFDSv4 collection from your Postman desktop application by using the NewmanCustomFDSv4Demo Postman environment. In end, you can also create a Newman node in a containerized Node-RED instance and configure it to run the NewmanCustomFDSv4 Postman collection using the NewmanCustomFDSv4NewmanDemo Postman environment.

Learn more about CustomFDS in N. Bharot, M. Soderi, and J. G. Breslin, "A Demo of Microservice for Customized Faulty Product Detection System in Smart Manufacturing," 2023 IEEE International Conference on Smart Computing (SMARTCOMP), 2023. You may also want to have a look to N. Bharot, M. Soderi, P. Verma, and J. G. Breslin, "Improving Product Quality Control in Smart Manufacturing through Transfer Learning-Based Fault Detection," 2023 IEEE International Conference on Smart Computing (SMARTCOMP), 2023.

Learn more about BLE Nodes, in Soderi, Mirco, and John Gerard. "BLE Servers and Ubiquitous Analytics AAS.", Digital Book of Abstracts, 30th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2022), December 8th - 9th, 2022, Cork, Ireland. (See poster: AICS_2022_Poster.pdf). Use the extended version of the mentioned poster paper as a guidance for a proof of concept that we have implemented and that consists of an Android app and of a system distributed across two laptops entirely built by running a Postman collection.

Learn more about Crazy Nodes in Soderi, Mirco, and John G. Breslin. "Crazy nodes: towards ultimate flexibility in ubiquitous big data stream engineering, visualisation, and analytics, in smart factories." International Symposium on Leveraging Applications of Formal Methods. Springer, Cham, 2022.. You may also want to run the updated version of the demo presented at SmartComp 2022; you can use the dedicated step by step PowerPoint presentation for guidance.

Learn more about Control Room, in M. Soderi, V. Kamath and J. G. Breslin, "A Demo of a Software Platform for Ubiquitous Big Data Engineering, Visualization, and Analytics, via Reconfigurable Micro-Services, in Smart Factories," 2022 IEEE International Conference on Smart Computing (SMARTCOMP), 2022, pp. 1-3, doi: 10.1109/SMARTCOMP55677.2022.00041.. The demo consists in a collection of API requests that are illustrated step by step in a dedicated PowerPoint presentation.

Learn more about AIS Nodes in M. Soderi, V. Kamath and J. G. Breslin, "Toward an API-Driven Infinite Cyber-Screen for Custom Real-Time Display of Big Data Streams," 2022 IEEE International Conference on Smart Computing (SMARTCOMP), 2022, pp. 153-155, doi: 10.1109/SMARTCOMP55677.2022.00036.

Learn more about AIS Nodes also in M. Soderi, V. Kamath, J. Morgan and J. G. Breslin, "Advanced Analytics as a Service in Smart Factories," 2022 IEEE 20th Jubilee World Symposium on Applied Machine Intelligence and Informatics (SAMI), 2022, pp. 000425-000430, doi: 10.1109/SAMI54271.2022.9780768.

Learn more about EMQ Broker Nodes and the implementation of access control for them, Service Nodes and the implementation of the access control for them, and the Transformation Library, in M. Soderi, V. Kamath, J. Morgan and J. G. Breslin, "Ubiquitous System Integration as a Service in Smart Factories," 2021 IEEE International Conference on Internet of Things and Intelligence Systems (IoTaIS), 2021, pp. 261-267, doi: 10.1109/IoTaIS53735.2021.9628434.. A SQL script is used for initializing the RDB instance used for the proof of concept included in this paper.

OpenAPI specifications

Complete OpenAPI specifications in yaml format are available for Network Factory, Service Nodes, Crazy Nodes, and BLE Nodes. Import them in the Swagger Editor for a preview.

Credits

CustomFDS.tar is a Docker volume backup meant to be restored to containers based on the msoderi/py4customfds Docker Image, and it contains a customization of the artifacts published by Nitesh Bharot in the GitHub repository CustomFDS in the context of a Seed Project led by me for the Confirm SFI Research Centre for Smart Manufacturing.

About

Organic set of software components that assists in implementing resilient distributed systems that excel in reconfigurability, monitorability, modularity, extensibility, and openness. It includes libraries of ready-to-use modules for Big Data data engineering, analytics, and visualization. It is designed with Reconfigurable Manufacturing in mind.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published