ETL pipeline to parse character image from pictures and create dataset of English Characters

A web app where an emergency worker can input a new message and get classification results in several categories. The web app will also display visualizations of the data.

ETL process which loads and transforms Medicare hospital data using Python and Hive

Modeling event data to create a non-relational database 🗃 and ETL Pipeline 🔀 for Sparkify music streaming 🎼 service using Apache Cassandra 👁

Data Engineering techniques to analyze the data from Figure Eight to build a classification model for an API that classify the Disaster message into one of the given 36 categories and contact the particular category company to help these peoples.

Repository for tutorials, information, notes and projects about data engineering.

Data modeling in a PostgreSQL database

Build disaster response pipelines with the message data from Figure Eight. A ETL pipeline was built to process the message data from major natural disasters around the world. A ML pipeline was built to categorize these events according to the messages. A web app was set up to visualize the data and classify the message after typing in.

As a recruiter, screening candidates can turn into a repetitive, time-consuming task. This GUI application was created as a viable solution! Today on Desktop, Tomorrow on the Cloud.

GCP App Engine ETL from publicly available API. (Home project to play with GCP products.)

This project focuses on the current job market for data-related jobs in four different countries: The United States (US), Canada, Australia, and Singapore. Using the Extraction, Loading, Transformation (ETL) methodology, the project will extract all data-related job postings from Indeed in these four countries, transform them, and load them into…

Coalesced and transformed various data sources to create a comprehensive data lake for the USA tourism sector.

Data Pipeline Analytics Platform is an end-to-end generic Big Data pipeline. Involves following tech stack: AWS S3, AWS Redshift, AWS EMR Cluster, Apache Spark, Apache Airflow.

A repository concentrating on using High end parallel pipelines to perform ETL across various data sources

Resolvendo problema de uma empresa que realiza revenda de carros.

Google Cloud Function built to perform an ETL Job to Collect StripeAPI Data and Transform it to be able to Import it to Bigquery.

Analyzing messaging data for a disaster response. The model generated in this project is deployed as a web app and can be used to categorize messages.

An ETL Pipeline using Apache Spark in Python

Olá pessoal, esse é um repositório tratando de fim a fim, uma pipeline de dados relacionados a produtos e suas subcategorias, onde simulo que isso seja um pedido do time de negócios, com granularidade diária a ser entregue.

Classifying real messages that were sent during disaster events so that they can be sent to an appropriate disaster relief agency.