Tech stack demand

An example microservice pipeline to visualize the tech stack demand on the current job market.

Show who is using certain technologies:

Search for company:

Installation

• Install docker/docker-compose
• Assure docker has enough resources (Preferences > Advanced; tested with 4 CPUs and 6 GiB RAM)
• Run docker-compose up
• Open in your browser http://localhost:5000/
• Collect new data (reload page to see progress; this can take up to 20 min)

How to use the UI

• Check out this basic action wiki

Technical sidenotes

Architecture

Browse kafka topics:

• Kafka logs are not mounted
• While collecting new data, check http://localhost:9000/

Manage Neo4j data:

• Data mounted to /neo4j_data_loader/neo4j_data
• Open Neo4j browser: http://localhost:7474/
• Login with user: neo4j password: password

Run tests:

• docker-compose -f docker-compose-tests.yml up

Run keywords api update:

• Delete old data saved in keywords_api/redis_data
• Run: docker-compose -f docker-compose-keywords-update.yml up