Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature]: add test pipeline for testing Postgres parser using different applications #1880

Open
wants to merge 420 commits into
base: main
Choose a base branch
from

Conversation

harshsbhat
Copy link

@harshsbhat harshsbhat commented May 18, 2024

closes [feature]: add test pipeline for testing Postgres parser using different applications
This pull request adds a new GitHub Actions workflow file named test-go-postgres.yml for testing the GinApp application with PostgreSQL. The workflow is designed to automate the testing process and includes the following steps:

Checkout Repository:
Checks out the main repository where the workflow is defined.
Download Keploy Build Binary: Downloads a Keploy build binary artifact from the actions artifact storage.
Give Permission to Binary: Sets executable permissions for the Keploy build binary.
Download Latest Keploy Binary: Downloads the latest Keploy binary release from GitHub and moves it to /usr/local/bin.
Get the Pilot for Tests and Mocks Assertion: Downloads the latest pilot binary for test and mock assertions, sets the executable permission, and moves it to /usr/local/bin.
Checkout to the Samples-Go Repository: Checks out the samples-go repository at the native-linux branch.
Build the Gin-Postgres Application: Navigates to the gin-postgres directory and builds the Go application.
Add Noisy Fields in the Config File: Placeholder step for adding any noisy fields in the configuration file if required.
Run Testing Script for Gin-Postgres Application: Uses a custom GitHub action (.github/actions/tester) to run the testing script for the gin-postgres application with specified parameters.

PostgreSQL Service:
Services Section: Configures a PostgreSQL service using the postgres:latest Docker image.
Environment Variables: Sets up the POSTGRES_USER, POSTGRES_PASSWORD, and POSTGRES_DB environment variables.
Ports and Health Checks: Exposes port 5432 and sets up health checks to ensure the PostgreSQL service is ready before running the tests.
This new workflow ensures that the GinApp application using PostgreSQL is properly tested with Keploy, automating the setup and execution of tests in a CI/CD pipeline.

Changes Introduced:
Added a new GitHub Actions workflow file: .github/workflows/test-go-postgres.yml.
Instructions for Reviewers:
Please review the new workflow file for accuracy and completeness.
Ensure that the PostgreSQL service configuration aligns with the application's requirements.
Verify that the steps and commands for building and testing the application are correct.

I have read the CLA Document and I hereby sign the CLA.

Hermione2408 and others added 30 commits October 30, 2023 15:22
* MySQLparser-v2

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* node changes

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* node app changes

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* WIP

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* ok packet changes

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* Bug fix

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* Bug fix

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

* Removed log

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>

---------

Signed-off-by: Hermione Dadheech <hermionedadheech@gmail.com>
Co-authored-by: Sarthak Shyngle <50234097+Sarthak160@users.noreply.github.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
* fix: removed unnecessary log

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>

* fix: testing version problem change

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>

* fix: updated path of keploy logs and added function to delete them

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>

* fix: testing sentry

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>

* fix: checking

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>

* fix: made changes according to the comments

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>

---------

Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
* fix: https mock not generating in docker

Signed-off-by: gouravkrosx <gouravgreatkr@gmail.com>

* docs: update README.md

---------

Signed-off-by: gouravkrosx <gouravgreatkr@gmail.com>
* add:one click install example

Signed-off-by: shivamsouravjha <2019145@iiitdmj.ac.in>

* refactor:without one click update

Signed-off-by: shivamsouravjha <2019145@iiitdmj.ac.in>

* refactor:flag name

---------

Signed-off-by: shivamsouravjha <2019145@iiitdmj.ac.in>
the decode function is not able to distinguish the closing braces of the string json and sectionSingle, so used the prefix and suffix in place of regex to extract the document correctly

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>
…le to decode (keploy#1047)

* feat: create a keploy config file

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add keploy-config.yaml

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add testset entry in keploy-config.yaml

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: added code for fetching noise config

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: build

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add regex code

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: keploy commands failing when no config file

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: update keploy-config

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: added the removed function

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* refactor: condition to match non readable payload

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* refactor: condition to match non readable payload

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* refactor: save encoded string in case of large buffer data

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* refactor: remove buffered string from pg response

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* feat: make unreadable data base64 in datarow packet

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

---------

Signed-off-by: Akash Kumar <meakash7902@gmail.com>
Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>
Co-authored-by: Akash Kumar <meakash7902@gmail.com>
Co-authored-by: Akash Kumar <91385321+AkashKumar7902@users.noreply.github.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
Signed-off-by: Aditya Tomar <adi02032002@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
Signed-off-by: gouravkrosx <gouravgreatkr@gmail.com>
* fix: api request time for mock filtering

Signed-off-by: gouravkrosx <gouravgreatkr@gmail.com>

* refactor: update amd c files

Signed-off-by: gouravkrosx <gouravgreatkr@gmail.com>

---------

Signed-off-by: gouravkrosx <gouravgreatkr@gmail.com>
…lity (keploy#1072)

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>
* update:serve with app command

* refactor:serve cmd

* refactor:remove fetch hook command

* fix:path for serve

Signed-off-by: shivamsouravjha <shivamsouravjha@gmail.com>

---------

Signed-off-by: shivamsouravjha <shivamsouravjha@gmail.com>
* added flag for req timestamp

Signed-off-by: 404fixer <govindtomar94@gmail.com>

* renamed flag

Signed-off-by: 404fixer <govindtomar94@gmail.com>

* fixed formatting

Signed-off-by: 404fixer <govindtomar94@gmail.com>

---------

Signed-off-by: 404fixer <govindtomar94@gmail.com>
…olve conflct in matching (keploy#1075)

* fix: updates the decode function for OpMsg section

the decode function is not able to distinguish the closing braces of the string json and sectionSingle, so used the prefix and suffix in place of regex to extract the document correctly

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>

* fix: adds the ismaster query in mongo config for node

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>

---------

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>
* update:serve with app command

* refactor:remove fetch hook command

* refactor:keploy example flag

Signed-off-by: shivamsouravjha <shivamsouravjha@gmail.com>

* refactor:removing unused flag

* removing keploy.sh file after install

---------

Signed-off-by: shivamsouravjha <shivamsouravjha@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
…y#1089)

* feat: add support for authentication of mongo

adds scram support to authenticate into mongoDb during test. DocumentDB is supported since it uses SCRAM-SHA-1 for authentication

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>

---------

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>
Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
Signed-off-by: Pranshu Srivastava <iampranshu24@gmail.com>
updates the authentication calls in mongo protocol to config calls for the database connection and fixes the issue of accessing an empty array after simulation by fetching tcs mocks after connection.read call

Signed-off-by: re-Tick <jain.ritik.1001@gmail.com>
* added automatic portChange feature if current port is busy

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added for for continuous port change if consecutive ports are busy

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* defined a upper and lower bound and maxatempts

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added a case where the system can't find a available port

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* improved code indentation

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed emoji

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed logger

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* removed useless imports

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* removed unnecessary logs

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

---------

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>
* feat: create a keploy config file

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add keploy-config.yaml

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add testset entry in keploy-config.yaml

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: added code for fetching noise config

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: build

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add regex code

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: keploy commands failing when no config file

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: update keploy-config

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: added the removed function

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: noise field in graphql

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: refactor to correct flow of code

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add new flag for config-path

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: changed noise to globalNoise in config yaml

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: build

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add cmd for generate keploy config

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* feat: add comments in keploy config yaml

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: removed keploy-config

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: address review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

* fix: adress review comments

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

---------

Signed-off-by: Akash Kumar <meakash7902@gmail.com>
shivamsouravjha and others added 11 commits May 3, 2024 14:07
* added re-record in v2

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added re-record feature

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added port check

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added re-record v2

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added re-record in test

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* cancellation after rerecord completion

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* cleaned up

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* cleaned up

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* added context cancellation

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* resolved lint errors

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* resolved lint errors

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* resolved lint errors

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* resolved merge conflict

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* resolved merge conflict

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed go lint

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed go lint

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed go lint

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed go lint

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed go lint

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* fixed go lint

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* removed unnecessary timer

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* tidy logs for re-record

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* feat: remove linting issue

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* added a ticker

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* correct ticker

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* used gettestcases function

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* used gettestcases function

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* corrected errors

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* corrected errors

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* corrected errors

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* changed cmd dialog

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* refactor: remove prints and format code

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* moved waiting port functions out

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>

* refactor: remove prints and format code

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

---------

Signed-off-by: EraKin575 <tejaskumar574@gmail.com>
Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>
Co-authored-by: Sarthak Shyngle <50234097+Sarthak160@users.noreply.github.com>
Co-authored-by: Sarthak160 <rocksarthak45@gmail.com>
* [Feature]: Create a normalisation command in keploy CLI (keploy#1589)

* Initialised Normalise Command

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Path init

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Successfully found the expected and actual responses for failing tests

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Files are being updated but structs are not proper: func WriteTestcase doesnt match yaml file

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Updating testcase files successfully

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Added flags to collect test-set and test-cases from user

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Fixed lint errors

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Fixed lint errors

Signed-off-by: Akash <akashsingh2210670@gmail.com>

* Refactored PR for Keploy v2 and fixed merge conflicts

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Normalise Test cases

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Addressed comments and refactored changes

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Fixed Lint

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Fixed Lint

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Refactoring and Adding Additional Flags

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Added Normalise to replay service

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

* Fixed lint

Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>

---------

Signed-off-by: Akash <akashsingh2210670@gmail.com>
Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>
Co-authored-by: Animesh Pathak <53110238+Sonichigo@users.noreply.github.com>

* feat: add normalization

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* fix: bugs

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* fix: bugs

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* fix: bugs

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* fix: resolve comments

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

---------

Signed-off-by: Akash <akashsingh2210670@gmail.com>
Signed-off-by: Akash Singh <akashsingh2210670@gmail.com>
Signed-off-by: charankamarapu <kamarapucharan@gmail.com>
Co-authored-by: Sky Singh <114267538+Akash-Singh04@users.noreply.github.com>
Co-authored-by: Animesh Pathak <53110238+Sonichigo@users.noreply.github.com>
Signed-off-by: Amogh Umesh <amoghumesh02@gmail.com>
Co-authored-by: Sarthak Shyngle <50234097+Sarthak160@users.noreply.github.com>
* chore: add docker ip log and remove a switch

Signed-off-by: Shubham Jain <shubhamkjain@outlook.com>

* fix: check container ip before starting tests

Signed-off-by: Shubham Jain <shubhamkjain@outlook.com>

---------

Signed-off-by: Shubham Jain <shubhamkjain@outlook.com>
ougoung -> outgoing

Signed-off-by: Priyansh Agrawal <agrawal.priyansh@yahoo.in>
Co-authored-by: Sarthak Shyngle <50234097+Sarthak160@users.noreply.github.com>
* fix: folder permissions

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* fix: folder permissions

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* fix: unset umask

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* doc: add comment

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

* doc: add comment

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>

---------

Signed-off-by: charankamarapu <kamarapucharan@gmail.com>
* refactor: sync mysql changes acc to latest code

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

* Fixed lint issues

Signed-off-by: Chinmay <chinmaysharma1020@gmail.com>

* fix: Added an argument to DecodeMySQL function to find out the mode

Signed-off-by: Chinmay <chinmaysharma1020@gmail.com>

* refactor: remove set mocks function form the memdb interface

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>

---------

Signed-off-by: Sarthak160 <rocksarthak45@gmail.com>
Signed-off-by: Chinmay <chinmaysharma1020@gmail.com>
Co-authored-by: Chinmay <chinmaysharma1020@gmail.com>
Copy link

github-actions bot commented May 18, 2024

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you and congratulations 🎉 for opening your very first pull request in keploy

Signed-off-by: harshsbhat <icybhat@gmail.com>
@harshsbhat
Copy link
Author

PR for issue #1880 @gouravkrosx

@harshsbhat
Copy link
Author

harshsbhat commented May 18, 2024

I have read the CLA Document and I hereby sign the CLA

@harshsbhat
Copy link
Author

I have read the CLA Document and I hereby sign the CLA .

@harshsbhat
Copy link
Author

recheck

@nehagup nehagup requested a review from gouravkrosx May 18, 2024 11:44
@slayerjain
Copy link
Member

recheck

* fix: example in config file

Signed-off-by: Animesh Pathak <kurosakiichigo.songoku@gmail.com>

* fix: example in config file

Signed-off-by: Animesh Pathak <kurosakiichigo.songoku@gmail.com>

* Update utils.go

* Update utils.go

* feat: consisten usage example form docs

---------

Signed-off-by: Animesh Pathak <kurosakiichigo.songoku@gmail.com>
@harshsbhat
Copy link
Author

recheck

@harshbhat04
Copy link

I have read the CLA Document and I hereby sign the CLA

@harshbhat04
Copy link

recheck

@gouravkrosx
Copy link
Member

@harshbhat04, The workflow has not been triggered, Please check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[feature]: add test pipeline for testing Postgres parser using different applications