Proposal: Integrate Dashboard test results with outside services.

[jennifer-shehane]

We'd like to have test results integrate with third party services like:

• Zapier
• Slack (Slack/Email Notifications #578)
• GitHub (DONE)
• Jira
• TestRail
• Gitlab
• Confluence
• Sentry
• Google Docs
• PagerDuty
• ??? have a service you'd like? You can express interest and see progress for this feature on our Roadmap from our Dashboard's product board here: https://portal.productboard.com/cypress-io/1-cypress-dashboard (Look for the card with your desired integration or Submit idea if you don't see it)

Our Dashboard Service knows when tests fail, it has details about a user's recorded CI run. When certain conditions are met, it can notify other integrated services and do things like:

• Send basic pass/fail stats
• Send Stdout (or link)
• Send Screenshots (or link)
• Send Video (or link)
• Analytics like...was this test failing before? the last time it passed?
• ??? have data you'd like? comment below!

The Dashboard should likely have a UI interface allowing the user to set what integrations they would like and what data they would like sent (on project? on organization?)

This proposal is a WIP currently, so we'd love to hear from the community on what you'd like to see.

You can express interest and see progress for this feature on our Roadmap from our Dashboard's product board here: https://portal.productboard.com/cypress-io/1-cypress-dashboard (Look for the card with your desired integration or Submit idea if you don't see it)

[cameronc56]

Hipchat would be great

[jennifer-shehane]

Conversations in Gitter surrounding integrations:

My main goal is to have some sort of notification that a test failed on a git commit
Lots of ways to slice this, a simple solution I found was using a custom reporter but I needed to use multiples to accomplish the goal
Built in slack integration would be best
https://gitter.im/cypress-io/cypress?at=597a2d90c101bc4e3ac74704

yes that solution for slack notification used to work with my protractor tests. I changed because I have not enough information about the failure (only exit code 1). Using the afterLaunch function I have more information about which test is failing...
Thats right with cypress I can send the notification to slack using the link to the dashboard to check the failure report. I love the dashboard.
Have you got any plans to integrate with slack o email failure notifications? Have you got any idea/workarround how can I develop the notifications?
https://gitter.im/cypress-io/cypress?at=58bee213e961e53c7fb0f7b7

Is there currently a way to hook Cypress up to a github pull request? If not, will github integration be a feature at some point?
https://gitter.im/cypress-io/cypress?at=5a1ec30571ad3f873644c9f1

[frankiethekneeman]

Would really love it if these could be implemented as a post-run/post-upload event so we can write our own service integration plugins.

[frankiethekneeman]

I mean, most of the major ones, probably, but it'd be nice to be able to write my own hooks for services outside that ecosystem, or even totally custom automation, around the results of these tests.

[amirrustam]

@jennifer-shehane I would be happy to contribute GitHub and Jira integrations. And due Atlassian Connect, the Jira work would also cover Bitbucket integration. I think this would help with #1173. I'm familiar with the APIs, and would love to help. 🙂

[acthilina2020]

Cucumber integration?

[jennifer-shehane]

@acthilina There is a cucumber plugin that integrates with Cypress you may want to check out. https://docs.cypress.io/plugins/

[acthilina2020]

Thank you @jennifer-shehane , If you have more info, please send my way. :-)

[soccer99]

Gitlab please!

[amirrustam]

@soccer99 Thanks for the suggestion. Gitlab currently isn't on our immediate integrations roadmap. But it might be in the future, specially if there is user demand for it. We'd be glad to hear what level of integration you're looking for.

[gscott]

Before doing too many custom integrations I’d recommend implementing outbound webhooks; they can be used to integrate just about anything.

[salty-blue-mango]

We're looking for a possible integration with confluence that prints out bar charts, pass/fail, etc.

[amirrustam]

@Nate-Watts feel free to elaborate further on the Confluence integration for the sake of documentation. We might integrate with the Atlassian stack down the line, and it would be nice to have your feedback in our considerations. Thank you.

[prescottprue]

Would love to see this.

I agree with @gscott on outbound webhooks. If those exist, almost anything should be able to be wired up, then the more specific integrations can come in the future (potentially built on that same webhook system).

[Shabbir124]

How we integrate Test-link to cypress? :(

[Knaledge]

@jennifer-shehane - is the idea with this proposal that Cypress itself will be able to integrate with all (or at least some) of the listed services? Or is this the Cypress Dashboard service integrating with the listed services?

I'm hoping for the former, though I'd understand the latter.

[jennifer-shehane]

@Knaledge The scope of this issue is for the Dashboard Service (any test using --recording), otherwise we don't have the data to integrate or know when to trigger notifications.

There is a separate issue regarding native notifications that could be triggered when running the Cypress test runner by itself here: #2145

[jennifer-shehane]

@bdim Could you open a pull request to add this plugin to our documentation? Thanks! Instructions here.

[bdimitrovski]

@jennifer-shehane Here it is cypress-io/cypress-documentation#1444. Thanks! :)

[nir0s]

PagerDuty please :)

[pareshkrc]

integration with qTest https://www.qasymphony.com/

[trydionel]

Hi all! We're continuing to make progress on adding GitHub integration to the Cypress Dashboard and would love to get some hands-on user feedback at this point. Does anyone have a few minutes to talk through the designs with me? You'd get a chance to try out this new integration in your own project, and you'll get to keep access to the feature afterwards!

Please leave a comment here, or email me directly if you're interested. Thank you!

[salty-blue-mango]

I'd like to try it out!

…

On Fri, Mar 29, 2019, 11:25 AM Jeff Tucker ***@***.***> wrote: Hi all! We're continuing to make progress on adding GitHub integration to the Cypress Dashboard and would love to get some hands-on user feedback at this point. Does anyone have a few minutes to talk through the designs with me? You'd get a chance to try out this new integration in your own project, and you'll get to keep access to the feature afterwards! Please leave a comment here, or email me directly if you're interested. Thank you! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AkqCPo7_fZA5S59dXOAWKX-6hEUpFrnSks5vbkyTgaJpZM4QvSL-> .

[nir0s]

Same here! On Fri, Mar 29, 2019, 20:45 MangoVerde-con-Sal <notifications@github.com> wrote:

…

I'd like to try it out! On Fri, Mar 29, 2019, 11:25 AM Jeff Tucker ***@***.***> wrote: > Hi all! We're continuing to make progress on adding GitHub integration to > the Cypress Dashboard and would love to get some hands-on user feedback at > this point. Does anyone have a few minutes to talk through the designs with > me? You'd get a chance to try out this new integration in your own project, > and you'll get to keep access to the feature afterwards! > > Please leave a comment here, or email me directly if you're interested. > Thank you! > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#981 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AkqCPo7_fZA5S59dXOAWKX-6hEUpFrnSks5vbkyTgaJpZM4QvSL- > > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#981 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB6QhL1qXO_8Jzc09RJ120AyP0YySRDKks5vblE9gaJpZM4QvSL-> .

[prescottprue]

@trydionel Same here!

[trydionel]

Awesome, thank you so much @Nate-Watts, @nir0s, @prescottprue! Just to confirm, you'll need to have 1) a project recording to the Cypress Dashboard, and 2) the repo for this project hosted on github.com, either publicly or privately. If you meet those criteria, please grab a time slot on my calendar:

https://calendly.com/trydionel/github-integration-research?month=2019-04

Looking forward to talking soon!

[freedomflyer]

@trydionel would also love to hop on this beta, we meet the criteria

[trydionel]

@freedomflyer Sure thing! Can you get in touch with me at the email listed on my GitHub profile?

[sravan-s]

+1 to the webhooks, it would be great if we can have webhooks with information such as -

• Job xxx has finished
• Summary of the job (success/fail/skip count, time taken etc)
• Link to get detailed information about the job

[timiyay]

We use Flowdock for our company chat, so this integration would be useful for us.

[amirrustam]

👋 Hi everyone, we're excited to release GitHub Integration for the Cypress Dashboard today. The Cypress GitHub App tightly integrates with your workflow via commit status checks and pull-request comments. Read all about it below, and let us know what you think (@ me).

Blog post: https://cypress.io/blog/2019/07/29/github-integration-for-the-cypress-dashboard/
Docs: https://on.cypress.io/github-integration.

[mitar]

Awesome! :-) Please do GitLab next. :-)

[cryptiklemur]

Love the integration, but the experience of using it with Slack needs some love.

Right now, there are too many images being used in the commit reply, and it Slack, the message is basically a big block of image links. I like to suggest the use of emoji over the icons that are shown there and the removal of the header image.

[deluxetom]

Love the integration, but the experience of using it with Slack needs some love.

Right now, there are too many images being used in the commit reply, and it Slack, the message is basically a big block of image links. I like to suggest the use of emoji over the icons that are shown there and the removal of the header image.

It's great to be able to see the Cypress test results on the PR page but as @aequasi mentioned, it makes the integration very noisy in slack and pretty much unusable.

[trydionel]

Hey @aequasi, @thomaspicquet,

Yea, that's a bit rough 😬 Would you mind to answer some questions about your workflow?

• Are you using Slack as your primary collaboration hub, or is this comment just noisy compared to others?
• If we provided direct Slack integration, would you opt into that feature over PR comments?
• We've also looked into GitHub's "checks" API to place Cypress results into each PR's "Checks" tab. Do you currently use PR "Checks"? If so, would you prefer that Cypress post results there instead of as a PR comment?

Thanks for the feedback!

PS: If you have any other feedback for the Cypress Dashboard, or are just curious about what we're working on, please check out our updated roadmap at https://portal.productboard.com/cypress-io/1-cypress-dashboard/tabs/1-under-consideration.

[deluxetom]

Hey @trydionel,

We use Slack for a lot of things and we have a channel for our CI with:

• Github pull requests and status checks (circleci / cypress)
• We use pull reminders (which was just acquired by GitHub) to see the messages on the pull requests and get reminders about code reviews

It's nice to see the Cypress PR message and I'd like to keep that over the Checks tab on github but the message is really big and not really useful when it gets to Slack, unfortunately.
A slack integration sounds good or maybe an option for the Github integration to turn off the message?

[trydionel]

Cool, I've added your feedback to our product backlog. Thanks for the info!

Also, if you find the PR comments too noisy, you can turn them off independently of the commit status checks. See our documentation.

[arifmasood]

@jennifer-shehane @amirrustam @trydionel +1 for sending cypress run reports to designated emails.

It will be great if we can have summarized results of cypress run (from dashboard) being delivered to specific email addresses. Raising a red alarm in case of any failures or sending a green report in case tests passed.

[jennifer-shehane]

You can express interest and see progress for this feature on our Roadmap from our Dashboard's product board here: https://portal.productboard.com/cypress-io/1-cypress-dashboard (Look for the card with your desired integration or Submit idea if you don't see it)

I will be closing this issue and closing it to all comments since this is related to our Dashboard.

All related work for Dashboard features is handled in that ProductBoard and will be addressed by the Dashboard team directly when you comment there. We have already noted the initial comments from this issue there. Please refer all future comments there. Thank you.

[jennifer-shehane]

Slack integration is now available within our Cypress Dashboard. It allows you to see your Cypress test results directly in your teams’ Slack channels. Please read our Slack Integration docs to get started.