Privacy Bug: Over-collection in Opt-in Performance Statistics

[David-Fryd]

Performance analytics are collected and written to mcaptcha_pow_analytics, even if the user has not opted in, or if they have opted out after previously opting in.

The handling of every verify_pow request results in the generation of a CreatePerformanceAnalytics and subsequent call to analysis_save, where the information is collected regardless of the opt-in status of the sitekey.

[realaravinth]

Hello 👋

This is not a bug, this is the intended behavior. The checkbox controls weather the performance data collected should be published to the larger mCaptcha net.

PoW in CAPTCHAs is new, it is hard to determine the correct range of difficulty factors that offer effective protection while also being accessible to a wide range of devices. This data tells how each difficulty factor performs.

Also, right now, we are describing CAPTCHAs in terms of difficulty factors, which is not very intuitive. I think ifwe start describing them in terms of seconds (as in time), then not only will the protection be more effective but also significantly improve UX. To switch to a seconds model, we'll have to know how PoW performs. We have a cronjob that takes the average of all recorded times for each difficulty factor and offer instance local statistics. This stuff isn't shared with other mCaptcha instances when the checkbox is enabled though.

These are the reasons why I've enabled performance statistics collection by default. I'm open for discussion on this matter :)

[David-Fryd]

Hello! 👋

Thank you for the prompt response! :)

Why is data deleted from mcaptcha_pow_analytics when users revoke their consent for publishing data? This seems to imply that the collection of user data is somehow related to the opt-in behavior. From the system you described, it would seem as if you would only want to remove the key's pseudo_id, and not the data stored in mcaptcha_pow_analytics.

As a side note- I wasn't able to find the cronjob that you referenced- is that in the repo somewhere?

[realaravinth]

Why is data deleted from mcaptcha_pow_analytics when users revoke their consent for publishing data? This seems to imply that the collection of user data is somehow related to the opt-in behavior. From the system you described, it would seem as if you would only want to remove the key's pseudo_id, and not the data stored in mcaptcha_pow_analytics.

Oh. Good catch, that is a bug. There's also another bug in the same feature, that I just noticed: we are not calculating PoW difficulty factor based on new parameters to update the CAPTCHA configuration. The infra is there, but I seem to have missed it out in the update subroutine. Will send a patch by the end of the week.

The cronjob that analyzes the performance stats. The codebase needs some clean, it shouldn't be this difficult to find stuff 😅