Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Mastodon to blocked services #100

Merged
merged 7 commits into from Nov 15, 2022
Merged

Conversation

sandervankasteel
Copy link
Contributor

@sandervankasteel sandervankasteel commented Nov 8, 2022

I've added the top 30 most popular Mastodon servers plus the two "official" instances run by Mastodon gGmbH themselves.

This is list is based on the "total_users" property from Mastodon's own page / API response (https://api.joinmastodon.org/servers).

This PR also might warrant a discussion about grabbing the content of the previously mentioned API call and build that list dynamically upon each Github workflow run due to the federated nature of Mastodon. Which I am more then happy to implement, if needed. ☺️

assets/services.json Outdated Show resolved Hide resolved
@ameshkov
Copy link
Member

ameshkov commented Nov 9, 2022

This PR also might warrant a discussion about grabbing the content of the previously mentioned API call and build that list dynamically upon each Github workflow run due to the federated nature of Mastodon. Which I am more then happy to implement, if needed.

Do they often change? Would be great if you can, thank you!

@sandervankasteel
Copy link
Contributor Author

sandervankasteel commented Nov 9, 2022

This PR also might warrant a discussion about grabbing the content of the previously mentioned API call and build that list dynamically upon each Github workflow run due to the federated nature of Mastodon. Which I am more then happy to implement, if needed.

Do they often change? Would be great if you can, thank you!

Due to the federated nature of Mastodon, new instances (and independent instances) are added constantly to the Mastodon network (especially given the current Twitter influx of users).

Just as an FYI, the servers on the previously mentioned endpoint are the instances that adhere to the Mastodon Covenant.
So if a Mastodon instance doesn't want to commit to the Mastodon Covenant it will not be added to the list (or simply does not want to be added to the list).
So it's impossible to 100% block all instances, but this should cover a very large percentage of the Mastodon instances.

Do they often change? Would be great if you can, thank you!

I will make sure the list will be updated on each Github Action run.
Are there things I need to take into consideration when I add the logic for building the server list?

@sandervankasteel sandervankasteel requested review from IldarKamalov and ameshkov and removed request for IldarKamalov November 12, 2022 21:27
@sandervankasteel
Copy link
Contributor Author

sandervankasteel commented Nov 13, 2022

Hi @ameshkov and @IldarKamalov, I finished updating the PR by adding the dynamic server list code. ☺️

As a small rundown, I also took the opportunity to update the Node version in the pipeline to the latest LTS version (version 18.x).
The primary reason behind the upgrade is to have support for fetch() without resorting to external dependencies like request.

If you have any questions or comments, don't hesitate to reach out.

@sandervankasteel sandervankasteel requested review from IldarKamalov and removed request for ameshkov November 13, 2022 13:57
hostlists-builder/index.js Outdated Show resolved Hide resolved
hostlists-builder/index.js Outdated Show resolved Hide resolved
const mastodonRules = result.map((element) => {
return `||${element.domain}^`
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need additional checks for mastodonRules:

  1. Not less than N elements (how many is okay)?
  2. Not more than X elements (how many is a good upper limit? 100?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100 seems like a fair limit.

Just as an additional check, you want to sort them by total user count ? So that only the top 100 servers are blocked?

@@ -10,6 +10,7 @@ const HOSTLISTS_URL = 'https://adguardteam.github.io/HostlistsRegistry/assets';
const CONFIGURATION_FILE = 'configuration.json';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since Mastodon logic is not related to hostlists and for better clarity I'd prefer to have it all in a separate file (mastodon.js) imported into index.js

@ameshkov
Copy link
Member

Thank you for the contribution! I've got a couple of nitpicks here and there, please take a look.

@sandervankasteel
Copy link
Contributor Author

Thank you for the contribution! I've got a couple of nitpicks here and there, please take a look.

Perfect! Thanks for the nitpicks, I will pick them up and ping you once they are ready for another round of reviews :)

@sandervankasteel
Copy link
Contributor Author

@ameshkov it's ready for another round of reviews 😄. I took the liberty of sorting the server list by the total_users count from high to low.

@ameshkov ameshkov merged commit b13933d into AdguardTeam:main Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants