Can we support UOS with puppeteer?

[huan]

According to this Pull Request to ItChat created by @luvletter2333, it seems we can improve our puppeteer version as well.

UOS

I believe the most important is the headers.

UOS_PATCH_CLIENT_VERSION = '2.0.0'
UOS_PATCH_EXTSPAM = 'Gp8ICJkIEpkICggwMDAwMDAwMRAGGoAI1GiJSIpeO1RZTq9QBKsRbPJdi84ropi16EYI10WB6g74sGmRwSNXjPQnYUKYotKkvLGpshucCaeWZMOylnc6o2AgDX9grhQQx7fm2DJRTyuNhUlwmEoWhjoG3F0ySAWUsEbH3bJMsEBwoB//0qmFJob74ffdaslqL+IrSy7LJ76/G5TkvNC+J0VQkpH1u3iJJs0uUYyLDzdBIQ6Ogd8LDQ3VKnJLm4g/uDLe+G7zzzkOPzCjXL+70naaQ9medzqmh+/SmaQ6uFWLDQLcRln++wBwoEibNpG4uOJvqXy+ql50DjlNchSuqLmeadFoo9/mDT0q3G7o/80P15ostktjb7h9bfNc+nZVSnUEJXbCjTeqS5UYuxn+HTS5nZsPVxJA2O5GdKCYK4x8lTTKShRstqPfbQpplfllx2fwXcSljuYi3YipPyS3GCAqf5A7aYYwJ7AvGqUiR2SsVQ9Nbp8MGHET1GxhifC692APj6SJxZD3i1drSYZPMMsS9rKAJTGz2FEupohtpf2tgXm6c16nDk/cw+C7K7me5j5PLHv55DFCS84b06AytZPdkFZLj7FHOkcFGJXitHkX5cgww7vuf6F3p0yM/W73SoXTx6GX4G6Hg2rYx3O/9VU2Uq8lvURB4qIbD9XQpzmyiFMaytMnqxcZJcoXCtfkTJ6pI7a92JpRUvdSitg967VUDUAQnCXCM/m0snRkR9LtoXAO1FUGpwlp1EfIdCZFPKNnXMeqev0j9W9ZrkEs9ZWcUEexSj5z+dKYQBhIICviYUQHVqBTZSNy22PlUIeDeIs11j7q4t8rD8LPvzAKWVqXE+5lS1JPZkjg4y5hfX1Dod3t96clFfwsvDP6xBSe1NBcoKbkyGxYK0UvPGtKQEE0Se2zAymYDv41klYE9s+rxp8e94/H8XhrL9oGm8KWb2RmYnAE7ry9gd6e8ZuBRIsISlJAE/e8y8xFmP031S6Lnaet6YXPsFpuFsdQs535IjcFd75hh6DNMBYhSfjv456cvhsb99+fRw/KVZLC3yzNSCbLSyo9d9BI45Plma6V8akURQA/qsaAzU0VyTIqZJkPDTzhuCl92vD2AD/QOhx6iwRSVPAxcRFZcWjgc2wCKh+uCYkTVbNQpB9B90YlNmI3fWTuUOUjwOzQRxJZj11NsimjOJ50qQwTTFj6qQvQ1a/I+MkTx5UO+yNHl718JWcR3AXGmv/aa9rD1eNP8ioTGlOZwPgmr2sor2iBpKTOrB83QgZXP+xRYkb4zVC+LoAXEoIa1+zArywlgREer7DLePukkU6wHTkuSaF+ge5Of1bXuU4i938WJHj0t3D8uQxkJvoFi/EYN/7u2P1zGRLV4dHVUsZMGCCtnO6BBigFMAA='

headers = {
  'client-version' : UOS_PATCH_CLIENT_VERSION,
  'extspam' : UOS_PATCH_EXTSPAM,
  'referer' : 'https://wx.qq.com/?&lang=zh_CN&target=t'
}

To be investigated.

[huan]

It seems not to work on my account (can not log in via web) with the following modifications:

https://github.com/wechaty/wechaty-puppet-wechat/blob/04fae37abc714b21b19ad66eca827971d3a63a35/src/bridge.ts#L228-L242

@luvletter2333 Please feel free to join us if you are interested. :-)

[luvletter2333]

I am willing to help but I have no idea about typescript and its HTTP library...

[luvletter2333]

This patch is still working, I checked it just before

[luvletter2333]

This patch should be applied to the third step of login.

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

When requesting cgi-bin/mmwebwx-bin/webwxnewloginpage, this patch is needed. Besides, wxsid and wxuin should be obtained from Cookie, not XML response.

[huan]

Hi @luvletter2333 !

Thank you very much for your reply, and it's great to know that the patch is still working!

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

Cloud you explain how to get a response contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage" when visit https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login?

I tried in the chrome browser and it seems that the webwxnewloginpage does not exist.

Could you please share a minimum code to reproduce it in Python code, or CURL if possible? Appreciate it!

[luvletter2333]

Hi @luvletter2333 !

Thank you very much for your reply, and it's great to know that the patch is still working!

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

Cloud you explain how to get a response contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage" when visit https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login?

I tried in the chrome browser and it seems that the webwxnewloginpage does not exist.

Could you please share a minimum code to reproduce it in Python code, or CURL if possible? Appreciate it!

I'm sorry to reply so late.
You would need to switch to Firefox for inspection since Chrome Devtools may lose Response Body.

You could follow this repo to try adding custom Headers. Then you may login Wechat Web like UOS.
The pre-generated token is published in issues by me.

[luvletter2333]

About the analysis of the protocol of Wechat Web, you could check this blog https://blog.csdn.net/zhangmiaoping23/article/details/78733231

[huan]

Thank you so much for the detailed explanation!

I'll try Firefox to reproduce it later, and I believe it will not be hard to enable it in Puppeteer because we can study from the Firefox ModHeader plugin.

[artxia]

不知道是不是 redirect_uri 的关系？使用 988dd56 依然不能绕开登录限制

[huan]

@artxia If you want to help, I'd like to suggest that you follow this repo https://github.com/BlueSky-07/wechat-token and make everything work with the browser plugin as the repo mentioned.

After you can reproduce the login successfully with the browser plugin, then I believe we will be easily to copy the setting from the browser plugin to the puppeteer configurations.

[artxia]

@artxia If you want to help, I'd like to suggest that you follow this repo https://github.com/BlueSky-07/wechat-token and make everything work with the browser plugin as the repo mentioned.

After you can reproduce the login successfully with the browser plugin, then I believe we will be easily to copy the setting from the browser plugin to the puppeteer configurations.

https://github.com/adamyi/wechrome
This browser plugin can be used directly.

[huan]

Awesome, it looks great!

Would you like to send a PR to configure our puppeteer by following the setting from this plugin?

This improvement will be an epic milestone for our Web Protocol since 2017!

[artxia]

I am very sorry, my code programming skills are not enough for PR.

Also, the above browser plugin may require adding an address "https://wx2.qq.com/*" to "wechatUrls".

[leochen-g]

This is an exciting news

[huan]

According to puppeteer/puppeteer#659 and Stackoverflow: No, google-chrome in headless mode doesn't support extensions yet., we can not use https://github.com/adamyi/wechrome directly with the puppeteer.

Need to move the logic to native puppeteer implementation.

[leochen-g]

Hi @luvletter2333 !

Thank you very much for your reply, and it's great to know that the patch is still working!

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

Cloud you explain how to get a response contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage" when visit https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login?

I tried in the chrome browser and it seems that the webwxnewloginpage does not exist.

Could you please share a minimum code to reproduce it in Python code, or CURL if possible? Appreciate it!

Thank you so much for the detailed explanation!

I'll try Firefox to reproduce it later, and I believe it will not be hard to enable it in Puppeteer because we can study from the Firefox ModHeader plugin.

我本地测试了一个最简化的代码，发现https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage是扫码点击确定后的跳转地址，所以不扫码确定看不到这个path。以下是我使用puppeteer写的最小demo，本地测试已经可以成功登陆网页端，希望能有帮助

const pptr = require('puppeteer');

async function bootstrap() {
    const browser = await pptr.launch({
        headless: false,
        slowMo: 250,
    });
    const page = await browser.newPage();
    // 开启请求拦截
    await page.setRequestInterception(true);
    page.on('request', async req => {
        let url = new URL(req.url());
        if (url.pathname == "/cgi-bin/mmwebwx-bin/webwxnewloginpage") {
            console.log(req.headers())
            const override = {
                headers: {
                    ...req.headers(),
                    extspam: "Gp8ICJkIEpkICggwMDAwMDAwMRAGGoAI1GiJSIpeO1RZTq9QBKsRbPJdi84ropi16EYI10WB6g74sGmRwSNXjPQnYUKYotKkvLGpshucCaeWZMOylnc6o2AgDX9grhQQx7fm2DJRTyuNhUlwmEoWhjoG3F0ySAWUsEbH3bJMsEBwoB//0qmFJob74ffdaslqL+IrSy7LJ76/G5TkvNC+J0VQkpH1u3iJJs0uUYyLDzdBIQ6Ogd8LDQ3VKnJLm4g/uDLe+G7zzzkOPzCjXL+70naaQ9medzqmh+/SmaQ6uFWLDQLcRln++wBwoEibNpG4uOJvqXy+ql50DjlNchSuqLmeadFoo9/mDT0q3G7o/80P15ostktjb7h9bfNc+nZVSnUEJXbCjTeqS5UYuxn+HTS5nZsPVxJA2O5GdKCYK4x8lTTKShRstqPfbQpplfllx2fwXcSljuYi3YipPyS3GCAqf5A7aYYwJ7AvGqUiR2SsVQ9Nbp8MGHET1GxhifC692APj6SJxZD3i1drSYZPMMsS9rKAJTGz2FEupohtpf2tgXm6c16nDk/cw+C7K7me5j5PLHv55DFCS84b06AytZPdkFZLj7FHOkcFGJXitHkX5cgww7vuf6F3p0yM/W73SoXTx6GX4G6Hg2rYx3O/9VU2Uq8lvURB4qIbD9XQpzmyiFMaytMnqxcZJcoXCtfkTJ6pI7a92JpRUvdSitg967VUDUAQnCXCM/m0snRkR9LtoXAO1FUGpwlp1EfIdCZFPKNnXMeqev0j9W9ZrkEs9ZWcUEexSj5z+dKYQBhIICviYUQHVqBTZSNy22PlUIeDeIs11j7q4t8rD8LPvzAKWVqXE+5lS1JPZkjg4y5hfX1Dod3t96clFfwsvDP6xBSe1NBcoKbkyGxYK0UvPGtKQEE0Se2zAymYDv41klYE9s+rxp8e94/H8XhrL9oGm8KWb2RmYnAE7ry9gd6e8ZuBRIsISlJAE/e8y8xFmP031S6Lnaet6YXPsFpuFsdQs535IjcFd75hh6DNMBYhSfjv456cvhsb99+fRw/KVZLC3yzNSCbLSyo9d9BI45Plma6V8akURQA/qsaAzU0VyTIqZJkPDTzhuCl92vD2AD/QOhx6iwRSVPAxcRFZcWjgc2wCKh+uCYkTVbNQpB9B90YlNmI3fWTuUOUjwOzQRxJZj11NsimjOJ50qQwTTFj6qQvQ1a/I+MkTx5UO+yNHl718JWcR3AXGmv/aa9rD1eNP8ioTGlOZwPgmr2sor2iBpKTOrB83QgZXP+xRYkb4zVC+LoAXEoIa1+zArywlgREer7DLePukkU6wHTkuSaF+ge5Of1bXuU4i938WJHj0t3D8uQxkJvoFi/EYN/7u2P1zGRLV4dHVUsZMGCCtnO6BBigFMAA=",
                    'client-version': "2.0.0",
                }
            }
            await req.continue(override);
            return
        }
        await req.continue();

    });

    page.on('response', async res => {
        if (res.url().indexOf('/header') >= 0) {
            console.log(res.status());
            console.log(await res.text());
        }
    });

    page.on('requestfinished', req => {
        // console.log(`请求完成: ${req.url()}`);
    });

    page.on('requestfailed', req => {
        console.log(`请求失败: ${req.url()}`);
    });
    await page.goto('https://wx.qq.com/?target=t');
}

bootstrap();

[luvletter2333]

我本地测试了一个最简化的代码，发现https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage是扫码点击确定后的跳转地址，所以不扫码确定看不到这个path。以下是我使用puppeteer写的最小demo，本地测试已经可以成功登陆网页端，希望能有帮助

是这样的（

[huan]

@gengchen528 Thank you very much for your feedback and minimum reproducible code, it will be a great help and I'll definitely try it today and hope we can integrate it into our puppet soon!

[huan]

Thanks to @gengchen528 for adding UOS support for our Wechaty Puppet WeChat!

The PR has been tested by myself and it works like a charm, I have merged it as wechaty-puppet-wechat@0.27.1 and we can expect for using it soon!

This is a HUGE improvement for our Wechaty ecosystem because this fix lets most of the users can get started with Wechaty more easily. I'd like to invite @gengchen528 to write a blog post for announcing this great improvement, it would be great if we have it soon and I'm looking forward to reading it!

[leochen-g]

Thanks to @gengchen528 for adding UOS support for our Wechaty Puppet WeChat!

The PR has been tested by myself and it works like a charm, I have merged it as wechaty-puppet-wechat@0.27.1 and we can expect for using it soon!

This is a HUGE improvement for our Wechaty ecosystem because this fix lets most of the users can get started with Wechaty more easily. I'd like to invite @gengchen528 to write a blog post for announcing this great improvement, it would be great if we have it soon and I'm looking forward to reading it!

非常荣幸能够写这篇博客，感谢 @artxia 提供的chrom插件给了我思路。希望这种方式能够一直可用

[huan]

Close this issue because it has been implemented perfectly.

Thanks to @luvletter2333 for letting me know UOS by your Python PR, @artxia for linking to the chrome extension to make everything clear, @gengchen528 for creating the PR for our puppet system!

[suntong]

Thanks you guys to have made such tremendous breakthrough. Truly a milestone in WX bot history. @gengchen528 your blog post is spreading like a wildfire in WX bot community, 😄😄

One question,

You could follow this repo to try adding custom Headers. Then you may login Wechat Web like UOS.
The pre-generated token is published in issues by me.

Currently, the whole world is using a fixed extspam value from @gengchen528's blog post, however, wechat-token says that the extspam value should be associated with each person's profile. So

@luvletter2333 / @gengchen528, do you think it'd be a problem, if the whole world is using the same fixed extspam value? thx

[huan]

@luvletter2333 / @gengchen528, do you think it'd be a problem, if the whole world is using the same fixed extspam value? thx

Thanks for pointing this out! I'd also have this question and I believe it should not use the same extspam value for every users.

[leochen-g]

目前看来同一个extspam不会有太大问题，但是后期没法保证。如果如果把wechat-token集成进puppet或者让用户自行生成的话，会不会增加使用门槛，而且我看使用wechat-token是需要安装docker环境的

我在想是否能够由我们提前生成一批extspam，在启动的时候随机使用其中一个呢 这只是我目前个人的建议

[suntong]

是否能够由我们提前生成一批extspam，在启动的时候随机使用其中一个呢

I think that's the best solution so far, I was thinking that too.
Otherwise, it can easily be identified by WX, and becomes the biggest giveaway, yelling to WX, "block me, I'm such an easy target and so simple to block"

[zpaimon]

I think we can build a token server.When user start puppet,the puppet call token api and get an extspam token.

[ghost]

我可以正常登录 UOS 微信，但是无法通过 wechaty 登陆。

扫码后手机微信提示“当前账号无法使用该服务”。

[austincityuwen]

It's too late for me to read this issue at this moment, my question is how to generate personal extspam that associated with each person's profile? In case that it is detected by wechat and may block it someday？