Twitter oauth POST fails when content-type includes '; charset=UTF-8'

[dbellavista]

Present since request 2.50. I have created a minimal working example: https://gist.github.com/dbellavista/a7b911f87b56bb9c47fc

The following request option

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "headers": {
  "Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"
 },
 "json": true,
 "body": "status=Test%20status",
 "oauth": {
  "token": "token",
  "token_secret": "tokensecret",
  "consumer_key": "consumerkey",
  "consumer_secret": "consumersecret"
 }
}

produce the following response from twitter:

{ "errors": [ { "message": "Could not authenticate you", "code": 32 } ] }

Removing ; charset=UTF-8 solves the problem. However, request 2.49 seems unaffected.

[simov]

Why not use the form option for sending url encoded form data

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "json": true,
 "form": {status:'Test status'},
 "oauth": {
  "token": "token",
  "token_secret": "tokensecret",
  "consumer_key": "consumerkey",
  "consumer_secret": "consumersecret"
 }
}

I'm not sure that this is the exact problem, but you can try this set of options as well.

[simov]

@dbellavista , also note that specifically for Twitter you need to escape special characters by yourself, I plan to fix that, but in the meantime you can use the code from this #1287 (comment)

[dbellavista]

Yes @simov I already do the proper escaping and I wasn't using form on purpose.
By the way, I was wrong again (sorry!), this ticket should be edited or marked as invalid.

The issue is the following and it has nothing to do with charset:

// Works on request 2.49, but not in >= 2.50
reqOptions.headers['Content-Type'] = 'application/x-www-form-urlencoded';
reqOptions.body = qs.stringify({
  status: 'Test status'
});
// Works on both version:
reqOptions.form = {status: 'Test status'};

Should it be considered valid or should I always use the form property when sending application/x-www-form-urlencode?

[simov]

Can you give us the whole options object in all cases (not only the parts you think are related)? I'm asking this because I assume that there are more options in reqOptions

[dbellavista]

Not working with 2.50, works on 2.49:

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "headers": {
  "Content-Type": "application/x-www-form-urlencoded"
 },
 "json": true,
 "body": "status=Test%20status",
 "oauth": {
  "token": "token",
  "token_secret": "token_secret",
  "consumer_key": "consumer_key",
  "consumer_secret": "consumer_secret"
 }
}

Not working with 2.50, works on 2.49 (different from what I said in the latter comment):

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "headers": {},
 "json": true,
 "form": {
  "status": "Test status"
 },
 "oauth": {
  "token": "token",
  "token_secret": "token_secret",
  "consumer_key": "consumer_key",
  "consumer_secret": "consumer_secret"
 }
}

[simov]

Oh wait, in your previous comment I saw >=2.50 so I assumed 2.51

2.50 had a serious bug in it, and you should not use it

Can you make the same tests with 2.51 compared to 2.49

[dbellavista]

Thanks! Finally I'm able to reproduce the original issue!

Not working with 2.51, works with 2.49:

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "headers": {
  "Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"
 },
 "json": true,
 "body": "status=Test%20status",
 "oauth": {
  "token": "token",
  "token_secret": "token_secret",
  "consumer_key": "consumer",
  "consumer_secret": "consumer_secret"
 }
}

Works with both 2.51 and 2.49:

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "headers": {
  "Content-Type": "application/x-www-form-urlencoded"
 },
 "json": true,
 "body": "status=Test%20status",
 "oauth": {
  "token": "token",
  "token_secret": "token_secret",
  "consumer_key": "consumer",
  "consumer_secret": "consumer_secret"
 }
}

Works with both 2.51 and 2.49:

{
 "uri": "https://api.twitter.com/1.1/statuses/update.json",
 "method": "POST",
 "headers": {},
 "json": true,
 "form": {
  "status": "Test status"
 },
 "oauth": {
  "token": "token",
  "token_secret": "token_secret",
  "consumer_key": "consumer",
  "consumer_secret": "consumer_secret"
 }
}

[simov]

@dbellavista thanks for the thorough bug report it's fixed here #1314