warning ... YAMLLoadWarning ... deprecated

[drandreaskrueger]

first time user here. I get this RED warning message:

~/env/.../python3.5/site-packages/searchtweets/credentials.py:34:
YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, 
as the default Loader is unsafe. 
Please read https://msg.pyyaml.org/load for full details.
  search_creds = yaml.load(f)[yaml_key]

when executing this code:

    premium_search_args = searchtweets.load_credentials("twitter_credentials.yaml",
                                       yaml_key="tweets_search_fullarchive",
                                       env_overwrite=False)

[igorbrigadir]

This was fixed in #68 - It's worth updating to the latest version on github, eg with pip:

pip install --upgrade git+https://github.com/twitterdev/search-tweets-python.git

[drandreaskrueger]

Great, thanks. Why not / When deploy that to pypi ?

Also, I might leave you again - because now actually tweepy is trying out ways to support the premium search, hooray. And I have plenty of tweepy code already so that might make it easier for me then.

[drandreaskrueger]

Oh, and now that I have your attention as someone at twitter ... what about these 2 suggestions:

• a public repo for feature requests; see what I wrote about it above here:
• a new API endpoint API.retweeted_with_comment(original_tweet_id) which returns "quoted retweets", i.e. "retweet with comment" tweets (which do NOT show up in the standard retweets API results), which would otherwise cause THOUSANDS of calls to your API as a workaround?

[igorbrigadir]

For requests for new features this is the place to post them: https://twitterdevfeedback.uservoice.com/ (this is for API features, not UI)

Or there's always https://twittercommunity.com/

For searching for retweets with comment, you need to use the Search API: tweepy/tweepy#1291 (comment)

[drandreaskrueger]

Thanks a lot for your suggestions, very helpful.

searching for retweets with comment, you need to use the Search API

yes, I am aware of that - and I am looking into Premium search ... exactly for that.

However, are you aware that I will be causing >3000 API calls instead of 1 ?
And each repeat, again ... several thousand calls.

I suggest you discuss how that can be avoided. Probably with a new API endpoint, no?

[igorbrigadir]

That's a lot of calls to make, it will get very expensive with Premium - 5000 calls on 30day endpoint is $1300 for example. Remember that Premium supports longer queries, so it's more efficient to batch calls together - so instead of searching for quotes of a single tweets, you'd search for quotes of multiple ones with OR, eg: url:1194044665164640257 OR url:1193958590689796104 OR url:1193906225366405120 This is a better question for https://twittercommunity.com/c/premium instead of github issues

[drandreaskrueger]

Thanks a lot @igorbrigadir

you'd search for quotes of multiple ones with OR

Yes, I had though of that already.
Then just taking care to stay below the max length of the query, and run it in chunked batches until this workaround has collected all tweet_ids of all quoted retweets.
And THEN check for retweets of THOSE tweet_ids, because that is what I am really interested in.

I don't quite understand the problem why I have to come up with such complicated (and expensive, as I learn now) workaround. Why not just give us an appropriate endpoint?

Is there perhaps a political reason that I do not understand, to just not to provide one of these endpoints:

• quoted_retweets_of_tweet(tweet_id) or
• quoted_retweets_of_all_tweets_of_user(user_id) or even
• retweets_of_quoted_retweets_of_user(user_id).

Especially the last one would save a LOT of queries, coding time, and money.

Can you help me to understand WHAT exactly is the problem here? The twitter API is super amazing, and fun to work with - but this lack of functionality is almost a "bug-like flaw". Perhaps it's intended? But then ... why?

[igorbrigadir]

This would be the perfect place for these ideas: https://twitterdevfeedback.uservoice.com/forums/921790-twitter-developer-labs

[drandreaskrueger]

This would be the perfect place for these ideas: ... twitterdevfeedback.uservoice ...

VERY good idea, thanks for pointing that out to me.

It's there now: https://twitterdevfeedback.uservoice.com/forums/921790-twitter-developer-labs/suggestions/38617114-add-endpoint-to-get-retweet-with-comment-aka-quot

Let's see what happens.