Export your personal Stackexchange data

Setting up

1. The easiest way is pip3 install --user git+https://github.com/karlicoss/stexport.

   Alternatively, use git clone --recursive, or git pull && git submodule update --init. After that, you can use pip3 install --editable.

2. Get API key Register a new Oauth app this for information. Once you’ve registered, find your app in the list and take note of
   • ‘Key’ – this is your key parameter
   • ‘Client Id’ and ‘Oauth Domain/Redirect URI’ – this might be useful for the next step
3. Get API access_token This is optional, since in principle you can run the script without access token. But some functionality won’t work (e.g. --user-sites), and you might miss some of your private data, so it’s recommended to do.

   Visit the following link, but

   • instead of 12345, substitute ‘Client Id’ from step 3
   • instead of http://example.com, substitute ‘Redirect URI’ from step 3

   https://stackoverflow.com/oauth/dialog?client_id=12345&redirect_uri=http://example.com&scope=private_info+no_expiry
       

   After that you should be authorised and redirected. Copy the access_token somewhere, you’ll need it later.

Exporting

NOTE: even if you decided not to use access_token, you still need to pass the parameter, just set it to empty string.

Usage:

Recommended: create secrets.py keeping your api parameters, e.g.:

key = "KEY"
access_token = "ACCESS_TOKEN"

After that, use:

python3 -m stexport.export --secrets /path/to/secrets.py

That way you type less and have control over where you keep your plaintext secrets.

Alternatively, you can pass parameters directly, e.g.

python3 -m stexport.export --key <key> --access_token <access_token>

However, this is verbose and prone to leaking your keys/tokens/passwords in shell history.

I highly recommend checking exported files at least once just to make sure they contain everything you expect from your export. If not, please feel free to ask or raise an issue!

Using data

You can use stexport.dal (stands for “Data Access/Abstraction Layer”) to access your exported data, even offline. I elaborate on motivation behind it here.

• main usecase is to be imported as python module to allow for programmatic access to your data.

  You can find some inspiration in =my.= package that I’m using as an API to all my personal data.

• to test it against your export, simply run: python3 -m stexport.dal --source /path/to/export
• you can also try it interactively: python3 -m stexport.dal --source /path/to/export --interactive