Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert books downloaded with licenserequest #3

Closed
mkb79 opened this issue Aug 4, 2019 · 42 comments · Fixed by #22
Closed

Convert books downloaded with licenserequest #3

mkb79 opened this issue Aug 4, 2019 · 42 comments · Fixed by #22

Comments

@mkb79
Copy link
Owner

mkb79 commented Aug 4, 2019

Books downloaded with

license, _ = client.post(
   "content/{asin}/licenserequest",
    body={
        "drm_type": "Adrm",
        "consumption_type": "Download",
        "quality":"Extreme"
    }
)
content_url = license['content_license']['content_metadata']['content_url']['offline_url']

can’t converted with ffmpeg -activation_bytes ... at this moment.

Read here

Audible uses a new aaxc format when downloading via audible app on android/iOS or the api. But with this client you can get the response of a licenserequest. Example shorten on many places with ... :

{
    "content_license": {
        "acr": "CR!...",
        "asin": "B07DDKJH33",
        "content_metadata": {
            "content_url": {
                "offline_url": "https://dyrrggeck87jc.cloudfront.net/.../bk_adko_003749ade_lc_128_44100_2.aax?voucherId=cdn:...&Policy=...&Signature=...&Key-Pair-Id=..."
            }
        },
        "drm_type": "Adrm",
        "license_response": "...==",
        "message": "Eligibility details:[GrantRightsReason [reasonCode=MEMBERSHIP, reason=No need to verify active offline license when request is not for offline consumption. OWNERSHIP-user does not have expected ownership rights over the parent title asin. OWNERSHIP-title does not qualify for long title child part ownership or customer does not own child part. OWNERSHIP-user has ownership rights. GEO_RIGHTS-user has geo-rights[DE] over the title[B07DDKJH33] in marketplaceId[AN7V1F1VY261K]. TITLE_ATTRIBUTES-title does not have rodizio plan association. GEO_RIGHTS-user has geo-rights[DE] over the title[B07DDKJH33] in marketplaceId[AN7V1F1VY261K]. No need to verify active offline license when request is not for offline consumption. BENEFIT-user has valid Radio benefit associated[RadioStub]. TITLE_ATTRIBUTES-title does not have radio plan association. Client Asin Mapping validation skipped since client id is null. AAA Client with id: ApolloEnv:AudibleApiExternalRouterService/EU/Prod, does not have access to asin: B07DDKJH33. Client does not have plans that support asin benefits.]]. Licensing details:[ADRM license granted]",
        "request_id": "...",
        "status_code": "Granted",
        "voucher_id": "cdn:..."
    },
    "response_groups": [
        "always-returned"
    ]
}

Maybe someone has the know-how to use this informations to decode the new aaxc format.

Meanwhile you can grab the audible web page for the download urls. You can use cookies to authenticate like so:

import audible
import requests

auth = audible.FileAuthenticator(...)
cookies = auth.login_cookies

r = requests.get("https://www.audible.com/...", cookies=cookies)

Edit 2022-01-11:
Since ffmpeg 4.4 decryption of aaxc files should be build in. You have to use ffmpeg with the --audible_iv and --audible_key options and the correct iv/key pair from the decrypted licenserequest response!

@mkb79 mkb79 added the help wanted Extra attention is needed label Aug 4, 2019
@omarroth
Copy link

omarroth commented Aug 4, 2019

You can also use:

asin = "some_asin"
codec = "some_codec" # Desired quality from /1.0/library/{asin}?response_groups=product_attrs,relationships

auth = audible.FileAuthenticator(...)
client = audible.AudibleAPI(auth)

content_url = f"https://cde-ta-g7g.amazon.com/FionaCDEServiceEngine/FSDownloadContent"
params = {
    'type': 'AUDI',
    'currentTransportMethod': 'WIFI',
    'key': asin,
    'codec': codec
}

signed_headers = client._sign_request('GET', content_url, params, {})
headers = client.headers.copy()
for item in signed_headers:
    headers[item] = signed_headers[item]

r = client.session.request('GET', content_url, headers=headers, params=params, json={}, allow_redirects=False)
print(r.headers['Location'])

To get files in .aa (Format4) or .aax (Enhanced) format.

@mkb79
Copy link
Owner Author

mkb79 commented Aug 5, 2019

I can’t download any books that way. It gives me a download link. But if I want to download the file it gives me an Unspecified database error.. And I‘m not the only one with that problem.

I wrote about this multiple times on reddit to you.

@mkb79
Copy link
Owner Author

mkb79 commented Aug 5, 2019

With help from omarroth here is a workaround to solve the problem:

import audible


asin = "some_asin"
codec = "some_codec"  # Desired quality from /1.0/library/{asin}?response_groups=product_attrs,relationships

auth = audible.FileAuthenticator(...)
client = audible.AudibleAPI(auth)

content_url = f"https://cde-ta-g7g.amazon.com/FionaCDEServiceEngine/FSDownloadContent"
params = {
    'type': 'AUDI',
    'currentTransportMethod': 'WIFI',
    'key': asin,
    'codec': codec
}

signed_headers = client._sign_request('GET', content_url, params, {})
headers = client.headers.copy()
for item in signed_headers:
    headers[item] = signed_headers[item]

r = client.session.request('GET', content_url, headers=headers, params=params, json={}, allow_redirects=False)
link = r.headers['Location']

# prepare link to fit correct tld
tld = auth.locale.audible_api.split("api.audible.")[1]
new_link = link.replace("cds.audible.com", f"cds.audible.{tld}")

print(new_link)

To get files in best quality codec is to be set to LC_128_44100_stereo

mkb79 added a commit that referenced this issue Aug 5, 2019
mkb79 added a commit that referenced this issue Aug 5, 2019
omarroth added a commit to omarroth/audible.cr that referenced this issue Aug 5, 2019
@mkb79 mkb79 closed this as completed Aug 8, 2019
@omarroth
Copy link

omarroth commented Oct 8, 2020

Make a request to /1.0/content/#{asin}/licenserequest, with this body:

{
  "quality: "Extreme",
  "consumption_type": "Download",
  "drm_type": "Adrm"
}

The expected body should provide an offline_url and license_response. Downloading the offline_url produces a .aax (aaxc) file.

Following information from this patch to ffmpeg, get the audible_key and audible_iv by decrypting license_response with a hash produced from customer_id, device_type, device_serial, and asin.

In the patch notes Vesselin provides this code:

def decrypt_voucher(deviceSerialNumber, customerId, deviceType, asin, voucher):
    buf = (deviceType + deviceSerialNumber + customerId + asin).encode("ascii")
    digest = hashlib.sha256(buf).digest()
    key = digest[0:16]
    iv = digest[16:]

    # decrypt "voucher" using AES in CBC mode with no padding
    cipher = AES.new(key, AES.MODE_CBC, iv)
    plaintext = cipher.decrypt(voucher).rstrip(b"\x00")
    return json.loads(plaintext)

deviceSerialNumber and deviceType are the same as those submitted to /auth/register. For example A2CZJZGLK2JJVM and AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA. The customerId is returned from /auth/register and looks like amzn1.account.AAAAAAAAAAAAAAAAAAAAAAAAAAAA. voucher is the base64-decoded license_response data.

Successfully decrypting license_response should produce a json object:

{
    "key": "ffffffffffffffffffffffffffffffff",
    "iv": "ffffffffffffffffffffffffffffffff",
    "refreshDate": "2020-10-28T00:15:10Z",
    "removalOnExpirationDate": "2020-12-07T00:15:10Z",
    "rules": [
        {
            "parameters": [
                {
                    "expireDate": "2020-11-07T00:15:10Z",
                    "type": "EXPIRES"
                }
            ],
            "name": "DefaultExpiresRule"
        },
        {
            "parameters": [
                {
                    "directedIds": [
                        "amzn1.account.AAAAAAAAAAAAAAAAAAAAAAAAAAAA"
                    ],
                    "type": "DIRECTED_IDS"
                }
            ],
            "name": "AllowedUsersRule"
        }
    ]
}

The key and iv can then be passed to a version of ffmpeg with Vesselin's patch applied to decrypt the desired aaxc file.

Any Audible Plus book should be able to be downloaded without purchase. I tested B084C68ZJK and B00URXOQ1E. The same method works for regular books as well. I would consider it a better alternative to the FSDownloadContent method.

@mkb79 mkb79 reopened this Oct 8, 2020
@mkb79
Copy link
Owner Author

mkb79 commented Oct 8, 2020

Hi omarroth,

Thank you and Vesselin for your very important information and hard work. I will look at this as fast as possible. It‘s definitely a better option than FSDownloadContent.

@mkb79
Copy link
Owner Author

mkb79 commented Oct 8, 2020

I can verify that it works for me too. I got the needed key and iv from voucher. I will implement them in my project.

Thanks again for your work!

@mkb79 mkb79 linked a pull request Oct 8, 2020 that will close this issue
@mkb79
Copy link
Owner Author

mkb79 commented Oct 8, 2020

I‘ve open a pr with function to decrypt voucher gained from licenserequest and an example how to download aaxc files and decrypt there voucher to retrieve the iv and key for further purposes.

This is running for me with Pythonista 3. Can someone test if it running on other systems?

@mkb79
Copy link
Owner Author

mkb79 commented Oct 8, 2020

@omarroth you have a patched ffmpeg binary for testing purposes?

@omarroth
Copy link

omarroth commented Oct 8, 2020

https://omar.yt/27c9ffdd33130cf7511f8605bcde868ce08b92ba/ffmpeg

Statically built with some patches to alpine's APKBUILD.

Should be able to build with:

$ git clone https://git.ffmpeg.org/ffmpeg.git
$ cd ffmpeg
$ curl -o https://patchwork.ffmpeg.org/project/ffmpeg/patch/17559601585196510@sas2-2fa759678732.qloud-c.yandex.net/raw/
$ git apply aaxc.patch
$ ./configure
$ make
$ ...

Use with:

./ffmpeg -audible_key ... -audible_iv ... -i in.aax out.mp3

@mkb79
Copy link
Owner Author

mkb79 commented Oct 8, 2020

Thank you for the binary. Have you applied the patch? When I execute this I got the error Unrecognized option 'audible_key'.

@omarroth
Copy link

omarroth commented Oct 8, 2020

Yes. Make sure you're running the correct binary:

$ ./ffmpeg
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Alpine 10.2.0)
  configuration: --prefix=/usr --pkg-config-flags=--static --extra-ldexeflags=-static --disable-gnutls --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libx264 --enable-pic --enable-pthreads --disable-stripping --enable-static --enable-libopus --disable-debug
...

@mkb79
Copy link
Owner Author

mkb79 commented Oct 8, 2020

Ah, my mistake. I‘m executed the wrong version. Thanks for your hind and your work!!!

@rmcrackan
Copy link
Contributor

@omarroth , @mkb79 -- the above patched binary appears to be linux only. Is there a link to the patched windows version?

@mkb79
Copy link
Owner Author

mkb79 commented Oct 10, 2020

@rmcrackan

You can take a look on this page. You only have to find out how you can apply the aaxc patch!

@mkb79
Copy link
Owner Author

mkb79 commented Oct 11, 2020

@rmcrackan

I've tested something to compile ffmpeg on windows. And I got a working and patched ffmpeg bin. But the bin is very huge, because I compiled it with most of the features. You can compile it according to your wish very easy. Only follow these steps:

  1. Download this package and extract it.
  2. Download ffmpeg_extra.zip and extract the zip file into the build folder of the new dir.
  3. Execute media-autobuild_suite.bat in the main dir and answer the questions to build your custom ffmpeg.
  4. Drink a coffee. And drink a coffee again. Do this for the next hours until your compilation is finished.

@mkb79 mkb79 closed this as completed in #22 Oct 15, 2020
@mkb79 mkb79 pinned this issue Oct 15, 2020
TimothyGu pushed a commit to FFmpeg/FFmpeg that referenced this issue Nov 5, 2020
The AAXC container format is the same as the (already supported) Audible
AAX format but it uses a different encryption scheme.

Note: audible_key and audible_iv values are variable (per file) and are
externally fed.

It is possible to extend https://github.com/mkb79/Audible to derive the
audible_key and audible_key values.

Relevant code:

def decrypt_voucher(deviceSerialNumber, customerId, deviceType, asin, voucher):
    buf = (deviceType + deviceSerialNumber + customerId + asin).encode("ascii")
    digest = hashlib.sha256(buf).digest()
    key = digest[0:16]
    iv = digest[16:]

    # decrypt "voucher" using AES in CBC mode with no padding
    cipher = AES.new(key, AES.MODE_CBC, iv)
    plaintext = cipher.decrypt(voucher).rstrip(b"\x00")  # improve this!
    return json.loads(plaintext)

The decrypted "voucher" has the required audible_key and audible_iv
values.

Update (Nov-2020): This patch has now been tested by multiple folks -
details at the following URL:

mkb79/Audible#3

Signed-off-by: Vesselin Bontchev <vesselin.bontchev@yandex.com>
@mkb79 mkb79 removed the help wanted Extra attention is needed label Dec 18, 2020
@oaksisfolks
Copy link

@mkb79 would it be possible to give me instructions on how to build a version of ffmpeg with this new commit? I just built ffmpeg straight from the git repo and it didn't have the option -audible_key. The patch also fails.

@mkb79
Copy link
Owner Author

mkb79 commented Dec 20, 2020

@oaksisfolks

Which OS you are running? If you have Windows, you can try the guide above.

@tobi1220
Copy link

I can't seem to build ffmpeg with the patch on Windows, even with you guide @mkb79
Is there an already built version of ffmpeg with said patch applied?

@mkb79
Copy link
Owner Author

mkb79 commented Jan 15, 2021

I build ffmpeg for windows as I wrote the guide above. ffmpeg was running smooth. But I build ffmpeg with most of features, so the binary was very huge. But I doesn’t have it anymore (I use ffmpeg on my linux machine).

You can try the ffmpeg bin build for alpine above and run it with wsl2 under windows. Or you can take a look here. Maybe these bins are compiled with the patch. Otherwise you can search for a guide to build ffmpeg for windows and apply the patch yourself before compiling.

@tobi1220
Copy link

Thanks a lot, I tried the git full binaries built from gyan.dev and they already have the patch applied! This could also be interesting for you @oaksisfolks

@mkb79
Copy link
Owner Author

mkb79 commented Jan 15, 2021

Thank you for this very important information @KlaraStoff .

@celtichawk
Copy link

I've been following this and I've two questions

  1. So I know htey have to be download in .aaxc format, but is that script speific or Audible being Audible?
  2. Given th e./ffmpeg command, is there a way to cut the book up into chapters or a way to enable continue from where I left off?

@mkb79
Copy link
Owner Author

mkb79 commented Jun 3, 2021

  1. I don’t understand your question. You can use the licenserequest endpoint to download books in aaxc format. The same endpoint is used by the Audible app for Android or iOS.
  2. The aaxc file contains chapter information. With this information you can split and convert the file with ffmpeg. Using the metadata or licenserequest API endpoint gives you also the last heard position (if you are using the right response group)

@celtichawk
Copy link

I understand I can get the metadata from the .aaxc file, but I'm struggling on how to reinsert it. Using the above .ffmpeg command as the example, what would I add to it? When I try it just gives me a message was repeated error and doesn't add the metadata when I've extracted the metadata into the metadata file

@mkb79
Copy link
Owner Author

mkb79 commented Jun 3, 2021

Maybe you can take a look here. This is a command for my audible-cli package which uses ffmpeg to replace chapter metadata from aaxc file with the one obtained from licenserequest API endpoint.

@celtichawk
Copy link

That'd do what I'm asking. What's the simplest way to make that working with the pip version since that's how I've gone the pip3 route to get it installed and working.

@mkb79
Copy link
Owner Author

mkb79 commented Jun 3, 2021

Simple use audible-cli and copy the command file to the plug in dir. Please check the README for audible-cli.

@celtichawk
Copy link

Sorry for all the stupid questions but do I need to ask @omarroth for advice on building the ffmpeg mentioned in the thread? I tried and got nowhere with it so I'm not sure if I need to spin that off somewhere more appropriate however

@mkb79
Copy link
Owner Author

mkb79 commented Jun 3, 2021

You can try the last build from here. The aaxc patch should be included in this build.

@jscholes
Copy link

@mkb79 Question: is there a reason to use the newer aaxc format over aax? E.g. are there books no longer available in aax format, and/or is the quality of aaxc better? Or does it not matter for now.

@mkb79
Copy link
Owner Author

mkb79 commented Jul 18, 2021

AAXC-files use a different encryption than AAXC-files. To be clear, AAXC and AAX are only container which contains the audio file in aac-codec, the cover art and some metadata. So booth have the same quality.

Audible Plus titles and some other are only delivered as AAXC. So if you want them, you have to support AAXC.

@jscholes
Copy link

@mkb79 Thanks so much for the explanation, and of course for this software! As an Audible UK customer I don't have access to Audible Plus, but this is good to know. Luckily none of the titles in my library failed to download as AAX but I will be on the lookout for errors.

@Mbucari
Copy link
Contributor

Mbucari commented Sep 25, 2021

For those looking to work with AAXC files without ffmpeg, let me shamelessly promote my new C# library.

https://github.com/Mbucari/AAXClean

And for the record, they're actually encrypted in the same way as AAX files. The only difference is that AAX files contain key derivation information inside a custom mpeg4 box named Adrm. Keys are derived with activation bytes + Adrm data, and after you derive the key the decryption is exactly the same as with AAXC.

@rmcrackan
Copy link
Contributor

@Mbucari -- I didn't know you wanted to promote AAXClean. I frequently refer people to this page that I curate for audiobook software. I'll be glad to add you. What kind of description would you like me to include?

@mkb79
Copy link
Owner Author

mkb79 commented Sep 25, 2021

@rmcrackan

Maybe you can add audible-cli to this page?

@rmcrackan
Copy link
Contributor

Maybe you can add audible-cli to this page?

Added your cli and AAXClean. Let me know if you'd like any wording changed

@mkb79
Copy link
Owner Author

mkb79 commented Sep 25, 2021

Thank you very much!

@normal1ze
Copy link

@mkb79 I know this is closed but is this still the recommended way to convert AACX files with ffmpeg? Getting down the git and applying the patch was recommended 2 years ago. Is there a better way or is this still the solution?

@jscholes
Copy link

@normal1ze The patch should be bundled with any reasonably recent build of FFmpeg. So unless your source for FFmpeg builds is extremely out of date, there should be no need for patching and compiling anything manually. There are other options, such as that described by @Mbucari in #3 (comment). There is also, I believe, an optional plug-in for audible-cli, but AFAIK it uses FFmpeg underneath.

@normal1ze
Copy link

@jscholes thanks for the feedback mate. I have just installed the latest FFmpeg so I'll give it a go with vanilla install. Cheers.

@mkb79
Copy link
Owner Author

mkb79 commented Jan 11, 2022

@jscholes Thanks from me too!

@normal1ze Since ffmpeg v4.4 this function should be build in.

@devnoname120
Copy link
Sponsor

devnoname120 commented Oct 11, 2022

The Ubuntu APT repositories currently have an outdated version of ffmpeg which doesn't have that option.

This 3rd-party APT repository has an up-to-date version and I can confirm that it solves the issue for me on Ubuntu.

Here are the current instructions to install it:

sudo add-apt-repository ppa:savoury1/ffmpeg4
sudo add-apt-repository ppa:savoury1/ffmpeg5
sudo apt-get update
sudo apt-get upgrade && sudo apt-get dist-upgrade
sudo apt-get install ffmpeg

Alternatively you can download the latest statically-linked ffmpeg from here: https://johnvansickle.com/ffmpeg/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants