NOTICE: The Uitzending Gemist tvOS App has reached its End of Life. Use TV Gemist instead.

[dyve]

I don't see any new programs appear since last Sunday (Jan 28). The live views work, the existing programs from the past weeks work, but the "By Day" view shows nothing, and the "Program" view shows no new episodes. Did NPO change something?

[MatrixView]

Yup, the old NPO API (apps-api.uitzendinggemist.nl) is likely being discontinued because the broadcast data requests are producing empty json replies the last few days. Of course it is possible to parse/scrape the npo website, but that beats the purpose of an API... Of which they have a new version, but probably won't give you access to...

[4np]

Yes, unfortunately the old API -that was replaced by NPO 2.0 in spring 2017- has now finally been pulled offline. While local cache ensures you can still watch some programs you have favourited as well as live TV, this is pretty much the end of where this app can go.

Having said that, I do have a working prototype I built from scratch a while ago with very basic functionality for the new API (browsing programs, filtering them and watching them). Unfortunately the challenge is that it relies on an API key, which the NPO can kill any time. Making it a bit harder is that they use SSL Pinning in the new mobile app, so you cannot reverse engineer the app very well. And, of course, all the other functionality still needs to be written. There are also some other challenges with the API itself, as it cannot do one of the most basic things like sorting alphabetically 😏 I guess they think you need to search if you want to find a particular program instead of paginating through a list? What if you don't want that :)

[martysmartySE]

I'm actually only seeing SSL Pinning on the latest version of the iPad app, at the point where it needs to retrieve the ApiKey. Once it has that cached, it seems to no longer use that on other requests. I'm able to do a couple requests there. The new API is located at https://start-api.npo.nl. With the parameter ApiKey you can actually supply your API key.

For my internal app I don't need paginating, so I haven't looked into that, but the app should be able to no?

I'll see if I can decode more endpoints during the week.

Do please post if you find out how to retrieve an ApiKey

[4np]

I have an APIKey, but I am bit hesitant to throw it open in the public considering the NPO might disable it ;)

Pagination / infinite scrolling works much more speedy than through the deprecated API. I have most requests sorted out, there aren't that many. The only part I am having trouble with is getting the new player (using Apple's FairPlay Streaming FPS) up and running. I am currently up to the point that I have the most of the pieces -Fairplay CERT, Server Playback Context (SPC)-, but I am having trouble obtaining the Content Key Context (CKC). If you have any requests you can intercept that successfully set up a FairPlay-stream that would help :) I am getting the impression the NPO apps are using the proprietary player from nepworldwide.nl (that take care of the streams) using DASH / Widevine, so I am not even sure FairPlay Streaming should work...

Relevant API Endpoints:

• Home: https://start-api.npo.nl/page/home
• Series API (Plural) : https://start-api.npo.nl/page/catalogue?page=2
• Series API (Singular): https://start-api.npo.nl/page/franchise/KN_1676589?page=1
• Episode API (Singular): https://start-api.npo.nl/page/episode/POW_03567086
• Episode media information: https://start-api.npo.nl/media/AT_2083054 (qualityOptions & SRT subtitles)
• Subtitles: http://tt888.omroep.nl/tt888/AT_2085754 (WebVTT)
• Episode Recommendations: https://start-api.npo.nl/media/AT_2083054/recommendations
• Search API: https://start-api.npo.nl/search?pageSize=7&query=keur&page=1
• Live API: https://start-api.npo.nl/page/live
• TV Guide API: https://start-api.npo.nl/channel
• EPG API: https://start-api.npo.nl/epg/2017-07-14?type=tv (tv / radio)
• Images: http://images.poms.omroep.nl/image/s295/c295x166/911848.jpg

I am thinking of open sourcing the WIP-app and networking kit (it's all in Swift 4) when I have taken it a bit further. Not sure what to do with the APIKey though... Perhaps I'll include it at some point when I don't need to debug the code anymore.

[MatrixView]

@4np Wow, that is great stuff. I really hope you'll open this up further...
The apikey probably is consistent across platform/app/user/network, so throwing it "out there" is probably not as bad as you might expect because their own app wouldn't work either if they blocked it. (unless they wrote it so it'll refresh)

Besides... the P in NPO stands for Public. Just sayin'.. :-) They used to have a hackathon a few years ago and many open API's supporting it (NPO Backstage). I kid you not!

[dyve]

ApiKey can be found at https://packagist.org/packages/kro-ncrv/npoplayer?q=&p=0&hFR[type][0]=concrete5-package Published by a KRO-NCRV Employee to embed a player on websites. If the ApiKey is in a config file in the app then it could easily be changed by someone who is comfortable with sideloading apps.

[Reino17]

• Subtitles: http://tt888.omroep.nl/tt888/AT_2085754 (WebVTT)

This is the old api, afaik. The new api looks like https://rs.poms.omroep.nl/v1/api/subtitles/AT_2085754/nl_NL/CAPTION.vtt. Or I've mistaken it for yet another api.

[martysmartySE]

FYI, it looks like part of the old API is still available. I do a weekly update on all series, starting at http://apps-api.uitzendinggemist.nl/series.json, which seems still available.

Only the daily updates have indeed gone away (https://apps-api.uitzendinggemist.nl/broadcasts/2018-01-30.json)

[MatrixView]

@MKaterbarg Sure, the old API is still working, but there's less and less data added to it.
The series.json you mention does contain data, but (besides new series) you need details from:
../series/{series-id}.json requests, and those have no episode info (yet) of 30-01-2018 and 31-01-2018.
The latest episodes that were added in the last few days are from 28-01-2018 and 29-01-2018. I know this because I create RSS feeds from this old API. My feeling is that NPO will slowly kill it by not feeding it new data. That way users will seek refuge to new apps and the NPO website.

[martysmartySE]

Ah thanks @MatrixView . My script hadn't run for a few days, so it still found about 2000 new eps :)

[4np]

Hah, nice work @dyve, that's a different API Key than I have :) That means I can publish the code I am working on using that publicly available API Key which makes it a lot easier :)

@Reino17, that tt888 endpoint is indeed the old one, the new media API (e.g. https://start-api.npo.nl/media/AT_2083054) provides references to the SRT subtitles.

Like @MatrixView says, the old API is in part still functioning, but the content is lagging behind (if updated at all). When I have some time I'll create a new repo to publish the working code I have right now.

[MatrixView]

@4np Ah, that'd be great!

NPO seems to have a github channel too... with a different API.
https://github.com/npo-poms

...Unfortunately they didn't slip any APIkey and/or credentials in with that source. :-(

[4np]

I have just published NPOKit. A Swift 4 Framework for interfacing with the NPO 2.0 APIs. Note though that this is still very much alpha and method signatures might change without notice. I have a working tvOS app as well, but I need to clean the code up a bit more before I publish it to GitHub.

[MatrixView]

@4np Thx for the NPOKit! I will look into it. Although I don't use Swift or even a Mac... But I'm interested in the code and see what I can use for my shell scripts.

BTW. the old API seems indeed lagging, as I've seen "new" episodes of January 30th and 31st roll in. And even some of yesterday. Let's wait and see...

[4np]

@MatrixView: You can use it from the command line as well. And as Swift if available on Linux that might work as well (I haven't tested that use case yet). Please take a look at the HOWTO for a command line example...

[MatrixView]

@4np O, thx for the tips but please don't bother! I'm using Powershell and Xidel under Windows. I've taken a quick look at your NPOKit and I'll probably manage to convert your http requests to a script when I have the time to do so.

In the meantime I'm using code from Reino17's BatchGemist to get my TVguide. (This scrapes the NPO-Gids webpage).

Thanx again for your work! If I have any questions I'll know where to find you!
--MV

[4np]

No worries, I just wanted to make sure that use case was covered as well :) That allows for people to write command line tools as well as by using a Swift server side framework (Vapor, Perfect, Kitura, etcetera) :)

[4np]

I have just pushed the first working version of TV Gemist. It currently supports paging through programs, filtering programs, paging through episodes and playing episodes. Please take a look :)

[MatrixView]

@4np Jeroen, ook al heb ik geen AppleTV, bedankt voor het coden van TV Gemist! In dit soort projecten zitten altijd leerzame dingen voor me.

[sanderdg]

I'm just getting started with this wonderful app and I can report that it works great for me.