Start moving to ytdl-org

2019-03-09 19:15:16 +07:00
63 changed files with 945 additions and 1992 deletions
@@ -6,8 +6,8 @@
 ---
-### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2019.04.07*. If it's not, read [this FAQ entry](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
+### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2019.03.09*. If it's not, read [this FAQ entry](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2019.04.07**
+- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2019.03.09**
 ### Before submitting an *issue* make sure you have:
 - [ ] At least skimmed through the [README](https://github.com/ytdl-org/youtube-dl/blob/master/README.md), **most notably** the [FAQ](https://github.com/ytdl-org/youtube-dl#faq) and [BUGS](https://github.com/ytdl-org/youtube-dl#bugs) sections
@@ -36,7 +36,7 @@ Add the `-v` flag to **your command line** you run youtube-dl with (`youtube-dl
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
-[debug] youtube-dl version 2019.04.07
+[debug] youtube-dl version 2019.03.09
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -1,83 +1,3 @@
 version 2019.04.07
 Core
 + [downloader/external] Pass rtmp_conn to ffmpeg
 Extractors
 + [ruutu] Add support for audio podcasts (#20473, #20545)
 + [xvideos] Extract all thumbnails (#20432)
 + [platzi] Add support for platzi.com (#20562)
 * [dvtv] Fix extraction (#18514, #19174)
 + [vrv] Add basic support for individual movie links (#19229)
 + [bfi:player] Add support for player.bfi.org.uk (#19235)
 * [hbo] Fix extraction and extract subtitles (#14629, #13709)
 * [youtube] Extract srv[1-3] subtitle formats (#20566)
 * [adultswim] Fix extraction (#18025)
 * [teamcoco] Fix extraction and add suport for subdomains (#17099, #20339)
 * [adn] Fix subtitle compatibility with ffmpeg
 * [adn] Fix extraction and add support for positioning styles (#20549)
 * [vk] Use unique video id (#17848)
 * [newstube] Fix extraction
 * [rtl2] Actualize extraction
 + [adobeconnect] Add support for adobeconnect.com (#20283)
 + [gaia] Add support for authentication (#14605)
 + [mediasite] Add support for dashed ids and named catalogs (#20531)
 version 2019.04.01
 Core
 * [utils] Improve int_or_none and float_or_none (#20403)
 * Check for valid --min-sleep-interval when --max-sleep-interval is specified
  (#20435)
 Extractors
 + [weibo] Extend URL regular expression (#20496)
 + [xhamster] Add support for xhamster.one (#20508)
 + [mediasite] Add support for catalogs (#20507)
 + [teamtreehouse] Add support for teamtreehouse.com (#9836)
 + [ina] Add support for audio URLs
 * [ina] Improve extraction
 * [cwtv] Fix episode number extraction (#20461)
 * [npo] Improve DRM detection
 + [pornhub] Add support for DASH formats (#20403)
 * [svtplay] Update API endpoint (#20430)
 version 2019.03.18
 Core
 * [extractor/common] Improve HTML5 entries extraction
 + [utils] Introduce parse_bitrate
 * [update] Hide update URLs behind redirect
 * [extractor/common] Fix url meta field for unfragmented DASH formats (#20346)
 Extractors
 + [yandexvideo] Add extractor
 * [openload] Improve embed detection
 + [corus] Add support for bigbrothercanada.ca (#20357)
 + [orf:radio] Extract series (#20012)
 + [cbc:watch] Add support for gem.cbc.ca (#20251, #20359)
 - [anysex] Remove extractor (#19279)
 + [ciscolive] Add support for new URL schema (#20320, #20351)
 + [youtube] Add support for invidiou.sh (#20309)
 - [anitube] Remove extractor (#20334)
 - [ruleporn] Remove extractor (#15344, #20324)
 * [npr] Fix extraction (#10793, #13440)
 * [biqle] Fix extraction (#11471, #15313)
 * [viddler] Modernize
 * [moevideo] Fix extraction
 * [primesharetv] Remove extractor
 * [hypem] Modernize and extract more metadata (#15320)
 * [veoh] Fix extraction
 * [escapist] Modernize
 - [videomega] Remove extractor (#10108)
 + [beeg] Add support for beeg.porn (#20306)
 * [vimeo:review] Improve config url extraction and extract original format
  (#20305)
 * [fox] Detect geo restriction and authentication errors (#20208)
 version 2019.03.09
 Core
@@ -642,7 +642,6 @@ The simplest case is requesting a specific format, for example with `-f 22` you
 You can also use a file extension (currently `3gp`, `aac`, `flv`, `m4a`, `mp3`, `mp4`, `ogg`, `wav`, `webm` are supported) to download the best quality format of a particular file extension served as a single file, e.g. `-f webm` will download the best quality format with the `webm` extension served as a single file.
 You can also use special names to select particular edge case formats:
 - `best`: Select the best quality format represented by a single file with video and audio.
 - `worst`: Select the worst quality format represented by a single file with video and audio.
 - `bestvideo`: Select the best quality video-only format (e.g. DASH video). May not be available.
@@ -659,7 +658,6 @@ If you want to download several formats of the same video use a comma as a separ
 You can also filter the video formats by putting a condition in brackets, as in `-f "best[height=720]"` (or `-f "[filesize>10M]"`).
 The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `>=`, `=` (equals), `!=` (not equals):
 - `filesize`: The number of bytes, if known in advance
 - `width`: Width of the video, if known
 - `height`: Height of the video, if known
@@ -670,7 +668,6 @@ The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `
 - `fps`: Frame rate
 Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends with), `*=` (contains) and following string meta fields:
 - `ext`: File extension
 - `acodec`: Name of the audio codec in use
 - `vcodec`: Name of the video codec in use
@@ -28,7 +28,6 @@
 - **acast:channel**
 - **AddAnime**
 - **ADN**: Anime Digital Network
 - **AdobeConnect**
 - **AdobeTV**
 - **AdobeTVChannel**
 - **AdobeTVShow**
@@ -45,7 +44,9 @@
 - **AmericasTestKitchen**
 - **anderetijden**: npo.nl, ntr.nl, omroepwnl.nl, zapp.nl and npo3.nl
 - **AnimeOnDemand**
 - **anitube.se**
 - **Anvato**
 - **AnySex**
 - **APA**
 - **Aparat**
 - **AppleConnect**
@@ -102,7 +103,6 @@
 - **Bellator**
 - **BellMedia**
 - **Bet**
 - **bfi:player**
 - **Bigflix**
 - **Bild**: Bild.de
 - **BiliBili**
@@ -347,6 +347,7 @@
 - **Groupon**
 - **Hark**
 - **hbo**
 - **hbo:episode**
 - **HearThisAt**
 - **Heise**
 - **HellPorno**
@@ -489,8 +490,6 @@
 - **Medialaan**
 - **Mediaset**
 - **Mediasite**
 - **MediasiteCatalog**
 - **MediasiteNamedCatalog**
 - **Medici**
 - **megaphone.fm**: megaphone.fm embedded players
 - **Meipai**: 美拍
@@ -673,8 +672,6 @@
 - **Piksel**
 - **Pinkbike**
 - **Pladform**
 - **Platzi**
 - **PlatziCourse**
 - **play.fm**
 - **PlayPlusTV**
 - **PlaysTV**
@@ -701,6 +698,7 @@
 - **PornoXO**
 - **PornTube**
 - **PressTV**
 - **PrimeShareTV**
 - **PromptFile**
 - **prosiebensat1**: ProSiebenSat.1 Digital
 - **puhutv**
@@ -720,7 +718,7 @@
 - **radio.de**
 - **radiobremen**
 - **radiocanada**
- - **radiocanada:audiovideo**
+ - **RadioCanadaAudioVideo**
 - **radiofrance**
 - **RadioJavan**
 - **Rai**
@@ -767,6 +765,7 @@
 - **RTVS**
 - **Rudo**
 - **RUHD**
 - **RulePorn**
 - **rutube**: Rutube videos
 - **rutube:channel**: Rutube channels
 - **rutube:embed**: Rutube embedded videos
@@ -874,7 +873,6 @@
 - **teachertube:user:collection**: teachertube.com user and collection videos
 - **TeachingChannel**
 - **Teamcoco**
 - **TeamTreeHouse**
 - **TechTalks**
 - **techtv.mit.edu**
 - **ted**
@@ -1012,6 +1010,7 @@
 - **video.mit.edu**
 - **VideoDetective**
 - **videofy.me**
 - **VideoMega**
 - **videomore**
 - **videomore:season**
 - **videomore:video**
@@ -1128,7 +1127,6 @@
 - **yandexmusic:album**: Яндекс.Музыка - Альбом
 - **yandexmusic:playlist**: Яндекс.Музыка - Плейлист
 - **yandexmusic:track**: Яндекс.Музыка - Трек
 - **YandexVideo**
 - **YapFiles**
 - **YesJapan**
 - **yinyuetai:video**: 音悦Tai
@@ -107,184 +107,6 @@ class TestInfoExtractor(unittest.TestCase):
        self.assertRaises(ExtractorError, self.ie._download_json, uri, None)
        self.assertEqual(self.ie._download_json(uri, None, fatal=False), None)
    def test_parse_html5_media_entries(self):
        # from https://www.r18.com/
        # with kpbs in label
        expect_dict(
            self,
            self.ie._parse_html5_media_entries(
                'https://www.r18.com/',
                r'''
                <video id="samplevideo_amateur" class="js-samplevideo video-js vjs-default-skin vjs-big-play-centered" controls preload="auto" width="400" height="225" poster="//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg">
                    <source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_sm_w.mp4" type="video/mp4"  res="240" label="300kbps">
                    <source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dm_w.mp4" type="video/mp4"  res="480" label="1000kbps">
                    <source id="video_source" src="https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dmb_w.mp4" type="video/mp4"  res="740" label="1500kbps">
                    <p>Your browser does not support the video tag.</p>
                </video>
                ''', None)[0],
            {
                'formats': [{
                    'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_sm_w.mp4',
                    'ext': 'mp4',
                    'format_id': '300kbps',
                    'height': 240,
                    'tbr': 300,
                }, {
                    'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dm_w.mp4',
                    'ext': 'mp4',
                    'format_id': '1000kbps',
                    'height': 480,
                    'tbr': 1000,
                }, {
                    'url': 'https://awscc3001.r18.com/litevideo/freepv/m/mgm/mgmr105/mgmr105_dmb_w.mp4',
                    'ext': 'mp4',
                    'format_id': '1500kbps',
                    'height': 740,
                    'tbr': 1500,
                }],
                'thumbnail': '//pics.r18.com/digital/amateur/mgmr105/mgmr105jp.jpg'
            })
        # from https://www.csfd.cz/
        # with width and height
        expect_dict(
            self,
            self.ie._parse_html5_media_entries(
                'https://www.csfd.cz/',
                r'''
                <video width="770" height="328" preload="none" controls  poster="https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360" >
                    <source src="https://video.csfd.cz/files/videos/157/750/157750813/163327358_eac647.mp4" type="video/mp4" width="640" height="360">
                    <source src="https://video.csfd.cz/files/videos/157/750/157750813/163327360_3d2646.mp4" type="video/mp4" width="1280" height="720">
                    <source src="https://video.csfd.cz/files/videos/157/750/157750813/163327356_91f258.mp4" type="video/mp4" width="1920" height="1080">
                    <source src="https://video.csfd.cz/files/videos/157/750/157750813/163327359_962b4a.webm" type="video/webm" width="640" height="360">
                    <source src="https://video.csfd.cz/files/videos/157/750/157750813/163327361_6feee0.webm" type="video/webm" width="1280" height="720">
                    <source src="https://video.csfd.cz/files/videos/157/750/157750813/163327357_8ab472.webm" type="video/webm" width="1920" height="1080">
                    <track src="https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt" type="text/x-srt" kind="subtitles" srclang="cs" label="cs">
                </video>
                ''', None)[0],
            {
                'formats': [{
                    'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327358_eac647.mp4',
                    'ext': 'mp4',
                    'width': 640,
                    'height': 360,
                }, {
                    'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327360_3d2646.mp4',
                    'ext': 'mp4',
                    'width': 1280,
                    'height': 720,
                }, {
                    'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327356_91f258.mp4',
                    'ext': 'mp4',
                    'width': 1920,
                    'height': 1080,
                }, {
                    'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327359_962b4a.webm',
                    'ext': 'webm',
                    'width': 640,
                    'height': 360,
                }, {
                    'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327361_6feee0.webm',
                    'ext': 'webm',
                    'width': 1280,
                    'height': 720,
                }, {
                    'url': 'https://video.csfd.cz/files/videos/157/750/157750813/163327357_8ab472.webm',
                    'ext': 'webm',
                    'width': 1920,
                    'height': 1080,
                }],
                'subtitles': {
                    'cs': [{'url': 'https://video.csfd.cz/files/subtitles/163/344/163344115_4c388b.srt'}]
                },
                'thumbnail': 'https://img.csfd.cz/files/images/film/video/preview/163/344/163344118_748d20.png?h360'
            })
        # from https://tamasha.com/v/Kkdjw
        # with height in label
        expect_dict(
            self,
            self.ie._parse_html5_media_entries(
                'https://tamasha.com/v/Kkdjw',
                r'''
                <video crossorigin="anonymous">
                        <source src="https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4" type="video/mp4" label="AUTO" res="0"/>
                                <source src="https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4" type="video/mp4"
                                        label="240p" res="240"/>
                                <source src="https://s-v2.tamasha.com/statics/videos_file/20/00/Kkdjw_200041c66f657fc967db464d156eafbc1ed9fe6f_n_144.mp4" type="video/mp4"
                                        label="144p" res="144"/>
                </video>
                ''', None)[0],
            {
                'formats': [{
                    'url': 'https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4',
                }, {
                    'url': 'https://s-v2.tamasha.com/statics/videos_file/19/8f/Kkdjw_198feff8577d0057536e905cce1fb61438dd64e0_n_240.mp4',
                    'ext': 'mp4',
                    'format_id': '240p',
                    'height': 240,
                }, {
                    'url': 'https://s-v2.tamasha.com/statics/videos_file/20/00/Kkdjw_200041c66f657fc967db464d156eafbc1ed9fe6f_n_144.mp4',
                    'ext': 'mp4',
                    'format_id': '144p',
                    'height': 144,
                }]
            })
        # from https://www.directvnow.com
        # with data-src
        expect_dict(
            self,
            self.ie._parse_html5_media_entries(
                'https://www.directvnow.com',
                r'''
                <video id="vid1" class="header--video-masked active" muted playsinline>
                    <source data-src="https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4" type="video/mp4" />
                </video>
                ''', None)[0],
            {
                'formats': [{
                    'ext': 'mp4',
                    'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
                }]
            })
        # from https://www.directvnow.com
        # with data-src
        expect_dict(
            self,
            self.ie._parse_html5_media_entries(
                'https://www.directvnow.com',
                r'''
                <video id="vid1" class="header--video-masked active" muted playsinline>
                    <source data-src="https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4" type="video/mp4" />
                </video>
                ''', None)[0],
            {
                'formats': [{
                    'url': 'https://cdn.directv.com/content/dam/dtv/prod/website_directvnow-international/videos/DTVN_hdr_HBO_v3.mp4',
                    'ext': 'mp4',
                }]
            })
        # from https://www.klarna.com/uk/
        # with data-video-src
        expect_dict(
            self,
            self.ie._parse_html5_media_entries(
                'https://www.directvnow.com',
                r'''
                <video loop autoplay muted class="responsive-video block-kl__video video-on-medium">
                    <source src="" data-video-desktop data-video-src="https://www.klarna.com/uk/wp-content/uploads/sites/11/2019/01/KL062_Smooth3_0_DogWalking_5s_920x080_.mp4" type="video/mp4" />
                </video>
                ''', None)[0],
            {
                'formats': [{
                    'url': 'https://www.klarna.com/uk/wp-content/uploads/sites/11/2019/01/KL062_Smooth3_0_DogWalking_5s_920x080_.mp4',
                    'ext': 'mp4',
                }],
            })
    def test_extract_jwplayer_data_realworld(self):
        # from http://www.suffolk.edu/sjc/
        expect_dict(
@@ -752,8 +574,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                # Also tests duplicate representation ids, see
                # https://github.com/ytdl-org/youtube-dl/issues/15111
                'float_duration',
-                'http://unknown/manifest.mpd',  # mpd_url
+                'http://unknown/manifest.mpd',
                None,  # mpd_base_url
                [{
                    'manifest_url': 'http://unknown/manifest.mpd',
                    'ext': 'm4a',
@@ -833,8 +654,7 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
            ), (
                # https://github.com/ytdl-org/youtube-dl/pull/14844
                'urls_only',
-                'http://unknown/manifest.mpd',  # mpd_url
+                'http://unknown/manifest.mpd',
                None,  # mpd_base_url
                [{
                    'manifest_url': 'http://unknown/manifest.mpd',
                    'ext': 'mp4',
@@ -913,61 +733,15 @@ jwplayer("mediaplayer").setup({"abouttext":"Visit Indie DB","aboutlink":"http:\/
                    'width': 1920,
                    'height': 1080,
                }]
            ), (
                # https://github.com/ytdl-org/youtube-dl/issues/20346
                # Media considered unfragmented even though it contains
                # Initialization tag
                'unfragmented',
                'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',  # mpd_url
                'https://v.redd.it/hw1x7rcg7zl21',  # mpd_base_url
                [{
                    'url': 'https://v.redd.it/hw1x7rcg7zl21/audio',
                    'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
                    'ext': 'm4a',
                    'format_id': 'AUDIO-1',
                    'format_note': 'DASH audio',
                    'container': 'm4a_dash',
                    'acodec': 'mp4a.40.2',
                    'vcodec': 'none',
                    'tbr': 129.87,
                    'asr': 48000,
                }, {
                    'url': 'https://v.redd.it/hw1x7rcg7zl21/DASH_240',
                    'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
                    'ext': 'mp4',
                    'format_id': 'VIDEO-2',
                    'format_note': 'DASH video',
                    'container': 'mp4_dash',
                    'acodec': 'none',
                    'vcodec': 'avc1.4d401e',
                    'tbr': 608.0,
                    'width': 240,
                    'height': 240,
                    'fps': 30,
                }, {
                    'url': 'https://v.redd.it/hw1x7rcg7zl21/DASH_360',
                    'manifest_url': 'https://v.redd.it/hw1x7rcg7zl21/DASHPlaylist.mpd',
                    'ext': 'mp4',
                    'format_id': 'VIDEO-1',
                    'format_note': 'DASH video',
                    'container': 'mp4_dash',
                    'acodec': 'none',
                    'vcodec': 'avc1.4d401e',
                    'tbr': 804.261,
                    'width': 360,
                    'height': 360,
                    'fps': 30,
                }]
            )
        ]
-        for mpd_file, mpd_url, mpd_base_url, expected_formats in _TEST_CASES:
+        for mpd_file, mpd_url, expected_formats in _TEST_CASES:
            with io.open('./test/testdata/mpd/%s.mpd' % mpd_file,
                         mode='r', encoding='utf-8') as f:
                formats = self.ie._parse_mpd_formats(
                    compat_etree_fromstring(f.read().encode('utf-8')),
-                    mpd_base_url=mpd_base_url, mpd_url=mpd_url)
+                    mpd_url=mpd_url)
                self.ie._sort_formats(formats)
                expect_value(self, formats, expected_formats, None)
@@ -33,13 +33,11 @@ from youtube_dl.utils import (
    ExtractorError,
    find_xpath_attr,
    fix_xml_ampersands,
    float_or_none,
    get_element_by_class,
    get_element_by_attribute,
    get_elements_by_class,
    get_elements_by_attribute,
    InAdvancePagedList,
    int_or_none,
    intlist_to_bytes,
    is_html,
    js_to_json,
@@ -57,7 +55,6 @@ from youtube_dl.utils import (
    parse_count,
    parse_iso8601,
    parse_resolution,
    parse_bitrate,
    pkcs1pad,
    read_batch_urls,
    sanitize_filename,
@@ -470,21 +467,6 @@ class TestUtil(unittest.TestCase):
            shell_quote(args),
            """ffmpeg -i 'ñ€ß'"'"'.mp4'""" if compat_os_name != 'nt' else '''ffmpeg -i "ñ€ß'.mp4"''')
    def test_float_or_none(self):
        self.assertEqual(float_or_none('42.42'), 42.42)
        self.assertEqual(float_or_none('42'), 42.0)
        self.assertEqual(float_or_none(''), None)
        self.assertEqual(float_or_none(None), None)
        self.assertEqual(float_or_none([]), None)
        self.assertEqual(float_or_none(set()), None)
    def test_int_or_none(self):
        self.assertEqual(int_or_none('42'), 42)
        self.assertEqual(int_or_none(''), None)
        self.assertEqual(int_or_none(None), None)
        self.assertEqual(int_or_none([]), None)
        self.assertEqual(int_or_none(set()), None)
    def test_str_to_int(self):
        self.assertEqual(str_to_int('123,456'), 123456)
        self.assertEqual(str_to_int('123.456'), 123456)
@@ -1048,13 +1030,6 @@ class TestUtil(unittest.TestCase):
        self.assertEqual(parse_resolution('4k'), {'height': 2160})
        self.assertEqual(parse_resolution('8K'), {'height': 4320})
    def test_parse_bitrate(self):
        self.assertEqual(parse_bitrate(None), None)
        self.assertEqual(parse_bitrate(''), None)
        self.assertEqual(parse_bitrate('300kbps'), 300)
        self.assertEqual(parse_bitrate('1500kbps'), 1500)
        self.assertEqual(parse_bitrate('300 kbps'), 300)
    def test_version_tuple(self):
        self.assertEqual(version_tuple('1'), (1,))
        self.assertEqual(version_tuple('10.23.344'), (10, 23, 344))
@@ -1,28 +0,0 @@
 <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 <MPD mediaPresentationDuration="PT54.915S" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
    <Period duration="PT54.915S">
        <AdaptationSet segmentAlignment="true" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
            <Representation bandwidth="804261" codecs="avc1.4d401e" frameRate="30" height="360" id="VIDEO-1" mimeType="video/mp4" startWithSAP="1" width="360">
                <BaseURL>DASH_360</BaseURL>
                <SegmentBase indexRange="915-1114" indexRangeExact="true">
                    <Initialization range="0-914"/>
                </SegmentBase>
            </Representation>
            <Representation bandwidth="608000" codecs="avc1.4d401e" frameRate="30" height="240" id="VIDEO-2" mimeType="video/mp4" startWithSAP="1" width="240">
                <BaseURL>DASH_240</BaseURL>
                <SegmentBase indexRange="913-1112" indexRangeExact="true">
                    <Initialization range="0-912"/>
                </SegmentBase>
            </Representation>
        </AdaptationSet>
        <AdaptationSet>
            <Representation audioSamplingRate="48000" bandwidth="129870" codecs="mp4a.40.2" id="AUDIO-1" mimeType="audio/mp4" startWithSAP="1">
                <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
                <BaseURL>audio</BaseURL>
                <SegmentBase indexRange="832-1007" indexRangeExact="true">
                    <Initialization range="0-831"/>
                </SegmentBase>
            </Representation>
        </AdaptationSet>
    </Period>
 </MPD>
@@ -309,8 +309,6 @@ class YoutubeDL(object):
    The following options are used by the post processors:
    prefer_ffmpeg:     If False, use avconv instead of ffmpeg if both are available,
                       otherwise prefer ffmpeg.
    ffmpeg_location:   Location of the ffmpeg/avconv binary; either the path
                       to the binary or its containing directory.
    postprocessor_args: A list of additional command-line arguments for the
                        postprocessor.
@@ -166,8 +166,6 @@ def _real_main(argv=None):
    if opts.max_sleep_interval is not None:
        if opts.max_sleep_interval < 0:
            parser.error('max sleep interval must be positive or 0')
        if opts.sleep_interval is None:
            parser.error('min sleep interval must be specified, use --min-sleep-interval')
        if opts.max_sleep_interval < opts.sleep_interval:
            parser.error('max sleep interval must be greater than or equal to min sleep interval')
    else:
@@ -289,7 +289,6 @@ class FFmpegFD(ExternalFD):
            tc_url = info_dict.get('tc_url')
            flash_version = info_dict.get('flash_version')
            live = info_dict.get('rtmp_live', False)
            conn = info_dict.get('rtmp_conn')
            if player_url is not None:
                args += ['-rtmp_swfverify', player_url]
            if page_url is not None:
@@ -304,11 +303,6 @@ class FFmpegFD(ExternalFD):
                args += ['-rtmp_flashver', flash_version]
            if live:
                args += ['-rtmp_live', 'live']
            if isinstance(conn, list):
                for entry in conn:
                    args += ['-rtmp_conn', entry]
            elif isinstance(conn, compat_str):
                args += ['-rtmp_conn', conn]
        args += ['-i', url, '-c', 'copy']
@@ -21,6 +21,7 @@ from ..utils import (
    intlist_to_bytes,
    long_to_bytes,
    pkcs1pad,
    srt_subtitles_timecode,
    strip_or_none,
    urljoin,
 )
@@ -41,18 +42,6 @@ class ADNIE(InfoExtractor):
    }
    _BASE_URL = 'http://animedigitalnetwork.fr'
    _RSA_KEY = (0xc35ae1e4356b65a73b551493da94b8cb443491c0aa092a357a5aee57ffc14dda85326f42d716e539a34542a0d3f363adf16c5ec222d713d5997194030ee2e4f0d1fb328c01a81cf6868c090d50de8e169c6b13d1675b9eeed1cbc51e1fffca9b38af07f37abd790924cd3bee59d0257cfda4fe5f3f0534877e21ce5821447d1b, 65537)
    _POS_ALIGN_MAP = {
        'start': 1,
        'end': 3,
    }
    _LINE_ALIGN_MAP = {
        'middle': 8,
        'end': 4,
    }
    @staticmethod
    def _ass_subtitles_timecode(seconds):
        return '%01d:%02d:%02d.%02d' % (seconds / 3600, (seconds % 3600) / 60, seconds % 60, (seconds % 1) * 100)
    def _get_subtitles(self, sub_path, video_id):
        if not sub_path:
@@ -60,14 +49,14 @@ class ADNIE(InfoExtractor):
        enc_subtitles = self._download_webpage(
            urljoin(self._BASE_URL, sub_path),
-            video_id, 'Downloading subtitles data', fatal=False)
+            video_id, fatal=False)
        if not enc_subtitles:
            return None
        # http://animedigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js
        dec_subtitles = intlist_to_bytes(aes_cbc_decrypt(
            bytes_to_intlist(compat_b64decode(enc_subtitles[24:])),
-            bytes_to_intlist(binascii.unhexlify(self._K + '083db5aebd9353b4')),
+            bytes_to_intlist(binascii.unhexlify(self._K + '9032ad7083106400')),
            bytes_to_intlist(compat_b64decode(enc_subtitles[:24]))
        ))
        subtitles_json = self._parse_json(
@@ -78,27 +67,23 @@ class ADNIE(InfoExtractor):
        subtitles = {}
        for sub_lang, sub in subtitles_json.items():
-            ssa = '''[Script Info]
+            srt = ''
-ScriptType:V4.00
+            for num, current in enumerate(sub):
-[V4 Styles]
+                start, end, text = (
 Format: Name,Fontname,Fontsize,PrimaryColour,SecondaryColour,TertiaryColour,BackColour,Bold,Italic,BorderStyle,Outline,Shadow,Alignment,MarginL,MarginR,MarginV,AlphaLevel,Encoding
 Style: Default,Arial,18,16777215,16777215,16777215,0,-1,0,1,1,0,2,20,20,20,0,0
 [Events]
 Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
            for current in sub:
                start, end, text, line_align, position_align = (
                    float_or_none(current.get('startTime')),
                    float_or_none(current.get('endTime')),
-                    current.get('text'), current.get('lineAlign'),
+                    current.get('text'))
                    current.get('positionAlign'))
                if start is None or end is None or text is None:
                    continue
-                alignment = self._POS_ALIGN_MAP.get(position_align, 2) + self._LINE_ALIGN_MAP.get(line_align, 0)
+                srt += os.linesep.join(
-                ssa += os.linesep + 'Dialogue: Marked=0,%s,%s,Default,,0,0,0,,%s%s' % (
+                    (
-                    self._ass_subtitles_timecode(start),
+                        '%d' % num,
-                    self._ass_subtitles_timecode(end),
+                        '%s --> %s' % (
-                    '{\\a%d}' % alignment if alignment != 2 else '',
+                            srt_subtitles_timecode(start),
-                    text.replace('\n', '\\N').replace('<i>', '{\\i1}').replace('</i>', '{\\i0}'))
+                            srt_subtitles_timecode(end)),
                        text,
                        os.linesep,
                    ))
            if sub_lang == 'vostf':
                sub_lang = 'fr'
@@ -106,8 +91,8 @@ Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
                'ext': 'json',
                'data': json.dumps(sub),
            }, {
-                'ext': 'ssa',
+                'ext': 'srt',
-                'data': ssa,
+                'data': srt,
            }])
        return subtitles
@@ -115,15 +100,7 @@ Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        player_config = self._parse_json(self._search_regex(
-            r'playerConfig\s*=\s*({.+});', webpage,
+            r'playerConfig\s*=\s*({.+});', webpage, 'player config'), video_id)
            'player config', default='{}'), video_id, fatal=False)
        if not player_config:
            config_url = urljoin(self._BASE_URL, self._search_regex(
                r'(?:id="player"|class="[^"]*adn-player-container[^"]*")[^>]+data-url="([^"]+)"',
                webpage, 'config url'))
            player_config = self._download_json(
                config_url, video_id,
                'Downloading player config JSON metadata')['player']
        video_info = {}
        video_info_str = self._search_regex(
@@ -152,15 +129,12 @@ Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
            encrypted_message = long_to_bytes(pow(bytes_to_long(padded_message), e, n))
            authorization = base64.b64encode(encrypted_message).decode()
            links_data = self._download_json(
-                urljoin(self._BASE_URL, links_url), video_id,
+                urljoin(self._BASE_URL, links_url), video_id, headers={
                'Downloading links JSON metadata', headers={
                    'Authorization': 'Bearer ' + authorization,
                })
            links = links_data.get('links') or {}
            metas = metas or links_data.get('meta') or {}
-            sub_path = sub_path or links_data.get('subtitles') or \
+            sub_path = (sub_path or links_data.get('subtitles')) + '&token=' + token
                'index.php?option=com_vodapi&task=subtitles.getJSON&format=json&id=' + video_id
            sub_path += '&token=' + token
            error = links_data.get('error')
        title = metas.get('title') or video_info['title']
@@ -168,11 +142,9 @@ Format: Marked,Start,End,Style,Name,MarginL,MarginR,MarginV,Effect,Text'''
        for format_id, qualities in links.items():
            if not isinstance(qualities, dict):
                continue
-            for quality, load_balancer_url in qualities.items():
+            for load_balancer_url in qualities.values():
                load_balancer_data = self._download_json(
-                    load_balancer_url, video_id,
+                    load_balancer_url, video_id, fatal=False) or {}
                    'Downloading %s %s JSON metadata' % (format_id, quality),
                    fatal=False) or {}
                m3u8_url = load_balancer_data.get('location')
                if not m3u8_url:
                    continue
@@ -1,37 +0,0 @@
 # coding: utf-8
 from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..compat import (
    compat_parse_qs,
    compat_urlparse,
 )
 class AdobeConnectIE(InfoExtractor):
    _VALID_URL = r'https?://\w+\.adobeconnect\.com/(?P<id>[\w-]+)'
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        title = self._html_search_regex(r'<title>(.+?)</title>', webpage, 'title')
        qs = compat_parse_qs(self._search_regex(r"swfUrl\s*=\s*'([^']+)'", webpage, 'swf url').split('?')[1])
        is_live = qs.get('isLive', ['false'])[0] == 'true'
        formats = []
        for con_string in qs['conStrings'][0].split(','):
            formats.append({
                'format_id': con_string.split('://')[0],
                'app': compat_urlparse.quote('?' + con_string.split('?')[1] + 'flvplayerapp/' + qs['appInstance'][0]),
                'ext': 'flv',
                'play_path': 'mp4:' + qs['streamName'][0],
                'rtmp_conn': 'S:' + qs['ticket'][0],
                'rtmp_live': is_live,
                'url': con_string,
            })
        return {
            'id': video_id,
            'title': self._live_title(title) if is_live else title,
            'formats': formats,
            'is_live': is_live,
        }
@@ -1,19 +1,13 @@
 # coding: utf-8
 from __future__ import unicode_literals
 import json
 import re
 from .turner import TurnerBaseIE
 from ..utils import (
    determine_ext,
    float_or_none,
    int_or_none,
    mimetype2ext,
    parse_age_limit,
    parse_iso8601,
    strip_or_none,
-    try_get,
+    url_or_none,
 )
@@ -27,8 +21,8 @@ class AdultSwimIE(TurnerBaseIE):
            'ext': 'mp4',
            'title': 'Rick and Morty - Pilot',
            'description': 'Rick moves in with his daughter\'s family and establishes himself as a bad influence on his grandson, Morty.',
-            'timestamp': 1543294800,
+            'timestamp': 1493267400,
-            'upload_date': '20181127',
+            'upload_date': '20170427',
        },
        'params': {
            # m3u8 download
@@ -49,7 +43,6 @@ class AdultSwimIE(TurnerBaseIE):
            # m3u8 download
            'skip_download': True,
        },
        'skip': '404 Not Found',
    }, {
        'url': 'http://www.adultswim.com/videos/decker/inside-decker-a-new-hero/',
        'info_dict': {
@@ -68,9 +61,9 @@ class AdultSwimIE(TurnerBaseIE):
    }, {
        'url': 'http://www.adultswim.com/videos/attack-on-titan',
        'info_dict': {
-            'id': 'attack-on-titan',
+            'id': 'b7A69dzfRzuaXIECdxW8XQ',
            'title': 'Attack on Titan',
-            'description': 'md5:41caa9416906d90711e31dc00cb7db7e',
+            'description': 'md5:6c8e003ea0777b47013e894767f5e114',
        },
        'playlist_mincount': 12,
    }, {
@@ -85,118 +78,83 @@ class AdultSwimIE(TurnerBaseIE):
            # m3u8 download
            'skip_download': True,
        },
        'skip': '404 Not Found',
    }]
    def _real_extract(self, url):
        show_path, episode_path = re.match(self._VALID_URL, url).groups()
        display_id = episode_path or show_path
-        query = '''query {
+        webpage = self._download_webpage(url, display_id)
-  getShowBySlug(slug:"%s") {
+        initial_data = self._parse_json(self._search_regex(
-    %%s
+            r'AS_INITIAL_DATA(?:__)?\s*=\s*({.+?});',
-  }
+            webpage, 'initial data'), display_id)
 }''' % show_path
        if episode_path:
            query = query % '''title
    getVideoBySlug(slug:"%s") {
      _id
      auth
      description
      duration
      episodeNumber
      launchDate
      mediaID
      seasonNumber
      poster
      title
      tvRating
    }''' % episode_path
            ['getVideoBySlug']
        else:
            query = query % '''metaDescription
    title
    videos(first:1000,sort:["episode_number"]) {
      edges {
        node {
           _id
           slug
        }
      }
    }'''
        show_data = self._download_json(
            'https://www.adultswim.com/api/search', display_id,
            data=json.dumps({'query': query}).encode(),
            headers={'Content-Type': 'application/json'})['data']['getShowBySlug']
        if episode_path:
            video_data = show_data['getVideoBySlug']
            video_id = video_data['_id']
            episode_title = title = video_data['title']
            series = show_data.get('title')
            if series:
                title = '%s - %s' % (series, title)
            info = {
                'id': video_id,
                'title': title,
                'description': strip_or_none(video_data.get('description')),
                'duration': float_or_none(video_data.get('duration')),
                'formats': [],
                'subtitles': {},
                'age_limit': parse_age_limit(video_data.get('tvRating')),
                'thumbnail': video_data.get('poster'),
                'timestamp': parse_iso8601(video_data.get('launchDate')),
                'series': series,
                'season_number': int_or_none(video_data.get('seasonNumber')),
                'episode': episode_title,
                'episode_number': int_or_none(video_data.get('episodeNumber')),
            }
-            auth = video_data.get('auth')
+        is_stream = show_path == 'streams'
-            media_id = video_data.get('mediaID')
+        if is_stream:
-            if media_id:
+            if not episode_path:
-                info.update(self._extract_ngtv_info(media_id, {
+                episode_path = 'live-stream'
                    # CDN_TOKEN_APP_ID from:
                    # https://d2gg02c3xr550i.cloudfront.net/assets/asvp.e9c8bef24322d060ef87.bundle.js
                    'appId': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBJZCI6ImFzLXR2ZS1kZXNrdG9wLXB0enQ2bSIsInByb2R1Y3QiOiJ0dmUiLCJuZXR3b3JrIjoiYXMiLCJwbGF0Zm9ybSI6ImRlc2t0b3AiLCJpYXQiOjE1MzI3MDIyNzl9.BzSCk-WYOZ2GMCIaeVb8zWnzhlgnXuJTCu0jGp_VaZE',
                }, {
                    'url': url,
                    'site_name': 'AdultSwim',
                    'auth_required': auth,
                }))
-            if not auth:
+            video_data = next(stream for stream_path, stream in initial_data['streams'].items() if stream_path == episode_path)
-                extract_data = self._download_json(
+            video_id = video_data.get('stream')
-                    'https://www.adultswim.com/api/shows/v1/videos/' + video_id,
+
-                    video_id, query={'fields': 'stream'}, fatal=False) or {}
+            if not video_id:
-                assets = try_get(extract_data, lambda x: x['data']['video']['stream']['assets'], list) or []
+                entries = []
-                for asset in assets:
+                for episode in video_data.get('archiveEpisodes', []):
-                    asset_url = asset.get('url')
+                    episode_url = url_or_none(episode.get('url'))
-                    if not asset_url:
+                    if not episode_url:
                        continue
-                    ext = determine_ext(asset_url, mimetype2ext(asset.get('mime_type')))
+                    entries.append(self.url_result(
-                    if ext == 'm3u8':
+                        episode_url, 'AdultSwim', episode.get('id')))
-                        info['formats'].extend(self._extract_m3u8_formats(
+                return self.playlist_result(
-                            asset_url, video_id, 'mp4', m3u8_id='hls', fatal=False))
+                    entries, video_data.get('id'), video_data.get('title'),
-                    elif ext == 'f4m':
+                    strip_or_none(video_data.get('description')))
                        continue
                        # info['formats'].extend(self._extract_f4m_formats(
                        #     asset_url, video_id, f4m_id='hds', fatal=False))
                    elif ext in ('scc', 'ttml', 'vtt'):
                        info['subtitles'].setdefault('en', []).append({
                            'url': asset_url,
                        })
            self._sort_formats(info['formats'])
            return info
        else:
-            entries = []
+            show_data = initial_data['show']
-            for edge in show_data.get('videos', {}).get('edges', []):
+
-                video = edge.get('node') or {}
+            if not episode_path:
-                slug = video.get('slug')
+                entries = []
-                if not slug:
+                for video in show_data.get('videos', []):
-                    continue
+                    slug = video.get('slug')
-                entries.append(self.url_result(
+                    if not slug:
-                    'http://adultswim.com/videos/%s/%s' % (show_path, slug),
+                        continue
-                    'AdultSwim', video.get('_id')))
+                    entries.append(self.url_result(
-            return self.playlist_result(
+                        'http://adultswim.com/videos/%s/%s' % (show_path, slug),
-                entries, show_path, show_data.get('title'),
+                        'AdultSwim', video.get('id')))
-                strip_or_none(show_data.get('metaDescription')))
+                return self.playlist_result(
                    entries, show_data.get('id'), show_data.get('title'),
                    strip_or_none(show_data.get('metadata', {}).get('description')))
            video_data = show_data['sluggedVideo']
            video_id = video_data['id']
        info = self._extract_cvp_info(
            'http://www.adultswim.com/videos/api/v0/assets?platform=desktop&id=' + video_id,
            video_id, {
                'secure': {
                    'media_src': 'http://androidhls-secure.cdn.turner.com/adultswim/big',
                    'tokenizer_src': 'http://www.adultswim.com/astv/mvpd/processors/services/token_ipadAdobe.do',
                },
            }, {
                'url': url,
                'site_name': 'AdultSwim',
                'auth_required': video_data.get('auth'),
            })
        info.update({
            'id': video_id,
            'display_id': display_id,
            'description': info.get('description') or strip_or_none(video_data.get('description')),
        })
        if not is_stream:
            info.update({
                'duration': info.get('duration') or int_or_none(video_data.get('duration')),
                'timestamp': info.get('timestamp') or int_or_none(video_data.get('launch_date')),
                'season_number': info.get('season_number') or int_or_none(video_data.get('season_number')),
                'episode': info['title'],
                'episode_number': info.get('episode_number') or int_or_none(video_data.get('episode_number')),
            })
            info['series'] = video_data.get('collection_title') or info.get('series')
            if info['series'] and info['series'] != info['title']:
                info['title'] = '%s - %s' % (info['series'], info['title'])
        return info
@@ -0,0 +1,30 @@
 from __future__ import unicode_literals
 from .nuevo import NuevoBaseIE
 class AnitubeIE(NuevoBaseIE):
    IE_NAME = 'anitube.se'
    _VALID_URL = r'https?://(?:www\.)?anitube\.se/video/(?P<id>\d+)'
    _TEST = {
        'url': 'http://www.anitube.se/video/36621',
        'md5': '59d0eeae28ea0bc8c05e7af429998d43',
        'info_dict': {
            'id': '36621',
            'ext': 'mp4',
            'title': 'Recorder to Randoseru 01',
            'duration': 180.19,
        },
        'skip': 'Blocked in the US',
    }
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        key = self._search_regex(
            r'src=["\']https?://[^/]+/embed/([A-Za-z0-9_-]+)', webpage, 'key')
        return self._extract_nuevo(
            'http://www.anitube.se/nuevo/econfig.php?key=%s' % key, video_id)
@@ -0,0 +1,61 @@
 from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..utils import (
    parse_duration,
    int_or_none,
 )
 class AnySexIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?anysex\.com/(?P<id>\d+)'
    _TEST = {
        'url': 'http://anysex.com/156592/',
        'md5': '023e9fbb7f7987f5529a394c34ad3d3d',
        'info_dict': {
            'id': '156592',
            'ext': 'mp4',
            'title': 'Busty and sexy blondie in her bikini strips for you',
            'description': 'md5:de9e418178e2931c10b62966474e1383',
            'categories': ['Erotic'],
            'duration': 270,
            'age_limit': 18,
        }
    }
    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
        video_id = mobj.group('id')
        webpage = self._download_webpage(url, video_id)
        video_url = self._html_search_regex(r"video_url\s*:\s*'([^']+)'", webpage, 'video URL')
        title = self._html_search_regex(r'<title>(.*?)</title>', webpage, 'title')
        description = self._html_search_regex(
            r'<div class="description"[^>]*>([^<]+)</div>', webpage, 'description', fatal=False)
        thumbnail = self._html_search_regex(
            r'preview_url\s*:\s*\'(.*?)\'', webpage, 'thumbnail', fatal=False)
        categories = re.findall(
            r'<a href="http://anysex\.com/categories/[^"]+" title="[^"]*">([^<]+)</a>', webpage)
        duration = parse_duration(self._search_regex(
            r'<b>Duration:</b> (?:<q itemprop="duration">)?(\d+:\d+)', webpage, 'duration', fatal=False))
        view_count = int_or_none(self._html_search_regex(
            r'<b>Views:</b> (\d+)', webpage, 'view count', fatal=False))
        return {
            'id': video_id,
            'url': video_url,
            'ext': 'mp4',
            'title': title,
            'description': description,
            'thumbnail': thumbnail,
            'categories': categories,
            'duration': duration,
            'view_count': view_count,
            'age_limit': 18,
        }
@@ -9,8 +9,8 @@ from ..utils import (
 class BeegIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?beeg\.(?:com|porn(?:/video)?)/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?beeg\.com/(?P<id>\d+)'
-    _TESTS = [{
+    _TEST = {
        'url': 'http://beeg.com/5416503',
        'md5': 'a1a1b1a8bc70a89e49ccfd113aed0820',
        'info_dict': {
@@ -24,13 +24,7 @@ class BeegIE(InfoExtractor):
            'tags': list,
            'age_limit': 18,
        }
-    }, {
+    }
        'url': 'https://beeg.porn/video/5416503',
        'only_matching': True,
    }, {
        'url': 'https://beeg.porn/5416503',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        video_id = self._match_id(url)
@@ -1,37 +0,0 @@
 # coding: utf-8
 from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..utils import extract_attributes
 class BFIPlayerIE(InfoExtractor):
    IE_NAME = 'bfi:player'
    _VALID_URL = r'https?://player\.bfi\.org\.uk/[^/]+/film/watch-(?P<id>[\w-]+)-online'
    _TEST = {
        'url': 'https://player.bfi.org.uk/free/film/watch-computer-doctor-1974-online',
        'md5': 'e8783ebd8e061ec4bc6e9501ed547de8',
        'info_dict': {
            'id': 'htNnhlZjE60C9VySkQEIBtU-cNV1Xx63',
            'ext': 'mp4',
            'title': 'Computer Doctor',
            'description': 'md5:fb6c240d40c4dbe40428bdd62f78203b',
        },
        'skip': 'BFI Player films cannot be played outside of the UK',
    }
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        entries = []
        for player_el in re.findall(r'(?s)<[^>]+class="player"[^>]*>', webpage):
            player_attr = extract_attributes(player_el)
            ooyala_id = player_attr.get('data-video-id')
            if not ooyala_id:
                continue
            entries.append(self.url_result(
                'ooyala:' + ooyala_id, 'Ooyala',
                ooyala_id, player_attr.get('data-label')))
        return self.playlist_result(entries)
@@ -2,96 +2,39 @@
 from __future__ import unicode_literals
 from .common import InfoExtractor
 from .vk import VKIE
 from ..utils import (
    HEADRequest,
    int_or_none,
 )
 class BIQLEIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?biqle\.(?:com|org|ru)/watch/(?P<id>-?\d+_\d+)'
    _TESTS = [{
-        # Youtube embed
+        'url': 'http://www.biqle.ru/watch/847655_160197695',
-        'url': 'https://biqle.ru/watch/-115995369_456239081',
+        'md5': 'ad5f746a874ccded7b8f211aeea96637',
        'md5': '97af5a06ee4c29bbf9c001bdb1cf5c06',
        'info_dict': {
-            'id': '8v4f-avW-VI',
+            'id': '160197695',
            'ext': 'mp4',
-            'title': "PASSE-PARTOUT - L'ete c'est fait pour jouer",
+            'title': 'Foo Fighters - The Pretender (Live at Wembley Stadium)',
-            'description': 'Passe-Partout',
+            'uploader': 'Andrey Rogozin',
-            'uploader_id': 'mrsimpsonstef3',
+            'upload_date': '20110605',
-            'uploader': 'Phanolito',
+        }
            'upload_date': '20120822',
        },
    }, {
-        'url': 'http://biqle.org/watch/-44781847_168547604',
+        'url': 'https://biqle.org/watch/-44781847_168547604',
        'md5': '7f24e72af1db0edf7c1aaba513174f97',
        'info_dict': {
-            'id': '-44781847_168547604',
+            'id': '168547604',
            'ext': 'mp4',
            'title': 'Ребенок в шоке от автоматической мойки',
            'timestamp': 1396633454,
            'uploader': 'Dmitry Kotov',
            'upload_date': '20140404',
            'uploader_id': '47850140',
        },
        'skip': ' This video was marked as adult.  Embedding adult videos on external sites is prohibited.',
    }]
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        embed_url = self._proto_relative_url(self._search_regex(
-            r'<iframe.+?src="((?:https?:)?//daxab\.com/[^"]+)".*?></iframe>',
+            r'<iframe.+?src="((?:http:)?//daxab\.com/[^"]+)".*?></iframe>', webpage, 'embed url'))
            webpage, 'embed url'))
        if VKIE.suitable(embed_url):
            return self.url_result(embed_url, VKIE.ie_key(), video_id)
        self._request_webpage(
            HEADRequest(embed_url), video_id, headers={'Referer': url})
        video_id, sig, _, access_token = self._get_cookies(embed_url)['video_ext'].value.split('%3A')
        item = self._download_json(
            'https://api.vk.com/method/video.get', video_id,
            headers={'User-Agent': 'okhttp/3.4.1'}, query={
                'access_token': access_token,
                'sig': sig,
                'v': 5.44,
                'videos': video_id,
            })['response']['items'][0]
        title = item['title']
        formats = []
        for f_id, f_url in item.get('files', {}).items():
            if f_id == 'external':
                return self.url_result(f_url)
            ext, height = f_id.split('_')
            formats.append({
                'format_id': height + 'p',
                'url': f_url,
                'height': int_or_none(height),
                'ext': ext,
            })
        self._sort_formats(formats)
        thumbnails = []
        for k, v in item.items():
            if k.startswith('photo_') and v:
                width = k.replace('photo_', '')
                thumbnails.append({
                    'id': width,
                    'url': v,
                    'width': int_or_none(width),
                })
        return {
-            'id': video_id,
+            '_type': 'url_transparent',
-            'title': title,
+            'url': embed_url,
            'formats': formats,
            'comment_count': int_or_none(item.get('comments')),
            'description': item.get('description'),
            'duration': int_or_none(item.get('duration')),
            'thumbnails': thumbnails,
            'timestamp': int_or_none(item.get('date')),
            'uploader': item.get('owner_id'),
            'view_count': int_or_none(item.get('views')),
        }
@@ -360,7 +360,7 @@ class CBCWatchVideoIE(CBCWatchBaseIE):
 class CBCWatchIE(CBCWatchBaseIE):
    IE_NAME = 'cbc.ca:watch'
-    _VALID_URL = r'https?://(?:gem|watch)\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
+    _VALID_URL = r'https?://watch\.cbc\.ca/(?:[^/]+/)+(?P<id>[0-9a-f-]+)'
    _TESTS = [{
        # geo-restricted to Canada, bypassable
        'url': 'http://watch.cbc.ca/doc-zone/season-6/customer-disservice/38e815a-009e3ab12e4',
@@ -386,9 +386,6 @@ class CBCWatchIE(CBCWatchBaseIE):
            'description': 'Arthur, the sweetest 8-year-old aardvark, and his pals solve all kinds of problems with humour, kindness and teamwork.',
        },
        'playlist_mincount': 30,
    }, {
        'url': 'https://gem.cbc.ca/media/this-hour-has-22-minutes/season-26/episode-20/38e815a-0108c6c6a42',
        'only_matching': True,
    }]
    def _real_extract(self, url):
@@ -65,8 +65,8 @@ class CiscoLiveBaseIE(InfoExtractor):
 class CiscoLiveSessionIE(CiscoLiveBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?ciscolive(?:\.cisco)?\.com/[^#]*#/session/(?P<id>[^/?&]+)'
+    _VALID_URL = r'https?://ciscolive\.cisco\.com/on-demand-library/\??[^#]*#/session/(?P<id>[^/?&]+)'
-    _TESTS = [{
+    _TEST = {
        'url': 'https://ciscolive.cisco.com/on-demand-library/?#/session/1423353499155001FoSs',
        'md5': 'c98acf395ed9c9f766941c70f5352e22',
        'info_dict': {
@@ -79,13 +79,7 @@ class CiscoLiveSessionIE(CiscoLiveBaseIE):
            'uploader_id': '5647924234001',
            'location': '16B Mezz.',
        },
-    }, {
+    }
        'url': 'https://www.ciscolive.com/global/on-demand-library.html?search.event=ciscoliveemea2019#/session/15361595531500013WOU',
        'only_matching': True,
    }, {
        'url': 'https://www.ciscolive.com/global/on-demand-library.html?#/session/1490051371645001kNaS',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        rf_id = self._match_id(url)
@@ -94,7 +88,7 @@ class CiscoLiveSessionIE(CiscoLiveBaseIE):
 class CiscoLiveSearchIE(CiscoLiveBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?ciscolive(?:\.cisco)?\.com/(?:global/)?on-demand-library(?:\.html|/)'
+    _VALID_URL = r'https?://ciscolive\.cisco\.com/on-demand-library/'
    _TESTS = [{
        'url': 'https://ciscolive.cisco.com/on-demand-library/?search.event=ciscoliveus2018&search.technicallevel=scpsSkillLevel_aintroductory&search.focus=scpsSessionFocus_designAndDeployment#/',
        'info_dict': {
@@ -104,9 +98,6 @@ class CiscoLiveSearchIE(CiscoLiveBaseIE):
    }, {
        'url': 'https://ciscolive.cisco.com/on-demand-library/?search.technology=scpsTechnology_applicationDevelopment&search.technology=scpsTechnology_ipv6&search.focus=scpsSessionFocus_troubleshootingTroubleshooting#/',
        'only_matching': True,
    }, {
        'url': 'https://www.ciscolive.com/global/on-demand-library.html?search.technicallevel=scpsSkillLevel_aintroductory&search.event=ciscoliveemea2019&search.technology=scpsTechnology_dataCenter&search.focus=scpsSessionFocus_bestPractices#/',
        'only_matching': True,
    }]
    @classmethod
@@ -44,7 +44,6 @@ from ..utils import (
    compiled_regex_type,
    determine_ext,
    determine_protocol,
    dict_get,
    error_to_compat_str,
    ExtractorError,
    extract_attributes,
@@ -57,16 +56,13 @@ from ..utils import (
    JSON_LD_RE,
    mimetype2ext,
    orderedSet,
    parse_bitrate,
    parse_codecs,
    parse_duration,
    parse_iso8601,
    parse_m3u8_attributes,
    parse_resolution,
    RegexNotFoundError,
    sanitized_Request,
    sanitize_filename,
    str_or_none,
    unescapeHTML,
    unified_strdate,
    unified_timestamp,
@@ -112,13 +108,10 @@ class InfoExtractor(object):
                                   for RTMP - RTMP URL,
                                   for HLS - URL of the M3U8 media playlist,
                                   for HDS - URL of the F4M manifest,
-                                   for DASH
+                                   for DASH - URL of the MPD manifest or
-                                     - HTTP URL to plain file media (in case of
+                                              base URL representing the media
-                                       unfragmented media)
+                                              if MPD manifest is parsed from
-                                     - URL of the MPD manifest or base URL
+                                              a string,
                                       representing the media if MPD manifest
                                       is parsed froma string (in case of
                                       fragmented media)
                                   for MSS - URL of the ISM manifest.
                    * manifest_url
                                 The URL of the manifest file in case of
@@ -2144,6 +2137,8 @@ class InfoExtractor(object):
                        bandwidth = int_or_none(representation_attrib.get('bandwidth'))
                        f = {
                            'format_id': '%s-%s' % (mpd_id, representation_id) if mpd_id else representation_id,
                            # NB: mpd_url may be empty when MPD manifest is parsed from a string
                            'url': mpd_url or base_url,
                            'manifest_url': mpd_url,
                            'ext': mimetype2ext(mime_type),
                            'width': int_or_none(representation_attrib.get('width')),
@@ -2282,14 +2277,10 @@ class InfoExtractor(object):
                                    fragment['duration'] = segment_duration
                                fragments.append(fragment)
                            representation_ms_info['fragments'] = fragments
-                        # If there is a fragments key available then we correctly recognized fragmented media.
+                        # NB: MPD manifest may contain direct URLs to unfragmented media.
-                        # Otherwise we will assume unfragmented media with direct access. Technically, such
+                        # No fragments key is present in this case.
                        # assumption is not necessarily correct since we may simply have no support for
                        # some forms of fragmented media renditions yet, but for now we'll use this fallback.
                        if 'fragments' in representation_ms_info:
                            f.update({
                                # NB: mpd_url may be empty when MPD manifest is parsed from a string
                                'url': mpd_url or base_url,
                                'fragment_base_url': base_url,
                                'fragments': [],
                                'protocol': 'http_dash_segments',
@@ -2300,10 +2291,6 @@ class InfoExtractor(object):
                                    f['url'] = initialization_url
                                f['fragments'].append({location_key(initialization_url): initialization_url})
                            f['fragments'].extend(representation_ms_info['fragments'])
                        else:
                            # Assuming direct URL to unfragmented media.
                            f['url'] = base_url
                        # According to [1, 5.3.5.2, Table 7, page 35] @id of Representation
                        # is not necessarily unique within a Period thus formats with
                        # the same `format_id` are quite possible. There are numerous examples
@@ -2485,43 +2472,18 @@ class InfoExtractor(object):
            media_info['thumbnail'] = absolute_url(media_attributes.get('poster'))
            if media_content:
                for source_tag in re.findall(r'<source[^>]+>', media_content):
-                    s_attr = extract_attributes(source_tag)
+                    source_attributes = extract_attributes(source_tag)
-                    # data-video-src and data-src are non standard but seen
+                    src = source_attributes.get('src')
                    # several times in the wild
                    src = dict_get(s_attr, ('src', 'data-video-src', 'data-src'))
                    if not src:
                        continue
-                    f = parse_content_type(s_attr.get('type'))
+                    f = parse_content_type(source_attributes.get('type'))
                    is_plain_url, formats = _media_formats(src, media_type, f)
                    if is_plain_url:
-                        # width, height, res, label and title attributes are
+                        # res attribute is not standard but seen several times
-                        # all not standard but seen several times in the wild
+                        # in the wild
                        labels = [
                            s_attr.get(lbl)
                            for lbl in ('label', 'title')
                            if str_or_none(s_attr.get(lbl))
                        ]
                        width = int_or_none(s_attr.get('width'))
                        height = (int_or_none(s_attr.get('height')) or
                                  int_or_none(s_attr.get('res')))
                        if not width or not height:
                            for lbl in labels:
                                resolution = parse_resolution(lbl)
                                if not resolution:
                                    continue
                                width = width or resolution.get('width')
                                height = height or resolution.get('height')
                        for lbl in labels:
                            tbr = parse_bitrate(lbl)
                            if tbr:
                                break
                        else:
                            tbr = None
                        f.update({
-                            'width': width,
+                            'height': int_or_none(source_attributes.get('res')),
-                            'height': height,
+                            'format_id': source_attributes.get('label'),
                            'tbr': tbr,
                            'format_id': s_attr.get('label') or s_attr.get('title'),
                        })
                        f.update(formats[0])
                        media_info['formats'].append(f)
@@ -13,9 +13,9 @@ class CorusIE(ThePlatformFeedIE):
                        (?:www\.)?
                        (?P<domain>
                            (?:globaltv|etcanada)\.com|
-                            (?:hgtv|foodnetwork|slice|history|showcase|bigbrothercanada)\.ca
+                            (?:hgtv|foodnetwork|slice|history|showcase)\.ca
                        )
-                        /(?:video/(?:[^/]+/)?|(?:[^/]+/)+(?:videos/[a-z0-9-]+-|video\.html\?.*?\bv=))
+                        /(?:video/|(?:[^/]+/)+(?:videos/[a-z0-9-]+-|video\.html\?.*?\bv=))
                        (?P<id>\d+)
                    '''
    _TESTS = [{
@@ -42,12 +42,6 @@ class CorusIE(ThePlatformFeedIE):
    }, {
        'url': 'http://www.showcase.ca/eyewitness/video/eyewitness++106/video.html?v=955070531919&p=1&s=da#video',
        'only_matching': True,
    }, {
        'url': 'http://www.bigbrothercanada.ca/video/1457812035894/',
        'only_matching': True
    }, {
        'url': 'https://www.bigbrothercanada.ca/video/big-brother-canada-704/1457812035894/',
        'only_matching': True
    }]
    _TP_FEEDS = {
@@ -79,10 +73,6 @@ class CorusIE(ThePlatformFeedIE):
            'feed_id': '9H6qyshBZU3E',
            'account_id': 2414426607,
        },
        'bigbrothercanada': {
            'feed_id': 'ChQqrem0lNUp',
            'account_id': 2269680845,
        },
    }
    def _real_extract(self, url):
@@ -79,7 +79,7 @@ class CWTVIE(InfoExtractor):
        season = str_or_none(video_data.get('season'))
        episode = str_or_none(video_data.get('episode'))
        if episode and season:
-            episode = episode[len(season):]
+            episode = episode.lstrip(season)
        return {
            '_type': 'url_transparent',
@@ -10,16 +10,16 @@ from ..utils import (
    int_or_none,
    js_to_json,
    mimetype2ext,
    try_get,
    unescapeHTML,
    parse_iso8601,
 )
 class DVTVIE(InfoExtractor):
    IE_NAME = 'dvtv'
    IE_DESC = 'http://video.aktualne.cz/'
    _VALID_URL = r'https?://video\.aktualne\.cz/(?:[^/]+/)+r~(?P<id>[0-9a-f]{32})'
    _TESTS = [{
        'url': 'http://video.aktualne.cz/dvtv/vondra-o-ceskem-stoleti-pri-pohledu-na-havla-mi-bylo-trapne/r~e5efe9ca855511e4833a0025900fea04/',
        'md5': '67cb83e4a955d36e1b5d31993134a0c2',
@@ -28,13 +28,11 @@ class DVTVIE(InfoExtractor):
            'ext': 'mp4',
            'title': 'Vondra o Českém století: Při pohledu na Havla mi bylo trapně',
            'duration': 1484,
            'upload_date': '20141217',
            'timestamp': 1418792400,
        }
    }, {
        'url': 'http://video.aktualne.cz/dvtv/dvtv-16-12-2014-utok-talibanu-boj-o-kliniku-uprchlici/r~973eb3bc854e11e498be002590604f2e/',
        'info_dict': {
-            'title': r'DVTV 16. 12. 2014: útok Talibanu, boj o kliniku, uprchlíci',
+            'title': r're:^DVTV 16\. 12\. 2014: útok Talibanu, boj o kliniku, uprchlíci',
            'id': '973eb3bc854e11e498be002590604f2e',
        },
        'playlist': [{
@@ -86,8 +84,6 @@ class DVTVIE(InfoExtractor):
            'ext': 'mp4',
            'title': 'Zeman si jen léčí mindráky, Sobotku nenávidí a Babiš se mu teď hodí, tvrdí Kmenta',
            'duration': 1103,
            'upload_date': '20170511',
            'timestamp': 1494514200,
        },
        'params': {
            'skip_download': True,
@@ -95,59 +91,43 @@ class DVTVIE(InfoExtractor):
    }, {
        'url': 'http://video.aktualne.cz/v-cechach-poprve-zazni-zelenkova-zrestaurovana-mse/r~45b4b00483ec11e4883b002590604f2e/',
        'only_matching': True,
    }, {
        # Test live stream video (liveStarter) parsing
        'url': 'https://video.aktualne.cz/dvtv/zive-mistryne-sveta-eva-samkova-po-navratu-ze-sampionatu/r~182654c2288811e990fd0cc47ab5f122/',
        'md5': '2e552e483f2414851ca50467054f9d5d',
        'info_dict': {
            'id': '8d116360288011e98c840cc47ab5f122',
            'ext': 'mp4',
            'title': 'Živě: Mistryně světa Eva Samková po návratu ze šampionátu',
            'upload_date': '20190204',
            'timestamp': 1549289591,
        },
        'params': {
            # Video content is no longer available
            'skip_download': True,
        },
    }]
-    def _parse_video_metadata(self, js, video_id, timestamp):
+    def _parse_video_metadata(self, js, video_id, live_js=None):
        data = self._parse_json(js, video_id, transform_source=js_to_json)
        if live_js:
            data.update(self._parse_json(
                live_js, video_id, transform_source=js_to_json))
        title = unescapeHTML(data['title'])
        live_starter = try_get(data, lambda x: x['plugins']['liveStarter'], dict)
        if live_starter:
            data.update(live_starter)
        formats = []
-        for tracks in data.get('tracks', {}).values():
+        for video in data['sources']:
-            for video in tracks:
+            video_url = video.get('file')
-                video_url = video.get('src')
+            if not video_url:
-                if not video_url:
+                continue
-                    continue
+            video_type = video.get('type')
-                video_type = video.get('type')
+            ext = determine_ext(video_url, mimetype2ext(video_type))
-                ext = determine_ext(video_url, mimetype2ext(video_type))
+            if video_type == 'application/vnd.apple.mpegurl' or ext == 'm3u8':
-                if video_type == 'application/vnd.apple.mpegurl' or ext == 'm3u8':
+                formats.extend(self._extract_m3u8_formats(
-                    formats.extend(self._extract_m3u8_formats(
+                    video_url, video_id, 'mp4', entry_protocol='m3u8_native',
-                        video_url, video_id, 'mp4', entry_protocol='m3u8_native',
+                    m3u8_id='hls', fatal=False))
-                        m3u8_id='hls', fatal=False))
+            elif video_type == 'application/dash+xml' or ext == 'mpd':
-                elif video_type == 'application/dash+xml' or ext == 'mpd':
+                formats.extend(self._extract_mpd_formats(
-                    formats.extend(self._extract_mpd_formats(
+                    video_url, video_id, mpd_id='dash', fatal=False))
-                        video_url, video_id, mpd_id='dash', fatal=False))
+            else:
-                else:
+                label = video.get('label')
-                    label = video.get('label')
+                height = self._search_regex(
-                    height = self._search_regex(
+                    r'^(\d+)[pP]', label or '', 'height', default=None)
-                        r'^(\d+)[pP]', label or '', 'height', default=None)
+                format_id = ['http']
-                    format_id = ['http']
+                for f in (ext, label):
-                    for f in (ext, label):
+                    if f:
-                        if f:
+                        format_id.append(f)
-                            format_id.append(f)
+                formats.append({
-                    formats.append({
+                    'url': video_url,
-                        'url': video_url,
+                    'format_id': '-'.join(format_id),
-                        'format_id': '-'.join(format_id),
+                    'height': int_or_none(height),
-                        'height': int_or_none(height),
+                })
                    })
        self._sort_formats(formats)
        return {
@@ -156,29 +136,41 @@ class DVTVIE(InfoExtractor):
            'description': data.get('description'),
            'thumbnail': data.get('image'),
            'duration': int_or_none(data.get('duration')),
-            'timestamp': int_or_none(timestamp),
+            'timestamp': int_or_none(data.get('pubtime')),
            'formats': formats
        }
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        timestamp = parse_iso8601(self._html_search_meta(
            'article:published_time', webpage, 'published time', default=None))
-        items = re.findall(r'(?s)playlist\.push\(({.+?})\);', webpage)
+        # live content
-        if items:
+        live_item = self._search_regex(
-            return self.playlist_result(
+            r'(?s)embedData[0-9a-f]{32}\.asset\.liveStarter\s*=\s*(\{.+?\});',
                [self._parse_video_metadata(i, video_id, timestamp) for i in items],
                video_id, self._html_search_meta('twitter:title', webpage))
        item = self._search_regex(
            r'(?s)BBXPlayer\.setup\((.+?)\);',
            webpage, 'video', default=None)
        # single video
        item = self._search_regex(
            r'(?s)embedData[0-9a-f]{32}\[["\']asset["\']\]\s*=\s*(\{.+?\});',
            webpage, 'video', default=None)
        if item:
-            # remove function calls (ex. htmldeentitize)
+            return self._parse_video_metadata(item, video_id, live_item)
-            # TODO this should be fixed in a general way in the js_to_json
+
-            item = re.sub(r'\w+?\((.+)\)', r'\1', item)
+        # playlist
-            return self._parse_video_metadata(item, video_id, timestamp)
+        items = re.findall(
            r"(?s)BBX\.context\.assets\['[0-9a-f]{32}'\]\.push\(({.+?})\);",
            webpage)
        if not items:
            items = re.findall(r'(?s)var\s+asset\s*=\s*({.+?});\n', webpage)
        if items:
            return {
                '_type': 'playlist',
                'id': video_id,
                'title': self._og_search_title(webpage),
                'entries': [self._parse_video_metadata(i, video_id) for i in items]
            }
        raise ExtractorError('Could not find neither video nor playlist')
@@ -1,11 +1,14 @@
 from __future__ import unicode_literals
 import json
 from .common import InfoExtractor
 from ..utils import (
    determine_ext,
    clean_html,
    int_or_none,
    float_or_none,
    sanitized_Request,
 )
@@ -33,7 +36,7 @@ def _decrypt_config(key, string):
 class EscapistIE(InfoExtractor):
-    _VALID_URL = r'https?://?(?:(?:www|v1)\.)?escapistmagazine\.com/videos/view/[^/]+/(?P<id>[0-9]+)'
+    _VALID_URL = r'https?://?(?:www\.)?escapistmagazine\.com/videos/view/[^/?#]+/(?P<id>[0-9]+)-[^/?#]*(?:$|[?#])'
    _TESTS = [{
        'url': 'http://www.escapistmagazine.com/videos/view/the-escapist-presents/6618-Breaking-Down-Baldurs-Gate',
        'md5': 'ab3a706c681efca53f0a35f1415cf0d1',
@@ -58,12 +61,6 @@ class EscapistIE(InfoExtractor):
            'duration': 304,
            'uploader': 'The Escapist',
        }
    }, {
        'url': 'http://escapistmagazine.com/videos/view/the-escapist-presents/6618',
        'only_matching': True,
    }, {
        'url': 'https://v1.escapistmagazine.com/videos/view/the-escapist-presents/6618-Breaking-Down-Baldurs-Gate',
        'only_matching': True,
    }]
    def _real_extract(self, url):
@@ -77,20 +74,19 @@ class EscapistIE(InfoExtractor):
        video_id = ims_video['videoID']
        key = ims_video['hash']
-        config = self._download_webpage(
+        config_req = sanitized_Request(
-            'http://www.escapistmagazine.com/videos/vidconfig.php',
+            'http://www.escapistmagazine.com/videos/'
-            video_id, 'Downloading video config', headers={
+            'vidconfig.php?videoID=%s&hash=%s' % (video_id, key))
-                'Referer': url,
+        config_req.add_header('Referer', url)
-            }, query={
+        config = self._download_webpage(config_req, video_id, 'Downloading video config')
                'videoID': video_id,
                'hash': key,
            })
-        data = self._parse_json(_decrypt_config(key, config), video_id)
+        data = json.loads(_decrypt_config(key, config))
        video_data = data['videoData']
        title = clean_html(video_data['title'])
        duration = float_or_none(video_data.get('duration'), 1000)
        uploader = video_data.get('publisher')
        formats = [{
            'url': video['src'],
@@ -103,9 +99,8 @@ class EscapistIE(InfoExtractor):
            'id': video_id,
            'formats': formats,
            'title': title,
-            'thumbnail': self._og_search_thumbnail(webpage) or data.get('poster'),
+            'thumbnail': self._og_search_thumbnail(webpage),
            'description': self._og_search_description(webpage),
-            'duration': float_or_none(video_data.get('duration'), 1000),
+            'duration': duration,
-            'uploader': video_data.get('publisher'),
+            'uploader': uploader,
            'series': video_data.get('show'),
        }
@@ -20,7 +20,6 @@ from .acast import (
 )
 from .addanime import AddAnimeIE
 from .adn import ADNIE
 from .adobeconnect import AdobeConnectIE
 from .adobetv import (
    AdobeTVIE,
    AdobeTVShowIE,
@@ -39,7 +38,9 @@ from .alphaporno import AlphaPornoIE
 from .amcnetworks import AMCNetworksIE
 from .americastestkitchen import AmericasTestKitchenIE
 from .animeondemand import AnimeOnDemandIE
 from .anitube import AnitubeIE
 from .anvato import AnvatoIE
 from .anysex import AnySexIE
 from .aol import AolIE
 from .allocine import AllocineIE
 from .aliexpress import AliExpressLiveIE
@@ -107,7 +108,6 @@ from .behindkink import BehindKinkIE
 from .bellmedia import BellMediaIE
 from .beatport import BeatportIE
 from .bet import BetIE
 from .bfi import BFIPlayerIE
 from .bigflix import BigflixIE
 from .bild import BildIE
 from .bilibili import (
@@ -442,7 +442,10 @@ from .goshgay import GoshgayIE
 from .gputechconf import GPUTechConfIE
 from .groupon import GrouponIE
 from .hark import HarkIE
-from .hbo import HBOIE
+from .hbo import (
    HBOIE,
    HBOEpisodeIE,
 )
 from .hearthisat import HearThisAtIE
 from .heise import HeiseIE
 from .hellporno import HellPornoIE
@@ -631,11 +634,7 @@ from .massengeschmacktv import MassengeschmackTVIE
 from .matchtv import MatchTVIE
 from .mdr import MDRIE
 from .mediaset import MediasetIE
-from .mediasite import (
+from .mediasite import MediasiteIE
    MediasiteIE,
    MediasiteCatalogIE,
    MediasiteNamedCatalogIE,
 )
 from .medici import MediciIE
 from .megaphone import MegaphoneIE
 from .meipai import MeipaiIE
@@ -868,10 +867,6 @@ from .picarto import (
 from .piksel import PikselIE
 from .pinkbike import PinkbikeIE
 from .pladform import PladformIE
 from .platzi import (
    PlatziIE,
    PlatziCourseIE,
 )
 from .playfm import PlayFMIE
 from .playplustv import PlayPlusTVIE
 from .plays import PlaysTVIE
@@ -906,6 +901,7 @@ from .puhutv import (
    PuhuTVSerieIE,
 )
 from .presstv import PressTVIE
 from .primesharetv import PrimeShareTVIE
 from .promptfile import PromptFileIE
 from .prosiebensat1 import ProSiebenSat1IE
 from .puls4 import Puls4IE
@@ -982,6 +978,7 @@ from .rtvnh import RTVNHIE
 from .rtvs import RTVSIE
 from .rudo import RudoIE
 from .ruhd import RUHDIE
 from .ruleporn import RulePornIE
 from .rutube import (
    RutubeIE,
    RutubeChannelIE,
@@ -1121,7 +1118,6 @@ from .teachertube import (
 )
 from .teachingchannel import TeachingChannelIE
 from .teamcoco import TeamcocoIE
 from .teamtreehouse import TeamTreeHouseIE
 from .techtalks import TechTalksIE
 from .ted import TEDIE
 from .tele5 import Tele5IE
@@ -1303,6 +1299,7 @@ from .viddler import ViddlerIE
 from .videa import VideaIE
 from .videodetective import VideoDetectiveIE
 from .videofyme import VideofyMeIE
 from .videomega import VideoMegaIE
 from .videomore import (
    VideomoreIE,
    VideomoreVideoIE,
@@ -1408,7 +1405,7 @@ from .webofstories import (
    WebOfStoriesPlaylistIE,
 )
 from .weibo import (
-    WeiboIE,
+    WeiboIE, 
    WeiboMobileIE
 )
 from .weiqitv import WeiqiTVIE
@@ -1452,13 +1449,12 @@ from .yahoo import (
    YahooIE,
    YahooSearchIE,
 )
 from .yandexdisk import YandexDiskIE
 from .yandexmusic import (
    YandexMusicTrackIE,
    YandexMusicAlbumIE,
    YandexMusicPlaylistIE,
 )
-from .yandexvideo import YandexVideoIE
+from .yandexdisk import YandexDiskIE
 from .yapfiles import YapFilesIE
 from .yesjapan import YesJapanIE
 from .yinyuetai import YinYueTaiIE
@@ -6,12 +6,10 @@ import uuid
 from .adobepass import AdobePassIE
 from ..compat import (
    compat_HTTPError,
    compat_str,
    compat_urllib_parse_unquote,
 )
 from ..utils import (
    ExtractorError,
    int_or_none,
    parse_age_limit,
    parse_duration,
@@ -50,7 +48,6 @@ class FOXIE(AdobePassIE):
        'url': 'https://www.fox.com/watch/30056b295fb57f7452aeeb4920bc3024/',
        'only_matching': True,
    }]
    _GEO_BYPASS = False
    _HOME_PAGE_URL = 'https://www.fox.com/'
    _API_KEY = 'abdcbed02c124d393b39e818a4312055'
    _access_token = None
@@ -61,22 +58,9 @@ class FOXIE(AdobePassIE):
        }
        if self._access_token:
            headers['Authorization'] = 'Bearer ' + self._access_token
-        try:
+        return self._download_json(
-            return self._download_json(
+            'https://api2.fox.com/v2.0/' + path,
-                'https://api2.fox.com/v2.0/' + path,
+            video_id, data=data, headers=headers)
                video_id, data=data, headers=headers)
        except ExtractorError as e:
            if isinstance(e.cause, compat_HTTPError) and e.cause.status == 403:
                entitlement_issues = self._parse_json(
                    e.cause.read().decode(), video_id)['entitlementIssues']
                for e in entitlement_issues:
                    if e.get('errorCode') == 1005:
                        raise ExtractorError(
                            'This video is only available via cable service provider '
                            'subscription. You may want to use --cookies.', expected=True)
                messages = ', '.join([e['message'] for e in entitlement_issues])
                raise ExtractorError(messages, expected=True)
            raise
    def _real_initialize(self):
        if not self._access_token:
@@ -97,15 +81,7 @@ class FOXIE(AdobePassIE):
        title = video['name']
        release_url = video['url']
-        try:
+        m3u8_url = self._download_json(release_url, video_id)['playURL']
            m3u8_url = self._download_json(release_url, video_id)['playURL']
        except ExtractorError as e:
            if isinstance(e.cause, compat_HTTPError) and e.cause.status == 403:
                error = self._parse_json(e.cause.read().decode(), video_id)
                if error.get('exception') == 'GeoLocationBlocked':
                    self.raise_geo_restricted(countries=['US'])
                raise ExtractorError(error['description'], expected=True)
            raise
        formats = self._extract_m3u8_formats(
            m3u8_url, video_id, 'mp4',
            entry_protocol='m3u8_native', m3u8_id='hls')
@@ -4,17 +4,12 @@ from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
-from ..compat import (
+from ..compat import compat_str
    compat_str,
    compat_urllib_parse_unquote,
 )
 from ..utils import (
    ExtractorError,
    int_or_none,
    str_or_none,
    strip_or_none,
    try_get,
    urlencode_postdata,
 )
@@ -51,29 +46,6 @@ class GaiaIE(InfoExtractor):
            'skip_download': True,
        },
    }]
    _NETRC_MACHINE = 'gaia'
    _jwt = None
    def _real_initialize(self):
        auth = self._get_cookies('https://www.gaia.com/').get('auth')
        if auth:
            auth = self._parse_json(
                compat_urllib_parse_unquote(auth.value),
                None, fatal=False)
        if not auth:
            username, password = self._get_login_info()
            if username is None:
                return
            auth = self._download_json(
                'https://auth.gaia.com/v1/login',
                None, data=urlencode_postdata({
                    'username': username,
                    'password': password
                }))
            if auth.get('success') is False:
                raise ExtractorError(', '.join(auth['messages']), expected=True)
        if auth:
            self._jwt = auth.get('jwt')
    def _real_extract(self, url):
        display_id, vtype = re.search(self._VALID_URL, url).groups()
@@ -87,12 +59,8 @@ class GaiaIE(InfoExtractor):
        media_id = compat_str(vdata['nid'])
        title = node['title']
        headers = None
        if self._jwt:
            headers = {'Authorization': 'Bearer ' + self._jwt}
        media = self._download_json(
-            'https://brooklyn.gaia.com/media/' + media_id,
+            'https://brooklyn.gaia.com/media/' + media_id, media_id)
            media_id, headers=headers)
        formats = self._extract_m3u8_formats(
            media['mediaUrls']['bcHLS'], media_id, 'mp4')
        self._sort_formats(formats)
@@ -4,28 +4,16 @@ from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..compat import compat_str
 from ..utils import (
    xpath_text,
    xpath_element,
    int_or_none,
    parse_duration,
    urljoin,
 )
-class HBOIE(InfoExtractor):
+class HBOBaseIE(InfoExtractor):
    IE_NAME = 'hbo'
    _VALID_URL = r'https?://(?:www\.)?hbo\.com/(?:video|embed)(?:/[^/]+)*/(?P<id>[^/?#]+)'
    _TEST = {
        'url': 'https://www.hbo.com/video/game-of-thrones/seasons/season-8/videos/trailer',
        'md5': '8126210656f433c452a21367f9ad85b3',
        'info_dict': {
            'id': '22113301',
            'ext': 'mp4',
            'title': 'Game of Thrones - Trailer',
        },
        'expected_warnings': ['Unknown MIME type application/mp4 in DASH manifest'],
    }
    _FORMATS_INFO = {
        'pro7': {
            'width': 1280,
@@ -65,17 +53,10 @@ class HBOIE(InfoExtractor):
        },
    }
-    def _real_extract(self, url):
+    def _extract_from_id(self, video_id):
-        display_id = self._match_id(url)
+        video_data = self._download_xml(
-        webpage = self._download_webpage(url, display_id)
+            'http://render.lv3.hbo.com/data/content/global/videos/data/%s.xml' % video_id, video_id)
-        location_path = self._parse_json(self._html_search_regex(
+        title = xpath_text(video_data, 'title', 'title', True)
            r'data-state="({.+?})"', webpage, 'state'), display_id)['video']['locationUrl']
        video_data = self._download_xml(urljoin(url, location_path), display_id)
        video_id = xpath_text(video_data, 'id', fatal=True)
        episode_title = title = xpath_text(video_data, 'title', fatal=True)
        series = xpath_text(video_data, 'program')
        if series:
            title = '%s - %s' % (series, title)
        formats = []
        for source in xpath_element(video_data, 'videos', 'sources', True):
@@ -147,23 +128,68 @@ class HBOIE(InfoExtractor):
                    'width': width,
                })
        subtitles = None
        caption_url = xpath_text(video_data, 'captionUrl')
        if caption_url:
            subtitles = {
                'en': [{
                    'url': caption_url,
                    'ext': 'ttml'
                }],
            }
        return {
            'id': video_id,
            'title': title,
            'duration': parse_duration(xpath_text(video_data, 'duration/tv14')),
            'series': series,
            'episode': episode_title,
            'formats': formats,
            'thumbnails': thumbnails,
            'subtitles': subtitles,
        }
 class HBOIE(HBOBaseIE):
    IE_NAME = 'hbo'
    _VALID_URL = r'https?://(?:www\.)?hbo\.com/video/video\.html\?.*vid=(?P<id>[0-9]+)'
    _TEST = {
        'url': 'http://www.hbo.com/video/video.html?autoplay=true&g=u&vid=1437839',
        'md5': '2c6a6bc1222c7e91cb3334dad1746e5a',
        'info_dict': {
            'id': '1437839',
            'ext': 'mp4',
            'title': 'Ep. 64 Clip: Encryption',
            'thumbnail': r're:https?://.*\.jpg$',
            'duration': 1072,
        }
    }
    def _real_extract(self, url):
        video_id = self._match_id(url)
        return self._extract_from_id(video_id)
 class HBOEpisodeIE(HBOBaseIE):
    IE_NAME = 'hbo:episode'
    _VALID_URL = r'https?://(?:www\.)?hbo\.com/(?P<path>(?!video)(?:(?:[^/]+/)+video|watch-free-episodes)/(?P<id>[0-9a-z-]+))(?:\.html)?'
    _TESTS = [{
        'url': 'http://www.hbo.com/girls/episodes/5/52-i-love-you-baby/video/ep-52-inside-the-episode.html?autoplay=true',
        'md5': '61ead79b9c0dfa8d3d4b07ef4ac556fb',
        'info_dict': {
            'id': '1439518',
            'display_id': 'ep-52-inside-the-episode',
            'ext': 'mp4',
            'title': 'Ep. 52: Inside the Episode',
            'thumbnail': r're:https?://.*\.jpg$',
            'duration': 240,
        },
    }, {
        'url': 'http://www.hbo.com/game-of-thrones/about/video/season-5-invitation-to-the-set.html?autoplay=true',
        'only_matching': True,
    }, {
        'url': 'http://www.hbo.com/watch-free-episodes/last-week-tonight-with-john-oliver',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        path, display_id = re.match(self._VALID_URL, url).groups()
        content = self._download_json(
            'http://www.hbo.com/api/content/' + path, display_id)['content']
        video_id = compat_str((content.get('parsed', {}).get(
            'common:FullBleedVideo', {}) or content['selectedEpisode'])['videoId'])
        info_dict = self._extract_from_id(video_id)
        info_dict['display_id'] = display_id
        return info_dict
@@ -1,11 +1,18 @@
 from __future__ import unicode_literals
 import json
 import time
 from .common import InfoExtractor
-from ..utils import int_or_none
+from ..compat import compat_urllib_parse_urlencode
 from ..utils import (
    ExtractorError,
    sanitized_Request,
 )
 class HypemIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?hypem\.com/track/(?P<id>[0-9a-z]{5})'
+    _VALID_URL = r'https?://(?:www\.)?hypem\.com/track/(?P<id>[^/]+)/'
    _TEST = {
        'url': 'http://hypem.com/track/1v6ga/BODYWORK+-+TAME',
        'md5': 'b9cc91b5af8995e9f0c1cee04c575828',
@@ -14,36 +21,41 @@ class HypemIE(InfoExtractor):
            'ext': 'mp3',
            'title': 'Tame',
            'uploader': 'BODYWORK',
            'timestamp': 1371810457,
            'upload_date': '20130621',
        }
    }
    def _real_extract(self, url):
        track_id = self._match_id(url)
-        response = self._download_webpage(url, track_id)
+        data = {'ax': 1, 'ts': time.time()}
        request = sanitized_Request(url + '?' + compat_urllib_parse_urlencode(data))
        response, urlh = self._download_webpage_handle(
            request, track_id, 'Downloading webpage with the url')
-        track = self._parse_json(self._html_search_regex(
+        html_tracks = self._html_search_regex(
-            r'(?s)<script\s+type="application/json"\s+id="displayList-data">(.+?)</script>',
+            r'(?ms)<script type="application/json" id="displayList-data">(.+?)</script>',
-            response, 'tracks'), track_id)['tracks'][0]
+            response, 'tracks')
        try:
            track_list = json.loads(html_tracks)
            track = track_list['tracks'][0]
        except ValueError:
            raise ExtractorError('Hypemachine contained invalid JSON.')
        key = track['key']
        track_id = track['id']
        title = track['song']
-        final_url = self._download_json(
+        request = sanitized_Request(
-            'http://hypem.com/serve/source/%s/%s' % (track_id, track['key']),
+            'http://hypem.com/serve/source/%s/%s' % (track_id, key),
-            track_id, 'Downloading metadata', headers={
+            '', {'Content-Type': 'application/json'})
-                'Content-Type': 'application/json'
+        song_data = self._download_json(request, track_id, 'Downloading metadata')
-            })['url']
+        final_url = song_data['url']
        artist = track.get('artist')
        return {
            'id': track_id,
            'url': final_url,
            'ext': 'mp3',
            'title': title,
-            'uploader': track.get('artist'),
+            'uploader': artist,
            'duration': int_or_none(track.get('time')),
            'timestamp': int_or_none(track.get('ts')),
            'track': title,
        }
@@ -1,83 +1,36 @@
 # coding: utf-8
 from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..utils import (
    determine_ext,
    int_or_none,
    strip_or_none,
    xpath_attr,
    xpath_text,
 )
 class InaIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?ina\.fr/(?:video|audio)/(?P<id>[A-Z0-9_]+)'
+    _VALID_URL = r'https?://(?:www\.)?ina\.fr/video/(?P<id>I?[A-Z0-9]+)'
-    _TESTS = [{
+    _TEST = {
        'url': 'http://www.ina.fr/video/I12055569/francois-hollande-je-crois-que-c-est-clair-video.html',
        'md5': 'a667021bf2b41f8dc6049479d9bb38a3',
        'info_dict': {
            'id': 'I12055569',
            'ext': 'mp4',
            'title': 'François Hollande "Je crois que c\'est clair"',
            'description': 'md5:3f09eb072a06cb286b8f7e4f77109663',
        }
-    }, {
+    }
        'url': 'https://www.ina.fr/video/S806544_001/don-d-organes-des-avancees-mais-d-importants-besoins-video.html',
        'only_matching': True,
    }, {
        'url': 'https://www.ina.fr/audio/P16173408',
        'only_matching': True,
    }, {
        'url': 'https://www.ina.fr/video/P16173408-video.html',
        'only_matching': True,
    }]
    def _real_extract(self, url):
-        video_id = self._match_id(url)
+        mobj = re.match(self._VALID_URL, url)
        info_doc = self._download_xml(
            'http://player.ina.fr/notices/%s.mrss' % video_id, video_id)
        item = info_doc.find('channel/item')
        title = xpath_text(item, 'title', fatal=True)
        media_ns_xpath = lambda x: self._xpath_ns(x, 'http://search.yahoo.com/mrss/')
        content = item.find(media_ns_xpath('content'))
-        get_furl = lambda x: xpath_attr(content, media_ns_xpath(x), 'url')
+        video_id = mobj.group('id')
-        formats = []
+        mrss_url = 'http://player.ina.fr/notices/%s.mrss' % video_id
-        for q, w, h in (('bq', 400, 300), ('mq', 512, 384), ('hq', 768, 576)):
+        info_doc = self._download_xml(mrss_url, video_id)
            q_url = get_furl(q)
            if not q_url:
                continue
            formats.append({
                'format_id': q,
                'url': q_url,
                'width': w,
                'height': h,
            })
        if not formats:
            furl = get_furl('player') or content.attrib['url']
            ext = determine_ext(furl)
            formats = [{
                'url': furl,
                'vcodec': 'none' if ext == 'mp3' else None,
                'ext': ext,
            }]
-        thumbnails = []
+        self.report_extraction(video_id)
-        for thumbnail in content.findall(media_ns_xpath('thumbnail')):
+
-            thumbnail_url = thumbnail.get('url')
+        video_url = info_doc.find('.//{http://search.yahoo.com/mrss/}player').attrib['url']
            if not thumbnail_url:
                continue
            thumbnails.append({
                'url': thumbnail_url,
                'height': int_or_none(thumbnail.get('height')),
                'width': int_or_none(thumbnail.get('width')),
            })
        return {
            'id': video_id,
-            'formats': formats,
+            'url': video_url,
-            'title': title,
+            'title': info_doc.find('.//title').text,
            'description': strip_or_none(xpath_text(item, 'description')),
            'thumbnails': thumbnails,
        }
@@ -13,8 +13,6 @@ from ..utils import (
    ExtractorError,
    float_or_none,
    mimetype2ext,
    str_or_none,
    try_get,
    unescapeHTML,
    unsmuggle_url,
    url_or_none,
@@ -22,11 +20,8 @@ from ..utils import (
 )
 _ID_RE = r'(?:[0-9a-f]{32,34}|[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12,14})'
 class MediasiteIE(InfoExtractor):
-    _VALID_URL = r'(?xi)https?://[^/]+/Mediasite/(?:Play|Showcase/(?:default|livebroadcast)/Presentation)/(?P<id>%s)(?P<query>\?[^#]+|)' % _ID_RE
+    _VALID_URL = r'(?xi)https?://[^/]+/Mediasite/(?:Play|Showcase/(?:default|livebroadcast)/Presentation)/(?P<id>[0-9a-f]{32,34})(?P<query>\?[^#]+|)'
    _TESTS = [
        {
            'url': 'https://hitsmediaweb.h-its.org/mediasite/Play/2db6c271681e4f199af3c60d1f82869b1d',
@@ -98,11 +93,6 @@ class MediasiteIE(InfoExtractor):
            'url': 'https://mediasite.ntnu.no/Mediasite/Showcase/default/Presentation/7d8b913259334b688986e970fae6fcb31d',
            'only_matching': True,
        },
        {
            # dashed id
            'url': 'https://hitsmediaweb.h-its.org/mediasite/Play/2db6c271-681e-4f19-9af3-c60d1f82869b1d',
            'only_matching': True,
        }
    ]
    # look in Mediasite.Core.js (Mediasite.ContentStreamType[*])
@@ -119,7 +109,7 @@ class MediasiteIE(InfoExtractor):
        return [
            unescapeHTML(mobj.group('url'))
            for mobj in re.finditer(
-                r'(?xi)<iframe\b[^>]+\bsrc=(["\'])(?P<url>(?:(?:https?:)?//[^/]+)?/Mediasite/Play/%s(?:\?.*?)?)\1' % _ID_RE,
+                r'(?xi)<iframe\b[^>]+\bsrc=(["\'])(?P<url>(?:(?:https?:)?//[^/]+)?/Mediasite/Play/[0-9a-f]{32,34}(?:\?.*?)?)\1',
                webpage)]
    def _real_extract(self, url):
@@ -231,136 +221,3 @@ class MediasiteIE(InfoExtractor):
            'formats': formats,
            'thumbnails': thumbnails,
        }
 class MediasiteCatalogIE(InfoExtractor):
    _VALID_URL = r'''(?xi)
                        (?P<url>https?://[^/]+/Mediasite)
                        /Catalog/Full/
                        (?P<catalog_id>{0})
                        (?:
                            /(?P<current_folder_id>{0})
                            /(?P<root_dynamic_folder_id>{0})
                        )?
                    '''.format(_ID_RE)
    _TESTS = [{
        'url': 'http://events7.mediasite.com/Mediasite/Catalog/Full/631f9e48530d454381549f955d08c75e21',
        'info_dict': {
            'id': '631f9e48530d454381549f955d08c75e21',
            'title': 'WCET Summit: Adaptive Learning in Higher Ed: Improving Outcomes Dynamically',
        },
        'playlist_count': 6,
        'expected_warnings': ['is not a supported codec'],
    }, {
        # with CurrentFolderId and RootDynamicFolderId
        'url': 'https://medaudio.medicine.iu.edu/Mediasite/Catalog/Full/9518c4a6c5cf4993b21cbd53e828a92521/97a9db45f7ab47428c77cd2ed74bb98f14/9518c4a6c5cf4993b21cbd53e828a92521',
        'info_dict': {
            'id': '9518c4a6c5cf4993b21cbd53e828a92521',
            'title': 'IUSM Family and Friends Sessions',
        },
        'playlist_count': 2,
    }, {
        'url': 'http://uipsyc.mediasite.com/mediasite/Catalog/Full/d5d79287c75243c58c50fef50174ec1b21',
        'only_matching': True,
    }, {
        # no AntiForgeryToken
        'url': 'https://live.libraries.psu.edu/Mediasite/Catalog/Full/8376d4b24dd1457ea3bfe4cf9163feda21',
        'only_matching': True,
    }, {
        'url': 'https://medaudio.medicine.iu.edu/Mediasite/Catalog/Full/9518c4a6c5cf4993b21cbd53e828a92521/97a9db45f7ab47428c77cd2ed74bb98f14/9518c4a6c5cf4993b21cbd53e828a92521',
        'only_matching': True,
    }, {
        # dashed id
        'url': 'http://events7.mediasite.com/Mediasite/Catalog/Full/631f9e48-530d-4543-8154-9f955d08c75e',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
        mediasite_url = mobj.group('url')
        catalog_id = mobj.group('catalog_id')
        current_folder_id = mobj.group('current_folder_id') or catalog_id
        root_dynamic_folder_id = mobj.group('root_dynamic_folder_id')
        webpage = self._download_webpage(url, catalog_id)
        # AntiForgeryToken is optional (e.g. [1])
        # 1. https://live.libraries.psu.edu/Mediasite/Catalog/Full/8376d4b24dd1457ea3bfe4cf9163feda21
        anti_forgery_token = self._search_regex(
            r'AntiForgeryToken\s*:\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
            webpage, 'anti forgery token', default=None, group='value')
        if anti_forgery_token:
            anti_forgery_header = self._search_regex(
                r'AntiForgeryHeaderName\s*:\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
                webpage, 'anti forgery header name',
                default='X-SOFO-AntiForgeryHeader', group='value')
        data = {
            'IsViewPage': True,
            'IsNewFolder': True,
            'AuthTicket': None,
            'CatalogId': catalog_id,
            'CurrentFolderId': current_folder_id,
            'RootDynamicFolderId': root_dynamic_folder_id,
            'ItemsPerPage': 1000,
            'PageIndex': 0,
            'PermissionMask': 'Execute',
            'CatalogSearchType': 'SearchInFolder',
            'SortBy': 'Date',
            'SortDirection': 'Descending',
            'StartDate': None,
            'EndDate': None,
            'StatusFilterList': None,
            'PreviewKey': None,
            'Tags': [],
        }
        headers = {
            'Content-Type': 'application/json; charset=UTF-8',
            'Referer': url,
            'X-Requested-With': 'XMLHttpRequest',
        }
        if anti_forgery_token:
            headers[anti_forgery_header] = anti_forgery_token
        catalog = self._download_json(
            '%s/Catalog/Data/GetPresentationsForFolder' % mediasite_url,
            catalog_id, data=json.dumps(data).encode(), headers=headers)
        entries = []
        for video in catalog['PresentationDetailsList']:
            if not isinstance(video, dict):
                continue
            video_id = str_or_none(video.get('Id'))
            if not video_id:
                continue
            entries.append(self.url_result(
                '%s/Play/%s' % (mediasite_url, video_id),
                ie=MediasiteIE.ie_key(), video_id=video_id))
        title = try_get(
            catalog, lambda x: x['CurrentFolder']['Name'], compat_str)
        return self.playlist_result(entries, catalog_id, title,)
 class MediasiteNamedCatalogIE(InfoExtractor):
    _VALID_URL = r'(?xi)(?P<url>https?://[^/]+/Mediasite)/Catalog/catalogs/(?P<catalog_name>[^/?#&]+)'
    _TESTS = [{
        'url': 'https://msite.misis.ru/Mediasite/Catalog/catalogs/2016-industrial-management-skriabin-o-o',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
        mediasite_url = mobj.group('url')
        catalog_name = mobj.group('catalog_name')
        webpage = self._download_webpage(url, catalog_name)
        catalog_id = self._search_regex(
            r'CatalogId\s*:\s*["\'](%s)' % _ID_RE, webpage, 'catalog id')
        return self.url_result(
            '%s/Catalog/Full/%s' % (mediasite_url, catalog_id),
            ie=MediasiteCatalogIE.ie_key(), video_id=catalog_id)
@@ -1,12 +1,15 @@
 # coding: utf-8
 from __future__ import unicode_literals
 import json
 import re
 from .common import InfoExtractor
 from ..utils import (
-    clean_html,
+    ExtractorError,
    int_or_none,
    sanitized_Request,
    urlencode_postdata,
 )
@@ -14,8 +17,8 @@ class MoeVideoIE(InfoExtractor):
    IE_DESC = 'LetitBit video services: moevideo.net, playreplay.net and videochart.net'
    _VALID_URL = r'''(?x)
        https?://(?P<host>(?:www\.)?
-        (?:(?:moevideo|playreplay|videochart)\.net|thesame\.tv))/
+        (?:(?:moevideo|playreplay|videochart)\.net))/
-        (?:video|framevideo|embed)/(?P<id>[0-9a-z]+\.[0-9A-Za-z]+)'''
+        (?:video|framevideo)/(?P<id>[0-9]+\.[0-9A-Za-z]+)'''
    _API_URL = 'http://api.letitbit.net/'
    _API_KEY = 'tVL0gjqo5'
    _TESTS = [
@@ -54,26 +57,58 @@ class MoeVideoIE(InfoExtractor):
    ]
    def _real_extract(self, url):
-        host, video_id = re.match(self._VALID_URL, url).groups()
+        mobj = re.match(self._VALID_URL, url)
        video_id = mobj.group('id')
        webpage = self._download_webpage(
-            'http://%s/video/%s' % (host, video_id),
+            'http://%s/video/%s' % (mobj.group('host'), video_id),
            video_id, 'Downloading webpage')
        title = self._og_search_title(webpage)
        thumbnail = self._og_search_thumbnail(webpage)
        description = self._og_search_description(webpage)
-        embed_webpage = self._download_webpage(
+        r = [
-            'http://%s/embed/%s' % (host, video_id),
+            self._API_KEY,
-            video_id, 'Downloading embed webpage')
+            [
-        video = self._parse_json(self._search_regex(
+                'preview/flv_link',
-            r'mvplayer\("#player"\s*,\s*({.+})',
+                {
-            embed_webpage, 'mvplayer'), video_id)['video']
+                    'uid': video_id,
                },
            ],
        ]
        r_json = json.dumps(r)
        post = urlencode_postdata({'r': r_json})
        req = sanitized_Request(self._API_URL, post)
        req.add_header('Content-type', 'application/x-www-form-urlencoded')
        response = self._download_json(req, video_id)
        if response['status'] != 'OK':
            raise ExtractorError(
                '%s returned error: %s' % (self.IE_NAME, response['data']),
                expected=True
            )
        item = response['data'][0]
        video_url = item['link']
        duration = int_or_none(item['length'])
        width = int_or_none(item['width'])
        height = int_or_none(item['height'])
        filesize = int_or_none(item['convert_size'])
        formats = [{
            'format_id': 'sd',
            'http_headers': {'Range': 'bytes=0-'},  # Required to download
            'url': video_url,
            'width': width,
            'height': height,
            'filesize': filesize,
        }]
        return {
            'id': video_id,
            'title': title,
-            'thumbnail': video.get('poster') or self._og_search_thumbnail(webpage),
+            'thumbnail': thumbnail,
-            'description': clean_html(self._og_search_description(webpage)),
+            'description': description,
-            'duration': int_or_none(self._og_search_property('video:duration', webpage)),
+            'duration': duration,
-            'url': video['ourUrl'],
+            'formats': formats,
        }
@@ -1,17 +1,12 @@
 # coding: utf-8
 from __future__ import unicode_literals
-import base64
+import re
 import hashlib
 from .common import InfoExtractor
 from ..aes import aes_cbc_decrypt
 from ..utils import (
-    bytes_to_intlist,
+    ExtractorError,
    int_or_none,
    intlist_to_bytes,
    parse_codecs,
    parse_duration,
 )
@@ -19,7 +14,7 @@ class NewstubeIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?newstube\.ru/media/(?P<id>.+)'
    _TEST = {
        'url': 'http://www.newstube.ru/media/telekanal-cnn-peremestil-gorod-slavyansk-v-krym',
-        'md5': '9d10320ad473444352f72f746ccb8b8c',
+        'md5': '801eef0c2a9f4089fa04e4fe3533abdc',
        'info_dict': {
            'id': '728e0ef2-e187-4012-bac0-5a081fdcb1f6',
            'ext': 'mp4',
@@ -30,45 +25,84 @@ class NewstubeIE(InfoExtractor):
    }
    def _real_extract(self, url):
-        video_id = self._match_id(url)
+        mobj = re.match(self._VALID_URL, url)
        video_id = mobj.group('id')
-        page = self._download_webpage(url, video_id)
+        page = self._download_webpage(url, video_id, 'Downloading page')
        title = self._html_search_meta(['og:title', 'twitter:title'], page, fatal=True)
        video_guid = self._html_search_regex(
-            r'<meta\s+property="og:video(?::(?:(?:secure_)?url|iframe))?"\s+content="https?://(?:www\.)?newstube\.ru/embed/(?P<guid>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})',
+            r'<meta property="og:video:url" content="https?://(?:www\.)?newstube\.ru/freshplayer\.swf\?guid=(?P<guid>[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})',
            page, 'video GUID')
-        enc_data = base64.b64decode(self._download_webpage(
+        player = self._download_xml(
-            'https://www.newstube.ru/embed/api/player/getsources2',
+            'http://p.newstube.ru/v2/player.asmx/GetAutoPlayInfo6?state=&url=%s&sessionId=&id=%s&placement=profile&location=n2' % (url, video_guid),
-            video_guid, query={
+            video_guid, 'Downloading player XML')
-                'guid': video_guid,
+
-                'ff': 3,
+        def ns(s):
-            }))
+            return s.replace('/', '/%(ns)s') % {'ns': '{http://app1.newstube.ru/N2SiteWS/player.asmx}'}
-        key = hashlib.pbkdf2_hmac(
+
-            'sha1', video_guid.replace('-', '').encode(), enc_data[:16], 1)[:16]
+        error_message = player.find(ns('./ErrorMessage'))
-        dec_data = aes_cbc_decrypt(
+        if error_message is not None:
-            bytes_to_intlist(enc_data[32:]), bytes_to_intlist(key),
+            raise ExtractorError('%s returned error: %s' % (self.IE_NAME, error_message.text), expected=True)
-            bytes_to_intlist(enc_data[16:32]))
+
-        sources = self._parse_json(intlist_to_bytes(dec_data[:-dec_data[-1]]), video_guid)
+        session_id = player.find(ns('./SessionId')).text
        media_info = player.find(ns('./Medias/MediaInfo'))
        title = media_info.find(ns('./Name')).text
        description = self._og_search_description(page)
        thumbnail = media_info.find(ns('./KeyFrame')).text
        duration = int(media_info.find(ns('./Duration')).text) / 1000.0
        formats = []
-        for source in sources:
+
-            source_url = source.get('Src')
+        for stream_info in media_info.findall(ns('./Streams/StreamInfo')):
-            if not source_url:
+            media_location = stream_info.find(ns('./MediaLocation'))
            if media_location is None:
                continue
-            height = int_or_none(source.get('Height'))
+
-            f = {
+            server = media_location.find(ns('./Server')).text
-                'format_id': 'http' + ('-%dp' % height if height else ''),
+            app = media_location.find(ns('./App')).text
-                'url': source_url,
+            media_id = stream_info.find(ns('./Id')).text
-                'width': int_or_none(source.get('Width')),
+            name = stream_info.find(ns('./Name')).text
            width = int(stream_info.find(ns('./Width')).text)
            height = int(stream_info.find(ns('./Height')).text)
            formats.append({
                'url': 'rtmp://%s/%s' % (server, app),
                'app': app,
                'play_path': '01/%s' % video_guid.upper(),
                'rtmp_conn': ['S:%s' % session_id, 'S:%s' % media_id, 'S:n2'],
                'page_url': url,
                'ext': 'flv',
                'format_id': 'rtmp' + ('-%s' % name if name else ''),
                'width': width,
                'height': height,
-            }
+            })
-            source_type = source.get('Type')
+
-            if source_type:
+        sources_data = self._download_json(
-                f.update(parse_codecs(self._search_regex(
+            'http://www.newstube.ru/player2/getsources?guid=%s' % video_guid,
-                    r'codecs="([^"]+)"', source_type, 'codecs', fatal=False)))
+            video_guid, fatal=False)
-            formats.append(f)
+        if sources_data:
            for source in sources_data.get('Sources', []):
                source_url = source.get('Src')
                if not source_url:
                    continue
                height = int_or_none(source.get('Height'))
                f = {
                    'format_id': 'http' + ('-%dp' % height if height else ''),
                    'url': source_url,
                    'width': int_or_none(source.get('Width')),
                    'height': height,
                }
                source_type = source.get('Type')
                if source_type:
                    mobj = re.search(r'codecs="([^,]+),\s*([^"]+)"', source_type)
                    if mobj:
                        vcodec, acodec = mobj.groups()
                        f.update({
                            'vcodec': vcodec,
                            'acodec': acodec,
                        })
                formats.append(f)
        self._check_formats(formats, video_guid)
        self._sort_formats(formats)
@@ -76,8 +110,8 @@ class NewstubeIE(InfoExtractor):
        return {
            'id': video_guid,
            'title': title,
-            'description': self._html_search_meta(['description', 'og:description'], page),
+            'description': description,
-            'thumbnail': self._html_search_meta(['og:image:secure_url', 'og:image', 'twitter:image'], page),
+            'thumbnail': thumbnail,
-            'duration': parse_duration(self._html_search_meta('duration', page)),
+            'duration': duration,
            'formats': formats,
        }
@@ -181,7 +181,10 @@ class NPOIE(NPOBaseIE):
    def _real_extract(self, url):
        video_id = self._match_id(url)
-        return self._get_info(url, video_id) or self._get_old_info(video_id)
+        try:
            return self._get_info(url, video_id)
        except ExtractorError:
            return self._get_old_info(video_id)
    def _get_info(self, url, video_id):
        token = self._download_json(
@@ -203,7 +206,6 @@ class NPOIE(NPOBaseIE):
        player_token = player['token']
        drm = False
        format_urls = set()
        formats = []
        for profile in ('hls', 'dash-widevine', 'dash-playready', 'smooth'):
@@ -225,8 +227,7 @@ class NPOIE(NPOBaseIE):
            if not stream_url or stream_url in format_urls:
                continue
            format_urls.add(stream_url)
-            if stream.get('protection') is not None or stream.get('keySystemOptions') is not None:
+            if stream.get('protection') is not None:
                drm = True
                continue
            stream_type = stream.get('type')
            stream_ext = determine_ext(stream_url)
@@ -245,11 +246,6 @@ class NPOIE(NPOBaseIE):
                    'url': stream_url,
                })
        if not formats:
            if drm:
                raise ExtractorError('This video is DRM protected.', expected=True)
            return
        self._sort_formats(formats)
        info = {
@@ -1,6 +1,7 @@
 from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..compat import compat_urllib_parse_urlencode
 from ..utils import (
    int_or_none,
    qualities,
@@ -8,16 +9,16 @@ from ..utils import (
 class NprIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?npr\.org/(?:sections/[^/]+/)?\d{4}/\d{2}/\d{2}/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?npr\.org/player/v2/mediaPlayer\.html\?.*\bid=(?P<id>\d+)'
    _TESTS = [{
-        'url': 'https://www.npr.org/sections/allsongs/2015/10/21/449974205/new-music-from-beach-house-chairlift-cmj-discoveries-and-more',
+        'url': 'http://www.npr.org/player/v2/mediaPlayer.html?id=449974205',
        'info_dict': {
            'id': '449974205',
            'title': 'New Music From Beach House, Chairlift, CMJ Discoveries And More'
        },
        'playlist_count': 7,
    }, {
-        'url': 'https://www.npr.org/sections/deceptivecadence/2015/10/09/446928052/music-from-the-shadows-ancient-armenian-hymns-and-piano-jazz',
+        'url': 'http://www.npr.org/player/v2/mediaPlayer.html?action=1&t=1&islist=false&id=446928052&m=446929930&live=1',
        'info_dict': {
            'id': '446928052',
            'title': "Songs We Love: Tigran Hamasyan, 'Your Mercy is Boundless'"
@@ -31,46 +32,30 @@ class NprIE(InfoExtractor):
                'duration': 402,
            },
        }],
    }, {
        # mutlimedia, not media title
        'url': 'https://www.npr.org/2017/06/19/533198237/tigers-jaw-tiny-desk-concert',
        'info_dict': {
            'id': '533198237',
            'title': 'Tigers Jaw: Tiny Desk Concert',
        },
        'playlist': [{
            'md5': '12fa60cb2d3ed932f53609d4aeceabf1',
            'info_dict': {
                'id': '533201718',
                'ext': 'mp4',
                'title': 'Tigers Jaw: Tiny Desk Concert',
                'duration': 402,
            },
        }],
        'expected_warnings': ['Failed to download m3u8 information'],
    }]
    def _real_extract(self, url):
        playlist_id = self._match_id(url)
-        story = self._download_json(
+        config = self._download_json(
-            'http://api.npr.org/query', playlist_id, query={
+            'http://api.npr.org/query?%s' % compat_urllib_parse_urlencode({
                'id': playlist_id,
-                'fields': 'audio,multimedia,title',
+                'fields': 'titles,audio,show',
                'format': 'json',
                'apiKey': 'MDAzMzQ2MjAyMDEyMzk4MTU1MDg3ZmM3MQ010',
-            })['list']['story'][0]
+            }), playlist_id)
        playlist_title = story.get('title', {}).get('$text')
-        KNOWN_FORMATS = ('threegp', 'm3u8', 'smil', 'mp4', 'mp3')
+        story = config['list']['story'][0]
        KNOWN_FORMATS = ('threegp', 'mp4', 'mp3')
        quality = qualities(KNOWN_FORMATS)
        entries = []
-        for media in story.get('audio', []) + story.get('multimedia', []):
+        for audio in story.get('audio', []):
-            media_id = media['id']
+            title = audio.get('title', {}).get('$text')
-
+            duration = int_or_none(audio.get('duration', {}).get('$text'))
            formats = []
-            for format_id, formats_entry in media.get('format', {}).items():
+            for format_id, formats_entry in audio.get('format', {}).items():
                if not formats_entry:
                    continue
                if isinstance(formats_entry, list):
@@ -79,30 +64,19 @@ class NprIE(InfoExtractor):
                if not format_url:
                    continue
                if format_id in KNOWN_FORMATS:
-                    if format_id == 'm3u8':
+                    formats.append({
-                        formats.extend(self._extract_m3u8_formats(
+                        'url': format_url,
-                            format_url, media_id, 'mp4', 'm3u8_native',
+                        'format_id': format_id,
-                            m3u8_id='hls', fatal=False))
+                        'ext': formats_entry.get('type'),
-                    elif format_id == 'smil':
+                        'quality': quality(format_id),
-                        smil_formats = self._extract_smil_formats(
+                    })
                            format_url, media_id, transform_source=lambda s: s.replace(
                                'rtmp://flash.npr.org/ondemand/', 'https://ondemand.npr.org/'))
                        self._check_formats(smil_formats, media_id)
                        formats.extend(smil_formats)
                    else:
                        formats.append({
                            'url': format_url,
                            'format_id': format_id,
                            'quality': quality(format_id),
                        })
            self._sort_formats(formats)
            entries.append({
-                'id': media_id,
+                'id': audio['id'],
-                'title': media.get('title', {}).get('$text') or playlist_title,
+                'title': title,
-                'thumbnail': media.get('altImageUrl', {}).get('$text'),
+                'duration': duration,
                'duration': int_or_none(media.get('duration', {}).get('$text')),
                'formats': formats,
            })
        playlist_title = story.get('title', {}).get('$text')
        return self.playlist_result(entries, playlist_id, playlist_title)
@@ -243,16 +243,18 @@ class PhantomJSwrapper(object):
 class OpenloadIE(InfoExtractor):
    _DOMAINS = r'(?:openload\.(?:co|io|link|pw)|oload\.(?:tv|stream|site|xyz|win|download|cloud|cc|icu|fun|club|info|pw|live|space))'
    _VALID_URL = r'''(?x)
                    https?://
                        (?P<host>
                            (?:www\.)?
-                            %s
+                            (?:
                                openload\.(?:co|io|link|pw)|
                                oload\.(?:tv|stream|site|xyz|win|download|cloud|cc|icu|fun|club|info|pw|live|space)
                            )
                        )/
                        (?:f|embed)/
                        (?P<id>[a-zA-Z0-9-_]+)
-                    ''' % _DOMAINS
+                    '''
    _TESTS = [{
        'url': 'https://openload.co/f/kUEfGclsU9o',
@@ -357,8 +359,8 @@ class OpenloadIE(InfoExtractor):
    @staticmethod
    def _extract_urls(webpage):
        return re.findall(
-            r'<iframe[^>]+src=["\']((?:https?://)?%s/embed/[a-zA-Z0-9-_]+)'
+            r'<iframe[^>]+src=["\']((?:https?://)?(?:openload\.(?:co|io)|oload\.tv)/embed/[a-zA-Z0-9-_]+)',
-            % OpenloadIE._DOMAINS, webpage)
+            webpage)
    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
@@ -176,8 +176,7 @@ class ORFRadioIE(InfoExtractor):
                'description': subtitle,
                'duration': (info['end'] - info['start']) / 1000,
                'timestamp': info['start'] / 1000,
-                'ext': 'mp3',
+                'ext': 'mp3'
                'series': data.get('programTitle')
            }
        entries = [extract_entry_dict(t, data['title'], data['subtitle']) for t in data['streams']]
@@ -36,7 +36,7 @@ class PandaTVIE(InfoExtractor):
            'https://www.panda.tv/api_room_v2?roomid=%s' % video_id, video_id)
        error_code = config.get('errno', 0)
-        if error_code != 0:
+        if error_code is not 0:
            raise ExtractorError(
                '%s returned error %s: %s'
                % (self.IE_NAME, error_code, config['errmsg']),
@@ -1,217 +0,0 @@
 # coding: utf-8
 from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..compat import (
    compat_b64decode,
    compat_str,
 )
 from ..utils import (
    clean_html,
    ExtractorError,
    int_or_none,
    str_or_none,
    try_get,
    url_or_none,
    urlencode_postdata,
    urljoin,
 )
 class PlatziIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
                            platzi\.com/clases|           # es version
                            courses\.platzi\.com/classes  # en version
                        )/[^/]+/(?P<id>\d+)-[^/?\#&]+
                    '''
    _LOGIN_URL = 'https://platzi.com/login/'
    _NETRC_MACHINE = 'platzi'
    _TESTS = [{
        'url': 'https://platzi.com/clases/1311-next-js/12074-creando-nuestra-primera-pagina/',
        'md5': '8f56448241005b561c10f11a595b37e3',
        'info_dict': {
            'id': '12074',
            'ext': 'mp4',
            'title': 'Creando nuestra primera página',
            'description': 'md5:4c866e45034fc76412fbf6e60ae008bc',
            'duration': 420,
        },
        'skip': 'Requires platzi account credentials',
    }, {
        'url': 'https://courses.platzi.com/classes/1367-communication-codestream/13430-background/',
        'info_dict': {
            'id': '13430',
            'ext': 'mp4',
            'title': 'Background',
            'description': 'md5:49c83c09404b15e6e71defaf87f6b305',
            'duration': 360,
        },
        'skip': 'Requires platzi account credentials',
        'params': {
            'skip_download': True,
        },
    }]
    def _real_initialize(self):
        self._login()
    def _login(self):
        username, password = self._get_login_info()
        if username is None:
            return
        login_page = self._download_webpage(
            self._LOGIN_URL, None, 'Downloading login page')
        login_form = self._hidden_inputs(login_page)
        login_form.update({
            'email': username,
            'password': password,
        })
        urlh = self._request_webpage(
            self._LOGIN_URL, None, 'Logging in',
            data=urlencode_postdata(login_form),
            headers={'Referer': self._LOGIN_URL})
        # login succeeded
        if 'platzi.com/login' not in compat_str(urlh.geturl()):
            return
        login_error = self._webpage_read_content(
            urlh, self._LOGIN_URL, None, 'Downloading login error page')
        login = self._parse_json(
            self._search_regex(
                r'login\s*=\s*({.+?})(?:\s*;|\s*</script)', login_error, 'login'),
            None)
        for kind in ('error', 'password', 'nonFields'):
            error = str_or_none(login.get('%sError' % kind))
            if error:
                raise ExtractorError(
                    'Unable to login: %s' % error, expected=True)
        raise ExtractorError('Unable to log in')
    def _real_extract(self, url):
        lecture_id = self._match_id(url)
        webpage = self._download_webpage(url, lecture_id)
        data = self._parse_json(
            self._search_regex(
                r'client_data\s*=\s*({.+?})\s*;', webpage, 'client data'),
            lecture_id)
        material = data['initialState']['material']
        desc = material['description']
        title = desc['title']
        formats = []
        for server_id, server in material['videos'].items():
            if not isinstance(server, dict):
                continue
            for format_id in ('hls', 'dash'):
                format_url = url_or_none(server.get(format_id))
                if not format_url:
                    continue
                if format_id == 'hls':
                    formats.extend(self._extract_m3u8_formats(
                        format_url, lecture_id, 'mp4',
                        entry_protocol='m3u8_native', m3u8_id=format_id,
                        note='Downloading %s m3u8 information' % server_id,
                        fatal=False))
                elif format_id == 'dash':
                    formats.extend(self._extract_mpd_formats(
                        format_url, lecture_id, mpd_id=format_id,
                        note='Downloading %s MPD manifest' % server_id,
                        fatal=False))
        self._sort_formats(formats)
        content = str_or_none(desc.get('content'))
        description = (clean_html(compat_b64decode(content).decode('utf-8'))
                       if content else None)
        duration = int_or_none(material.get('duration'), invscale=60)
        return {
            'id': lecture_id,
            'title': title,
            'description': description,
            'duration': duration,
            'formats': formats,
        }
 class PlatziCourseIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
                            platzi\.com/clases|           # es version
                            courses\.platzi\.com/classes  # en version
                        )/(?P<id>[^/?\#&]+)
                    '''
    _TESTS = [{
        'url': 'https://platzi.com/clases/next-js/',
        'info_dict': {
            'id': '1311',
            'title': 'Curso de Next.js',
        },
        'playlist_count': 22,
    }, {
        'url': 'https://courses.platzi.com/classes/communication-codestream/',
        'info_dict': {
            'id': '1367',
            'title': 'Codestream Course',
        },
        'playlist_count': 14,
    }]
    @classmethod
    def suitable(cls, url):
        return False if PlatziIE.suitable(url) else super(PlatziCourseIE, cls).suitable(url)
    def _real_extract(self, url):
        course_name = self._match_id(url)
        webpage = self._download_webpage(url, course_name)
        props = self._parse_json(
            self._search_regex(r'data\s*=\s*({.+?})\s*;', webpage, 'data'),
            course_name)['initialProps']
        entries = []
        for chapter_num, chapter in enumerate(props['concepts'], 1):
            if not isinstance(chapter, dict):
                continue
            materials = chapter.get('materials')
            if not materials or not isinstance(materials, list):
                continue
            chapter_title = chapter.get('title')
            chapter_id = str_or_none(chapter.get('id'))
            for material in materials:
                if not isinstance(material, dict):
                    continue
                if material.get('material_type') != 'video':
                    continue
                video_url = urljoin(url, material.get('url'))
                if not video_url:
                    continue
                entries.append({
                    '_type': 'url_transparent',
                    'url': video_url,
                    'title': str_or_none(material.get('name')),
                    'id': str_or_none(material.get('id')),
                    'ie_key': PlatziIE.ie_key(),
                    'chapter': chapter_title,
                    'chapter_number': chapter_num,
                    'chapter_id': chapter_id,
                })
        course_id = compat_str(try_get(props, lambda x: x['course']['id']))
        course_title = try_get(props, lambda x: x['course']['name'], compat_str)
        return self.playlist_result(entries, course_id, course_title)
@@ -14,7 +14,6 @@ from ..compat import (
 )
 from .openload import PhantomJSwrapper
 from ..utils import (
    determine_ext,
    ExtractorError,
    int_or_none,
    orderedSet,
@@ -276,10 +275,6 @@ class PornHubIE(PornHubBaseIE):
                    r'/(\d{6}/\d{2})/', video_url, 'upload data', default=None)
                if upload_date:
                    upload_date = upload_date.replace('/', '')
            if determine_ext(video_url) == 'mpd':
                formats.extend(self._extract_mpd_formats(
                    video_url, video_id, mpd_id='dash', fatal=False))
                continue
            tbr = None
            mobj = re.search(r'(?P<height>\d+)[pP]?_(?P<tbr>\d+)[kK]', video_url)
            if mobj:
@@ -0,0 +1,62 @@
 from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..utils import (
    ExtractorError,
    sanitized_Request,
    urlencode_postdata,
 )
 class PrimeShareTVIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?primeshare\.tv/download/(?P<id>[\da-zA-Z]+)'
    _TEST = {
        'url': 'http://primeshare.tv/download/238790B611',
        'md5': 'b92d9bf5461137c36228009f31533fbc',
        'info_dict': {
            'id': '238790B611',
            'ext': 'mp4',
            'title': 'Public Domain - 1960s Commercial - Crest Toothpaste-YKsuFona',
        },
    }
    def _real_extract(self, url):
        video_id = self._match_id(url)
        webpage = self._download_webpage(url, video_id)
        if '>File not exist<' in webpage:
            raise ExtractorError('Video %s does not exist' % video_id, expected=True)
        fields = self._hidden_inputs(webpage)
        headers = {
            'Referer': url,
            'Content-Type': 'application/x-www-form-urlencoded',
        }
        wait_time = int(self._search_regex(
            r'var\s+cWaitTime\s*=\s*(\d+)',
            webpage, 'wait time', default=7)) + 1
        self._sleep(wait_time, video_id)
        req = sanitized_Request(
            url, urlencode_postdata(fields), headers)
        video_page = self._download_webpage(
            req, video_id, 'Downloading video page')
        video_url = self._search_regex(
            r"url\s*:\s*'([^']+\.primeshare\.tv(?::443)?/file/[^']+)'",
            video_page, 'video url')
        title = self._html_search_regex(
            r'<h1>Watch\s*(?:&nbsp;)?\s*\((.+?)(?:\s*\[\.\.\.\])?\)\s*(?:&nbsp;)?\s*<strong>',
            video_page, 'title')
        return {
            'id': video_id,
            'url': video_url,
            'title': title,
            'ext': 'mp4',
        }
@@ -147,7 +147,7 @@ class RadioCanadaIE(InfoExtractor):
 class RadioCanadaAudioVideoIE(InfoExtractor):
-    IE_NAME = 'radiocanada:audiovideo'
+    'radiocanada:audiovideo'
    _VALID_URL = r'https?://ici\.radio-canada\.ca/([^/]+/)*media-(?P<id>[0-9]+)'
    _TESTS = [{
        'url': 'http://ici.radio-canada.ca/audio-video/media-7527184/barack-obama-au-vietnam',
@@ -21,7 +21,7 @@ from ..utils import (
 class RTL2IE(InfoExtractor):
    IE_NAME = 'rtl2'
-    _VALID_URL = r'https?://(?:www\.)?rtl2\.de/sendung/[^/]+/(?:video/(?P<vico_id>\d+)[^/]+/(?P<vivi_id>\d+)-|folge/)(?P<id>[^/?#]+)'
+    _VALID_URL = r'http?://(?:www\.)?rtl2\.de/[^?#]*?/(?P<id>[^?#/]*?)(?:$|/(?:$|[?#]))'
    _TESTS = [{
        'url': 'http://www.rtl2.de/sendung/grip-das-motormagazin/folge/folge-203-0',
        'info_dict': {
@@ -34,11 +34,10 @@ class RTL2IE(InfoExtractor):
            # rtmp download
            'skip_download': True,
        },
        'expected_warnings': ['Unable to download f4m manifest', 'Failed to download m3u8 information'],
    }, {
        'url': 'http://www.rtl2.de/sendung/koeln-50667/video/5512-anna/21040-anna-erwischt-alex/',
        'info_dict': {
-            'id': 'anna-erwischt-alex',
+            'id': '21040-anna-erwischt-alex',
            'ext': 'mp4',
            'title': 'Anna erwischt Alex!',
            'description': 'Anna nimmt ihrem Vater nicht ab, dass er nicht spielt. Und tatsächlich erwischt sie ihn auf frischer Tat.'
@@ -47,29 +46,31 @@ class RTL2IE(InfoExtractor):
            # rtmp download
            'skip_download': True,
        },
        'expected_warnings': ['Unable to download f4m manifest', 'Failed to download m3u8 information'],
    }]
    def _real_extract(self, url):
-        vico_id, vivi_id, display_id = re.match(self._VALID_URL, url).groups()
+        # Some rtl2 urls have no slash at the end, so append it.
-        if not vico_id:
+        if not url.endswith('/'):
-            webpage = self._download_webpage(url, display_id)
+            url += '/'
-            mobj = re.search(
+        video_id = self._match_id(url)
-                r'data-collection="(?P<vico_id>\d+)"[^>]+data-video="(?P<vivi_id>\d+)"',
+        webpage = self._download_webpage(url, video_id)
-                webpage)
+
-            if mobj:
+        mobj = re.search(
-                vico_id = mobj.group('vico_id')
+            r'<div[^>]+data-collection="(?P<vico_id>\d+)"[^>]+data-video="(?P<vivi_id>\d+)"',
-                vivi_id = mobj.group('vivi_id')
+            webpage)
-            else:
+        if mobj:
-                vico_id = self._html_search_regex(
+            vico_id = mobj.group('vico_id')
-                    r'vico_id\s*:\s*([0-9]+)', webpage, 'vico_id')
+            vivi_id = mobj.group('vivi_id')
-                vivi_id = self._html_search_regex(
+        else:
-                    r'vivi_id\s*:\s*([0-9]+)', webpage, 'vivi_id')
+            vico_id = self._html_search_regex(
                r'vico_id\s*:\s*([0-9]+)', webpage, 'vico_id')
            vivi_id = self._html_search_regex(
                r'vivi_id\s*:\s*([0-9]+)', webpage, 'vivi_id')
        info = self._download_json(
-            'https://service.rtl2.de/api-player-vipo/video.php',
+            'http://www.rtl2.de/sites/default/modules/rtl2/mediathek/php/get_video_jw.php',
-            display_id, query={
+            video_id, query={
                'vico_id': vico_id,
                'vivi_id': vivi_id,
            })
@@ -88,7 +89,7 @@ class RTL2IE(InfoExtractor):
                'format_id': 'rtmp',
                'url': rtmp_url,
                'play_path': stream_url,
-                'player_url': 'https://www.rtl2.de/sites/default/modules/rtl2/jwplayer/jwplayer-7.6.0/jwplayer.flash.swf',
+                'player_url': 'http://www.rtl2.de/flashplayer/vipo_player.swf',
                'page_url': url,
                'flash_version': 'LNX 11,2,202,429',
                'rtmp_conn': rtmp_conn,
@@ -98,12 +99,12 @@ class RTL2IE(InfoExtractor):
        m3u8_url = video_info.get('streamurl_hls')
        if m3u8_url:
-            formats.extend(self._extract_akamai_formats(m3u8_url, display_id))
+            formats.extend(self._extract_akamai_formats(m3u8_url, video_id))
        self._sort_formats(formats)
        return {
-            'id': display_id,
+            'id': video_id,
            'title': title,
            'thumbnail': video_info.get('image'),
            'description': video_info.get('beschreibung'),
@@ -0,0 +1,44 @@
 from __future__ import unicode_literals
 from .nuevo import NuevoBaseIE
 class RulePornIE(NuevoBaseIE):
    _VALID_URL = r'https?://(?:www\.)?ruleporn\.com/(?:[^/?#&]+/)*(?P<id>[^/?#&]+)'
    _TEST = {
        'url': 'http://ruleporn.com/brunette-nympho-chick-takes-her-boyfriend-in-every-angle/',
        'md5': '86861ebc624a1097c7c10eaf06d7d505',
        'info_dict': {
            'id': '48212',
            'display_id': 'brunette-nympho-chick-takes-her-boyfriend-in-every-angle',
            'ext': 'mp4',
            'title': 'Brunette Nympho Chick Takes Her Boyfriend In Every Angle',
            'description': 'md5:6d28be231b981fff1981deaaa03a04d5',
            'age_limit': 18,
            'duration': 635.1,
        }
    }
    def _real_extract(self, url):
        display_id = self._match_id(url)
        webpage = self._download_webpage(url, display_id)
        video_id = self._search_regex(
            r'lovehomeporn\.com/embed/(\d+)', webpage, 'video id')
        title = self._search_regex(
            r'<h2[^>]+title=(["\'])(?P<url>.+?)\1',
            webpage, 'title', group='url')
        description = self._html_search_meta('description', webpage)
        info = self._extract_nuevo(
            'http://lovehomeporn.com/media/nuevo/econfig.php?key=%s&rp=true' % video_id,
            video_id)
        info.update({
            'display_id': display_id,
            'title': title,
            'description': description,
            'age_limit': 18
        })
        return info
@@ -59,20 +59,6 @@ class RuutuIE(InfoExtractor):
            'url': 'http://www.ruutu.fi/video/3193728',
            'only_matching': True,
        },
        {
            # audio podcast
            'url': 'https://www.supla.fi/supla/3382410',
            'md5': 'b9d7155fed37b2ebf6021d74c4b8e908',
            'info_dict': {
                'id': '3382410',
                'ext': 'mp3',
                'title': 'Mikä ihmeen poltergeist?',
                'description': 'md5:bbb6963df17dfd0ecd9eb9a61bf14b52',
                'thumbnail': r're:^https?://.*\.jpg$',
                'age_limit': 0,
            },
            'expected_warnings': ['HTTP Error 502: Bad Gateway'],
        }
    ]
    def _real_extract(self, url):
@@ -108,12 +94,6 @@ class RuutuIE(InfoExtractor):
                        continue
                        formats.extend(self._extract_mpd_formats(
                            video_url, video_id, mpd_id='dash', fatal=False))
                    elif ext == 'mp3' or child.tag == 'AudioMediaFile':
                        formats.append({
                            'format_id': 'audio',
                            'url': video_url,
                            'vcodec': 'none',
                        })
                    else:
                        proto = compat_urllib_parse_urlparse(video_url).scheme
                        if not child.tag.startswith('HTTP') and proto != 'rtmp':
@@ -185,7 +185,7 @@ class SVTPlayIE(SVTPlayBaseIE):
    def _extract_by_video_id(self, video_id, webpage=None):
        data = self._download_json(
-            'https://api.svt.se/video/%s' % video_id,
+            'https://api.svt.se/videoplayer-api/video/%s' % video_id,
            video_id, headers=self.geo_verification_headers())
        info_dict = self._extract_video(data, video_id)
        if not info_dict.get('title'):
@@ -16,7 +16,7 @@ from ..utils import (
 class TeamcocoIE(TurnerBaseIE):
-    _VALID_URL = r'https?://(?:\w+\.)?teamcoco\.com/(?P<id>([^/]+/)*[^/?#]+)'
+    _VALID_URL = r'https?://teamcoco\.com/(?P<id>([^/]+/)*[^/?#]+)'
    _TESTS = [
        {
            'url': 'http://teamcoco.com/video/mary-kay-remote',
@@ -79,20 +79,15 @@ class TeamcocoIE(TurnerBaseIE):
        }, {
            'url': 'http://teamcoco.com/israel/conan-hits-the-streets-beaches-of-tel-aviv',
            'only_matching': True,
        }, {
            'url': 'https://conan25.teamcoco.com/video/ice-cube-kevin-hart-conan-share-lyft',
            'only_matching': True,
        }
    ]
    def _graphql_call(self, query_template, object_type, object_id):
        find_object = 'find' + object_type
        return self._download_json(
-            'https://teamcoco.com/graphql', object_id, data=json.dumps({
+            'http://teamcoco.com/graphql/', object_id, data=json.dumps({
                'query': query_template % (find_object, object_id)
-            }).encode(), headers={
+            }))['data'][find_object]
                'Content-Type': 'application/json',
            })['data'][find_object]
    def _real_extract(self, url):
        display_id = self._match_id(url)
@@ -150,12 +145,7 @@ class TeamcocoIE(TurnerBaseIE):
                'accessTokenType': 'jws',
            }))
        else:
-            d = self._download_json(
+            video_sources = self._graphql_call('''{
                'https://teamcoco.com/_truman/d/' + video_id,
                video_id, fatal=False) or {}
            video_sources = d.get('meta') or {}
            if not video_sources:
                video_sources = self._graphql_call('''{
  %s(id: "%s") {
    src
  }
@@ -1,140 +0,0 @@
 # coding: utf-8
 from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..utils import (
    clean_html,
    determine_ext,
    ExtractorError,
    float_or_none,
    get_element_by_class,
    get_element_by_id,
    parse_duration,
    remove_end,
    urlencode_postdata,
    urljoin,
 )
 class TeamTreeHouseIE(InfoExtractor):
    _VALID_URL = r'https?://(?:www\.)?teamtreehouse\.com/library/(?P<id>[^/]+)'
    _TESTS = [{
        # Course
        'url': 'https://teamtreehouse.com/library/introduction-to-user-authentication-in-php',
        'info_dict': {
            'id': 'introduction-to-user-authentication-in-php',
            'title': 'Introduction to User Authentication in PHP',
            'description': 'md5:405d7b4287a159b27ddf30ca72b5b053',
        },
        'playlist_mincount': 24,
    }, {
        # WorkShop
        'url': 'https://teamtreehouse.com/library/deploying-a-react-app',
        'info_dict': {
            'id': 'deploying-a-react-app',
            'title': 'Deploying a React App',
            'description': 'md5:10a82e3ddff18c14ac13581c9b8e5921',
        },
        'playlist_mincount': 4,
    }, {
        # Video
        'url': 'https://teamtreehouse.com/library/application-overview-2',
        'info_dict': {
            'id': 'application-overview-2',
            'ext': 'mp4',
            'title': 'Application Overview',
            'description': 'md5:4b0a234385c27140a4378de5f1e15127',
        },
        'expected_warnings': ['This is just a preview'],
    }]
    _NETRC_MACHINE = 'teamtreehouse'
    def _real_initialize(self):
        email, password = self._get_login_info()
        if email is None:
            return
        signin_page = self._download_webpage(
            'https://teamtreehouse.com/signin',
            None, 'Downloading signin page')
        data = self._form_hidden_inputs('new_user_session', signin_page)
        data.update({
            'user_session[email]': email,
            'user_session[password]': password,
        })
        error_message = get_element_by_class('error-message', self._download_webpage(
            'https://teamtreehouse.com/person_session',
            None, 'Logging in', data=urlencode_postdata(data)))
        if error_message:
            raise ExtractorError(clean_html(error_message), expected=True)
    def _real_extract(self, url):
        display_id = self._match_id(url)
        webpage = self._download_webpage(url, display_id)
        title = self._html_search_meta(['og:title', 'twitter:title'], webpage)
        description = self._html_search_meta(
            ['description', 'og:description', 'twitter:description'], webpage)
        entries = self._parse_html5_media_entries(url, webpage, display_id)
        if entries:
            info = entries[0]
            for subtitles in info.get('subtitles', {}).values():
                for subtitle in subtitles:
                    subtitle['ext'] = determine_ext(subtitle['url'], 'srt')
            is_preview = 'data-preview="true"' in webpage
            if is_preview:
                self.report_warning(
                    'This is just a preview. You need to be signed in with a Basic account to download the entire video.', display_id)
                duration = 30
            else:
                duration = float_or_none(self._search_regex(
                    r'data-duration="(\d+)"', webpage, 'duration'), 1000)
                if not duration:
                    duration = parse_duration(get_element_by_id(
                        'video-duration', webpage))
            info.update({
                'id': display_id,
                'title': title,
                'description': description,
                'duration': duration,
            })
            return info
        else:
            def extract_urls(html, extract_info=None):
                for path in re.findall(r'<a[^>]+href="([^"]+)"', html):
                    page_url = urljoin(url, path)
                    entry = {
                        '_type': 'url_transparent',
                        'id': self._match_id(page_url),
                        'url': page_url,
                        'id_key': self.ie_key(),
                    }
                    if extract_info:
                        entry.update(extract_info)
                    entries.append(entry)
            workshop_videos = self._search_regex(
                r'(?s)<ul[^>]+id="workshop-videos"[^>]*>(.+?)</ul>',
                webpage, 'workshop videos', default=None)
            if workshop_videos:
                extract_urls(workshop_videos)
            else:
                stages_path = self._search_regex(
                    r'(?s)<div[^>]+id="syllabus-stages"[^>]+data-url="([^"]+)"',
                    webpage, 'stages path')
                if stages_path:
                    stages_page = self._download_webpage(
                        urljoin(url, stages_path), display_id, 'Downloading stages page')
                    for chapter_number, (chapter, steps_list) in enumerate(re.findall(r'(?s)<h2[^>]*>\s*(.+?)\s*</h2>.+?<ul[^>]*>(.+?)</ul>', stages_page), 1):
                        extract_urls(steps_list, {
                            'chapter': chapter,
                            'chapter_number': chapter_number,
                        })
                    title = remove_end(title, ' Course')
            return self.playlist_result(
                entries, display_id, title, description)
@@ -1,10 +1,13 @@
 from __future__ import unicode_literals
 import re
 import json
 from .common import InfoExtractor
 from ..utils import (
    int_or_none,
-    parse_duration,
+    ExtractorError,
-    qualities,
+    sanitized_Request,
 )
@@ -13,9 +16,9 @@ class VeohIE(InfoExtractor):
    _TESTS = [{
        'url': 'http://www.veoh.com/watch/v56314296nk7Zdmz3',
-        'md5': '9e7ecc0fd8bbee7a69fe38953aeebd30',
+        'md5': '620e68e6a3cff80086df3348426c9ca3',
        'info_dict': {
-            'id': 'v56314296nk7Zdmz3',
+            'id': '56314296',
            'ext': 'mp4',
            'title': 'Straight Backs Are Stronger',
            'uploader': 'LUMOback',
@@ -53,6 +56,29 @@ class VeohIE(InfoExtractor):
        'only_matching': True,
    }]
    def _extract_formats(self, source):
        formats = []
        link = source.get('aowPermalink')
        if link:
            formats.append({
                'url': link,
                'ext': 'mp4',
                'format_id': 'aow',
            })
        link = source.get('fullPreviewHashLowPath')
        if link:
            formats.append({
                'url': link,
                'format_id': 'low',
            })
        link = source.get('fullPreviewHashHighPath')
        if link:
            formats.append({
                'url': link,
                'format_id': 'high',
            })
        return formats
    def _extract_video(self, source):
        return {
            'id': source.get('videoId'),
@@ -67,37 +93,38 @@ class VeohIE(InfoExtractor):
        }
    def _real_extract(self, url):
-        video_id = self._match_id(url)
+        mobj = re.match(self._VALID_URL, url)
-        video = self._download_json(
+        video_id = mobj.group('id')
            'https://www.veoh.com/watch/getVideo/' + video_id,
            video_id)['video']
        title = video['title']
-        thumbnail_url = None
+        if video_id.startswith('v'):
-        q = qualities(['HQ', 'Regular'])
+            rsp = self._download_xml(
-        formats = []
+                r'http://www.veoh.com/api/findByPermalink?permalink=%s' % video_id, video_id, 'Downloading video XML')
-        for f_id, f_url in video.get('src', {}).items():
+            stat = rsp.get('stat')
-            if not f_url:
+            if stat == 'ok':
-                continue
+                return self._extract_video(rsp.find('./videoList/video'))
-            if f_id == 'poster':
+            elif stat == 'fail':
-                thumbnail_url = f_url
+                raise ExtractorError(
-            else:
+                    '%s said: %s' % (self.IE_NAME, rsp.find('./errorList/error').get('errorMessage')), expected=True)
                formats.append({
                    'format_id': f_id,
                    'quality': q(f_id),
                    'url': f_url,
                })
        self._sort_formats(formats)
-        return {
+        webpage = self._download_webpage(url, video_id)
-            'id': video_id,
+        age_limit = 0
-            'title': title,
+        if 'class="adultwarning-container"' in webpage:
-            'description': video.get('description'),
+            self.report_age_confirmation()
-            'thumbnail': thumbnail_url,
+            age_limit = 18
-            'uploader': video.get('author', {}).get('nickname'),
+            request = sanitized_Request(url)
-            'duration': int_or_none(video.get('lengthBySec')) or parse_duration(video.get('length')),
+            request.add_header('Cookie', 'confirmedAdult=true')
-            'view_count': int_or_none(video.get('views')),
+            webpage = self._download_webpage(request, video_id)
-            'formats': formats,
+
-            'average_rating': int_or_none(video.get('rating')),
+        m_youtube = re.search(r'http://www\.youtube\.com/v/(.*?)(\&|"|\?)', webpage)
-            'comment_count': int_or_none(video.get('numOfComments')),
+        if m_youtube is not None:
-        }
+            youtube_id = m_youtube.group(1)
            self.to_screen('%s: detected Youtube video.' % video_id)
            return self.url_result(youtube_id, 'Youtube')
        info = json.loads(
            self._search_regex(r'videoDetailsJSON = \'({.*?})\';', webpage, 'info').replace('\\\'', '\''))
        video = self._extract_video(info)
        video['age_limit'] = age_limit
        return video
@@ -1,16 +1,19 @@
 from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..compat import (
    compat_urllib_parse_urlencode,
    compat_urlparse,
 )
 from ..utils import (
    float_or_none,
    int_or_none,
    sanitized_Request,
 )
 class ViddlerIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?viddler\.com/(?:v|embed|player)/(?P<id>[a-z0-9]+)(?:.+?\bsecret=(\d+))?'
+    _VALID_URL = r'https?://(?:www\.)?viddler\.com/(?:v|embed|player)/(?P<id>[a-z0-9]+)'
    _TESTS = [{
        'url': 'http://www.viddler.com/v/43903784',
        'md5': '9eee21161d2c7f5b39690c3e325fab2f',
@@ -75,18 +78,23 @@ class ViddlerIE(InfoExtractor):
    }]
    def _real_extract(self, url):
-        video_id, secret = re.match(self._VALID_URL, url).groups()
+        video_id = self._match_id(url)
        query = {
            'video_id': video_id,
            'key': 'v0vhrt7bg2xq1vyxhkct',
        }
        qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
        secret = qs.get('secret', [None])[0]
        if secret:
            query['secret'] = secret
-        data = self._download_json(
+        headers = {'Referer': 'http://static.cdn-ec.viddler.com/js/arpeggio/v2/embed.html'}
-            'http://api.viddler.com/api/v2/viddler.videos.getPlaybackDetails.json',
+        request = sanitized_Request(
-            video_id, headers={'Referer': url}, query=query)['video']
+            'http://api.viddler.com/api/v2/viddler.videos.getPlaybackDetails.json?%s'
            % compat_urllib_parse_urlencode(query), None, headers)
        data = self._download_json(request, video_id)['video']
        formats = []
        for filed in data['files']:
@@ -0,0 +1,60 @@
 # coding: utf-8
 from __future__ import unicode_literals
 import re
 from .common import InfoExtractor
 from ..utils import (
    decode_packed_codes,
    sanitized_Request,
 )
 class VideoMegaIE(InfoExtractor):
    _VALID_URL = r'(?:videomega:|https?://(?:www\.)?videomega\.tv/(?:(?:view|iframe|cdn)\.php)?\?ref=)(?P<id>[A-Za-z0-9]+)'
    _TESTS = [{
        'url': 'http://videomega.tv/cdn.php?ref=AOSQBJYKIDDIKYJBQSOA',
        'md5': 'cc1920a58add3f05c6a93285b84fb3aa',
        'info_dict': {
            'id': 'AOSQBJYKIDDIKYJBQSOA',
            'ext': 'mp4',
            'title': '1254207',
            'thumbnail': r're:^https?://.*\.jpg$',
        }
    }, {
        'url': 'http://videomega.tv/cdn.php?ref=AOSQBJYKIDDIKYJBQSOA&width=1070&height=600',
        'only_matching': True,
    }, {
        'url': 'http://videomega.tv/view.php?ref=090051111052065112106089103052052103089106112065052111051090',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        video_id = self._match_id(url)
        iframe_url = 'http://videomega.tv/cdn.php?ref=%s' % video_id
        req = sanitized_Request(iframe_url)
        req.add_header('Referer', url)
        req.add_header('Cookie', 'noadvtday=0')
        webpage = self._download_webpage(req, video_id)
        title = self._html_search_regex(
            r'<title>(.+?)</title>', webpage, 'title')
        title = re.sub(
            r'(?:^[Vv]ideo[Mm]ega\.tv\s-\s*|\s*-\svideomega\.tv$)', '', title)
        thumbnail = self._search_regex(
            r'<video[^>]+?poster="([^"]+)"', webpage, 'thumbnail', fatal=False)
        real_codes = decode_packed_codes(webpage)
        video_url = self._search_regex(
            r'"src"\s*,\s*"([^"]+)"', real_codes, 'video URL')
        return {
            'id': video_id,
            'title': title,
            'url': video_url,
            'thumbnail': thumbnail,
            'http_headers': {
                'Referer': iframe_url,
            },
        }
@@ -195,32 +195,6 @@ class VimeoBaseInfoExtractor(InfoExtractor):
            'subtitles': subtitles,
        }
    def _extract_original_format(self, url, video_id):
        download_data = self._download_json(
            url, video_id, fatal=False,
            query={'action': 'load_download_config'},
            headers={'X-Requested-With': 'XMLHttpRequest'})
        if download_data:
            source_file = download_data.get('source_file')
            if isinstance(source_file, dict):
                download_url = source_file.get('download_url')
                if download_url and not source_file.get('is_cold') and not source_file.get('is_defrosting'):
                    source_name = source_file.get('public_name', 'Original')
                    if self._is_valid_url(download_url, video_id, '%s video' % source_name):
                        ext = (try_get(
                            source_file, lambda x: x['extension'],
                            compat_str) or determine_ext(
                            download_url, None) or 'mp4').lower()
                        return {
                            'url': download_url,
                            'ext': ext,
                            'width': int_or_none(source_file.get('width')),
                            'height': int_or_none(source_file.get('height')),
                            'filesize': parse_filesize(source_file.get('size')),
                            'format_id': source_name,
                            'preference': 1,
                        }
 class VimeoIE(VimeoBaseInfoExtractor):
    """Information extractor for vimeo.com."""
@@ -685,11 +659,29 @@ class VimeoIE(VimeoBaseInfoExtractor):
            comment_count = None
        formats = []
-
+        download_request = sanitized_Request('https://vimeo.com/%s?action=load_download_config' % video_id, headers={
-        source_format = self._extract_original_format(
+            'X-Requested-With': 'XMLHttpRequest'})
-            'https://vimeo.com/' + video_id, video_id)
+        download_data = self._download_json(download_request, video_id, fatal=False)
-        if source_format:
+        if download_data:
-            formats.append(source_format)
+            source_file = download_data.get('source_file')
            if isinstance(source_file, dict):
                download_url = source_file.get('download_url')
                if download_url and not source_file.get('is_cold') and not source_file.get('is_defrosting'):
                    source_name = source_file.get('public_name', 'Original')
                    if self._is_valid_url(download_url, video_id, '%s video' % source_name):
                        ext = (try_get(
                            source_file, lambda x: x['extension'],
                            compat_str) or determine_ext(
                            download_url, None) or 'mp4').lower()
                        formats.append({
                            'url': download_url,
                            'ext': ext,
                            'width': int_or_none(source_file.get('width')),
                            'height': int_or_none(source_file.get('height')),
                            'filesize': parse_filesize(source_file.get('size')),
                            'format_id': source_name,
                            'preference': 1,
                        })
        info_dict_config = self._parse_config(config, video_id)
        formats.extend(info_dict_config['formats'])
@@ -948,7 +940,7 @@ class VimeoGroupsIE(VimeoAlbumIE):
 class VimeoReviewIE(VimeoBaseInfoExtractor):
    IE_NAME = 'vimeo:review'
    IE_DESC = 'Review pages on vimeo'
-    _VALID_URL = r'(?P<url>https://vimeo\.com/[^/]+/review/(?P<id>[^/]+)/[0-9a-f]{10})'
+    _VALID_URL = r'https://vimeo\.com/[^/]+/review/(?P<id>[^/]+)'
    _TESTS = [{
        'url': 'https://vimeo.com/user21297594/review/75524534/3c257a1b5d',
        'md5': 'c507a72f780cacc12b2248bb4006d253',
@@ -1000,8 +992,7 @@ class VimeoReviewIE(VimeoBaseInfoExtractor):
            data = self._parse_json(self._search_regex(
                r'window\s*=\s*_extend\(window,\s*({.+?})\);', webpage, 'data',
                default=NO_DEFAULT if video_password_verified else '{}'), video_id)
-            config = data.get('vimeo_esi', {}).get('config', {})
+            config_url = data.get('vimeo_esi', {}).get('config', {}).get('configUrl')
            config_url = config.get('configUrl') or try_get(config, lambda x: x['clipData']['configUrl'])
        if config_url is None:
            self._verify_video_password(webpage_url, video_id, webpage)
            config_url = self._get_config_url(
@@ -1009,13 +1000,10 @@ class VimeoReviewIE(VimeoBaseInfoExtractor):
        return config_url
    def _real_extract(self, url):
-        page_url, video_id = re.match(self._VALID_URL, url).groups()
+        video_id = self._match_id(url)
        config_url = self._get_config_url(url, video_id)
        config = self._download_json(config_url, video_id)
        info_dict = self._parse_config(config, video_id)
        source_format = self._extract_original_format(page_url, video_id)
        if source_format:
            info_dict['formats'].append(source_format)
        self._vimeo_sort_formats(info_dict['formats'])
        info_dict['id'] = video_id
        return info_dict
@@ -6,7 +6,10 @@ import re
 import sys
 from .common import InfoExtractor
-from ..compat import compat_urlparse
+from ..compat import (
    compat_str,
    compat_urlparse,
 )
 from ..utils import (
    clean_html,
    ExtractorError,
@@ -100,7 +103,7 @@ class VKIE(VKBaseIE):
            'url': 'http://vk.com/videos-77521?z=video-77521_162222515%2Fclub77521',
            'md5': '7babad3b85ea2e91948005b1b8b0cb84',
            'info_dict': {
-                'id': '-77521_162222515',
+                'id': '162222515',
                'ext': 'mp4',
                'title': 'ProtivoGunz - Хуёвая песня',
                'uploader': 're:(?:Noize MC|Alexander Ilyashenko).*',
@@ -114,7 +117,7 @@ class VKIE(VKBaseIE):
            'url': 'http://vk.com/video205387401_165548505',
            'md5': '6c0aeb2e90396ba97035b9cbde548700',
            'info_dict': {
-                'id': '205387401_165548505',
+                'id': '165548505',
                'ext': 'mp4',
                'title': 'No name',
                'uploader': 'Tom Cruise',
@@ -129,7 +132,7 @@ class VKIE(VKBaseIE):
            'url': 'http://vk.com/video_ext.php?oid=32194266&id=162925554&hash=7d8c2e0d5e05aeaa&hd=1',
            'md5': 'c7ce8f1f87bec05b3de07fdeafe21a0a',
            'info_dict': {
-                'id': '32194266_162925554',
+                'id': '162925554',
                'ext': 'mp4',
                'uploader': 'Vladimir Gavrin',
                'title': 'Lin Dan',
@@ -146,7 +149,7 @@ class VKIE(VKBaseIE):
            'md5': 'a590bcaf3d543576c9bd162812387666',
            'note': 'Only available for registered users',
            'info_dict': {
-                'id': '-8871596_164049491',
+                'id': '164049491',
                'ext': 'mp4',
                'uploader': 'Триллеры',
                'title': '► Бойцовский клуб / Fight Club 1999 [HD 720]',
@@ -160,7 +163,7 @@ class VKIE(VKBaseIE):
            'url': 'http://vk.com/hd_kino_mania?z=video-43215063_168067957%2F15c66b9b533119788d',
            'md5': '4d7a5ef8cf114dfa09577e57b2993202',
            'info_dict': {
-                'id': '-43215063_168067957',
+                'id': '168067957',
                'ext': 'mp4',
                'uploader': 'Киномания - лучшее из мира кино',
                'title': ' ',
@@ -174,7 +177,7 @@ class VKIE(VKBaseIE):
            'md5': '0c45586baa71b7cb1d0784ee3f4e00a6',
            'note': 'ivi.ru embed',
            'info_dict': {
-                'id': '-43215063_169084319',
+                'id': '60690',
                'ext': 'mp4',
                'title': 'Книга Илая',
                'duration': 6771,
@@ -188,7 +191,7 @@ class VKIE(VKBaseIE):
            'url': 'https://vk.com/video30481095_171201961?list=8764ae2d21f14088d4',
            'md5': '091287af5402239a1051c37ec7b92913',
            'info_dict': {
-                'id': '30481095_171201961',
+                'id': '171201961',
                'ext': 'mp4',
                'title': 'ТюменцевВВ_09.07.2015',
                'uploader': 'Anton Ivanov',
@@ -203,10 +206,10 @@ class VKIE(VKBaseIE):
            'url': 'https://vk.com/video276849682_170681728',
            'info_dict': {
                'id': 'V3K4mi0SYkc',
-                'ext': 'mp4',
+                'ext': 'webm',
                'title': "DSWD Awards 'Children's Joy Foundation, Inc.' Certificate of Registration and License to Operate",
                'description': 'md5:bf9c26cfa4acdfb146362682edd3827a',
-                'duration': 178,
+                'duration': 179,
                'upload_date': '20130116',
                'uploader': "Children's Joy Foundation Inc.",
                'uploader_id': 'thecjf',
@@ -236,7 +239,7 @@ class VKIE(VKBaseIE):
            'url': 'http://vk.com/video-110305615_171782105',
            'md5': 'e13fcda136f99764872e739d13fac1d1',
            'info_dict': {
-                'id': '-110305615_171782105',
+                'id': '171782105',
                'ext': 'mp4',
                'title': 'S-Dance, репетиции к The way show',
                'uploader': 'THE WAY SHOW | 17 апреля',
@@ -251,17 +254,14 @@ class VKIE(VKBaseIE):
        {
            # finished live stream, postlive_mp4
            'url': 'https://vk.com/videos-387766?z=video-387766_456242764%2Fpl_-387766_-2',
            'md5': '90d22d051fccbbe9becfccc615be6791',
            'info_dict': {
-                'id': '-387766_456242764',
+                'id': '456242764',
                'ext': 'mp4',
-                'title': 'ИгроМир 2016 День 1 — Игромания Утром',
+                'title': 'ИгроМир 2016 — день 1',
                'uploader': 'Игромания',
                'duration': 5239,
-                # TODO: use act=show to extract view_count
+                'view_count': int,
                # 'view_count': int,
                'upload_date': '20160929',
                'uploader_id': '-387766',
                'timestamp': 1475137527,
            },
        },
        {
@@ -465,7 +465,7 @@ class VKIE(VKBaseIE):
        self._sort_formats(formats)
        return {
-            'id': video_id,
+            'id': compat_str(data.get('vid') or video_id),
            'formats': formats,
            'title': title,
            'thumbnail': data.get('jpg'),
@@ -150,10 +150,9 @@ class VRVIE(VRVBaseIE):
    def _real_extract(self, url):
        video_id = self._match_id(url)
-        object_data = self._call_cms(self._get_cms_resource(
+        episode_path = self._get_cms_resource(
-            'cms:/objects/' + video_id, video_id), video_id, 'object')['items'][0]
+            'cms:/episodes/' + video_id, video_id)
-        resource_path = object_data['__links__']['resource']['href']
+        video_data = self._call_cms(episode_path, video_id, 'video')
        video_data = self._call_cms(resource_path, video_id, 'video')
        title = video_data['title']
        streams_path = video_data['__links__'].get('streams', {}).get('href')
@@ -19,7 +19,7 @@ from ..utils import (
 class WeiboIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?weibo\.com/[0-9]+/(?P<id>[a-zA-Z0-9]+)'
+    _VALID_URL = r'https?://weibo\.com/[0-9]+/(?P<id>[a-zA-Z0-9]+)'
    _TEST = {
        'url': 'https://weibo.com/6275294458/Fp6RGfbff?type=comment',
        'info_dict': {
@@ -20,7 +20,7 @@ from ..utils import (
 class XHamsterIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
-                        (?:.+?\.)?xhamster\.(?:com|one)/
+                        (?:.+?\.)?xhamster\.com/
                        (?:
                            movies/(?P<id>\d+)/(?P<display_id>[^/]*)\.html|
                            videos/(?P<display_id_2>[^/]*)-(?P<id_2>\d+)
@@ -91,9 +91,6 @@ class XHamsterIE(InfoExtractor):
        # new URL schema
        'url': 'https://pt.xhamster.com/videos/euro-pedal-pumping-7937821',
        'only_matching': True,
    }, {
        'url': 'https://xhamster.one/videos/femaleagent-shy-beauty-takes-the-bait-1509445',
        'only_matching': True,
    }]
    def _real_extract(self, url):
@@ -57,17 +57,10 @@ class XVideosIE(InfoExtractor):
            webpage, 'title', default=None,
            group='title') or self._og_search_title(webpage)
-        thumbnails = []
+        thumbnail = self._search_regex(
-        for preference, thumbnail in enumerate(('', '169')):
+            (r'setThumbUrl\(\s*(["\'])(?P<thumbnail>(?:(?!\1).)+)\1',
-            thumbnail_url = self._search_regex(
+             r'url_bigthumb=(?P<thumbnail>.+?)&amp'),
-                r'setThumbUrl%s\(\s*(["\'])(?P<thumbnail>(?:(?!\1).)+)\1' % thumbnail,
+            webpage, 'thumbnail', fatal=False, group='thumbnail')
                webpage, 'thumbnail', default=None, group='thumbnail')
            if thumbnail_url:
                thumbnails.append({
                    'url': thumbnail_url,
                    'preference': preference,
                })
        duration = int_or_none(self._og_search_property(
            'duration', webpage, default=None)) or parse_duration(
            self._search_regex(
@@ -105,6 +98,6 @@ class XVideosIE(InfoExtractor):
            'formats': formats,
            'title': title,
            'duration': duration,
-            'thumbnails': thumbnails,
+            'thumbnail': thumbnail,
            'age_limit': 18,
        }
@@ -1,90 +0,0 @@
 # coding: utf-8
 from __future__ import unicode_literals
 from .common import InfoExtractor
 from ..utils import (
    int_or_none,
    url_or_none,
 )
 class YandexVideoIE(InfoExtractor):
    _VALID_URL = r'''(?x)
                    https?://
                        (?:
                            yandex\.ru(?:/portal/(?:video|efir))?/?\?.*?stream_id=|
                            frontend\.vh\.yandex\.ru/player/
                        )
                        (?P<id>[\da-f]+)
                    '''
    _TESTS = [{
        'url': 'https://yandex.ru/portal/video?stream_id=4dbb262b4fe5cf15a215de4f34eee34d',
        'md5': '33955d7ae052f15853dc41f35f17581c',
        'info_dict': {
            'id': '4dbb262b4fe5cf15a215de4f34eee34d',
            'ext': 'mp4',
            'title': 'В Нью-Йорке баржи и теплоход оторвались от причала и расплылись по Гудзону',
            'description': '',
            'thumbnail': r're:^https?://.*\.jpg$',
            'timestamp': 0,
            'duration': 30,
            'age_limit': 18,
        },
    }, {
        'url': 'https://yandex.ru/portal/efir?stream_id=4dbb36ec4e0526d58f9f2dc8f0ecf374&from=morda',
        'only_matching': True,
    }, {
        'url': 'https://yandex.ru/?stream_id=4dbb262b4fe5cf15a215de4f34eee34d',
        'only_matching': True,
    }, {
        'url': 'https://frontend.vh.yandex.ru/player/4dbb262b4fe5cf15a215de4f34eee34d?from=morda',
        'only_matching': True,
    }, {
        # vod-episode, series episode
        'url': 'https://yandex.ru/portal/video?stream_id=45b11db6e4b68797919c93751a938cee',
        'only_matching': True,
    }, {
        # episode, sports
        'url': 'https://yandex.ru/?stream_channel=1538487871&stream_id=4132a07f71fb0396be93d74b3477131d',
        'only_matching': True,
    }]
    def _real_extract(self, url):
        video_id = self._match_id(url)
        content = self._download_json(
            'https://frontend.vh.yandex.ru/v22/player/%s.json' % video_id,
            video_id, query={
                'stream_options': 'hires',
                'disable_trackings': 1,
            })['content']
        m3u8_url = url_or_none(content.get('content_url')) or url_or_none(
            content['streams'][0]['url'])
        title = content.get('title') or content.get('computed_title')
        formats = self._extract_m3u8_formats(
            m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
            m3u8_id='hls')
        self._sort_formats(formats)
        description = content.get('description')
        thumbnail = content.get('thumbnail')
        timestamp = (int_or_none(content.get('release_date')) or
                     int_or_none(content.get('release_date_ut')) or
                     int_or_none(content.get('start_time')))
        duration = int_or_none(content.get('duration'))
        series = content.get('program_title')
        age_limit = int_or_none(content.get('restriction_age'))
        return {
            'id': video_id,
            'title': title,
            'description': description,
            'thumbnail': thumbnail,
            'timestamp': timestamp,
            'duration': duration,
            'series': series,
            'age_limit': age_limit,
            'formats': formats,
        }
@@ -351,8 +351,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                            (?:www\.)?hooktube\.com/|
                            (?:www\.)?yourepeat\.com/|
                            tube\.majestyc\.net/|
-                            (?:(?:www|dev)\.)?invidio\.us/|
+                            (?:www\.)?invidio\.us/|
                            (?:www\.)?invidiou\.sh/|
                            (?:www\.)?invidious\.snopyta\.org/|
                            (?:www\.)?invidious\.kabi\.tk/|
                            (?:www\.)?vid\.wxzm\.sx/|
@@ -484,7 +483,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
        # RTMP (unnamed)
        '_rtmp': {'protocol': 'rtmp'},
    }
-    _SUBTITLE_FORMATS = ('srv1', 'srv2', 'srv3', 'ttml', 'vtt')
+    _SUBTITLE_FORMATS = ('ttml', 'vtt')
    _GEO_BYPASS = False
@@ -31,7 +31,7 @@ def rsa_verify(message, signature, key):
 def update_self(to_screen, verbose, opener):
    """Update the program file with the latest version from the repository"""
-    UPDATE_URL = 'https://yt-dl.org/update/'
+    UPDATE_URL = 'https://ytdl-org.github.io/youtube-dl/update/'
    VERSION_URL = UPDATE_URL + 'LATEST_VERSION'
    JSON_URL = UPDATE_URL + 'versions.json'
    UPDATES_RSA_KEY = (0x9d60ee4d8f805312fdb15a62f87b95bd66177b91df176765d13514a0f1754bcd2057295c5b6f1d35daa6742c3ffc9a82d3e118861c207995a8031e151d863c9927e304576bc80692bc8e094896fcf11b66f3e29e04e3a71e9a11558558acea1840aec37fc396fb6b65dc81a1c4144e03bd1c011de62e3f1357b327d08426fe93, 65537)
@@ -1798,14 +1798,6 @@ def parse_resolution(s):
    return {}
 def parse_bitrate(s):
    if not isinstance(s, compat_str):
        return
    mobj = re.search(r'\b(\d+)\s*kbps', s)
    if mobj:
        return int(mobj.group(1))
 def month_by_name(name, lang='en'):
    """ Return the number of a month by (locale-independently) English name """
@@ -1922,7 +1914,7 @@ def int_or_none(v, scale=1, default=None, get_attr=None, invscale=1):
        return default
    try:
        return int(v) * invscale // scale
-    except (ValueError, TypeError):
+    except ValueError:
        return default
@@ -1943,7 +1935,7 @@ def float_or_none(v, scale=1, invscale=1, default=None):
        return default
    try:
        return float(v) * invscale / scale
-    except (ValueError, TypeError):
+    except ValueError:
        return default
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals
-__version__ = '2019.04.07'
+__version__ = '2019.03.09'
`@@ -1,3 +1,3 @@`
	`from __future__ import unicode_literals`	`from __future__ import unicode_literals`

	`__version__ = '2019.04.07'`	`__version__ = '2019.03.09'`