mollusk/youtube-dl
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7,283 Commits 9 Branches 1,077 Tags
b9b42f2ea0c564f3e75a8f052bfe0dfe21cf320f
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jaime Marquínez Ferrándiz b9b42f2ea0 [npo] Convert to new subtitles system 2015-02-18 17:57:10 +01:00
Jaime Marquínez Ferrándiz 311c393838 [lynda] Convert to new subtitles system 2015-02-18 17:46:33 +01:00
Jaime Marquínez Ferrándiz 18c1c42405 [drtv] Convert to new subtitles system 2015-02-18 17:20:22 +01:00
Jaime Marquínez Ferrándiz 37dd5d4629 [mit] Don't set the subtitles field 
YouTube already provides them in more formats
 2015-02-18 17:19:56 +01:00
Jaime Marquínez Ferrándiz edab9dbf4d [YoutubeDL] use the 'render_table' function for listing the subtitles 2015-02-17 22:59:19 +01:00
Jaime Marquínez Ferrándiz 9868ea4936 [extractor/common] Simplify subtitles handling methods 
Initially I was going to use a single method for handling both subtitles and automatic captions, that's why I used the 'list_subtitles' and the 'subtitles' variables.
 2015-02-17 22:16:29 +01:00
Jaime Marquínez Ferrándiz 85920dd01d [bliptv] Convert to new subtitles system 2015-02-17 21:56:25 +01:00
Jaime Marquínez Ferrándiz bd7fe0cf66 [walla] Convert to new subtitles system 2015-02-17 21:23:09 +01:00
Jaime Marquínez Ferrándiz 48246541da [ceskatelevize] Convert to new subtitles system 2015-02-17 21:17:47 +01:00
Jaime Marquínez Ferrándiz 360e1ca5cc [youtube] Convert to new subtitles system 
The automatic captions are stored in the 'automactic_captions' field, which is used if no normal subtitles are found for an specific language.
 2015-02-16 22:47:39 +01:00
Jaime Marquínez Ferrándiz a1f2a06b34 [dailymotion] Convert to new subtitles system 2015-02-16 21:51:08 +01:00
Jaime Marquínez Ferrándiz c84dd8a90d [YoutubeDL] store the subtitles to download in the 'requested_subtitles' field 
We need to keep the orginal subtitles information, so that the '--load-info' option can be used to list or select the subtitles again.
We'll also be able to have a separate field for storing the automatic captions info.
 2015-02-16 21:51:08 +01:00
Jaime Marquínez Ferrándiz 65469a7f8b [vimeo] Convert to new subtitles system 
Removed some tests, the behaviour should be checked in a test for the YoutubeDL class
 2015-02-16 21:51:07 +01:00
Jaime Marquínez Ferrándiz 6b597516c1 [atresplayer] Convert to new subtitles system 2015-02-16 21:51:07 +01:00
Jaime Marquínez Ferrándiz b5857f62e2 [crunchyroll] Convert to new subtitles system 2015-02-16 21:51:07 +01:00
Jaime Marquínez Ferrándiz a504ced097 Improve subtitles support 
For each language the extractor builds a list with the available formats sorted (like for video formats), then YoutubeDL selects one of them using the '--sub-format' option which now allows giving the format preferences (for example 'ass/srt/best').
For each format the 'url' field can be set so that we only download the contents if needed, or if the contents needs to be processed (like in crunchyroll) the 'data' field can be used.

The reasons for this change are:
* We weren't checking that the format given with '--sub-format' was available, checking it in each extractor would be repetitive.
* It allows to easily support giving a format preference.
* The subtitles were automatically downloaded in the extractor, but I think that if you use for example the '--dump-json' option you want to finish as fast as possible.

Currently only the ted extractor has been updated, but the old system still works.
 2015-02-16 21:51:03 +01:00
Jaime Marquínez Ferrándiz 8fb474fb17 [test/subtitles] Fix some tests 
The checksym for the CeskaTelevize subtitles has changed again, so we just test that it has a reasonable length.
 2015-02-15 15:01:07 +01:00
Jaime Marquínez Ferrándiz 6ca7732d5e [extractor/common] Fix link to external documentation 2015-02-14 22:20:24 +01:00
Jaime Marquínez Ferrándiz b0ab0fac49 Remove unused imports 2015-02-14 22:19:58 +01:00
Jaime Marquínez Ferrándiz cffcbc02de [postprocessor/ffmpeg] Don't let ffmpeg read from stdin (fixes #4945) 
If you run 'while read aurl ; do youtube-dl --extract-audio "${aurl}"; done < path_to_batch_file'  (batch_file contains one url per line) each call to youtube-dl consumed some characters and 'read' would assing to 'aurl' a non valid url, something like 'tube.com/watch?v=<id>'.
 2015-02-13 22:25:34 +01:00
Jaime Marquínez Ferrándiz 36e7a4ca2e [test/subtitles] Update checksums 2015-02-13 14:43:50 +01:00
Jaime Marquínez Ferrándiz ae6423d704 [bambuser] Fix 'uploader_id' extraction (fixes #4944) 2015-02-13 11:36:33 +01:00
Jaime Marquínez Ferrándiz 2d30521ab9 [youtube] Extract average rating (closes #2362) 2015-02-11 18:39:31 +01:00
Jaime Marquínez Ferrándiz 050fa43561 flake8: Ignore some error added in pep8 1.6 
* E402: we exectute code between imports, like modifying 'sys.path' in the tests
* E731: we assign to lambdas in a lot of places, we may want to consider defining functions in a single line instead (what pep8 recommends)
 2015-02-11 18:15:15 +01:00
Jaime Marquínez Ferrándiz f36f92f4da [aes] style: Put __all__ variable at the end of the file 2015-02-11 18:15:15 +01:00
Jaime Marquínez Ferrándiz d304209a85 [test/parameters.json] Set 'fixup' to 'never' 
The fixed audio files for Youtube have a size lower that the minimum required.
 2015-02-11 17:25:04 +01:00
Jaime Marquínez Ferrándiz ba61796458 [youtube] Don't override format info from the dash manifest (fixes #4911) 2015-02-09 15:04:22 +01:00
Jaime Marquínez Ferrándiz 0161353d7d [test/test_YoutubeDL] Remove debug print call 2015-02-06 23:58:01 +01:00
Jaime Marquínez Ferrándiz 2b4ecde2c8 [test/YoutubeDL] Add a simple test for postprocesors 
Just checks that the 'keepvideo' option works as intended.
 2015-02-06 23:54:25 +01:00
Jaime Marquínez Ferrándiz b3a286d69d [YoutubeDL] _calc_cookies: add get_header method to _PseudoRequest (#4861) 2015-02-06 22:23:06 +01:00
Jaime Marquínez Ferrándiz 467d3c9a0c [ffmpeg] --extrac-audio: Use the same options for avconv and ffmpeg 
They have been available in ffmpeg since version 0.9, and we require 1.0 or higher.
 2015-02-06 22:05:11 +01:00
Jaime Marquínez Ferrándiz a57b562cff [nfl] Add support for articles pages (fixes #4848) 2015-02-02 23:17:00 +01:00
Jaime Marquínez Ferrándiz e41b1f7385 Fix flake8 errors 2015-01-31 10:51:39 +01:00
Jaime Marquínez Ferrándiz cd596028d6 [rtve] Recognize mobile urls (fixes #4823) 2015-01-30 23:46:55 +01:00
Jaime Marquínez Ferrándiz cc57bd33a8 [rtve] Fix extraction 
Skip live stream test, we can't use the f4m manifest yet
 2015-01-30 23:46:55 +01:00
Jaime Marquínez Ferrándiz a7a14d9586 [YoutubeDL] set the 'thumbnails' field if the info_dict has the 'thumbnails' field 
Since the '--write-thumbnail' uses the 'thumbnails' field and we didn't updated the info_dict, it wouldn't detect the thumbnail. (fixes #4812)
 2015-01-29 20:15:38 +01:00
Jaime Marquínez Ferrándiz 4f264c02c7 [utils] YoutubeDLHTTPSHandler.https_open: pass all required arguments to do_open 
With this change the '--no-check-certificate' works again (#4807).
 2015-01-29 15:01:27 +01:00
Jaime Marquínez Ferrándiz e72c7e4123 [YoutubeDL] Always set the '_filename' field in the info_dict (reported in #4053) 
It's also useful when you use the '--write-info-json' option.
 2015-01-26 12:01:43 +01:00
Jaime Marquínez Ferrándiz 6d2749aac4 [drtv] Prefer the version without spoken subtitles (fixes #4779) 
For example for http://www.dr.dk/tv/se/moderne-klassikere/moderne-klassikere-one-republic-apologize#!/, there's a version where everytime someone speaks in English a computer voice translates it.
 2015-01-25 18:56:04 +01:00
Jaime Marquínez Ferrándiz 1070711d60 [YoutubeDL._calc_cookies] Restore the 'is_unverifiable' 
I should have check everything was copied before commiting 4b405cfc6e.
 2015-01-24 20:12:47 +01:00
Jaime Marquínez Ferrándiz 4b405cfc6e [YoutubeDL._calc_cookies] Restore the 'has_header' method 
I didn't copied it from downloader/external
 2015-01-24 20:08:24 +01:00
Jaime Marquínez Ferrándiz e5660ee6ae [YoutubeDL] Fill the info dict 'http_headers' field with all the headers available 
Useful for external tools using the json output.

The methods '_calc_headers' and '_calc_cookies' have been copied from the downloader/external, now they just use "info_dict['http_headers']".
 2015-01-24 18:56:04 +01:00
Jaime Marquínez Ferrándiz 587a9c2749 [downloader/external] Use the 'http_headers' field 2015-01-24 18:25:09 +01:00
Jaime Marquínez Ferrándiz e1554a407d [extractors] Use http_headers for setting the User-Agent and the Referer 2015-01-24 18:23:53 +01:00
Jaime Marquínez Ferrándiz 3fcfb8e9fa [utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent 
Setting the 'User-Agent' header is enough
 2015-01-24 18:07:21 +01:00
Jaime Marquínez Ferrándiz fdaaaaa878 README: Recommend using flake8 instead of pyflake and pep8 separately 2015-01-23 21:10:10 +01:00
Jaime Marquínez Ferrándiz 12d1fb5aa9 [twitch] PEP8 2015-01-23 21:05:07 +01:00
Jaime Marquínez Ferrándiz 48f00d15b1 [auengine] Remove extractor 
The test is probably infringing copyright and nobody has provided a new test (see #4643).
 2015-01-23 21:03:00 +01:00
Jaime Marquínez Ferrándiz 0920e5830f [atresplayer] Don't include f4m formats if they are protected by DRM (fixes #4705) 2015-01-23 16:39:23 +01:00
Jaime Marquínez Ferrándiz bf7fa94ec7 [downloader/f4m] build_fragments_list: Support videos with more than 1 segment 2015-01-23 16:31:52 +01:00
Jaime Marquínez Ferrándiz 47e0e1e0e2 [nbc] Fix pep8 issue 2015-01-21 10:36:15 +01:00
Jaime Marquínez Ferrándiz efcddaebe9 [cnn] Use edition.cnn.com for getting the information (fixes #4757) 
Some videos (like http://edition.cnn.com/videos/us/2015/01/20/orig-yellowstone-oil-spill.cnn) will fail if we use cnn.com.
 2015-01-21 10:31:57 +01:00
Jaime Marquínez Ferrándiz 5fe5112589 [CNNArticle] Update test 2015-01-21 10:27:18 +01:00
Jaime Marquínez Ferrándiz 2875cf01bb FFmpegEmbedSubtitlePP: simplify command 2015-01-16 13:37:37 +01:00
Jaime Marquínez Ferrándiz e205db3bcd FFmpegEmbedSubtitlePP: don't fail if the video doesn't have an audio stream (fixes #4718) 
Instead of specifying which streams ffmpeg must copy, we tell it to copy all.
 2015-01-16 13:29:01 +01:00
Jaime Marquínez Ferrándiz f3ff1a3696 YoutubeDL: Make the decision about removing the original file after each postprocessor is run (fixes #2261) 
If one of the processors said the file should be kept, it wouldn't pay
attention to the response from the following processors. This was wrong if the
'keep_video' option was False, if the first extractor modifies the original file
and then we extract its audio we don't want to keep the original video file.
 2015-01-11 11:35:18 +01:00
Jaime Marquínez Ferrándiz 9932a65370 [vk] Remove debug assert statement (fixes #4672, fixes #4514) 2015-01-09 20:13:53 +01:00
Jaime Marquínez Ferrándiz c5d666d374 Fix build with python 2.6 
* Packages cannot be executed
* '.format' needs the index of the argument

(Reported in https://github.com/Homebrew/homebrew/issues/35616)
 2015-01-07 16:09:43 +01:00
Jaime Marquínez Ferrándiz 5d8993b06a [extractor/__init__] Remove unused import 2015-01-07 11:59:15 +01:00
Jaime Marquínez Ferrándiz c758bf9fd7 [nrktv] Remove 'proxy' parameter from tests 2015-01-07 11:56:22 +01:00
Jaime Marquínez Ferrándiz 76b3c61012 [youtube] Add formats 308 and 315 (closes #4650) 2015-01-06 11:59:41 +01:00
Jaime Marquínez Ferrándiz 8f9529cd05 [motorsport] Fix extraction and make trailing '/' optional 
They directly embed a youtube video now.
 2015-01-05 19:19:01 +01:00
Jaime Marquínez Ferrándiz a285b6377b [normalboots] Skip download in test, it uses rtmp 2015-01-05 13:59:49 +01:00
Jaime Marquínez Ferrándiz cd791a5ea0 [ted] Add support for embed-ssl.ted.com embedded videos 2015-01-05 13:11:13 +01:00
Jaime Marquínez Ferrándiz 87830900a9 [generic] Update some tests 2015-01-05 13:07:24 +01:00
Jaime Marquínez Ferrándiz 75311a7e16 .travis.yml: Remove my email from the list 2015-01-05 12:29:32 +01:00
Jaime Marquínez Ferrándiz 628bc4d1e7 [khanacademy] Update test 2015-01-05 12:28:35 +01:00
Jaime Marquínez Ferrándiz a4c3f48639 [vimple] Replace tests 
The first one seems to be no longer available and the second was an episode from a tv show.
 2015-01-05 11:54:14 +01:00
Jaime Marquínez Ferrándiz 2f985f4bb4 [youtube:toplist] Remove extractor 
They use now normal playlists (their id is PL*).
 2015-01-05 00:18:43 +01:00
Jaime Marquínez Ferrándiz 1d2d0e3ff2 utils: Remove blank line at the end of file 2015-01-04 14:07:06 +01:00
Jaime Marquínez Ferrándiz 9fda6ee39f [tf1] Remove unused import 2015-01-04 14:06:23 +01:00
Jaime Marquínez Ferrándiz bc3e582fe4 Don't use '-shortest' option for merging formats (closes #4220, closes #4580) 
With avconv and older versions of ffmpeg the video is partially copied.
The duration difference between the audio and the video seem to be really small, so it's probably not noticeable.
 2015-01-04 14:02:17 +01:00
Jaime Marquínez Ferrándiz 63948fc62c [downloader/hls] Respect the 'prefer_ffmpeg' option 2015-01-04 13:41:49 +01:00
Jaime Marquínez Ferrándiz d26b1317ed [downloader/mplayer] Use check_executable 2015-01-03 00:33:36 +01:00
Jaime Marquínez Ferrándiz a221f22969 [crunchyroll] Fix format extraction 
Reported in https://github.com/rg3/youtube-dl/issues/2782#issuecomment-68556780
 2015-01-02 21:17:10 +01:00
Jaime Marquínez Ferrándiz 817f786fbb [canalplus] Raise an error if the video is georestricted (closes #4472) 2015-01-02 21:02:34 +01:00
Jaime Marquínez Ferrándiz f86d543ebb [pbs] Catch geoblocking errors (closes #4516) 2014-12-31 17:43:49 +01:00
Jaime Marquínez Ferrándiz 60e47a2699 [youtube] Use '_download_xml' for getting the available subtitles 2014-12-31 15:44:15 +01:00
Jaime Marquínez Ferrándiz 7d900ef1bf [youtube] Add support for automatically translated subtitles (fixes #4555) 
They have a manually uploaded subtitles track and YouTube can transtale it.
 2014-12-31 14:15:16 +01:00
Jaime Marquínez Ferrándiz e0ae1814b1 [sportdeutschland] Fix extraction (fixes #4544) 2014-12-22 22:24:19 +01:00
Jaime Marquínez Ferrándiz f00fd51dae Don't write the description file if info_dict['description'] is None (#3166) 2014-12-21 20:49:14 +01:00
Jaime Marquínez Ferrándiz 5d3808524d [extractor/common] Update docstring: replace FileDownloader with YoutubeDL 2014-12-21 16:58:29 +01:00
Jaime Marquínez Ferrándiz c8f167823f [dbtv] Make sure the 'id' field is a string 2014-12-21 16:57:07 +01:00
Jaime Marquínez Ferrándiz 70f6796e7d [telecinco] Rename 'episode' group to 'id' in the _VALID_URL regex 
MiTeleIE now uses '_match_id'
 2014-12-21 16:54:53 +01:00
Jaime Marquínez Ferrándiz 85d253af6b [internetvideoarchive] Update test's duration field 2014-12-21 15:37:21 +01:00
Jaime Marquínez Ferrándiz a86cbf5876 [rtp] Fix test's id field 2014-12-21 15:28:40 +01:00
Jaime Marquínez Ferrándiz 3f1399de8a [tmz] Fix test's thumbnail field 2014-12-21 15:26:00 +01:00
Jaime Marquínez Ferrándiz 1f809a8560 [nerdcubed] Style fixes 2014-12-21 15:22:30 +01:00
Jaime Marquínez Ferrándiz 653d14e2f9 [yahoo] Update extraction process 
Their webpage uses now https://video.media.yql.yahoo.com/v1/video/sapi/streams/ for getting the video info.
 2014-12-21 14:47:44 +01:00
Jaime Marquínez Ferrándiz 85fab7e47b [yahoo] Replace two tests 
The first one returned an internal server error.
The other doesn't seem to contain a video anymore.
 2014-12-21 14:47:12 +01:00
Jaime Marquínez Ferrándiz 3aa9176f08 [yahoo] Improve video id detection (fixes #4521) 2014-12-21 14:09:00 +01:00
Jaime Marquínez Ferrándiz e54a3a2f01 [screencastomatic] Remove unused variable 2014-12-17 14:56:30 +01:00
Jaime Marquínez Ferrándiz 0e4cb4f406 YoutubeDL: style fix 2014-12-17 14:55:27 +01:00
Jaime Marquínez Ferrándiz a16af51873 flake8: Add more ignored files 
* setup.py: the '__version__' variable is not defined in the script, it is loadded from youtube_dl/version.py
* devscripts/buildserver.py: Produces a lot of messages
 2014-12-16 20:38:59 +01:00
Jaime Marquínez Ferrándiz dc9a441bfa Move flake8 configuration to setup.cfg 
It will be used when calling flake8 from any directory in the project
 2014-12-16 20:34:07 +01:00
Jaime Marquínez Ferrándiz ee6dfe8308 Use flake8 instead of pyflakes and pep8 
It combines both tools
 2014-12-16 20:34:07 +01:00
Jaime Marquínez Ferrándiz 2cb5b03e53 [test/test_unicode_literals] Remove duplicated imports 2014-12-16 20:33:23 +01:00
Jaime Marquínez Ferrándiz bb18d787b5 [aljazeera] Add extractor (closes #4487) 2014-12-16 15:48:01 +01:00
Jaime Marquínez Ferrándiz 4b7df0d30c [youtube:playlist] Work around buggy playlists (fixes #4449) 
They show a "Load more" button, but they don't have more videos.
The continuation url in the json file was a link to itself, so we ended up in an infinite loop.
 2014-12-15 19:19:15 +01:00
Jaime Marquínez Ferrándiz 3013bbb27d Remove unused imports 2014-12-15 08:24:50 +01:00