Compare commits

..

148 Commits

Author SHA1 Message Date
Sergey M․ fe0049d1a7 Start moving to ytdl-org 2019-03-09 18:25:26 +07:00
Philipp Hagemeister d2e9088ee4 prevent access to .git
The repository wasn't secret per se, but there's no reason to allow access to the git files.

Thanks to Ben Stock and CISPA for notifying us.
2017-02-03 22:07:13 +01:00
Sergey M․ 167e13273e Use latest in download URLs 2016-07-17 18:45:31 +07:00
Sergey M․ 40f283d159 Add some more shortcuts 2016-06-26 02:20:59 +07:00
Sergey M․ 6a020a3218 Add yt-dl.org/questions shortcut 2016-06-26 02:00:20 +07:00
Sergey M․ e275a2f1ab [update_latest.sh] Fix root latest redirection 2016-06-11 03:55:38 +07:00
Sergey M․ 8afb663891 [update_latest.sh] Fix download links redirection (2) 2016-06-11 03:19:16 +07:00
Sergey M․ 1df8afcc18 [update_latest.sh] Fix download links redirection 2016-06-11 02:56:11 +07:00
Sergey M․ b0035c3448 Allow latest without trailing slash 2016-06-11 02:25:05 +07:00
Sergey M․ 8c6f08911f [update_latest.sh] Redirect all releases to GitHub 2016-06-11 02:20:39 +07:00
Philipp Hagemeister 7fc3fe2b54 save traffic on 302 2016-06-04 11:58:05 +02:00
Philipp Hagemeister 6a6bbc01f5 ignore latest_version file 2016-06-04 11:41:53 +02:00
Philipp Hagemeister 774e0d72cd modify index 2016-06-04 11:41:04 +02:00
Philipp Hagemeister 00221fad40 add better rewrite rules 2016-06-04 11:39:40 +02:00
Philipp Hagemeister c0716886d8 readd /latest directory 2016-06-04 10:20:14 +02:00
Philipp Hagemeister 386cc2f23e update latest etc. 2016-06-04 10:18:57 +02:00
Philipp Hagemeister 1a93d2ce6e Redirect current version to GitHub releases
Our hoster uberspace noted that we're by far exceeding our traffic limit.
For now, redirect everything to GitHub releases.
2016-06-04 10:08:23 +02:00
Philipp Hagemeister 86d55e2f99 Add shortcut URL for 403 errors when used in webbrowser 2015-12-30 13:13:17 +01:00
Philipp Hagemeister 1d97efec15 [htaccess] Add a shortcut for missing example URLs 2015-02-18 10:32:26 +01:00
Philipp Hagemeister 31326e8739 FAQ for -citw 2015-02-11 20:30:16 +01:00
Philipp Hagemeister 843060b2e0 Correct htaccess 2015-02-03 14:22:45 +01:00
Philipp Hagemeister 9c49a1a8bd Add a redirect for anime/illegal sites 2015-02-03 14:20:54 +01:00
Philipp Hagemeister da9084a061 [.htaccess] Link to https 2015-01-04 02:43:34 +01:00
Philipp Hagemeister caae7db812 [.htaccess] Add short URL yt-dl.org/donate 2015-01-04 02:41:24 +01:00
Philipp Hagemeister a29e9ec53c Change link to support on yt-dl.org (#4484) 2014-12-16 00:27:34 +01:00
Philipp Hagemeister 29a8bfbaa9 [/update] Fix hash 2014-11-14 00:41:41 +01:00
Philipp Hagemeister 5e4ecfe225 [/latest] Correct content type and prevent caching 2014-11-13 23:37:57 +01:00
Philipp Hagemeister b2ee8bf786 add /update 302 2014-11-13 22:42:59 +01:00
Philipp Hagemeister da0a0d59a1 Ignore test from master branch 2014-11-13 22:41:21 +01:00
Philipp Hagemeister 52f8db8b79 Filter string latest in list of latest versions 2014-06-01 10:27:01 +02:00
Philipp Hagemeister 644f92f6a5 Write downloads .htaccess, not main one 2014-04-30 10:40:14 +02:00
Philipp Hagemeister 54a1852f08 Add a redirect for downloads/latest 2014-04-30 10:39:23 +02:00
Philipp Hagemeister 0b33704b43 Re-add +Indexes 2014-04-30 10:34:58 +02:00
Philipp Hagemeister 17e311adf0 Add latest symlink in downloads (Fixes #2835) 2014-04-30 10:21:09 +02:00
Philipp Hagemeister 809987a65c Do not escape redirect URL 2014-04-19 15:00:04 +02:00
Philipp Hagemeister 2f0e8ad0a6 Rename reporting shortener 2014-04-19 14:57:36 +02:00
Philipp Hagemeister a5bd123ec0 Shorten bug reporting URL (Suggested by @AGSPhoenix) 2014-04-19 14:55:46 +02:00
Philipp Hagemeister c74d3d5d80 echo latest version, too 2013-06-25 20:54:43 +02:00
Philipp Hagemeister c20b2894ff Linkify latest 2013-06-25 20:53:54 +02:00
Philipp Hagemeister 005caca467 Disable automatic directory detection 2013-06-25 20:51:29 +02:00
Philipp Hagemeister abb88dcb2f Allow latest to be called from everywhere 2013-06-25 19:13:59 +02:00
Philipp Hagemeister c052029267 Add an error message if number of arguments is incorrect 2013-06-25 19:04:30 +02:00
Philipp Hagemeister 0669b923c0 Allow listing latest/ 2013-06-25 18:32:45 +02:00
Philipp Hagemeister 110c975154 Script to update the latest/ directory (#925) 2013-06-25 18:29:07 +02:00
Philipp Hagemeister 8005b349a4 Add latest/ directory 2013-06-25 18:00:37 +02:00
Philipp Hagemeister d185f03455 Record changes to download server 2013-06-25 17:33:24 +02:00
Philipp Hagemeister 420cf88aeb Remove copyright note (this page is trivial) 2013-06-25 17:28:55 +02:00
Philipp Hagemeister 504afbe65d Update gitignore 2013-05-10 02:09:43 +02:00
Philipp Hagemeister d929dd31d1 IP check 2013-05-10 02:09:39 +02:00
Philipp Hagemeister 49698bfaa3 Merge pull request #828 from jaimeMF/download-server
Fix float display
2013-05-09 16:48:07 -07:00
Jaime Marquínez Ferrándiz 633f6f9b39 Fix float display 2013-05-07 14:25:15 +02:00
Filippo Valsorda 71c8858fcd fixes 2012-12-31 17:04:16 +01:00
Philipp Hagemeister 83f5182bcf let travis-lint pass as well ... 2012-12-12 20:37:20 +01:00
Philipp Hagemeister fb65f76630 Tell travis not to test this 2012-12-12 20:35:36 +01:00
Philipp Hagemeister 1f69efb978 Correct download links 2012-12-12 20:32:00 +01:00
Philipp Hagemeister 3bcc0a565c New downloads page for git 2012-12-12 20:21:16 +01:00
Philipp Hagemeister 2840ae3a8f Update homepage 2012-12-11 17:52:56 +01:00
Philipp Hagemeister b51355ef99 Credit @FiloSottile 2012-12-11 17:43:50 +01:00
Philipp Hagemeister 1762b604b1 new release 2012-12-11 17:34:56 +01:00
Philipp Hagemeister 8c64aa48d2 New generate-download for the new build mechanism 2012-12-11 17:34:33 +01:00
Philipp Hagemeister 8c3a45b6e4 Update supported extractors 2012-11-30 08:38:14 +01:00
Philipp Hagemeister dc9e2d8e0c Document CollegeHumor and arte.tv 2012-11-28 17:57:06 +01:00
Philipp Hagemeister e7d727efc2 Drop 2.5 support 2012-11-28 03:30:44 +01:00
Philipp Hagemeister a89c5d20c7 2012.11.29 2012-11-27 18:37:27 +01:00
Philipp Hagemeister 044cb16590 release 2012.11.28 2012-11-27 16:16:51 +01:00
Philipp Hagemeister 81db2d8040 Document --restrict-filenames, remove stitle 2012-11-27 16:06:59 +01:00
Philipp Hagemeister 2811949f4c Update doc for 2012.11.27 2012-11-27 00:26:05 +01:00
Philipp Hagemeister 3a2c78a2da Document configuration options 2012-11-26 23:23:33 +01:00
Philipp Hagemeister 23cf656bd1 Automatically update copyright (Closes #549) 2012-11-26 11:19:40 +01:00
Philipp Hagemeister 41ff5333f7 ignore temporary files 2012-11-26 11:18:31 +01:00
Philipp Hagemeister 571d094858 Clarify required Python version (#549) 2012-11-26 11:18:08 +01:00
Philipp Hagemeister 96679b26cf release 2012.11.17 2012-11-26 04:05:23 +01:00
Philipp Hagemeister b7d1a2247e Python 2.6 instead of 2.5 in about (Closes #516) 2012-11-07 01:49:35 +01:00
Philipp Hagemeister 9746c518a5 update docs 2012-10-10 19:33:08 +02:00
Philipp Hagemeister 959fa33ef9 release youtube-dl 2012.10.09 2012-10-10 19:27:50 +02:00
Philipp Hagemeister a44c76cdf4 release release 2012.09.27 2012-09-28 13:27:54 +02:00
Philipp Hagemeister 6a3ff26a01 Add recent breakages to FAQ (#433) 2012-09-27 23:29:40 +02:00
Philipp Hagemeister 60a1c34662 Drop 2.5 compatibility 2012-09-27 15:33:20 +02:00
Philipp Hagemeister 49bd2af54f Clarify 402 2012-02-28 02:00:00 +01:00
Philipp Hagemeister a5999b83fd release 2012.02.27 2012-02-27 20:20:08 +01:00
Philipp Hagemeister 82eedd7d83 update docs for 2012.02.26 2012-02-27 00:43:20 +01:00
Philipp Hagemeister 759320387a Use stitle instead of title in doc 2012-01-12 20:15:54 +01:00
Philipp Hagemeister 0be94d4c6a release 2012.01.08b docs 2012-01-08 17:24:46 +01:00
Philipp Hagemeister 6bd03617e6 docs for 2012.01.05 2012-01-05 11:10:14 +01:00
Philipp Hagemeister a3d06cfe6d doc for 2011.12.18 2011-12-17 01:35:45 +01:00
Philipp Hagemeister cd891850ec Release 2011.12.15 2011-12-15 20:33:49 +01:00
Philipp Hagemeister cdee681a2c Release 2011.12.08 2011-12-08 21:41:43 +01:00
Philipp Hagemeister db2c6281eb Release 2011.11.23: Update webpage 2011-11-23 10:39:50 +01:00
Philipp Hagemeister 06a129c9dd Release 2011.11.22 2011-11-22 15:39:04 +01:00
Philipp Hagemeister 5a938b5d04 +Makefile 2011-11-22 15:38:32 +01:00
Philipp Hagemeister b0e67d2404 Release 2011.11.21 2011-11-21 21:54:04 +01:00
Philipp Hagemeister 92961845c3 docs for 2011.10.19 2011-10-19 00:40:55 +02:00
Philipp Hagemeister f9c3398449 release 2011.09.30 2011-09-30 09:08:53 +02:00
Philipp Hagemeister bffe76ba4b Remove empty <tr>s in index 2011-09-28 09:55:54 +02:00
Philipp Hagemeister 0d29baa7c6 Make documentation valid HTML 2011-09-28 09:53:50 +02:00
Philipp Hagemeister dd9d3efcfb Make FAQ valid HTML 2011-09-28 09:52:01 +02:00
Philipp Hagemeister 9014805f26 Fix stray </a> in FAQ 2011-09-28 09:50:04 +02:00
Philipp Hagemeister f973da57a4 Switch to HTML5 doctype 2011-09-28 09:49:19 +02:00
Philipp Hagemeister a073eb792f Update documentation and FAQ 2011-09-28 09:47:54 +02:00
Philipp Hagemeister f202278fe1 Close <ul> in first FAQ question 2011-09-28 09:29:19 +02:00
Philipp Hagemeister 3328f34eb3 Update homepage for 2011.09.27 2011-09-27 21:44:59 +02:00
Philipp Hagemeister 50d7a239ae Raise correct args in exception 2011-09-27 21:44:02 +02:00
Philipp Hagemeister 81eb98e4bd Update docs 2011-09-27 21:17:50 +02:00
Philipp Hagemeister 78ef538449 Updated bug reporting instructions in the FAQ 2011-09-27 20:58:37 +02:00
Philipp Hagemeister 6e3ed6385c Download regeneration works with python 2.5 2011-09-27 20:50:03 +02:00
Ricardo Garcia a2dafa7316 Add creative commons license to webpages 2011-09-27 20:17:34 +02:00
Philipp Hagemeister 86e5e08789 release 2011.09.18c 2011-09-17 00:59:45 +02:00
Philipp Hagemeister e397cf5b47 2011.09.18b 2011-09-16 22:33:54 +02:00
Philipp Hagemeister 3fd01c88c5 Update documentation for 2011.09.18 2011-09-15 19:29:53 +02:00
Philipp Hagemeister d2cbb552b7 Update documentation for 2011.09.17 2011-09-15 19:25:07 +02:00
Philipp Hagemeister 9c358a8982 Release 2011.09.16 2011-09-15 18:49:20 +02:00
Philipp Hagemeister 7923514ea0 Release 2011.09.15 2011-09-14 22:56:50 +02:00
Philipp Hagemeister 004759c8f2 Add a prominent Develop link in the documentation (Closes #155) 2011-09-14 00:42:45 +02:00
Philipp Hagemeister 95f029a5d8 Release 2011.09.14 2011-09-14 00:02:16 +02:00
Philipp Hagemeister ac47cfe369 Update generate-download 2011-09-13 23:06:09 +02:00
Philipp Hagemeister 463fe92d26 Release new version 2011-09-13 23:05:58 +02:00
Ricardo Garcia ba8a90b0bf Regenerate download page 2011-08-04 19:16:25 +02:00
Ricardo Garcia 3a6c68881a Regenerate download page 2011-03-29 20:33:22 +02:00
Ricardo Garcia a290358c2f Regenerate download page 2011-02-26 00:48:41 +01:00
Ricardo Garcia 17faaa4548 Regenerate download page 2011-02-25 21:54:55 +01:00
Ricardo Garcia 3ea19089f9 Regenerate download page 2011-02-25 20:13:32 +01:00
Ricardo Garcia 9077650490 Add Gergely Imreh to the author list 2011-02-20 18:03:52 +01:00
Ricardo Garcia 2ecbfb4cbc Properly update Python version requirement in the documentation 2011-01-30 13:14:45 +01:00
Ricardo Garcia 3519974510 Regenerate download page 2011-01-30 13:12:45 +01:00
Ricardo Garcia e76e4dc8b1 Add Paweł Paprota to the list of authors in the documentation 2011-01-30 13:12:10 +01:00
Ricardo Garcia 01e013b5e3 Mention support for YouTube.com user videos in the documentation 2011-01-30 13:10:22 +01:00
Ricardo Garcia 64ea994344 Update documentation to reflect Python 2.5 requirement 2011-01-30 12:56:52 +01:00
Ricardo Garcia a3e9208c01 Add Witold Baryluk to the author list 2011-01-21 18:19:43 +01:00
Ricardo Garcia ec0d2e46c5 Add link to the project page from the about page 2011-01-07 10:43:13 +01:00
Ricardo Garcia 1b3dbefb04 Modify pages increasing the copyright year 2011-01-03 10:54:47 +01:00
Ricardo Garcia c3003bb4db Add question about option -g to the FAQ 2011-01-03 10:54:07 +01:00
Ricardo Garcia 6c7d5d8d14 Regenerate download page 2010-12-09 19:58:30 +01:00
Ricardo Garcia 35c0ad4587 Regenerate download page 2010-12-09 19:38:15 +01:00
Ricardo Garcia 941383109a Document new DepositFiles InfoExtractor 2010-12-08 10:59:59 +01:00
Ricardo Garcia c95ac4f062 Regenerate download page 2010-11-19 19:42:22 +01:00
Ricardo Garcia 0732b1f39f Document change from "ord" to "autonumber" in template 2010-11-19 19:38:36 +01:00
Ricardo Garcia 8d78373ea0 Document new upload_date template parameter 2010-11-19 19:38:07 +01:00
Ricardo Garcia 17082ad9db Put the main buttons ordered vertically 2010-11-19 18:24:03 +01:00
Ricardo Garcia 725d763c48 Reorganize website 2010-11-19 18:19:41 +01:00
Ricardo Garcia a993428ac9 Document rtmpdump's optional requirement in the main page 2010-11-06 18:46:50 +01:00
Ricardo Garcia 7136abc8f3 Improve documentation on "SyntaxError" to take Python 3.x into account 2010-11-06 18:45:10 +01:00
Ricardo Garcia 790622091e Renamed "generate-home" to "generate-index" for consistency 2010-11-06 11:29:10 +01:00
Ricardo Garcia db03ff92a6 Changed a few style aspects to make the page look better under IE 2010-11-06 11:29:10 +01:00
Ricardo Garcia b294d9fa38 Use SSL in URLs where possible 2010-11-06 11:28:56 +01:00
Ricardo Garcia 5de4c645e2 Make web pages valid HTML and remove a few glitches 2010-11-02 21:28:56 +01:00
Ricardo Garcia 05703c8f9c Minor changes to stylesheet 2010-11-01 11:28:23 +01:00
Ricardo Garcia f37b138766 Add copyright notice at the bottom of every page 2010-11-01 08:23:57 +01:00
Ricardo Garcia 5c7359365f Initial pages commit, using the wiki contents 2010-10-31 23:50:48 +01:00
15 changed files with 260 additions and 1193 deletions
+13
View File
@@ -0,0 +1,13 @@
*.kate-swp
downloads/*
updates_key.pem
youtube_dl.egg-info
test
build/
dist/
youtube-dl.1
youtube-dl.fish
youtube-dl.zsh
README.txt
youtube-dl.bash-completion
latest_version
-4
View File
@@ -1,4 +0,0 @@
syntax: glob
index.html
youtube-dl-*
.*.swp
+24
View File
@@ -0,0 +1,24 @@
Options +Indexes
RewriteEngine On
RewriteRule ^ip/?$ ip.php
RewriteRule ^bugs?/? https://github.com/ytdl-org/youtube-dl/issues [R=302,L]
RewriteRule ^reporting/? https://github.com/ytdl-org/youtube-dl/blob/master/README.md#bugs [R=302,L,NE]
RewriteRule ^update/?$ https://github.com/ytdl-org/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl [R=302,L,NE]
RewriteRule ^donat(e|ions)/?$ https://ytdl-org.github.io/youtube-dl/donations.html [R=302,L]
RewriteRule ^(faq-)?anime/?$ https://github.com/ytdl-org/youtube-dl/blob/master/README.md#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free [R=302,L,NE]
RewriteRule ^(faq-)?citw/?$ https://github.com/ytdl-org/youtube-dl/blob/master/README.md#do-i-always-have-to-pass-in---max-quality-format-or--citw [R=302,L,NE]
RewriteRule ^(?:example-url|questions)/?$ https://github.com/ytdl-org/youtube-dl/blob/master/CONTRIBUTING.md#is-the-description-of-the-issue-itself-sufficient [R=302,L,NE]
RewriteRule ^g403/?$ https://github.com/ytdl-org/youtube-dl/blob/master/README.md#i-extracted-a-video-url-with--g-but-it-does-not-play-on-another-machine--in-my-webbrowser [R=302,L,NE]
RewriteRule ^format-selection/?$ https://github.com/ytdl-org/youtube-dl#format-selection [R=302,L,NE]
RewriteRule ^output-template/?$ https://github.com/ytdl-org/youtube-dl#output-template [R=302,L,NE]
RewriteRule ^latest/version/?$ latest_version [L,T=text/plain]
RewriteRule ^latest_version/?$ - [T=text/plain]
RewriteRule ^latest(?:/(.*))?$ /downloads/latest/$1 [R=302,L]
RewriteRule ^\.git - [F]
ErrorDocument 302 "302"
+3
View File
@@ -0,0 +1,3 @@
# Just a placeholder script so that travis doesn't complain
language: python
script: true
View File
+5
View File
@@ -0,0 +1,5 @@
RewriteEngine On
RewriteRule ^$ https://github.com/ytdl-org/youtube-dl/releases
RewriteRule latest(.*) /downloads/2019.03.09$1 [L,R=302]
+5
View File
@@ -0,0 +1,5 @@
RewriteEngine On
RewriteRule ^$ https://github.com/ytdl-org/youtube-dl/releases/tag/2016.06.03_tmp [R=302,L]
RewriteRule ^(.+)$ https://github.com/ytdl-org/youtube-dl/releases/download/2016.06.03_tmp/$1 [R=302,L]
-15
View File
@@ -1,15 +0,0 @@
#!/usr/bin/env python
import hashlib
import subprocess
template = file('index.html.in', 'r').read()
version = subprocess.Popen(['./youtube-dl', '--version'], stdout=subprocess.PIPE).communicate()[0].strip()
data = file('youtube-dl', 'rb').read()
md5sum = hashlib.md5(data).hexdigest()
sha1sum = hashlib.sha1(data).hexdigest()
sha256sum = hashlib.sha256(data).hexdigest()
template = template.replace('@PROGRAM_VERSION@', version)
template = template.replace('@PROGRAM_MD5SUM@', md5sum)
template = template.replace('@PROGRAM_SHA1SUM@', sha1sum)
template = template.replace('@PROGRAM_SHA256SUM@', sha256sum)
file('index.html', 'w').write(template)
-214
View File
@@ -1,214 +0,0 @@
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-type" content="text/html; charset=UTF-8" />
<title>youtube-dl: Download videos from YouTube.com</title>
<style type="text/css"><!--
body {
font-family: sans-serif;
font-size: small;
}
h1 {
text-align: center;
text-decoration: underline;
color: #006699;
}
h2 {
color: #006699;
}
p {
text-align: justify;
margin-left: 5%;
margin-right: 5%;
}
ul {
margin-left: 5%;
margin-right: 5%;
list-style-type: square;
}
li {
margin-bottom: 0.5ex;
}
.smallnote {
font-size: x-small;
text-align: center;
}
--></style>
</head>
<body>
<h1>youtube-dl: Download videos from YouTube.com</h1>
<p class="smallnote">(and more...)</p>
<h2>What is it?</h2>
<p><em>youtube-dl</em> is a small command-line program to download videos
from YouTube.com. It requires the <a href="http://www.python.org/">Python
interpreter</a>, version 2.4 or later, and it's not platform specific.
It should work in your Unix box, in Windows or in Mac OS X. The latest version
is <strong>@PROGRAM_VERSION@</strong>. It's released to the public domain,
which means you can modify it, redistribute it or use it however you like.</p>
<p>I'll try to keep it updated if YouTube.com changes the way you access
their videos. After all, it's a simple and short program. However, I can't
guarantee anything. If you detect it stops working, check for new versions
and/or inform me about the problem, indicating the program version you
are using. If the program stops working and I can't solve the problem but
you have a solution, I'd like to know it. If that happens and you feel you
can maintain the program yourself, tell me. My contact information is
at <a href="http://freshmeat.net/~rg3/">freshmeat.net</a>.</p>
<p>Thanks for all the feedback received so far. I'm glad people find my
program useful.</p>
<h2>Usage instructions</h2>
<p>In Windows, once you have installed the Python interpreter, save the
program with the <em>.py</em> extension and put it somewhere in the PATH.
Try to follow the
<a href="http://rg03.wordpress.com/youtube-dl-under-windows-xp/">guide to
install youtube-dl under Windows XP</a>.</p>
<p>In Unix, download it, give it execution permission and copy it to one
of the PATH directories (typically, <em>/usr/local/bin</em>).</p>
<p>After that, you should be able to call it from the command line as
<em>youtube-dl</em> or <em>youtube-dl.py</em>. I will use <em>youtube-dl</em>
in the following examples. Usage instructions are easy. Use <em>youtube-dl</em>
followed by a video URL or identifier. Example: <em>youtube-dl
"http://www.youtube.com/watch?v=foobar"</em>. The video will be saved
to the file <em>foobar.flv</em> in that example. As YouTube.com
videos are in Flash Video format, their extension should be <em>flv</em>.
In Linux and other unices, video players using a recent version of
<em>ffmpeg</em> can play them. That includes MPlayer, VLC, etc. Those two
work under Windows and other platforms, but you could also get a
specific FLV player of your taste.</p>
<p>If you try to run the program and you receive an error message containing the
keyword <em>SyntaxError</em> near the end, it means your Python interpreter
is too old.</p>
<h2>More usage tips</h2>
<ul>
<li>You can change the file name of the video using the -o option, like in
<em>youtube-dl -o vid.flv "http://www.youtube.com/watch?v=foobar"</em>.
Read the <a href="#otpl">Output template</a> section for more details on
this.</li>
<li>Some videos require an account to be downloaded, mostly because they're
flagged as mature content. You can pass the program a username and password
for a YouTube.com account with the -u and -p options, like <em>youtube-dl
-u myusername -p mypassword "http://www.youtube.com/watch?v=foobar"</em>.</li>
<li>The account data can also be read from the user .netrc file by indicating
the -n or --netrc option. The machine name is <em>youtube</em> in that
case.</li>
<li>The <em>simulate mode</em> (activated with -s or --simulate) can be used
to just get the real video URL and use it with a download manager if you
prefer that option.</li>
<li>The <em>quiet mode</em> (activated with -q or --quiet) can be used to
supress all output messages. This allows, in systems featuring /dev/stdout
and other similar special files, outputting the video data to standard output
in order to pipe it to another program without interferences.</li>
<li>The program can be told to simply print the final video URL to standard
output using the -g or --get-url option.</li>
<li>In a similar line, the -e or --get-title option tells the program to print
the video title.</li>
<li>The default filename is <em>video_id.flv</em>. But you can also use the
video title in the filename with the -t or --title option, or preserve the
literal title in the filename with the -l or --literal option.</li>
<li>You can make the program append <em>&amp;fmt=something</em> to the URL
by using the -f or --format option. This makes it possible to download high
quality versions of the videos when available.</li>
<li>The -b or --best-quality option is an alias for -f 18.</li>
<li>The -m or --mobile-version option is an alias for -f 17.</li>
<li>Normally, the program will stop on the first error, but you can tell it
to attempt to download every video with the -i or --ignore-errors option.</li>
<li>The -a or --batch-file option lets you specify a file to read URLs from.
The file must contain one URL per line.</li>
<li><em>youtube-dl</em> honors the <em>http_proxy</em> environment variable
if you want to use a proxy. Set it to something like
<em>http://proxy.example.com:8080</em>, and do not leave the <em>http://</em>
prefix out.</li>
<li>You can get the program version by calling it as <em>youtube-dl
-v</em> or <em>youtube-dl --version</em>.</li>
<li>For usage instructions, use <em>youtube-dl -h</em> or <em>youtube-dl
--help.</em></li>
<li>You can cancel the program at any time pressing Ctrl+C. It may print
some error lines saying something about <em>KeyboardInterrupt</em>.
That's ok.</li>
</ul>
<h2 id="otpl">Download it</h2>
<p>Note that if you directly click on these hyperlinks, your web browser will
most likely display the program contents. It's usually better to
right-click on it and choose the appropriate option, normally called <em>Save
Target As</em> or <em>Save Link As</em>, depending on the web browser you
are using.</p>
<p><a href="youtube-dl">@PROGRAM_VERSION@</a></p>
<ul>
<li><strong>MD5</strong>: @PROGRAM_MD5SUM@</li>
<li><strong>SHA1</strong>: @PROGRAM_SHA1SUM@</li>
<li><strong>SHA256</strong>: @PROGRAM_SHA256SUM@</li>
</ul>
<h2>Output template</h2>
<p>The -o option allows users to indicate a template for the output file names.
The basic usage is not to set any template arguments when downloading a single
file, like in <em>youtube-dl -o funny_video.flv 'http://some/video'</em>.
However, it may contain special sequences that will be replaced when
downloading each video. The special sequences have the format
<strong>%(NAME)s</strong>. To clarify, that's a percent symbol followed by a
name in parenthesis, followed by a lowercase S. Allowed names are:</p>
<ul>
<li><em>id</em>: The sequence will be replaced by the video identifier.</li>
<li><em>url</em>: The sequence will be replaced by the video URL.</li>
<li><em>uploader</em>: The sequence will be replaced by the nickname of the
person who uploaded the video.</li>
<li><em>title</em>: The sequence will be replaced by the literal video
title.</li>
<li><em>stitle</em>: The sequence will be replaced by a simplified video
title, restricted to alphanumeric characters and dashes.</li>
<li><em>ext</em>: The sequence will be replaced by the appropriate
extension (like <em>flv</em> or <em>mp4</em>).</li>
</ul>
<p>As you may have guessed, the default template is <em>%(id)s.%(ext)s</em>.
When some command line options are used, it's replaced by other templates like
<em>%(title)s-%(id)s.%(ext)s</em>. You can specify your own.</p>
<h2>Authors</h2>
<ul>
<li>Ricardo Garcia Gonzalez: program core, YouTube.com InfoExtractor,
metacafe.com InfoExtractor and YouTube playlist InfoExtractor.</li>
<li>Many other people contributing patches, code, ideas and kind messages. Too
many to be listed here. You know who you are. Thank you very much.</li>
</ul>
<p class="smallnote">Copyright &copy; 2006-2007 Ricardo Garcia Gonzalez</p>
</body>
</html>
+36
View File
@@ -0,0 +1,36 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8">
<title>youtube-dl</title>
<link rel="stylesheet" href="style.css" type="text/css">
</head>
<body>
<h1>youtube-dl downloads</h1>
<?php
$latest = file_get_contents('latest_version');
echo '<div class="latest">';
echo '<div><a href="latest">Latest</a> (v' . htmlspecialchars($latest) . ') downloads:</div>';
echo '<a href="downloads/latest/youtube-dl">youtube-dl</a> ';
echo '<a href="downloads/latest/youtube-dl.exe">youtube-dl.exe</a> ';
echo '<a href="downloads/latest/youtube-dl-' . htmlspecialchars($latest) . '.tar.gz">youtube-dl-' . htmlspecialchars($latest) . '.tar.gz</a>';
echo '</div>';
?>
See the right for more resources.
<table border="0" id="rgb" style="float: right;">
<tr><td><a class="button" id="main-homepage" href="http://ytdl-org.github.com/youtube-dl/">Homepage</a></td></tr>
<tr><td><a class="button" id="g" href="http://ytdl-org.github.com/youtube-dl/download.html">Download instructions</a></td></tr>
<tr><td><a class="button" id="r" href="http://ytdl-org.github.com/youtube-dl/documentation.html">Documentation</a></td></tr>
<tr><td><a class="button" id="main-support" href="https://github.com/ytdl-org/youtube-dl/issues/">Support</a></td></tr>
<tr><td><a class="button" id="y" href="https://github.com/ytdl-org/youtube-dl/">Develop</a></td></tr>
<tr><td><a class="button" id="b" href="http://ytdl-org.github.com/youtube-dl/about.html">About</a></td></tr>
</table>
</body>
</html>
+3
View File
@@ -0,0 +1,3 @@
<?php
header('Content-Type: text/plain');
echo $_SERVER['REMOTE_ADDR'];
+152
View File
@@ -0,0 +1,152 @@
body {
font-family: sans-serif;
margin-left: 10%;
margin-right: 10%;
margin-top: 2ex;
margin-bottom: 3ex;
background-color: white;
color: black;
/*background-color: #fff1db;*/
background-color: white;
/*
background-image: url("gradient.png");
background-repeat: repeat-x;
*/
/*
background-image: url("gradient2.png");
background-repeat: repeat-y;
*/
/*
background-image: url("gradient3.png");
background-repeat: repeat-x;
*/
/*
background-image: url("gradient4.png");
background-repeat: repeat-y;
*/
background-image: url("gradient5.png");
background-repeat: repeat-x;
}
.heading {
border: 0;
color: black;
font-size: xx-large;
font-weight: bold;
padding-bottom: 1ex;
border-bottom: 1px solid black;
margin-bottom: 2ex;
width: 100%;
}
.heading tr {
border: 0;
}
.heading td {
border: 0;
}
.heading a {
text-decoration: none;
color: black;
}
.title {
text-align: left;
}
.subtitle {
text-align: right;
}
.toc {
padding-left: 2ex;
border: 1px solid #aaaaaa;
background-color: white;
padding-bottom: 1ex;
border-radius: 10px;
-moz-border-radius: 10px;
}
.toc ul {
margin: 0; list-style-type: none;
}
hr {
margin-top: 3ex;
margin-bottom: 3ex;
width: 50%;
}
.note {
margin-top: 10ex;
text-align: center;
font-size: x-small;
clear: both;
}
h1 {
font-size: x-large;
margin-top: 2ex;
color: black;
margin-left: 2%;
margin-right: 2%;
}
h2 {
font-size: large;
margin-left: 5%;
margin-right: 5%;
}
p {
margin-left: 5%;
margin-right: 5%;
}
ul {
margin-left: 5%;
margin-right: 5%;
}
li {
margin-left: 3%;
margin-top: 0.5ex;
margin-bottom: 0.5ex;
}
tt {
padding-left: 0.5ex;
padding-right: 0.5ex;
background: #dddddd;
}
#rgb {
width: 33%;
margin: 3ex auto;
}
.button {
color: white;
font-weight: bold;
font-size: x-large;
text-decoration: none;
text-align: center;
display: block;
padding: 2ex;
border-radius: 10px;
-moz-border-radius: 10px;
}
#r {
background-color: #884444;
border: 2px solid #880000;
}
#g {
background-color: #448844;
border: 2px solid #006600;
}
#b {
background-color: #444488;
border: 2px solid #000088;
}
#y {
background-color: #888844;
border: 2px solid #666600;
}
#main-homepage {
background-color: #848;
border: 2px solid #808;
}
#main-support {
background-color: #448888;
border: 2px solid #008888;
}
.all-versions {
float: left;
}
View File
+19
View File
@@ -0,0 +1,19 @@
#/bins/sh
set -e
version="$1"
if [ -z "$version" ]; then
echo "Usage: $0 version"
exit 1
fi
cd "$(dirname $(readlink -f $0))"
echo -n "$version" > latest_version
echo -e "RewriteEngine On" > downloads/.htaccess
echo -e "RewriteRule ^$ https://github.com/ytdl-org/youtube-dl/releases" >> downloads/.htaccess
echo -e "RewriteRule ^(\d{4}\.\d{2}\.\d{2}(?:\.\d+)?/?)$ https://github.com/ytdl-org/youtube-dl/releases/tag/\$1" >> downloads/.htaccess
echo -e "RewriteRule ^(\d{4}\.\d{2}\.\d{2}(?:\.\d+)?/.+)$ https://github.com/ytdl-org/youtube-dl/releases/download/\$1" >> downloads/.htaccess
echo -e "RewriteRule latest(.*) /downloads/$1\$1 [L,R=302]" >> downloads/.htaccess
-960
View File
@@ -1,960 +0,0 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Author: Ricardo Garcia Gonzalez
# License: Public domain code
import htmlentitydefs
import httplib
import locale
import math
import netrc
import os
import os.path
import re
import socket
import string
import sys
import time
import urllib
import urllib2
std_headers = {
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5',
'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
'Accept': 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5',
'Accept-Language': 'en-us,en;q=0.5',
}
simple_title_chars = string.ascii_letters.decode('ascii') + string.digits.decode('ascii')
class DownloadError(Exception):
"""Download Error exception.
This exception may be thrown by FileDownloader objects if they are not
configured to continue on errors. They will contain the appropriate
error message.
"""
pass
class SameFileError(Exception):
"""Same File exception.
This exception will be thrown by FileDownloader objects if they detect
multiple files would have to be downloaded to the same file on disk.
"""
pass
class PostProcessingError(Exception):
"""Post Processing exception.
This exception may be raised by PostProcessor's .run() method to
indicate an error in the postprocessing task.
"""
pass
class FileDownloader(object):
"""File Downloader class.
File downloader objects are the ones responsible of downloading the
actual video file and writing it to disk if the user has requested
it, among some other tasks. In most cases there should be one per
program. As, given a video URL, the downloader doesn't know how to
extract all the needed information, task that InfoExtractors do, it
has to pass the URL to one of them.
For this, file downloader objects have a method that allows
InfoExtractors to be registered in a given order. When it is passed
a URL, the file downloader handles it to the first InfoExtractor it
finds that reports being able to handle it. The InfoExtractor returns
all the information to the FileDownloader and the latter downloads the
file or does whatever it's instructed to do.
File downloaders accept a lot of parameters. In order not to saturate
the object constructor with arguments, it receives a dictionary of
options instead. These options are available through the get_params()
method for the InfoExtractors to use. The FileDownloader also registers
itself as the downloader in charge for the InfoExtractors that are
added to it, so this is a "mutual registration".
Available options:
username: Username for authentication purposes.
password: Password for authentication purposes.
usenetrc: Use netrc for authentication instead.
quiet: Do not print messages to stdout.
forceurl: Force printing final URL.
forcetitle: Force printing title.
simulate: Do not download the video files.
format: Video format code.
outtmpl: Template for output names.
ignoreerrors: Do not stop on download errors.
ratelimit: Download speed limit, in bytes/sec.
"""
_params = None
_ies = []
_pps = []
def __init__(self, params):
"""Create a FileDownloader object with the given options."""
self._ies = []
self._pps = []
self.set_params(params)
@staticmethod
def pmkdir(filename):
"""Create directory components in filename. Similar to Unix "mkdir -p"."""
components = filename.split(os.sep)
aggregate = [os.sep.join(components[0:x]) for x in xrange(1, len(components))]
aggregate = ['%s%s' % (x, os.sep) for x in aggregate] # Finish names with separator
for dir in aggregate:
if not os.path.exists(dir):
os.mkdir(dir)
@staticmethod
def format_bytes(bytes):
if bytes is None:
return 'N/A'
if bytes == 0:
exponent = 0
else:
exponent = long(math.log(float(bytes), 1024.0))
suffix = 'bkMGTPEZY'[exponent]
converted = float(bytes) / float(1024**exponent)
return '%.2f%s' % (converted, suffix)
@staticmethod
def calc_percent(byte_counter, data_len):
if data_len is None:
return '---.-%'
return '%6s' % ('%3.1f%%' % (float(byte_counter) / float(data_len) * 100.0))
@staticmethod
def calc_eta(start, now, total, current):
if total is None:
return '--:--'
dif = now - start
if current == 0 or dif < 0.001: # One millisecond
return '--:--'
rate = float(current) / dif
eta = long((float(total) - float(current)) / rate)
(eta_mins, eta_secs) = divmod(eta, 60)
if eta_mins > 99:
return '--:--'
return '%02d:%02d' % (eta_mins, eta_secs)
@staticmethod
def calc_speed(start, now, bytes):
dif = now - start
if bytes == 0 or dif < 0.001: # One millisecond
return '%10s' % '---b/s'
return '%10s' % ('%s/s' % FileDownloader.format_bytes(float(bytes) / dif))
@staticmethod
def best_block_size(elapsed_time, bytes):
new_min = max(bytes / 2.0, 1.0)
new_max = min(max(bytes * 2.0, 1.0), 4194304) # Do not surpass 4 MB
if elapsed_time < 0.001:
return int(new_max)
rate = bytes / elapsed_time
if rate > new_max:
return int(new_max)
if rate < new_min:
return int(new_min)
return int(rate)
@staticmethod
def parse_bytes(bytestr):
"""Parse a string indicating a byte quantity into a long integer."""
matchobj = re.match(r'(?i)^(\d+(?:\.\d+)?)([kMGTPEZY]?)$', bytestr)
if matchobj is None:
return None
number = float(matchobj.group(1))
multiplier = 1024.0 ** 'bkmgtpezy'.index(matchobj.group(2).lower())
return long(round(number * multiplier))
def set_params(self, params):
"""Sets parameters."""
if type(params) != dict:
raise ValueError('params: dictionary expected')
self._params = params
def get_params(self):
"""Get parameters."""
return self._params
def add_info_extractor(self, ie):
"""Add an InfoExtractor object to the end of the list."""
self._ies.append(ie)
ie.set_downloader(self)
def add_post_processor(self, pp):
"""Add a PostProcessor object to the end of the chain."""
self._pps.append(pp)
pp.set_downloader(self)
def to_stdout(self, message, skip_eol=False):
"""Print message to stdout if not in quiet mode."""
if not self._params.get('quiet', False):
print u'%s%s' % (message, [u'\n', u''][skip_eol]),
sys.stdout.flush()
def to_stderr(self, message):
"""Print message to stderr."""
print >>sys.stderr, message
def fixed_template(self):
"""Checks if the output template is fixed."""
return (re.search(ur'(?u)%\(.+?\)s', self._params['outtmpl']) is None)
def trouble(self, message=None):
"""Determine action to take when a download problem appears.
Depending on if the downloader has been configured to ignore
download errors or not, this method may throw an exception or
not when errors are found, after printing the message. If it
doesn't raise, it returns an error code suitable to be returned
later as a program exit code to indicate error.
"""
if message is not None:
self.to_stderr(message)
if not self._params.get('ignoreerrors', False):
raise DownloadError(message)
return 1
def slow_down(self, start_time, byte_counter):
"""Sleep if the download speed is over the rate limit."""
rate_limit = self._params.get('ratelimit', None)
if rate_limit is None or byte_counter == 0:
return
now = time.time()
elapsed = now - start_time
if elapsed <= 0.0:
return
speed = float(byte_counter) / elapsed
if speed > rate_limit:
time.sleep((byte_counter - rate_limit * (now - start_time)) / rate_limit)
def report_destination(self, filename):
"""Report destination filename."""
self.to_stdout(u'[download] Destination: %s' % filename)
def report_progress(self, percent_str, data_len_str, speed_str, eta_str):
"""Report download progress."""
self.to_stdout(u'\r[download] %s of %s at %s ETA %s' %
(percent_str, data_len_str, speed_str, eta_str), skip_eol=True)
def report_finish(self):
"""Report download finished."""
self.to_stdout(u'')
def download(self, url_list):
"""Download a given list of URLs."""
retcode = 0
if len(url_list) > 1 and self.fixed_template():
raise SameFileError(self._params['outtmpl'])
for url in url_list:
suitable_found = False
for ie in self._ies:
if not ie.suitable(url):
continue
# Suitable InfoExtractor found
suitable_found = True
all_results = ie.extract(url)
results = [x for x in all_results if x is not None]
if len(results) != len(all_results):
retcode = self.trouble()
if len(results) > 1 and self.fixed_template():
raise SameFileError(self._params['outtmpl'])
for result in results:
# Forced printings
if self._params.get('forcetitle', False):
print result['title']
if self._params.get('forceurl', False):
print result['url']
# Do nothing else if in simulate mode
if self._params.get('simulate', False):
continue
try:
filename = self._params['outtmpl'] % result
self.report_destination(filename)
except (ValueError, KeyError), err:
retcode = self.trouble('ERROR: invalid output template or system charset: %s' % str(err))
continue
try:
self.pmkdir(filename)
except (OSError, IOError), err:
retcode = self.trouble('ERROR: unable to create directories: %s' % str(err))
continue
try:
outstream = open(filename, 'wb')
except (OSError, IOError), err:
retcode = self.trouble('ERROR: unable to open for writing: %s' % str(err))
continue
try:
self._do_download(outstream, result['url'])
outstream.close()
except (OSError, IOError), err:
retcode = self.trouble('ERROR: unable to write video data: %s' % str(err))
continue
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
retcode = self.trouble('ERROR: unable to download video data: %s' % str(err))
continue
try:
self.post_process(filename, result)
except (PostProcessingError), err:
retcode = self.trouble('ERROR: postprocessing: %s' % str(err))
continue
break
if not suitable_found:
retcode = self.trouble('ERROR: no suitable InfoExtractor: %s' % url)
return retcode
def post_process(self, filename, ie_info):
"""Run the postprocessing chain on the given file."""
info = dict(ie_info)
info['filepath'] = filename
for pp in self._pps:
info = pp.run(info)
if info is None:
break
def _do_download(self, stream, url):
request = urllib2.Request(url, None, std_headers)
data = urllib2.urlopen(request)
data_len = data.info().get('Content-length', None)
data_len_str = self.format_bytes(data_len)
byte_counter = 0
block_size = 1024
start = time.time()
while True:
# Progress message
percent_str = self.calc_percent(byte_counter, data_len)
eta_str = self.calc_eta(start, time.time(), data_len, byte_counter)
speed_str = self.calc_speed(start, time.time(), byte_counter)
self.report_progress(percent_str, data_len_str, speed_str, eta_str)
# Download and write
before = time.time()
data_block = data.read(block_size)
after = time.time()
data_block_len = len(data_block)
if data_block_len == 0:
break
byte_counter += data_block_len
stream.write(data_block)
block_size = self.best_block_size(after - before, data_block_len)
# Apply rate limit
self.slow_down(start, byte_counter)
self.report_finish()
if data_len is not None and str(byte_counter) != data_len:
raise ValueError('Content too short: %s/%s bytes' % (byte_counter, data_len))
class InfoExtractor(object):
"""Information Extractor class.
Information extractors are the classes that, given a URL, extract
information from the video (or videos) the URL refers to. This
information includes the real video URL, the video title and simplified
title, author and others. It is returned in a list of dictionaries when
calling its extract() method. It is a list because a URL can refer to
more than one video (think of playlists). The dictionaries must include
the following fields:
id: Video identifier.
url: Final video URL.
uploader: Nickname of the video uploader.
title: Literal title.
stitle: Simplified title.
ext: Video filename extension.
Subclasses of this one should re-define the _real_initialize() and
_real_extract() methods, as well as the suitable() static method.
Probably, they should also be instantiated and added to the main
downloader.
"""
_ready = False
_downloader = None
def __init__(self, downloader=None):
"""Constructor. Receives an optional downloader."""
self._ready = False
self.set_downloader(downloader)
@staticmethod
def suitable(url):
"""Receives a URL and returns True if suitable for this IE."""
return False
def initialize(self):
"""Initializes an instance (authentication, etc)."""
if not self._ready:
self._real_initialize()
self._ready = True
def extract(self, url):
"""Extracts URL information and returns it in list of dicts."""
self.initialize()
return self._real_extract(url)
def set_downloader(self, downloader):
"""Sets the downloader for this IE."""
self._downloader = downloader
def to_stdout(self, message):
"""Print message to stdout if downloader is not in quiet mode."""
if self._downloader is None or not self._downloader.get_params().get('quiet', False):
print message
def to_stderr(self, message):
"""Print message to stderr."""
print >>sys.stderr, message
def _real_initialize(self):
"""Real initialization process. Redefine in subclasses."""
pass
def _real_extract(self, url):
"""Real extraction process. Redefine in subclasses."""
pass
class YoutubeIE(InfoExtractor):
"""Information extractor for youtube.com."""
_VALID_URL = r'^((?:http://)?(?:\w+\.)?youtube\.com/(?:(?:v/)|(?:(?:watch(?:\.php)?)?\?(?:.+&)?v=)))?([0-9A-Za-z_-]+)(?(1).+)?$'
_LANG_URL = r'http://uk.youtube.com/?hl=en&persist_hl=1&gl=US&persist_gl=1&opt_out_ackd=1'
_LOGIN_URL = 'http://www.youtube.com/signup?next=/&gl=US&hl=en'
_AGE_URL = 'http://www.youtube.com/verify_age?next_url=/&gl=US&hl=en'
_NETRC_MACHINE = 'youtube'
@staticmethod
def suitable(url):
return (re.match(YoutubeIE._VALID_URL, url) is not None)
def report_lang(self):
"""Report attempt to set language."""
self.to_stdout(u'[youtube] Setting language')
def report_login(self):
"""Report attempt to log in."""
self.to_stdout(u'[youtube] Logging in')
def report_age_confirmation(self):
"""Report attempt to confirm age."""
self.to_stdout(u'[youtube] Confirming age')
def report_webpage_download(self, video_id):
"""Report attempt to download webpage."""
self.to_stdout(u'[youtube] %s: Downloading video webpage' % video_id)
def report_information_extraction(self, video_id):
"""Report attempt to extract video information."""
self.to_stdout(u'[youtube] %s: Extracting video information' % video_id)
def report_video_url(self, video_id, video_real_url):
"""Report extracted video URL."""
self.to_stdout(u'[youtube] %s: URL: %s' % (video_id, video_real_url))
def _real_initialize(self):
if self._downloader is None:
return
username = None
password = None
downloader_params = self._downloader.get_params()
# Attempt to use provided username and password or .netrc data
if downloader_params.get('username', None) is not None:
username = downloader_params['username']
password = downloader_params['password']
elif downloader_params.get('usenetrc', False):
try:
info = netrc.netrc().authenticators(self._NETRC_MACHINE)
if info is not None:
username = info[0]
password = info[2]
else:
raise netrc.NetrcParseError('No authenticators for %s' % self._NETRC_MACHINE)
except (IOError, netrc.NetrcParseError), err:
self.to_stderr(u'WARNING: parsing .netrc: %s' % str(err))
return
# No authentication to be performed
if username is None:
return
# Set language
request = urllib2.Request(self._LOGIN_URL, None, std_headers)
try:
self.report_lang()
urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'WARNING: unable to set language: %s' % str(err))
return
# Log in
login_form = {
'current_form': 'loginForm',
'next': '/',
'action_login': 'Log In',
'username': username,
'password': password,
}
request = urllib2.Request(self._LOGIN_URL, urllib.urlencode(login_form), std_headers)
try:
self.report_login()
login_results = urllib2.urlopen(request).read()
if re.search(r'(?i)<form[^>]* name="loginForm"', login_results) is not None:
self.to_stderr(u'WARNING: unable to log in: bad username or password')
return
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'WARNING: unable to log in: %s' % str(err))
return
# Confirm age
age_form = {
'next_url': '/',
'action_confirm': 'Confirm',
}
request = urllib2.Request(self._AGE_URL, urllib.urlencode(age_form), std_headers)
try:
self.report_age_confirmation()
age_results = urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'ERROR: unable to confirm age: %s' % str(err))
return
def _real_extract(self, url):
# Extract video id from URL
mobj = re.match(self._VALID_URL, url)
if mobj is None:
self.to_stderr(u'ERROR: invalid URL: %s' % url)
return [None]
video_id = mobj.group(2)
# Downloader parameters
format_param = None
if self._downloader is not None:
params = self._downloader.get_params()
format_param = params.get('format', None)
# Extension
video_extension = {'18': 'mp4', '17': '3gp'}.get(format_param, 'flv')
# Normalize URL, including format
normalized_url = 'http://www.youtube.com/watch?v=%s&gl=US&hl=en' % video_id
if format_param is not None:
normalized_url = '%s&fmt=%s' % (normalized_url, format_param)
request = urllib2.Request(normalized_url, None, std_headers)
try:
self.report_webpage_download(video_id)
video_webpage = urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'ERROR: unable to download video webpage: %s' % str(err))
return [None]
self.report_information_extraction(video_id)
# "t" param
mobj = re.search(r', "t": "([^"]+)"', video_webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract "t" parameter')
return [None]
video_real_url = 'http://www.youtube.com/get_video?video_id=%s&t=%s' % (video_id, mobj.group(1))
if format_param is not None:
video_real_url = '%s&fmt=%s' % (video_real_url, format_param)
self.report_video_url(video_id, video_real_url)
# uploader
mobj = re.search(r"var watchUsername = '([^']+)';", video_webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract uploader nickname')
return [None]
video_uploader = mobj.group(1)
# title
mobj = re.search(r'(?im)<title>YouTube - ([^<]*)</title>', video_webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract video title')
return [None]
video_title = mobj.group(1).decode('utf-8')
video_title = re.sub(ur'(?u)&(.+?);', lambda x: unichr(htmlentitydefs.name2codepoint[x.group(1)]), video_title)
video_title = video_title.replace(os.sep, u'%')
# simplified title
simple_title = re.sub(ur'(?u)([^%s]+)' % simple_title_chars, ur'_', video_title)
simple_title = simple_title.strip(ur'_')
# Return information
return [{
'id': video_id.decode('utf-8'),
'url': video_real_url.decode('utf-8'),
'uploader': video_uploader.decode('utf-8'),
'title': video_title,
'stitle': simple_title,
'ext': video_extension.decode('utf-8'),
}]
class MetacafeIE(InfoExtractor):
"""Information Extractor for metacafe.com."""
_VALID_URL = r'(?:http://)?(?:www\.)?metacafe\.com/watch/([^/]+)/([^/]+)/.*'
_DISCLAIMER = 'http://www.metacafe.com/family_filter/'
_youtube_ie = None
def __init__(self, youtube_ie, downloader=None):
InfoExtractor.__init__(self, downloader)
self._youtube_ie = youtube_ie
@staticmethod
def suitable(url):
return (re.match(MetacafeIE._VALID_URL, url) is not None)
def report_disclaimer(self):
"""Report disclaimer retrieval."""
self.to_stdout(u'[metacafe] Retrieving disclaimer')
def report_age_confirmation(self):
"""Report attempt to confirm age."""
self.to_stdout(u'[metacafe] Confirming age')
def report_download_webpage(self, video_id):
"""Report webpage download."""
self.to_stdout(u'[metacafe] %s: Downloading webpage' % video_id)
def report_extraction(self, video_id):
"""Report information extraction."""
self.to_stdout(u'[metacafe] %s: Extracting information' % video_id)
def _real_initialize(self):
# Retrieve disclaimer
request = urllib2.Request(self._DISCLAIMER, None, std_headers)
try:
self.report_disclaimer()
disclaimer = urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'ERROR: unable to retrieve disclaimer: %s' % str(err))
return
# Confirm age
disclaimer_form = {
'filters': '0',
'submit': "Continue - I'm over 18",
}
request = urllib2.Request('http://www.metacafe.com/', urllib.urlencode(disclaimer_form), std_headers)
try:
self.report_age_confirmation()
disclaimer = urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'ERROR: unable to confirm age: %s' % str(err))
return
def _real_extract(self, url):
# Extract id and simplified title from URL
mobj = re.match(self._VALID_URL, url)
if mobj is None:
self.to_stderr(u'ERROR: invalid URL: %s' % url)
return [None]
video_id = mobj.group(1)
# Check if video comes from YouTube
mobj2 = re.match(r'^yt-(.*)$', video_id)
if mobj2 is not None:
return self._youtube_ie.extract('http://www.youtube.com/watch?v=%s' % mobj2.group(1))
simple_title = mobj.group(2).decode('utf-8')
video_extension = 'flv'
# Retrieve video webpage to extract further information
request = urllib2.Request('http://www.metacafe.com/watch/%s/' % video_id)
try:
self.report_download_webpage(video_id)
webpage = urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'ERROR: unable retrieve video webpage: %s' % str(err))
return [None]
# Extract URL, uploader and title from webpage
self.report_extraction(video_id)
mobj = re.search(r'(?m)"mediaURL":"(http.*?\.flv)"', webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract media URL')
return [None]
mediaURL = mobj.group(1).replace('\\', '')
mobj = re.search(r'(?m)"gdaKey":"(.*?)"', webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract gdaKey')
return [None]
gdaKey = mobj.group(1)
video_url = '%s?__gda__=%s' % (mediaURL, gdaKey)
mobj = re.search(r'(?im)<title>(.*) - Video</title>', webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract title')
return [None]
video_title = mobj.group(1).decode('utf-8')
mobj = re.search(r'(?m)<li id="ChnlUsr">.*?Submitter:<br />(.*?)</li>', webpage)
if mobj is None:
self.to_stderr(u'ERROR: unable to extract uploader nickname')
return [None]
video_uploader = re.sub(r'<.*?>', '', mobj.group(1))
# Return information
return [{
'id': video_id.decode('utf-8'),
'url': video_url.decode('utf-8'),
'uploader': video_uploader.decode('utf-8'),
'title': video_title,
'stitle': simple_title,
'ext': video_extension.decode('utf-8'),
}]
class YoutubePlaylistIE(InfoExtractor):
"""Information Extractor for YouTube playlists."""
_VALID_URL = r'(?:http://)?(?:\w+\.)?youtube.com/view_play_list\?p=(.+)'
_TEMPLATE_URL = 'http://www.youtube.com/view_play_list?p=%s&page=%s&gl=US&hl=en'
_VIDEO_INDICATOR = r'/watch\?v=(.+?)&'
_MORE_PAGES_INDICATOR = r'/view_play_list?p=%s&amp;page=%s'
_youtube_ie = None
def __init__(self, youtube_ie, downloader=None):
InfoExtractor.__init__(self, downloader)
self._youtube_ie = youtube_ie
@staticmethod
def suitable(url):
return (re.match(YoutubePlaylistIE._VALID_URL, url) is not None)
def report_download_page(self, playlist_id, pagenum):
"""Report attempt to download playlist page with given number."""
self.to_stdout(u'[youtube] PL %s: Downloading page #%s' % (playlist_id, pagenum))
def _real_initialize(self):
self._youtube_ie.initialize()
def _real_extract(self, url):
# Extract playlist id
mobj = re.match(self._VALID_URL, url)
if mobj is None:
self.to_stderr(u'ERROR: invalid url: %s' % url)
return [None]
# Download playlist pages
playlist_id = mobj.group(1)
video_ids = []
pagenum = 1
while True:
self.report_download_page(playlist_id, pagenum)
request = urllib2.Request(self._TEMPLATE_URL % (playlist_id, pagenum), None, std_headers)
try:
page = urllib2.urlopen(request).read()
except (urllib2.URLError, httplib.HTTPException, socket.error), err:
self.to_stderr(u'ERROR: unable to download webpage: %s' % str(err))
return [None]
# Extract video identifiers
ids_in_page = []
for mobj in re.finditer(self._VIDEO_INDICATOR, page):
if mobj.group(1) not in ids_in_page:
ids_in_page.append(mobj.group(1))
video_ids.extend(ids_in_page)
if (self._MORE_PAGES_INDICATOR % (playlist_id, pagenum + 1)) not in page:
break
pagenum = pagenum + 1
information = []
for id in video_ids:
information.extend(self._youtube_ie.extract('http://www.youtube.com/watch?v=%s' % id))
return information
class PostProcessor(object):
"""Post Processor class.
PostProcessor objects can be added to downloaders with their
add_post_processor() method. When the downloader has finished a
successful download, it will take its internal chain of PostProcessors
and start calling the run() method on each one of them, first with
an initial argument and then with the returned value of the previous
PostProcessor.
The chain will be stopped if one of them ever returns None or the end
of the chain is reached.
PostProcessor objects follow a "mutual registration" process similar
to InfoExtractor objects.
"""
_downloader = None
def __init__(self, downloader=None):
self._downloader = downloader
def to_stdout(self, message):
"""Print message to stdout if downloader is not in quiet mode."""
if self._downloader is None or not self._downloader.get_params().get('quiet', False):
print message
def to_stderr(self, message):
"""Print message to stderr."""
print >>sys.stderr, message
def set_downloader(self, downloader):
"""Sets the downloader for this PP."""
self._downloader = downloader
def run(self, information):
"""Run the PostProcessor.
The "information" argument is a dictionary like the ones
returned by InfoExtractors. The only difference is that this
one has an extra field called "filepath" that points to the
downloaded file.
When this method returns None, the postprocessing chain is
stopped. However, this method may return an information
dictionary that will be passed to the next postprocessing
object in the chain. It can be the one it received after
changing some fields.
In addition, this method may raise a PostProcessingError
exception that will be taken into account by the downloader
it was called from.
"""
return information # by default, do nothing
### MAIN PROGRAM ###
if __name__ == '__main__':
try:
# Modules needed only when running the main program
import getpass
import optparse
# General configuration
urllib2.install_opener(urllib2.build_opener(urllib2.ProxyHandler()))
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPCookieProcessor()))
socket.setdefaulttimeout(300) # 5 minutes should be enough (famous last words)
# Parse command line
parser = optparse.OptionParser(
usage='Usage: %prog [options] url...',
version='2009.01.31',
conflict_handler='resolve',
)
parser.add_option('-h', '--help',
action='help', help='print this help text and exit')
parser.add_option('-v', '--version',
action='version', help='print program version and exit')
parser.add_option('-u', '--username',
dest='username', metavar='UN', help='account username')
parser.add_option('-p', '--password',
dest='password', metavar='PW', help='account password')
parser.add_option('-o', '--output',
dest='outtmpl', metavar='TPL', help='output filename template')
parser.add_option('-q', '--quiet',
action='store_true', dest='quiet', help='activates quiet mode', default=False)
parser.add_option('-s', '--simulate',
action='store_true', dest='simulate', help='do not download video', default=False)
parser.add_option('-t', '--title',
action='store_true', dest='usetitle', help='use title in file name', default=False)
parser.add_option('-l', '--literal',
action='store_true', dest='useliteral', help='use literal title in file name', default=False)
parser.add_option('-n', '--netrc',
action='store_true', dest='usenetrc', help='use .netrc authentication data', default=False)
parser.add_option('-g', '--get-url',
action='store_true', dest='geturl', help='simulate, quiet but print URL', default=False)
parser.add_option('-e', '--get-title',
action='store_true', dest='gettitle', help='simulate, quiet but print title', default=False)
parser.add_option('-f', '--format',
dest='format', metavar='FMT', help='video format code')
parser.add_option('-b', '--best-quality',
action='store_const', dest='format', help='alias for -f 18', const='18')
parser.add_option('-m', '--mobile-version',
action='store_const', dest='format', help='alias for -f 17', const='17')
parser.add_option('-i', '--ignore-errors',
action='store_true', dest='ignoreerrors', help='continue on download errors', default=False)
parser.add_option('-r', '--rate-limit',
dest='ratelimit', metavar='L', help='download rate limit (e.g. 50k or 44.6m)')
parser.add_option('-a', '--batch-file',
dest='batchfile', metavar='F', help='file containing URLs to download')
(opts, args) = parser.parse_args()
# Batch file verification
batchurls = []
if opts.batchfile is not None:
try:
batchurls = [line.strip() for line in open(opts.batchfile, 'r')]
except IOError:
sys.exit(u'ERROR: batch file could not be read')
all_urls = batchurls + args
# Conflicting, missing and erroneous options
if len(all_urls) < 1:
sys.exit(u'ERROR: you must provide at least one URL')
if opts.usenetrc and (opts.username is not None or opts.password is not None):
sys.exit(u'ERROR: using .netrc conflicts with giving username/password')
if opts.password is not None and opts.username is None:
sys.exit(u'ERROR: account username missing')
if opts.outtmpl is not None and (opts.useliteral or opts.usetitle):
sys.exit(u'ERROR: using output template conflicts with using title or literal title')
if opts.usetitle and opts.useliteral:
sys.exit(u'ERROR: using title conflicts with using literal title')
if opts.username is not None and opts.password is None:
opts.password = getpass.getpass(u'Type account password and press return:')
if opts.ratelimit is not None:
numeric_limit = FileDownloader.parse_bytes(opts.ratelimit)
if numeric_limit is None:
sys.exit(u'ERROR: invalid rate limit specified')
opts.ratelimit = numeric_limit
# Information extractors
youtube_ie = YoutubeIE()
metacafe_ie = MetacafeIE(youtube_ie)
youtube_pl_ie = YoutubePlaylistIE(youtube_ie)
# File downloader
charset = locale.getdefaultlocale()[1]
if charset is None:
charset = 'ascii'
fd = FileDownloader({
'usenetrc': opts.usenetrc,
'username': opts.username,
'password': opts.password,
'quiet': (opts.quiet or opts.geturl or opts.gettitle),
'forceurl': opts.geturl,
'forcetitle': opts.gettitle,
'simulate': (opts.simulate or opts.geturl or opts.gettitle),
'format': opts.format,
'outtmpl': ((opts.outtmpl is not None and opts.outtmpl.decode(charset))
or (opts.usetitle and u'%(stitle)s-%(id)s.%(ext)s')
or (opts.useliteral and u'%(title)s-%(id)s.%(ext)s')
or u'%(id)s.%(ext)s'),
'ignoreerrors': opts.ignoreerrors,
'ratelimit': opts.ratelimit,
})
fd.add_info_extractor(youtube_pl_ie)
fd.add_info_extractor(metacafe_ie)
fd.add_info_extractor(youtube_ie)
retcode = fd.download(all_urls)
sys.exit(retcode)
except DownloadError:
sys.exit(1)
except SameFileError:
sys.exit(u'ERROR: fixed output name but more than one file to download')
except KeyboardInterrupt:
sys.exit(u'\nERROR: Interrupted by user')