Output file size with -s or -g #5

http://imgflip.com/public_files/youtube-dl

Author

Owner

@acuity12 commented on GitHub (Nov 24, 2011):

youtube-dl was in dire need of a --get-filesize option and I needed it now, so I just wrote it. your welcome. I don't know how github works yet so maybe someone else can request this be added.

@acuity12 commented on GitHub (Nov 24, 2011): youtube-dl was in dire need of a --get-filesize option and I needed it now, so I just wrote it. your welcome. I don't know how github works yet so maybe someone else can request this be added. http://imgflip.com/public_files/youtube-dl

deekerman commented

Author

Owner

@phihag commented on GitHub (Nov 25, 2011):

@acuity The best way would be a github pull request, since that can be commented line-by-line. However, you can also provide your own git repository, or a patches. Providing a link to the changed version means I have to create the patch manually.

Note that why your implementation works in some cases, it can also output the undocumented None and any non-integer values. Additionally, it doesn't work for RTMP resources (and silently downloads these). Also, depending on the OS and its configuration, it may actually download quite a significant chunk of data and then throw it away - you're sending an HTTP GET request, read data, and then just leave it. And it creates the output file even if --get-filesize is given.

However, since --get-filesize seems to get requested over and over, I'll think about implementing it. My main issue with it is that I can't see a use case. @sivaramn mentioned ISP bandwidth limitations in #190, and that may be one.

@phihag commented on GitHub (Nov 25, 2011): @acuity The best way would be a github pull request, since that can be commented line-by-line. However, you can also provide your own git repository, or a patches. Providing a link to the changed version means I have to create the patch manually. Note that why your implementation works in some cases, it can also output the undocumented `None` and any non-integer values. Additionally, it doesn't work for RTMP resources (and silently downloads these). Also, depending on the OS and its configuration, it may actually download quite a significant chunk of data and then throw it away - you're sending an HTTP GET request, read data, and then just leave it. And it creates the output file even if `--get-filesize` is given. However, since `--get-filesize` seems to get requested over and over, I'll think about implementing it. My main issue with it is that I can't see a use case. @sivaramn mentioned ISP bandwidth limitations in #190, and that may be one.

deekerman commented

Author

Owner

@sivaramn commented on GitHub (Nov 29, 2011):

indeed, you won't believe the plans here. 10Gb at 512kbs capped and then 256 kbs beyond 10gb. And that's the 'unlimited' plan

@sivaramn commented on GitHub (Nov 29, 2011): indeed, you won't believe the plans here. 10Gb at 512kbs capped and then 256 kbs beyond 10gb. And that's the 'unlimited' plan

deekerman commented

Author

Owner

@acuity12 commented on GitHub (Dec 6, 2011):

Yea, I just did only what was necessary to make it work for my project. I just needed to set a download limit on youtube video size.

Another thing I added was a cheat to get around the youtube bandwidth limitation. After downloading about 1MB of video, it limits your bandwidth severely. So I added another option --partial that only downloads 1MB of video, then resets the connection using the partial file. This lets you utilize youtube's bandwidth, and is about 60 times faster after the first 1MB (on my server). It probably has more problems outside what I'm using it for though, so I probably shouldn't share that version.

@acuity12 commented on GitHub (Dec 6, 2011): Yea, I just did only what was necessary to make it work for my project. I just needed to set a download limit on youtube video size. Another thing I added was a cheat to get around the youtube bandwidth limitation. After downloading about 1MB of video, it limits your bandwidth severely. So I added another option --partial that only downloads 1MB of video, then resets the connection using the partial file. This lets you utilize youtube's bandwidth, and is about 60 times faster after the first 1MB (on my server). It probably has more problems outside what I'm using it for though, so I probably shouldn't share that version.

deekerman commented

Author

Owner

@Bzzz commented on GitHub (Sep 6, 2015):

@phihag Another use case for --get-filesize? Determining the size of many videos in advance, like when downloading a complete YouTube channel. Not as a single-option-there-is-your-total-download-size-solution, but with reasonable parsing effort afterwards. For single videos, you can always start the download and stop immediately. For multiple videos (read: possibly hundreds) that's not a practical way to do it. --skip-download skips the download before actually getting/printing the size.
So, why would I enjoy this feature? I have a medium speed internet connection, too fast to just let it run overnight (might fill up hard disk to the brim if there are large-ish videos that I'm not aware of), but also too slow to just get stuff on demand without annoying everybody, killing VOIP, etc. So I'd like to know how much total DL size will be, so that I can pull a reasonable amount of videos during the idle time at night - finishing before the first user starts in the morning, not accidentally filling my HDD, not wasting precious download time.

@Bzzz commented on GitHub (Sep 6, 2015): @phihag Another use case for --get-filesize? Determining the size of many videos in advance, like when downloading a complete YouTube channel. Not as a single-option-there-is-your-total-download-size-solution, but with reasonable parsing effort afterwards. For single videos, you can always start the download and stop immediately. For multiple videos (read: possibly hundreds) that's not a practical way to do it. --skip-download skips the download before actually getting/printing the size. So, why would I enjoy this feature? I have a medium speed internet connection, too fast to just let it run overnight (might fill up hard disk to the brim if there are large-ish videos that I'm not aware of), but also too slow to just get stuff on demand without annoying everybody, killing VOIP, etc. So I'd like to know how much total DL size will be, so that I can pull a reasonable amount of videos during the idle time at night - finishing before the first user starts in the morning, not accidentally filling my HDD, not wasting precious download time.

deekerman commented

Author

Owner

@BlacklightShining commented on GitHub (Dec 2, 2015):

Out of curiosity, is this a lot harder to implement than it sounds? I mean, as far as I know, the progress indicators always contain the file size, so it seems like it should be pretty easy. Meanwhile, this request's been open for literally over five years…

@BlacklightShining commented on GitHub (Dec 2, 2015): Out of curiosity, is this a lot harder to implement than it sounds? I mean, as far as I know, the progress indicators always contain the file size, so it _seems_ like it should be pretty easy. Meanwhile, this request's been open for _literally_ over five years…

deekerman commented

Author

Owner

@hrxn commented on GitHub (Dec 3, 2015):

Well, if you have a progress indicator, then yes, that is pretty obvious.

But there are other cases, like RTMP, HLS, HDS, all that streaming crap ;)

You don't have a single video file, instead there are lots of fragments. You somehow could fetch the sizes of these, I think, but yes, it is a lot harder to implement than a single, direct download ;)

@hrxn commented on GitHub (Dec 3, 2015): Well, if you have a progress indicator, then yes, that is pretty obvious. But there are other cases, like RTMP, HLS, HDS, all that streaming crap ;) You don't have a single video file, instead there are lots of fragments. You somehow could fetch the sizes of these, I think, but yes, it is a lot harder to implement than a single, direct download ;)

deekerman commented

Author

Owner

@BlacklightShining commented on GitHub (Dec 18, 2015):

@Hrxn Maybe --get-size could print just the sizes that are available (so not the ones for streams, for example), and cause youtube-dl to exit unsuccessfully if some of them weren't available?

@BlacklightShining commented on GitHub (Dec 18, 2015): @Hrxn Maybe `--get-size` could print just the sizes that are available (so not the ones for streams, for example), and cause `youtube-dl` to exit unsuccessfully if some of them weren't available?

deekerman commented

Author

Owner

@hrxn commented on GitHub (Dec 18, 2015):

Yes, that should be possible, but as someone stated earlier, I'm also not sure about a good use case..

@hrxn commented on GitHub (Dec 18, 2015): Yes, that should be possible, but as someone stated earlier, I'm also not sure about a good use case..

deekerman commented

Author

Owner

@sebma commented on GitHub (Apr 13, 2016):

Hi,

1°) I suggest to force the user to add the format along with the --get-size so that there might be only one result:
youtube-dl --get-size -f 18 http://www.youtube.com/watch?v=29S1T7RNHGM
2°) If YouTube does not provide the filesize information for the specified format, you could still get it from the Content-Length by doing a HTTP HEAD request (you don't have to download the file :) ), example with python urllib :

$ python
Python 2.7.8 (default, Jun 18 2015, 18:54:19) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib, os
>>> link = "http://ftp.lip6.fr/pub/linux/distributions/Ubuntu/releases/trusty/ubuntu-14.04.4-desktop-amd64.iso"
>>> meta = urllib.urlopen(link).info()
>>> print "Content-Length:", meta.getheader("Content-Length")
Content-Length: 1069547520
>>>

@sebma commented on GitHub (Apr 13, 2016): Hi, 1°) I suggest to force the user to add the format along with the `--get-size` so that there might be only one result: `youtube-dl --get-size -f 18 http://www.youtube.com/watch?v=29S1T7RNHGM` 2°) If YouTube does not provide the filesize information for the specified format, you could still get it from the `Content-Length` by doing a HTTP HEAD request (you don't have to download the file :) ), example with python `urllib` : ```python $ python Python 2.7.8 (default, Jun 18 2015, 18:54:19) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import urllib, os >>> link = "http://ftp.lip6.fr/pub/linux/distributions/Ubuntu/releases/trusty/ubuntu-14.04.4-desktop-amd64.iso" >>> meta = urllib.urlopen(link).info() >>> print "Content-Length:", meta.getheader("Content-Length") Content-Length: 1069547520 >>> ```

deekerman commented

Author

Owner

@sebma commented on GitHub (Apr 13, 2016):

Another example with python httplib (still without downloading the file :) ) :

$ python
Python 2.7.8 (default, Jun 18 2015, 18:54:19) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import httplib
>>> url = "http://cdimage.ubuntu.com/kubuntu/releases/14.04.4/release/kubuntu-14.04.4-desktop-amd64.iso"
>>> domain = url.split("//")[-1].split("/")[0]
>>> conn = httplib.HTTPConnection(domain)
>>> conn.request("HEAD", url)
>>> response = conn.getresponse()
>>> print "Content-Length:", response.getheader('content-length')
Content-Length: 1105133568
>>>

@sebma commented on GitHub (Apr 13, 2016): Another example with python `httplib` (still without downloading the file :) ) : ```python $ python Python 2.7.8 (default, Jun 18 2015, 18:54:19) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import httplib >>> url = "http://cdimage.ubuntu.com/kubuntu/releases/14.04.4/release/kubuntu-14.04.4-desktop-amd64.iso" >>> domain = url.split("//")[-1].split("/")[0] >>> conn = httplib.HTTPConnection(domain) >>> conn.request("HEAD", url) >>> response = conn.getresponse() >>> print "Content-Length:", response.getheader('content-length') Content-Length: 1105133568 >>> ```

deekerman commented

Author

Owner

@sebma commented on GitHub (Apr 13, 2016):

Even easier with python requests :) (Requests: HTTP for Humans):

$ python
Python 2.7.8 (default, Jun 18 2015, 18:54:19) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> url = "http://cdimage.ubuntu.com/kubuntu/releases/14.04.4/release/kubuntu-14.04.4-desktop-amd64.iso"
>>> r = requests.head(url)
>>> print r.headers['content-length']
1105133568

@sebma commented on GitHub (Apr 13, 2016): Even easier with python `requests` :) ([Requests: HTTP for Humans](http://docs.python-requests.org/en/latest/user/quickstart/)): ```python $ python Python 2.7.8 (default, Jun 18 2015, 18:54:19) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import requests >>> url = "http://cdimage.ubuntu.com/kubuntu/releases/14.04.4/release/kubuntu-14.04.4-desktop-amd64.iso" >>> r = requests.head(url) >>> print r.headers['content-length'] 1105133568 ```

deekerman commented

Author

Owner

@YosefBerger commented on GitHub (Feb 9, 2020):

If we are still looking for use cases, I often have to send videos over messaging applications (whatsapp, messenger etc.), and it would be useful to know if there is a version that is within the file size limits when using the -F flag.

@YosefBerger commented on GitHub (Feb 9, 2020): If we are still looking for use cases, I often have to send videos over messaging applications (whatsapp, messenger etc.), and it would be useful to know if there is a version that is within the file size limits when using the -F flag.

deekerman commented

Author

Owner

@Jefferderp commented on GitHub (Feb 16, 2020):

I would love a "--size" flag for estimating the final size of a download. Ideally this would act as a dry run, returning the approximate size of a download.

@Jefferderp commented on GitHub (Feb 16, 2020): I would love a "--size" flag for estimating the final size of a download. Ideally this would act as a dry run, returning the approximate size of a download.

deekerman commented

Author

Owner

@blackdimund commented on GitHub (Jul 24, 2020):

AtomicParsley 0.9.0 in FreeBSD apparently does not embed thumbnails in video sizes over 4GB. On my arch linux it has AtomicParsley 0.9.6 and it works fine, but FreeBSD hasn't updated AtomicParsley yet. Youtube doesn't give file sizes for a lot of formats like mp4_dash or webm_dash containers. If youtube-dl had a way to get file size it would help avoid embedding files over 4GB.

@blackdimund commented on GitHub (Jul 24, 2020): AtomicParsley 0.9.0 in FreeBSD apparently does not embed thumbnails in video sizes over 4GB. On my arch linux it has AtomicParsley 0.9.6 and it works fine, but FreeBSD hasn't updated AtomicParsley yet. Youtube doesn't give file sizes for a lot of formats like mp4_dash or webm_dash containers. If youtube-dl had a way to get file size it would help avoid embedding files over 4GB.

deekerman commented