mirror of
https://github.com/ytdl-org/youtube-dl.git
synced 2026-03-02 19:17:00 -05:00
Having youtube-dl Grab From Source Instead of URL #8611
Closed
opened 2026-02-21 04:27:01 -05:00 by deekerman
·
16 comments
No Branch/Tag specified
master
pull/30733/head
gh-pages
df-fmt-ext-patch
dlp-fifa-backport
df-test-cleanup
pull/29816/head
download-server
totalwebcasting
rtmp_test
2024.07.11-nightly
2021.12.17
2021.06.06
2021.05.16
2021.04.26
2021.04.17
2021.04.07
2021.04.01
2021.03.31
2021.03.25
2021.03.14
2021.03.03
2021.03.02
2021.02.22
2021.02.10
2021.02.04.1
2021.02.04
2021.01.24.1
2021.01.24
2021.01.16
2021.01.08
2021.01.03
2020.12.31
2020.12.29
2020.12.26
2020.12.22
2020.12.14
2020.12.12
2020.12.09
2020.12.07
2020.12.05
2020.12.02
2020.11.29
2020.11.26
2020.11.24
2020.11.21.1
2020.11.21
2020.11.19
2020.11.18
2020.11.17
2020.11.12
2020.11.01.1
2020.11.01
2020.09.20
2020.09.14
2020.09.06
2020.07.28
2020.06.16.1
2020.06.16
2020.06.06
2020.05.29
2020.05.08
2020.05.03
2020.03.24
2020.03.08
2020.03.06
2020.03.01
2020.02.16
2020.01.24
2020.01.15
2020.01.01
2019.12.25
2019.11.28
2019.11.22
2019.11.05
2019.10.29
2019.10.22
2019.10.16
2019.09.28
2019.09.12.1
2019.09.12
2019.09.01
2019.08.13
2019.08.02
2019.07.30
2019.07.27
2019.07.16
2019.07.14
2019.07.12
2019.07.02
2019.06.27
2019.06.21
2019.06.08
2019.05.20
2019.05.11
2019.04.30
2019.04.24
2019.04.17
2019.04.07
2019.04.01
2019.03.18
2019.03.09
2019.03.01
2019.02.18
2019.02.08
2019.01.30.1
2019.01.30
2019.01.27
2019.01.24
2019.01.23
2019.01.17
2019.01.16
2019.01.10
2019.01.02
2018.12.31
2018.12.17
2018.12.09
2018.12.03
2018.11.23
2018.11.18
2018.11.07
2018.11.03
2018.10.29
2018.10.05
2018.09.26
2018.09.18
2018.09.10
2018.09.08
2018.09.01
2018.08.28
2018.08.22
2018.08.04
2018.07.29
2018.07.21
2018.07.10
2018.07.04
2018.06.25
2018.06.19
2018.06.18
2018.06.14
2018.06.11
2018.06.04
2018.06.02
2018.05.30
2018.05.26
2018.05.18
2018.05.09
2018.05.01
2018.04.25
2018.04.16
2018.04.09
2018.04.03
2018.03.26.1
2018.03.26
2018.03.20
2018.03.14
2018.03.10
2018.03.03
2018.02.26
2018.02.25
2018.02.22
2018.02.11
2018.02.08
2018.02.04
2018.02.03
2018.01.27
2018.01.21
2018.01.18
2018.01.14
2018.01.07
2017.12.31
2017.12.28
2017.12.23
2017.12.14
2017.12.10
2017.12.02
2017.11.26
2017.11.15
2017.11.06
2017.10.29
2017.10.20
2017.10.15.1
2017.10.15
2017.10.12
2017.10.07
2017.10.01
2017.09.24
2017.09.15
2017.09.11
2017.09.10
2017.09.02
2017.08.27.1
2017.08.27
2017.08.23
2017.08.18
2017.08.13
2017.08.09
2017.08.06
2017.07.30.1
2017.07.23
2017.07.15
2017.07.09
2017.07.02
2017.06.25
2017.06.23
2017.06.18
2017.06.12
2017.06.05
2017.05.29
2017.05.26
2017.05.23
2017.05.18.1
2017.05.18
2017.05.14
2017.05.09
2017.05.07
2017.05.01
2017.04.28
2017.04.26
2017.04.17
2017.04.16
2017.04.15
2017.04.14
2017.04.11
2017.04.09
2017.04.03
2017.04.02
2017.03.26
2017.03.24
2017.03.22
2017.03.20
2017.03.16
2017.03.15
2017.03.10
2017.03.07
2017.03.06
2017.03.05
2017.03.02
2017.02.28
2017.02.27
2017.02.24.1
2017.02.24
2017.02.22
2017.02.21
2017.02.17
2017.02.16
2017.02.14
2017.02.11
2017.02.10
2017.02.07
2017.02.04.1
2017.02.04
2017.02.01
2017.01.31
2017.01.29
2017.01.28
2017.01.25
2017.01.24
2017.01.22
2017.01.18
2017.01.16
2017.01.14
2017.01.10
2017.01.08
2017.01.05
2017.01.02
2016.12.31
2016.12.22
2016.12.20
2016.12.18
2016.12.15
2016.12.12
2016.12.09
2016.12.01
2016.11.27
2016.11.22
2016.11.18
2016.11.14.1
2016.11.14
2016.11.08.1
2016.11.08
2016.11.04
2016.11.02
2016.10.31
2016.10.30
2016.10.26
2016.10.25
2016.10.21.1
2016.10.21
2016.10.19
2016.10.16
2016.10.12
2016.10.07
2016.10.02
2016.09.27
2016.09.24
2016.09.19
2016.09.18
2016.09.15
2016.09.11.1
2016.09.11
2016.09.08
2016.09.04.1
2016.09.04
2016.09.03
2016.08.31
2016.08.28
2016.08.24.1
2016.08.24
2016.08.22
2016.08.19
2016.08.17
2016.08.13
2016.08.12
2016.08.10
2016.08.07
2016.08.06
2016.08.01
2016.07.30
2016.07.28
2016.07.26.2
2016.07.26.1
2016.07.26
2016.07.24
2016.07.22
2016.07.17
2016.07.16
2016.07.13
2016.07.11
2016.07.09.2
2016.07.09.1
2016.07.09
2016.07.07
2016.07.06
2016.07.05
2016.07.03.1
2016.07.03
2016.07.02
2016.07.01
2016.06.30
2016.06.27
2016.06.26
2016.06.25
2016.06.23.1
2016.06.23
2016.06.22
2016.06.20
2016.06.19.1
2016.06.19
2016.06.18.1
2016.06.18
2016.06.16
2016.06.14
2016.06.12
2016.06.11.3
2016.06.11.2
2016.06.11.1
2016.06.11
2016.06.05
2016.06.04
2016.06.03_tmp
2016.06.03
2016.06.02
2016.05.30.2
2016.05.30.1
2016.05.30
2016.05.21.2
2016.05.21.1
2016.05.21
2016.05.16
2016.05.10
2016.05.01
2016.04.24
2016.04.19
2016.04.13
2016.04.06
2016.04.05
2016.04.01
2016.03.27
2016.03.26
2016.03.25
2016.03.18
2016.03.14
2016.03.06
2016.03.01
2016.02.27
2016.02.22
2016.02.13
2016.02.10
2016.02.09.1
2016.02.09
2016.02.05.1
2016.02.05
2016.02.04
2016.02.01
2016.01.31
2016.01.29
2016.01.27
2016.01.23
2016.01.15
2016.01.14
2016.01.09
2016.01.01
2015.12.31
2015.12.29
2015.12.23
2015.12.21
2015.12.18
2015.12.13
2015.12.10
2015.12.09
2015.12.06
2015.12.05
2015.11.27.1
2015.11.27
2015.11.24
2015.11.23
2015.11.21
2015.11.19
2015.11.18
2015.11.15
2015.11.13
2015.11.10
2015.11.02
2015.11.01
2015.10.24
2015.10.23
2015.10.18
2015.10.16
2015.10.13
2015.10.12
2015.10.09
2015.10.06.2
2015.10.06.1
2015.10.06
2015.09.28
2015.09.22
2015.09.09
2015.09.03
2015.08.28
2015.08.23
2015.08.16.1
2015.08.16
2015.08.09
2015.08.06.1
2015.08.06
2015.07.28
2015.07.21
2015.07.18
2015.07.07
2015.07.04
2015.06.25
2015.06.15
2015.06.04.1
2015.06.04
2015.05.29
2015.05.20
2015.05.15
2015.05.10
2015.05.04
2015.05.03
2015.04.28
2015.04.26
2015.04.17
2015.04.09
2015.04.03
2015.03.28
2015.03.24
2015.03.18
2015.03.15
2015.03.09
2015.03.03.1
2015.03.03
2015.02.28
2015.02.26.2
2015.02.26.1
2015.02.26
2015.02.24.2
2015.02.24.1
2015.02.24
2015.02.23.1
2015.02.23
2015.02.21
2015.02.20
2015.02.19.3
2015.02.19.2
2015.02.19.1
2015.02.19
2015.02.18.1
2015.02.18
2015.02.17.2
2015.02.17.1
2015.02.17
2015.02.16.1
2015.02.16
2015.02.11
2015.02.10.5
2015.02.10.4
2015.02.10.3
2015.02.10.2
2015.02.10.1
2015.02.10
2015.02.09.3
2015.02.09.2
2015.02.09.1
2015.02.09
2015.02.08
2015.02.06
2015.02.04
2015.02.03.1
2015.02.03
2015.02.02.5
2015.02.02.4
2015.02.02.3
2015.02.02.2
2015.02.02.1
2015.02.02
2015.02.01
2015.01.30.2
2015.01.30.1
2015.01.30
2015.01.25
2015.01.23.4
2015.01.23.3
2015.01.23.2
2015.01.23.1
2015.01.23
2015.01.22
2015.01.16
2015.01.15.1
2015.01.15
2015.01.11
2015.01.10.2
2015.01.10.1
2015.01.10
2015.01.09.2
2015.01.09.1
2015.01.09
2015.01.08
2015.01.07.2
2015.01.07.1
2015.01.07
2015.01.05.1
2015.01.05
2015.01.04
2015.01.03
2015.01.02
2015.01.01
2014.12.17.2
2014.12.17.1
2014.12.17
2014.12.16.2
2014.12.16.1
2014.12.16
2014.12.15
2014.12.14
2014.12.13.1
2014.12.13
2014.12.12.7
2014.12.12.6
2014.12.12.5
2014.12.12.4
2014.12.12.3
2014.12.12.2
2014.12.12.1
2014.12.12
2014.12.11
2014.12.10.3
2014.12.10.2
2014.12.10.1
2014.12.10
2014.12.06.1
2014.12.06
2014.12.04.2
2014.12.04.1
2014.12.04
2014.12.03
2014.12.01
2014.11.27
2014.11.26.4
2014.11.26.3
2014.11.26.2
2014.11.26.1
2014.11.26
2014.11.25.1
2014.11.25
2014.11.24
2014.11.23.1
2014.11.23
2014.11.21.1
2014.11.21
2014.11.20.1
2014.11.20
2014.11.16
2014.11.15.1
2014.11.15
2014.11.14
2014.11.13.3
2014.11.13.2
2014.11.13.1
2014.11.13
2014.11.12.1
2014.11.12
2014.11.09
2014.11.04
2014.11.02.1
2014.11.02
2014.10.30
2014.10.29
2014.10.27
2014.10.26.2
2014.10.26.1
2014.10.26
2014.10.25
2014.10.24
2014.10.23
2014.10.18
2014.10.15
2014.10.13
2014.10.12
2014.10.05.2
2014.10.05.1
2014.10.05
2014.10.02
2014.09.29.2
2014.09.29.1
2014.09.29
2014.09.28.1
2014.09.28
2014.09.25
2014.09.24.1
2014.09.24
2014.09.22.1
2014.09.22
2014.09.19
2014.09.18
2014.09.16.1
2014.09.16
2014.09.15.1
2014.09.15
2014.09.14.3
2014.09.14.2
2014.09.14.1
2014.09.14
2014.09.12
2014.09.10.1
2014.09.10
2014.09.06
2014.09.04.3
2014.09.04.2
2014.09.04.1
2014.09.04
2014.09.01.2
2014.09.01.1
2014.09.01
2014.08.29
2014.08.28.2
2014.08.28.1
2014.08.28
2014.08.27.1
2014.08.27
2014.08.26
2014.08.25.3
2014.08.25.2
2014.08.25.1
2014.08.25
2014.08.24.6
2014.08.24.5
2014.08.24.4
2014.08.24.3
2014.08.24.2
2014.08.24.1
2014.08.24
2014.08.23
2014.08.22.3
2014.08.22.2
2014.08.22.1
2014.08.22
2014.08.21.3
2014.08.21.2
2014.08.21.1
2014.08.21
2014.08.10
2014.08.05
2014.08.02.1
2014.08.02
2014.07.30
2014.07.25.1
2014.07.25
2014.07.24
2014.07.23.2
2014.07.23.1
2014.07.23
2014.07.22
2014.07.21
2014.07.20.2
2014.07.20.1
2014.07.20
2014.07.15
2014.07.11.3
2014.07.11.2
2014.07.11.1
2014.07.11
2014.07.10
2014.06.26
2014.06.25
2014.06.24.1
2014.06.24
2014.06.19
2014.06.16
2014.06.09
2014.06.07
2014.06.04
2014.06.02
2014.05.31.4
2014.05.31.3
2014.05.31.2
2014.05.31.1
2014.05.31
2014.05.30.1
2014.05.30
2014.05.19
2014.05.17
2014.05.16.1
2014.05.16
2014.05.13
2014.05.12
2014.05.05
2014.04.30.1
2014.04.30
2014.04.21.6
2014.04.21.5
2014.04.21.4
2014.04.21.3
2014.04.21.2
2014.04.21.1
2014.04.21
2014.04.19
2014.04.13
2014.04.11.2
2014.04.11.1
2014.04.11
2014.04.07.4
2014.04.07.3
2014.04.07.2
2014.04.07.1
2014.04.07
2014.04.04.7
2014.04.04.6
2014.04.04.5
2014.04.04.4
2014.04.04.2
2014.04.04.3
2014.04.04.1
2014.04.04
2014.04.03.3
2014.04.03.2
2014.04.03.1
2014.04.03
2014.04.02
2014.04.01.3
2014.04.01.2
2014.04.01.1
2014.04.01
2014.03.30.1
2014.03.30
2014.03.29
2014.03.28
2014.03.27.1
2014.03.27
2014.03.25.1
2014.03.25
2014.03.24.5
2014.03.24.4
2014.03.24.3
2014.03.24.2
2014.03.24.1
2014.03.24
2013.03.24.2
2013.03.24.1
2013.03.24
2014.03.23
2014.03.21.5
2014.03.21.4
2014.03.21.3
2014.03.21.2
2014.03.21.1
2014.03.21
2014.03.20
2014.03.18.1
2014.03.18
2014.03.17
2014.03.12
2014.03.11
2014.03.10
2014.03.07.1
2014.03.07
2014.03.06
2014.03.04.2
2014.03.04.1
2014.03.04
2014.03.03
2014.02.28
2014.02.27.1
2014.02.27
2014.02.26
2014.02.25.1
2014.02.25
2014.02.24
2014.02.22.1
2014.02.22
2014.02.21.1
2014.02.21
2014.02.20
2014.02.19.1
2014.02.19
2014.02.17
2014.02.13
2014.02.10
2014.02.08.2
2014.02.08.1
2014.02.08
2014.02.06.3
2014.02.06.2
2014.02.06.1
2014.02.06
2014.02.05
2014.02.04.1
2014.02.04
2014.02.03.1
2014.02.03
2014.01.30.2
2014.01.30.1
2014.01.30
2014.01.29
2014.01.28.1
2014.01.28
2014.01.27.2
2014.01.27.1
2014.01.27
2014.01.23.4
2014.01.23.3
2014.01.23.2
2014.01.23.1
2014.01.23
2014.01.22.5
2014.01.22.4
2014.01.22.3
2014.01.22.2
2014.01.22.1
2014.01.22
2014.01.21.1
2014.01.21
2014.01.20
2014.01.17.2
2013.01.17.1
2013.01.17
2014.01.08
2014.01.07.5
2014.01.07.4
2014.01.07.3
2014.01.07.2
2014.01.07.1
2014.01.07
2014.01.06.1
2014.01.06
2014.01.05.6
2014.01.05.5
2014.01.05.4
2014.01.05.3
2014.01.05.1
2014.01.05
2014.01.03
2013.12.26
2013.12.23.4
2013.12.23.3
2013.12.23.2
2013.12.23.1
2013.12.23
2013.12.20
2013.12.17.2
2013.12.17.1
2013.12.17
2013.12.16.7
2013.12.16.6
2013.12.16.5
2013.12.16.4
2013.12.16.3
2013.12.16.2
2013.12.16.1
2013.12.16
2013.12.11.2
2013.12.11.1
2013.12.11
2013.12.10
2013.12.09.4
2013.12.09.3
2013.12.09.2
2013.12.09.1
2013.12.09
2013.12.08.1
2013.12.08
2013.12.04
2013.12.03
2013.12.02
2013.11.29
2013.11.28.1
2013.11.28
2013.11.26
2013.11.25.3
2013.11.25.2
2013.11.25.1
2013.11.25
2013.11.24.1
2013.11.24
2013.11.22.2
2013.11.22.1
2013.11.22
2013.11.21
2013.11.20
2013.11.19
2013.11.18.1
2013.11.18
2013.11.17
2013.11.15.1
2013.11.15
2013.11.13
2013.11.11
2013.11.07
2013.11.06.1
2013.11.06
2013.11.03
2013.11.02
2013.10.30
2013.10.29
2013.10.28
2013.10.23.2
2013.10.23.1
2013.10.23
2013.10.22
2013.10.18.2
2013.10.18.1
2013.10.18
2013.10.17
2013.10.15
2013.10.09
2013.10.07
2013.10.06
2013.10.04
2013.10.01.1
2013.10.01
2013.09.29
2013.09.24.2
2013.09.24.1
2013.09.24
2013.09.20.1
2013.09.20
2013.09.17
2013.09.16
2013.09.12
2013.11.09
2013.09.10
2013.09.07
2013.09.06.1
2013.09.06
2013.09.05
2013.09.04
2013.08.30
2013.08.29
2013.08.28.1
2013.08.28
2013.08.27
2013.08.23
2013.08.22
2013.08.21
2013.08.17
2013.08.15
2013.08.14
2013.08.09
2013.08.08.1
2013.08.08
2013.08.02
2013.07.31
2013.07.25.2
2013.07.25.1
2013.07.25
2013.07.24.2
2013.07.24.1
2013.07.24
2013.07.23.1
2013.07.23
2013.07.19
2013.07.18
2013.07.17.1
2013.07.17
2013.07.12
2013.07.11
2013.07.10
2013.07.08.1
2013.07.08
2013.07.07.01
2013.07.07
2013.07.05
2013.07.04
2013.07.02
2013.06.34.4
2013.06.34.3
2013.06.34.2
2013.06.34.1
2013.06.34
2013.06.33
2013.06.32
2013.06.31
2013.06.30
2013.06.29
2013.06.28
2013.06.27
2013.06.26
2013.06.25
2013.06.23
2013.06.21
2013.05.23
2013.05.14
2013.05.13
2013.05.10
2013.05.07
2013.05.06
2013.05.05
2013.05.04
2013.05.01
2013.04.31
2013.04.30
2013.04.28
2013.04.27
2013.04.22
2013.04.21
2013.04.18
2013.04.11
2013.04.03
2013.03.29
2013.02.25
2013.02.22
2012.02.22
2013.02.19
2013.02.18
2013.02.02
2013.02.01
2013.01.28
2013.01.27
2013.01.13
2013.01.12
2012.12.99
2013.01.11
2013.01.08
2013.01.06
2013.01.02
2012.12.11
2012.11.29
2012.11.28
2012.11.27
2012.11.17
2012.10.09
2012.09.27
2012.02.27
2012.02.26
2012.01.08b
2012.01.08
2012.01.05
2011.12.18
2011.12.15
2011.12.08
2011.11.23
2011.11.22
2011.11.21
2011.10.19
2011.09.30
2011.09.27
2011.09.18c
2011.09.18b
2011.09.18
2011.09.17
2011.09.16
2011.09.15
2011.09.14
2011.09.13
2011.08.04
2011.03.29
2011.02.25c
2011.02.25b
2011.02.25
2011.01.30
2010.12.09
2010.11.19
2010.10.24
2010.10.03
2010.08.04
2010.07.24
2010.07.22
2010.07.14
2010.06.06
2010.04.04
2010.04.03
2010.04.02
2010.03.13
2010.03.07
2010.02.13
2010.01.19
2010.01.06
2010.01.05
2009.12.26
2009.09.13
2009.09.08
2009.08.08
2009.06.29
2009.05.30
2009.05.25
2009.05.23
2009.05.13
2009.05.11
2009.04.25
2009.04.06
2009.03.28
2009.03.03
2009.02.07
2009.01.31
2008.11.01
2008.10.16
2008.09.20
2008.08.09
2008.07.22
Labels
Clear labels
DRM
Good first issue
account-needed
broken-IE
bug
build/update
cant-reproduce
clarification-needed
documentation
duplicate
external-bugs
fixed
geo-restricted
gh-pages
help-wanted
hls
incomplete
invalid
linux
mpd
not-a-bug
nsfw
offtopic
out-of-scope
outdated-version
patch-available
pending-fixes
php
postprocessors
question
regression
request
request
site-support-request
site-update-request
spam
subtitles
test-needed
tv-provider-account-needed
windows
won't fix
yt-dlp
No labels
DRM
Good first issue
account-needed
broken-IE
bug
build/update
cant-reproduce
clarification-needed
documentation
duplicate
external-bugs
fixed
geo-restricted
gh-pages
help-wanted
hls
incomplete
invalid
linux
mpd
not-a-bug
nsfw
offtopic
out-of-scope
outdated-version
patch-available
pending-fixes
php
postprocessors
question
regression
request
request
site-support-request
site-update-request
spam
subtitles
test-needed
tv-provider-account-needed
windows
won't fix
yt-dlp
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
starred/youtube-dl-ytdl-org#8611
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @EnginePod on GitHub (Aug 31, 2016).
Before submitting an issue make sure you have:
What is the purpose of your issue?
Description of your issue, suggested solution and other information
This might be the second or third time that I request this feature because it would be so extremely useful to have.
There are tons of reasons to why this would be useful to have, but the main reason is that this way I could have youtube-dl extract video URLs on member restricted pages such as Facebook videos set to private.
YouTube videos are fine as I can use the username and password parameters to log in, but for other sites (like Facebook and Instagram among others) I'm shit out of luck so I have to resort to installing software to sniff the URLs.
If there was just a simple parameter like
--get-from-source C:\location\to\website_source.htmlwhich then dumped the video links then I might be free from bundled adware at the end of the day.@dstftw commented on GitHub (Aug 31, 2016):
Duplicate of #5768.
@EnginePod commented on GitHub (Aug 31, 2016):
I went through the #5768 and what @jaimeMF wrote doesn't completely make sense and I'll explain why.
Here are the reasons he posted to why this feature could not be added
Reason 1) It would unnecessarily complicate the code, we would need to modify the extractors so that they work both by giving an url and the webpage. (I gess that you are problably only interested on youtube, but we support more sites).
Answer 1) Not really, it would be extremely simple to add as it would only be a matter of adding an if-statement checking if the
--get-from-sourceparameter was set. If it is set then it doesn't download the page source, but uses the data from the parameter. Example:The main page source is only downloaded once so this if-statement would also only have to be used once. Most hosts also don't restrict the direct video link to the video so youtube-dl would have no issue working with the source as a usual request after that point.
Reason 2) What happens if we decide to use another page for getting the info? You would get error messages and we would need to explain what has changed, and this is internal behaviour so we shouldn't have to.
Answer 2) Like I explained in my previous answer, nothing would change other than adding that if-statement so no additional errors would have to be caught. Everything would be exactly as a normal request only that the user was allowed to input the source.
Reason 3) You would need to specify an extractor, which is another source of bug reports because people would use the wrong extractor and we sometimes split extractors or add a special one for handling some urls.
Answer 3) There's a point here, but it would be very easy to solve as well. The user would simply add the URL along with the parameter just like any other command. For example
youtube-dl http://web.com/video --get-from-source@dstftw commented on GitHub (Aug 31, 2016):
@EnginePod commented on GitHub (Aug 31, 2016):
You're not understanding how this would work and once again I'll explain why.
I think we can agree that currently youtube-dl usually works something like this:
User provide a URL -> Youtube-dl -> Downloads URL source
-> Downloads other assets which were LOADED THROUGH THE PAGE SOURCE
With the new page source parameter you would simply only change the "Downloads URL source" part which leaves the rest of the process untouched. It would look like this:
User provide a URL -> Youtube-dl -> page source PROVIDED THROUGH PARAMETER
-> Downloads other assets which were LOADED THROUGH THE PAGE SOURCE
Now as I said in my PREVIOUS answer, youtube-dl would in most cases get the direct link RIGHT AWAY (this is the case for Facebook and Instagram and many more as most pages have the video URL right on the page) from the page source WITHOUT having to download any additional assets or it would download a few assets first. Now like I said, the assets usually are NOT restricted by the user session as they're on static file servers which in very rare cases just check the IP.
@remitamine commented on GitHub (Aug 31, 2016):
we don't always request the url passed by the user, sometimes we use other sources to extract information, these are some examples of the extractors that do this: CBSIE, CWTVIE, CTVIE, TouTvIE, WatIE...
@EnginePod commented on GitHub (Aug 31, 2016):
@remitamine: I was expecting this to be mentioned sooner or later. My answer is that this is very, very rare and almost never happens. Take a look at the size of the sites that you mentioned for example and besides 1 extra request for these rare cases won't really cause any performance issues.
@dstftw commented on GitHub (Aug 31, 2016):
Do you even read my post? Core does not and should not be aware of which URL or set of URLs extractor needs and which actions this extractor should take to obtain them, which headers and cookies to provide and so on.
No it's not. And I did not say that.
No it's not. Even if this subset of dumb extractors that pull only the original URL without any HTTP request tweaking, without custom headers and cookies is generalized there are still extractors that does not follow this pattern and can't be generalized this way.
No, not necessarily.
How come? What if somebody wants to provide two first pages? Solution should be generic.
Fetching page source sets some cookies required for further extraction that are not available as bare file => extraction won't work.
Page source contains some expired hash, that is used in subsequent requests => extraction won't work.
etc.
Does not scale.
No it's not. Even assuming this to be true for a second, what about other extractors? Say, those that don't download source URL at all. Then the way you suggest it to be implemented does not fit them at all and this feature won't be available for them that leads to even more confusion.
Everything else may require.
So this does not worth the effort.
@yan12125 commented on GitHub (Aug 31, 2016):
@dstftw has explained things well. Here are just some interesting statistics:
@EnginePod commented on GitHub (Aug 31, 2016):
@yan12125: no reason to continue discussing this then as editing 900 files would take way too much time.
Unfortunate that this feature couldn't be added though, it would be really useful to get the download links on pages that youtube-dl can't reach due to login restrictions.
@yan12125 commented on GitHub (Aug 31, 2016):
Well, the point is not that we have lots of extractors, but your solution applies to few websites. If you want some website supporting logging in, open new issues for each one.
@EnginePod commented on GitHub (Aug 31, 2016):
This is still completely doable and the solution does actually apply 475 websites/extractors as you mentioned, but then again you'd have to add the if-statement to every single one of those extractors.
@yan12125 commented on GitHub (Aug 31, 2016):
I count extractors that downloads the web page, not extractors that only downloads the web page. I didn't find a quick way to count the latter, so I skip it. It should be much less than 475.
@EnginePod commented on GitHub (Aug 31, 2016):
@dstftw: now over to your reply:
Your post mentioned the core AFTER I wrote that, but it doesn't matter much as the solution could be applied to each extractor individually although it would be quite some amount of work.
How come? What if somebody wants to provide two first pages? Solution should be generic.The solution is generic, the only difference is that this would only work with a single URL.
If they want to download a video located on two URLs then they'd simply have to run two commands with youtube-dl.
Fetching page source sets some cookies required for further extraction that are not available as bare file => extraction won't work. Page source contains some expired hash, that is used in subsequent requests => extraction won't work. etc. Does not scale.I've explained this part twice now.
The source that has been fed through the parameter is almost always the only one that requires cookies sent through headers, the other assets don't require the login or any other cookies.
There is no reason for the page source to contain an expired hash as the source would be recently taken.
And like I said most websites don't require authentication for the direct video URL or the assets on that page while they do for the video page so there wouldn't be any issues with the cookies (try it for yourself).
`Everything else may require.
So this does not worth the effort.`
"I've explained this part twice now.
The source that has been fed through the parameter is almost always the only one that requires cookies sent through headers, the other assets don't require the login or any other cookies."
There is also no reason for there
Again, this is completely doable, the only thing in the way is that the if statement would have to be added to every single extractor that downloads the page source and like @yan12125 pointed out it's a few hundred files.
@yan12125 commented on GitHub (Sep 1, 2016):
This is the key point whether this is doable or not. Please give one example URL that you think this is doable.
Likely, it's your work to give examples to prove that your proposal works, not ours.
@EnginePod commented on GitHub (Sep 1, 2016):
It is indeed my duty to provide examples, but I didn't bother as I don't think anyone is interested in editing hundreds of files just to add this parameter (although it'd be extremely useful to have).
@dstftw commented on GitHub (Sep 1, 2016):
I pointed out that extraction process is an implementation detail that should be known only to extractor itself in the very first reply. You are just speculating with terms in order to continue stubborn arguing with facts.
Generic solution offers arbitrary control. By definition. With generic solution one should be able to control any depth of extraction process not only special case with N=1.
"almost always" and "usually" does not mean always. These are concrete extraction scenarios of concrete extractors in youtube-dl codebase.
No, you can't make such assumptions. You are also not allowed to make any assumptions on IP and way this page was obtained thus making it again impossible in case of IP-bound source page.
The point is not adding this parameter but adding support for these unsupported URLs (that you haven't even provided so far) directly, adding support for authentication for extractors that does not support it, adding support for password protected videos and so on.