mirror of
https://github.com/ytdl-org/youtube-dl.git
synced 2026-03-02 19:17:00 -05:00
Unable to download the YouTube channel member video #25281
Open
opened 2026-02-21 13:44:13 -05:00 by deekerman
·
14 comments
No Branch/Tag specified
master
pull/30733/head
gh-pages
df-fmt-ext-patch
dlp-fifa-backport
df-test-cleanup
pull/29816/head
download-server
totalwebcasting
rtmp_test
2024.07.11-nightly
2021.12.17
2021.06.06
2021.05.16
2021.04.26
2021.04.17
2021.04.07
2021.04.01
2021.03.31
2021.03.25
2021.03.14
2021.03.03
2021.03.02
2021.02.22
2021.02.10
2021.02.04.1
2021.02.04
2021.01.24.1
2021.01.24
2021.01.16
2021.01.08
2021.01.03
2020.12.31
2020.12.29
2020.12.26
2020.12.22
2020.12.14
2020.12.12
2020.12.09
2020.12.07
2020.12.05
2020.12.02
2020.11.29
2020.11.26
2020.11.24
2020.11.21.1
2020.11.21
2020.11.19
2020.11.18
2020.11.17
2020.11.12
2020.11.01.1
2020.11.01
2020.09.20
2020.09.14
2020.09.06
2020.07.28
2020.06.16.1
2020.06.16
2020.06.06
2020.05.29
2020.05.08
2020.05.03
2020.03.24
2020.03.08
2020.03.06
2020.03.01
2020.02.16
2020.01.24
2020.01.15
2020.01.01
2019.12.25
2019.11.28
2019.11.22
2019.11.05
2019.10.29
2019.10.22
2019.10.16
2019.09.28
2019.09.12.1
2019.09.12
2019.09.01
2019.08.13
2019.08.02
2019.07.30
2019.07.27
2019.07.16
2019.07.14
2019.07.12
2019.07.02
2019.06.27
2019.06.21
2019.06.08
2019.05.20
2019.05.11
2019.04.30
2019.04.24
2019.04.17
2019.04.07
2019.04.01
2019.03.18
2019.03.09
2019.03.01
2019.02.18
2019.02.08
2019.01.30.1
2019.01.30
2019.01.27
2019.01.24
2019.01.23
2019.01.17
2019.01.16
2019.01.10
2019.01.02
2018.12.31
2018.12.17
2018.12.09
2018.12.03
2018.11.23
2018.11.18
2018.11.07
2018.11.03
2018.10.29
2018.10.05
2018.09.26
2018.09.18
2018.09.10
2018.09.08
2018.09.01
2018.08.28
2018.08.22
2018.08.04
2018.07.29
2018.07.21
2018.07.10
2018.07.04
2018.06.25
2018.06.19
2018.06.18
2018.06.14
2018.06.11
2018.06.04
2018.06.02
2018.05.30
2018.05.26
2018.05.18
2018.05.09
2018.05.01
2018.04.25
2018.04.16
2018.04.09
2018.04.03
2018.03.26.1
2018.03.26
2018.03.20
2018.03.14
2018.03.10
2018.03.03
2018.02.26
2018.02.25
2018.02.22
2018.02.11
2018.02.08
2018.02.04
2018.02.03
2018.01.27
2018.01.21
2018.01.18
2018.01.14
2018.01.07
2017.12.31
2017.12.28
2017.12.23
2017.12.14
2017.12.10
2017.12.02
2017.11.26
2017.11.15
2017.11.06
2017.10.29
2017.10.20
2017.10.15.1
2017.10.15
2017.10.12
2017.10.07
2017.10.01
2017.09.24
2017.09.15
2017.09.11
2017.09.10
2017.09.02
2017.08.27.1
2017.08.27
2017.08.23
2017.08.18
2017.08.13
2017.08.09
2017.08.06
2017.07.30.1
2017.07.23
2017.07.15
2017.07.09
2017.07.02
2017.06.25
2017.06.23
2017.06.18
2017.06.12
2017.06.05
2017.05.29
2017.05.26
2017.05.23
2017.05.18.1
2017.05.18
2017.05.14
2017.05.09
2017.05.07
2017.05.01
2017.04.28
2017.04.26
2017.04.17
2017.04.16
2017.04.15
2017.04.14
2017.04.11
2017.04.09
2017.04.03
2017.04.02
2017.03.26
2017.03.24
2017.03.22
2017.03.20
2017.03.16
2017.03.15
2017.03.10
2017.03.07
2017.03.06
2017.03.05
2017.03.02
2017.02.28
2017.02.27
2017.02.24.1
2017.02.24
2017.02.22
2017.02.21
2017.02.17
2017.02.16
2017.02.14
2017.02.11
2017.02.10
2017.02.07
2017.02.04.1
2017.02.04
2017.02.01
2017.01.31
2017.01.29
2017.01.28
2017.01.25
2017.01.24
2017.01.22
2017.01.18
2017.01.16
2017.01.14
2017.01.10
2017.01.08
2017.01.05
2017.01.02
2016.12.31
2016.12.22
2016.12.20
2016.12.18
2016.12.15
2016.12.12
2016.12.09
2016.12.01
2016.11.27
2016.11.22
2016.11.18
2016.11.14.1
2016.11.14
2016.11.08.1
2016.11.08
2016.11.04
2016.11.02
2016.10.31
2016.10.30
2016.10.26
2016.10.25
2016.10.21.1
2016.10.21
2016.10.19
2016.10.16
2016.10.12
2016.10.07
2016.10.02
2016.09.27
2016.09.24
2016.09.19
2016.09.18
2016.09.15
2016.09.11.1
2016.09.11
2016.09.08
2016.09.04.1
2016.09.04
2016.09.03
2016.08.31
2016.08.28
2016.08.24.1
2016.08.24
2016.08.22
2016.08.19
2016.08.17
2016.08.13
2016.08.12
2016.08.10
2016.08.07
2016.08.06
2016.08.01
2016.07.30
2016.07.28
2016.07.26.2
2016.07.26.1
2016.07.26
2016.07.24
2016.07.22
2016.07.17
2016.07.16
2016.07.13
2016.07.11
2016.07.09.2
2016.07.09.1
2016.07.09
2016.07.07
2016.07.06
2016.07.05
2016.07.03.1
2016.07.03
2016.07.02
2016.07.01
2016.06.30
2016.06.27
2016.06.26
2016.06.25
2016.06.23.1
2016.06.23
2016.06.22
2016.06.20
2016.06.19.1
2016.06.19
2016.06.18.1
2016.06.18
2016.06.16
2016.06.14
2016.06.12
2016.06.11.3
2016.06.11.2
2016.06.11.1
2016.06.11
2016.06.05
2016.06.04
2016.06.03_tmp
2016.06.03
2016.06.02
2016.05.30.2
2016.05.30.1
2016.05.30
2016.05.21.2
2016.05.21.1
2016.05.21
2016.05.16
2016.05.10
2016.05.01
2016.04.24
2016.04.19
2016.04.13
2016.04.06
2016.04.05
2016.04.01
2016.03.27
2016.03.26
2016.03.25
2016.03.18
2016.03.14
2016.03.06
2016.03.01
2016.02.27
2016.02.22
2016.02.13
2016.02.10
2016.02.09.1
2016.02.09
2016.02.05.1
2016.02.05
2016.02.04
2016.02.01
2016.01.31
2016.01.29
2016.01.27
2016.01.23
2016.01.15
2016.01.14
2016.01.09
2016.01.01
2015.12.31
2015.12.29
2015.12.23
2015.12.21
2015.12.18
2015.12.13
2015.12.10
2015.12.09
2015.12.06
2015.12.05
2015.11.27.1
2015.11.27
2015.11.24
2015.11.23
2015.11.21
2015.11.19
2015.11.18
2015.11.15
2015.11.13
2015.11.10
2015.11.02
2015.11.01
2015.10.24
2015.10.23
2015.10.18
2015.10.16
2015.10.13
2015.10.12
2015.10.09
2015.10.06.2
2015.10.06.1
2015.10.06
2015.09.28
2015.09.22
2015.09.09
2015.09.03
2015.08.28
2015.08.23
2015.08.16.1
2015.08.16
2015.08.09
2015.08.06.1
2015.08.06
2015.07.28
2015.07.21
2015.07.18
2015.07.07
2015.07.04
2015.06.25
2015.06.15
2015.06.04.1
2015.06.04
2015.05.29
2015.05.20
2015.05.15
2015.05.10
2015.05.04
2015.05.03
2015.04.28
2015.04.26
2015.04.17
2015.04.09
2015.04.03
2015.03.28
2015.03.24
2015.03.18
2015.03.15
2015.03.09
2015.03.03.1
2015.03.03
2015.02.28
2015.02.26.2
2015.02.26.1
2015.02.26
2015.02.24.2
2015.02.24.1
2015.02.24
2015.02.23.1
2015.02.23
2015.02.21
2015.02.20
2015.02.19.3
2015.02.19.2
2015.02.19.1
2015.02.19
2015.02.18.1
2015.02.18
2015.02.17.2
2015.02.17.1
2015.02.17
2015.02.16.1
2015.02.16
2015.02.11
2015.02.10.5
2015.02.10.4
2015.02.10.3
2015.02.10.2
2015.02.10.1
2015.02.10
2015.02.09.3
2015.02.09.2
2015.02.09.1
2015.02.09
2015.02.08
2015.02.06
2015.02.04
2015.02.03.1
2015.02.03
2015.02.02.5
2015.02.02.4
2015.02.02.3
2015.02.02.2
2015.02.02.1
2015.02.02
2015.02.01
2015.01.30.2
2015.01.30.1
2015.01.30
2015.01.25
2015.01.23.4
2015.01.23.3
2015.01.23.2
2015.01.23.1
2015.01.23
2015.01.22
2015.01.16
2015.01.15.1
2015.01.15
2015.01.11
2015.01.10.2
2015.01.10.1
2015.01.10
2015.01.09.2
2015.01.09.1
2015.01.09
2015.01.08
2015.01.07.2
2015.01.07.1
2015.01.07
2015.01.05.1
2015.01.05
2015.01.04
2015.01.03
2015.01.02
2015.01.01
2014.12.17.2
2014.12.17.1
2014.12.17
2014.12.16.2
2014.12.16.1
2014.12.16
2014.12.15
2014.12.14
2014.12.13.1
2014.12.13
2014.12.12.7
2014.12.12.6
2014.12.12.5
2014.12.12.4
2014.12.12.3
2014.12.12.2
2014.12.12.1
2014.12.12
2014.12.11
2014.12.10.3
2014.12.10.2
2014.12.10.1
2014.12.10
2014.12.06.1
2014.12.06
2014.12.04.2
2014.12.04.1
2014.12.04
2014.12.03
2014.12.01
2014.11.27
2014.11.26.4
2014.11.26.3
2014.11.26.2
2014.11.26.1
2014.11.26
2014.11.25.1
2014.11.25
2014.11.24
2014.11.23.1
2014.11.23
2014.11.21.1
2014.11.21
2014.11.20.1
2014.11.20
2014.11.16
2014.11.15.1
2014.11.15
2014.11.14
2014.11.13.3
2014.11.13.2
2014.11.13.1
2014.11.13
2014.11.12.1
2014.11.12
2014.11.09
2014.11.04
2014.11.02.1
2014.11.02
2014.10.30
2014.10.29
2014.10.27
2014.10.26.2
2014.10.26.1
2014.10.26
2014.10.25
2014.10.24
2014.10.23
2014.10.18
2014.10.15
2014.10.13
2014.10.12
2014.10.05.2
2014.10.05.1
2014.10.05
2014.10.02
2014.09.29.2
2014.09.29.1
2014.09.29
2014.09.28.1
2014.09.28
2014.09.25
2014.09.24.1
2014.09.24
2014.09.22.1
2014.09.22
2014.09.19
2014.09.18
2014.09.16.1
2014.09.16
2014.09.15.1
2014.09.15
2014.09.14.3
2014.09.14.2
2014.09.14.1
2014.09.14
2014.09.12
2014.09.10.1
2014.09.10
2014.09.06
2014.09.04.3
2014.09.04.2
2014.09.04.1
2014.09.04
2014.09.01.2
2014.09.01.1
2014.09.01
2014.08.29
2014.08.28.2
2014.08.28.1
2014.08.28
2014.08.27.1
2014.08.27
2014.08.26
2014.08.25.3
2014.08.25.2
2014.08.25.1
2014.08.25
2014.08.24.6
2014.08.24.5
2014.08.24.4
2014.08.24.3
2014.08.24.2
2014.08.24.1
2014.08.24
2014.08.23
2014.08.22.3
2014.08.22.2
2014.08.22.1
2014.08.22
2014.08.21.3
2014.08.21.2
2014.08.21.1
2014.08.21
2014.08.10
2014.08.05
2014.08.02.1
2014.08.02
2014.07.30
2014.07.25.1
2014.07.25
2014.07.24
2014.07.23.2
2014.07.23.1
2014.07.23
2014.07.22
2014.07.21
2014.07.20.2
2014.07.20.1
2014.07.20
2014.07.15
2014.07.11.3
2014.07.11.2
2014.07.11.1
2014.07.11
2014.07.10
2014.06.26
2014.06.25
2014.06.24.1
2014.06.24
2014.06.19
2014.06.16
2014.06.09
2014.06.07
2014.06.04
2014.06.02
2014.05.31.4
2014.05.31.3
2014.05.31.2
2014.05.31.1
2014.05.31
2014.05.30.1
2014.05.30
2014.05.19
2014.05.17
2014.05.16.1
2014.05.16
2014.05.13
2014.05.12
2014.05.05
2014.04.30.1
2014.04.30
2014.04.21.6
2014.04.21.5
2014.04.21.4
2014.04.21.3
2014.04.21.2
2014.04.21.1
2014.04.21
2014.04.19
2014.04.13
2014.04.11.2
2014.04.11.1
2014.04.11
2014.04.07.4
2014.04.07.3
2014.04.07.2
2014.04.07.1
2014.04.07
2014.04.04.7
2014.04.04.6
2014.04.04.5
2014.04.04.4
2014.04.04.2
2014.04.04.3
2014.04.04.1
2014.04.04
2014.04.03.3
2014.04.03.2
2014.04.03.1
2014.04.03
2014.04.02
2014.04.01.3
2014.04.01.2
2014.04.01.1
2014.04.01
2014.03.30.1
2014.03.30
2014.03.29
2014.03.28
2014.03.27.1
2014.03.27
2014.03.25.1
2014.03.25
2014.03.24.5
2014.03.24.4
2014.03.24.3
2014.03.24.2
2014.03.24.1
2014.03.24
2013.03.24.2
2013.03.24.1
2013.03.24
2014.03.23
2014.03.21.5
2014.03.21.4
2014.03.21.3
2014.03.21.2
2014.03.21.1
2014.03.21
2014.03.20
2014.03.18.1
2014.03.18
2014.03.17
2014.03.12
2014.03.11
2014.03.10
2014.03.07.1
2014.03.07
2014.03.06
2014.03.04.2
2014.03.04.1
2014.03.04
2014.03.03
2014.02.28
2014.02.27.1
2014.02.27
2014.02.26
2014.02.25.1
2014.02.25
2014.02.24
2014.02.22.1
2014.02.22
2014.02.21.1
2014.02.21
2014.02.20
2014.02.19.1
2014.02.19
2014.02.17
2014.02.13
2014.02.10
2014.02.08.2
2014.02.08.1
2014.02.08
2014.02.06.3
2014.02.06.2
2014.02.06.1
2014.02.06
2014.02.05
2014.02.04.1
2014.02.04
2014.02.03.1
2014.02.03
2014.01.30.2
2014.01.30.1
2014.01.30
2014.01.29
2014.01.28.1
2014.01.28
2014.01.27.2
2014.01.27.1
2014.01.27
2014.01.23.4
2014.01.23.3
2014.01.23.2
2014.01.23.1
2014.01.23
2014.01.22.5
2014.01.22.4
2014.01.22.3
2014.01.22.2
2014.01.22.1
2014.01.22
2014.01.21.1
2014.01.21
2014.01.20
2014.01.17.2
2013.01.17.1
2013.01.17
2014.01.08
2014.01.07.5
2014.01.07.4
2014.01.07.3
2014.01.07.2
2014.01.07.1
2014.01.07
2014.01.06.1
2014.01.06
2014.01.05.6
2014.01.05.5
2014.01.05.4
2014.01.05.3
2014.01.05.1
2014.01.05
2014.01.03
2013.12.26
2013.12.23.4
2013.12.23.3
2013.12.23.2
2013.12.23.1
2013.12.23
2013.12.20
2013.12.17.2
2013.12.17.1
2013.12.17
2013.12.16.7
2013.12.16.6
2013.12.16.5
2013.12.16.4
2013.12.16.3
2013.12.16.2
2013.12.16.1
2013.12.16
2013.12.11.2
2013.12.11.1
2013.12.11
2013.12.10
2013.12.09.4
2013.12.09.3
2013.12.09.2
2013.12.09.1
2013.12.09
2013.12.08.1
2013.12.08
2013.12.04
2013.12.03
2013.12.02
2013.11.29
2013.11.28.1
2013.11.28
2013.11.26
2013.11.25.3
2013.11.25.2
2013.11.25.1
2013.11.25
2013.11.24.1
2013.11.24
2013.11.22.2
2013.11.22.1
2013.11.22
2013.11.21
2013.11.20
2013.11.19
2013.11.18.1
2013.11.18
2013.11.17
2013.11.15.1
2013.11.15
2013.11.13
2013.11.11
2013.11.07
2013.11.06.1
2013.11.06
2013.11.03
2013.11.02
2013.10.30
2013.10.29
2013.10.28
2013.10.23.2
2013.10.23.1
2013.10.23
2013.10.22
2013.10.18.2
2013.10.18.1
2013.10.18
2013.10.17
2013.10.15
2013.10.09
2013.10.07
2013.10.06
2013.10.04
2013.10.01.1
2013.10.01
2013.09.29
2013.09.24.2
2013.09.24.1
2013.09.24
2013.09.20.1
2013.09.20
2013.09.17
2013.09.16
2013.09.12
2013.11.09
2013.09.10
2013.09.07
2013.09.06.1
2013.09.06
2013.09.05
2013.09.04
2013.08.30
2013.08.29
2013.08.28.1
2013.08.28
2013.08.27
2013.08.23
2013.08.22
2013.08.21
2013.08.17
2013.08.15
2013.08.14
2013.08.09
2013.08.08.1
2013.08.08
2013.08.02
2013.07.31
2013.07.25.2
2013.07.25.1
2013.07.25
2013.07.24.2
2013.07.24.1
2013.07.24
2013.07.23.1
2013.07.23
2013.07.19
2013.07.18
2013.07.17.1
2013.07.17
2013.07.12
2013.07.11
2013.07.10
2013.07.08.1
2013.07.08
2013.07.07.01
2013.07.07
2013.07.05
2013.07.04
2013.07.02
2013.06.34.4
2013.06.34.3
2013.06.34.2
2013.06.34.1
2013.06.34
2013.06.33
2013.06.32
2013.06.31
2013.06.30
2013.06.29
2013.06.28
2013.06.27
2013.06.26
2013.06.25
2013.06.23
2013.06.21
2013.05.23
2013.05.14
2013.05.13
2013.05.10
2013.05.07
2013.05.06
2013.05.05
2013.05.04
2013.05.01
2013.04.31
2013.04.30
2013.04.28
2013.04.27
2013.04.22
2013.04.21
2013.04.18
2013.04.11
2013.04.03
2013.03.29
2013.02.25
2013.02.22
2012.02.22
2013.02.19
2013.02.18
2013.02.02
2013.02.01
2013.01.28
2013.01.27
2013.01.13
2013.01.12
2012.12.99
2013.01.11
2013.01.08
2013.01.06
2013.01.02
2012.12.11
2012.11.29
2012.11.28
2012.11.27
2012.11.17
2012.10.09
2012.09.27
2012.02.27
2012.02.26
2012.01.08b
2012.01.08
2012.01.05
2011.12.18
2011.12.15
2011.12.08
2011.11.23
2011.11.22
2011.11.21
2011.10.19
2011.09.30
2011.09.27
2011.09.18c
2011.09.18b
2011.09.18
2011.09.17
2011.09.16
2011.09.15
2011.09.14
2011.09.13
2011.08.04
2011.03.29
2011.02.25c
2011.02.25b
2011.02.25
2011.01.30
2010.12.09
2010.11.19
2010.10.24
2010.10.03
2010.08.04
2010.07.24
2010.07.22
2010.07.14
2010.06.06
2010.04.04
2010.04.03
2010.04.02
2010.03.13
2010.03.07
2010.02.13
2010.01.19
2010.01.06
2010.01.05
2009.12.26
2009.09.13
2009.09.08
2009.08.08
2009.06.29
2009.05.30
2009.05.25
2009.05.23
2009.05.13
2009.05.11
2009.04.25
2009.04.06
2009.03.28
2009.03.03
2009.02.07
2009.01.31
2008.11.01
2008.10.16
2008.09.20
2008.08.09
2008.07.22
Labels
Clear labels
DRM
Good first issue
account-needed
broken-IE
bug
build/update
cant-reproduce
clarification-needed
documentation
duplicate
external-bugs
fixed
geo-restricted
gh-pages
help-wanted
hls
incomplete
invalid
linux
mpd
not-a-bug
nsfw
offtopic
out-of-scope
outdated-version
patch-available
pending-fixes
php
postprocessors
question
regression
request
request
site-support-request
site-update-request
spam
subtitles
test-needed
tv-provider-account-needed
windows
won't fix
yt-dlp
No labels
DRM
Good first issue
account-needed
broken-IE
bug
build/update
cant-reproduce
clarification-needed
documentation
duplicate
external-bugs
fixed
geo-restricted
gh-pages
help-wanted
hls
incomplete
invalid
linux
mpd
not-a-bug
nsfw
offtopic
out-of-scope
outdated-version
patch-available
pending-fixes
php
postprocessors
question
regression
request
request
site-support-request
site-update-request
spam
subtitles
test-needed
tv-provider-account-needed
windows
won't fix
yt-dlp
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
starred/youtube-dl-ytdl-org#25281
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @fairfaxhshw on GitHub (May 29, 2022).
Checklist
Verbose log
Description
Trying to download the YouTube members only video, and unable to download the video.
I'm currently a member of the channel, and able to watch the YouTube video at the website.
I downloaded the most recent cookies and applied it on the script.
I'm able to download the other videos that do not require "members only" using Youtube-dl.
I was able to download the members only video about two weeks ago without a problem.
@dirkf commented on GitHub (May 29, 2022):
Your output is the same as that for non-members except for this:
So the cookie is having some effect. Apparently the cookie-driven page has a different structure such that the embedded hydration JSON (probably) isn't correctly extracted. The extractor then POSTs a query to
https://www.youtube.com/youtubei/v1/player, which gives JSON containing the logged error message.
If you use
--write-pagesand attach the resulting files, it should be possible to analyse the page and find the problem.See also #29928. Other historical issues seem to have been due to the cookie file being incorrect or not specified correctly.
@coletdjnz commented on GitHub (May 29, 2022):
this is fixed in yt-dlp by
github.com/yt-dlp/yt-dlp@ee27297f82test video: https://www.youtube.com/watch?v=tjjjtzRLHvA
@coletdjnz commented on GitHub (May 29, 2022):
yeah youtube-dl has no auth support with innertube, hence this error (was one of the early things fixed in yt-dlp). this player request itself is lacking many parameters too so it doesn't always work, so youtube-dl is reliant on extracting the data from the webpage (which is failing here).
@dirkf commented on GitHub (May 30, 2022):
Well, I could run the test video, but what are these unparseable sham JSON strings?
@dirkf commented on GitHub (May 30, 2022):
Really, that seems like a bit of a hack, unless there is a use case for
fatal=True, lenient=True. Don't we want to know when the extraction is going wrong?I have run the test video. Aha! The test video's title is the rather antagonistic
ハッシュタグ無し };if\n window.ytcsi(apparently "no hashtag};..."), which breaks the pattern used to extract the YTinitial data, as\ndoesn't match.+?withoutre.DOTALL. Also, we're looking for a block that's terminated by; var meta =whereas YT is now settingvar head =first. The fallback pattern then returns an initial substring of the JSON that crashes the parser.The initial hydration data may also contain a potentially confusing chunk of JS as the value of its
attestation.playerAttestationRendererinterpreterSafeScript.botguardData.privateDoNotAccessOrElseSafeScriptWrappedValuemember. As it's minified with fewer than 3339 variables, its variables are at most 2 characters.Finally, yt-dl has largely identical methods
YoutubeIE._extract_yt_initial_variable(),YoutubeBaseInfoExtractor._extract_yt_initial_data()that should be unified asYoutubeBaseInfoExtractor._extract_yt_initial_variable()(yt-dlp hasYoutubeBaseInfoExtractor.extract_yt_initial_data(), but it's not apparently used outside the YT extractor and the same could apply).If we strip the trailing
;from the main pattern and make this_YT_INITIAL_BOUNDARY_REthe JSON can be correctly extracted.
And the test video
tjjjtzRLHvA:@coletdjnz commented on GitHub (May 30, 2022):
@pukkandan (since you were the one that wrote it)
@jim60105 commented on GitHub (May 30, 2022):
ytarchive had a similar issue a few days ago, FYI
https://github.com/Kethsar/ytarchive/issues/93#issuecomment-1140275153
@fairfaxhshw commented on GitHub (May 30, 2022):
Alright. Just did "--write-pages" and attach the resulting files. I was unable to attach the dump files, so I have attach the compressed file
@pukkandan commented on GitHub (May 31, 2022):
Maybe
lenientis not a very good keyword. What it actually does is parse the json until an error is reached. In other words, it can parse json content embedded in a larger text (like{...}<..>)Originally, I attempted to fix this issue with just regex. But since python regex does not support recursive groups or even possessive quantifiers, it is impossible to write a foolproof regex to capture json without creating catastrophic backtracking. Eg:
r'ytInitialPlayerResponse\s*=\s*({(?:"(?:\\"|[^"])+"|[^"])+});'works, but hangs indefinitely if the regex is not found on the pageActually, it is not the first time I am encountering this issue. The same problem existed when trying to isolate
{...}code blocks forjsinterp. I had writtenJSInterpretter._seperate_at_parenfor this reason. So I could add quoting support to this (and move it to utils) to address this use-case. (Note that the regex must be changed to greedy since we can handle over-capturing, but not under-capturing)But when I thought about it more, this is what
json.loadsalready does inJSONDecoder.raw_decode. The only difference is that the stdlib raises when the unparsed section is not just whitespace. So we can just catch that error, trim the json at the point of error, and try to parse it again. This is how I ended up with the current implementation.Another solution could be to create a custom parser.
PS: Feel free to copy the code for any of these solutions (I honestly wouldn't recommend the regex though)
@dirkf commented on GitHub (May 31, 2022):
Yes, not a hack at all, or rather, an excellent hack, when you actually read the code properly. Finding the end of a JSON block with regex is clearly unviable in general so it's much better to use the parser from the json module.
I'm told that the Go JSON parser has this lenience built in, which is why the ytarchive change mentioned above also did this:
One comment:
Adding a
decoderkw (defaultjson.JSONDecoder) to_parse_json()might be good as some APIs may send XML or some other return in certain cases (error, eg) and it would be easier to handle that with a decoder class thantransform_source. It could also replacetransform_sourcethoughfatalhandling as above would be less straightforward. A function as below could be applied to make a class for atransform_sourcefunction:Or the
transform_sourcecould be tested and treated as a decoder class:@dirkf commented on GitHub (May 31, 2022):
Thanks. I checked the YT page that you dumped, and the same problem that I analysed above applies. So it should be fixed by the modified YT extractor.
@pukkandan commented on GitHub (Jun 6, 2022):
FYI, yt-dlp's implementation has been changed to use a custom decoder.
github.com/yt-dlp/yt-dlp@b7c47b7438@pukkandan commented on GitHub (Jun 6, 2022):
Why would you use _parse_json for XML? There is a different parser for it
@dirkf commented on GitHub (Jun 6, 2022):
The case I found was a JSON API at trt.com that returned JSON normally, except that when the API failed it returned XML instead.