New maintainer needed #99

Closed
opened 2026-02-20 21:02:24 -05:00 by deekerman · 16 comments
Owner

Originally created by @rg3 on GitHub (Aug 4, 2011).

Hello everyone,

As you may have noticed, I have almost stopped maintaining youtube-dl. This is due to time constraints. This year has been pretty intense professionally, I have several family issues open and I'm also trying to finish my master project (I should have done it 6 years ago when I finished my degree but I was dumb and left it pending, basically).

This forced me to get rid of a few time sinks and one of them was youtube-dl, so I simply left it there without looking at the pile of email that was accumulating and concentrated on the important stuff (sorry for not answering!). This situation will still last a few more months.

The program broke today so I was forced to release a new version, and used the fix from Philipp Hagemeister to make it work again (thanks for the code!). But, before it breaks again, I think the proper step is to give up maintainership and rely on someone with a bit more time to dedicate to the program.

If needed, I can still manage the master repository here and push out changes other people could send me. They only need to be rebased to the master branch to have clean pulls, but this would have to be coordinated with me so as not to steal more than a few minutes a week, unless the program breaks as it did today.

Thanks everyone and good luck to anyone who steps up.

It's been a good experience maintaining a semi-popular open source project and I learned a lot of things about interacting with my users and the way people work. The only advice I can give you as the old dog in the house is that I think I should have sticked to making the program work with YouTube alone, and I should really really have said no to many feature requests because the program has turned into a big monster for what it does. It should be much simpler, have much fewer options and do only one thing and do it well. Many times people request a small feature you can code in a few minutes, and you do it, and then someone asks for another feature and you think "if I said yes to the other feature, this one should be in too", and later you're buried in dozens of small features you need to maintain and do not respond to changes in other parts of the code as you had predicted. Today I would have also separated the output control from the FileDownloader, etc. The following article can be a good inspiration for maintaining open source projects.

"cat -v considered harmful" http://harmful.cat-v.org/cat-v/

Cheers.

Originally created by @rg3 on GitHub (Aug 4, 2011). Hello everyone, As you may have noticed, I have almost stopped maintaining youtube-dl. This is due to time constraints. This year has been pretty intense professionally, I have several family issues open and I'm also trying to finish my master project (I should have done it 6 years ago when I finished my degree but I was dumb and left it pending, basically). This forced me to get rid of a few time sinks and one of them was youtube-dl, so I simply left it there without looking at the pile of email that was accumulating and concentrated on the important stuff (sorry for not answering!). This situation will still last a few more months. The program broke today so I was forced to release a new version, and used the fix from Philipp Hagemeister to make it work again (thanks for the code!). But, before it breaks again, I think the proper step is to give up maintainership and rely on someone with a bit more time to dedicate to the program. If needed, I can still manage the master repository here and push out changes other people could send me. They only need to be rebased to the master branch to have clean pulls, but this would have to be coordinated with me so as not to steal more than a few minutes a week, unless the program breaks as it did today. Thanks everyone and good luck to anyone who steps up. It's been a good experience maintaining a semi-popular open source project and I learned a lot of things about interacting with my users and the way people work. The only advice I can give you as the old dog in the house is that I think I should have sticked to making the program work with YouTube alone, and I should really really have said no to many feature requests because the program has turned into a big monster for what it does. It should be much simpler, have much fewer options and do only one thing and do it well. Many times people request a small feature you can code in a few minutes, and you do it, and then someone asks for another feature and you think "if I said yes to the other feature, this one should be in too", and later you're buried in dozens of small features you need to maintain and do not respond to changes in other parts of the code as you had predicted. Today I would have also separated the output control from the FileDownloader, etc. The following article can be a good inspiration for maintaining open source projects. "cat -v considered harmful" http://harmful.cat-v.org/cat-v/ Cheers.
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

rg3 , Philipp's patch does not fix it. Most of the videos are still broken. Can you fix it this time please?

@anondavid commented on GitHub (Aug 4, 2011): rg3 , Philipp's patch does not fix it. Most of the videos are still broken. Can you fix it this time please?
Author
Owner

@rg3 commented on GitHub (Aug 4, 2011):

Can you give me a few examples that don't work, please?

@rg3 commented on GitHub (Aug 4, 2011): Can you give me a few examples that don't work, please?
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

I tried downloading the latest version so that I can give you videos, but could not for some reason (Get a 404 error on github)

http://rg3.github.com/youtube-dl/download.html

Here is one video that does not work...will send me you more shortly

http://www.youtube.com/watch?v=4XpnKHJAok8

@anondavid commented on GitHub (Aug 4, 2011): I tried downloading the latest version so that I can give you videos, but could not for some reason (Get a 404 error on github) http://rg3.github.com/youtube-dl/download.html Here is one video that does not work...will send me you more shortly http://www.youtube.com/watch?v=4XpnKHJAok8
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

oops, actually that youtube url works, let me find you some that dont work, sorry

@anondavid commented on GitHub (Aug 4, 2011): oops, actually that youtube url works, let me find you some that dont work, sorry
Author
Owner

@rg3 commented on GitHub (Aug 4, 2011):

I forgot to push the tags out and the URL was a 404. It's fixed now.

@rg3 commented on GitHub (Aug 4, 2011): I forgot to push the tags out and the URL was a 404. It's fixed now.
Author
Owner

@phihag commented on GitHub (Aug 4, 2011):

I'm probably too naive, but I do think it is possible to create complex programs with lots of features. youtube-dl is nowhere near as complex as cpython, Linux, or webkit. Since everybody needs a different set of features, having many features translates to being useful for many people. Simplicity is great, but being useful trumps it. youtube-dl should be able to convert any URL to a video file or video playlist. Nothing more, but also nothing less.

youtube-dl is missing two essential things required to grow:

  • Strict separation between UI and library. I don't think youtube-dl needs to be split in two parts, but it should have a nice API (And eventually have its name changed to youtube_dl to allow importing it ;) ).
  • Unit and integration tests. This allows it to add, alter, and remove stuff and still be sure it runs on all configurations. I've already prepared and uploaded some example videos, and writing test code is on the upper half of my youtube-dl TODO list. In the future, every feature should come with test code, and that test code better show off the elegance of the API of the implemented features.

I'd be honoured to co-maintain the official youtube-dl, but as you can read above, I'd take it into a different direction. Also, since I'm writing my Master's thesis, I won't have more than 5 hours per week until 2012.

@phihag commented on GitHub (Aug 4, 2011): I'm probably too naive, but I do think it is possible to create complex programs with lots of features. youtube-dl is nowhere near as complex as cpython, Linux, or webkit. Since everybody needs a different set of features, having many features translates to being useful for many people. Simplicity is great, but being useful trumps it. youtube-dl should be able to convert any URL to a video file or video playlist. Nothing more, but also nothing less. youtube-dl is missing two essential things required to grow: - Strict separation between UI and library. I don't think youtube-dl needs to be split in two parts, but it should have a nice API (And eventually have its name changed to `youtube_dl` to allow importing it ;) ). - Unit and integration tests. This allows it to add, alter, and remove stuff and still be sure it runs on all configurations. I've already prepared and uploaded some example videos, and writing test code is on the upper half of my youtube-dl TODO list. In the future, every feature should come with test code, and that test code better show off the elegance of the API of the implemented features. I'd be honoured to co-maintain the official youtube-dl, but as you can read above, I'd take it into a different direction. Also, since I'm writing my Master's thesis, I won't have more than 5 hours per week until 2012.
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

rg3: It seems to work. If it breaks on anything, will let you know here. Thanks.

Features Vs complexity: I agree with rg3 that adding too many features is a bad idea, even at the youtube-dl level. Unless someone is ready to keep those features up2date. Perhaps the ones who request the features need to keep them up2date and then youtube-dl can automatically pull that code, test it, then append it to the youtube-dl file at the right place for download. If the test fails, that feature is out. I like the idea of unittests from phihag, but i think youtube-dl should be a place to assimilate other people's small pieces of well maintained code...and in itself be a small piece of code that downloads youtube videos. -- just my 2 cents.

@anondavid commented on GitHub (Aug 4, 2011): rg3: It seems to work. If it breaks on anything, will let you know here. Thanks. Features Vs complexity: I agree with rg3 that adding too many features is a bad idea, even at the youtube-dl level. Unless someone is ready to keep those features up2date. Perhaps the ones who request the features need to keep them up2date and then youtube-dl can automatically pull that code, test it, then append it to the youtube-dl file at the right place for download. If the test fails, that feature is out. I like the idea of unittests from phihag, but i think youtube-dl should be a place to assimilate other people's small pieces of well maintained code...and in itself be a small piece of code that downloads youtube videos. -- just my 2 cents.
Author
Owner

@phihag commented on GitHub (Aug 4, 2011):

@anondavid Automatically pulling code is not feasible (for security reasons at the least). With a good test suite, the burden to either remove features or fix code is on the one who commits new code.

@phihag commented on GitHub (Aug 4, 2011): @anondavid Automatically pulling code is not feasible (for security reasons at the least). With a good test suite, the burden to either remove features or fix code is on the one who commits new code.
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

Can anyone first make the code pep8 compliant and document it a little better? -- If someone has time perhaps?

@anondavid commented on GitHub (Aug 4, 2011): Can anyone first make the code pep8 compliant and document it a little better? -- If someone has time perhaps?
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

Actually the documentation looks decent already. But pep8/pylint could use some work : Global evaluation : Your code has been rated at -2.59/10. :-(

@anondavid commented on GitHub (Aug 4, 2011): Actually the documentation looks decent already. But pep8/pylint could use some work : Global evaluation : Your code has been rated at -2.59/10. :-(
Author
Owner

@anondavid commented on GitHub (Aug 4, 2011):

@rg3 : I just did this "reindent youtube-dl" and then ran pylint...the new score is 7.27/10 ! Perfect! Can you try that and test it and see if you want to post that result instead of the current one.

@anondavid commented on GitHub (Aug 4, 2011): @rg3 : I just did this "reindent youtube-dl" and then ran pylint...the new score is 7.27/10 ! Perfect! Can you try that and test it and see if you want to post that result instead of the current one.
Author
Owner

@phihag commented on GitHub (Aug 5, 2011):

@anondavid youtube-dl may have some problems, but indentation is not one of them. Apart from having a better "score", which apparantly just depends on whitespace, the only effect of PEP8-ing the source is that developers have to reconfigure their editors to indent with 4 spaces instead of 1 tab.

@phihag commented on GitHub (Aug 5, 2011): @anondavid youtube-dl may have some problems, but indentation is not one of them. Apart from having a better "score", which apparantly just depends on whitespace, the only effect of PEP8-ing the source is that developers have to reconfigure their editors to indent with 4 spaces instead of 1 tab.
Author
Owner

@rg3 commented on GitHub (Aug 5, 2011):

@phihag It's naturally possible to create simple programs with lots of planned features. Things start breaking when they get a lot of unplanned features. I actually think youtube-dl needs to be internally separated a bit more to be more useful and flexible, and I agree on the tests side. I normally check against a few bookmarked videos and URLs, but I don't check absolutely everything because I don't have an official test suite. However, I'm not here to impose restrictions or any specific condition to any possible future maintainer, so do not consider anything I say more than simple advice that can be followed or not.

I can remark, however, that I think people like to be able to deploy youtube-dl by simply downloading the file and dropping it somewhere, and they do like the autoupdate feature, which works in most Unix systems if you have youtube-dl in a directory you control, like $HOME/bin. This is because, in some distributions, the package management tools do not repackage youtube-dl fast enough (others do, however), so youtube-dl is the kind of program they more or less have to or like to install manually. The version that works today may not work next week or tomorrow. Making that installation trivial is one of the keys to sucess.

Another advice I can give from experience is that you need to be careful when introducing new features and not rely on many features from modern Python versions. This is because a part of the program users use it from a rented server where they cannot fully control what's installed. Recently Red Hat released RHEL 6.1 which features Python 2.6. OpenBSD and Debian are shipping that version too, but it's worth keeping an eye on those details. Not long ago some people were still using CentOS and Red Hat 5.x, which only had Python 2.4.

@rg3 commented on GitHub (Aug 5, 2011): @phihag It's naturally possible to create simple programs with lots of planned features. Things start breaking when they get a lot of _unplanned_ features. I actually think youtube-dl needs to be internally separated a bit more to be more useful and flexible, and I agree on the tests side. I normally check against a few bookmarked videos and URLs, but I don't check absolutely everything because I don't have an official test suite. However, I'm not here to impose restrictions or any specific condition to any possible future maintainer, so do not consider anything I say more than simple advice that can be followed or not. I can remark, however, that I think people like to be able to deploy youtube-dl by simply downloading the file and dropping it somewhere, and they do like the autoupdate feature, which works in most Unix systems if you have youtube-dl in a directory you control, like $HOME/bin. This is because, in some distributions, the package management tools do not repackage youtube-dl fast enough (others do, however), so youtube-dl is the kind of program they more or less have to or like to install manually. The version that works today may not work next week or tomorrow. Making that installation trivial is one of the keys to sucess. Another advice I can give from experience is that you need to be careful when introducing new features and not rely on many features from modern Python versions. This is because a part of the program users use it from a rented server where they cannot fully control what's installed. Recently Red Hat released RHEL 6.1 which features Python 2.6. OpenBSD and Debian are shipping that version too, but it's worth keeping an eye on those details. Not long ago some people were still using CentOS and Red Hat 5.x, which only had Python 2.4.
Author
Owner

@5shekel commented on GitHub (Aug 30, 2011):

@rg3, i just want to thank you for the hard work and patience you put into this project.
yt-dl is the best out there. thanks to you. live long and prosper

@5shekel commented on GitHub (Aug 30, 2011): @rg3, i just want to thank you for the hard work and patience you put into this project. yt-dl is the best out there. thanks to you. live long and prosper
Author
Owner

@rg3 commented on GitHub (Sep 10, 2011):

phihag, as nobody else is willing to do this, I delegate on you from now on.

@rg3 commented on GitHub (Sep 10, 2011): phihag, as nobody else is willing to do this, I delegate on you from now on.
Author
Owner

@phihag commented on GitHub (Sep 10, 2011):

Great, thanks! Since my fork doesn't seem to have caused great instability, I'll push it over here soon.

@phihag commented on GitHub (Sep 10, 2011): Great, thanks! Since my fork doesn't seem to have caused great instability, I'll push it over here soon.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/youtube-dl#99
No description provided.