Handmade Hero » Forums » Code » Downloading youtube videos from win32 command line
Nioub
Joël Thieffry
3 posts
#2190 Downloading youtube videos from win32 command line
2 years, 8 months ago Edited by Joël Thieffry on Jan. 25, 2015, 9:49 a.m. Reason: OK, I'll copy-paste it

I've done a quick batch file to download 1080p youtube videos from windows command line. It is based on youtube-dl, but since youtube now uses its DASH format for 1080p, you have to download video and audio separately, then recombine them.

You need :
youtube-dl.exe from https://rg3.github.io/youtube-dl/download.html
ffmpeg.exe from http://ffmpeg.zeranoe.com/builds/
Please adapt the path to these static executables in the script.

Usage : to download "Handmade Hero Day 050 - Basic Minkowski-based Collision Detection", type
1
youtube-dl-dash.bat https://www.youtube.com/watch?v=_g8DLrNyVsQ


Now the script :
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
@REM Usage: youtube-dl-dash.bat https://www.youtube.com/watch?v=xxxxxxxxxxx
@REM Get the URL from the command line
SET YOUTUBE_URL=%1

@REM Set tools
SET YOUTUBEDL_EXE=D:\NoInstall\youtube-dl.exe
SET FFMPEG_EXE=D:\NoInstall\ffmpeg\bin\ffmpeg.exe

@REM Set DASH best quality for video and audio
SET VIDEO_Q=137
SET AUDIO_Q=141

@REM Get video and audio filename
"%YOUTUBEDL_EXE%" --get-filename -f %VIDEO_Q% "%YOUTUBE_URL%" > youtube-dl-dash-temp.txt
SET /p VIDEO_FILENAME=<youtube-dl-dash-temp.txt
"%YOUTUBEDL_EXE%" --get-filename -f %AUDIO_Q% "%YOUTUBE_URL%" > youtube-dl-dash-temp.txt
SET /p AUDIO_FILENAME=<youtube-dl-dash-temp.txt
del youtube-dl-dash-temp.txt

@REM Download video and audio files
"%YOUTUBEDL_EXE%" -f %VIDEO_Q% "%YOUTUBE_URL%"
"%YOUTUBEDL_EXE%" -f %AUDIO_Q% "%YOUTUBE_URL%"

@REM Recombine video and audio
SET FILEOUT=NEW-%VIDEO_FILENAME%
"%FFMPEG_EXE%" -i "%VIDEO_FILENAME%" -i "%AUDIO_FILENAME%" -acodec copy -vcodec copy -threads 0 "%FILEOUT%"

@REM Clean up
del "%VIDEO_FILENAME%"
del "%AUDIO_FILENAME%"
ren "%FILEOUT%" "%VIDEO_FILENAME%"
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#2197 Downloading youtube videos from win32 command line
2 years, 8 months ago

You really don't need manually combine audio and video files. youtube-dl will do that automatically if you have ffmpeg executable avaialble in PATH (or current folder). So simply running:
1
youtube-dl -f 137+141 https://www.youtube.com/watch?v=_g8DLrNyVsQ
will create one mp4 file with video and audio in it.
Nioub
Joël Thieffry
3 posts
#2200 Downloading youtube videos from win32 command line
2 years, 8 months ago

Just tested, it works very well. Excellent!

Thank you for the tip.
Miblo
Matt Mascarenhas
105 posts
2 projects

Annotator, supper gatherer and slammer of thunderous high fives

#2204 Downloading youtube videos from win32 command line
2 years, 8 months ago Edited by Matt Mascarenhas on Jan. 26, 2015, 12:15 a.m. Reason: Bug in the wget assets line

Cheers, for both of these tips, chaps. So the youtube line in my own dlhmh (zsh, although I think it's all bash-compatible) script now reads:

1
youtube-dl -i -r 800K -f 137+141 --download-archive "${VIDDIR}/.dlarchive" -o "${VIDDIR}/%(title)s-%(id)s.%(ext)s" --dateafter "$(date +%Y%m%d -d'4 days ago')" "https://www.youtube.com/user/handmadeheroarchive"


The script also downloads the latest source .zip and has a commented line ready for the assets.

1
2
wget -O "${SRCDIR}/handmade_hero_source.zip" "${HMHDIR}/${HMHSRC}"
#wget -O "${SRCDIR}/handmade_hero_assets.zip" "${HMHDIR}/${HMHASSETS}"
chizran
Matej Kac
6 posts
#2253 Downloading youtube videos from win32 command line
2 years, 8 months ago Edited by Matej Kac on Jan. 28, 2015, 9:35 a.m.

I have made a Windows only download script at the start of the series.

You can find the instructions at:

http://www.reddit.com/r/HandmadeH...hzo/handmadehero_download_script/

Currently it only supports downloading the source code. I will be adding assets downloading support later.
chizran
Matej Kac
6 posts
#2960 Downloading youtube videos from win32 command line
2 years, 6 months ago Edited by Matej Kac on March 10, 2015, 8:15 a.m. Reason: Added link to youtube-dl documentation

If anybody is interested, I have added the ability to download assets from sendowl and pre stream Q&A from Twitch to my LINQPad daily download script. As before, it can also download the current source code zip file from sendowl and the latest video uploaded to the YouTube archive.

Requirements:

LINQPad installed.

To be able to download the source code and the assets, you obviously need to preorder the game and supply your sendowl URL per the instructions (below).

For YouTube video download, you need to have both ffmpeg and youtube-dl in your PATH. youtube-dl is required for both Twitch and YouTube, ffmpeg is required only for YouTube.

Instructions:
  • •Download, install and run LINQPad.
  • •In LINQPad go to File>Open, paste link to the script and click Open.
  • •If you want to download videos you have install both ffmpeg and youtube-dl. Easiest way to get them is via chocolatey.
  • •Set your parameters and click Execute (F5)
  • •When you run the script for the first time, it will ask you for the sendowl URL. You can also set it manually via LINQPads builtin password manager (File>Password Manager) and adding password with the name 'handmadehero.sendowlurl' and value of your full sendowl URL. Passwords are securedly stored with the Windows Data Protection API (check the LINQPad FAQ)

Parameters:
  • •downloadSrc : (true|false) Download the source code zip file
  • •downloadVideo : (true|false) Download the latest YouTube file
  • •outputDir : (string) Path to the output dir (without the trailing backslash)
  • •srcCodeFileName : (string) filename of the zip file
  • •youtubeFormat : (string) youTube-dl format (-f option, 137+141 for the best quality, check youTube-dl documentation for details)
  • •downloadPreStream : (true|false) Download the pre stream Q&A
  • •downloadAssets : (true|false) Download the assets file(s)

Command-line usage

If you have installed LINQPad, you can run the scripts via commandline. You have to save the script somewhere on your system (File>Save).

1
lprun <full-path-to-the-script> <downloadSrc> <downloadVideo> <outputDir> <srcCodeFileName> <youtubeFormat> <downloadPreStream> <downloadAssets>


1
lprun "E:\Linqpad\Queries\HandMadeHero_DownloadLatest.linq" true true "d:\Download\HandMadeHero" handmadehero.zip 137+141 true true
aameen951
Ameen Sayegh
49 posts
#4282 Downloading youtube videos from win32 command line
2 years, 2 months ago

I am interesting in how youtube-dl extract the URL of a YouTube video.
I looked at the source code but it is complicated python code
but I think it is more likely inside this magic function _extract_signature_function

if anyone knows python better and can tell me how it is extracting the URL, it would be appreciated.
Or simply if I can use the tool to just extract the URL because I want to use a faster downloader and I just want to give it the link.
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#4283 Downloading youtube videos from win32 command line
2 years, 2 months ago

When I'm using youtube-dl it downloads video with my maximum Internet speed. I don't see how using other downloader would help.

But if you want to use youtube-dl to get URL of actual video file the "--get-url" argument will do that. Look at "youtube-dl --help" for more stuff - like getting title or other info.

If you want to extract URL manually, you can do that from big block of JavaScript code under <div id="player-api"> element.
gazto
Carlos Gabriel Hasbun Comandari
30 posts

|· noise ·|

#4979 Downloading youtube videos from win32 command line
1 year, 11 months ago

Thanks. It is very useful.
I love Open Source command line tools.

|· We were all born equal... except for all those things that set us apart. ·|
gazto
Carlos Gabriel Hasbun Comandari
30 posts

|· noise ·|

#5359 Downloading youtube videos from win32 command line
1 year, 9 months ago

Do you know why Youtube-dl can't download playlists? It is supposed to.

|· We were all born equal... except for all those things that set us apart. ·|
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#5360 Downloading youtube videos from win32 command line
1 year, 9 months ago

It downloads for me just fine.
Try "--print-traffic --verbose" arguments to see various debugging information, maybe it will contain some helpful information why it fails for you.
gazto
Carlos Gabriel Hasbun Comandari
30 posts

|· noise ·|

#5361 Downloading youtube videos from win32 command line
1 year, 9 months ago Edited by Carlos Gabriel Hasbun Comandari on Dec. 11, 2015, 9:27 p.m.

Yeah, it is weird. I am downloading a series (Youtube playlist)of Japanese stories and converting it to .mp3. It works with that list but not for Handmade Hero's Debug Infrastructure playlist. I'll check the verbose debug output from youtube-dl.

[Edit] I am now downloading all the Debug Infrastructure playlist as audio files, it is working properly, I guess it has some issues with the video. [/Edit]

|· We were all born equal... except for all those things that set us apart. ·|
petradragon
6 posts
#5376 Downloading youtube videos from win32 command line
1 year, 9 months ago

chizran
If anybody is interested, I have added the ability to download assets from sendowl and pre stream Q&A from Twitch to my LINQPad daily download script. As before, it can also download the current source code zip file from sendowl and the latest video uploaded to the YouTube archive.

Requirements:

LINQPad installed.

To be able to download the source code and the assets, you obviously need to preorder the game and supply your sendowl URL per the instructions (below).

For YouTube video download, you need to have both ffmpeg and youtube-dl in your PATH. youtube-dl is required for both Twitch and YouTube, ffmpeg is required only for YouTube.

Instructions:
  • •Download, install and run LINQPad.
  • •In LINQPad go to File>Open, paste link to the script and click Open.
  • •If you want to download videos you have install both ffmpeg and youtube-dl. Easiest way to get them is via chocolatey.
  • •Set your parameters and click Execute (F5)
  • •When you run the script for the first time, it will ask you for the sendowl URL. You can also set it manually via LINQPads builtin password manager (File>Password Manager) and adding password with the name 'handmadehero.sendowlurl' and value of your full sendowl URL. Passwords are securedly stored with the Windows Data Protection API (check the LINQPad FAQ)



@chizran a quick question - I just found this post - I see that you have pre stream as an option here, I wonder how you download and differentiate it exclusively from the rest of the stream - is it that for (prestream == yes) you get it from Twitch and if no then Youtube? Would you mind shedding some light on it and More importantly, do you have all the previous pre streams and can you make them available somehow? (Read - https://hero.handmadedev.org/foru...on/969-pre-stream-technical-noise)
mmozeiko
Mārtiņš Možeiko
1477 posts
1 project
#5377 Downloading youtube videos from win32 command line
1 year, 9 months ago

In his script he downloads prestream video from twitch by specifying to download 2nd, not the 1st most recent video. Youtube-dl can download specified videos in the playlist. You simply pass whole handmade hero archive as a playlist url and item index 1 to youtube-dl, and it will save pre stream video.
chizran
Matej Kac
6 posts
#5379 Downloading youtube videos from win32 command line
1 year, 9 months ago

As mmozeiko explained, downloading the prestream videos works by specifying the video from the Twitch playlist. Unfortunately, since a few episodes ago, this hasn't been working as expected. YouTube-dl downloads only one video file per broadcast from Twitch. I do have all the files archived, but the latest files are quite large, since these are whole episodes. My upload speed is not the best, but can I least try to get some of them online during the holidays.