I want to download a file in a shell-script using curl on my Mac: http://dumps.wikimedia.org/barwiki/latest/barwiki-latest-pages-articles.xml.bz2. We will download an file with this library and place the downloaded file to a special place. Enclosed I'll send you the code we use until now. The curleasysetopt functions return all true, but the file will not be downloaded. Also the Cocoa Application will cause an crash at the end. Is there anyone out there who can help me?
Updated 2017-08-01 to fix typo and use new demo URL. Also note that the Homebrew package manager has matured since this post was first published in 2012. Once you’ve configured Homebrew, installing a tool like wget is as simple as runningbrew install wget
wget is an incredibly useful GNU tool on Linux. Unfortunately, it doesn’t come with OS X (as of Mountain Lion). OS X includes curl, which is a very handy tool but lacks at least one important feature of wget: the ability to use wildcards to get multiple files at the same time. For example, let’s say you want to download a subset of files from an FTP server. With wget, you could type:
Here is how to mimic that process with curl and a few UNIX command-line tricks.
1. Download the directory listing and save it in a file.
2. Use grep with regular expressions to parse the .html file, extract the .igs file names and save them in a text file.
Curl File
3. Use a bash loop to iterate over the text file and fetch each file with curl.
Windows Curl Download File
Curl Download File List
Another advantage of the curl approach is that wget doesn’t support wildcard characters with HTTP URLs (only FTP).