How to download files using cURL (basic & advanced techniques)

Introduction

In this tutorial, we are going to dive into using cURL to download files from the internet. cURL, short for ‘Client for URLs’, is a command line tool and library for transferring data with URLs. It is supported on a wide range of platforms and provides capabilities to download files from HTTP, HTTPS, FTP, and SCP, among others.

Before we start, ensure you have cURL installed on your system. Most Unix-based systems like Linux and macOS come with cURL pre-installed. Windows users can download cURL from the official website or use the version now included in Windows 10 and newer.

Basic Download

To download a file using cURL, the basic syntax is:

curl -O [URL]

This command tells cURL to download the file specified in the [URL]. The -O flag instructs cURL to save the file with the same name as in the URL.

Downloading to a Specific Filename

If you wish to save the file under a different name, you can use the -o option followed by the desired file name:

curl -o newfilename.zip [URL]

This will download the file and save it as newfilename.zip.

Downloading Multiple Files

cURL can also download multiple files simultaneously. To do so, use multiple -O options in the same command:

curl -O [URL1] -O [URL2]

This will download both files specified in the URLs. Make sure to use the -O option for each file you wish to download with its original name.

Using cURL with Authentication

Some files may be protected with HTTP authentication. To download such files, use the -u option:

curl -u username:password -O [URL]

Replace username and password with your actual credentials. It is important to note that this method exposes your credentials in the command line history, so use it with caution.

Downloading Files from FTP

cURL also supports downloading files from FTP servers. The syntax is quite similar:

curl -u ftpusername:ftppassword -O ftp://ftp.example.com/path/to/file.zip

This command connects to the specified FTP server with the provided username and password, then downloads the specified file.

Resuming an Interrupted Download

If a download is interrupted, you can resume it using the -C - option:

curl -C - -O [URL]

This tells cURL to resume the download from where it left off. If the server supports resuming downloads, cURL will continue downloading the remaining part of the file.

Using cURL with Proxies

If you are behind a proxy, cURL has got you covered. Use the -x option to specify the proxy address:

curl -x http://proxyserver:port -O [URL]

This will tell cURL to route the request through the proxy specified.

Limiting Download Speed

It is sometimes necessary to limit the bandwidth used for downloading files. cURL allows you to do this with the --limit-rate option:

curl --limit-rate 500k -O [URL]

This restricts the download speed to 500 kilobytes per second, helping ensure other network activities remain unaffected.

Conclusion

cURL is a powerful tool for downloading files from the internet. It offers flexibility with its wide range of options for various scenarios. By understanding and utilizing these options, you can leverage cURL to suit your specific needs. Whether you’re downloading a single file, multiple files, files requiring authentication, or from an FTP server, cURL offers a command to get the job done efficiently and securely.

We hope this guide has been helpful in getting you started with downloading files using cURL. Remember to consult the cURL man page (man curl) or the official documentation for more complex scenarios or to learn more about the options available.