How to Download Files from Linux Terminal Using wget and curl
If you are working on a Linux server or a desktop without a graphical user interface, you may need to download files from the internet using the command line. There are many ways to do this, but two of the most popular and versatile tools are wget and curl. In this article, you will learn how to use these commands to download files from various sources and protocols.
download file linux
Introduction
What are wget and curl?
wget and curl are command line utilities that can transfer data from or to a server using various protocols, such as HTTP, HTTPS, FTP, SFTP, SCP, etc. They can also handle cookies, authentication, redirections, proxies, and other features that are common in web browsing. Both tools are available in most Linux distributions and can be easily installed using the package manager.
Why use wget and curl to download files?
There are many reasons why you may want to use wget or curl to download files from the command line. Some of them are:
You are working on a remote server that does not have a graphical user interface or a web browser.
You want to automate the download process using scripts or cron jobs.
You want to download multiple files at once or an entire website.
You want to resume an interrupted download or check the integrity of a downloaded file.
You want to download a file that requires authentication or special headers.
You want to have more control and flexibility over the download options and parameters.
How to download a file with wget
Basic syntax of wget
The basic syntax of wget is very simple. You just need to provide the URL of the file you want to download. For example:
wget [1](
This will download the file with its original name in the current directory. You will see some information about the download progress, such as the file size, transfer speed, and estimated time of completion.
Downloading a file with a different name
If you want to save the downloaded file with a different name, you can use the -O (uppercase O) option followed by the desired filename. For example:
wget -O new_file.tar [1](
This will save the downloaded file as new_file.tar in the current directory. This is useful when you want to avoid overwriting an existing file or when you want to give a more meaningful name to the downloaded file.
Downloading multiple files at once
If you want to download more than one file at once, you can create a text file that contains the URLs of the files you want to download, one per line. Then, you can use the -i option followed by the name of the text file. For example:
wget -i download_files.txt
This will download all the files listed in <code Downloading a folder or a website
If you want to download a folder or an entire website, you can use the -r option to enable recursive downloading. This will follow all the links and subdirectories in the given URL and download them as well. For example:
download file linux terminal
download file linux command line
download file linux curl
download file linux wget
download file linux ssh
download file linux scp
download file linux ftp
download file linux http
download file linux python
download file linux bash
download file linux script
download file linux url
download file linux server
download file linux ubuntu
download file linux centos
download file linux debian
download file linux redhat
download file linux fedora
download file linux mint
download file linux suse
download file linux arch
download file linux zip
download file linux tar
download file linux gzip
download file linux bzip2
download file linux xz
download file linux rar
download file linux 7z
download file linux iso
download file linux img
download file linux pdf
download file linux txt
download file linux csv
download file linux xml
download file linux json
download file linux html
download file linux md5
download file linux sha1
download file linux sha256
download file linux sha512
download file linux gpg
download file linux pgp
download file linux asc
download file linux sig
download file linux torrent
download file linux magnet link
wget -r [2](
This will download the folder and all its contents in the current directory. You can also use the -l option to specify the maximum depth of recursion. For example, -l 2 will download only two levels of subdirectories. You can also use the -p option to download all the files that are necessary to display the web pages, such as images, stylesheets, scripts, etc. For example:
wget -r -p [3](
This will download the website and all its dependencies in the current directory. You can also use the -k option to convert the links in the downloaded files to make them suitable for local viewing. For example:
wget -r -p -k [3](
This will download the website and make it ready for offline browsing.
Resuming incomplete downloads
If your download is interrupted or stopped for some reason, you can resume it from where you left off using the -c option. This will check the size of the existing file and continue downloading from there. For example:
wget -c [1](
This will resume downloading the file if it was partially downloaded before. This is useful when you have a slow or unstable internet connection or when you want to pause and resume your downloads at your convenience.
How to download a file with curl
Basic syntax of curl
The basic syntax of curl is also very simple. You just need to provide the URL of the file you want to download. For example:
curl [1](
This will download the file and display its contents on the standard output (the terminal screen). If you want to save the downloaded file, you need to redirect the output to a file using the > operator. For example:
curl [1]( > file.tar
This will save the downloaded file as file.tar in the current directory.
Downloading a file with a different name
If you want to save the downloaded file with a different name, you can use the -o (lowercase o) option followed by the desired filename. For example:
curl -o new_file.tar [1](
This will save the downloaded file as new_file.tar in the current directory.
Downloading multiple files at once
If you want to download more than one file at once, you can provide multiple URLs as arguments to curl. For example:
curl [4]( [5]( [6](
This will download all three files and display their contents on the standard output. If you want to save them with their original names, you can use the -O (uppercase O) option. For example:
curl -O [4]( -O [5]( -O [6](
This will save all three files with their original names in the current directory. Downloading a file with authentication
If you want to download a file that requires authentication, such as a username and password, you can use the -u option followed by the credentials in the format username:password. For example:
curl -u admin:secret [7](
This will download the file using the given username and password. You can also omit the password and curl will prompt you to enter it interactively. For example:
curl -u admin [7](
This will ask you to enter the password for the user admin.
Downloading a file with progress bar
If you want to see a progress bar while downloading a file, you can use the -# option. This will display a series of hashes that indicate the percentage of completion. For example:
curl -# [1](
This will show something like this:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 100M 100 100M 0 0 5.0M 0 0:00:20 0:00:20 --:--:-- 5.1M
This is useful when you want to monitor the download progress and speed.
Conclusion
Summary of the main points
In this article, you learned how to use wget and curl to download files from Linux terminal using various options and parameters. You learned how to download files with different names, multiple files at once, folders or websites, files with authentication, and files with progress bars. You also learned how to resume incomplete downloads and check the integrity of downloaded files.
Comparison of wget and curl
Both wget and curl are powerful and versatile tools that can handle most of the common download scenarios. However, they have some differences and advantages over each other. Here are some of them:
wget
curl
Better for recursive downloading of folders or websites.
Better for downloading files from multiple protocols or sources.
Better for resuming interrupted downloads.
Better for downloading files with authentication or headers.
Better for checking the integrity of downloaded files.
Better for displaying the download progress and speed.
Easier to use for simple downloads.
More flexible and customizable for complex downloads.
Further resources and links
If you want to learn more about wget and curl, you can check out their official documentation and some useful tutorials:
: The official manual of wget.
: The official manual of curl.
: A tutorial on how to use wget with examples.
: A tutorial on how to use curl with examples.
: A comparison of wget and curl with usage examples.
: A guide on how to use wget on Linux.
: A guide on how to use curl on Linux.
I hope you enjoyed this article and learned something new. If you have any questions or feedback, please feel free to leave a comment below. Happy downloading!
FAQs
What is the difference between wget and curl?
wget and curl are both command line tools that can download files from the internet using various protocols. However, they have some differences and advantages over each other. For example, wget is better for recursive downloading of folders or websites, while curl is better for downloading files from multiple sources or with authentication. You can see a more detailed comparison in the conclusion section of this article.
How do I download a file with wget or curl in the background?
If you want to download a file with wget or curl in the background, you can use the & operator at the end of the command. This will run the command in a subshell and return you to the prompt. For example:
wget [1]( & curl [1]( > file.tar &
This will download the file in the background and let you continue working on the terminal. You can check the status of the background jobs using the jobs command.
How do I download a file with wget or curl over HTTPS?
If you want to download a file with wget or curl over HTTPS, you just need to provide the URL with the prefix. For example:
wget [15]( curl [15]( > file.tar
This will download the file over a secure connection. However, if the server has a self-signed or invalid certificate, you may encounter an error message. In that case, you can use the --no-check-certificate option for wget or the -k option for curl to skip the certificate verification. For example:
wget --no-check-certificate [15]( curl -k [15]( > file.tar
This will download the file without checking the certificate validity. However, this is not recommended as it may expose you to security risks.
How do I download a file with wget or curl from a specific IP address?
If you want to download a file with wget or curl from a specific IP address, you can use the --bind-address option for wget or the --interface option for curl. These options allow you to specify the local IP address or interface to use for the outgoing connection. For example:
wget --bind-address 192.168.1.100 [1]( curl --interface 192.168.1.100 [1]( > file.tar
This will download the file using the IP address 192.168.1.100 as the source address. This is useful when you have multiple network interfaces or IP addresses on your system and want to use a specific one for downloading.
How do I download a file with wget or curl and check its checksum?
If you want to download a file with wget or curl and check its checksum, you can use the --checksum-file option for wget. This option allows you to provide a file that contains the checksums of the files you want to download, in the format of <hash type> <hash value> <filename>. For example:
wget --checksum-file checksums.txt
This will download the files listed in checksums.txt and verify their checksums using the specified hash type and value. For example, the file checksums.txt could look like this:
md5 9a0d2f3b2c4c5fb9c53e4d34a72f0c74 file1.tar sha1 86f7e437faa5a7fce15d1ddcb9eaeaea377667b8 file2.tar sha256 4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce file3.tar
If you want to use curl to download a file and check its checksum, you can use the -o option to save the file and then use a separate command to calculate and compare the checksum. For example:
curl -o file.tar [1]( md5sum -c <(echo "9a0d2f3b2c4c5fb9c53e4d34a72f0c74 file.tar")
This will download the file and compare its MD5 checksum with the given value. You can also use other commands such as sha1sum, sha256sum, etc. to check other types of checksums. 44f88ac181