How to download the entire file in a website folder

The ability to download the entire file in a website folder is really useful because you don’t need to download each file one by one. In addition, when the list on the file is long, the level of difficulty increases and you have a high chance of making mistakes, missing file, download the same file several times.

This post provides the information you need to download the entire file in a website folder.

You can use a plug-in, program, or commands that are available in the Windows command prompt or a Linux terminal to get the job done.

How to download the entire file in a website folder with wget

You can use the wget utility in the Windows 10 command prompt to download folders, subfolders and files on a website. The Wget utility supports multiple protocols such as HTTP, HTTPS, FTP and FTPS.

Go to the Windows 10 search bar, type cmd, and then click Command Prompt. Or

Simultaneously press Windows key and R key to open run, type cmd, and then press Enter.

By default, wget download the files to the current directory, if not you want, enter the folder where you want to download the files.

In the Command Prompt window, type the command below and then press Enter.

wget -rcN https://www.bible.audio

-r to tell wget that you want recursive download

-c to tell wget that you want resume a partially downloaded file

-N to tell wget that you don’t want re-retrieve files unless newer than local

If you don’t want to download ascend to the parent directory, use the command below and hit enter.

wget -rcN -np https://www.gotquestions.org

-np to tell wget that you don’t want ascend to the parent directory

Some website gives you different content depending your web browser and your operating system. In addition, some websites may detect you as a bot and block your IP address. To prevent or make this type of detection difficult, you can use your preferred user agent to make the web server feel like you are a web browser like Google Chrome, Safari, Mozilla Firefox, etc. If you want to use Google Chrome version 70.0.3538.77 as a user agent, type the command below and press Enter.

wget -rcNnp -U "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)" https://htopskills.com/media/

To use Mozilla Firefox version 77.0 as a user agent, type the command below and press Enter.

wget -rcNnp --user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:77.0) Gecko/20190101 Firefox/77.0" https://htopskills.com/media/

Note: If you want to see more things you can do with the wget utility, type the command below and hit Enter.

wget --help

Video to show you how to download the whole file to a website folder

How to download a website for offline use

To be able to download a website for offline use, you can use a free software called HTTrack that is available on multiple platforms such as Microsoft Windows, Linux, and Android. In this demonstration, I will be using the portable Windows version of HTTrack with graphical user interface.

After downloading the HTTrack software, extract it somewhere on your computer and then open the executable file. By the way, you can use HTTrack in two ways: command line with httrack.exe and WinHTTrack.exe for graphical user interface (GUI).

When you open the HTTrack for the first time, select the language in which you want to use the software, then click OK.

Click Next.

Give a name to the project (for example the name of the website), a name for the project category which is optional. The base path is usually correct, but you can change it to something else if you don’t want your file saved in the default folder.

When you are finished, click Next.

In the Web Addresses box, type the URL of the website you want to use offline, or copy the URL directly from your browser.

In preferences and mirror options area, click Set options if you want something more personalized like set the type of file you want to include, the file you want to exclude, add a proxy configuration, etc.

When you are finished, click Next and then click Finish.

Note: You can stop the operation at any time by clicking the Cancel button. And, to update the offline website, open HTTrack, go to File, and select the website.

What To Read Next

If you like the content of this post or if it has been useful to you, please consider sharing it on your social media and follow me on Facebook and Twitter for more exclusive content.