The HTTrack interface, step by step

HTTrack copies a website to your disk so you can read it offline. The same engine ships behind three interfaces, and this page covers all three: pick yours below and the screenshots and notes follow along.

WinHTTrack Windows WebHTTrack Linux, macOS and Unix HTTrack Android

Before you start

The three versions differ in their chrome, not in what they do. Every option below means the same thing in each of them, and all three write the same profile format. Where a version genuinely lacks something, it is marked not on Android or similar.

WebHTTrack runs as a small local web server and opens in your usual browser. Nothing is uploaded anywhere; the pages you see come from your own machine. Do leave the window open while a mirror runs: the interface pings the server every 30 seconds, and the server shuts itself down once those pings stop.

The Android app needs Android 7.0 or later and is on Google Play.

Each option also lists its command-line equivalent, so anything you set here can later be scripted. The command-line guide covers that side.

First launch. HTTrack offers to take all-files access, which saves your mirrors in a public HTTrack folder other apps can open; tap OK and Android's own All files access screen opens to make the grant. Not now also works, but mirrors then stay in a private folder Android erases if you uninstall HTTrack or clear its storage.

If an older release left mirrors behind, a second prompt offers to bring them in. Tap Import to move them, or Not now.

First launch. WinHTTrack opens on its About box, which carries a Language preference dropdown at the foot. The choice is remembered, and Preferences > Language preference... changes it later. The publisher line above it is the name on the code-signing certificate, which is why it reads as it does.

1. Open HTTrack

The welcome screen has nothing to fill in. Click NextTap Next to create a project, or open one you already made.

The welcome screen.

The dropdown shows the language the interface is already in. Picking another one reloads the page in it.

On macOS, brew install httrack installs WebHTTrack alongside the command line tool. Open HTTrack.app if you have it, or run webhttrack in a terminal; either way the interface opens in your browser and behaves exactly as described here.

2. Name the project

A project is one mirror: its name becomes the folder your files land in, so give it something you will recognise in a year. The category is optional and only groups related projects together in the list.

Project name, category, and where the files go.

Base path is the folder holding all your mirrors. Keeping every project under one folder is worth doing, if only so an update finds the previous copy. To reopen or update an existing project, pick its name from the list instead of typing a new one.

The base path is fixed inside the app's storage and cannot be changed. To update an existing project, pick its name from the list instead of typing a new one.

3. Enter the addresses

Type the address you want to copy. Several addresses, one per line, are mirrored together and keep their links to each other.

The addresses to copy, and what to do with them.

Action decides what the engine does with them:

Download web site(s)-w, --mirror

The normal choice. Copies the sites with the current options.

Download web site(s) + questions-W, --mirror-wizard

The same, but asks before following anything it is unsure about. WebHTTrack cannot answer the questions and does not offer it.

Get individual files-g, --get-files

Downloads exactly the addresses you listed and follows nothing. Use it to grab a handful of files rather than a site.

Download all sites in pages-Y, --mirrorlinks

Mirrors every site linked from the pages you listed. Pointed at a bookmarks file, it copies everything you bookmarked.

Test links in pages--testlinks

Checks that the links resolve without saving anything. Useful against a bookmarks file.

Continue interrupted download--continue

Picks a crashed or cancelled mirror back up where it stopped. Offered only when the project already exists.

Update existing download--update

Re-checks the whole site and fetches only what changed. Offered only when the project already exists, and it needs the cache from the previous run, which is why Use a cache for updates should stay on.

The Add a URL button takes one address at a time. It can attach a login and password to that one address, for a page behind basic authentication.

It can also capture an address out of your browser, for a page you can only reach by submitting a form or following a script: HTTrack listens on a local port, you point your browser at it as a proxy and click the link or the submit button as usual, and the address the browser asks for is added to the project.

Adding a single address, with an optional login.

Everything else lives behind Set options, described in the option panel below. The defaults are sane; a first mirror needs nothing from there except, quite often, a scan rule.

On Android the option tabs are a list.

4. Last checks

One pane stands between you and the mirror. It can dial a connection first, hang up when the mirror ends, shut the machine down afterwards, or hold the whole thing for up to 24 hours. Leave it alone if you are already online and want to start now, and click Finish.

One page stands between you and the mirror. Start begins it; the save-only choice writes the project settings and stops, which is handy when you are preparing a mirror to run later.

Android has no such screen: Start on the address screen begins the mirror straight away.

The last screen before the mirror runs.

5. Watch it run

The engine reports what it is doing: bytes written, links scanned, transfer rate, errors so far, and one line per transfer in flight. A mirror of any size takes a while. That is the server's pace as much as yours, and the Limits options exist to keep it polite.

Android 13 and later ask whether HTTrack may post notifications. The prompt appears over the progress pane as the crawl starts. Tap Allow. Without the permission the app cannot tell you that the mirror finished while you were in another app, or offer to resume one Android stopped. HTTrack does not ask again once you answer. To change your mind later, go to its entry in Android's settings.

A crawl in flight.

You can stop at any point without losing what has been written, and a stopped mirror resumes later with Continue interrupted download. Individual transfers can be cancelled too, which is the quick way past one enormous file. Several options (the number of connections, the limits) can be changed while the mirror runs.

Tap Abort to stop. Nothing already written is lost, and the mirror resumes later with Continue interrupted download.

6. Read the result

When the crawl ends, open the copy and read it as you would the real site. Links between mirrored pages are rewritten to point at each other, so it browses offline in any browser.

Done.

Read the log before you trust a mirror. It records every error, and a site that looks complete can still be missing images that a scan rule would have caught. If something is missing, the FAQ and troubleshooting page starts with the usual causes.

The files sit under the base path you chose, in a folder named after the project. Opening index.html there browses the mirror without HTTrack.

Mirrors are written to /storage/emulated/0/HTTrack/Websites, in a folder named after the project. That is shared storage, so a file manager or a USB cable can reach it, and opening index.html there browses the mirror without the app.

The option panel

Eleven tabs, and you can ignore almost all of them. The defaults mirror a site correctly; the options are there for the sites that need persuading. Each entry below names the setting as the interface shows it, and the command-line option it corresponds to.

Scan Rules Limits Flow Control Links Build Browser ID Spider Proxy Log, Index, Cache MIME Types Experts Only

Find an option

Scan Rules

The most useful tab in the panel, and the answer to most “why is this missing?” questions. A scan rule accepts or refuses addresses by pattern: a whole directory, a domain, a file type. When a mirror comes back without its images because they live on another host, this is where you let them in.

Include link(s)+pattern

Accept addresses matching the pattern, even ones the engine would otherwise leave alone. +*.example.com/*.jpg takes the JPEGs from a neighbouring host; +*/images/landscapes/* takes one directory wherever it appears.

Exclude link(s)-pattern

Refuse addresses matching the pattern. -*.zip skips the archives, -*/forum/* skips a section you do not want.

Rule list from a file-%S, --urllist <file>

Reads rules from a text file, one per line, instead of the box.

Rules are read in order and the last match wins, so a broad exclusion followed by a narrow inclusion does what it looks like. The full pattern syntax, including size and MIME conditions, is on the filter page.

Limits

Ceilings on how much the mirror may cost you, and cost the server. The two rate limits at the bottom are the ones that keep you welcome.

Maximum mirroring depth-rN, --depth N

How many clicks from your starting addresses the engine may travel. A depth of 3 means the pages you listed, plus everything within two more clicks. Left empty it is effectively unlimited, which is usually right: the engine already refuses to leave the site.

Maximum external depth-%eN, --ext-depth N

How far to follow links off the site, past what the scan rules allow. It overrides the other limits, so raise it with care. The default is zero.

Max size of any non-HTML file-mN, --max-files N

Per-file ceiling in bytes: a larger image or archive is skipped.

Max size of any HTML file-mN,N2, --max-files

The same for pages. One command-line option carries both ceilings, non-HTML first.

Site size limit-MN, --max-size N

Total bytes the mirror may download before it stops.

Pause after downloading-GN, --max-pause N

Pauses once that many bytes have arrived and waits for you to delete a lock file. The way to mirror a site larger than your free space: back up and clear the files during the pause.

Pause between files-%G, --pause MIN[:MAX]

Waits a random number of seconds between downloads. 2:8 picks a fresh delay in that range each time. The gentlest way to crawl a small server, and less blunt than capping the rate.

Max time overall-EN, --max-time N

Seconds the whole mirror may take. 3600 is an hour.

Max transfer rate-AN, --max-rate N

Bytes per second, across the whole mirror. Keeps HTTrack from taking the whole line.

Max connections / seconds-%cN, --connection-per-second N

New connections per second, and the politest thing on this page. Fractions are allowed: 0.1 is one connection every ten seconds. Setting it to zero removes the limit and can flatten a small server; do not, unless the server is yours.

Maximum number of links-#LN, --advanced-maxlinks N

How many addresses may be held in memory, downloaded or not. The engine stops dead on reaching it, so do not set it low. The default of 100,000 suits most sites and costs memory to raise.

Flow Control

How hard to push, and when to give up on a server that is not answering.

Number of connections-cN, --sockets N

Transfers running at once. Four is the default and eight is comfortable on an ordinary site; drop to one or two when the files are large, since parallel transfers of big files help nobody.

Timeout-TN, --timeout N

Seconds to wait on a silent server before abandoning the link. It also bounds host name resolution. Around 120 suits most connections.

Retries-RN, --retries N

How many times to retry after a timeout or another non-fatal error. It will not rescue a 404: nothing retries a definite answer.

Min transfer rate-JN, --min-rate N

Bytes per second below which a transfer is treated as stalled and dropped.

Abandon host on timeout or slow transfer-HN, --host-control N

Whether one bad transfer condemns the whole host: never (0), on timeout (1), on a slow transfer (2), or either (3). The checkboxes beside the timeout and rate fields set this. Anything but never can drop a lot of links over one slow moment, so leave it off unless a host is actively wasting your time.

Links

What counts as a link worth following.

Attempt to detect all links-%P, --extended-parsing

Looks for addresses everywhere, including unknown tags and JavaScript. On by default. It earns its keep on script-heavy pages, at the price of the occasional request for something that was never a link.

Get non-HTML files related to a link-n, --near

Fetches the images and other files a page refers to even when they sit outside the mirror. The usual fix for a page whose pictures live on another host.

Test validity of all links-t, --test

Requests every link found, including ones the rules forbid, and logs the failures. A link checker rather than a mirroring option.

Get HTML files first-p7, --priority 7

Takes the pages before the images. The structure of the site is known sooner, which makes the rest of the crawl better informed.

Build

How the copy is laid out on disk.

Local structure type-NN, --structure N, --user-structure

The default reproduces the site's own folders and names. The alternatives sort files by kind instead: all the pages here, all the images there. A custom pattern such as --structure “%h%p/%n%q.%t” builds names from the host, path, name, query and type. --user-structure takes the same pattern and never a preset number.

DOS names (8+3)-L0, --long-names 0

Cuts every name to eight characters and a three-letter extension.

ISO9660 names (CDROM)-L2, --long-names 2

Names that survive being burned to a CD or DVD.

No error pages-o0, --do-not-generate-errors

By default a page that returned 404 is saved as a note saying so. This drops those, leaving nothing at all where the page was.

No external pages-x, --replace-external

Replaces links that leave the mirror with a page saying you need to be online. Keeps the offline copy from silently reaching for the network.

Hide passwords-%x, --disable-passwords

Strips credentials out of the links written into the saved pages, so the copy cannot leak them.

Hide query strings-%q0, --include-query-string 0

Local files rarely need the ?a=1&b=2 part, though it can carry a hint about what a page was. Some limited browsers choke on it.

Do not purge old files-X0, --purge-old 0

After an update HTTrack deletes local files that are gone from the site or now excluded. This keeps them.

Inline assets as data: URIs-%Z, --single-file

Rewrites each saved page with its stylesheets, scripts, images and fonts embedded, so a single file opens anywhere on its own. Links between mirrored pages stay relative; audio and video stay links. --single-file-max-size caps each embedded asset, 10 MB by default.

Browser ID

What HTTrack says about itself in its requests.

Browser identity-F, --user-agent

The User-Agent sent with every request. Some sites serve different pages, or refuse to serve at all, depending on what they read here.

HTML footer-%F, --footer

A comment added to each saved page, recording where it came from. Named fields are substituted: {url}, {date}, {addr}, {path}, {lastmodified}, {version}, {mime}, {charset}, {status} and {size}. For example <!-- Mirrored from {url} on {date} -->. You can turn it off, but a page with no record of its origin is worth less later.

Preferred language-%l, --language

The Accept-Language header, as in “fr, en, *”. It decides which translation a multilingual site hands over.

Default referer-%R, --referer

The Referer sent with the first request. Occasionally needed by sites that refuse traffic arriving from nowhere.

Spider

How the engine behaves towards the server it is talking to.

Accept cookies-bN, --cookies N

On by default. Sites that hand out a session before showing anything need it.

Check document type-uN, --check-type N

When the address does not reveal the file type, say a .cgi that returns an image, the engine asks the server so the file gets a sensible name locally. Turning this off entirely produces a mirror full of files no browser will open.

Parse scripts-jN, --parse-java N

Reads scripts looking for addresses. On by default.

Follow robots.txt-sN, --robots N

Whether to respect the site's crawling rules: never (0), sometimes (1), always (2, the default), or always including the strict rules (3). Ignoring them on a site you do not own is how mirroring gets people blocked.

Update hacks-%s, --updatehack

Works around servers that misreport what changed, treating a file of identical size as unchanged even when the timestamp moved. Saves a great deal of traffic on dynamic sites, at a small risk of keeping a stale page.

Tolerant requests-%B, --tolerant

Accepts responses that break the rules. Off by default, because a server that lies about a file's length usually leaves you with a truncated one.

Force HTTP/1.0 requests-%h, --http-10

For servers and proxies too old to cope with anything newer. It costs you most of the update machinery, so use it only when a site genuinely fails without it.

URL hacks-%u, --urlhack

Treats addresses that differ only cosmetically as the same page (www.example.com and example.com, doubled slashes, reordered query keys), so the same page is not downloaded twice.

Three checkboxes opt out of one part each when a site really does treat the difference as meaningful: Keep the www. prefix (--keep-www-prefix), Keep double slashes (--keep-double-slashes) and Keep the original query-string order (--keep-query-order).

Seed the crawl from the site's sitemap-%m, --sitemap

Reads the site's sitemap, the Sitemap: lines in robots.txt and then /sitemap.xml, and adds every address it lists as a starting point. It reaches pages no link on the site points to. Sitemap address (--sitemap-url) names one directly instead of probing.

Load cookies from file-%K, --cookies-file

Preloads cookies from a Netscape cookies.txt before the crawl starts. How you mirror something that needs you logged in, by exporting the session from your browser.

Strip query keys-%g, --strip-query

Comma-separated query keys to drop when naming saved files, such as sid,utm_source. Keeps one page from being saved several times under tracking parameters that change nothing. You can scope a rule to the addresses it covers, for example shop.example.com/*=sid.

The box takes one rule per line. Only the last matching rule applies, so keep the keys for one scope together on a line rather than on lines of their own.

Host aliases-%C, --host-alias

Names the other hostnames one site answers to, one rule per line, written www2.example.com,m.example.com=example.com. They fold onto that host: their links travel under the same rules and each page is saved once instead of once per name. Either side may carry a scheme, as in legacy.example.com=https://example.com. A trailing slash is ignored, but a path is refused: this maps hosts, not addresses.

Proxy

Proxy type-P scheme://…

The proxy protocol. HTTP is an ordinary proxy. HTTP (CONNECT tunnel) sends every request through a CONNECT tunnel, which CONNECT-only proxies such as Tor's HTTPTunnelPort require. SOCKS5 defaults to port 1080.

Proxy address and port-P, --proxy

The proxy to send requests through. On the command line one string carries the lot, including a scheme and credentials where needed: --proxy socks5://user:pass@proxy.example.com:1080.

Proxy login and password-P user:pass@host:port

Filled in behind the Configure button, for a proxy that demands authentication.

not on Android The Android app takes a host and a port only.

Use proxy for FTP transfers-%f, --httpproxy-ftp

Sends ftp:// transfers through the HTTP proxy as well. On by default, and worth keeping: it gets FTP links through a firewall, and proxied FTP tends to be more reliable than the engine's own client.

Log, Index, Cache

Store all files in cache-k, --store-all-in-cache

Normally only pages are cached, since that is all an update needs. This caches everything, which lets you rebuild the mirror in a different layout later without downloading it again, and makes the cache as large as the mirror.

Do not re-download locally erased files-%n, --do-not-recatch

A file you deleted stays deleted through the next update, instead of coming straight back. Useful while pruning something large by hand.

Create log files-f, --file-log

On, and worth leaving on: without a log you have no way to find out what failed. The debug level beside it controls how much detail lands there.

Make an index-I, --index

Writes an index.html at the top of the project, listing the mirrored sites. The front door to the copy.

Make a word database-%I, --search-index

Builds a searchable word index of every mirrored page.

Write a WARC archive of the crawl-%r, --warc

Saves every fetched response into an ISO-28500 WARC/1.1 archive alongside the mirror, headers and all. The mirror is a browsable copy; the WARC is the archival record of what the server actually sent, which is what replay tools and web archives want.

WARC archive name (--warc-file) sets the base name, blank to auto-name it under the output directory. WARC segment size limit (--warc-max-size) starts a new segment past that many bytes; blank or 0 keeps one file.

Write a CDXJ index of the WARC archive-%rc, --warc-cdx

Adds a sorted CDXJ index listing every record in the archive, which replay tools use to find a capture without reading the whole file.

Package the crawl as a WACZ file--wacz

Bundles the WARC, its CDXJ index and the mirrored pages into one WACZ file. It implies both of the two above.

Report what changed since the previous mirror-%d, --changes

Writes hts-changes.json listing what this crawl left new, changed, unchanged and gone against the previous mirror. The quick way to see what an update actually did.

MIME Types

Type / MIME associations-%A, --assume

Tells the engine what an extension always means, so it stops asking the server. On a site full of .asp links this is a large speed-up: without it every unknown extension costs a request before the engine knows what it has.

Enter a file type and the MIME type it maps to, asp to text/html, and list several at once with commas: asp,php,php3. On the command line the same thing reads --assume asp,php,php3=text/html, and --assume standard covers the usual page extensions in one go.

It also renames on the way in. If you know the site's .dat files are really ZIPs, map dat to application/x-zip and they land correctly named. Mapping an unknown type to application/octet-stream simply stops the engine checking it.

Android The table holds up to eight entries.

Experts Only

The defaults here are right for almost every mirror. The travel modes are the exception: they are how you widen a crawl deliberately.

Use a cache for updates-CN, --cache N

Keep this on. The cache is what makes Update existing download and Continue interrupted download possible at all; turning it off saves a little disk and costs you both.

Primary scan rule-pN, --priority N

What gets saved: nothing (0, for checking links), pages only (1), everything but pages (2), everything (3, the default), or pages first then the rest (7).

Travel mode-S -D -U -B

Which directories the crawl may enter relative to your starting address: this one only (-S), it and below (-D, the default), it and above (-U), or both directions (-B).

Global travel mode-a -d -l -e

How far the crawl may stray from the address you gave it: same address only (-a, the default), same domain (-d), same top-level domain (-l), or anywhere at all (-e). The last one will follow the web until a limit stops it.

Rewrite links-KN, --keep-links N

What the links in the saved pages look like: relative, so the copy browses offline (K0, the default), absolute, or the original online addresses (K4), which gives you the pages as archived documents rather than a browsable mirror.

Activate debug mode-%H, --debug-headers

Writes the HTTP headers and other internals to the log. For diagnosing a site that behaves strangely, not for normal use.

Where to go next

• Filter syntax: the full pattern language behind Scan Rules.
• Command-line guide: the same engine, scripted.
• Manual page: every option, authoritative.
• FAQ and troubleshooting: when a mirror comes back wrong.
• How not to use HTTrack: worth two minutes before pointing it at someone else's server.