(DIR) <- Back
       
       
       # hurl: HTTP, HTTPS and Gopher file grabber
       
       Last modification on 2020-07-20
       
       hurl is a relatively simple HTTP, HTTPS and Gopher client/file grabber.
       
       
       ## Why?
       
       Sometimes (or most of the time?) you just want to fetch a file via the HTTP,
       HTTPS or Gopher protocol.
       
       The focus of this tool is only this.
       
       
       ## Features
       
       * Uses OpenBSD pledge(2) and unveil(2). Allow no filesystem access (writes to
         stdout).
       * Impose time-out and maximum size limits.
       * Use well-defined exitcodes for reliable scripting (curl sucks at this).
       * Send as little information as possible (no User-Agent etc by default).
       
       
       ## Anti-features
       
       * No HTTP byte range support.
       * No HTTP User-Agent.
       * No HTTP If-Modified-Since/If-* support.
       * No HTTP auth support.
       * No HTTP/2+ support.
       * No HTTP keep-alive.
       * No HTTP chunked-encoding support.
       * No HTTP redirect support.
       * No (GZIP) compression support.
       * No cookie-jar or cookie parsing support.
       * No Gopher text handling (".\r\n").
       * ... etc...
       
       
       ## Dependencies
       
       * C compiler (C99).
       * libc + some BSD functions like err() and strlcat().
       * LibreSSL(-portable)
       * libtls (part of LibreSSL).
       
       
       ## Optional dependencies
       
       * POSIX make(1) (for Makefile).
 (HTM) * mandoc for documentation: »https://mdocml.bsd.lv/«
       
       
       ## Clone
       
               git clone git://git.codemadness.org/hurl
       
       
       ## Browse
       
       You can browse the source-code at:
       
 (HTM) * https://git.codemadness.org/hurl/
 (DIR) * gopher://codemadness.org/1/git/hurl
       
       
       ## Download releases
       
       Releases are available at:
       
 (HTM) * https://codemadness.org/releases/hurl/
 (DIR) * gopher://codemadness.org/1/releases/hurl
       
       
       ## Build and install
       
               $ make
               # make install
       
       
       ## Examples
       
       Fetch the Atom feed from this site using a maximum filesize limit of 1MB and
       a time-out limit of 15 seconds:
       
               hurl -m 1048576 -t 15 "https://codemadness.org/atom.xml"
       
       There is an -H option to add custom headers. This way some of the anti-features
       listed above are supported. For example some CDNs like Cloudflare are known to
       block empty or certain User-Agents.
       
       User-Agent:
       
               hurl -H 'User-Agent: some browser' 'https://codemadness.org/atom.xml'
       
       HTTP Basic Auth (base64-encoded username:password):
       
               hurl -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=' \
                       'https://codemadness.org/atom.xml'
       
       GZIP (this assumes the served response Content-Type is gzip):
       
               hurl -H 'Accept-Encoding: gzip' 'https://somesite/' | gzip -d