Read File From Server With Some Offset

How can I read file from server starting with some offset (Similar behavior to wget -c)? What headers I must send to server? What futures must server support?

Solution 1:

You should use the Range header in the request. But you may use it only if the server informs you that it accept range request by Accept-Ranges response header.

This is an example session. Suppose we are interested in getting a part of this picture. First, we send a HTTP HEAD request to determine: a) if the server supports byte ranges, b) the content-length:

>HEAD/2238/2758537173_670161cac7_b.jpgHTTP/1.1>Host:farm3.static.flickr.com>Accept:*/*><HTTP/1.1200OK<Date:Thu,08Jul2010 12:22:12 GMT<Content-Type:image/jpeg<Connection:keep-alive<Server:Apache/2.0.52(RedHat)<Expires:Mon,28Jul2014 23:30:00 GMT<Last-Modified:Wed,13Aug2008 06:13:54 GMT<Accept-Ranges:bytes<Content-Length:350015

Next, we send a GET request with the Range header asking for the first 11 bytes of the picure:

>GET/2238/2758537173_670161cac7_b.jpgHTTP/1.1>Host:farm3.static.flickr.com>Accept:*/*>Range:bytes=0-10><HTTP/1.1206PartialContent<Date:Thu,08Jul2010 12:26:54 GMT<Content-Type:image/jpeg<Connection:keep-alive<Server:Apache/2.0.52(RedHat)<Expires:Mon,28Jul2014 23:30:00 GMT<Last-Modified:Wed,13Aug2008 06:13:54 GMT<Accept-Ranges:bytes<Content-Range:bytes0-10/350015<Content-Length:11<

This is a hex dump of the first 11 bytes:

00000000  ff d8 ff e0 00 10 4a 46  49 46 00                 |......JFIF.|
0000000b

For more info see the Range header specification in HTTP RFC 2616.

Solution 2:

In http://www.gnu.org/software/wget/manual/wget.html

Note that ‘-c’ only works with ftp servers and with http servers that support the Range header.

In https://www.rfc-editor.org/rfc/rfc2616

Examples of byte-ranges-specifier values (assuming an entity-body of length 10000):

  - The first 500 bytes (byte offsets 0-499, inclusive):  bytes=0-
    499
  - The second 500 bytes (byte offsets 500-999, inclusive):
    bytes=500-999
  - The final 500 bytes (byte offsets 9500-9999, inclusive):
    bytes=-500
  - Or bytes=9500-

  - The first and last bytes only (bytes 0 and 9999):  bytes=0-0,-1

  - Several legal but not canonical specifications of the second

500 bytes (byte offsets 500-999, inclusive): bytes=500-600,601-999 bytes=500-700,601-999

So you should send

Range:bytes=9500-

To test if a server support it you can test the accept-range as such

Origin servers that accept byte-range requests MAY send

Accept-Ranges: bytes

but are not required to do so. Clients MAY generate byte-range requests without having received this header for the resource involved. Range units are defined in section 3.12.

Servers that do not accept any kind of range request for a resource MAY send

Accept-Ranges: none

to advise the client not to attempt a range request.