Here is a small Haskell package for pushing files to Amazon Glacier: https://github.com/carlohamalainen/glacier-push. It uses Brendan Hay’s amazonka API, in particular amazonka-glacier.

One thing that I couldn’t find in amazonka was a way to calculate the tree hash of a file. The Glacier API needs this for each part that is uploaded as well as the whole file. Amazon explains how to calculate the tree hash in their Glacier docs and provides sample code in Java and C++. Since the algorithm is recursive, it is quite short in Haskell:

To push a large file to Glacier we do three things: initiate the multipart upload, upload each part (say, 100Mb chunks), and then finalize the upload.

We do this to get an uploadId which is then used for each of the multipart uploads. We use initiateMultipartUpload, and need to set the part size.

With the response from initiating the multipart upload (the mu parameter in uploadOnePart) we can push a single part using uploadMultipartPart. Here we have to also set the checksum and content range:

Completing the multipart upload lets Glacier know that it should start its job of assembling all the parts into a full archive. We have to set the archive size and the tree hash of the entire file.

## Notes

In each of these functions I used a helper for sending the request:

I run the main block of work in a KatipContextT since I am using katip for structured logging. Adding new key-value info to the log context is accomplished using katipAddContext.

I found it handy to write this little helper function to turn each header from an amazonka ServiceError into a Katip context key/value pair:

Katip can write to ElasticSearch using katip-elasticsearch. Then you’d be able to search for errors on specific header fields, etc.

## Sample run

glacier-push-exe basement myfile


The lines are pretty long (as they are intended for consumption into ElasticSearch, not human parsing) so here is one with line breaks:

[2017-08-30 12:45:45]
[glacier-push.main]
[Info]
[x4]
[1724]
[partEnd:268435455]
[partStart:134217728]

$git clone https://github.com/carlohamalainen/glacier-push.git$ cd glacier-push