2,651
edits
Eiger: Difference between revisions
update paragraph about compression
No edit summary |
(update paragraph about compression) |
||
| Line 5: | Line 5: | ||
# For faster processing, the shell script below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). This script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary file that Dectris' H5ToXds writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes). | # For faster processing, the shell script below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). This script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary file that Dectris' H5ToXds writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes). | ||
A suitable XDS.INP should normally be written by the beamline software; [[generate_XDS.INP]] does not (yet) write it. The XDS_from_H5.py script can be used if XDS.INP is not available | A suitable XDS.INP should normally be written by the beamline software; [[generate_XDS.INP]] does not (yet) write it. The XDS_from_H5.py script (below) can be used if XDS.INP is not available. | ||
The number of pixels of the Eiger 16M is three times higher than that of the Pilatus 6M, but since the Eiger firmware update in November 2015, the ("bit shufflle LZ4") compression of the .h5 files containing data is better than that of CBF files, which mostly compensates for the increased number of pixels. However, the size of the *master.h5 file from a Eiger 16M experiment at SLS X06SA is more than 300MB, ''no matter how many frames are collected''. It is therefore advisable to compress (by ~75%) the *master.h5 files on-site, before transferring them home using disk or internet. The fastest (parallel) program with the best compression that I found is [http://lbzip2.org lbzip2] (available from the EPEL repository for RHEL clones). It is supposedly fully compatible with bzip2. | |||
== A script for faster XDS processing of Eiger data == | == A script for faster XDS processing of Eiger data == | ||