2,651
edits
Eiger: Difference between revisions
no edit summary
No edit summary |
|||
| Line 3: | Line 3: | ||
== General aspects == | == General aspects == | ||
# It is advisable to use the most recent 64bit version of XDS (since version Oct 15, 2015 the 32bit versions are no longer distributed anyway). The idea of the new framecache in XDS is that RAM is used to save on I/O. To this end, XDS tries to store NUMBER_OF_IMAGES_IN_CACHE=DELPHI/OSCILLATION_RANGE images in memory. Each frame is stored as (number of pixels)*(4 bytes) which means 72 MB in case of the Eiger 16M. As an example: if DELPHI=20 and OSCILLATION_RANGE=0.05 your computer has to have 400*72MB = 29GB of memory (plus some more for the program and the operating system). If it has not, the fallback is to the old behaviour of reading each frame three times. | # It is advisable to use the most recent 64bit version of XDS (since version Oct 15, 2015 the 32bit versions are no longer distributed anyway). The idea of the new framecache in XDS is that RAM is used to save on I/O. To this end, XDS tries to store NUMBER_OF_IMAGES_IN_CACHE=DELPHI/OSCILLATION_RANGE images in memory. Each frame is stored as (number of pixels)*(4 bytes) which means 72 MB in case of the Eiger 16M. As an example: if DELPHI=20 and OSCILLATION_RANGE=0.05 your computer has to have 400*72MB = 29GB of memory (plus some more for the program and the operating system). If it has not, the fallback is to the old behaviour of reading each frame three times. | ||
# Dectris provides [https://www.dectris.com/news.html?page=2 H5ToXds] (Linux only!) which is needed by XDS. H5ToXds should be copied to e.g. /usr/local/bin/H5ToXds.bin - note the .bin filename extension! As an alternative, one could use GlobalPhasing's hdf2mini-cbf program or, from http://www.mrc-lmb.cam.ac.uk/harry/imosflm/ver721/downloads, the | # Dectris provides [https://www.dectris.com/news.html?page=2 H5ToXds] (Linux only!) which is needed by XDS. H5ToXds should be copied to e.g. /usr/local/bin/H5ToXds.bin - note the .bin filename extension! As an alternative, one could use GlobalPhasing's hdf2mini-cbf program (needs autoPROC license) or, from http://www.mrc-lmb.cam.ac.uk/harry/imosflm/ver721/downloads, the eiger2cbf-osx or eiger2cbf-linux program written by T. Nakane (the latter were called miniCBF-Linux and miniCBF-OSX before March 13, 2016). | ||
# For faster processing, the [[Eiger#A_script_for_faster_XDS_processing_of_Eiger_data|shell script]] below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). This script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary file that H5ToXds/ | # For faster processing, the [[Eiger#A_script_for_faster_XDS_processing_of_Eiger_data|shell script]] below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). This script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary file that H5ToXds/eiger2cbf/hdf2mini-cbf writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes). | ||
A suitable XDS.INP should normally be written by the beamline software; [[generate_XDS.INP]] does not (yet) write it. The XDS_from_H5.py script (below) can be used if XDS.INP is not available. | A suitable XDS.INP should normally be written by the beamline software; [[generate_XDS.INP]] does not (yet) write it. The XDS_from_H5.py script (below) can be used if XDS.INP is not available. | ||
| Line 33: | Line 33: | ||
so there's still some room for improvement. | so there's still some room for improvement. | ||
With program versions as of 2016-03-10, | With program versions as of 2016-03-10, eiger2cbf-linux is practically as fast as the H5ToXds binary; hdf2mini-cbf is somewhat slower. | ||
When unpacking the .h5 files to .cbf files and processing those, I get (on the same machine and with same processing parameters) on three successive runs: | When unpacking the .h5 files to .cbf files and processing those, I get (on the same machine and with same processing parameters) on three successive runs: | ||
| Line 49: | Line 49: | ||
#!/bin/bash | #!/bin/bash | ||
# Kay Diederichs 10/2015 | # Kay Diederichs 10/2015 | ||
# 3/2016 adapt for | # 3/2016 adapt for eiger2cbf-linux/eiger2cbf-osx/hdf2min-cbf | ||
# for the latter see https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;58a4ee1.1603 and | # for the latter see https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;58a4ee1.1603 and | ||
# https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;a048b4e8.1603 | # https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;a048b4e8.1603 | ||
| Line 65: | Line 65: | ||
tempfile="/dev/shm/H5ToXds${PWD//\//_}.$3" | tempfile="/dev/shm/H5ToXds${PWD//\//_}.$3" | ||
# | # | ||
# comment out the next 2 lines when using hdf2mini-cbf or | # comment out the next 2 lines when using hdf2mini-cbf or eiger2cbf; uncomment for H5ToXds.bin | ||
/usr/local/bin/H5ToXds.bin $1 $2 "$tempfile" | /usr/local/bin/H5ToXds.bin $1 $2 "$tempfile" | ||
ln -sf "$tempfile" $3 2>/dev/null | ln -sf "$tempfile" $3 2>/dev/null | ||
# | # | ||
# comment out the next 2 lines when using H5ToXds.bin or hdf2mini-cbf; uncomment for | # comment out the next 2 lines when using H5ToXds.bin or hdf2mini-cbf; uncomment for eiger2cbf | ||
#/usr/local/bin/ | #/usr/local/bin/eiger2cbf-osx $1 $2 "$tempfile" >& /dev/null | ||
#ln -sf "$tempfile" $3 2>/dev/null | #ln -sf "$tempfile" $3 2>/dev/null | ||
# | # | ||
# comment out the next 3 lines when using H5ToXds.bin or | # comment out the next 3 lines when using H5ToXds.bin or eiger2cbf; uncomment for hdf2mini-cbf | ||
#/usr/local/bin/hdf2mini-cbf -image $2 -m $1 -o "$tempfile" >& /dev/null | #/usr/local/bin/hdf2mini-cbf -image $2 -m $1 -o "$tempfile" >& /dev/null | ||
#mv "$tempfile"$2.cbf "$tempfile" | #mv "$tempfile"$2.cbf "$tempfile" | ||