Eiger: Difference between revisions

Jump to navigation Jump to search

Eiger (view source)

Revision as of 18:32, 27 February 2021

2,776 bytes removed ,  27 February 2021
→‎Less efficient way of processing Eiger data, using conversion to CBF
(47 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Processing of [https://www.dectris.com/EIGER_X_Features.html Eiger] data is different from processing of conventional data, because the frames are wrapped into [http://www.hdfgroup.org HDF5] files (ending with .h5).
Processing of [https://www.dectris.com/EIGER_X_Features.html Eiger] data is different from processing of conventional data, because the frames are wrapped into [http://www.hdfgroup.org HDF5] files (often ending with .h5). However, with the [[LIB]] feature of XDS and a suitable plugin ([https://github.com/dectris/neggia ''Neggia''] or [https://github.com/DiamondLightSource/durin ''Durin'']), processing is as straightforward as before.


== General aspects ==
== General aspects ==
# The framecache of XDS uses memory to save on I/O; it saves a frame in RAM after reading it for the first time. By default, each XDS (or mcolspot/mintegrate) process stores NUMBER_OF_IMAGES_IN_CACHE=DELPHI/OSCILLATION_RANGE images in memory which corresponds to one DELPHI-sized batch of data. This requires (number of pixels)*(number of jobs)*4 Bytes per frame which amounts to 72 MB in case of the Eiger 16M when running with MAXIMUM_NUBER_OF_JOBS=1. (If DELPHI=20 and OSCILLATION_RANGE=0.05 your computer thus has to have at least 400*72MB = 29GB of memory for each process). If it has not, the fallback is to the old behaviour of reading each frame three times (instead of once). There is an upper limit (2GB?) to the amount of memory that will be used by default; if the required memory is more than that, a message will be printed and the user must explicitly include a NUMBER_OF_IMAGES_IN_CACHE= line in XDS.INP.
# The framecache of XDS uses memory to save on I/O; it saves a frame in RAM after reading it for the first time. By default, each XDS (or mcolspot/mintegrate) job stores NUMBER_OF_IMAGES_IN_CACHE=DELPHI/OSCILLATION_RANGE images in memory which corresponds to one DELPHI-sized batch of data. This requires (number of pixels)*(number of jobs)*4 Bytes per frame which amounts to 72 MB in case of the Eiger 16M when running with MAXIMUM_NUBER_OF_JOBS=1. (If DELPHI=20 and OSCILLATION_RANGE=0.05 your computer thus has to have at least 400*72MB = 29GB of memory for each job!). If memory allocation fails, the fallback is to the old behaviour of reading each frame three times (instead of once).
# Dectris provides [https://www.dectris.com/news.html?page=2 H5ToXds] (Linux only!) which is needed by XDS. That program converts (as the name indicates) the HDF5 files to CBF files; however, it does not write the geometry and other information into the CBF header (therefore, [[generate_XDS.INP]] does not work with these files). As an alternative, one could use GlobalPhasing's hdf2mini-cbf program (needs autoPROC license) or, from http://www.mrc-lmb.cam.ac.uk/harry/imosflm/ver721/downloads, the eiger2cbf-osx or eiger2cbf-linux program written by T. Nakane. These programs do write a useful CBF header.
# Dectris provides the ''Neggia'' library ([https://github.com/dectris/neggia source],[https://www.dectris.com/support/downloads/sign-in binary]) for native reading of HDF5 files, which can be loaded into XDS at runtime using the <code>[[LIB]]=</code> [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_parameters.html#LIB= keyword]. With this library (which can also be found at https://{{SERVERNAME}}/pub/linux_bin for Linux, and at https://{{SERVERNAME}}/pub/mac_bin for MacOS), no conversion to CBF or otherwise is necessary. It is therefore just as fast and efficient to read HDF5 files as any other file format. At Diamond Light Source, a different HDF5 format was developed, and this requires the [https://github.com/DiamondLightSource/durin/releases/latest ''Durin'' plugin]. The latter can also read the HDF5 files written by the Dectris software.
# For faster processing (Linux only; script needs to be adapted for OSX), the [[Eiger#A_script_for_faster_XDS_processing_of_Eiger_data|shell script]] below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). The binary H5ToXds then should be named e.g. /usr/local/bin/H5ToXds.bin - note the .bin filename extension! The script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary CBF file that H5ToXds/eiger2cbf/hdf2mini-cbf writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes).


A suitable [[XDS.INP]] may have been written by the data collection (beamline) software. Latest [[generate_XDS.INP]] (<code>generate_XDS.INP xxx_master.h5</code>) or the [[Eiger#XDS_from_H5.py_script_for_generating_XDS.INP_given_a_master_.h5_file|XDS_from_H5.py script]] can be used if XDS.INP is not available.
A suitable [[XDS.INP]] may have been written by the data collection (beamline) software. Latest [[generate_XDS.INP]] (<code>generate_XDS.INP xxx_master.h5</code>) or the [[Eiger#Script_for_generating_XDS.INP_from_master.h5|XDS_from_H5.py script]] can be used if XDS.INP is not available.


== Compression ==
== Compression ==
Line 15: Line 14:


Update 2016-06-05 (Toine Schreurs): a HDF5 file may be compressed with [https://www.hdfgroup.org/HDF5/docNewFeatures/FileSpace/h5repack.htm h5repack], ''e.g.'' by <code>h5repack -i <in.h5> -o <out.h5> -f GZIP=6</code> (6 is the default compression level of gzip). This should be a good way to reduce the size of master files while keeping them compatible with processing, but needs to be tested. Whether h5repack uses parallel gzip is not clear from the docs.
Update 2016-06-05 (Toine Schreurs): a HDF5 file may be compressed with [https://www.hdfgroup.org/HDF5/docNewFeatures/FileSpace/h5repack.htm h5repack], ''e.g.'' by <code>h5repack -i <in.h5> -o <out.h5> -f GZIP=6</code> (6 is the default compression level of gzip). This should be a good way to reduce the size of master files while keeping them compatible with processing, but needs to be tested. Whether h5repack uses parallel gzip is not clear from the docs.
== A benchmark ==
Any comparisons should be based on a common dataset. I downloaded from https://www.dectris.com/datasets.html their latest dataset
ftp://dectris.com/EIGER_16M_Nov2015.tar.bz2 (900 frames) and processed it on a single unloaded CentOS7.2 64bit machine with dual Intel(R) Xeon(R) CPU E5-2667 v2 @ 3.30GHz , HT enabled (showing 32 processors in /proc/cpuinfo), on a local XFS filesystem (all defaults), with four JOBs and 12 PROCESSORS (the XDS.INP that Dectris provides suggests 8 JOBs of 12 PROCESSORS, but I changed that). The numbers below refer to the H5ToXds binary as used in the [[Eiger#A_script_for_faster_XDS_processing_of_Eiger_data|script]] below.
The timing, using the XDS (BUILT=20151231), is on the first run
INIT:  elapsed wall-clock time      12.0 sec
COLSPOT: elapsed wall-clock time      44.9 sec
INTEGRATE: total elapsed wall-clock time      65.1 sec
CORRECT: elapsed wall-clock time        2.9 sec
Total elapsed wall-clock time for XDS      133.6 sec
When I repeat this, I get
Total elapsed wall-clock time for XDS      128.3 sec
Repeat once again:
Total elapsed wall-clock time for XDS      129.3 sec
So a bit of cache-warming helps, but not much. This machine has 64GB RAM. From the output of "top", the highest memory usage occurs during INTEGRATE, when each of the mintegrate_par processes consumes up to 7.4% of the memory. In other words, in this way less than 20GB of total memory are used. "top" shows a CPU consumption around (on average) 4 times 650%.
The number of JOBs and PROCESSORs could be optimized. I tried 6 JOBs and get
Total elapsed wall-clock time for XDS      120.1 sec
so there's still some room for improvement.
With program versions as of 2016-03-10, eiger2cbf-linux is practically as fast as the H5ToXds binary; hdf2mini-cbf is somewhat slower.
When unpacking the .h5 files to .cbf files and processing those, I get on the same machine and with same processing parameters:
Total elapsed wall-clock time for XDS      96.3 sec
which indicates a 24% overhead due to the HDF5-to-CBF conversion. However, one has to add to this the time for the HDF5-to-CBF conversion, which is (with 18 parallel H5ToXds jobs each converting 50 frames) 34.2 sec, so overall the "on-the-fly" route using the script below is faster than the "pre-conversion" route, at least on this machine.
On multi-socket machines, there are additional considerations having to do with their NUMA architecture - see [[Performance]].
=== Xeon Phi (Knights Landing, KNL) ===
The benchmark was run on a single KNL7210 processor (256 cores) set to quadrant mode and using the MCDRAM as cache. The environment variable OMP_PROC_BIND was set to false (if this is not done, the scheduler seems to put all threads on one core). XDS was compiled with the -xMIC-AVX512 option of ifort.
Deviating from the above benchmark setup, BACKGROUND_RANGE was set to a more realistic value of 1 50 (instead of 1 9).
This gives
COLSPOT:        elapsed wall-clock time      48.3 sec
INTEGRATE: total elapsed wall-clock time      61.2 sec
when run with MAXIMUM_NUMBER_OF_JOBS=16 and MAXIMUM_NUMBER_OF_PROCESSORS=16. These parameters, as well as the KNL setup could still be optimized.
Update Feb 21, 2017 using XDS BUILT=20161205, and the CentOS-7.3 default kernel 3.10.0-514.6.1.el7:
INIT:            elapsed wall-clock time      33.4 sec
COLSPOT:        elapsed wall-clock time      49.3 sec
INTEGRATE: total elapsed wall-clock time      59.8 sec
Using a pre-release library that makes use of the <code>LIB=</code> [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#LIB= option] of XDS:
INIT:            elapsed wall-clock time      30.4 sec
COLSPOT:        elapsed wall-clock time      40.7 sec
INTEGRATE: total elapsed wall-clock time      52.9 sec
Now additionally running with <code>numactl --preferred=1 xds_par</code> after having modified the forkintegrate script such that it starts mintegrate_par with the same numactl parameters:
INIT.LP:        elapsed wall-clock time      29.8 sec
COLSPOT:        elapsed wall-clock time      40.0 sec
INTEGRATE: total elapsed wall-clock time      51.3 sec
This was running with a 8GB/8GB split (''hybrid'') MCDRAM. The same run, but with 8 JOBS and 32 PROCESSORS, takes
INIT.LP:        elapsed wall-clock time      25.3 sec
COLSPOT:        elapsed wall-clock time      40.1 sec
INTEGRATE: total elapsed wall-clock time      53.1 sec
Back to 16 JOBS and 16 PROCESSORS, but with MCDRAM in ''flat'' mode und <code>numactl --preferred=1 xds_par</code> (thus using all 16GB for arrays, and nothing for cache):
INIT.LP:        elapsed wall-clock time      29.5 sec
COLSPOT:        elapsed wall-clock time      38.6 sec
INTEGRATE: total elapsed wall-clock time      53.2 sec
Now setting the KNL to SNC4 mode, and the MCDRAM to cache (using it in flat mode is impractical because the --preferred argument takes only 1 argument; to determine the correct argument requires scripting):
INIT.LP:        elapsed wall-clock time      29.6 sec
COLSPOT.LP:      elapsed wall-clock time      37.8 sec
INTEGRATE: total elapsed wall-clock time      49.6 sec
Conclusions: since INIT benefits from more PROCESSORs, one could run XDS twice for fastest turnaround; the first run with JOBS=XYCORR INIT and a high number of processors (99 is maximum). The second run with JOB=COLSPOT IDXREF DEFPIX INTEGRATE CORRECT, and an optimized JOBS/PROCESSORS combination. The SNC4 mode is indeed fastest - to do better than the cache mode of the MCDRAM, one needs to adapt the forkcolspot and forkintegrate script- see [[Performance]].


== Troubleshooting ==
== Troubleshooting ==
* make sure that master.h5 and the corresponding data.h5 files remain together as collected, and '''don't rename the data.h5 files''' - they are referred to from master.h5.  If you change the names of the data.h5 files or copy them somewhere else, that link is broken unless you fix master.h5. 
* make sure that master.h5 and the corresponding data.h5 files remain together as collected, and '''don't rename the data.h5 files''' - they are referred to from master.h5.  If you change the names of the data.h5 files or copy them somewhere else, that link is broken unless you fix master.h5.
* the programs get a lot of testing on RHEL/CentOS/SL. To test if the conversion program work (e.g. on uncommon distros like Mint), run it outside XDS, e.g. <pre> H5ToXds master.h5 1:100 out.cbf </pre> If this creates CBF-compressed files for the first 100 images of your dataset, all is good.


== A script for faster XDS processing of Eiger data ==
== Script by Andreas Förster (Dectris) for generating XDS.INP from master.h5 ==
<div class="mw-collapsible mw-collapsed">
Expand code section below (i.e. click on blue <code>[Expand]</code> at the end of this line if there is no code visible), download it and save as XDS_from_H5.py .
<div class="mw-collapsible-content">
<pre>
<pre>
#!/bin/bash
# -*- coding: utf-8 -*-
# Kay Diederichs 10/2015
# 3/2016 adapt for eiger2cbf-linux and hdf2min-cbf
# for the latter see https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;58a4ee1.1603 and
# https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;a048b4e8.1603
#
# Idea: put temporary files into fast local directory, instead of NFS
#
# Installation: Rename Dectris' H5ToXds to H5ToXds.bin
#              This script should be called H5ToXds and reside in $PATH
#              Modify this script according to which binary you use - see comments below.
#
# Recommendation:
# - for the fast local directory one should use a RAMdisk (one GB size at most)
# - /dev/shm seems to be set up for that purpose on most distributions
#
tempfile="/dev/shm/H5ToXds${PWD//\//_}.$3"
#
# choose between H5ToXds.bin,  eiger2cbf and hdf2mini-cbf; un/comment accordingly
/usr/local/bin/H5ToXds.bin $1 $2 "$tempfile" || rm "$tempfile"
#/usr/local/bin/eiger2cbf-linux $1 $2 "$tempfile" >& /dev/null  || rm "$tempfile"
#/usr/local/bin/hdf2mini-cbf $1 $2 "$tempfile"  || rm "$tempfile"
ln -sf "$tempfile" $3 2>/dev/null
</pre>


== XDS_from_H5.py script for generating XDS.INP given a master .h5 file ==
__author__ = "AndF"
This script could be made executable and put into /usr/local/bin. It requires the [https://www.dectris.com/albula.html#main_head_navigation ALBULA API] to be installed. If you get the error message
__date__ = "2017/03/08"
ImportError: No module named numpy.core.multiarray
__reviewer__ = ""
you should
__version__ = "0.1.1"
yum -y install numpy
as root.
<pre>
#!/usr/bin/python
# coding: utf8


import sys
import sys
# Path needs to be be set only if dectris.albula is not found
# Path needs to be be set only if dectris.albula is not found
# i.e. if ALBULA was installed without "--python=</path/to/python_interpreter>"
# i.e. if ALBULA was installed without "--python=</path/to/python_interpreter>"
# Define correct path to ALBULA API:
# Uncomment below (and define correct path to ALBULA API)
sys.path.insert(0,"/usr/local/dectris/albula/3.1/python")
# sys.path.insert(0,"/usr/local/dectris/albula/3.2/python")
try:
try:
   import dectris.albula as dec
   import dectris.albula as dec
Line 151: Line 57:
!    Characters to the right of an exclamation mark are comments.
!    Characters to the right of an exclamation mark are comments.
!
!
!    This file was autogenerated by XDS_from_H5.py (Oct 2015).
!    This file was autogenerated by XDS_from_H5.py (Mar 2017).
!    Please check default values before processing.
!    Please check default values before processing.
!
!
Line 162: Line 68:
!====================== DETECTOR PARAMETERS ==================================
!====================== DETECTOR PARAMETERS ==================================
  DETECTOR=%(family)s
  DETECTOR=%(family)s
LIB= /usr/local/lib64/dectris-neggia.so
  MINIMUM_VALID_PIXEL_VALUE=0
  MINIMUM_VALID_PIXEL_VALUE=0
  OVERLOAD= %(cutoff)i ! taken from HDF5 header item
  OVERLOAD= %(cutoff)i ! taken from HDF5 header item
Line 177: Line 84:


!====================== JOB CONTROL PARAMETERS ===============================
!====================== JOB CONTROL PARAMETERS ===============================
  JOB= XYCORR INIT COLSPOT IDXREF DEFPIX ! XPLAN INTEGRATE CORRECT
!JOB= XYCORR INIT COLSPOT IDXREF DEFPIX ! XPLAN INTEGRATE CORRECT
  JOB= XYCORR INIT COLSPOT IDXREF DEFPIX INTEGRATE CORRECT
!JOB= INTEGRATE CORRECT


!Set maximum number of jobs and processors so that their products comes close
!to the number of CPUs of the machine.
  MAXIMUM_NUMBER_OF_JOBS=8  !Speeds up COLSPOT & INTEGRATE on multicore machine
  MAXIMUM_NUMBER_OF_JOBS=8  !Speeds up COLSPOT & INTEGRATE on multicore machine
  MAXIMUM_NUMBER_OF_PROCESSORS=4!<32;ignored by single cpu version of xds
  MAXIMUM_NUMBER_OF_PROCESSORS=4!<99;ignored by single cpu version of xds
!SECONDS=0  !Maximum number of seconds to wait until data image must appear
!SECONDS=0  !Maximum number of seconds to wait until data image must appear
!TEST=1    !Test flag. 1,2 additional diagnostics and images
!TEST=1    !Test flag. 1,2 additional diagnostics and images
Line 187: Line 98:
!ORGX and ORGY are often close to the image center, i.e. ORGX=NX/2, ORGY=NY/2
!ORGX and ORGY are often close to the image center, i.e. ORGX=NX/2, ORGY=NY/2
  ORGX= %(orgx).1f  ORGY= %(orgy).1f    !Detector origin (pixels).  ORGX=NX/2; ORGY=NY/2
  ORGX= %(orgx).1f  ORGY= %(orgy).1f    !Detector origin (pixels).  ORGX=NX/2; ORGY=NY/2
  DETECTOR_DISTANCE= %(dist)i   ! [mm]
  DETECTOR_DISTANCE= %(dist).2f   ! [mm]


  ROTATION_AXIS= 1.0 0.0 0.0
  ROTATION_AXIS= 1.0 0.0 0.0
Line 212: Line 123:
!REIDX=  0  0 -1  0  0 -1  0  0 -1  0  0  0
!REIDX=  0  0 -1  0  0 -1  0  0 -1  0  0  0


FRIEDEL'S_LAW=FALSE ! Default is TRUE.
!FRIEDEL'S_LAW=FALSE ! Default is TRUE.


!REFERENCE_DATA_SET= CK.HKL  ! Name of a reference data set (optional)
!REFERENCE_DATA_SET= CK.HKL  ! Name of a reference data set (optional)
Line 299: Line 210:
!REFLECTING_RANGE_E.S.D.=  0.113 !half-width (mosaicity) of REFLECTING_RANGE
!REFLECTING_RANGE_E.S.D.=  0.113 !half-width (mosaicity) of REFLECTING_RANGE


  NUMBER_OF_PROFILE_GRID_POINTS_ALONG_ALPHA/BETA=21!used by: INTEGRATE
!The next two values could be increased up to 21 for best profiles.
  NUMBER_OF_PROFILE_GRID_POINTS_ALONG_GAMMA=21     !used by: INTEGRATE
  NUMBER_OF_PROFILE_GRID_POINTS_ALONG_ALPHA/BETA=13!used by: INTEGRATE
  NUMBER_OF_PROFILE_GRID_POINTS_ALONG_GAMMA=13     !used by: INTEGRATE


!DELPHI= 6.0!controls the number of reference profiles and scaling factors
!DELPHI= 6.0!controls the number of reference profiles and scaling factors
Line 441: Line 353:
         'orgx': float(parameters["/entry/instrument/detector/beam_center_x"]),
         'orgx': float(parameters["/entry/instrument/detector/beam_center_x"]),
         'orgy': float(parameters["/entry/instrument/detector/beam_center_y"]),
         'orgy': float(parameters["/entry/instrument/detector/beam_center_y"]),
         'dist': int(float(parameters["/entry/instrument/detector/detector_distance"]) * 1000.0),
         'dist': float(parameters["/entry/instrument/detector/detector_distance"]) * 1000.0,
         'osc_range': float(parameters["/entry/sample/goniometer/omega_range_average"]),
         'osc_range': float(parameters["/entry/sample/goniometer/omega_range_average"]),
         'wavelength': float(parameters["/entry/instrument/beam/incident_wavelength"]),
         'wavelength': float(parameters["/entry/instrument/beam/incident_wavelength"]),
Line 502: Line 414:
             gap[1] + 1 + offset + n_excluded_edge_pixels,
             gap[1] + 1 + offset + n_excluded_edge_pixels,
             0,
             0,
             detector_families[fam]['sizes'][det][1] + 1))
             detector_families[fam]['sizes'][det][1] + offset))
     param_lines.append('\n')
     param_lines.append('\n')
     param_lines.append('!EXCLUSION OF HORIZONTAL DEAD AREAS OF THE '
     param_lines.append('!EXCLUSION OF HORIZONTAL DEAD AREAS OF THE '
Line 511: Line 423:
         param_lines.append(' UNTRUSTED_RECTANGLE= %4d %4d  %4d %4d \n' % (
         param_lines.append(' UNTRUSTED_RECTANGLE= %4d %4d  %4d %4d \n' % (
             0,
             0,
             detector_families[fam]['sizes'][det][0] + 1,
             detector_families[fam]['sizes'][det][0] + offset,
             gap[0] - 1 + offset - n_excluded_edge_pixels,
             gap[0] - 1 + offset - n_excluded_edge_pixels,
             gap[1] + 1 + offset + n_excluded_edge_pixels))
             gap[1] + 1 + offset + n_excluded_edge_pixels))
Line 519: Line 431:
     return ('\nThis script extracts from a given HDF5 master file all metadata\n'
     return ('\nThis script extracts from a given HDF5 master file all metadata\n'
             'required to write XDS.INP.  The user is prompted for missing metadata.\n'
             'required to write XDS.INP.  The user is prompted for missing metadata.\n'
            'If there are errors in the metadata, XDS.INP will be incorrect.\n'
             '\n'
             '\n'
            'WARNING - This script is a proof-of-principle, pre-alpha.\n'
             'Please report shortcomings and errors to docandreas@gmail.com\n')
            'Do not rely on it for anything serious.  Things will go wrong.\n'
            'In particular, this does not work for data collected in ROI mode.\n'
            '\n'
             'Please report shortcomings and errors to andreas.foerster@dectris.com\n')


def help():
def help():
Line 574: Line 483:
         return raw_input("Please enter the maximum trusted pixel value.\n")
         return raw_input("Please enter the maximum trusted pixel value.\n")
     elif (parameter == resolution_cutoff):
     elif (parameter == resolution_cutoff):
         return raw_input("Please enter a resolution limit for processing.\n")
         print "Please enter a resolution limit for processing."
        return raw_input("Enter '0' to let XDS decide.\n") or 0
     else:
     else:
         print "Unknown software version.  Please check."
         print "Unknown software version.  Please check."
Line 681: Line 591:
                 else:
                 else:
                     print "\nThe HDF5 file was created with version %s of the detector firmware" % (para_version)
                     print "\nThe HDF5 file was created with version %s of the detector firmware" % (para_version)
                     print "This script supports versions 1.2 and up."
                     print "This script supports versions 1.5 and up."
                     print "\nFile XDS.INP was not created."
                     print "\nFile XDS.INP was not created."
                     print "Please extract metadata with hdfview or h5dump.\n"
                     print "Please extract metadata with hdfview or h5dump.\n"
Line 699: Line 609:
         exit(-1)
         exit(-1)
</pre>
</pre>
</div>
</div>
Then,
* Make script executable and put into /usr/local/bin.
* Install [https://www.dectris.com/albula.html#main_head_navigation ALBULA API]
* Install numpy (yum -y install numpy) as root if you get the error message
** ImportError: No module named numpy.core.multiarray
Once XDS.INP has been generated,
* Make sure no nonsense has been extracted from master.h5.
* Make sure INCIDENT_BEAM_DIRECTION= corresponds to the experimental geometry.
* Point LIB= to where Neggia is saved (if in current directory, use <code>LIB=./dectris-neggia.so</code> i.e. specify directory!).
** Comment out LIB= if Neggia isn't used (not recommended).
* Set MAXIMUM_NUMBER_OF_JOBS= and MAXIMUM_NUMBER_OF_PROCESSORS= to similar values whose product is slightly smaller than the total number of threads on your system.


= Less efficient way of processing Eiger data, using conversion to CBF =


== See also ==
Since the release of Neggia, a plugin for XDS that parallelizes the reading of images from HDF5 data, conversion by H5ToXds should no longer be required in most usage scenarios. The sections below nevertheless describe this possibility, since preliminary experience with some less common network file systems (apparently GPFS, but not NFS) seems to indicate low performance of Neggia.
 
Conversion program options: Dectris provides [https://www.dectris.com/news.html?page=2 H5ToXds] (Linux only!). That program converts (as the name indicates) the HDF5 files to CBF files; however, it does not write the geometry and other information into the CBF header (therefore, [[generate_XDS.INP]] or MOSFLM does not work with these files). Alternatives are GlobalPhasing's hdf2mini-cbf program (does ''not'' need autoPROC license) or, from http://www.mrc-lmb.cam.ac.uk/harry/imosflm/ver721/downloads, the eiger2cbf-osx or eiger2cbf-linux program written by T. Nakane. The latter programs do write a useful CBF header.
 
H5ToXds and eiger2cbf-osx / eiger2cbf-linux do not work with files produced at Diamond Light Source.
 
== A script for faster XDS processing of CBF-converted Eiger data (this is only shown out of historic interest) ==
 
For faster processing, the [[Eiger#A_script_for_faster_XDS_processing_of_CBF-converted Eiger data|shell script]] below should be copied to /usr/local/bin/H5ToXds and made executable (<code>chmod a+rx /usr/local/bin/H5ToXds*</code>). The binary H5ToXds then should be named e.g. /usr/local/bin/H5ToXds.bin - note the .bin filename extension! The script ''also'' uses RAM to speed up processing; it uses it for fast storage of the temporary CBF file that H5ToXds/eiger2cbf/hdf2mini-cbf writes, and that each parallel thread ("processor") of XDS reads. The amount of additional RAM this requires is modest (about (number of pixels)*(number of threads) bytes).
 
<pre>
#!/bin/bash
# Kay Diederichs 10/2015
# 3/2017 include RAMdisk creation for MacOS; only lightly tested!
# 3/2016 adapt for eiger2cbf and hdf2mini-cbf
# for the latter see https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;58a4ee1.1603 and
# https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ccp4bb;a048b4e8.1603
#
# Idea: put temporary files into fast local directory, instead of NFS
#
# Installation: Rename Dectris' H5ToXds to H5ToXds.bin
#              This script should be called H5ToXds and reside in $PATH
#              Modify this script according to which binary you use - see comments below.
#
# Recommendation:
# - for the fast local directory one should use a RAMdisk (one GB size at most)
# - /dev/shm seems to be already set up for that purpose on most Linux distributions
# - on MacOS you can easily set this up as described at http://stackoverflow.com/questions/2033362/does-os-x-have-an-equivalent-to-dev-shm
# example on MacOS for 1GB RAMdisk (needs to be repeated after booting):
# diskutil eraseVolume HFS+ RAMdisk $(hdiutil attach -nomount ram://$((2 * 1024 * 1000)))
#
# on MacOS the next line should then be:
# tempfile="/Volumes/RAMdisk/H5ToXds${PWD//\//_}.$3"
# and on Linux:
tempfile="/dev/shm/H5ToXds${PWD//\//_}.$3"
#
# choose between H5ToXds.bin,  eiger2cbf and hdf2mini-cbf; un/comment accordingly
/usr/local/bin/H5ToXds.bin $1 $2 "$tempfile" || rm "$tempfile"
#/usr/local/bin/eiger2cbf-linux $1 $2 "$tempfile" >& /dev/null  || rm "$tempfile"
#/usr/local/bin/eiger2cbf-osx $1 $2 "$tempfile" >& /dev/null  || rm "$tempfile"
#/usr/local/bin/hdf2mini-cbf $1 $2 "$tempfile"  || rm "$tempfile"
ln -sf "$tempfile" $3 2>/dev/null
</pre>
 
= See also =


[[Performance]]
[[Performance]]
[https://github.com/keitaroyam/yamtbx/blob/master/doc/eiger-en.md Keitaro Yamashita's Eiger page, with some emphasis on SPring-8]
Kay
Bureaucrats
2,651

edits

Retrieved from "https://wiki.uni-konstanz.de/xds/index.php/Special:MobileDiff/3493...4260"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.
More information

Navigation menu