2,652
edits
Eiger: Difference between revisions
Jump to navigation
Jump to search
fix upper limit for memory used for the cache if NUMBER_OF_IMAGES_IN_CACHE is _not_ specified
(fix upper limit for memory used for the cache if NUMBER_OF_IMAGES_IN_CACHE is _not_ specified) |
|||
| Line 2: | Line 2: | ||
== General aspects == | == General aspects == | ||
# The framecache of XDS uses memory to save on I/O; it saves a frame in RAM after reading it for the first time. By default, each XDS (or mcolspot/mintegrate) job stores NUMBER_OF_IMAGES_IN_CACHE=DELPHI/OSCILLATION_RANGE images in memory which corresponds to one DELPHI-sized batch of data. This requires (number of pixels)*(number of jobs)*4 Bytes per frame which amounts to 72 MB in case of the Eiger 16M when running with MAXIMUM_NUBER_OF_JOBS=1. (If DELPHI=20 and OSCILLATION_RANGE=0.05 your computer thus has to have at least 400*72MB = 29GB of memory for each job). If it has not, the fallback is to the old behaviour of reading each frame three times (instead of once). There is an upper limit ( | # The framecache of XDS uses memory to save on I/O; it saves a frame in RAM after reading it for the first time. By default, each XDS (or mcolspot/mintegrate) job stores NUMBER_OF_IMAGES_IN_CACHE=DELPHI/OSCILLATION_RANGE images in memory which corresponds to one DELPHI-sized batch of data. This requires (number of pixels)*(number of jobs)*4 Bytes per frame which amounts to 72 MB in case of the Eiger 16M when running with MAXIMUM_NUBER_OF_JOBS=1. (If DELPHI=20 and OSCILLATION_RANGE=0.05 your computer thus has to have at least 400*72MB = 29GB of memory for each job). If it has not, the fallback is to the old behaviour of reading each frame three times (instead of once). There is an upper limit (8GB) to the amount of memory that will be used by default; if the required memory is more than that, a message will be printed and the user must explicitly include a NUMBER_OF_IMAGES_IN_CACHE= line in XDS.INP. | ||
# Dectris provides a library [https://github.com/dectris/neggia] for native reading of HDF5 files, which can be loaded into XDS at runtime using the <code>LIB=</code> [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#LIB= keyword]. With this library, no conversion to CBF or otherwise is necessary. It is therefore just as fast and efficient to read HDF5 files as any other file format. | # Dectris provides a library [https://github.com/dectris/neggia] for native reading of HDF5 files, which can be loaded into XDS at runtime using the <code>LIB=</code> [http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#LIB= keyword]. With this library, no conversion to CBF or otherwise is necessary. It is therefore just as fast and efficient to read HDF5 files as any other file format. | ||
# The XDS BUILT=20170215 has a problem with constructing the master filename, see [[Eiger#Troubleshooting]]. | # The XDS BUILT=20170215 has a problem with constructing the master filename, see [[Eiger#Troubleshooting]]. | ||