Cookies help us deliver our services. By using our services, you agree to our use of cookies. More information


From XDSwiki


5,270 bytes added, 21:17, 26 October 2020
IDXREF prints !!! WARNING !!! message
== XDS crashes ==
XDS should never crash (if it terminates with an error message, this does not count as crash). If it does, it is either a bug in the program which should be brought to the attention of Wolfgang Kabsch or Kay Diederichs, and will be fixed, or it is a problem with your computer, or (rarely) it is due to improper inputs ([[Problems#INTEGRATE_cell_and.2For_distance_run_away.3B_xds_crashes_or_has_to_be_killed|this]] is the only example I know of).
If it crashes for the second reason, these are the things to try/consider:
ulimit -s 102400
The numbers above mean a 10-fold increase over the default, and should be enough. I've found this to be necessary for unusually large frames (32 MB).
xds_par in this case also might need an increase of the environment variable OMP_STACKSIZE (e.g. "setenv OMP_STACKSIZE 128M").
=== memory consumption exceeding xds_par in this case also might need an increase of the 4GB barrier when using the 32bit binary ===environment variable OMP_STACKSIZE (e.g. "setenv OMP_STACKSIZE 128M").
As pointed out by James Holton, the memory consumed by huge frames in combination with many threads may be larger than 32 bits allow. Execution then stops with forrtl: severe (41): insufficient virtual memory The workaround is According to limit the [http://homesstackoverflow.mpimf-heidelberg.mpg.decom/~kabschquestions/xds13264274/html_doc/xds_parameters.html MAXIMUM_NUMBER_OF_PROCESSORS], or to use XDS instead of XDS_PAR. An even better solution why-segmentation-fault-is to switch to -happening-in-this-openmp-code one should also check the 64bit binary, but that of course requires a 64bit operating system"virtual address space limit" with ulimit (increase with ulimit -v) .
=== Problems with OpenMP ===
If the "xds_par" binary crashed, try "xds". xds_par uses OpenMP for parallelization, which adds complexity. If it works with xds, but not with xds_par, then there is a chance that some environment variable needs to be set/changed. In any case the XDS developers would like to learn about this.
=== limitation of the parallel 32bit binary === The 32bit OpenMP binary may not be able to allocate enough storage when using many threads (e.g. >10). The error message might be something like:  OMP: Error #34: System unable to allocate necessary resources for OMP thread: OMP: System error #11: Resource temporarily unavailable === 64bit binary on a 32 bit 32bit operating system ===
If the error message is e.g.
-bash: /usr/local/bin/xds: cannot execute binary file
make sure to use the 32bit version of XDS instead - you try to run the 64bit version on a 32bit operating system; this won't work. === CPU without SSE2 support === If the error message is  forrtl: severe (168): Program Exception - illegal instruction Image PC Routine Line Source xds 08055250 Unknown Unknown Unknown xds 0804B3F6 Unknown Unknown Unknown this means that unfortunately the CPU of your machine is too old Since October 2015, you need a 64bit Linux operating system to be supported by run XDS (which in this respect because the 32bit version is compiled with the default options of the ifort compiler). This may happen e.g. with old AMD CPUs that don't support SSE2 (on Linux check the ''flags'' field of /proc/cpuinfo for the presence of the sse2 flag)no longer provided.
According to [] the following Intel-compatible CPUs did not implement SSE2 after SSE2 was introduced (2001):
* AMD CPUs prior to Athlon 64, including all Socket A-based CPUs
* Intel CPUs prior to Pentium 4
* VIA C3
* Transmeta Crusoe
since otherwise, FRAME.cbf is overwritten by INTEGRATE.
To look at '''all''' spot positions found by COLSPOT, you could try echo "set yrange can use the COLSPOT tab of [[XDSGUI]] reverse ; plot 'SPOT.XDS' us 1:2 w dots" | gnuplot -persistThis nicely shows ice rings, and may also help to find shaded regions on the detector. === ugly diffraction pattern === If the spots are split or unclean, you may get a seemingly terrible match between observed and calculated spot positions: ***** REFINED SOLUTION BASED ON INDEXED REFLECTIONS IN SUBTREE # 1 ***** REFINED VALUES OF DIFFRACTION PARAMETERS DERIVED FROM 2916 INDEXED SPOTS REFINED PARAMETERS: AXIS BEAM ORIENTATION CELL STANDARD DEVIATION OF SPOT POSITION (PIXELS) 11.74 STANDARD DEVIATION OF SPINDLE POSITION (DEGREES) 5.94which leads to very many reflections being rejected from refinement: ***** INDEXING OF OBSERVED SPOTS IN SPACE GROUP # 1 ***** 1594 OUT OF 27897 SPOTS INDEXED. 0 REJECTED REFLECTIONS (REASON: OVERLAP) 26303 REJECTED REFLECTIONS (REASON: TOO FAR FROM IDEAL POSITION)The fix is to e.g. double the parameters MAXIMUM_ERROR_OF_SPOT_POSITION= 6.0 MAXIMUM_ERROR_OF_SPINDLE_POSITION= 4.0which then leads to less rejections: ***** INDEXING OF OBSERVED SPOTS IN SPACE GROUP # 1 ***** 11669 OUT OF 27897 SPOTS INDEXED. 2 REJECTED REFLECTIONS (REASON: OVERLAP) 16226 REJECTED REFLECTIONS (REASON: TOO FAR FROM IDEAL POSITION)and surprisingly good integration.
=== IDXREF ends with !!! ERROR !!! message ===
This is printed out for you to actually read, and take action accordingly. In most cases you just change the JOBS - JOB= line in XDS.INP to read
and then continue to run XDS. In other cases you may want to change SPOT.XDS, or other keywords in [[XDS.INP]] (see also below). But in any case this is an important alert that should make you check the correctness of the parameters that describe the data collection (X-RAY_WAVELENGTH, DETECTOR_DISTANCE, ORGX, ORGY, OSCILLATION_RANGE, NAME_TEMPLATE_OF_DATA_FRAMES).
==== Things to watch out for in IDXREF.LP =produces too short cell parameter(s) ===
The first table "CLUSTER COORDINATES AND INDICES WITH RESPECT TO REC. LATTICE BASIS VECTORS" should show close-to-integer difference vectors. The following is an example how it should ''not'' look like:
1 0.0004199-0.0012633-0.0043826 2442. 0.97 -0.04 -0.00
2 0.0101757 0.0060701-0.0019630 2175. -0.03 -1.02 0.01
3 -0.0076118 0.0114167-0.0040603 1965. 0.13 0.04 0.50 <---- half-integer indices!?
4 0.0100552 0.0071646 0.0025337 1944. -1.01 -0.99 0.01
5 0.0072840-0.0101455 0.0084405 1841. -1.10 -0.01 -0.49 <---- half-integer indices!?
6 0.0000976 0.0027828 0.0089584 1792. -2.00 0.00 -0.00
7 0.0103103 0.0043851-0.0063902 1790. 0.98 -1.02 0.00
8 0.0025742 0.0163995-0.0098507 1760. 0.95 -0.99 0.51 <---- half-integer indices!? 9 0.0068686-0.0089437 0.0128884 1724. -2.08 0.03 -0.49 <---- half-integer indices!? 10 -0.0174443 0.0123504 0.0161324 1694. -4.01 0.99 0.52 <---- half-integer indices!? 11 0.0272195-0.0050005-0.0136142 1678. 2.99 -1.97 -0.50 <---- half-integer indices!?
34.71 84.72 213.67 90.35 90.27 90.06
Do you see the half-integer difference vectors in the last column? This clearly indicates that the third axis above the table (with 1/length=34.71) actually should be twice that size. If this happens, the histogram of indexed spots often has two equally-large subtrees:
5 6
Remedy: take the reduced cell (as found below the table), but double the 34.71, and stick it into the line
UNIT_CELL_CONSTANTS=69.42 84.72 213.67 90.35 90.27 90.06
in XDS.INP. To make XDS actually ''use '' that line, specify
and leave the space group determination for later.
Alternatively, try several well-separated SPOT_RANGEs instead of 1. For example, instead of
SPOT_RANGE=801 900
In most the caseswhere I've tried this, this makes splitting of SPOT_RANGE into several wedges always resulted in IDXREF pick picking up the correct cell.  The reason for the indexing failure in these cases seems to be the fact that the crystal changed its orientation within the SPOT_RANGE by more than ~ 0.1°. '''The other reason for this failure mode''' - but in this case there is only one SUBTREE with high POPULATION - is when the default <code>SEPMIN= 6.00 CLUSTER_RADIUS= 3</code> are used, but the spots are closer than 6 pixels. This often happens with Pilatus data; the recommendation for this detector is to use <code>SEPMIN= 4.00 CLUSTER_RADIUS= 2</code> in XDS.INP, or even <code>SEPMIN= 2.00 CLUSTER_RADIUS= 1</code>.  === Difference vectors are neither integers nor halfs === This can happen if SPACE_GROUP_NUMBER is wrong, i.e. the user "forces" a lattice (e.g. body-centered) that does not match the true one (which may be primitive). So the first thing to try is SPACE_GROUP_NUMBER=0. Sometimes, IDXREF nevertheless finds no good lattice:  # COORDINATES OF VECTOR CLUSTER FREQUENCY CLUSTER INDICES 1 0.0090336-0.0044767 0.0137041 1636. 0.99 0.99 -0.00 2 -0.0239422 0.0085088 0.0078304 1556. -2.00 1.00 0.00 3 0.0088799 0.0092726 0.0077661 1555. 0.50 0.50 -0.56 4 -0.0000164 0.0138382-0.0058215 1486. -0.49 -0.49 -0.56 <---- 0.56 is neither close to 0.5 nor to 0 or 1 5 -0.0148222 0.0040131 0.0216634 1468. -1.00 2.00 -0.00 6 0.0098829 0.0070246-0.0042191 1430. 0.50 -0.50 -0.44 <---- same here 7 -0.0030325 0.0110238 0.0056577 1422. -0.50 0.50 -0.44 <---- and here ... If it is not a case of SEPMIN and CLUSTER_RADIUS being too large (see above), try to increase INDEX_ERROR - in this case, it indexes beautifully with INDEX_ERROR=0.14 . === IDXREF produces too long axes === This may (rarely) happen when MAXIMUM_NUMBER_OF_JOBS is > 1. In this case, different JOBS' COLSPOT runs may report some reflections twice in SPOT.XDS. Since their phi values are close, they correspond to long unit cell parameters. This happens more easily if the mosaicity is high and therefore reflections extend over the borders between JOBs.
I have seen this working well in This effect may be mitigated by having as many cases of too short cell parametersSPOT_RANGEs as JOBs, and leaving gaps between the SPOT_RANGEs.
=== IDXREF prints !!! WARNING !!! message ===
If you see
in XDS.INP. If it is commented out with a !, remove the !. Then, change it to have
REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS ! DISTANCEPOSITIONi.e. remove DISTANCE POSITION from the list of refinable parameters. Once that is done, save XDS.INP and re-run the IDXREF step. This problem occurs if the POSITION value (called DISTANCE in former XDS versions) is large, and XDS cannot refine it meaningfully.
If this does not help, try to refine even less items, e.g. leave out AXIS. === IDXREF prints ERROR IN REFINE !!! RETURN CODE IS IER= 0 === This problem occurs is due to either wrong inputs in XDS.INP, or due to bad data, e.g. spots from many crystals whose diffraction patterns overlap. The first possibility applies if the DISTANCE diffraction pattern is largeclean. The second applies if the diffraction pattern is ugly, and XDS cannot refine it meaningfullyunsuitable for indexing. In that case, maybe try a different SPOT_RANGE.
=== IDXREF.LP does not show the expected lattice ===
By specifying the spacegroup and unitcell, you tell XDS that it should index based on those reflections that match that spacegroup and unitcell. In a sense, you _force_ that spacegroup and cell. This will discard other (spurious?) reflections, and usually leads to a clean list of Bravais lattice possibilities.
If the data reduction fails nevertheless (in terms of bad R-factors and [[CORRECT#An estimate for the overall quality of an experimental setup|ISa]] in the [[CORRECT]] step), then chances are that you specified some parameter wrongly, or not accurate enough (ORGY ORGX and ORGY are the most likely candidates), or that the crystal does not match your idea about its spacegroup and unit cell. Actually the latter happens pretty frequently (which is why it is always the safest way to collect 180° of spindle rotation unless you know your crystals very well; this is also a good strategy in other respects).
If you tell XDS that you do not to know the spacegroup (SPACE_GROUP_NUMBER= 0), then [[IDXREF]] takes all observed reflections into account. By design, the spacegroup decision is then postponed until the CORRECT step, or rather to a run of [[pointless]] after CORRECT, and it may be not vital to closely inspect IDXREF.LP, because CORRECT.LP has basically the same information, plus more.
=== none of the lattices in IDXREF.LP (except aP) has a good QUALITY OF FIT ===
== Integration failure ==
=== INTEGRATE stops with error message ===
If INTEGRATE stops after e.g.
then there are two ways to resolve this:* you could insert a line DELPHI=10 (or DELPHI=20, for 10 or 20 degrees batches; the default is 5 degree batches) into XDS.INP and re-run INTEGRATE* as the error message suggests, you should reduce the upper limit of the DATA_RANGE, to stop before the problematic frames, and re-run INTEGRATE. In this example, you would modify XDS.INP to have
Save XDS.INP, run XDS and inspect INTEGRATE.LP, to find the lines (e.g.)
Copy them to XDS.INP. Restore the original DATA_RANGE and continue.
If this problem happens with multiple XDS jobs working on the same data set, you might also get a message like
Image PC Routine Line Source
xds_par 0000000000592B91 Unknown Unknown Unknown
xds_par 00000000004BEF21 joinintegrate_ 8995 MAIN_XDS.f90
xds_par 0000000000410A5B xds_ 21690 MAIN_XDS.f90
xds_par 000000000040B3A4 MAIN__ 1 MAIN_XDS.f90
xds_par 00000000004083F6 Unknown Unknown Unknown 00007FB40222E830 Unknown Unknown Unknown
xds_par 00000000004082E9 Unknown Unknown Unknown
which looks like a real crash of the program, but in this case with a known reason.
Another error mode of INTEGRATE (in processing of small-molecule data) is ...
The error message is misleading in this case: there are too few reflections to build the average profile. The fix is: restart INTEGRATE after inserting e.g.
DELPHI=20 ! default is 5, so try with e.g. 10, 20, 45, 90, 180
in XDS.INP and re-run INTEGRATE. === INTEGRATE cell and/or distance run away; xds crashes or has to be killed ===If during INTEGRATE the cell keeps increasing or the distance decreasing or both, then xds starts to consume large amounts of memory and becomes very slow. This may finally exhaust all available memory, and the job either crashes or has to be killed by the user. The fix here is to use REFINE(INTEGRATE)= BEAM POSITION ORIENTATION ! CELL instead of (what used to be the default until June 2017) REFINE(INTEGRATE)= BEAM POSITION ORIENTATION CELL Quite generally, the more conservative setting without CELL refinement is adequate unless your crystals diffract to quite high resolution. A sure sign that you should not be refining CELL is that the refined cell and/or distance values in INTEGRATE.LP fluctuate without obvious physical reason. Moreover, distance ("POSITION") refinement nicely soaks up any cell change resulting from radiation damage. A compromise (if you suspect that the cell parameters actually change ''differently'') would be REFINE(INTEGRATE)= BEAM ORIENTATION CELL ! POSITIONThis should also avoid the run-away but gives more freedom to the refinement. Of course it requires a refined distance (from IDXREF, or rather from GXPARM.XDS->XPARM.XDS renaming).
== See also ==
[[Low dose data]]

Navigation menu