Difference between revisions of "Xdscc12"

From XDSwiki
(fix links)
Line 1: Line 1:
XDSCC12 is a program for generating delta-CC<sub>1/2</sub> and delta-CC<sub>1/2</sub>ano values for XDS_ASCII.HKL (written by XDS), or for XSCALE.HKL containing several files of type XDS_ASCII.HKL after scaling in XSCALE (with MERGE=FALSE).  
+
XDSCC12 is a program for generating delta-CC<sub>1/2</sub> and delta-CC<sub>1/2</sub>ano values for XDS_ASCII.HKL (written by [[XDS]]), or for XSCALE.HKL containing several files of type XDS_ASCII.HKL after scaling in [[XSCALE]] (with MERGE=FALSE).  
  
 
It implements the method described in Assmann, Brehm and Diederichs (2016) Identification of rogue datasets in serial crystallography. J. Appl. Cryst. 49, 1021 [http://journals.iucr.org/j/issues/2016/03/00/zw5005/zw5005.pdf], and it does this not only for the individual datasets in XSCALE.HKL, but also for individual frames, or groups of frames, of a single dataset collected with the rotation method and processed by [[XDS]].
 
It implements the method described in Assmann, Brehm and Diederichs (2016) Identification of rogue datasets in serial crystallography. J. Appl. Cryst. 49, 1021 [http://journals.iucr.org/j/issues/2016/03/00/zw5005/zw5005.pdf], and it does this not only for the individual datasets in XSCALE.HKL, but also for individual frames, or groups of frames, of a single dataset collected with the rotation method and processed by [[XDS]].
Line 7: Line 7:
 
Usage (this text can be obtained with <code>xdscc12 -h</code>):
 
Usage (this text can be obtained with <code>xdscc12 -h</code>):
 
<pre>
 
<pre>
xdscc12 KD 2018-9-6. Academic use only; no redistribution. -h option shows options.
+
xdscc12 KD 2019-04-30. Academic use only; no redistribution. -h option shows options.
 
Please cite Assmann, G., Brehm, W., Diederichs, K. (2016) J.Appl.Cryst. 49, 1021-1028
 
Please cite Assmann, G., Brehm, W., Diederichs, K. (2016) J.Appl.Cryst. 49, 1021-1028
usage: xdscc12 [-dmin <lowres>] [-dmax <highres>] [-nbin <nbin>] [-mode <1 or 2>] [-<abcdeftwz>] [-r <ref>] FILE_NAME
+
running 'xdscc12 -h' on 20190502 at 16:11:46 +0200
dmax (default 999A), dmin (default 1A) and nbin (default 10) have the usual meanings.
+
usage: xdscc12 [-dmin <lowres>] [-dmax <highres>] [-nbin <nbin>] [-mode <1 or 2>] [-<abcdefstwz>] [-r <ref>] FILE_NAME
 +
dmin (default 999A), dmax (default 1A) and nbin (default 10) have the usual meanings.
 
mode can be 1 (equal volumes of resolution shells) or 2 (increasing volumes; default).
 
mode can be 1 (equal volumes of resolution shells) or 2 (increasing volumes; default).
  -t: total oscillation (degree) to batch fine-sliced frames into
+
  -r: next parameter: ASCII reference file with lines: h,k,l,Fcalc or h,k,l,Fcalc+,Fcalc-
  -r: next parameter: ASCII reference file with lines: h,k,l,Fcalc
+
      this allows calculation of CC of isomorphous signal with reference
 +
  -s: read two columns from reference: Fcalc(+), Fcalc(-).
 +
      this allows calculation of CC of anomalous signal with that of reference
 +
  -t: total oscillation (degree) to batch fine-sliced frames into
 
  FILE_NAME can be XDS or XSCALE reflection file
 
  FILE_NAME can be XDS or XSCALE reflection file
 
  other options can be combined (e.g. -def), and switch the following off:
 
  other options can be combined (e.g. -def), and switch the following off:
  -a: individual isomorphous summary values
+
  -a: individual isomorphous summary values
  -b: individual (Fisher-transformed) delta-CC1/2 values
+
  -b: individual (Fisher-transformed) delta-CC1/2 values
  -c: individual delta-CC1/2 reflection numbers
+
  -c: individual delta-CC1/2 reflection numbers
  -d: individual anomalous summary values
+
  -d: individual anomalous summary values
  -e: individual (Fisher-transformed) delta-CC1/2ano values
+
  -e: individual (Fisher-transformed) delta-CC1/2ano values
  -f: individual delta-CC1/2ano reflection numbers
+
  -f: individual delta-CC1/2ano reflection numbers
  -w: weighting of intensities with their sigmas
+
  -w: weighting of intensities with their sigmas
  -z: no Fisher transformation of delta-CC1/2 values
+
  -z: no Fisher transformation of delta-CC1/2 values
 
</pre>
 
</pre>
  
Line 32: Line 36:
 
Statistics are given (in resolution shells) for the isomorphous and the anomalous signal.
 
Statistics are given (in resolution shells) for the isomorphous and the anomalous signal.
  
Important: to identify outliers in [[XSCALE]]d data, you should use the -w option. Otherwise, a and b are adjusted such that the sigmas are very high, which reduces the delta-CC<sub>1/2</sub> signal.
+
Important: to more clearly identify outliers in [[XSCALE]]d data, you may want use the -w option. This can be helpful since the ''a'' and ''b'' parameters of the error model are adjusted by XSCALE such that the sigmas are very high, which reduces the delta-CC<sub>1/2</sub> signal.
  
 
A complete description of how to process serial crystallography data with XDS/XSCALE is given in [[SSX]]. A program that implements the method of [https://doi.org/10.1107/S1399004713025431 Brehm and Diederichs (2014)] and theory of [https://doi.org/10.1107/S2059798317000699 Diederichs (2017)] is [[xscale_isocluster]].
 
A complete description of how to process serial crystallography data with XDS/XSCALE is given in [[SSX]]. A program that implements the method of [https://doi.org/10.1107/S1399004713025431 Brehm and Diederichs (2014)] and theory of [https://doi.org/10.1107/S2059798317000699 Diederichs (2017)] is [[xscale_isocluster]].
  
== Preparing a reference data set ==
+
== Correlation against a reference data set (-r <reference> option) ==
If the refinement was done with phenix.refine, one could use
+
The correlation of the experimental data set against the user-supplied reference data is shown in the lines starting with r.
 +
To prepare a reference data set if the refinement was done with phenix.refine, one could use e.g.
 
<pre>
 
<pre>
 
mtz2various hklin 2bn3_refine_001.mtz hklout temp.hkl <<eof
 
mtz2various hklin 2bn3_refine_001.mtz hklout temp.hkl <<eof
Line 54: Line 59:
 
</pre>
 
</pre>
  
== Fcalc with anomalous signal as reference ==
+
=== Reference data with anomalous signal (additional -s option) ===
 +
The correlation of the anomalous difference of the experimental data set against the anomalous signal of the user-supplied reference data is shown in the lines starting with s.
 
A simple way to obtain Fcalc(+) and Fcalc(-) is to run <code>phenix.refine</code> with options (in case of S as anomalous scatterer)
 
A simple way to obtain Fcalc(+) and Fcalc(-) is to run <code>phenix.refine</code> with options (in case of S as anomalous scatterer)
  refinement.input.xray_data.labels="F(+),SIGF(+),F(-),SIGF(-),merged"  
+
  refinement.input.xray_data.labels="F(+),SIGF(+),F(-),SIGF(-),merged"  refinement.refine.anomalous_scatterers.group.selection="element S" strategy=individual_sites+individual_adp+group_anomalous+occupancies
  refinement.refine.anomalous_scatterers.group.selection="element S"
 
strategy=individual_sites+individual_adp+group_anomalous+occupancies
 
 
and then
 
and then
 
<pre>
 
<pre>
Line 68: Line 72:
 
eof
 
eof
 
</pre>
 
</pre>
in which case <code>sftools</code> outputs only the acentric reflections - only those have anomalous differences. <code>XDSCC12</code> then has to be run with the <code>-s -r anom-reference.hkl</code> option. This allows a direct comparison of CC Iobs-Icalc, with Icalc either as (Fcalc(+) + Fcalc(-))/2 (these are the output lines starting with r) or with Iobs(+) being compared with Fcalc(+)<sup>2</sup>, and Iobs(-) being compared with Fcalc(-)<sup>2</sup> (output lines starting with s). Preliminary insight: the r and s lines do not differ significantly.
+
in which case <code>sftools</code> outputs only the acentric reflections - only those have anomalous differences. <code>XDSCC12</code> then has to be run with the <code>-s -r anom-reference.hkl</code> option.

Revision as of 14:21, 2 May 2019

XDSCC12 is a program for generating delta-CC1/2 and delta-CC1/2ano values for XDS_ASCII.HKL (written by XDS), or for XSCALE.HKL containing several files of type XDS_ASCII.HKL after scaling in XSCALE (with MERGE=FALSE).

It implements the method described in Assmann, Brehm and Diederichs (2016) Identification of rogue datasets in serial crystallography. J. Appl. Cryst. 49, 1021 [1], and it does this not only for the individual datasets in XSCALE.HKL, but also for individual frames, or groups of frames, of a single dataset collected with the rotation method and processed by XDS.

The program can be downloaded for Linux or Mac.

Usage (this text can be obtained with xdscc12 -h):

xdscc12 KD 2019-04-30. Academic use only; no redistribution. -h option shows options.
Please cite Assmann, G., Brehm, W., Diederichs, K. (2016) J.Appl.Cryst. 49, 1021-1028
running 'xdscc12 -h' on 20190502 at 16:11:46 +0200
usage: xdscc12 [-dmin <lowres>] [-dmax <highres>] [-nbin <nbin>] [-mode <1 or 2>] [-<abcdefstwz>] [-r <ref>] FILE_NAME
dmin (default 999A), dmax (default 1A) and nbin (default 10) have the usual meanings.
mode can be 1 (equal volumes of resolution shells) or 2 (increasing volumes; default).
  -r: next parameter: ASCII reference file with lines: h,k,l,Fcalc or h,k,l,Fcalc+,Fcalc-
      this allows calculation of CC of isomorphous signal with reference
  -s: read two columns from reference: Fcalc(+), Fcalc(-). 
      this allows calculation of CC of anomalous signal with that of reference
  -t: total oscillation (degree) to batch fine-sliced frames into
 FILE_NAME can be XDS or XSCALE reflection file
 other options can be combined (e.g. -def), and switch the following off:
  -a: individual isomorphous summary values
  -b: individual (Fisher-transformed) delta-CC1/2 values
  -c: individual delta-CC1/2 reflection numbers
  -d: individual anomalous summary values
  -e: individual (Fisher-transformed) delta-CC1/2ano values
  -f: individual delta-CC1/2ano reflection numbers
  -w: weighting of intensities with their sigmas
  -z: no Fisher transformation of delta-CC1/2 values

The program output is terse but supposed to be self-explanatory. The isomorphous delta-CC1/2 of a batch of frames (width chosen with the -t option) relative to all data is most easily visualized via XDSGUI (Statistics tab); the anomalous delta-CC1/2 may be plotted with e.g. gnuplot after grepping the relevant lines from the output.

For multiple datasets, the output lines show the contribution of each dataset toward the total CC1/2. Negative numbers indicate a worsening of the overall signal.

Statistics are given (in resolution shells) for the isomorphous and the anomalous signal.

Important: to more clearly identify outliers in XSCALEd data, you may want use the -w option. This can be helpful since the a and b parameters of the error model are adjusted by XSCALE such that the sigmas are very high, which reduces the delta-CC1/2 signal.

A complete description of how to process serial crystallography data with XDS/XSCALE is given in SSX. A program that implements the method of Brehm and Diederichs (2014) and theory of Diederichs (2017) is xscale_isocluster.

Correlation against a reference data set (-r <reference> option)

The correlation of the experimental data set against the user-supplied reference data is shown in the lines starting with r. To prepare a reference data set if the refinement was done with phenix.refine, one could use e.g.

mtz2various hklin 2bn3_refine_001.mtz hklout temp.hkl <<eof
OUTPUT USER *
LABIN FC=F-model PHIC=PHIF-model
END
eof

- the column corresponding to PHIC will not be used by xdscc12. Alternatively,

sftools
read mymodel_001.mtz
write temp.hkl format(3i5,f10.3) col F-model
y
quit

Reference data with anomalous signal (additional -s option)

The correlation of the anomalous difference of the experimental data set against the anomalous signal of the user-supplied reference data is shown in the lines starting with s. A simple way to obtain Fcalc(+) and Fcalc(-) is to run phenix.refine with options (in case of S as anomalous scatterer)

refinement.input.xray_data.labels="F(+),SIGF(+),F(-),SIGF(-),merged"  refinement.refine.anomalous_scatterers.group.selection="element S" strategy=individual_sites+individual_adp+group_anomalous+occupancies

and then

sftools <<eof
read mymodel_001.mtz
write anom-reference.hkl format(3i5,2f10.3) col "F-model(+)" "F-model(-)"
y
quit
eof

in which case sftools outputs only the acentric reflections - only those have anomalous differences. XDSCC12 then has to be run with the -s -r anom-reference.hkl option.