XDSwiki - Recent changes [en]

XDSgui

2013-06-17T11:23:45Z

‎TOOLS:

NORMAL.HKL

2013-06-14T13:38:08Z

Created page with "This is the description of filetype NORMAL.HKL (used until 2002). Files of this type may be used as REFERENCE_DATA_SET which may be useful in specific situations. ..."

New page

This is the description of filetype NORMAL.HKL (used until 2002).
Files of this type may be used as REFERENCE_DATA_SET which may be useful in specific situations.

NORMAL.HKL (ASCII, formatted sequential)

Description of file format NORMAL (or OLDHKL) as produced by XDS
(NORMAL.HKL) and XSCALE (XSCALE.HKL) for representing unique
intensities in the absence of anomalous scattering effects.

Symmetry related reflections including Friedel-pairs are averaged
and recorded by FORTRAN FORMAT(3I5,4E12.4).

Each record consists of the following items

h,k,l,I,SDI

h,k,l - unique reflection indices.
h=10000 indicates the last record in file.

I,SDI - Mean weighted intensity and its standard deviation
of all reflections symmetry related to h, k, l or
-h,-k,-l
In case SDI is missing on an input file of this type,
XSCALE will assume SDI=0.1*I.

Indexing

2013-06-13T15:00:02Z

‎Indexing images from non-merohedrally twinned crystals (i.e. several lattices):

Generate XDS.INP

2013-06-12T10:17:23Z

‎Variant script (xds_generate_all): remove

Show changes

XDSgui

2013-06-11T12:52:25Z

‎Frame:

Generate XDS.INP

2013-06-11T12:31:42Z

version 0.35 - document use of INCLUDE/EXCLUDE_RESOLUTION_RANGE by IDXREF; lower DEFPIX cutoff from 7000 to 6000

XDSgui

2013-06-11T05:18:03Z

‎Known bugs / problems / workarounds: spaces in directory names

@@ Line 2: / Line 2: @@
 It graphically displays the ASCII and [http://www.bernstein-plus-sons.com/software/CBF cbf] files that XDS writes, and can run useful shell commands with a simple mouse click. The design goal of the program is to enable XDS data processing without the commandline, and to supply additional graphical information, in a simple, user-modifiable and user-extensible way.
-The main window of XDSgui has the following sub-windows organized as tabs:
+== How to use the GUI ==
-== Projects ==
+=== Projects ===
 stores names of directories which contain files from XDS processing. If none of these is selected, the current directory is used. A new directory may be created. The list of project directories is saved to ~/.xds-gui .
-== Frame ==
+=== Frame ===
-to display existing 2-dimensional data frames, or to start XDS data processing by displaying a raw data frame. In the latter case, the "generate XDS.INP" button runs the [[generate_XDS.INP]] script for the raw data frame that is being displayed. The blue circles correspond to the areas of the detector that are within TRUSTED_REGION; the green circles correspond to INCLUDE_RESOLUTION_RANGE, and the red hatched regions correspond to EXCLUDE_RESOLUTION_RANGEs. The locations of the circles are not as accurate as those that XDS uses internally, because a simplified formula (and only values from XDS.INP, not the refined ones from XPARM.XDS) is used to calculate the resolution; this e.g. does not take care of detector swingout or otherwise skew geometry. Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab.
+to display existing 2-dimensional data frames, or to start XDS data processing by displaying a raw data frame. In the latter case, the "generate XDS.INP" button runs the [[generate_XDS.INP]] script for the raw data frame that is being displayed. The blue circles correspond to the areas of the detector that are within TRUSTED_REGION; the green circles correspond to INCLUDE_RESOLUTION_RANGE, and the red hatched regions correspond to EXCLUDE_RESOLUTION_RANGEs. The locations of the circles are not as accurate as those that XDS uses internally, because a simplified formula (and only values from XDS.INP, not the refined ones from XPARM.XDS) is used to calculate the resolution; this e.g. does not take care of detector swingout or otherwise skew geometry.
-== XDS.INP ==
+Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab.
 this shows the current [[XDS.INP]] file. Coloured items correspond to the circles of the FRAME tab. Changes can be made with the simple editor, and the new version can be saved. A run of xds_par can be started (and killed). <br>
 The XDS.INP tab is directly connected with the FRAME tab; changing a value in the XDS.INP tab results in immediate change in the FRAME tab. This means that e.g. an UNTRUSTED area that was positioned wrongly can be edited or simply removed in the XDS.INP tab.
-== Output from XDS ==
+=== XYCORR ===
+=== INIT ===
-=== XYCORR, INIT, COLSPOT, IDXREF, DEFPIX ===
+=== COLSPOT ===
 each tab shows its .LP file, respectively.
@@ Line 34: / Line 52: @@
 The border between the left and right side of the window may be adjusted with the mouse.<br>
-== TOOLS ==
+=== TOOLS ===
 offers possibilities for running short scripts.
@@ Line 40: / Line 58: @@
 Several such scripts are pre-defined; the user may create her own scripts. If scripts are modified, they are saved to ~/.xds-gui . The names of the buttons (e.g. "User defined command 1") can be changed by editing ~/.xds-gui .
-The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" should be replaced by "cp -p" since FAT32 does not support symlinks.
+* The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" should be replaced by "cp -p" since FAT32 does not support symlinks.
-The second item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDS-gui (which is always the case) . Thus, one should simply press the "Reload" button.
+* The second item ("Optimizing data quality") offers commands that manipulate [[XDS.INP]] in several ways. Please note: the popup "XDS.INP has been changed externally" is emitted by the Qt system and cannot be switched off. It appears if one of the scripts changes XDS.INP while it is opened by XDS-gui (which is always the case) . Thus, one should simply press the "Reload" button. The user-definable commandline may be used e.g. for
-The third item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory.
+* The third item ("Saving and comparing good results") offers commands to save/restore the current data processing files to/from a "save" directory. Make sure to replace "xdiff" with "xxdiff" or "pkdiff", if one of the latter is available.
-The last item ("Further analyses") may be used for commands that e.g. run [[pointless]] against the XDS_ASCII.HKL file.
+* The last item ("Further analyses") may be used for commands, e.g. running [[pointless]] against the XDS_ASCII.HKL file. The user-definable commandline may be used e.g. for
-== XDSSTAT ==
+=== XDSSTAT ===
 This tab allows to run [[XDSSTAT]] at the press of a button. As soon as XDSSTAT.LP exists, it displays the main plots derived from it. Another button allows to run [[XDS-viewer]] to visualize the control images that XDSSTAT generates, namely anom.pck, scales.pck, rf.pck, misfits.pck, corr.pck, rlps.pck, peaks.pck and nobs.pck .
-== XDSCONV ==
+=== XDSCONV ===
 a simple XDSCONV.INP is provided by default. It may be edited by the user, and saved. Upon clicking a button, [[xdsconv]] is run and if necessary a MTZ file is  produced. XDSCONV.LP may be displayed.
@@ Line 60: / Line 80: @@
 The program is under development and probably has bugs. If it crashes, it should simply be restarted. A crash of the program does not interfere with the operation of [[XDS]].
-The current version of the program can be downloaded from [ftp://turn5.biologie.uni-konstanz.de/pub/xdsGUI.rhel6.64] (Linux 64bit), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsGUI.rhel6.32] (Linux 32bit), or [ftp://turn5.biologie.uni-konstanz.de/pub/xds-gui.dmg] (Mac OSX 10.7).
+The current version of the program can be downloaded, by academic users, from [ftp://turn5.biologie.uni-konstanz.de/pub/xdsGUI.rhel6.64] (Linux 64bit), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsGUI.rhel6.32] (Linux 32bit), or [ftp://turn5.biologie.uni-konstanz.de/pub/xds-gui.dmg] (Mac OSX 10.7). Industrial users of XDS: pls contact me directly.
 === Dependencies ===
@@ Line 68: / Line 88: @@
 Technically, the word "dependency" means that these scripts/programs should be in your $PATH - an 'alias' is often not good enough because the TOOLS commands use bash as a shell, and that shell does not inherit the aliases from e.g. (t)csh.
-== Known bugs / problems / workarounds ==
+=== Installation ===
 # ''Question concerning TOOLS: "Show frame..."  on Mac: it creates everything shown in the commandline but it seem to us that everything runs in the background. Therefore xds-viewer does not open anywhere as it is not seen. Also, it would be great if this image together with the predictions would be re-directed to the FRAME tab.'' <br> Answer: On Linux, the xds-viewer window is brought to the foreground automatically, whereas on the Mac this does not seem to happen. However, an icon for xds-viewer appears in the dock (on the right) and I can double-click it to see the xds-viewer window. I have no idea how to bring xds-viewer to the foreground automatically, and I need input from people who know Macs and tell me how to do it (on the commandline!). The reason why we do not open automatically in the FRAME tab is that one can have several xds-viewer windows open at the same time, and compare the patterns. As a '''workaround''', you can manually open the resulting temp/FRAME_$X.cbf in the FRAME tab.
 # ''Another question concerning TOOLS: "Further analyses" on Mac: again pointless runs nicely but there is no output for the user.'' <br> Answer: Could it be that you started xds-gui by double-clicking its icon in the Finder? In that case, there is no output visible, because there is no console. Unfortunately, for all the Tools, the output is in the console window where xds-gui was started. We will try to find out how to open a window where the output is then shown. Thus, currently the '''workaround''' for this problem is to start xds-gui in a console window.

@@ Line 15: / Line 15: @@
 '''3.''' split the indexed/non-indexed spots:
-  % grep " 0       0       0" SPOT.XDS > SPOT.XDS_bad
+  % grep " 0 0 0" SPOT.XDS > SPOT.XDS_bad
-  % grep -v " 0       0       0" SPOT.XDS > SPOT.XDS_good
+ % # for old XDS versions use: grep " 0       0       0" SPOT.XDS > SPOT.XDS_bad
   % mv SPOT.XDS SPOT.XDS_orig
   % cp -p SPOT.XDS_good SPOT.XDS
@@ Line 22: / Line 23: @@
   % xds
 Now you have (basically) indexed lattice 1. You could repeat this step a bit further:
-  % grep  -v " 0       0       0" SPOT.XDS > SPOT.XDS_tmp
+  % grep  -v " 0 0 0" SPOT.XDS > SPOT.XDS_tmp
   % mv SPOT.XDS_tmp SPOT.XDS
   % xds
@@ Line 34: / Line 35: @@
   % xds
 Are there still too many spots rejected? Check with
-  % grep -c " 0       0       0" SPOT.XDS    # gives non-indexed spots
+  % grep -c " 0 0 0" SPOT.XDS    # gives non-indexed spots
-  % grep -vc " 0       0       0" SPOT.XDS   # gives indexed spots
+  % grep -vc " 0 0 0" SPOT.XDS   # gives indexed spots
 Maybe you have a third (fourth etc) lattice? Then repeat step 3 for this one ... etc

@@ Line 11: / Line 11: @@
 == Frame ==
-to display existing 2-dimensional data frames, or to start XDS data processing by displaying a raw data frame. In the latter case, the "generate XDS.INP" button runs the [[generate_XDS.INP]] script for the raw data frame that is being displayed. The blue circles correspond to the areas of the detector that are within TRUSTED_REGION; the green circles correspond to INCLUDE_RESOLUTION_RANGE, and the red hatched regions correspond to EXCLUDE_RESOLUTION_RANGEs. The locations of the circles are not as accurate as those that XDS uses internally, because a simplified formula is used to calculate the resolution; this e.g. does not take care of detector swingout or otherwise skew geometry. Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab.
+to display existing 2-dimensional data frames, or to start XDS data processing by displaying a raw data frame. In the latter case, the "generate XDS.INP" button runs the [[generate_XDS.INP]] script for the raw data frame that is being displayed. The blue circles correspond to the areas of the detector that are within TRUSTED_REGION; the green circles correspond to INCLUDE_RESOLUTION_RANGE, and the red hatched regions correspond to EXCLUDE_RESOLUTION_RANGEs. The locations of the circles are not as accurate as those that XDS uses internally, because a simplified formula (and only values from XDS.INP, not the refined ones from XPARM.XDS) is used to calculate the resolution; this e.g. does not take care of detector swingout or otherwise skew geometry. Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab.
 == XDS.INP ==

@@ Line 50: / Line 50: @@
 # revision 0.33 . Keitaro 5/2013 - automatically set ROTATION_AXIS=-1 0 0 for SPring-8 BL32XU/41XU/44XU beamlines based on detector serial numbers.
 # revision 0.34 . Keitaro 5/2013 - recognize ADSC detectors in Photon Factory and choose correct beam center convention based on detector serial numbers.
-REVISION="0.34 (27-May-2013)"
+# revision 0.35 . KD 6/2013 - reduce 7000 to 6000 for shadow detection; insert comment about *_RESOLUTION_RANGE lines
+#
 # usage: e.g. generate_XDS.INP "frms/mydata_1_???.img"
@@ Line 328: / Line 329: @@
 UNIT_CELL_CONSTANTS= 70 80 90 90 90 90 ! put correct values if known
 INCLUDE_RESOLUTION_RANGE=50 0  ! after CORRECT, insert high resol limit; re-run CORRECT
 FRIEDEL'S_LAW=FALSE     ! This acts only on the CORRECT step
@@ Line 345: / Line 347: @@
 ! parameters with changes wrt default values:
 TRUSTED_REGION=0.00 1.2  ! partially use corners of detectors; 1.41421=full use
-VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS=7000. 30000. ! often 8000 is ok
+VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS=6000. 30000. ! often 7000 or 8000 is ok
 STRONG_PIXEL=4           ! COLSPOT: only use strong reflections (default is 3)
 MINIMUM_NUMBER_OF_PIXELS_IN_A_SPOT=3 ! default of 6 is sometimes too high
@@ Line 817: / Line 819: @@
 INCLUDE_RESOLUTION_RANGE=50 0  ! After CORRECT, insert high resol limit; re-run CORRECT
-!                                Ice rings can be excluded from scaling with the keyword
+!                            Ice rings can be excluded from indexing and scaling with the keyword
-!                                EXCLUDE_RESOLUTION_RANGE, e.g. EXCLUDE_RESOLUTION_RANGE=2.28 2.22
+!                            EXCLUDE_RESOLUTION_RANGE, e.g. EXCLUDE_RESOLUTION_RANGE=2.28 2.22
 !SECONDS=60 !Uncomment this line to tell XDS to wait 1min for the next image before aborting.
@@ Line 848: / Line 850: @@
 TRUSTED_REGION=0.00 1.2  ! Partially use corners of detectors; 1.41421=full use, 1.0=edge
 !****************************************************************************************
 !Other parameters:
 !****************************************************************************************
-VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS=7000. 30000. ! Often 8000 is ok.
+VALUE_RANGE_FOR_TRUSTED_DETECTOR_PIXELS=6000. 30000. ! Often 7000 or 8000 is ok.
 STRONG_PIXEL=4           ! COLSPOT: only use strong reflections (default is 3; 6 may be better for strong data)
 MINIMUM_NUMBER_OF_PIXELS_IN_A_SPOT=3 ! default of 6 is sometimes too high

@@ Line 71: / Line 71: @@
 # ''Question concerning TOOLS: "Show frame..."  on Mac: it creates everything shown in the commandline but it seem to us that everything runs in the background. Therefore xds-viewer does not open anywhere as it is not seen. Also, it would be great if this image together with the predictions would be re-directed to the FRAME tab.'' <br> Answer: On Linux, the xds-viewer window is brought to the foreground automatically, whereas on the Mac this does not seem to happen. However, an icon for xds-viewer appears in the dock (on the right) and I can double-click it to see the xds-viewer window. I have no idea how to bring xds-viewer to the foreground automatically, and I need input from people who know Macs and tell me how to do it (on the commandline!). The reason why we do not open automatically in the FRAME tab is that one can have several xds-viewer windows open at the same time, and compare the patterns. As a '''workaround''', you can manually open the resulting temp/FRAME_$X.cbf in the FRAME tab.
 # ''Another question concerning TOOLS: "Further analyses" on Mac: again pointless runs nicely but there is no output for the user.'' <br> Answer: Could it be that you started xds-gui by double-clicking its icon in the Finder? In that case, there is no output visible, because there is no console. Unfortunately, for all the Tools, the output is in the console window where xds-gui was started. We will try to find out how to open a window where the output is then shown. Thus, currently the '''workaround''' for this problem is to start xds-gui in a console window.
 If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , ideally with enough information/data to reproduce the bug.