Cookies help us deliver our services. By using our services, you agree to our use of cookies. More information

Changes

From XDSwiki

XDSGUI

7,423 bytes added, 29 June
Libraries and software that the program depends on: small fixes
XDSGUI is a GUI (graphical user interface) for XDS that is supposed to help both novice and experienced users.
It graphically displays the ASCII and [http://www.bernstein-plus-sons.com/software/CBF cbf] files that XDS writes, and can run useful shell commands with a simple mouse click. The design goal of the program is to enable XDS data processing without the commandline, and to supply additional graphical information, in a simple, user-modifiable and user-extensible way.
 
In its latest version, XDSGUI gives access to George Sheldrick's SHELXC, SHELXD and SHELXE programs, via the [[XDSGUI#SHELX|SHELX]] tab, and to Isabel Uson's ARCIMBOLDO programs, via the [[XDSGUI#ARCIMBOLDO|ARCIMBOLDO]] tab.
== How to use XDSGUI ==
After finally moving to the [[XDSGUI#XDS.INP|XDS.INP]] tab, other parameters can be adjusted by the user, and finally XDS may be run by clicking a button. The resulting output files from XDS will then be displayed in the next tabs.
After finishing this first round of processing, the [[XDSGUI#TOOLS|TOOLS]] tab may be used, which - among other things - offers those three options that I found most useful to optimize the data processing. After choosing (by mouse click) one of these three options, the user should go back to the [[XDSGUI#XDS.INP|XDS.INP]] tab, specify JOBSJOB=DEFPIX INTEGRATE CORRECT, and run XDS again. Ideally, each of the three options should be tried separately, and its effect should be compared with the previous processing to verify that it ''really'' improved the processed data. A significant increase in [[ISa]] (> 1%) indicates that the processing has improved; a slight decrease in ISa (<1%) often is accompanied by an increase of CC1/2 at high resolution, and thus should be tolerated.
Those options that improve ISa can then be used in combination.
* the simple editor operating in the XDS.INP, XDSCONV and XSCALE tabs understands ctrl-X (cut), ctrl-c (copy), ctrl-v (paste), ctrl-z (undo) and ctrl-Z (redo).
* ctrl-+ (control-plus) and ctrl-- (control-minus) enlarge or decrease the font size of characters (available since April 22, 2014).
* flexibility of operation: all commands in the TOOLS tab may be modified by the user, simply by modifying the (bash) code in the text field below the button. The user-changed command is automagically and permanently stored in ~/.xds-gui . The user may also change what is written on a button, by editing ~/.xds-gui.
* Menu / Settings is used for setting "Paths" for binaries (xds_par, xscale_par, shelxc/d/e, ...), "Library" (typically /usr/local/lib64/dectris-neggia.so), and "Appearance" (font size and default plot height).
== Tabs ==
Untrusted areas can be specified by the user, using two (UNTRUSTED_ELLIPSE; UNTRUSTED_RECTANGLE) or four (UNTRUSTED QUADRILATERAL) right mouse clicks. The resulting areas are shown with red outline, and the keyword/parameter pairs are shown in the XDS.INP tab. Step-by-step:
* "Load" a raw frame or a CBF file (e.g. FRAME.cbf) to have it displayed. The pane can be "dragged" with the left mouse button; the mouse wheel zooms. The parameters in the XDS.INP tab are taken for resolution calculations (i.e. the frame header is not being interpreted in any way).
* if XDS.INP does not yet exist, click "generate XDS.INP" (this ''will'' read the header). Check the XDS.INP tab afterwards but then go back to the Frame tab. Note that the current [[generate_XDS.INP ]] works well for Pilatus, ADSC, Mar , [[Eiger]] and some Rigaku detectors; for other kinds of detectors the values marked XXX in XDS.INP have to be filled in manually.
* left-click on "Untrusted areas" -> a pulldown menu appears
* left-click on (say) "Untrusted Rectangle (2 clicks)"
=== [[IDXREF]] ===
detailed explanation of output: see [[IDXREF.LP]].The right side of the tab shows the indexed (black) and not-indexed (red) reflections of [[SPOT.XDS]].
=== DEFPIX ===
The border between the left and right side of the window may be adjusted with the mouse.<br>
=== TOOLS tools ===
offers possibilities for running short scripts.
Several such scripts are pre-defined; the user may create her own scripts. If scripts are modified, they are saved to ~/.xds-gui . The names of the buttons (e.g. "User defined command 1") can be changed by editing ~/.xds-gui .
* The first item of the left panel ("Show frame with predicted spots") generates the predicted pattern of reflections for a user-specified frame, overlaid on the frame, for display with [[XDS-viewer]]. The file FRAME.cbf (produced by INTEGRATE) is renamed to FRAME_$X.cbf (where X is the user-specified frame number) and remains in the temp subdirectory. It may of course be opened in the FRAME tab, but starting XDS-viewer automatically has the advantage that several frames with predictions may be inspected on the screen, at the same time. Please note: if the XDS directory resides in a FAT32 filesystem (which is often the case on a USB stick or disk), then "ln -s" (of the script line) should be replaced by "cp -p" since FAT32 does not support symlinks. Also note: for the script to work correctly, NAME_TEMPLATE_OF_DATA_FRAMES in XDS.INP has to specify an absolute, not a relative path.
* The second item ("Optimizing data qualitySaving and comparing good results") offers commands that manipulate [[XDS.INP]] in several ways. Please note: to save/restore the popup current data processing files to/from a "XDSsave" directory.INP has been changed externallyMake sure to replace "xdiff" with "xxdiff" or " is emitted by the Qt system and cannot be switched off. It appears tkdiff", if one of the scripts changes XDSlatter is available.INP while it is opened by XDSGUI (which If [http://www.globalphasing.com/autoproc autoPROC] is always the case) . Thusinstalled, one should simply press I suggest to use the "ReloadUser defined command 2" button. The userfor <code>mkdir staraniso; cd staraniso; aP_scale -definable commandline may be used ehkl .g. for grep -v "JOB=" XDS/XDS_ASCII.INP > xHKL; echo "JOB= DEFPIX INTEGRATE CORRECT" > XDS.INP; cat x >anisotropy-corrected files are in staraniso subdirectory</code> XDS.INP
* The third item ("Saving and comparing good resultsOptimizing data quality") offers commands to save/restore that manipulate [[XDS.INP]] in several ways. Please note: the current data processing files to/from a popup "saveXDS.INP has been changed externally" directoryis emitted by the Qt system and cannot be switched off. Make sure to replace It appears if one of the scripts changes XDS.INP while it is opened by XDSGUI (which is always the case) . Thus, one should simply press the "xdiffReload" button. XDSGUI sometimes hangs (and must be killed with "xxdiff" or "tkdiff", if one Ctrl-C) when running <pre>grep _E INTEGRATE.LP | tail -2 >x; sed -e 's/ BEAM_DIVERGENCE=/! BEAM_DIVERGENCE=/' -e 's/ REFLECTING_RANGE=/! REFLECTING_RANGE=/' XDS.INP >>x; mv x XDS.INP</pre> I suggest to replace the final part of the latter command <pre>mv x XDS.INP</pre> with <pre>cat x > XDS.INP; rm x</pre> because this workaround conserves the inode. This bug will be fixed in the code. The user-definable command may be used e.g. for <pre>echo RELRAD=7 ! default is 5 >x; grep -v BEAM_DIVERGENCE XDS.INP >>x; cat x > XDS.INP; rm x</pre> This provides more pixels to the background estimation (for another INTEGRATE), and is availableuseful for weakly exposed data sets which have mostly pixels with zero counts.
* The last item ("Further analyses") may be used for commands, e.g. running [[pointless]] against the XDS_ASCII.HKL file. The user-definable commandline may be used e.g. for
phenixgrep -s allow-duplicate-sequence-numbers ~/.xtriage XDS_ASCIIcoot || echo "(allow-duplicate-sequence-numbers)" >>~/.coot ; spot2pdb -r 1; coot SPOT-*.pdbsince that runs [[spot2pdb]] to show you the reflections of SPOT.XDS in reciprocal space, with coot. You see the indexed reflections (indexed in IDXREF) in yellow, the not-indexed ones in pink, the rotation axis in blue and the origin as a blue cross.HKL
=== XDSSTAT statistics ===
This tab allows to run [[XDSSTAT]] and [[XDSCC12]] with a mouseclick.  As soon as XDSSTAT.LP exists, the main plots derived from it are displayed(second and further panes). Another The "view" button allows to run [[XDS-viewer]] to visualize the control images that XDSSTAT generates, namely anom.pck, scales.pck, rf.pck, misfits.pck, corr.pck, rlps.pck, peaks.pck and nobs.pck . Likewise, as soon as the output file XDSCC12.LP exists, a plot derived from those lines starting with "a:" and "b:" will be generated. In black, this shows the overall contribution of a frame, or of several frames pooled, to the CC<sub>1/2</sub> value of those reflections that this frame (or these frames) contributes to. The rainbow-coloured curves divide the total resolution range into a user-selectable number (chosen with the -nbin option of XDSCC12) of resolution ranges; blue is low resolution and red is high resolution.Fine-sliced frames (0.1°) can be pooled into 1°-pseudoframes by using the -t option, to increase the number of reflections contributing to the plot and to make it less noisy.
=== XDSCONV ===
a simple XSCALE.INP is provided by default. It may be edited by the user, and saved. Upon clicking a button, [[xscale]] is run. XSCALE.LP may be displayed.
 
=== SHELX ===
 
This gives access to SHELXC, SHELXD and SHELXE ([http://shelx.uni-goettingen.de/ documentation]) in a similar manner to [[ccp4com:Hkl2map|hkl2map]]. As in other tabs, the commands for running these programs are shown, and the user can modify and customize them.
 
=== ARCIMBOLDO ===
 
This gives access to ARCIMBOLDO ([http://chango.ibmb.csic.es/ documentation]). As in other tabs, the commands for running these programs are shown, and the user can modify and customize them.
== Availability ==
The [https://sourceforge.net/u/joseptrivino/xdsgui/ci/master/tree/ program's source code] is released under the terms of the [https://gnu.org/licenses/old-licenses/gpl-2.0.txt GPLv2]; it uses the [http://www.qt.io/ Qt library] which was released under the terms of the [https://gnu.org/licenses/lgpl.txt LGPL]. The program is under development and probably has bugs. If it crashes, it should simply be restarted. A crash of the program ''does not interfere'' with the operation of [[XDS]]; likewise, closing the program window does not influence any XDS run started from XDSGUI.
=== Dependencies ===
XDSGUI depends on [[generate_XDS.INP]], [[XDS-viewer]], and of course [[XDS]] (both the xds and xds_par binaries!), [[XDSCONV]], and [[XDSCONV2cbf]]. The XDSSTAT ''statistics'' tab requires [[XDSSTAT]] and [[XDSCC12]]. One of [[Eiger]] data processing requires <code>h5dump</code>, and either the items in <code>H5ToXds</code> binary or the TOOLS tab calls <code>dectris-neggia</code> library (download from Dectris!). The latter should be put into a graphical file comparison program; if the default directory (xdiffe.g. /usr/local/lib64) is not available, [http:and the full path to it (e.g. /usr/furius.calocal/xxdiff xxdiff] or [http:lib64//sourceforgedectris-neggia.net/projects/tkdiff tkdiff] may so) should be used entered into the "generic frame library" entry, in Menu- please adjust the commandline below the button accordingly>Settings.
One of the items in the <code>tools</code> tab calls a graphical file comparison program; if the default (xdiff) is not available, [http://furius.ca/xxdiff xxdiff] or [http://sourceforge.net/projects/tkdiff tkdiff] or [http://kdiff3.sourceforge.net/ kdiff3] or [http://meldmerge.org/ meld] may be used - please adjust the command line below the button accordingly. See also [[Installation]]. Technically, the word "dependency" means that these scripts/programs should be in your $PATH - an 'alias' is often not good enough because the ''TOOLS '' commands use <code>bash </code> as a shell, and that shell does not inherit the aliases from e.g. (t)csh.
=== Installation ===
The current version of the program can be downloaded, by academic users, for Linux [ftphttps://turn5.biologie.uni-konstanz.de{{SERVERNAME}}/pub/xdsgui.rhel6.64 64bit] (compiled on RHEL6), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel6.32 32bit] (compiled on RHEL6), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel5.64 64bit] (compiled on RHEL5, statically linked with Qt), [ftp://turn5.biologie.uni-konstanz.de/pub/xdsgui.rhel5.32 32bit] (compiled on RHEL5, statically linked with Qt), or [ftphttps://turn5.biologie.uni-konstanz.de{{SERVERNAME}}/pub/xdsgui.dmg Mac] (works on OSX 10.6 and up). Industrial users of XDS: pls contact me directly.
In case of '''Linux''', after downloading the binary fileThe easiest way to install XDSGUI, its name should be changed to "xdsgui" and then be made executable ("chmod a+x xdsgui"). The binary may then be copied into e.g. the directory /usr/local/bin by the administrator, or to the $HOME/bin directory by the user. Any other directory should be suitable as long as it all XDSGUI-related programs is documented in your $PATH[[Installation]].
Alternatively, in case of '''Linux''', after downloading the binary file, its name should be changed to "xdsgui" and then be made executable ("chmod a+x xdsgui"). The binary may then be copied into e.g. the directory /usr/local/bin by the administrator, or to the $HOME/bin directory by the user. Any other directory should be suitable as long as it is in your $PATH. In case of a '''Mac''', the DMG file may be installed in the usual way, by double-clicking it and dragging the icon into the Applications folder. '''It is advantageous to place a symbolic link, like'''
sudo ln -s /Applications/xdsgui.app/Contents/MacOS/xdsgui /usr/local/bin/xdsgui
'''and to start the GUI from a terminal window, by typing xdsgui'''. In a similar way, XDS-viewer should be put into the $PATH under the name xds-viewer, e.g. using
sudo ln -s /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin /usr/local/bin/xds-viewer
==== Libraries and software that the program depends on ====In case of '''Ubuntu 14.04Fedora''' has libmng2 instead of libmng. Thereforeor Redhat Enterprise Linux ('''RHEL'''; same as CentOS) distributions, on 32bit systems you should try sudo ln -s /usr/lib/i386-linux-gnu/libmng.so.2 /usr/lib/i386-linux-gnu/libmng.so.1 and on 64bit systemsmay have to sudo ln -s /usr/lib/x86_64-linux-gnu/libmng.so.2 /usr/lib/x86_64-linuxyum install qt qt-gnu/libmng.so.1 x11
if you get the error message
xdsgui: error while loading shared libraries: libmnglibQtGui.so.4: cannot open shared object file: No such file or directoryor similar for libQtCore.so.4.Similarly, the error message xds-viewer: error while loading shared libraries: libpng12.so.0: cannot open shared object file: No such file or directorymeans that you need to yum install libpng12Generally, if in case of RHEL or related distributions you don't know the name of the package, e.g. yum provides "*/libQtCore.so.4"would tell you about it. RedHat Enterprise Linux 8 (CentOS 8) does not seem to provide by default the needed Qt libraries. They can be installed using dnf install https://extras.getpagespeed.com/redhat/8/noarch/RPMS/getpagespeed-extras-release-8-6.el8.gps.noarch.rpm dnf install qt-x11 mesa-libGLUand answering "y" to all questions (attention - this installs a non-default repository). Alternatively, the Fedora 29 packages can be installed: wget https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/Everything/x86_64/os/Packages/q/qt-4.8.7-41.fc29.x86_64.rpm wget https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/Everything/x86_64/os/Packages/q/qt-common-4.8.7-41.fc29.noarch.rpm wget https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/Everything/x86_64/os/Packages/q/qt-x11-4.8.7-41.fc29.x86_64.rpm dnf install qt-4.8.7-41.fc29.x86_64.rpm qt-common-4.8.7-41.fc29.noarch.rpm qt-x11-4.8.7-41.fc29.x86_64.rpmIn addition, the qt-settings RPM may have to be installed. It is available from EPEL, or also from Fedora 29.  '''Ubuntu''' needs the libglu1-mesa package if you get the error message xdsgui: error while loading shared libraries: libGLU.so.1: cannot open shared object file: No such file or directoryIf in the case of Ubuntu you don't know the name of the package that provides a certain library, the sudo apt-file search libGLU.so.1 command would find it for you. This needs a one-time installation by sudo apt install apt-file sudo apt-file update  On the '''Mac''' (e.g. Yosemite) one needs [https://itunes.apple.com/us/app/xcode/id497799835?ls=1&mt=12 Xcode] installed. Otherwise one may obtain a popup error “You can’t open the application because it is not supported on this type of Mac.” After installation of Xcode, run the <code>strings</code> command (once) and accept the license agreement!
== Known bugs, problems and workarounds ==
# ''Question concerning TOOLS: "Show frame..." on Mac: it creates everything shown in the commandline but it seem to us that everything runs in the background. Therefore xds-viewer does not open anywhere as it is not seen. Also, it would be great if this image together with the predictions would be re-directed to the FRAME tab.'' <br> Answer: On Linux, the xds-viewer window is brought to the foreground automatically, whereas on the Mac this does not seem to happen. However, an icon for xds-viewer appears in the dock (on the right) and I can double-click it to see the xds-viewer window. I have no idea how to bring xds-viewer to the foreground automatically, and I need input from people who know Macs and tell me how to do it. The reason why we do not open automatically in the FRAME tab is that one can have several xds-viewer windows open at the same time, and compare the patterns. As a '''workaround''', you can manually open the resulting temp/FRAME_$X.cbf in the FRAME tab.
# ''Another question concerning TOOLS: "Further analyses" on Mac: again pointless runs nicely but there is no output for the user.'' <br> Answer: Could it be that you started xdsgui by double-clicking its icon in the Finder? In that case, there is no output visible, because there is no console. Unfortunately, for all the Tools, the output is in the console window where xdsgui was started. We will try to find out how to open a window where the output is then shown. Thus, currently the '''workaround''' for this problem is to you should really start xdsgui in a console window.# both generate_XDS.INP and [[XDS]] do not seem to like tolerate spaces ("blanks") in file/directory names. While frame names rarely have a space in them, directories might, e.g. if an external disk has the label "John's USB disk" then this would be mounted on Linux as /media/John's USB disk/ and that would result in a error message from generate_XDS.INP and/or XDS. The '''workaround''' is to use a symlink on a different hard disk. version 0.45 of generate_XDS.INP makes a start towards a fix for this problem, but XDS also will have to be adjusted.
# ''Problem description: On the Mac, after loading frames, by clicking “generate XDS.INP”, the program gives some strange symbol “” in XDS.INP. And the more you click “save” button, the more “” appeared. This looks like <br>SPACE_GROUP_NUMBER=0  ! 0 if unknown <br>UNIT_CELL_CONSTANTS= 70 80 90 90 90 90  .''<br> The problem is due to the “Rich text” format in TextEdit when saving "generate_XDS.INP". It is solved by re-downloading the script, and changing format to Plain - everything is working now.
# on a Mac, you may find a stream of the following error messages in the console window: <code>Jan 31 14:35:17 xdsgui[20939] <Error>: Error: this application, or a library it uses, has passed an invalid numeric value (NaN, or not-a-number) to CoreGraphics API and this value is being ignored. Please fix this problem. Jan 31 14:35:17 xdsgui[20939] <Error>: If you want to see the backtrace, please set CG_NUMERICS_SHOW_BACKTRACE environmental variable.</code> This sounds terrible, but is actually harmless. Since it is a message from Qt (which XDSGUI uses for graphics), ''it does not compromise the XDS data processing in any way''. If it bothers you, just suppress the printout of these messages, by <code>xdsgui 2>&1 | egrep -v 'CoreGraphics|CG_NUMERICS'</code>
# there is a bug in the Jan-2017 version of XDSGUI that prevents it from displaying non-CBF files. The symptom is that the frames appear to be blank. The workaround is: go to (upper left) "Menu"->"Settings"->remove the string "/usr/local/lib64/dectris-neggia.so". The fix is in the Oct-2017 version (thanks Wolfgang!).
# in the COLSPOT tab, one sometimes has to jiggle a bit the border in the middle of the page, between the log output and the two plots, to trigger showing the plots.
# XDSGUI sometimes hangs (and must be killed with Ctrl-C) when running <pre>grep _E INTEGRATE.LP | tail -2 >x; sed -e 's/ BEAM_DIVERGENCE=/! BEAM_DIVERGENCE=/' -e 's/ REFLECTING_RANGE=/! REFLECTING_RANGE=/' XDS.INP >>x; mv x XDS.INP</pre> in the tools tab. A potential workaround is given [[XDSGUI#tools|above]] - pls try!
If you find a bug, please send email to Kay dot Diederichs at uni-konstanz dot de , ideally with enough information/data to reproduce the bug.
== Limitations ==
* The program depends on [[generate_XDS.INP]] to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types) and , MAR (CCD and image plate) detectors, and the Bruker PHOTON II. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like oscillation rangeROTATION_AXIS, OSCILLATION_RANGE, wavelengthX-RAY_WAVELENGTH, distanceDETECTOR_DISTANCE, and direct beam position (or rather: point of detector that is closest to the crystal)XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].* The authors of [[generate_XDS.INP]] have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has a anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. We have no exhaustive list of these beamlines, and the frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction.
* The display of large frames in the FRAME tab may be slow over the network.
* The green INCLUDE_RESOLUTION and red semi-transparent EXCLUDE_RESOLUTION_RANGE circles in the Frame tab as well as all resolution values in the Frame and IDXREF tabs are calculated from values in XDS.INP (rather than from the refined values in XPARM.XDS, which XDS uses). For the (curved) Pilatus 12M, the calculations in XDSGUI do not give the correct results.
== News ==
update June 2, 2014: fix bugs
update December 11, 2014: fix bugs; SETTING SYMMETRY-BASED for [[pointless]] (in the TOOLS tab); radiation damage (R_d) plot with recommended limit shown in the XDSSTAT tab
 
update Sep 17, 2015: belatedly updated xdsgui.rhel6.* - this version correctly visualizes the tables of INTEGRATE.LP which have a different format since XDS version June-2015
 
update May 20, 2016: many bug fixes, rename XDSSTAT tab to <code>statistics</code> tab, include delta-CC<sub>1/2</sub> plot.
 
update Jan 31, 2017: many bug fixes. Now works with Eiger HDF5 - select the *master.h5 file in the FRAME tab.
 
update Oct 10, 2017: fix the bug that non-CBF files are not being displayed (see Bugs section above).
 
update Apr 27, 2018: fix the bug that the INTEGRATE tab is not updated when XDS runs. 32-bit Linux binaries are no longer made. Download source from https://sourceforge.net/u/joseptrivino/xdsgui/ci/master/tree/ .
 
update June 15, 2018: SHELXC, SHELXD and SHELXE are available through the SHELX tab.
 
update April 17, 2019: Arcimboldo programs are available through the ARCIMBOLDO tab.
== See also ==
[[Installation]]
2,284
edits

Navigation menu