Changes

From XDSwiki
Jump to navigationJump to search
m
Line 1: Line 1:  +
 +
 +
 +
 +
 +
 +
 +
 +
 +
 
This article has some little script snippets that should ease the installation of XDS and related programs like ([[XDS-viewer]], [[xdsstat]], [[XDSGUI|xdsgui]]). It is assumed that binaries (or links to binaries) should go to /usr/local/bin . This means that you should do this as the administrator (root), or have sudo rights (Mac, and some Linux distros).
 
This article has some little script snippets that should ease the installation of XDS and related programs like ([[XDS-viewer]], [[xdsstat]], [[XDSGUI|xdsgui]]). It is assumed that binaries (or links to binaries) should go to /usr/local/bin . This means that you should do this as the administrator (root), or have sudo rights (Mac, and some Linux distros).
   Line 6: Line 16:     
=== [[XDS]] package ===
 
=== [[XDS]] package ===
If you are an academic user, as root
+
If you are an academic user, as root (on Ubuntu, use "sudo -i" to become root)
 
<pre>
 
<pre>
 
  cd /usr/local/bin
 
  cd /usr/local/bin
  wget -O- ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-INTEL64_Linux_x86_64.tar.gz | tar xzvf -
+
  wget -O- https://xds.mr.mpg.de/XDS-INTEL64_Linux_x86_64.tar.gz | tar xzvf -
 
  ln -sf XDS-INTEL64_Linux_x86_64/* .
 
  ln -sf XDS-INTEL64_Linux_x86_64/* .
 
</pre>
 
</pre>
In case the [ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch above site] is unavailable, there is a [https://strucbio.biologie.uni-konstanz.de/pub/xds backup site].
+
In case the above website is unavailable, there is a [https://strucbio.biologie.uni-konstanz.de/pub/xds backup site].
   −
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]], [[generate_XDS.INP]]  ===
+
=== other programs: [[XDS-Viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]], [[spot2pdb]], [[generate_XDS.INP]]  ===
... can be downloaded from https://{{SERVERNAME}}/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then (as root)
+
... can be downloaded from https://{{SERVERNAME}}/pub/linux_bin/ . A simple way to obtain them is to create an empty folder, "cd" to that folder and then
  wget https://{{SERVERNAME}}/pub/linux_bin/get_folder.sh
+
  wget -N https://{{SERVERNAME}}/pub/linux_bin/get_folder.sh
 
  chmod +x get_folder.sh
 
  chmod +x get_folder.sh
 
  ./get_folder.sh
 
  ./get_folder.sh
You may repeat this, as root, every now and then, to update the programs.
+
You may repeat this every now and then to update the programs.
   −
A Qt4 version of xds-viewer is available as xds-viewer.qt4, for old Linux distributions.
+
As root, the script copies the programs into /usr/local/bin/linux_bin, and links them to /usr/local/bin. As a normal user, they are downloaded to $HOME/bin/linux_bin, and linked to your $HOME/bin. In the latter case, make sure that $HOME/bin is in your $PATH.
 +
 
 +
The Qt5 graphics library is now the default for xdsgui and xds-viewer. Qt4 versions are available as xdsgui.qt4 and xds-viewer.qt4, for old Linux distributions (and for use with x2goclient, which would otherwise need [https://wiki.x2go.org/doku.php/wiki:development:glx-xlib-workaround special action] for Qt5 binaries).
    
=== tools and helper programs for [[generate_XDS.INP]] and [[XDSGUI]], and libraries ===
 
=== tools and helper programs for [[generate_XDS.INP]] and [[XDSGUI]], and libraries ===
Line 43: Line 55:  
Ubuntu 18.04: if libQtOpenGL.so.4 is missing, use <code>sudo apt install libqt4-opengl</code>.
 
Ubuntu 18.04: if libQtOpenGL.so.4 is missing, use <code>sudo apt install libqt4-opengl</code>.
   −
Ubuntu 20.04: if libQtOpenGL.so.4 is missing, use
+
Ubuntu 20.04: if libQtOpenGL.so.4 is missing (for the Qt4 binary of [[XDSGUI]]), use
 
  sudo add-apt-repository -y ppa:rock-core/qt4
 
  sudo add-apt-repository -y ppa:rock-core/qt4
 
  sudo apt update
 
  sudo apt update
 
  sudo apt install libqt4-opengl
 
  sudo apt install libqt4-opengl
But it would be better to use the [https://strucbio.biologie.uni-konstanz.de/pub/linux_bin/xdsgui_qt5 Qt5 binary of XDSGUI].
+
If the Qt5 libraries are missing, use <code>sudo apt install libqt5gui5 libqt5printsupport5 libqt5opengl5</code>.  
 +
 
 +
On CentOS7, this would be <code>yum -y install qt5-qtbase-gui qt5-qtbase</code> .
    
<code>xxdiff</code> is likely available for your distribution - google for it, or try e.g.
 
<code>xxdiff</code> is likely available for your distribution - google for it, or try e.g.
Line 59: Line 73:  
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.
 
Alternatively, tkdiff may be installed, or other graphical comparison programs, like meld or kdiff3. vimdiff is also nice, and probably already installed.
   −
For [[Eiger]] data processing, <code>H5ToXDS</code> should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code>. Furthermore, you want the [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast Neggia] library or the [https://github.com/DiamondLightSource/durin Durin plugin].
+
For [[Eiger]] data processing, a h5dump binary (version 1.10 req'd for HDF5 data from DLS) and <code>H5ToXDS</code> is installed by the <code>get_folder.sh</code> script into /usr/local/bin. If the script is not used, <code>H5ToXDS</code> should be installed as [[Eiger|explained]], and <code>h5dump</code> should be installed by e.g. <code>yum -y install hdf5</code> (Ubuntu: <code>apt install hdf5-tools</code>) - but make sure that h5dump is version 1.10 or higher when processing HDF5 data from DLS.
   −
== Mac OSX ==
+
Furthermore, you want the [https://www.dectris.com/company/news/newsroom/news-details/process-eiger-data-with-xds-fast Neggia] library (does not work for HDF5 data from DLS) or the [https://github.com/DiamondLightSource/durin Durin plugin]. These may be installed through the <code>get_folder.sh</code> script, in <code>/usr/local/lib64</code>.
   −
Attention: [[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge). In my experience, you have to explicitly agree to the License terms when running a Command Line Tool (e.g. <code>strings</code>) for the first time.
+
== macOS ==
 
  −
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the installation tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root for installation, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time <code>echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile</code>)
  −
 
  −
So, prepare the following steps by opening the Terminal window, and then
  −
sudo su                    # this will make you root, and ask for your password
  −
mkdir /usr/local/bin  # only if /usr/local/bin was not created before
      
=== [[XDS]] package ===
 
=== [[XDS]] package ===
Line 76: Line 84:  
  echo you may have to "sudo su" first, to obtain administrator permissions!
 
  echo you may have to "sudo su" first, to obtain administrator permissions!
 
  cd /usr/local/bin
 
  cd /usr/local/bin
  curl -L -o - ftp://ftp.mpimf-heidelberg.mpg.de/pub/kabsch/XDS-OSX_64.tar.gz | tar xzvf -
+
  curl -L -o - https://xds.mr.mpg.de/XDS-OSX_64.tar.gz | tar xzvf -
 
  ln -sf XDS-OSX_64/* .
 
  ln -sf XDS-OSX_64/* .
   −
Starting with macOS Catalina, you also need
+
Starting with macOS Catalina, you may also need
 
  xattr -dr XDS-OSX_64/*
 
  xattr -dr XDS-OSX_64/*
 
if you get a security error popup (google "xattr com.apple.quarantine catalina").
 
if you get a security error popup (google "xattr com.apple.quarantine catalina").
 +
 +
Then make sure that the binaries are in the search path for executables, as explained in https://xds.mr.mpg.de/html_doc/downloading.html .
 +
 +
==== considerations for a macOS machine with Apple M1 processor ====
 +
 +
Change the above <code>curl</code> and <code>ln</code> commands to
 +
curl -L -o - https://xds.mr.mpg.de/XDS-Apple_M1.tar.gz | tar xzvf -
 +
ln -sf XDS-Apple_M1/* .
 +
 +
If you want to process .h5 files written by Eiger detectors, use the [https://strucbio.biologie.uni-konstanz.de/pub/mac_bin/durin-plugin-Apple-arm64.so Durin plugin for Apple ARM64 processors] ([https://github.com/DiamondLightSource/durin source code]).
 +
 +
A message like "Note: The following floating-point exceptions are signalling: IEEE_DIVIDE_BY_ZERO IEEE_UNDERFLOW_FLAG" may be printed after running XDS or XSCALE. This is harmless.
 +
 +
=== Prerequisites (Xcode, CCP4) for [[generate_XDS.INP]], [[XDSGUI]] and other programs ===
 +
[[generate_XDS.INP]] requires the [https://developer.apple.com/download/more/ Xcode command line tools] to be installed (free of charge): open a Terminal window, and enter <code>sudo xcode-select --install</code> (the download has a size of about 750MB; the downloader is much too pessimistic about the time it takes).
 +
 +
Although CCP4 is not required by XDS itself, some of the programs (XDSSTAT, and MTZ file creation in XDSCONV, POINTLESS and COOT) use CCP4 tools or libraries. Therefore, CCP4 should be installed. Open a Terminal and check the existence of the file .zprofile , and its content:<pre>
 +
kay@Kays-MacBook-Air-2018 ~ % ls -l .zprofile
 +
-rw-r--r--  1 kay  staff  107  7 Mär 09:41 .zprofile
 +
kay@Kays-MacBook-Air-2018 ~ % cat .zprofile
 +
source $HOME/.profile
 +
# Added by CCP4 package manager:
 +
. '/Applications/ccp4-7.1/bin/ccp4.setup-sh'
 +
</pre>If the file does not exist, create it - the easiest way is:
 +
<pre>
 +
echo . /Applications/ccp4-7.1/bin/ccp4.setup-sh >> $HOME/.zprofile
 +
</pre>The <code>get_folder.sh</code> script (below) checks the CCP4 setup, and creates the necessary files if needed.
 +
 +
Only Terminal windows that are opened afterwards will have access to the CCP4 programs!
 +
 +
Become familiar with the concept and ways to run commands as "root" - google "mac osx become root". All the installation tasks that are run in a Terminal window require root privileges, since some of the programs and their links are written to /usr/local/bin. (As an alternative that does not require root for installation, one may create a directory $HOME/bin and use that for the programs and links. That would also require modification of the $PATH, by a one-time <code>echo 'export PATH=$PATH:$HOME/bin' >> ~/.profile; echo 'export PATH=$PATH:$HOME/bin' >> ~/.zprofile</code>)
 +
 +
So, prepare the following steps by opening the Terminal window, and then
 +
sudo su              # this will make you root, and ask for your password
 +
mkdir /usr/local/bin  # only if /usr/local/bin was not created before
    
=== other programs: [[XDS-viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]], [[generate_XDS.INP]]  ===
 
=== other programs: [[XDS-viewer]], [[XDSSTAT]], [[XDSCC12]], [[XDSGUI]], [[XSCALE_ISOCLUSTER]], [[generate_XDS.INP]]  ===
Line 96: Line 139:     
In order to enable [[Eiger]] data processing, the <code>get_folder.sh</code> script  
 
In order to enable [[Eiger]] data processing, the <code>get_folder.sh</code> script  
# copies Dectris' Neggia library into /usr/local/lib64
+
# copies Dectris' Neggia library and DLS's Durin library into <code>/usr/local/lib64</code> (see above for Neggia and Durin on M1 Macs; the script does not yet handle this automatically)
# copies [[Eiger#General_aspects|eiger2cbf-osx]] into /usr/local/bin, but under the name <code>H5ToXds</code>
+
# copies [https://www.globalphasing.com GlobalPhasing's] <code>hdf2mini-cbf</code> into <code>/usr/local/bin</code>, but under the name <code>H5ToXds</code> (consider getting a [https://www.globalphasing.com/autoproc/ autoPROC] license!)
# copies [https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8/hdf5-1.8.7/bin/mac-intel-x86_64/utilities/h5dump version 1.8.7] of <code>h5dump</code> into /usr/local/bin  
+
# copies <code>h5dump</code> [https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.1/bin/unix/hdf5-1.12.1-Std-macos11_64-clang.tar.gz] into <code>/usr/local/bin</code>
   −
A Qt5 version of XDS-viewer (provided by T. Hauß) is available for MacOS [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.13.dmg 10.13], and for [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.15.dmg 10.15] and newer. Users must decide which DMG they install. The Qt4 version is still the default.
+
A [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.13.dmg 10.13] Qt5 version of XDS-viewer (compiled by T. Hauß) works on macOS Catalina, and is installed by <code>get_folder.sh</code>. Older Macs need [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-0.6.dmg this version]; newer macOS should install [https://{{SERVERNAME}}/pub/mac_bin/XDS-Viewer-macOS10.15.dmg 10.15]. For XDSGUI, the [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_qt4.dmg Qt4 version] is still the default, but a [https://{{SERVERNAME}}/pub/mac_bin/xdsgui_qt5.dmg Qt5 version] is available (but needs Qt5 libraries!).
   −
Lastly, it is useful (for e.g. XDSGUI) to install <code>xxdiff</code> from http://furius.ca/downloads/xxdiff/releases/macosx/ .
+
Lastly, it is useful (for e.g. XDSGUI) to install <code>xxdiff</code> from http://furius.ca/downloads/xxdiff/releases/macosx/ . I had to go to System Preferences > Security & Privacy > Privacy and add <code>xxdiff</code> to the programs with full disk access.
    
=== Troubleshooting ===
 
=== Troubleshooting ===
Line 113: Line 156:  
# in the same way, check the other programs that should be used
 
# in the same way, check the other programs that should be used
 
</pre>  
 
</pre>  
returns a message like <code>file or directory not found</code> then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink:
+
returns a message like <code>file or directory not found</code> then this means that you do not have the symlink. But even if these commands do not return errors, you should also check the targets of the symlink, as shown in the output of the <code>ls -l</code> commands.
<pre>
+
Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to install the programs properly (in /Applications) and make the symlink point to the correct location.
ls -l /Applications/xdsgui.app/Contents/MacOS/xdsgui
  −
ls -l /Applications/XDS-Viewer.app/Contents/MacOS/xds-viewer-bin
  −
</pre>
  −
Again, these commands should not return an error message. If they do, the programs are not installed in the location where the symlink points to. You'll have to either install the programs properly (in /Applications) or make the symlink point to the correct location.
      
A quick way is also
 
A quick way is also
Line 126: Line 165:  
== Windows ==
 
== Windows ==
   −
The programs of the [[XDS]] package as well as [[XDSGUI]], [[XDSSTAT]], [[XDSCC12]] and so on may be run on 64bit Windows 10 within the [https://docs.microsoft.com/windows/wsl/install-win10 Linux Subsystem for Windows] (WSL). This (easily!) installs a Ubuntu environment, which supports the apt package manager, so in principle any software available for Ubuntu may be installed (or other distros, see [https://blogs.msdn.microsoft.com/commandline/2017/05/11/new-distros-coming-to-bashwsl-via-windows-store/]).
+
The programs of the [[XDS]] package as well as [[XDSGUI]], [[XDSSTAT]], [[XDSCC12]] and so on may be run on 64bit Windows 10 within the [https://docs.microsoft.com/windows/wsl/install-win10 Linux Subsystem for Windows] (WSL, or preferably WSL2). This (easily!) installs e.g. an [https://ubuntu.com/wsl Ubuntu] environment, which supports the apt package manager, so in principle any software available for Ubuntu may be installed (or other distros, see [https://blogs.msdn.microsoft.com/commandline/2017/05/11/new-distros-coming-to-bashwsl-via-windows-store/]).
   −
Graphical Linux programs require installation of a X server like [https://sourceforge.net/projects/xming/files/latest/download Xming] or [https://sourceforge.net/projects/vcxsrv VcXsvr] on the Windows host. There are helpful writeups (e.g. [http://wsl-guide.org/en/latest/]). To use the X server, one has to say in the shell window
+
A lot of technical detail is at [https://docs.microsoft.com/en-us/windows/wsl/compare-versions] .
 +
 
 +
CCP4 documents use of WSL-1 ; see info at http://www.ccp4.ac.uk/download/#os=wsl .
 +
 
 +
Graphical Linux programs require installation of a X server like [https://sourceforge.net/projects/xming/files/latest/download Xming] or [https://sourceforge.net/projects/vcxsrv VcXsvr] or [https://mobaxterm.mobatek.net/ Mobaxterm] on the Windows host. There are helpful writeups (e.g. [https://wiki.ubuntu.com/WSL#Running_Graphical_Applications] and [http://wsl-guide.org/en/latest/]). VcXsrc seems to have a problem with coot for some people (see [https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=COOT;6ef4a967.2011 WSL2 installation] and responses). To use the X server with WSL, one has to say in the shell window
 
  export DISPLAY=:0
 
  export DISPLAY=:0
before running the program. Running XDSGUI under WSL's Ubuntu may require installation of a few packages with e.g.
+
before running the program. Running [[XDSGUI]] under WSL's Ubuntu 20.04 may require installation of a few packages with e.g.
  sudo apt-get install libgomp1 libqtgui4
+
  sudo apt-get install libgomp1 libqtgui4 binutils
(see [[XDSGUI#Libraries_and_software_that_the_program_depends_on|XDSGUI]] and above).  
+
(see [[XDSGUI#Libraries_and_software_that_the_program_depends_on|XDSGUI]] and above). If required, also install python and hdf5-tools.
 
  −
[[generate_XDS.INP]] requires a number of packages (some of which are only for specific detectors!); I'd start with
  −
sudo apt-get install coreutils binutils gawk sed bc grep
  −
and if required, also install python and hdf5-tools.
      
XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host.
 
XDSSTAT and the conversion to MTZ files by XDSCONV require a CCP4 installation accessible by WSL. In principle, CCP4 may be installed within WSL (have not tested this), or on the Windows host.
Line 145: Line 184:     
A detailed [https://strucbio.biologie.uni-konstanz.de/pub/xds%20%20prog%20installation%20in%20WSL%20(Lata%20Panicker).pdf writeup for Ubuntu 20.04 LTS] was provided by Dr Lata Panicker, SO(G), BARC, India.
 
A detailed [https://strucbio.biologie.uni-konstanz.de/pub/xds%20%20prog%20installation%20in%20WSL%20(Lata%20Panicker).pdf writeup for Ubuntu 20.04 LTS] was provided by Dr Lata Panicker, SO(G), BARC, India.
 +
 +
Update September 2021: With the current version of Windows 10 and WSL, the Qt5 binary of XDSGUI does not seem to allow editing in the XDS.INP tab. If this happens, either use an external editor or use xdsgui.qt4. In the latter case, you may have to install Qt4 libraries.
 +
 +
=== WSL2 ===
 +
 +
CCP4 7.1 including coot-0.9.5 works in a Ubuntu 20.04 LTS machine under WSL2 on Windows 10 version 20H2, and newer.
 +
 +
[https://docs.microsoft.com/windows/wsl/install-win10 After installation of WSL2], I installed [https://mobaxterm.mobatek.net/ MobaXterm] as the Xserver. When the Windows firewall asked whether it should allow MobaXterm communication, I answered "yes" for both private and public networks.
 +
 +
If you want to install CCP4 as a regular user, you need (as this is Linux! And replace MYWSLNAME with your WSL user name):
 +
sudo mkdir /opt/xtal
 +
sudo chown MYWSLNAME /opt/xtal
 +
 +
A few libraries are needed before CCP4 installation (tcsh is needed by CCP4; most of the others are graphics libraries; binutils provides <code>strings</code>):
 +
sudo apt install tcsh libqt5opengl5 libqt5printsupport5 libqt5gui5 libxcb-render0 libxcb-shm0 libglu1-mesa libgomp1 binutils
 +
When installing CCP4, let it "modify command line environment" for you (on one of the very first screens of the installation GUI). If you forgot this, insert a line into your ~/.bashrc :
 +
source /opt/xtal/ccp4-7.1/bin/ccp4.setup-sh
 +
afterwards (this is not WSL specific, but is just Linux). I created a file in the Windows filesystem (replace MYUSERNAME with your Windows user name):
 +
cd /mnt/c/Users/MYUSERNAME/
 +
echo "[wsl2]" > .wslconfig
 +
echo "kernelCommandLine = vsyscall=emulate" >> .wslconfig
 +
to make the shelx* programs work (see https://github.com/microsoft/WSL/issues/4694 ), and to make "vsyscall=emulate" appear in /proc/cmdline. After creating the file, WSL2 must be restarted with "wsl.exe --shutdown" (or the machine booted).
 +
 +
Performance of coot is good enough for occasional work. XDSGUI and multi-threaded XDS work as expected. For reading .h5 files, the NEGGIA plugin works correctly, but the DURIN plugin currently crashes (a [https://github.com/DiamondLightSource/durin/issues/22 bug] has been reported).
 +
 +
Update September 2021: With the current version of Windows 10 and WSL, the Qt5 binary of XDSGUI does not seem to allow editing in the XDS.INP tab. If this happens, either use an external editor or use xdsgui.qt4. In the latter case, you may have to install Qt4 libraries.
    
== See also ==
 
== See also ==
    
[[Cluster Installation]]
 
[[Cluster Installation]]
2,522

edits

Cookies help us deliver our services. By using our services, you agree to our use of cookies.

Navigation menu