Changes

From XDSwiki
Jump to navigationJump to search
→‎The script: take away POSITION from REFINE(CORRECT) if electron diffraction data
Line 10: Line 10:     
== The script ==
 
== The script ==
 +
 
<pre>
 
<pre>
   Line 104: Line 105:  
# revision 0.86 . Keitaro 03/05/2019 Add PILATUS3 6M, S/N 60-0123 at SSRF BL18U-1 with ROTATION_AXIS=-1 0 0
 
# revision 0.86 . Keitaro 03/05/2019 Add PILATUS3 6M, S/N 60-0123 at SSRF BL18U-1 with ROTATION_AXIS=-1 0 0
 
# revision 0.87 . KD 12/10/2019 Add PILATUS XXX, S/N XX-XXX at SSRF BL19U1 and MarCCD detector #43 at BL17B1 with ROTATION_AXIS=-1 0 0
 
# revision 0.87 . KD 12/10/2019 Add PILATUS XXX, S/N XX-XXX at SSRF BL19U1 and MarCCD detector #43 at BL17B1 with ROTATION_AXIS=-1 0 0
# revision 0.88 . KD 16/10/2019 fixes for SSRF, add "-maxdepth 0" to "find -H ..."
+
# revision 0.88 . KD 16/10/2019 fixes for SSRF, add "-maxdepth 1" to "find -H ..."
REVISION="0.88 (16-Oct-2019)"
+
# revision 0.89 . KD 21/10/2019 add ADSC S/N 905 at ALS 8.2.1, S/N 928 at Australian Synchrotron MX2 beamline; final SSRF fixes
 +
# revision 0.90 . KD 25/10/2019 add OLDMAR detector type. Tested w/ SBGrid data set 6. Anomalous signal may have wrong hand!
 +
# revision 0.91 . KD 16/01/2020 Allow negative starting angle for Eiger (found -33 at SLS !).
 +
# revision 0.92 . KD 27/02/2020 read *_master.h5 from Diamond Light Source
 +
# revision 0.93 . KD 13/03/2020 print out 2theta for MarCCD (DETECTOR_*_AXIS can be derived from this)
 +
# revision 0.94 . KD 16/03/2020 bugfix for Bruker-cbf to make bc accept e.g. 3.1e-005 by using awk printf "%.5f" instead of awk print
 +
# revision 0.95 . KD 29/07/2020 fix DLS Eiger HDF5 variant OSCILLATION_RANGE, STARTING_ANGLE. Attention: DLS Eiger variant needs h5dump 1.10 for OVERLOAD!
 +
# revision 0.96 . KD 03/10/2020 make script echo detector serial number if inverted ROTATION_AXIS is detected.
 +
# revision 0.97 . KD 19/10/2020 add UNTRUSTED_RECTANGLEs for Eiger2 (which has a few pixels less than Eiger (thanks to Andreas Förster)
 +
# revision 0.98 . Thomas Hauß (HZB) 06/11/2020 switch Python print syntax to Python3
 +
# revision 0.99 . Aaron Finke (CHESS) 07/11/2020 add ROTATION_AXIS=-1 0 0 for EIGER2 16M detector at MX beamline ID7B2 (CHESS)
 +
# revision 1.00 . Gleb Bourenkov 10/11/2020 add ROTATION_AXIS=0 -1 0 for beamline P14 (EMBL Hamburg) detectors Eiger 16M S/N E-32-0107; previously Eiger 4M S/N E-08-0107 and PILATUS 6M-F, S/N 60-0115-F
 +
# revision 1.01 . KD 07/12/2020 ROTATION_AXIS=0 -1 0 for Pilatus3 2M, S/N 24-0118 at ID23-2 (http://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings)
 +
# revision 1.02 . KD 11/01/2021 recognize mar555 detector as mar345 (thanks to Thomas Hauß, HZB)
 +
# revision 1.03 . Feng YU 18/7/2021 fix/expand information about Shanghai Synchrotron Radiation Facility (SSRF)
 +
# revision 1.04 . Zhipu Luo 13/08/2021 extract ORGX,ORGY for electron diffraction from SMV header if the wavelength value starts with 0.0
 +
REVISION="1.04 (13-AUG-2021)"
    
#                                                                                                             
 
#                                                                                                             
Line 158: Line 175:  
SEPMIN=7.0
 
SEPMIN=7.0
 
CLUSTER_RADIUS=3.5
 
CLUSTER_RADIUS=3.5
 +
REFINE_CORRECT="CELL BEAM ORIENTATION AXIS POSITION ! Default is: refine everything"
    
dname=`echo "$1" | xargs dirname`
 
dname=`echo "$1" | xargs dirname`
Line 168: Line 186:  
# cope with blanks in directory / file name
 
# cope with blanks in directory / file name
 
IFS=$'\n'
 
IFS=$'\n'
find -H -maxdepth 0 $dname -name "$bname" -or -name "${bname}.bz2" -or -name "${bname}.gz" -or -name "${bname}.xz" | egrep -v "_00000.cbf|_000.img" | sort > tmp1
+
find -H $dname -maxdepth 1 -name "$bname" -or -name "${bname}.bz2" -or -name "${bname}.gz" -or -name "${bname}.xz" | egrep -v "_00000.cbf|_000.img" | sort > tmp1
 
if [ ! -s tmp1 ]
 
if [ ! -s tmp1 ]
 
then
 
then
Line 229: Line 247:  
  grep -q "^SOURCE_WAVELENGTH= *1" tmp2 && DET=dtrek
 
  grep -q "^SOURCE_WAVELENGTH= *1" tmp2 && DET=dtrek
 
  grep -q BEAM_CENTER_X tmp2      && DET=adsc                                 
 
  grep -q BEAM_CENTER_X tmp2      && DET=adsc                                 
  grep -q mar345 tmp2 && DET=MAR345    
+
  grep -q mar345 tmp2 && DET=MAR345
 +
# rev. 1.02: in one case, mar555 data could be processed pretending it is mar345, so:
 +
grep -q mar555 tmp2 && DET=MAR345
 
  grep -q BRUKER tmp2 && grep -q CBF tmp2  && DET=Bruker-cbf   
 
  grep -q BRUKER tmp2 && grep -q CBF tmp2  && DET=Bruker-cbf   
  grep -q CMOS1 tmp2 && DET=adsc-CMOS1                  
+
  grep -q CMOS1 tmp2 && DET=adsc-CMOS1  
 +
grep -q MARCONTROL tmp2 && DET=OLDMAR 
 +
grep -q "WAVELENGTH=0.0" tmp2 && DET=experimental-ED                                   
 
else
 
else
 
  h5dump -d "/entry/instrument/detector/description" $FIRSTFRAME | grep -i Eiger > /dev/null && DET=eiger
 
  h5dump -d "/entry/instrument/detector/description" $FIRSTFRAME | grep -i Eiger > /dev/null && DET=eiger
Line 247: Line 269:  
elif [ "$DET" == "mccd" ]; then  
 
elif [ "$DET" == "mccd" ]; then  
 
   echo Data from a MarCCD detector
 
   echo Data from a MarCCD detector
 +
# http://www.sb.fsu.edu/~xray/Manuals/marCCD165header.html has header information
 
                                    
 
                                    
 
   DETECTOR="CCDCHESS MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65500"
 
   DETECTOR="CCDCHESS MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65500"
Line 268: Line 291:  
   if echo "${DET_SN}${REVERSEPHI_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
   if echo "${DET_SN}${REVERSEPHI_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
     REVERSE_PHI="yes"
 
     REVERSE_PHI="yes"
 +
    echo inverted ROTATION_AXIS since Detector Serial Number is "${DET_SN}"
 
   fi
 
   fi
   Line 282: Line 306:  
   DETECTOR_DISTANCE=$(od -t dI -j $SKIP -N 4 tmp2 | head -1 | awk '{print $2}')
 
   DETECTOR_DISTANCE=$(od -t dI -j $SKIP -N 4 tmp2 | head -1 | awk '{print $2}')
 
   DETECTOR_DISTANCE=`echo "scale=3; $DETECTOR_DISTANCE/1000" | bc -l`                                     
 
   DETECTOR_DISTANCE=`echo "scale=3; $DETECTOR_DISTANCE/1000" | bc -l`                                     
 +
 +
# Mar 12, 2020 KD
 +
  let SKIP=1724
 +
  TWOTHETA=$(od -t dI -j $SKIP -N 4 tmp2 | head -1 | awk '{print $2}')
 +
  TWOTHETA=`echo "scale=3; $TWOTHETA/1000" | bc -l`
 +
  echo 2THETA= $TWOTHETA
    
   let SKIP=1024+256+128+256+44
 
   let SKIP=1024+256+128+256+44
Line 346: Line 376:  
       BEAM_CENTER_X=`grep BEAM_CENTER_X tmp2 | sed s/BEAM_CENTER_X=//`
 
       BEAM_CENTER_X=`grep BEAM_CENTER_X tmp2 | sed s/BEAM_CENTER_X=//`
 
       BEAM_CENTER_Y=`grep BEAM_CENTER_Y tmp2 | sed s/BEAM_CENTER_Y=//`
 
       BEAM_CENTER_Y=`grep BEAM_CENTER_Y tmp2 | sed s/BEAM_CENTER_Y=//`
# fix 2010-04-26 - tell user about possible ORGX, ORGY alternatives -  
+
   
 
       COMMENT_ORGXY="
 
       COMMENT_ORGXY="
 
! Following are possible beam center interpretations for ADSC detectors"
 
! Following are possible beam center interpretations for ADSC detectors"
# at ESRF, PF, ALS 5.0.2 and ... (pls fill in!) the following should be used:         
+
# at ESRF, PF, ALS 5.0.2, AS MX2 and ... (pls fill in!) the following should be used:         
 
       ORGX1=`echo "scale=1; $BEAM_CENTER_Y/$QX" | bc -l`
 
       ORGX1=`echo "scale=1; $BEAM_CENTER_Y/$QX" | bc -l`
 
       ORGY1=`echo "scale=1; $BEAM_CENTER_X/$QX" | bc -l`
 
       ORGY1=`echo "scale=1; $BEAM_CENTER_X/$QX" | bc -l`
       echo - at ESRF, PF, APS Argonne BLs use: ORGX=$ORGX1 ORGY=$ORGY1                     
+
       echo - at ESRF, PF, ALS 8.2.1, APS Argonne BLs use: ORGX=$ORGX1 ORGY=$ORGY1                     
 
       COMMENT_ORGXY="${COMMENT_ORGXY}
 
       COMMENT_ORGXY="${COMMENT_ORGXY}
! ORGX= $ORGX1 ORGY= $ORGY1 ! For ESRF, PF, APS Argonne ..."
+
! ORGX= $ORGX1 ORGY= $ORGY1 ! For ESRF, PF, APS, AS MX2 ..."
 
# this 2nd alternative convention should be used at the following beamlines (pls complete the list): ALS 5.0.3, ...
 
# this 2nd alternative convention should be used at the following beamlines (pls complete the list): ALS 5.0.3, ...
 
       ORGX2=`echo "scale=1; $NX-$BEAM_CENTER_X/$QX" | bc -l `
 
       ORGX2=`echo "scale=1; $NX-$BEAM_CENTER_X/$QX" | bc -l `
Line 377: Line 407:  
       # Decision of beam center convention based on detector serial numbers.
 
       # Decision of beam center convention based on detector serial numbers.
 
       DET_SN=`grep DETECTOR_SN tmp2 | sed -e "s/DETECTOR_SN=//"`
 
       DET_SN=`grep DETECTOR_SN tmp2 | sed -e "s/DETECTOR_SN=//"`
       # For convention 1; Known PF detectors = 449: NW12A Q210, 472: NE3A Q270, 474: BL17A Q270, 912: BL5A Q315, 923: ALS BL5.0.2 Q315, 933: AichiSR BL2S1 Q315, 916: APS 24 IDE
+
      echo Detector serial number is $DET_SN
 +
       # For convention 1; Known PF detectors = 449: NW12A Q210, 472: NE3A Q270, 474: BL17A Q270, 912: BL5A Q315, 923: ALS BL5.0.2 Q315, 933: AichiSR BL2S1 Q315, 916: APS 24 IDE, 928: AS MX2
 
       ORG1_SNs="
 
       ORG1_SNs="
 
449
 
449
Line 388: Line 419:  
446
 
446
 
916
 
916
 +
905
 +
928
 
"
 
"
 
       ORG4_SNs="
 
       ORG4_SNs="
Line 395: Line 428:  
       ORGX=$ORGX1
 
       ORGX=$ORGX1
 
       ORGY=$ORGY1
 
       ORGY=$ORGY1
 +
      echo the following was chosen based on detector serial number:
 
       elif echo "${DET_SN}${ORG4_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
       elif echo "${DET_SN}${ORG4_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
       ORGX=$ORGX4
 
       ORGX=$ORGX4
 
       ORGY=$ORGY4
 
       ORGY=$ORGY4
 +
      echo the following was chosen based on detector serial number:
 
       else
 
       else
 
       ORGX=$ORGX3
 
       ORGX=$ORGX3
 
       ORGY=$ORGY3
 
       ORGY=$ORGY3
 +
      echo the following default was chosen because the detector serial number was not special-cased:
 
       fi
 
       fi
    
       # Check detector serial number and recognize beamline for reversed-phi setting.
 
       # Check detector serial number and recognize beamline for reversed-phi setting.
 
       # Known detectors for reversed-phi in SPring-8: 915: BL38B1 Q315; APS 19-ID: 458; BM30A: 924
 
       # Known detectors for reversed-phi in SPring-8: 915: BL38B1 Q315; APS 19-ID: 458; BM30A: 924
 +
      # 928 is at Australian Beamline MX2
 
       # revision 0.83 of this script removes 458 from the list!
 
       # revision 0.83 of this script removes 458 from the list!
 
       REVERSEPHI_SNs="
 
       REVERSEPHI_SNs="
 
915
 
915
 
924
 
924
 +
928
 
"
 
"
 
       if echo "${DET_SN}${REVERSEPHI_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
       if echo "${DET_SN}${REVERSEPHI_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
         REVERSE_PHI="yes"
 
         REVERSE_PHI="yes"
 +
        echo inverted ROTATION_AXIS since detector serial number is ${DET_SN}
 
       fi
 
       fi
   Line 434: Line 473:  
   DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=/-/`                                                                     
 
   DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=/-/`                                                                     
 
   OSCILLATION_RANGE=`grep OMEGA_DELTA tmp2 | sed s/OMEGA_DELTA=//`   
 
   OSCILLATION_RANGE=`grep OMEGA_DELTA tmp2 | sed s/OMEGA_DELTA=//`   
   DIRECTION_OF_DETECTOR_X_AXIS="-1 0 0"                                                                 
+
   DIRECTION_OF_DETECTOR_X_AXIS="-1 0 0"  
 +
                                                                  
 +
elif [ "$DET" == "experimental-ED" ]; then
 +
  DETECTOR="ADSC MINIMUM_VALID_PIXEL_VALUE= 1 OVERLOAD= 65000  ! OVERLOAD is really unknown "
 +
  echo --- Electron diffraction data in SMV format!
 +
  SENSOR_THICKNESS=0.01                                                       
 +
  sed s/\;// tmp2 > tmp1                                                     
 +
  mv tmp1 tmp2                                                               
 +
  X_RAY_WAVELENGTH=`grep WAVELENGTH tmp2 | head -1 | sed s/WAVELENGTH=//`
 +
  NX=`grep SIZE1 tmp2 | tail -1 | sed s/SIZE1=//`
 +
  QX=`grep PIXEL_SIZE tmp2 | sed s/PIXEL_SIZE=//`
 +
  NY=`grep SIZE2 tmp2 | tail -1 | sed s/SIZE2=//`                                           
 +
  QY=$QX 
 +
  BEAM_CENTER_X=`grep BEAM_CENTER_X tmp2 | sed s/BEAM_CENTER_X=//`
 +
  BEAM_CENTER_Y=`grep BEAM_CENTER_Y tmp2 | sed s/BEAM_CENTER_Y=//`
 +
  QXY=`echo "scale=1; $QX*($BEAM_CENTER_X+$BEAM_CENTER_Y)" | bc -l`
 +
      if [ "$QXY" "<" "$BEAM_CENTER_X" ] ; then
 +
      ORGX=`echo "scale=2; $BEAM_CENTER_Y/$QX" | bc -l`
 +
      ORGY=`echo "scale=2; $BEAM_CENTER_X/$QX" | bc -l`
 +
      else
 +
      ORGX=`echo "scale=2; $BEAM_CENTER_X/1" | bc -l`
 +
      ORGY=`echo "scale=2; $BEAM_CENTER_Y/1" | bc -l`
 +
      fi
 +
  DETECTOR_DISTANCE=`grep ^DISTANCE tmp2 | sed s/DISTANCE=//`                                                                   
 +
  OSCILLATION_RANGE=`grep OSC_RANGE tmp2 | sed s/OSC_RANGE=//`
 +
  STARTING_ANGLE=`grep OSC_START tmp2 | sed s/OSC_START=//`
 +
  echo --- For TIMEPIX detector, please fix ROTATION_AXIS yourself!
 +
  DIRECTION_OF_DETECTOR_X_AXIS="1 0 0"
 +
  REFINE_CORRECT="ORIENTATION CELL AXIS BEAM ! for ED, no POSITION when CELL is refined"
    
elif [ "$DET" == "pilatus" ]; then
 
elif [ "$DET" == "pilatus" ]; then
Line 488: Line 555:  
       if echo "${DET_SN}${REVERSEPHI_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
       if echo "${DET_SN}${REVERSEPHI_SNs}" | sort | uniq -d | grep [0-9] > /dev/null; then
 
         REVERSE_PHI="yes"
 
         REVERSE_PHI="yes"
 +
        echo inverted ROTATION_AXIS since detector serial number is ${DET_SN}
 
       fi
 
       fi
       if [ "$DET_SN" == "PILATUS XX, S/N XX-XXX" ] ; then
+
       if [ "$DET_SN" == "PILATUS XXX, S/N XX-XXX" ] ; then
 
         REVERSE_PHI="yes"
 
         REVERSE_PHI="yes"
 +
        echo inverted rotation axis at SSRF BL19U1
 
       fi
 
       fi
 
        
 
        
Line 497: Line 566:  
         if grep -q "Oscillation_axis X.CW +SLOW" tmp2 ; then
 
         if grep -q "Oscillation_axis X.CW +SLOW" tmp2 ; then
 
           rotation_axis="0 -1 0"
 
           rotation_axis="0 -1 0"
 +
          echo ROTATION_AXIS="0 -1 0" at Diamond I24
 
         fi
 
         fi
 
       fi
 
       fi
# insert similar code for Petra P14 here
+
# PETRA P14: raw data from Eiger are stored as CBF files so this is treated as Pilatus
 +
      if [ "$DET_SN" == "Dectris Eiger 16M, E-32-0107" -o "$DET_SN" == "Dectris Eiger 4M, E-08-0107" -o "$DET_SN" == "PILATUS 6M-F, S/N 60-0115-F" ] ; then
 +
        rotation_axis="0 -1 0"
 +
        echo ROTATION_AXIS="0 -1 0" at PETRA P14
 +
      fi
 +
# ESRF ID23-2:
 +
      if [ "$DET_SN" == "PILATUS3 2M, S/N 24-0118, ESRF ID23" ] ; then
 +
        rotation_axis="0 -1 0"
 +
        echo ROTATION_AXIS="0 -1 0" at ESRF ID23-2
 +
      fi
 +
 
 
elif [ "$DET" == "eiger" ]; then
 
elif [ "$DET" == "eiger" ]; then
   OVERLOAD=`h5dump -d "/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff" $FIRSTFRAME | awk '/\(0\):/{print $2}' `
+
# find out if HDF5 from Diamond (DLS=1) or Dectris (DLS=0)
 +
  DLS=0
 +
  OVERLOAD=`h5dump -d "/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff" $FIRSTFRAME 2>/dev/null` || DLS=1
 +
  if [ "$DLS" == 1 ]; then
 +
    echo Eiger HDF5 from Diamond
 +
# unfortunately h5dump 1.10 is required to get this right for the DLS .h5 files. This version is available at DLS but maybe not elsewhere
 +
    OVERLOAD=`h5dump -d "/entry/instrument/detector/saturation_value" $FIRSTFRAME | awk '/\(0\):/{print $2}'`
 +
  # v0.95: fix the next 2 lines by taking care of negative values with the \- , and stop after first "(0)"
 +
    OSCILLATION_RANGE=`h5dump -d "/entry/data/omega" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $3-$2;exit}'`
 +
    STARTING_ANGLE=`h5dump    -d "/entry/data/omega" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $2;exit}' | sed -e "s/,//"`
 +
    echo OSCILLATION_RANGE=$OSCILLATION_RANGE STARTING_ANGLE=$STARTING_ANGLE
 +
  # rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
 +
  # the above gives -1 0 0 for DLS data instead of the correct 1 0 0, so commented out for now
 +
   else
 +
    echo Eiger HDF5 from Dectris
 +
    OVERLOAD=`h5dump -d "/entry/instrument/detector/detectorSpecific/countrate_correction_count_cutoff" $FIRSTFRAME | awk '/\(0\):/{print $2}'`  
 +
    OSCILLATION_RANGE=`h5dump -d "/entry/sample/goniometer/omega_range_average" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
 +
  # STARTING_ANGLE:  the \- was introduced in version 0.91 to allow negative values :
 +
    STARTING_ANGLE=`h5dump -d "/entry/sample/goniometer/omega_start" $FIRSTFRAME | awk '/\(0\): [\-0-9]/{print $2}'`
 +
  # /entry/sample/goniometer/omega_start is missing in some eiger2 detectors (e.g. Eiger2 9M with fw version release-2020.2.1 and SIMPLON API 1.8) (Feng YU 2021-07-18)
 +
    if [ "$STARTING_ANGLE" == "" ]; then
 +
  echo "/entry/sample/goniometer/omega_start not found, try /entry/sample/goniometer/omega"
 +
  STARTING_ANGLE=`h5dump -d /entry/sample/goniometer/omega $FIRSTFRAME | grep "(0):" | head -n 1 | awk '{print $2}' | sed -e "s/,//g"`
 +
    fi
 +
    echo "STARTING_ANGLE=$STARTING_ANGLE"
 +
  # If rotation vector set (NeXus)
 +
    rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
 +
  # EIGER2 16M CHESS ID7B2 has S/N E-32-0123 (A. Finke 2020-11-07) v0.99
 +
    SN=`h5dump -d "/entry/instrument/detector/detector_number" $FIRSTFRAME | awk '/\(0\): /{print $2}' | sed s/\"//g`
 +
    if [ "$SN" == "E-32-0123" ]; then
 +
      rotation_axis="-1 0 0"
 +
      echo CHESS ID7B2 with inverted rotation axis
 +
    fi
 +
  # SSRF BL17U1 and SSRF BL10U2 (Feng YU 2021-07-18)
 +
  # Eiger X 16M (S/N E-32-0111) was installed at SSRF BL17U1 from Oct 2017 to Feb 2021. After Feb 2021, it was moved back to SSRF BL10U2.
 +
# 2021-02-01 00:00:00 CST time stamp is 1612108800
 +
    if [ "$SN" == "E-32-0111" ]; then
 +
      collection_time=`h5dump -d "/entry/instrument/detector/detectorSpecific/data_collection_date" $FIRSTFRAME | grep "(0):" | awk '{print $2}' | sed -e "s/\.\(.*\)/CST/g; s/\"//g"`
 +
      if [ `uname -s` == "Darwin" ]; then
 +
        collection_timestamp=`date -j -f "%Y-%m-%dT%H:%M:%S%Z" $collection_time +%s`
 +
      elif [ `uname -s` == "Linux" ]; then
 +
        collection_timestamp=`date -d $collection_time +%s`
 +
      else
 +
        collection_timestamp=0
 +
      fi
 +
      if [ $collection_timestamp -eq 0 ]; then
 +
        is_rotation_axis_set=1
 +
        rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0`
 +
        if [ "$is_rotation_axis_set" == "1" ]; then
 +
          rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"`
 +
          echo "The rotation axis of SSRF BL10U2 (Eiger X 16M) is $rotation_axis"
 +
        else
 +
          rotation_axis="-1 0 0    ! Cannot determine rotation axis. SSRF BL17U1: -1 0 0; SSRF BL02U1: 0 -1 0"
 +
        fi
 +
      elif [ $collection_timestamp -ge 0 ] && [ $collection_timestamp -le 1612108800 ]; then
 +
        # SSRF BL17U1
 +
        rotation_axis="-1 0 0"
 +
        echo "SSRF BL17U1 (Eiger X 16M) with inverted rotation axis"
 +
      else
 +
        # SSRF BL10U2
 +
        is_rotation_axis_set=1
 +
        rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0`
 +
        if [ "$is_rotation_axis_set" == "1" ]; then
 +
          rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"`
 +
          echo "The rotation axis of SSRF BL10U2 (Eiger X 16M) is $rotation_axis"
 +
        else
 +
          rotation_axis="0 -1 0"
 +
          echo "SSRF BL10U2 (Eiger X 16M) with vertical rotation axis"
 +
        fi
 +
      fi
 +
    fi
 +
  # SSRF BL02U1 (Feng YU 2021-07-18)
 +
  # EIGER2 S 9M SSRF BL02U1 has S/N E-18-0121
 +
    if [ "$SN" == "E-18-0121" ]; then
 +
      is_rotation_axis_set=1
 +
      rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME 2>/dev/null || is_rotation_axis_set=0`
 +
      if [ "$is_rotation_axis_set" == "1" ]; then
 +
        rotation_axis=`h5dump -d "/SSRF/RotationAxis" $FIRSTFRAME | grep "(0):" | sed -e "s/^.*://; s/,//g"`
 +
        echo "The rotation axis of SSRF BL02U1 (Eiger2 S 9M) is $rotation_axis"
 +
      else
 +
        rotation_axis="1 0 0"
 +
        echo "SSRF BL02U1 (Eiger2 S 9M) with horizontal rotation axis"
 +
      fi
 +
    fi
 +
  fi
 
   DETECTOR="EIGER MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= $OVERLOAD"
 
   DETECTOR="EIGER MINIMUM_VALID_PIXEL_VALUE=0 OVERLOAD= $OVERLOAD"
 
   QX=`h5dump -d "/entry/instrument/detector/x_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
 
   QX=`h5dump -d "/entry/instrument/detector/x_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
 
   QY=`h5dump -d "/entry/instrument/detector/y_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
 
   QY=`h5dump -d "/entry/instrument/detector/y_pixel_size" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
   −
   echo Data from a Eiger hdf5
+
   echo OVERLOAD=$OVERLOAD
 
   SENSOR_THICKNESS=`h5dump -d "/entry/instrument/detector/sensor_thickness" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
 
   SENSOR_THICKNESS=`h5dump -d "/entry/instrument/detector/sensor_thickness" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
 
   X_RAY_WAVELENGTH=`h5dump -d "/entry/instrument/beam/incident_wavelength" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
 
   X_RAY_WAVELENGTH=`h5dump -d "/entry/instrument/beam/incident_wavelength" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
Line 517: Line 681:  
   ORGY=`h5dump -d "/entry/instrument/detector/beam_center_y" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
 
   ORGY=`h5dump -d "/entry/instrument/detector/beam_center_y" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
   −
   # find DETECTOR_DISTANCE and OSCILLATION_RANGE:
+
   # find DETECTOR_DISTANCE :
   DETECTOR_DISTANCE=`h5dump -d "/entry/instrument/detector/detector_distance" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
+
   DETECTOR_DISTANCE=`h5dump -d "/entry/instrument/detector/detector_distance" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2*1000}'`
  OSCILLATION_RANGE=`h5dump -d "/entry/sample/goniometer/omega_range_average" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
  −
 
  −
  #STARTING_ANGLE:
  −
  STARTING_ANGLE=`h5dump -d "/entry/sample/goniometer/omega_start" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
  −
 
  −
  # If rotation vector set (NeXus)
  −
  rotation_axis=`h5dump -a "/entry/sample/transformations/omega/vector" $FIRSTFRAME 2>/dev/null | grep "(0):" | sed -e "s/^.*://; s/,//g"`
  −
  # Eiger 16M SSRF BL17U1
  −
  SN=`h5dump -d "/entry/instrument/detector/detector_number" $FIRSTFRAME | awk '/\(0\): [0-9]/{print $2}'`
  −
  if [ "$SN" == "E-32-0111" ]; then
  −
    rotation_axis="-1 0 0"
  −
  fi
      
   SEPMIN=4
 
   SEPMIN=4
 
   CLUSTER_RADIUS=2
 
   CLUSTER_RADIUS=2
 
+
 
 
elif [ "$DET" == "raxis" ]; then
 
elif [ "$DET" == "raxis" ]; then
 
   echo Data from a RAXIS detector
 
   echo Data from a RAXIS detector
Line 541: Line 693:  
   #let SKIP=768
 
   #let SKIP=768
 
   #NX=$(od -t x -j $SKIP -N 4 $FIRSTFRAME |awk 'NR==1{print toupper($2)}'|perl -nle '@array= $_ =~/.{2}/g; print "ibase=16;obase=A;".join("",reverse @array)'|bc)
 
   #NX=$(od -t x -j $SKIP -N 4 $FIRSTFRAME |awk 'NR==1{print toupper($2)}'|perl -nle '@array= $_ =~/.{2}/g; print "ibase=16;obase=A;".join("",reverse @array)'|bc)
   NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print "%.4d"%struct.unpack(">i",f.read(4))')
+
!  NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print "%.4d"%struct.unpack(">i",f.read(4))')
   NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print "%.4d"%struct.unpack(">i",f.read(4))')
+
!  NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print "%.4d"%struct.unpack(">i",f.read(4))')
 +
   NX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(768);print("%.4d"%struct.unpack(">i",f.read(4)))')
 +
   NY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(772);print("%.4d"%struct.unpack(">i",f.read(4)))')
   −
   DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print "-%.4f"%struct.unpack(">f",f.read(4))')
+
!  DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print "-%.4f"%struct.unpack(">f",f.read(4))')
 +
   DETECTOR_DISTANCE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(344);print("-%.4f"%struct.unpack(">f",f.read(4)))')
   −
   ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print "%.4f"%struct.unpack(">f",f.read(4))')
+
!  ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print "%.4f"%struct.unpack(">f",f.read(4))')
   ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print "%.4f"%struct.unpack(">f",f.read(4))')
+
!  ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print "%.4f"%struct.unpack(">f",f.read(4))')
 +
   ORGX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(540);print("%.4f"%struct.unpack(">f",f.read(4)))')
 +
   ORGY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(544);print("%.4f"%struct.unpack(">f",f.read(4)))')
   −
   OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print "%.4f"%(phie-phis)')
+
!  OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print "%.4f"%(phie-phis)')
 +
   OSCILLATION_RANGE=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(524);phis,phie=struct.unpack(">ff",f.read(8));print("%.4f"%(phie-phis))')
   −
   QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print "%.6f"%struct.unpack(">f",f.read(4))')
+
!  QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print "%.6f"%struct.unpack(">f",f.read(4))')
   QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print "%.6f"%struct.unpack(">f",f.read(4))')
+
!  QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print "%.6f"%struct.unpack(">f",f.read(4))')
 +
   QX=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(776);print("%.6f"%struct.unpack(">f",f.read(4)))')
 +
   QY=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(780);print("%.6f"%struct.unpack(">f",f.read(4)))')
   −
   X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print "%.6f"%struct.unpack(">f",f.read(4))')
+
!  X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print "%.6f"%struct.unpack(">f",f.read(4))')
 +
   X_RAY_WAVELENGTH=$(python -c 'import struct; f=open("'$FIRSTFRAME'","rb");f.seek(292);print("%.6f"%struct.unpack(">f",f.read(4)))')
    
elif [ "$DET" == "dtrek" ]; then
 
elif [ "$DET" == "dtrek" ]; then
Line 649: Line 810:  
  DETECTOR="MAR345  MINIMUM_VALID_PIXEL_VALUE=0  OVERLOAD=130000"
 
  DETECTOR="MAR345  MINIMUM_VALID_PIXEL_VALUE=0  OVERLOAD=130000"
 
  NX=`awk '/FORMAT/{print $2}' tmp2`
 
  NX=`awk '/FORMAT/{print $2}' tmp2`
  NY=$NX
+
# next line is rev 1.02 (previously NY= $NX):
 +
  NY=`awk -v NX=$NX '/FORMAT/{print $4/NX}' tmp2`
 
  QX=`awk '/PIXEL/{print $3/1000.}' tmp2`
 
  QX=`awk '/PIXEL/{print $3/1000.}' tmp2`
 
  QY=`awk '/PIXEL/{print $5/1000.}' tmp2`
 
  QY=`awk '/PIXEL/{print $5/1000.}' tmp2`
Line 664: Line 826:  
  X_RAY_WAVELENGTH=`grep 'WAVELENGTH' tmp2 | awk '{print $2}'`  
 
  X_RAY_WAVELENGTH=`grep 'WAVELENGTH' tmp2 | awk '{print $2}'`  
 
  OSCILLATION_RANGE=`grep 'PHI' tmp2 | awk '{print $5-$3}'`     
 
  OSCILLATION_RANGE=`grep 'PHI' tmp2 | awk '{print $5-$3}'`     
  TRUSTED_REGION="0 0.99"         
+
  TRUSTED_REGION="0 0.99"  
 +
 
 +
elif [ "$DET" == "OLDMAR" ]; then
 +
 
 +
echo  "Data from old type MAR image plate detector"
 +
DETECTOR="MAR  MINIMUM_VALID_PIXEL_VALUE=0  OVERLOAD=130000"
 +
NX=`awk 'NR==2 {print $2}' tmp2`
 +
NY=$NX
 +
QX=`awk 'NR==2 {print $15}' tmp2`
 +
QY=$QX
 +
ORGX=`awk 'NR==2 {print $19}' tmp2`
 +
ORGY=`awk 'NR==2 {print $20}' tmp2`
 +
DETECTOR_DISTANCE=`awk 'NR==2 {print $22}' tmp2`             
 +
X_RAY_WAVELENGTH=`awk 'NR==2 {print $21}' tmp2`
 +
OSCILLATION_RANGE=`awk 'NR==2 {print $24-$23}' tmp2`   
 +
TRUSTED_REGION="0 0.99" 
 +
rotation_axis="0 1 0" 
 +
echo unsure if sign of anomalous signal is correct - please verify or try both hands!
 +
        
 
elif [ "$DET" == "Bruker-cbf" ]; then
 
elif [ "$DET" == "Bruker-cbf" ]; then
 
    
 
    
Line 693: Line 873:     
  X_RAY_WAVELENGTH=`awk '/diffrn_radiation_wavelength.wavelength/{print $2}' tmp2`  
 
  X_RAY_WAVELENGTH=`awk '/diffrn_radiation_wavelength.wavelength/{print $2}' tmp2`  
  OMEGA=`awk '/OMEGA \? \? \?/{print $5}' tmp2`
+
# fix 16.3.20: instead of print, use printf "%.5f", because bc does not accept e.g. 3.1e-005
  DELTAOMEGA=`awk '/OMEGA \? \? \?/{print $6}' tmp2`
+
  OMEGA=`awk '/OMEGA \? \? \?/{printf "%.5f",$5}' tmp2`
  PHI=`awk '/PHI \? \? \?/{print $5}' tmp2`
+
  DELTAOMEGA=`awk '/OMEGA \? \? \?/{printf "%.5f",$6}' tmp2`
  DELTAPHI=`awk '/PHI \? \? \?/{printf "%5.4f",$6}' tmp2`
+
  PHI=`awk '/PHI \? \? \?/{printf "%.5f",$5}' tmp2`
  KAPPA=`awk '/CHI \? \? \?/{print $5}' tmp2`
+
  DELTAPHI=`awk '/PHI \? \? \?/{printf "%.5f",$6}' tmp2`
 +
  KAPPA=`awk '/CHI \? \? \?/{printf "%.5f",$5}' tmp2`
 
# echo OMEGA DELTAOMEGA PHI DELTAPHI KAPPA= $OMEGA $DELTAOMEGA $PHI $DELTAPHI $KAPPA
 
# echo OMEGA DELTAOMEGA PHI DELTAPHI KAPPA= $OMEGA $DELTAOMEGA $PHI $DELTAPHI $KAPPA
   Line 780: Line 961:  
SEPMIN=$SEPMIN  CLUSTER_RADIUS=$CLUSTER_RADIUS ! 4 and 2 for Pixel Array Detectors
 
SEPMIN=$SEPMIN  CLUSTER_RADIUS=$CLUSTER_RADIUS ! 4 and 2 for Pixel Array Detectors
 
! since XDS 01-MAR-2015, POSITION supersedes DISTANCE.
 
! since XDS 01-MAR-2015, POSITION supersedes DISTANCE.
! nowadays headers are usually correct so refine POSITION in INTEGRATE but not IDXREF
+
! nowadays headers are usually correct so refine POSITION in INTEGRATE but not IDXREF if low to medium resolution
REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS  ! refine POSITION only if known that header distance inaccurate
+
! however, if the spots from COLSPOT extend to 2A then POSITION could, and if 1.5A POSITION should be refined
REFINE(INTEGRATE)= POSITION BEAM ORIENTATION ! AXIS CELL
+
REFINE(IDXREF)=CELL BEAM ORIENTATION AXIS  ! add POSITION if high resolution, or DETECTOR_DISTANCE inaccurate
! REFINE(CORRECT)=CELL BEAM ORIENTATION AXIS POSITION ! Default is: refine everything
+
REFINE(INTEGRATE)= POSITION BEAM ORIENTATION ! AXIS CELL . If 1.5A or higher it is ok to refine CELL (unless electron diffraction)
 
+
REFINE(CORRECT)= $REFINE_CORRECT
 
! parameters specifically for this detector and beamline:
 
! parameters specifically for this detector and beamline:
 
DETECTOR= $DETECTOR
 
DETECTOR= $DETECTOR
Line 796: Line 977:     
if [ "$DET" == "eiger" ] && [ "$is_h5" == 1 ]; then
 
if [ "$DET" == "eiger" ] && [ "$is_h5" == 1 ]; then
   if [ -e /usr/local/lib64/dectris-neggia.so ]; then
+
   if [ "$DLS" == 0 ] ; then
    echo LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
+
    if [ -e /usr/local/lib64/dectris-neggia.so ]; then
    echo LIB= line was written to XDS.INP
+
      echo LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
 +
      echo LIB= line was written to XDS.INP
 +
    else
 +
      echo !LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
 +
      echo /usr/local/lib64/dectris-neggia.so was not found - specify location manually!
 +
    fi
 
   else
 
   else
     echo !LIB=/usr/local/lib64/dectris-neggia.so >> XDS.INP
+
     if [ -e /usr/local/lib64/durin-plugin.so ]; then
    echo /usr/local/lib64/dectris-neggia.so was not found - specify location manually!
+
      echo LIB=/usr/local/lib64/durin-plugin.so >> XDS.INP
   fi
+
      echo LIB= line was written to XDS.INP
 +
    else
 +
      echo !LIB=/usr/local/lib64/durin-plugin.so >> XDS.INP
 +
      echo /usr/local/lib64/durin-plugin.so was not found - specify location manually!
 +
    fi
 +
   fi  
 
fi  
 
fi  
   Line 866: Line 1,057:  
!                                                  and does not seem to have any downsides.  
 
!                                                  and does not seem to have any downsides.  
 
eof
 
eof
   if [ $NX == "1475" ]; then
+
   if [ "$NX" == "1028"  -a "$NY" == "1062" ]; then
 +
# Eiger2 1M ; v0.97 numbers from Andreas Förster
 +
    cat >> XDS.INP << eof
 +
UNTRUSTED_RECTANGLE=    0 1029    512  551
 +
eof
 +
  elif [ "$NX" == "1475" ]; then
 
     if ! grep -q Flat_field tmp2 ; then
 
     if ! grep -q Flat_field tmp2 ; then
 
     cat >> XDS.INP << eof
 
     cat >> XDS.INP << eof
Line 899: Line 1,095:  
eof
 
eof
 
     fi
 
     fi
   elif [ $NX == "2463" ]; then
+
   elif [ "$NX" == "2068"  -a "$NY" == "2162" ]; then
 +
# Eiger2 4M ; v0.97 numbers from Andreas Förster
 +
    cat >> XDS.INP << eof
 +
!EXCLUSION OF VERTICAL DEAD AREAS OF THE EIGER2 4M DETECTOR
 +
UNTRUSTED_RECTANGLE= 1028 1041      0 2163
 +
!EXCLUSION OF HORIZONTAL DEAD AREAS OF THE EIGER 4M DETECTOR
 +
UNTRUSTED_RECTANGLE=    0 2069    512  551
 +
UNTRUSTED_RECTANGLE=    0 2069  1062 1101
 +
UNTRUSTED_RECTANGLE=    0 2069  1612 1651
 +
eof
 +
  elif [ "$NX" == "2463" ]; then
 
# Pilatus 6M
 
# Pilatus 6M
 
# FIXME: here we could test if a Flat_field correction was applied like we do for 2M
 
# FIXME: here we could test if a Flat_field correction was applied like we do for 2M
Line 919: Line 1,125:  
UNTRUSTED_RECTANGLE=  0 2464  2315 2333
 
UNTRUSTED_RECTANGLE=  0 2464  2315 2333
 
eof
 
eof
   elif [ $NX == "3110"  -a $NY == "3269" ]; then
+
   elif [ "$NX" == "3110"  -a "$NY" == "3269" ]; then
 
# Eiger 9M
 
# Eiger 9M
 
     cat >> XDS.INP << eof
 
     cat >> XDS.INP << eof
Line 932: Line 1,138:  
UNTRUSTED_RECTANGLE= 0 3110 2717 2757
 
UNTRUSTED_RECTANGLE= 0 3110 2717 2757
 
eof
 
eof
   elif [ $NX == "4150" -a $NY == "4371" ]; then
+
   elif [ "$NX" == "3108"  -a "$NY" == "3262" ]; then
 +
# Eiger2 9M ; v0.97 numbers from Andreas Förster
 +
    cat >> XDS.INP << eof
 +
!EXCLUSION OF VERTICAL DEAD AREAS OF THE EIGER2 9M DETECTOR
 +
UNTRUSTED_RECTANGLE= 1028 1041      0 3262
 +
UNTRUSTED_RECTANGLE= 2068 2081      0 3263
 +
!EXCLUSION OF HORIZONTAL DEAD AREAS OF THE EIGER2 9M DETECTOR
 +
UNTRUSTED_RECTANGLE=    0 3109    512  551
 +
UNTRUSTED_RECTANGLE=    0 3109  1062 1101
 +
UNTRUSTED_RECTANGLE=    0 3109  1612 1651
 +
UNTRUSTED_RECTANGLE=    0 3109  2162 2201
 +
UNTRUSTED_RECTANGLE=    0 3109  2712 2751
 +
eof
 +
  elif [ "$NX" == "4150" -a "$NY" == "4371" ]; then
 
# Eiger 16M
 
# Eiger 16M
 
     cat >> XDS.INP << eof
 
     cat >> XDS.INP << eof
Line 947: Line 1,166:  
  UNTRUSTED_RECTANGLE= 2069 2082      0 4371  
 
  UNTRUSTED_RECTANGLE= 2069 2082      0 4371  
 
  UNTRUSTED_RECTANGLE= 3109 3122      0 4371
 
  UNTRUSTED_RECTANGLE= 3109 3122      0 4371
 +
eof
 +
  elif [ "$NX" == "4148" -a "$NY" == "4362" ]; then
 +
# Eiger2 16M ; v0.97 numbers from Andreas Förster
 +
    cat >> XDS.INP << eof
 +
!EXCLUSION OF HORIZONTAL DEAD AREAS OF THE EIGER2 16M DETECTOR
 +
UNTRUSTED_RECTANGLE= 1028 1041      0 4363
 +
UNTRUSTED_RECTANGLE= 2068 2081      0 4363
 +
UNTRUSTED_RECTANGLE= 3108 3121      0 4363
 +
!EXCLUSION OF VERTICAL DEAD AREAS OF THE EIGER2 16M DETECTOR
 +
UNTRUSTED_RECTANGLE=    0 4149    512  551
 +
UNTRUSTED_RECTANGLE=    0 4149  1062 1101
 +
UNTRUSTED_RECTANGLE=    0 4149  1612 1651
 +
UNTRUSTED_RECTANGLE=    0 4149  2162 2201
 +
UNTRUSTED_RECTANGLE=    0 4149  2712 2751
 +
UNTRUSTED_RECTANGLE=    0 4149  3262 3301
 +
UNTRUSTED_RECTANGLE=    0 4149  3812 3851
 
eof
 
eof
 
   fi
 
   fi
Line 956: Line 1,191:  
echo BKGPIX.cbf, and the agreement of predicted and observed spots in FRAME.cbf!
 
echo BKGPIX.cbf, and the agreement of predicted and observed spots in FRAME.cbf!
 
rm -f tmp1 tmp2
 
rm -f tmp1 tmp2
 +
 
</pre>
 
</pre>
   Line 969: Line 1,205:     
See also [[Generate_XDS.INP#Dependencies]] below, and the [[Installation]] article.
 
See also [[Generate_XDS.INP#Dependencies]] below, and the [[Installation]] article.
 +
 +
== Copying generate_XDS.INP from XDSwiki webserver ==
 +
On Linux:
 +
wget https://strucbio.biologie.uni-konstanz.de/pub/linux_bin/generate_XDS.INP
 +
chmod a+x generate_XDS.INP
 +
On Mac:
 +
curl -o generate_XDS.INP https://strucbio.biologie.uni-konstanz.de/pub/linux_bin/generate_XDS.INP
 +
chmod a+x generate_XDS.INP
 +
See also [[Installation]].
    
== Obtaining generate_XDS.INP from this webpage ==
 
== Obtaining generate_XDS.INP from this webpage ==
Line 1,004: Line 1,249:  
The script makes use of many GNU commands, like <code>ls, grep, egrep, awk, cut, cat, echo, wc, bc, head, sed, tail, cp, od, python</code>. Some of them (like <code>od</code> and <code>python</code>) are only used in case of specific detectors (MarCCD and RAXIS, respectively).  
 
The script makes use of many GNU commands, like <code>ls, grep, egrep, awk, cut, cat, echo, wc, bc, head, sed, tail, cp, od, python</code>. Some of them (like <code>od</code> and <code>python</code>) are only used in case of specific detectors (MarCCD and RAXIS, respectively).  
 
The script will only work if all the required commands are available. They reside in either the <code>coreutils</code> RPM, or specific RPMs (<code>gawk, sed, bc, grep, python</code> ...). Please note that to get the <code>strings</code> command on some Linux distributions (e.g. FC23) you need to install the <code>binutils</code> RPM package.
 
The script will only work if all the required commands are available. They reside in either the <code>coreutils</code> RPM, or specific RPMs (<code>gawk, sed, bc, grep, python</code> ...). Please note that to get the <code>strings</code> command on some Linux distributions (e.g. FC23) you need to install the <code>binutils</code> RPM package.
For Eiger data processing, the <code>h5dump</code> program must be installed. This is part of <code>hdf5-tools</code> (Ubuntu) or <code>hdf5</code> (RHEL).
+
For Eiger data processing, the <code>h5dump</code> program must be installed. This is part of <code>hdf5-tools</code> (Ubuntu) or <code>hdf5</code> (RHEL). The .h5 files collected at Diamond Light Source require a very new version of h5dump (namely h5dump 1.10) to extract the OVERLOAD parameter from the .h5 file; this version is available by default in Ubuntu 20.04 and RHEL/CentOS 8.
   −
On Mac OS X, installation of the "Command Line Tools" (from http://developer.apple.com/downloads; requires Apple ID) is required. These are also part of the (larger, but also free) [http://developer.apple.com/tools/xcode Xcode] package. This package comes with a license that has to be accepted by the user when running a Command Line Tool (e.g. <code>strings</code>) for the first time.
+
On Mac OS X, installation of the "Command Line Tools" (from http://developer.apple.com/downloads; requires Apple ID) is required (open a terminal and type <code>xcode-select --install</code>). These are also part of the (larger, but also free) [http://developer.apple.com/tools/xcode Xcode] package. This package comes with a license that has to be accepted by the user when running a Command Line Tool (e.g. <code>strings</code>) for the first time.
   −
An easy way to check for missing programs is
+
One way to check for missing programs is
  which ls grep egrep awk cut cat echo wc bc head sed tail cp od python strings h5dump
+
  #!/bin/bash
 +
for i in  ls grep egrep awk cut cat echo wc bc head sed tail cp od python strings h5dump ; do
 +
    if [ ! -x /bin/$i ] && [ ! -x /usr/bin/$i ]; then
 +
      echo $i not found
 +
    fi
 +
done
    
A command that should go a long way in providing all these tools for RedHat-derived distros is (as root)
 
A command that should go a long way in providing all these tools for RedHat-derived distros is (as root)
Line 1,021: Line 1,271:     
* The script tries to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and one Bruker detector. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like ROTATION_AXIS, OSCILLATION_RANGE, X-RAY_WAVELENGTH, DETECTOR_DISTANCE, and XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].
 
* The script tries to interpret the header of the frames, so is currently limited to Dectris (Pilatus, Eiger), ADSC (Quantum), Rigaku (several types), MAR (CCD and image plate) detectors, and one Bruker detector. Other detectors need some values to be manually filled into XDS.INP - the relevant places are marked with XXX. These are detector properties (type, pixel size and number, min and max counts in a pixel), and experimental parameters like ROTATION_AXIS, OSCILLATION_RANGE, X-RAY_WAVELENGTH, DETECTOR_DISTANCE, and XORG, YORG. For fine-tuning of detector parameters, see the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates].
* The authors have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has a anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.
+
* The authors have made a "best effort" to provide a XDS.INP that results in the correct sign of the anomalous signal. In the case of one detector type (internally called Rigaku SMV) this requires reversal of one detector axis, and a negative DETECTOR_DISTANCE, as is found in some of the [http://xds.mpimf-heidelberg.mpg.de/html_doc/xds_prepare.html detector-specific templates]. '''For an unusual or unknown detector setup, the correct sign of the anomalous signal needs to be established and verified e.g. with a good dataset from a test crystal that has an anomalous signal.''' The authors do not take any responsibility for problems arising from incorrect sign of the anomalous signal, nor - obviously! - for any other mischief arising in or from data processing.
* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. We have only just started the article [[Beamline notes]], and the frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction.
+
* At some beamlines, the ROTATION_AXIS should be -1 0 0 ("backwards") instead of the usual 1 0 0 ("horizontal"), or even 0 1 0 ("vertical") like at one of the PETRA Hamburg BLs. The frame headers do not have this information, so the default chosen by [[generate_XDS.INP]] may be wrong and need manual correction. Pls also see the article [[Beamline notes]].
 
* Sometimes, the x- and y- values of the primary beam position recorded in the header should be used for ORGY and ORGX (i.e reversed) instead of as ORGX and ORGY. For ADSC, this has been implemented in the script for a number of beamlines.
 
* Sometimes, the x- and y- values of the primary beam position recorded in the header should be used for ORGY and ORGX (i.e reversed) instead of as ORGX and ORGY. For ADSC, this has been implemented in the script for a number of beamlines.
 +
* there are apparently several flavours of HDF5 files produced at Diamond Light Source. They differ e.g. in the naming of the header items. This means that items like NX, NY, DETECTOR_DISTANCE and number of images cannot be found by the [[generate_XDS.INP]] script. Example: The data at /dls/i04-1/data/2021/mx28114-9/processing/Lenye_Diamini/ThiL/ThiL found during the CCP4 2021 online Cape Town workshop. A workaround is to use e.g. xia2 pipeline=3dii to process these files, and - if needed - extract those items from its output files, e.g. from DEFAULT/NATIVE/SWEEP1/index/XDS.INP .
    
== See also ==
 
== See also ==
2,450

edits

Cookies help us deliver our services. By using our services, you agree to our use of cookies.

Navigation menu