NOTE: Amber16 is optimally run on the amber128 queue.
Submitting Jobs
Short jobs (taking less than 10 minutes to run) can be run on the cottontail head node. This includes many of our python scripts. No header is needed.
sh jobname.sh
To submit a job that will take longer than this to the scheduler, a header is required (see below). Energy minimization and cpptraj will need the CPU header. Heating, equilibration, piggybacking, and dynamics will need the GPU header. To submit to the queuing system, enter into the terminal:
bsub < jobname.sh
It is acceptable to submit several jobs as a batch, but it is good practice to wait 1-2 minutes between submissions. For sequentially named jobs, this can be done with a loop:
for i in {1..5}
do
bsub < job_$i.sh
sleep 60
done
amber128 CPU Header
#!/bin/bash
#BSUB -q amber128
#BSUB -n 1
#BSUB -J jobname_CPU
#BSUB -o out
#BSUB -e error
# cd to working directory
cd /home/kscopino/SUMMER_20_A16/BPTI/EMIN
# call Amber16
/home/apps/CENTOS6/amber/amber16/bin/sander -O -i inputs.in -p prmtop_wat.prmtop -c coordinates_in.rst -r coordinates_out.rst -ref restraint_reference.rst -o energy_info.out
amber128 GPU Header
#!/bin/bash
#BSUB -q amber128
#BSUB -n 1
#BSUB -J jobname_GPU
#BSUB -o out
#BSUB -e err
#BSUB -R "rusage[gpu4=1:mem=12288],span[hosts=1]"
# GLOBAL VARIABLES
# cuda 8 & mpich for AMBER16
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export PATH=/usr/local/mpich-3.1.4/bin:/usr/local/amber16/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/mpich-3.1.4/lib:$LD_LIBRARY_PATH
# AMBER16
export PATH=/share/apps/CENTOS6/python/2.7.9/bin:$PATH
export LD_LIBRARY_PATH=/share/apps/CENTOS6/python/2.7.9/lib:$LD_LIBRARY_PATH
source /home/apps/CENTOS6/amber/amber16/amber.sh
# cd to working directory
cd /home/kscopino/SUMMER_20_A16/BPTI/EMIN
# call Amber16
n78.mpich3.wrapper pmemd.cuda.MPI -O -i inputs.in -p prmtop_wat.prmtop -c coordinates_in.rst -r coordinates_out.rst -ref restraint_reference.rst -o energy_info.out -x trajectory.mdcrd
Useful Troubleshooting Commands
# tells you the status of your jobs and which node they are running on
bjobs
# gives GPU identity (in the form of 00000000:3B:00.0) for each job on node 85, want 1 job per GPU
ssh n85 gpu-process
# gives information on % usage of each GPU (0 to 3) on node 85
ssh n85 gpu-info
# gives information on % usage of CPU for node 85
lsload n85
Additional Resources
Download Amber16 Manual
Wesleyan's Amber wiki
