Login to HPCC

ssh username@cluster.hpcc.ucr.edu

Create a Cluster

hpcc_cloud create <NameForYourCluster>

Show status of a Cluster

hpcc_cloud status <NameForYourCluster>

Output:

Status: cfncluster-new-cluster - CREATE_COMPLETE                                
Output:"MasterPublicIP"="52.52.227.148"
Output:"MasterPrivateIP"="172.31.24.51"
Output:"GangliaPublicURL"="http://52.52.227.148/ganglia/"
Output:"GangliaPrivateURL"="http://172.31.24.51/ganglia/"

Note the “MasterPublicIP” Address from the output. Use this IP Address when connecting to the cluster via “ssh” or uploading and downloading via “scp”.

Show running Clusters

hcpp_cloud list

Delete cluster

hpcc_cloud delete <NameForYourCluster>

Connecting to your cluster

Note - /path/to/your/key-file.pem = where you saved your AWS account key file

MasterPublicIP = Master Public IP address from the cluster status

ssh -i /path/to/your/key-file.pem centos@<MasterPublicIP>

Uploading data to your cluster

This will transfer the local files to your home directory on the cluster.

scp -i /path/to/your/key-file.pem <local-files-to-copy> centos@<MasterPublicIP>:.

Downloading data/results

This would be called from the HPCC cluster and it will download the specified remote files to your current directory.

scp -i /path/to/your/key-file.pem centos@<MasterPublicIP>:./<files-to-download> .

Running a job on your cluster

This will show all the steps needed to create a cluster and run a simple batch job

1. Start a new cluster

hpcc_cloud create new-cluster

2. Get the IP address of the new cluster

Once the cluster build is complete you will be presented with the status informatiom. Note the “MasterPublicIP” Address from the output. Use this IP Address when connecting to the cluster via “ssh” or uploading and downloading via “scp”.

Output:

hpcc_cloud create new-cluster
Beginning cluster creation for cluster: new-cluster
Creating stack named: cfncluster-new-cluster
Status: cfncluster-new-cluster - CREATE_COMPLETE                                
Output:"MasterPublicIP"="52.52.227.148"
Output:"MasterPrivateIP"="172.31.24.51"
Output:"GangliaPublicURL"="http://52.52.227.148/ganglia/"
Output:"GangliaPrivateURL"="http://172.31.24.51/ganglia/"

Find the “MasterPublicIP”

3. Upload your input files and slurm submission script

This will transfer the local files to your home directory on the cluster.

scp -i /path/to/your/key-file.pem <local-files-to-copy> centos@<MasterPublicIP>:.

4. SSH to your new cluster

ssh -i /path/to/your/key-file.pem centos@<MasterPublicIP>

5. Submit your job to the cluster

sbatch slurm-submission-script.sh

6. Monitor your job

squeue

7. Download results

This command would be called from the HPCC cluster and it will download the specified remote files to your current directory.

scp -i /path/to/your/key-file.pem centos@<MasterPublicIP>:./<results-to-download> .

8. Delete cluster

hpcc_cloud delete new-cluster 

Start sample cluster walk through (sped up)