Conda Installs

Presented by: Jordan Hayes (HPCC)

Summary

• Why Conda
  • Software Challenges
  • Solutions
• Configure
  • Virtual Environments
• Install Examples
  • Pandas
  • FastQC
  • Tensorflow

---

Why Conda: Software Challenges

• Non-Root access
• Dependency conflicts
• Sharable
• Easy to execute

---

Why Conda: Solutions

• Use non-root user
• Installs in separate environments
• Installs are modular and easy to share
• Easy to load/unload environments

---

Configure: Locations

Create the file ~/.condarc in your home, with the following content:

channels:
  - defaults
pkgs_dirs:
  - ~/bigdata/.conda/pkgs
envs_dirs:
  - ~/bigdata/.conda/envs
auto_activate_base: false

https://hpcc.ucr.edu/manuals/hpc_cluster/package_manage/#virtual-environments

---

Configure: Virtual Environment

First, lets request a compute node, since running on the head nodes would be slow:

srun -p short -c 2 --mem=10g --pty bash -l

Create conda virtual environment for pandasEnv, since pandas is a python pacakge, we will pre install Python:

conda create -y -n pandasEnv python=3.6.4

Activate pandasEnv virtual environment:

conda activate pandasEnv

---

Install Examples: Pandas

Now that we have a virtual environment, we can install packages within it. Visit anaconda website to search for a pagckage:

https://anaconda.org/

After you find the package, click on the name of the package to see install command.

#conda install -y -c anaconda pandas
conda install -y -n pandasEnv -c anaconda pandas

---

Install Examples: FastQC

Same proceedure as before:

1. Create fastqcEnv virtual environment
2. Activate fastqcEnv virtual environment
3. Look up fastqc on Anaconda
4. Install fastqc package
5. Deactivate fastqcEnv virtual environment

# Create virutal environment
conda create -y -n fastqcEnv

# Activate virutal environment
conda activate fastqcEnv

# Install fastqc
conda install -y -c bioconda fastqc

# Deactivate virtual environment
conda deactivate

---

Install Examples: Tensorflow

Same proceedure as before:

1. Create tensorflowEnv virtual environment
2. Activate tensorflowEnv virtual environment
3. Look up tensorflow on Anaconda
4. Install tensorflow package
5. Deactivate tensorflowEnv virtual environment

# Create virtual environment
conda create -y -n tensorflowEnv

# Activate virtual environment
conda activate tensorflow

# Install tensorflow
conda install -y -n tensorflowEnv -c conda-forge tensorflow

# Deactivate virtual environment
conda deactivate

---

Install Examples: Script

After the installs have completed, you may want to use 1 (or more) conda virtual environments within a job/script:

#!/bin/bash -l

#SBATCH -p short
#SBATCH -c 2
#SBATCH --mem=10gb
#SBATCH --time=10:00
#SBATCH -J "Conda Demo"

# Already loaded by default
module load miniconda2

#### pandasEnv ####
# Activate virtual environment
conda activate pandasEnv
# Show file path to where pandas is installed
python -c 'import pandas; print(pandas.__file__)'
# Deactivate virtual environment
conda deactivate

#### fastqcEnv ####
# Activate virtual environment
conda activate fastqcEnv
# Run help
fastqc --help
# Deactivate virtual environment
conda deactivate

#### tensorflowEnv ####
# Activate virtual environment
conda activate tensorflowEnv
# Add a scalar and a list
python -c 'import tensorflow as tf; x = [1, 2, 3, 4, 5]; y = 1; print(tf.add(x, y))'
# Deactivate virtual environment
conda deactivate