December 18, 2020

Outline

  • Background
  • Neovim-based IDE for R
  • Parallel R with batchtools
  • References

R Language

About

  • Statistical environment and programming language (CRAN) widely used in academia and data science.
  • Free and runs on all common operating systems
  • Large ecosystem of extension packages, e.g. Bioconductor and CRAN

Working environments

RStudio Server for Web-based HPCC Access

  • Integrated development environment (IDE) for R. RStudio local GUI and RStudio Server is web-based.
  • User access to RStudio Server on HPCC

Some useful RStudio shortcuts:

  • Ctrl+Enter: send code to R console
  • Ctrl+Alt+Enter: send code to terminal
  • Ctrl+Shift+C: comment/uncomment
  • Ctrl+1/2: switch window focus

Nvim-R-Tmux: Terminal-based R Environment

Advantages of Command-line UI

  • Knowledge of command-line interface is essential for working on a computer cluster efficiently
  • Main advantage: language agnostic approach that works with most computer languages
  • Users of Emacs may want to consider using ESS instead

Outline

  • Background
  • Neovim-based IDE for R
  • Parallel R with batchtools
  • References

Introduction to Nvim-R-Tmux

  • The following introduces Nvim-R combined with Tmux.
  • Similar instructions are available in HPCC’s Nvim-R-Tmux tutorial here.
  • Note: Nvim-R and Tmux are two independent tools that can be used independently or in combination, and both are useful for remote terminal work.
  • For simplicity, some of the following examples use Nvim-R without Tmux. Once users know the basics of both then it is trivial to combine them as needed.

Vim/Nvim Basics

The following opens a file (here myfile.txt) with nvim (or vim). This can be a new file or an existing one.

nvim myfile.txt # for neovim (or 'vim myfile.txt' for vim)

Modes

In Vim/Nvim there are three main modes: normal, insert and command mode. The most important commands for switching between the three modes are:

  • i: switches from the normal to the insert mode. The latter is used for typing.
  • Esc: switches from the insert mode back to the normal mode.
  • :: starts the command mode (from normal mode) at the bottom of the terminal window.

The cursor is moved with the arrow keys. In Nvim one can also enable mouse-based movements of the cursor. Fn Up/Down allows to page. In the following, all commands starting with : need to be typed in the command mode. All other commands are typed in the normal mode after pushing the Esc key.

[ Scroll down to continue ]



Important modifier keys to control Vim/Nvim

  • :w: saves changes to file. If in editing mode, Esc needs to be pressed first.
  • :q: quits file that has not been changed; use q! to quit file without saving changes.
  • :wq: saves and quits file

Useful resources for learning Vim/Nvim

Tmux for Managing Terminal Sessions

What is Tmux?

  • Tmux is a virtual terminal multiplexer providing re-attachable terminal sessions
  • Advantage: work in a terminal session cannot get lost due to internet disruptions or even when switching computers
  • Combined with the Nvim-r plugin it provides a flexible working environment for R
  • Users can send code from a script to the R console or command-line.
  • On HPCC both Nvim-R and Tmux are pre-configured and easy to install





Nvim-R-Tmux Configuration in HPCC User Accounts

Skip these steps if Nvim-R-Tmux is already configured in your account. Or follow the detailed instructions to install Nvim-R-Tmux from scratch on your own system (e.g. laptop or computer).

1. Log in to your user account on HPCC and execute on the command-line:

install_nvimRtmux

2. To enable the nvim-R-tmux environment, log out and in again.

3. Follow usage instructions of next section.

Typical Usage Workflow for Nvim-R-Tmux

1. Start tmux session from login node (not compute node!)

Running Nvim from tmux provides reattachment functionality. Skip this step if this is not required.

tmux # starts a new tmux session 
tmux a # attaches to an existing session 

2. Open nvim-connected R session

Open a *.R or *.Rmd file with nvim and initialize a connected R session with \rf. Note, the resulting split window among Nvim and R behaves like a split viewport in nvim or vim meaning the usage of Ctrl-w w followed by i and Esc is important for session navigation.

nvim myscript.R # or *.Rmd file

3. Send R code from nvim to the R pane

Single lines of code can be sent from nvim to the R console by pressing the space bar. To send several lines at once, one can select them in nvim’s visual mode and then hit the space bar.

[ Scroll down to continue ]



  • Please note, the default command for sending code lines in the nvim-r-plugin is \l. This key binding has been remapped in the provided .config/nvim/init.vim file to the space bar. Most other key bindings (shortcuts) still start with the \ as LocalLeader, e.g. \rh opens the help for a function/object where the cursor is located in nvim. More details on this are given on the next slide(s).
  • The most comprehensive manual on this is the official Nvim-R documentation here.

Keybindings to Control Environment

Important keybindings for nvim

  • \rf: opens vim-connected R session. If you do this the first time in your user account, you might be asked to create an R directory under ~/. If so approve this action by pressing y.
  • spacebar: sends code from vim to R; here remapped in init.vim from default \l
  • :split or :vsplit: splits viewport (similar to pane split in tmux)
  • gz: maximizes size of viewport in normal mode (similar to Tmux’s Ctrl-a z zoom utility)
  • Ctrl-w w: jumps cursor to R viewport and back; toggle between insert (i) and command (Esc) mode is required for navigation and controlling the environment.
  • Ctrl-w r: swaps viewports
  • Ctrl-w =: resizes splits to equal size
  • :resize <+5 or -5>: resizes height by specified value


[ Scroll down to continue ]





  • :vertical resize <+5 or -5>: resizes width by specified value
  • Ctrl-w H or Ctrl-w K: toggles between horizontal/vertical splits
  • Ctrl-spacebar: omni completion for R objects/functions when nvim is in insert mode. Note, this has been remapped in init.vim from difficult to type default Ctrl-x Ctrl-o.
  • :h nvim-R: opens nvim-R’s user manual; navigation works the same as for any Vim/Nvim help document
  • :Rhelp fct_name: opens help for a function from nvim’s command mode with text completion support
  • Ctrl-s and Ctrl-x: freezes/unfreezes vim (some systems)

Important keybindings for tmux

Pane-level commands

  • Ctrl-a %: splits pane vertically
  • Ctrl-a ": splits pane horizontally
  • Ctrl-a o: jumps cursor to next pane
  • Ctrl-a Ctrl-o: swaps panes
  • Ctrl-a <space bar>: rotates pane arrangement
  • Ctrl-a Alt <left or right>: resizes to left or right
  • Ctrl-a Esc <up or down>: resizes to left or right

Window-level comands

  • Ctrl-a n: switches to next tmux window
  • Ctrl-a Ctrl-a: switches to previous tmux window
  • Ctrl-a c: creates a new tmux window
  • Ctrl-a 1: switches to specific tmux window selected by number

Session-level comands

  • Ctrl-a d: detaches from current session
  • Ctrl-a s: switch between available tmux sessions
  • $ tmux new -s <name>: starts new session with a specific name
  • $ tmux ls: lists available tmux session(s)
  • $ tmux attach -t <id>: attaches to specific tmux session
  • $ tmux attach: reattaches to session
  • $ tmux kill-session -t <id>: kills a specific tmux session
  • Ctrl-a : kill-session: kills a session from tmux command mode

Use Same Environment for Other Languages

Basics

For languages other than R one can use the vimcmdline plugin for nvim (or vim). Supported languages include Bash, Python, Golang, Haskell, JavaScript, Julia, Jupyter, Lisp, Macaulay2, Matlab, Prolog, Ruby, and Sage. The nvim terminal also colorizes the output, as in the screenshot below, where different colors are used for general output, positive and negative numbers, and the prompt line.

vimcmdline

Install

To install it, one needs to copy from the vimcmdline repository the directories ftplugin, plugin and syntax and their files to ~/.config/nvim/. For user accounts of UCR’s HPCC, the above install script install_nvimRtmux includes the install of vimcmdline (since 09-Jun-18).

Usage

The usage of vimcmdline is very similar to nvim-R. To start a connected terminal session, one opens with nvim a code file with the extension of a given language (e.g. *.sh for Bash or *.py for Python), while the corresponding interactive interpreter session is initiated by pressing the key sequence \s (corresponds to \rf under nvim-R). Subsequently, code lines can be sent with the space bar. More details are available here.

Nvim-R Demo

To try out the following instructions, users want to log into their HPCC account via ssh, and then preferentially connect to a node by initializing an interactive srun session. The latter mimics the best practices for a real workflow but is not necessary for this basic exercise.

srun --x11 --partition=short --mem=2gb --cpus-per-task 4 --ntasks 1 --time 1:00:00 --pty bash -l
  • Under --partition it is important to assign the name of a partition a user has access to
    • Most users have access to: short, batch, intel and highmem
    • Students registered under the statistics department, only have access to: statsdept
    • Users of labs owning computer nodes also can access: <pi_name>lab
  • For more details on argument settings for srun, see here

Download R_for_HPC_demo.R file to you HPCC account as follows.

wget https://raw.githubusercontent.com/ucr-hpcc/ucr-hpcc.github.io/master/presentations/2020-12-18_Workshop/R_for_HPC/demo_files/R_for_HPC_demo.R

[ Scroll down to continue ]