On the other side of the screen, it all looks so easy. (TRON, 1982)

Goals

By the end of this introduction, you should be able to:

  • Log in to the class cluster.
  • Navigate and understand the directory structure.
  • Create and edit files and directories on the cluster.
  • Move files to and from the cluster.

Ask for help if any of these things aren’t working by the end!

Slack workspace

Slack is a messaging/discussion program that allows for large groups to interact as needed.
There is a slack workspace set up for people to ask questions and to have discussions.
The Slack Workspace is MBL-MOLE-2024
You can use the Slack app on your laptop and your phone.
If you’ve lost your invitation to join this workspace, ask one of us and we’ll resend the invitation to your email address.

Wi-Fi Login

You should have received instructions for connecting to Wi-Fi with your room key. If you have trouble connecting to the internet, let one of the TAs or course directors know and we’ll help you get on the network.

Downloads

Please download and install the following programs if you haven’t already:

Windows

  • Cyberduck (file transfer via sFTP)
  • Git for Windows (This is a UNIX emulator for running command-line programs)
  • Notepad++ After you have installed Notepad++, open it. Go to Settings, Preferences, New Document, and set the line endings to Unix.
  • Seaview (used to visualize small alignments)
  • FigTree (used to visualize phylogenetic trees)
  • Tracer (used for Bayesian divergence time analyses)

macOS

  • SCP and SSH installed by default
  • Cyberduck (file transfer via sFTP)
  • BBEdit (BBEdit is the 64 bit version of TextWrangler for contemporary macOS)
  • Seaview (used to visualize small alignments)
  • FigTree (used to visualize phylogenetic trees)
  • Tracer (used for Bayesian divergence time analyses)
  • iTerm2 (optional, but many of us like this terminal program better than the one that comes with macOS)

Linux

  • SCP and SSH installed by default
  • Seaview (used to visualize small alignments)
  • FigTree (used to visualize phylogenetic trees)
  • Tracer (used for Bayesian divergence time analyses)

You’re set!

We will be using more software throughout the workshop than what we’re asking you to download here, all of which should be accessible on the remote servers. Feel free to download and use any and all software on your personal device as well, just know that not everything will necessarily finish running in the allotted time, depending on your device.

Remote computer access software and SSH

You will use Secure Shell (SSH) and Secure File Transfer Protocol (sFTP) to connect to your own personal virtual machine (VM). You will interact with your VM solely by typing or pasting commands into a terminal window. This way of interacting with a computer may not be familiar to you, but you will get some practice during this lab.

Shells are computer programs that represent the outermost (i.e. closest to you) layer of the operating system. Shells hide most of the gory details, allowing you to interact with the operating system via commands. Another (better) name for a shell is command interpreter. SSH represents a shell that is secure in that every bit of text that moves between your keyboard and the operating system (and vice versa) is encrypted.

The cluster we will be using is from the Jetstream2 project at Indiana University. Jetstream was NSF’s first science and engineering research cloud, completed in early 2016. Jetstream is one source of computing infrastructure provided under the umbrella of Xsede, the Extreme Science and Engineering Discovery Environment, an NSF-sponsored program that provides cloud computing for researchers in academia.

SSH on Linux and macOS

First, open a terminal window:

  • Mac: Terminal or iTerm2 (in /Applications/Utilities)
  • Linux: Konsole (KDE), gnome-terminal (GNOME)

Login to your particular VM using the IP address on the sticker attached to the back of your name tag. If, for example, your IP address was 123.456.789.321, you would type the following into the terminal on your local computer (i.e. your laptop) and press the enter key:

ssh moleuser@123.456.789.321

SSH on Windows

On Windows you should use Git for Windows to access your virtual machine (VM). Assuming you’ve installed Git for Windows, choose Git BASH from the All Programs section of the Start menu. This will open a terminal running the shell program bash.

Now login to your particular virtual machine (VM) using the IP address on the sticker attached to the back of your name tag. If, for example, your IP address was 123.456.789.321, you would type the following into the terminal on your local computer (i.e. your laptop) and press the enter key:

ssh moleuser@123.456.789.321

Changing your password

The first thing to do once you have successfully logged onto your VM is to change your password. This is done by typing:

passwd

This will prompt you to enter a new password, so do so and press the enter/return key. Next you re-enter the new password and again press enter.

Once you have done this every time you ssh in to your VM you will use the new password you just created (so don’t forget it!).

Basic Syntax

Unix commands follow the general format of:

command -options target

Not all commands need options or targets. Options are sometimes called flags and are generally preceded by a single hyphen (-) or a double hyphen (–).

For example:

  • cd mydir uses the command cd (change directory) and the target mydir to move from the current directory into the subdirectory called mydir.

  • ls -l mydir uses the command ls (list), the option -l for long-list, and the target mydir to list the contents of mydir in the long list format, which provides more thorough descriptions than does the regular list command.

Notes on syntax for directory structure

  • Two dots (..) represents the parent directory of the present working directory. So, for example, cd .. will move you back one directory

  • One dot (.) represents the present working directory. So, for example, cd . will keep you where you are. There are times where the single dot can be more useful than this…

  • The tilde (~) represents your home directory. On your virtual machine, your home directory is /home/moleuser. The tilde is very helpful if you get lost while using the terminal – just type cd ~ and you’ll be back in your homedirectory.

  • A forward slash (/) by itself or at the start of a path refers to the root of the directory structure – the folder that contains all other folders.

Intro-to-Unix tutorial

If you are not familiar with typical linux commands, please go to the Linux tutorial and go through the step by step guide there.

Setting up Cyberduck

Creating a bookmark

When you open Cyberduck a screen should appear that has some default connections bookmarked such as Google Docs and Amazon.

Click the + button at the BOTTOM LEFT to create a new bookmark (NOT the one with the globe at the top). Here we will put in the details of our virtual machine login and save it so that each time we open Cyberduck we don’t have to re-enter the details.

  • Click the dropdown menu that says FTP (File Transfer Protocol) and select SFTP (SSH File Transfer Protocol);

  • Create a nickname for this bookmark. For example mine is called ‘mole vm’;

  • For server write the IP address of the virtual machine you were asssigned. For example, 123.456.789.321;

  • For username type moleuser;

Once all these details are entered close that screen. (e.g. on a mac click the red circle or on windows click the X).

A new bookmark should have appeared named whatever you set as the nickname. Double click on this.

If you get a popup dialog box with a lot of ugly text in it, e.g.

Unknown fingerprint

The fingerprint for the ED25519 key sent by the server is 
c6:41:86:61:7d:87:56:90:cc:c9:3a:a5:56:dc:dd:ac

just press the Allow button. This is just letting you know that you are logging into a machine that you’ve never connected to before. This would only be a concern if you see it a second time (which may mean someone has swapped the computer you thought you were connecting to with a different machine).

Cyberduck should ask you for the password. Type in the password you created earlier. There should be a button asking for you to Add to Keychain or something similar. Make sure this box is ticked. Once done, click to Login (or connect).

This may connect you directly to the virtual machine. If not, double click on the bookmark again.

You now should see your home directory on the cluster. You should see the folder named myfolder that we created earlier as well as the symlink named eg (which looks like an ordinary folder except that the folder icon contains a tiny little arrow symbol, which clues you into the fact that this is an alias and not a real folder.

Navigating through the class cluster in cyberduck is easy.

Double click on a folder to open that folder.

Above the window that lists files you should see your path: e.g., /home/moleuser. This tells you where you are in the virtual machine’s directory structure. If you click this dropdown menu you can see each nested folder in the directory hierarchy above that point. Clicking on any of these intermediate directories will allow you to navigate to that directory. If you get lost in folders you can click this dropdown and select /home/moleuser to return to your home directory.

There are two ways to download files from the virtual machine to your computer.

  1. If you double click on a file name, this file will be automatically downloaded to your default download folder. A transfer window should open up when you double click on a file. At the bottom there will be a line that starts local file. This is the location of that downloaded file on your own computer.

  2. An easier way to get files both to and from the class cluster through Cyberduck is through drag and drop. If you have an open finder window (mac) or explorer window (windows) you can click and drag a file from the Cyberduck window to the finder/explorer window directly. You also can do this in the reverse (drag a file from finder/explorer to Cyberduck) to place a file on the cluster in that folder.

Using Cyberduck for remote editing

Instead of using nano to edit files on the virtual machine you can use a text editor such as BBEdit (mac) or Notepad++ (windows) through Cyberduck.

In Cyberduck, select (mac) the Cyberduck main menu item and then select Preferences or (windows) the Edit menu item and then Preferences.

Click the Editor button.

There should be a dropdown menu where you can select a text editor. Select BBEdit on a Mac, Notepad++ on Windows.

Close this window.

Now when you single click on a file in Cyberduck, an Edit button with the icon for the editor you selected should appear in the top right corner. Click this button to open that file in the editor. Now you can edit the file on the cluster without downloading to your local computer. When you save this file it will be automatically updated on the class cluster.

Short Cyberduck exercise

The easiest way to transfer files using CyberDuck is with drag-and-drop. Try to drag a file from your desktop (or another folder of your choosing) from the CyberDuck window. This method allows you to choose unique destinations for each file you transfer.

In CyberDuck, double click example.txt in your home directory on the virtual machine. To where does this file download? You can change the default download locations using the Edit, Preferences, Transfers menu item.

Lastly, right clicking (Ctrl click on a mac) will allow you to select Download To or Download As. This will allow you to choose a location, and, if you choose, a new file name for the file.

These methods will all work to get files from the cluster to your computer, and each has their own benefits. Generally, you will be fine using any of them throughout the course.

Make a folder on your local computer to hold all of your lab materials from this course!

SCP is a command line alternative to Cyberduck

Navigate (using cd in a terminal window) to the folder (on your local computer) containing the file you wish to transfer to your virtual machine. Once there, type

scp testfile moleuser@123.456.789.321:

This would transfer the local file named _testfile_to your home directory on your virtual machine (assuming that the IP address of your virtual machine is 123.456.789.321). You could also copy the file to a subdirectory:

scp testfile moleuser@123.456.789.321:myfolder

To get files from your virtual machine to your local computer, type (from your local computer):

scp moleuser@123.456.789.321:testfile .

The dot at the end is significant: it stands for your present working directory (on your local computer).

You may think that typing moleuser@123.456.789.321 each time is a bit tedious, and you’d be correct. To simplify things, you can create an entry in your local computer’s ssh configuration file that creates short alias for the whole moleuser@123.456.789.321 combination.

Create, if necessary, a text file on your local computer as follows:

nano ~/.ssh/config

Enter the following text in this file:

Host molevm
    HostName 123.456.789.321
    User moleuser

Use the tab character to indent the second and third lines. Now make sure that your config file has the proper (very limited) permissions:

chmod 600 ~/.ssh/config

You should now be able to do this:

scp testfile molevm:

Your scp command can look up molevm in your ~/.ssh/config file to find what username and ip address to use!

The following table contains a list of commands that will allow us to navigate through the directory structure. The entries are linked to their Wikipedia pages, which contain very useful examples.

Linux/Mac Description Syntax (Linux/Mac)
pwd print working directory pwd
ls list directory contents ls
history display command history history
cd change directory cd directory_name
mkdir create directory mkdir directory_name
cp copy file(s) cp original_filename copied_filename
mv rename file(s) mv original_filename new_filename
clear clear the screen clear
exit quit command line exit
Ctrl-E go to the end of line  
Ctrl-A go to the beginning of line