Do more with Cloud Computing

One of the advantages of computing within CyVerse is shared credentials and user profiles on common computing resources. This makes it easy to transfer data, and to enable collaborator access securely. In this exercise, you will share your Atmosphere desktop with another CyVerse user.

Get the username of another CyVerse user and give them your IP address (from the Atmosphere website).
Connect to your Atmosphere instance via VNC.
In your Atmosphere instance, click the VNC icon in the upper right-hand corner of the Desktop; this will open a VNC Server window.
Click the More button on the VNC Server window and select Options...; A VNC Server Options window will open.
Select Users & Permissions from the VNC Server Options window, then click on Add.; Enter the CyVerse username of the collaborator you wish to share with; the click OK
Make any adjustments to the user permissions, click Apply, the click OK. Close the VNC Server window.
Your collaborator can now enter the IP address (+“:1”) to their VNC viewer as the server to connect to. They can connect using their CyVerse username and their user credentials.

Imaging an Atmosphere instance

Imaging allows you to save a copy of your Atmosphere instance (including data, installed software, etc.). This means that even after your terminate. These instructions give you an overview of the process but you will need to Read the Full Documentation, including which directories will be deleted or overwritten during the process.

Working on other Clouds

Choosing a cloud platform

The most important thing about The Cloud is choice - instead of purchasing a physical computer, you can obtain on-demand computing at almost any scale. This power comes with advantages and disadvantages:

Advantages of Cloud Computing Access large amounts of computing power on demand Full administrative rights - install anything * Use pre-configured images (software already installed)

Disadvantages of Cloud Computing Cloud computing costs money (you must keep track of your costs) If you need help, you may not have a local system administrator * Images may be poorly documented (you may not be clear on what is installed, or how to use it)

Cloud platform choices

There are several cloud providers to choose from. Some scientific clouds may either be free or allocate resources competitively. Commercial clouds are can be very powerful, but choice can be overwhelming. We will cover as much as we you need to get through the Data Carpentry lessons, but you will ultimately need to learn things not covered here so see the documentation below:

Commercial Clouds

Open Science Clouds

Launching an Cloud Instance (Virtual Machine)

Tip: Keep in mind, if you are attending a workshop this will have already been done for you!

We will provide instructions for working on the Amazon and the Atmosphere cloud. Follow the instructions for your platform of choice.

Launching an instance on Amazon Web Services

Prerequisites

Form of payment (credit card)*
Understanding of Amazon's billing and payment (See: Getting started with AWS Billing and Cost Management)
An image to launch: Amazon maintains a Catalogue of images ('amis').

In this tutorial: We will be referring to the Amazon instance used in Data Carpentry's Genomics Lessons: You can find this on Amazon by searching for: ami-5bc63136

Tip: You can use some of Amazon Web Services for free, or see if you qualify for an AWS Grant (See: https://aws.amazon.com/grants/ ) if you are using AWS for education. The free level of service will not be sufficient for working with the amount of data we are using for our lessons.

Create an AWS account

Go to Amazon Web Services https://aws.amazon.com/
Follow the button to sign up for an account - you will need to agree to Amazon's terms and conditions and provide credit card information.

Sign into AWS and Launch an Instance

Sign into AWS EC2 Dashboard: https://console.aws.amazon.com/ec2/
Click the 'Launch Instance' button
In 'Step 1' you will be asked to choose an Amazon Machine Image (AMI), on the lefthand side, look for 'Community AMIs' and then search for ami-5bc63136; select this image
For 'Choose and Instance Type' select t2.medium; then click 'Review and Launch'

Tip: You can select larger instances, but these will cost more to run. Also for the 'Review Instance Launch' you will get some warning about instance security. We will not go into detail here about security groups. However, you should know that when you launch an instance, you will want to take steps to make sure no one maliciously accesses your instance. For example, never give your security credentials (next step) to another user. 5. For the 'Review Instance Launch' step, click 'Launch' 6. You will be asked to Select an existing key pair or create a new key pair. Select 'Proceed without a key pair' and select the acknowledgement you are given. Then click 'Launch Instances'

You instance will now be launched. You should follow the links to 'Create billing alerts'.

Connect to Amazon Instance

Instructions for MAC

Log into your AWC EC2 Dashboard https://console.aws.amazon.com/ec2/
You should see that you have one instance, to proceed the instance state must be 'running' (if you just launched the instance it will take <5 min for the instance to start running)
At the bottom of the dashboard, you should see a Public DNS which will look something like ec2.12.2.45.678.compute-1.amazonaws.com. Copy that address (you may wish make a note of it as you will need this each time you connect.
Open the terminal application on your Mac and use 'ssh' to connect. Your command will be:

    $ ssh dcuser@your.amazon.dns

Your computer will be unable to verify the authenticity of the host... type yes to continue connecting
Then enter the password for this computer: 'data4Carp'

You should now be connected to your personal instance. You can confirm this with the following commands; whoami,pwd,ls, which should yield the following results:

Last login: Thu Jul 30 13:21:08 2015 from 8.sub-70-197-200.myvzw.com
$ whoami
dcuser
$ pwd
/home/dcuser
$ ls
dc_sample_data  FastQC  Trimmomatic-0.32
$

Instructions for PC

Download the PuTTY application at: http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe
Log into your AWC EC2 Dashboard https://console.aws.amazon.com/ec2/
You should see that you have one instance, make sure instance state is 'running' (if you just launched the instance it will take <5 min for the instance to start running)
At the bottom of the dashboard, you should see a Public DNS which will look something like ec2.12.2.45.678.compute-1.amazonaws.com. Copy that address (you may wish make a note of it as you will need this each time you connect.
Start PuTTY. In the section 'Specify the destination you want to connect to' for 'Host Name (or IP address)' paste in the DNS address and click 'Open'
When prompted to login as, enter 'dcuser'; you may be notified that the authenticity of the host cannot be verified - if so, ignore the warning an continue connecting
When prompted for a password enter 'data4Carp'