Cloud

Old documentation
Official Openstack documentation

Genostack is a cloud computing platform based on Openstack. It provides computing resources on-demand, allowing users to create and customize virtual machines of various sizes. Namely, Genostack provides the following functionalities:

Please keep in mind that Genostack is not fit for long-term deployments.
As computing resources are shared with the community, please delete your instances as soon as you are done.

Getting started

Genostack is a cloud computing platform based on OpenStack.
To gain access, you first need to activate your cloud account.
To do so, please go to my.genouest, and navigate to the "Details" -> "Genostack" tab. You can then click on "Activate cloud account".
The cloud web interface is accessible here with your Genouest credentials.

Configuring your SSH key

You need to add a public key in the OpenStack interface to access your instance with the SSH protocol.

To access your instance, you first need to connect to the Genouest frontend at openstack.genouest.org. From there, you can access your instance. Please visit this page to setup your connection to the frontend. (Access is the same between openstack.genouest.org and genossh.genouest.org)

Once you are connected to the openstack frontend, you will need to generate a new ssh keypair (for security reasons, you should avoid re-using your local keypair).

ssh-keygen -t rsa -b 4096

Then, copy your new public key :

cat ~/.ssh/id_rsa.pub

You can import this public key by going to the Genostack web interface (Using your Genouest credentials).
In the Project -> Compute -> Key Pairs tab, you will be able to select Import public key and paste it in the dialog box.
You can now launch a Virtual Machine and access it with ssh.

Launching a Virtual Machine

To launch a new instance (and monitor existing one), simply head to the “Instances” tab.
Then, click on “Launch instance”. The pop-up will then allow you to customize your instance.
Some parameters (marked by *) are required to launch a new instance.

Details

The “Details” tab will allow you to select the instance name, and the number of instances you want to launch. You can also see how many you can launch in regard to your quotas.

Source

In this tab, you can select the source of the instance you wish to use.

Flavors

Flavors are the “size” of the instance, in terms of memory, cores and disk space. (If you boot from a volume, the disk space will be the size of your volume, no matter the flavor you choose).

Security group

The “Default” security group will be selected. If you wish to use your own security group, please select it here.

Key pair

You can select the ssh key pair you wish to be injected in the instance here. Without it, you will not be able to access your instance.
If you only have one keypair, it will be selected by default.

Connecting to your Machine

Once connected to the openstack frontend, you can access your machine with this command : 

ssh root@your_ip

You can find your_ip in the Genostack dashboard, in the instance tab.
You can also copy your genocluster files on the VM using this command: 

scp myfile root@your_ip:

VM expiration

All instances have a default 2 months time limit. Afterwards, it will be deleted.
You can extend your VM duration through the "VM expiration" tab on the Genostack interface.

Boot source types

When booting a new instance in Genostack, you are given the choice between two sources: Images, or Volumes.
While the behaviour of the instance itself will not be affected by the type of source, there is a major difference in data persistence.

Using an image

When you boot from an image, you will get the same instance it was at the time of the snapshot. Booting 10 instances will produce the same result every time.
Booting from an image is better if you want to have a clean environnement for repeatability.
If you wish to save the state of your instance, you will need to create a snapshot of it, and use it as an image when you boot a new instance.

Using a volume

When you boot from a volume, the instance will be in the same state as you left it the last time you used the volume. Any modification you make on it will be saved in real-time.
Booting from a volume is better if you want to keep working on the same instance while saving your changes.
There is no need to save the state of your instance, you can boot from the same volume later.

Saving your own image

After creating and customizing your instance, you might want to save it as an image. You will then be able to use this image to re-create your instance with all your modifications saved.
You can also share this image with other users.

Cleaning up your instance

If the snapshot is for your private use, you can skip this step.
If you wish to share your image with other user, you should “clean it up”, meaning remove all private information on it. Private information includes data, but also your history and your ssh key.
You can remove your history using these commands: 

for logs in `find /var/log -type f`; do > $logs; done  
cat /dev/null > ~/.bash_history  
history -c

You can clear all ssh keys allowed in the instance using this command: 

cat /dev/null > ~/.ssh/authorized_keys

You will lose access to your instance the next time you disconnect, so make sure to only remove the ssh keys once you are ready to create the image.

Making a snapshot

To make a snapshot of one of your instances, simply head to the Instances and use the Create snapshot button. You will then be prompted for a name, and will be able to snapshot it.
The instance might stop during the process.
After the process is done, you will be able to use it as a boot source, by selecting Instance snapshot or Volume snapshot in the boot menu.

Converting your snapshot to an image

Snapshots and Images are virtually the same. The main difference is the way they are available from the interface. If you wish to convert your snapshot, simply head to the Image tab, and select the “Update Metadata” action. You then need to remove all metadata from your snapshot.

Making your image public

Please contact the support with the name of the image you wish to publicize.

Volumes

Volumes in Genostack serve three main purposes:

Even if the instance is deleted, the volumes will not, and can be re-used with another VM. Volumes are the best way to safely store any data generated on the cloud.
You are allocated 100 GB by default.

External data storage

If you need to increase the space available on one of your instances, one possibility is to attach a volume to it. Created volumes can be attached and detached from instances at will, allowing you to transfer data between several of your instances. You can also transfer one of your volumes to another user to share data with them.

Creation

To create a new volume, you need to go to the “Volume” tab. From there, click the “Create volume” button, fill out the name and description, select the size and click “Create volume”.
Your new volume will appears in the “Volumes” tab, ready to be used.

Attaching

First, you need to create an instance. When it is fully booted, you can attach your volume. To do so, go to the “Volumes” tab, and select “Manage Attachments” for the volume you wish to use.
From there, select your instance and attach the volume to it.

If this is the first time you are using this volume, you need to create a file system on it. To check if there is already one, use this command:

lsblk -f

The first line (with the mountpoint "/") should be your root filesystem. The next line should be your volume.
If there are no "FSTYPE" for your volume, you need to format it using this command :

mkfs.ext4 /dev/vdx

With vdx the location where your volume is mounted (see the lsblk command) Please use the value with a number at the end if available (Ex: vdb1).
You can then mount the volume anywhere you wish and start storing data.

mkdir /mnt/myfolder
mount /dev/vdx /mnt/myfolder

If an error occurs, please make sure the package nfs-common is installed.
For subsequent uses, you will just need to mount the volume where you want.
Do not use mkfs.ext4 again or you will lose everything on the volume.

Detaching

On your instance, make sure to unmount the volume

umount /dev/vdx

Then, in the Genostack interface, simply head back to the “Volumes” tab and the “Manage Attachments” button. You then need to click the “Detach volume” button. The volume will be unmounted if the instance is deleted

Boot partition extension

Openstack allows users to boot a new instance on a volume. Booting an instance on a volume allows you to keep any changes made on the instance, even if you delete and re-create it later on.
This allows you to keep working on a instance even after deletion, without any need to make a snapshot. Your instance will behave as normal, except that the root partition will be extended.

Booting from a volume

You should begin by creating a bootable volume. The easiest way would be to create an instance using an image, check the “Create New Volume” checkbox and select the volume size (it should be bigger than 20 Go).
The bootable volume will automatically be created at instance launch, and will remain after instance deletion. You can access it from the “Volumes” tab.
A bootable volume will behave as a normal volume, meaning you can attach it to an instance as a storage volume
Afterward, you will only need to select your volume in the “Source” tab at instance creation for subsequent uses.

Image to volume

If you created a snapshot from an instance without using a volume, you can convert it easily to a bootable volume by going to the “Images” tab, and using the “Create volume” action.

Volume to Image

If you wish to create a new image from one of your bootable volumes (for example because of quota issues), simply select the “Upload to image” action on your bootable volume.

Volume management

Copying Volumes

To copy a volume, you can either use the “Create snapshot” button in “Actions”, or create a new volume using the one you wish to copy as the source.

Transferring volume

You can transfer one of your volume (bootable or normal) to another user using the transfer function (You will loose the volume, so make sure to make a copy if you need to keep the data.).
To do so, use the “Create transfer” action on you volume. (Make sure it is not attached to an instance). You will need to fill out the transfer name.

Afterward, you will obtain the Transfer ID and the Authorization Key. Make sure to copy them, or you will have to start again from scratch. The user you wish to transfer to will need to use the “Accept transfer” in “Volumes”, and fill out the Transfer ID and Authorization Key. He will then be able to access the volume.

Sharing data between instances

The easiest way to share data, both between your own instances, and between users, is to use an NFS share. The Manila plugin allows you to easily create a secure NFS share, and control its access.

Creating a share

To create an NFS share, simply head to the “Share” tab in your project. From there, you can click the “Create share” button. You can then customize your share, including the name, description and share size. By default, you can create up to 50 shares with a total size of 100 Go.
You can also make your share visible to all users. If you use that option, make sure to secure your share access.

Managing access

By default, nobody can mount your share on their instance. You need to add security rules to decide how the share can be used. To customize the rules, please select the “Manage rules” option on your share. For now, you can only filter access by IP, meaning you need to choose which instances can mount your share, based on their IP.
You can also select whether the mount will be read/write or Read only, for each rule.

You will need to give the “Export locations” to any user you want to access your data.

You can select an IP range to be allowed. For instance, using “*” means all instances will be able to mount your share. If you made your share public at its creation, this could lead to a security risk. Please be careful when using IP ranges (you should at least restrict the access to read only).

Mounting your share

You can mount your share just as you would for any other volume. When in the “Shares” tab, click on your share's name. The “Export location” will be what you need to use in the mount command.

You can mount your share on your instance like this:

mount 192.168.100.2:/var/lib/manila/mnt/share-XX /mnt/my_share_folder

For a more permanent solution (persistent through reboot), you can fill out the /etc/fstab file. To mount the previous share in Read/write mode, add the following line to the file:

192.168.100.2:/var/lib/manila/mnt/share-XX /mnt/my_share_folder nfs rw 0 0

To mount the previous share in Read only mode, add the following line to the file:

192.168.100.2:/var/lib/manila/mnt/share-XX /mnt/my_share_folder nfs ro 0 0

and use

mount -a

to mount the share.

Firewall

Genostack uses an internal firewall limiting external access to the instances. Even if you open the ports on an instance, and redirect the http access, you will not be able to connect to it from the outside. You are, however, able to create or edit security groups and add your own security rules in there.

Default rules

By default, the ports 80, 443, and 22 are open to the outside. All ports are open between your instances using the default security group, meaning that all of your instances will be able to communicate with each other.

Custom rules

Rules are stored in Security groups. You can either add rules to an existing one, or create a new one. To do so, head to the "Project -> Network -> Security groups" tab. Once you have selected or created the security group you wish to use, use the Manage rules button. There, you can create or modify rules.

HTTP access

Default access

By default, you can access to your VM (port 80) from your web browser via this URL :
https://XXX-your-ip.vm.openstack.genouest.org
You will need to replace all “.” in the ip by “-“ (With the exception of the .vm.openstack.genouest.org):

XXX can be replaced by anything :

Both URLs will be forwarded to the same virtual machine, but the prefix (jupyter and myapp in the exampled, but any name can be used) can help you to do your own proxying in your VMs.
Using this URL scheme also automatically supports websockets.

Remote desktop

Using X2go, you can remotely access your instance desktop. Genostack provides several images “remote-desktop ready”. These images all have “desktop” in their name. You simply need to launch an instance using one of these images, setup x2go, and you are done.

Using a pre-configured image

Important: If you have the same set of SSH keys inside your local computer and on the frontend, you can launch X2go without a password, using an SSH connection (described below). Alternatively, you have to create your own root password or add a new special « virtual » user.

On your virtual machine

You can change the root password using the following command :

passwd root

You can create a new user using the following command :

adduser myuser

On your local machine

You can install X2go following this tutorial You should configure X2go as follows:

Host: openstack.genouest.org
Port: 22
Login: root/custom user

Here is the configuration for the proxy server (You will need to tick "Use proxy server for SSH connection")

Type: SSH
Host: openstack.genouest.org
Port: 22
Login: Your GenOuest login

The provided images use the Mate desktop

Creating your own image

You can modify one of your own instances to use a remote desktop. There are two main steps:

Installing x2go-server

Please refer to the official documentation here

Install a desktop

Please refer to the installation manual for the desktop you wish to use.

Murano deployment

Murano is a tool to simplify deployment and configuration of one or several instances.
For example, Murano can deploy a usage-ready Slurm cluster, or a Jupyter notebook, in only a few clics.

Murano uses “environnements”. Each environnement can contain several applications. Each environnement usually has his own security group.
An application is a set of one or more instances, aiming to be use-ready. For instance, an application can be a SQL database, or a whole cluster.

Selecting your package

To access the Murano interface, please select the Murano tab on the left sidebar.
From there, select the “Applications” tab. You then have access to a selection of ready to deploy applications.

Application configuration

To quickly launch your application, please use the “Quick deploy” button. A new environnement will be created, and you will then have to customize your deployment. The amount of customization will vary depending on the package. Parameters can includes:

Launching your application

After you are done configuring your application(s), you can launch the instances by pushing the “Deploy environnement” button. You will be able to follow in real-time the deployment of the application(s). Murano instances behave as normal instances. You can see all your instances in the “Project” tab, in the “Instances” category. You can connect to them with ssh as you would for normal instances.

Object storage

Please refer to this documentation for information on our object storage.