Sample Codes

Python Virtual Environment Example

Let us create a virtual environment called mlproject and test the Machine Learning Iris example using python's scikit-learn library.

To create the virtual environment:

abtakid91@raad2-gfx:~$ virtualenv mlproject

Start your interactive job. Why I need an interactive job? Read this

abtakid91@raad2-gfx:~$ sinteractive

To activate the virtual environment:

abtakid91@gfx1:~$ source mlproject/bin/activate

In the terminal, you will notice that (mlproject) is added, this means that you are working inside the python virtual environment.

The sample code can be found here: (DIRECTORY) (ml.py)

To run the code, we need to install the packages. For this specific example, we need: sklearn and pandas. To install them:

(mlproject) abtakid91@gfx1:~$ pip install pandas
(mlproject) abtakid91@gfx1:~$ pip install sklearn

Since everything is installed, we are ready to run the code:

(mlproject) abtakid91@gfx1:~$ python ml.py

The output of the code will look like this:

LR: 0.966667 (0.040825)
LDA: 0.975000 (0.038188)
KNN: 0.983333 (0.033333)

To deactivate the virtual environment:

(mlproject) abtakid91@gfx1:~$ deactivate

To exit to login node:

abtakid91@gfx1:~$ exit

Conda Python Virtual Environment Example

First, you need to create a Python Virtual environment using Conda, which will require submitting an interactive job.

Before creating the virtual environment, it is convenient to add the source line to the .bashrc by: (you have to do this only once)

echo "source /cm/shared/apps/anaconda3/etc/profile.d/conda.sh" >> .bashrc

Let us create a virtual environment called dlproject and test the Deep Learning Iris example using python's TensorFlow library.

abtakid91@raad2-gfx:~$ conda create -n dlproject python=3.6

Start your interactive job:

abtakid91@raad2-gfx:~$ sinteractive

To activate the created envirnment:

abtakid91@gfx1:~$ conda activate dlproject

In the terminal, you will notice that (dlproject) is added, this means that you are working inside the Conda python virtual environment. Here you can install any python packages without admin privileges.

The sample code can be found here: (DIRECTORY) (dl.py)

To verify the python version of your environment:

(dlproject) abtakid91@gfx1:~$ which python

The output should be something like this:

~/.conda/envs/dlproject/bin/python

To run the code, we need to install the packages. For this specific example, we need tensorflow, keras and cv2. To install it:

(dlproject) abtakid91@gfx1:~$ conda install tensorflow-gpu
(dlproject) abtakid91@gfx1:~$ conda install keras cv2

Since everything is installed, we are ready to run the code:

(dlproject) abtakid91@gfx1:~$ python dl.py

The output of the code will look like this:

Epoch 1/5
60000/60000 [==============================] - 17s 285us/sample - loss: 0.2200 - acc: 0.9356
Epoch 2/5
60000/60000 [==============================] - 17s 278us/sample - loss: 0.0981 - acc: 0.9700

To deactivate the current envirnemnt:

(dlproject) abtakid91@gfx1:~$ conda deactivate

To exit to login node:

abtakid91@gfx1:~$ exit

To see the list of conda environments you have:

abtakid91@raad2-gfx:~$ conda env list