Installation#
How to install#
The Autosubmit code is hosted in Git, at the BSC GitLab public repository. The Autosubmit Python package is available through PyPI, the primary source for Python packages.
Pre-requisites: bash, python3, sqlite3, git-scm > 1.8.2, subversion, dialog, curl, python-tk(tkinter in centOS), graphviz >= 2.41, pip3
Important
(SYSTEM) Graphviz version must be >= 2.38 except 2.40(not working). You can check the version using dot -v.
Python dependencies: configobj>=5.0.6, argparse>=1.4.0 , python-dateutil>=2.8.2, matplotlib==3.4.3, numpy==1.21.6, py3dotplus>=1.1.0, pyparsing>=3.0.7, paramiko>=2.9.2, mock>=4.0.3, six>=1.10, portalocker>=2.3.2, networkx==2.6.3, requests>=2.27.1, bscearth.utils>=0.5.2, cryptography>=36.0.1, setuptools>=60.8.2, xlib>=0.21, pip>=22.0.3, ruamel.yaml, pythondialog, pytest, nose, coverage, PyNaCl==1.4.0, six>=1.10.0, requests, xlib, Pygments, packaging==19, typing>=3.7, autosubmitconfigparser
Important
dot -v command should contain “dot”, pdf, png, SVG, Xlib in the device section.
Important
The host machine has to be able to access HPCs/Clusters via password-less ssh. Ensure that the ssh key is in PEM format ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM.
To install autosubmit, execute the following:
pip install autosubmit
Or download, unpack and:
python3 setup.py install
Hint
To check if Autosubmit is installed, run autosubmit -v. This command will print Autosubmit’s current version
Hint
To read Autosubmit’s readme file, run autosubmit readme
Hint
To see the changelog, use autosubmit changelog
The sequence of instructions to install Autosubmit and its dependencies with pip.#
Warning
The following instructions are for Ubuntu 20.04 LTS. The instructions may vary for other UNIX distributions.
# Update repositories
apt update
# Avoid interactive stuff
export DEBIAN_FRONTEND=noninteractive
# Dependencies
apt install wget curl python3 python3-tk python3-dev graphviz -y -q
# Additional dependencies related with pycrypto
apt install build-essential libssl-dev libffi-dev -y -q
# Install Autosubmit using pip
pip3 install autosubmit
# Check that we can execute autosubmit commands
autosubmit -h
For a very quick test, you can follow the next instructions to configure and run Autosubmit at the user level. Otherwise, please go directly to How to configure Autosubmit .
# Quick-configure ( user-level database)
autosubmit configure
# Install
autosubmit install
# Quick-start
# Get expid
autosubmit expid -H "local" -d "Test exp in local."
# Create with
# Since it was a new install, the expid will be a000
autosubmit create a000
# In case you want to use a remote platform
# Generate a key pair for password-less ssh. PEM format is recommended as others can cause problems
ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM
# Copy the public key to the remote machine
ssh-copy-id -i ~/.ssh/id_rsa.pub user@remotehost
# Add your key to the ssh-agent ( if encrypted )
# If not initialized, initialize it
eval `ssh-agent -s`
# Add the key
ssh-add ~/.ssh/id_rsa
# Where ~/.ssh/id_rsa is the path to your private key
# run
autosubmit run a000
The sequence of instructions to install Autosubmit and its dependencies with conda.#
Warning
The following instructions are for Ubuntu 20.04 LTS. The instructions may vary for other UNIX distributions.
Warning
This procedure is still WIP. You can follow the process at issue #864. We strongly recommend using the pip procedure.
If you don’t have conda installed yet, we recommend following Installing Miniconda.
# Download git
apt install git -y -q
# Download autosubmit
git clone https://earth.bsc.es/gitlab/es/autosubmit.git -b v4.0.0b
cd autosubmit
# Create a Conda environment from YAML with autosubmit dependencies
conda env create -f environment.yml -n autosubmitenv
# Activate env
conda activate autosubmitenv
# Install autosubmit
pip install autosubmit
# Test autosubmit
autosubmit -v
For a very quick test, you can follow the next instructions to configure and run Autosubmit at the user level. Otherwise, please go directly to How to configure Autosubmit
# Quick-configure ( user-level database)
autosubmit configure
# Install
autosubmit install
# Quick-start
# Get expid
autosubmit expid -H "local" -d "Test exp in local."
# Create with
# Since it was a new install, the expid will be a000
autosubmit create a000
# In case you want to use a remote platform
# Generate a key pair for password-less ssh. PEM format is recommended as others can cause problems
ssh-keygen -t rsa -b 4096 -C "email@email.com" -m PEM
# Copy the public key to the remote machine
ssh-copy-id -i ~/.ssh/id_rsa.pub user@remotehost
# Add your key to ssh agent ( if encrypted )
# If not initialized, initialize it
eval `ssh-agent -s`
# Add the key
ssh-add ~/.ssh/id_rsa
# Where ~/.ssh/id_rsa is the path to your private key
# run
autosubmit run a000
Hint
After installing the Conda, you may need to close the terminal and re-open it so the installation takes effect.
How to configure Autosubmit#
There are two methods of configuring the Autosubmit main paths.
autosubmit configureis suited for a personal/single user who wants to test Autosubmit in the scope of$HOME. It will generate an$HOME/.autosubmitrcfile that overrides the machine configuration.
Manually generate an autosubmitrc file in one of these locations, which is the recommended method for a production environment with a shared database in a manner that multiple users can share and view others’ experiments.
/etc/autosubmitrc, System level configuration.Set the environment variable
AUTOSUBMIT_CONFIGURATIONto the path of theautosubmitrcfile. This will override all other configuration files.
Important
.autosubmitrc user level precedes system configuration unless the environment variable is set. AUTOSUBMIT_CONFIGURATION > $HOME/.autosubmitrc > /etc/autosubmitrc
Experiments database installation#
As the last step, ensure to install the Autosubmit database. To do so, execute autosubmit install.
autosubmit install
This command will generate a blank database in the specified configuration path.