IDLab IT - speech

Getting started
Data organization
Running jobs

Accounts & passwords

You have several accounts. To avoid problems, keep the same user name for each of these accounts.

account	(re)set	used for
UGent	https://password.ugent.be
IDLab	https://account.idlab.technology
IDLab VPN	https://amos.intec.ugent.be/passwd/
ELIS	https://password.elis.ugent.be
VirtWall
imec

Usefull links

IDLab intranet: http://idlabintra.intec.ugent.be/index.php/IDLABintra
IDLab helpdesk: https://helpdesk.atlantis.ugent.be/
IDLab Mattermost: https://mattermost.ilabt.imec.be/
Reserving an IDLab meeting room: https://tw06v015.ugent.be/e/res/vergaderzalen/day.php?area=8
Reserving a UGent room: https://centauro.ugent.be/home
Updating your UGent publications: https://biblio.ugent.be/
The UGent Windows applications server: https://athena.ugent.be/Citrix/StorefrontWeb/
Tracking phd and master theses: https://plato.ea.ugent.be
imec expenses (Rydoo): https://accounts.rydoo.com
Minerva (website for the UGent courses): https://minerva.ugent.be/
Oasis (website for the UGent administration): https://oasis.ugent.be
UGent webmail: https://owa.ugent.be/
UGent SharePoint (OneDrive): https://ugentbe-my.sharepoint.com/
UGent telephone directory: https://telefoonboek.ugent.be/nl/simple
Belnet filesender: https://filesender.belnet.be/
Official UGent Holidays: https://www.ugent.be/intranet/nl/loopbaan/verlof/vakantie/

Getting started (login scripts and other settings)

Login script:
On the IDLab system, you may want to copy the following reference login scripts:
```
  cp ~spch/.profile ~
  cp ~spch/.bashrc ~
  cp ~spch/.inputrc ~
```
This way, you can access the common databases, software, etc.
Ssh credentials
Using the same credentials on your laptop as on the IDLab systems (IDLab homedir) makes things easier. Hence, copy your existing ssh-credentials if you already have ssh crendentials (e.g. you just made them on your laptop, then copy them your IDLab homedir). Herunder we give instructions on how to make new ssh-credentials:
```
  mkdir ~/.ssh
  cd ~/.ssh
  ssh-keygen -t rsa -f id_rsa -P ''
  ssh-keygen -t dsa -f id_dsa -P ''
  cat id_dsa.pub id_rsa.pub >> authorized_keys
  touch known_hosts
  chmod 600 known_hosts authorized_keys
  cp ~spch/.ssh/config .
  chmod 600 config
```
Seting up the laptop (Linux)
- If needed, create a user with the exact same user name as your UGent and IDLab user name.
  rationale: this is needed to use the UGent printers, and facilitates password-less log-through
```
old_usr="`whoami`"
echo -n "enter UGent user name : ";read new_usr
sudo adduser $new_usr
for group in `id -n -G` ; do [[ "$group" == "$old_usr`" ]] || sudo addgroup $new_usr $group ; done
```
- Setting up the VPN — just follow the IDLab-instructions
- Copy the IDLab ssh credentials and other handy config-files to your laptop:
```
ssh 10.10.133.31 'tar -c .ssh .inputrc' | tar -kxvf - -C ~/
```
- Setting up the email (Thunderbird)
  Follow the instructions on the UGent DICT help page and, send a e-mail to allow using an encrypted connection (Secure SMTP) to helpdesk@UGent.be
- Setting up the printer (Ricoh) — just follow the UGent-instructions

Useful commands

TODO

List item one
List item two
List item three

Etcetera

Principles

We have 3 separate tiers where speech related information is stored:

Every user has his/her own home directory. You can access someones homedirectory using either ∼<username> or using /home/speech/<username>. To allow easy collaboration, the default is that data in your homedirectory can be accessed by the other group members. Use the command chmod (see useful commands) to control who can access what.
Shared data is managed by a set of 3 meta-users:

spchdata:
Here you find all databases.

spchprog:
Here you find all our software packages.

spch:
Here you can find other shared resources such as presentations, published papers, conference processings, ...

You can access this shared data using either ∼<spchdata,spchprog,spch> or using /speech/data/, /speech/prog/, or /speech/spch.
A large storage pool that is not backed-up. Most experiments involve converting a database in yet antoher format. While it is relevant to backup all scripts and results of an experiments, backing-up yet another copy of a databases is just a waste of resources. Hence, you are encouraged to store intermediate results in a subdirectory of this disk system. By convention, this would be in /speechdump/<username>/<whatever_subdir_you_want>.

Databases

All databases are managed by the meta-user spchdata. The databases can be found using either

ls -l ∼spchdata/
ls -l /speech/data/

Software

Speech related tools are managed by the meta-user spchprog. The installed software packages can be found using either

ls -l ~spchprog/
ls -l /speech/prog

In case you use the advised login scripts, you should be able to access all software directly.

Other common resources

Other shared resources are managed by the meta-user spchg. You can find the common resources using either

ls -l ~spch/
ls -l /speech/spch

Work at home

Option1

IDLab has a VPN service that allows you to connect with the IDLab network from anywhere. The VPN service is normally enabled for every IDLab-speech account (but in case you cannot open the VPN, you may want to double check this). You can find a copy of the instructions to install the VPN on your computer here.

Option2

The speech group has an ssh-server that allows you to connect with the IDLab speech group computers from anywhere.

Login into the ssh-server; from thee you can access your data and log-trough to the other machines:
```
ssh -Y -p 4444 -l <username> lafite.ilabt.imec.be
```
Directly log into one of the compute servers:
```
ssh -Y -l <username> -J lafite.ilabt.imec.be:4444 <computer>
```
with <computer> one of the computers listed below.

Computers

GPU compute servers

poseidon30 (10.10.133.30)
poseidon31 (10.10.133.31)

Linux desktops

riesling (10.10.133.15)
merlot (10.10.133.16)
sahba (10.10.133.17)
lafite (10.10.133.21) — is our {web,ssh,file}-server; do not use for compute jobs

Accessing the data remotely (Linux)

You can usef sshfs to access all data on the IDLab speech servers remotely:

make a mount point on your laptop + copy user/group dbase (needs to be done only once):

sudo mkdir /mnt/idlab
sudo chown `id -un`:`id -gn` /mnt/idlab
sudo chmod 755 /mnt/idlab
ssh -p 4444 -l <username> lafite.ilabt.imec.be 'cat ~spch/.ssh/sshfs_uid_gid.tgz' | tar -kxvzf - -C ~/.ssh/

mount the IDLab disks on your laptop (this example starts from the root directory on the IDLab file-server, hence all volumes can be accessed, but with somewhat longer path names and broken absolute symbolic links):
```
sshfs -p 4444 -o IdentityFile=$HOME/.ssh/id_rsa <username>@lafite.ilabt.imec.be:/ /mnt/idlab
```
unmount the data (do not forget this, otherwise you risk stale files)!
```
fusermount -u /mnt/idlab
```
finding your homedirectory
```
ls -l /mnt/idlab/home/speech/<username>
```
options and remarks:
- you can mount the idlab disks somewhere in your home directory of your laptop as well by e.g. replacing "/mnt/idlab" by "~/idlab_disks"
- you can mount any subdirectory of the idlab disks as well:
```
sshfs -p 4444 -o idmap=file -o nomap=ignore -o uidfile=$HOME/.ssh/sshfs/idlab_uid.txt -o gidfile=$HOME/.ssh/sshfs/idlab_gid.txt -o IdentityFile=$HOME/.ssh/id_rsa <username>@lafite.ilabt.imec.be:<idlab_subdir> /mnt/idlab
```
- symbolic links on the idlab disks will only work when using relative references (not starting from the root directory), or if you mount all relevant subdirectories on the idlab disks in the exact same location on your laptop, or add the option "-o transform_symlinks" to sshfs, or if set symbolic links on your laptop to get the same directory structure as in IDLab (preferred solution), e.g.
```
sudo ln -s /mnt/idlab/speech /speech
sudo ln -s /mnt/idlab/speechdump /speechdump
sudo ln -s /mnt/idlab/home/speech /home/speech
sudo ln -s /mnt/idlab/home/students /home/students
```
- since your laptop and IDLab use different user and group DBases, the owner of a file (username and group) may be numerical if your laptop does not know that user or group. This has no ill effect on the operation of the mounted disks.
- all data is tranferred over the internet (with limited and short-time chaching only), so don't expect blazing fast performance.