Getting started

Quick installation

The quickest and safest way to start using our toolkit is to install its latest stable version locally in a virtual environment. To do so follow these steps:

1. Download the wheel file (.whl extension) of the latest version available in your local working folder

2. Create and activate a Conda virtual environment in python 3.10, named for example osmose_stable

conda create --name osmose python=3.10 -y
conda activate osmose

Note

We have made a tutorial on Conda if you are starting with it

3. Install the pip tool in your conda environment, go to the folder folder_wheel where the wheel file is located and use it to install the toolkit from the wheel file

conda install pip
cd <folder_wheel>
pip install <toolkit_version.whl>

Note

If it happens that the toolkit version has not changed, use instead

pip install --force-reinstall <toolkit_version.whl>

Warning

We discourage users to install the toolkit from our git repository, as it is under permanent development and thus potentially unstable and unfinished.

Note

If for any reason you get an unexpected error message or an incorrect result, please open a new issue in the issue tracker

Sample datasets

Our toolkit comes with a diversified set of sample datasets containing both audio recordings and auxiliary data, described in the table below.

Code name	Date	Time	Location	Sample frequency (Hz)
SPM2011	2011-09-19	11:00	Saint Pierre et Miquelon	48000
BallenyIslands2015	2011-09-19	11:00	Balleny Islands 2015	1000

Most of the examples found in modules and in the use case gallery use these audio recordings. To test and run the scripts, we recommend downloading these files to your local hard drive. The files can be downloaded from the GitHub repository

Tutorials

Github

If you happen to encounter this error “fatal: unable to access ‘https://github.com/Project-OSmOSE/osmose-toolkit.git/’: The requested URL returned error: 403” after running git push --force origin bugfix, here is how you can deal with it

git remote remove origin
git remote add origin git@github.com:Project-OSmOSE/osmose-toolkit.git

Conda

To activate a conda environments after its creation

conda activate ENV_NAME

To install a package in your activated environmnent

conda install PACKAGE_NAME

As an alternative way of installing python packages in your conda environment, you can also run conda install pip and then run pip install PACKAGE_NAME. Note that some packages (eg pysoundfile) are not available from conda install but only pip install (problem unsolved of Solving environment: failed with initial frozen solve. Retrying with flexible solve.)

To delete a conda environment, run

conda remove --name ENV_NAME

To get a list of all existing conda environments on your machine

conda info --envs

Bash and qsub

• ls : list all files and folders ;

• qstat -u myusername : this will list all your launched qsub jobs. This useful to check that your processing is running as expected, for example there is the right number of running jobs. The letter S stands for job status, which can be R = running, Q = en attente (dans la queue), H = en attente (on hold) ;

• cat some_file.txt : this will display the content of a text file (.txt, .csv) in your terminal.

• qdel job_id : this is to kill a launched job appearing in the qstat list. For example to kill the second job from the list below run qdel 1767379

  r2i1n25 datahome/dcazau% qstat -u dcazau datarmor0:

  Req’d Req’d Elap

  Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time ————— ——– ——– ———- —— — — —— —– - —– 1767039.datarmo dcazau ice_mt jupyterhub 42970 1 4 8192m 02:00 R 00:12 1767379.datarmo dcazau ice_mt jupyterhub 34170 1 8 16gb 02:00 R 01:03

• qstat -fx id_job : this will show you all information on your job, especially the address of servers on which it is or will run. Example : exec_vnode = (r2i1n25:ncpus=8:mem=16777216kb) informs you that your code is running on single machine located at r2i1n25 over 8 CPUS with 16 GB RAM

Joining datasets

The Auxiliary class automatically joins environmental and instrument data to acoustic data. By default, the processed acoustic dataset serves as the reference dataset and other data will be joined to corresponding time and localization. So far, only csv files and ERA5 re-analysis netCDF files can be joined to your acoustic data. The API functions to download ERA data can be found in the module but an internet access is necessary.