Roboy Audio

This project aims to implement human-like hearing skills. This with the use of the eight Microphones from the Matrix Creator board.

Status

Stable functionality:

  • Audio stream from the 8 Mics through ROS
  • Recieving Audio stream in python
  • Azimutal angle localization of one Audio source, with low resolution
  • Publishing the angles onto a rostopic

In development:

  • Recognition of several xylophone key tones
  • Localization with higher resolution

Contents:

Installation

We use catkin to build our tools. So make sure that you have a working catkin workspace.

Prerequisites & Dependencies

The roboy_communication_cognition package. It can be found in the roboy_communication repository (https://github.com/Roboy/roboy_communication). Best practice is to store the repository in the source folder of your catkin workspace.

libgstreamer1.0-0, libgstreamer1.0-dev, libgstreamer-plugins-base1.0-0, libgstreamer-plugins-base1.0-dev, gstreamer1.0-plugins-ugly & gstreamer1.0-plugins-good, fftw3, libgflags-dev To have them installed, you need to simply do sudo apt install <package_name>

In addition doa_estimation is just build and executable on a Raspberry Pi, since it uses it’s Libraries and the current version just works with a Matrix Creator and a Pi.

In the end it is mandatory for you to be the superuser when you’re using the audio programs.

Using command line

Clone the audio repository into the source folder of your catkin workspace. git clone https://github.com/Roboy/roboy_audio /path/to/catkin_workspace/src/roboy_audio

Navigate to your catkin workspace. cd /path/to/catkin_workspace

Make the packages. The available packages are: doa_estimation, roboy_audio_stream catkin_make --pkg <package_name>

Have fun with roslaunch. Start the direction of arrival estimation roslaunch doa_estimation matrix_doa.launch

Capture and stream audio from the default audio source roslaunch roboy_audio_stream capture.launch

Capture and stream audio from the 8 Matrix Creator microphones roslaunch roboy_audio_stream capture_matrix_mics.launch

Start the stream receiver template node roslaunch roboy_audio_stream receive_py.launch

Using IDE (Clion)

Clone the audio repository. git clone https://github.com/Roboy/roboy_audio

Now open the CMakeLists.txt from the package that you want to edit.

Getting started

How does it work?

You can roslaunch or rosrun every node that is currently released. But you will need a sound source from which the algorithms can get needed data. For that you can either use a Matrix Creator and run the rosnodes on it or try to change the rostopic names inside of the code and use it for your own sources.

How to extend it?

You can create new rosnodes, by adding new catkin packages to the root folder (roboy_audio). You can find a Tutorial for creating catkin packages on the ROS Tutorial side (http://wiki.ros.org/ROS/Tutorials/CreatingPackage).

API

About arc42

arc42, the Template for documentation of software and system architecture.

By Dr. Gernot Starke, Dr. Peter Hruschka and contributors.

Template Revision: 6.5 EN (based on asciidoc), Juni 2014

© We acknowledge that this document uses material from the arc 42 architecture template, http://www.arc42.de. Created by Dr. Peter Hruschka & Dr. Gernot Starke. For additional contributors see http://arc42.de/sonstiges/contributors.html

Note

This version of the template contains some help and explanations. It is used for familiarization with arc42 and the understanding of the concepts. For documentation of your own system you use better the plain version.

Literature and references

Starke-2014
Gernot Starke: Effektive Softwarearchitekturen - Ein praktischer Leitfaden. Carl Hanser Verlag, 6, Auflage 2014.
Starke-Hruschka-2011
Gernot Starke und Peter Hruschka: Softwarearchitektur kompakt. Springer Akademischer Verlag, 2. Auflage 2011.
Zörner-2013
Softwarearchitekturen dokumentieren und kommunizieren, Carl Hanser Verlag, 2012

Examples

Acknowledgements and collaborations

arc42 originally envisioned by Dr. Peter Hruschka and Dr. Gernot Starke.

Sources
We maintain arc42 in asciidoc format at the moment, hosted in GitHub under the aim42-Organisation.
Issues
We maintain a list of open topics and bugs.

We are looking forward to your corrections and clarifications! Please fork the repository mentioned over this lines and send us a pull request!

Collaborators

We are very thankful and acknowledge the support and help provided by all active and former collaborators, uncountable (anonymous) advisors, bug finders and users of this method.

Currently active
  • Gernot Starke
  • Stefan Zörner
  • Markus Schärtel
  • Ralf D. Müller
  • Peter Hruschka
  • Jürgen Krey
Former collaborators

(in alphabetical order)

  • Anne Aloysius
  • Matthias Bohlen
  • Karl Eilebrecht
  • Manfred Ferken
  • Phillip Ghadir
  • Carsten Klein
  • Prof. Arne Koschel
  • Axel Scheithauer