Joseph Wilk

Joseph Wilk

Things with code, creativity and computation.

Creating Sampled Instruments With Overtone

Overtone is an open source audio environment which uses Clojure and SuperCollider (an environment and programming language for real time audio synthesis and algorithmic composition).

Overtone makes it very easy to define instruments based on sound samples from freesound. We will walk through getting samples for a flute and then using them to define the flute instrument in Overtone.

The Flute

Getting the samples

Firstly we need to manually grab all the IDs from freesound. Sounds are often grouped into packs making this a little less painful.

We will be using the “Transverse Flute: Tenuto Vibrato C4-C7” pack.

A little Ruby script I use to produce a map of freesound ids to notes (which are usually in the filename of the sample):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

#!/usr/bin/env ruby -w
require 'rubygems'
require 'json'

raise Exception.new("You must pass in the pack id!") unless ARGV[0]

pack_id = ARGV[0]
note_extractor = λ{|filename| filename.gsub(/Transverse-Flute |\.wav|Tenuto|NonVibrato|-/, "").gsub(/ NonVibrato/, "")}

data = JSON.parse(`curl --silent "http://www.freesound.org/api/packs/#{pack_id}/sounds/?api_key=47efd585321048819a2328721507ee23"`)
number_of_pages = data["num_pages"] || 1

number_of_pages.times do |page|
  data = JSON.parse(`curl --silent "http://www.freesound.org/api/packs/#{pack_id}/sounds/?api_key=47efd585321048819a2328721507ee23&p=#{page + 1}"`)
  sound_ids = data["sounds"].map{|sound| sound["id"]}
  note_names = data["sounds"].map{|sound| note_extractor.call(sound["original_filename"])}

  sound_ids.each_with_index do |sound_id, index|
    puts "#{sound_id} :#{note_names[index]}"
  end
end

Adding these to our Clojure code we have a Freesound ID to note mapping:

1
2
3
4
5
6
7
8

(def FREESOUND-VIBRATO-FLUTE-SAMPLES
  "Freesound ids for all samples in the Vibrato Transverse Flute pack"
  {154274 :C7  154273 :B6 154272 :A#6 154271 :A6 154270 :G#6 154269 :G6  154268 :F#6
   154267 :F6  154266 :E6 154265 :D#6 154264 :D6 154263 :C#6 154262 :C6  154261 :B5
   154260 :A#5 154259 :A5 154258 :G#5 154257 :G5 154256 :F#5 154255 :F5  154254 :E5
   154253 :D#5 154252 :D5 154251 :C#5 154250 :C5 154249 :B4  154248 :A#4 154247 :A4
   154246 :G#4 154245 :G4 154244 :F#4 154243 :E4 154242 :F4  154241 :D#4 154240 :D4
   154239 :C#4 154238 :C4})

These samples are cached in the asset store (usually ~/.overtone/assets) so we don’t have to keep downloading them. We define a cache key for our downloaded samples.

1
2
3
4

(defn- registered-samples
  "Fetch flute samples from the asset store if they have been manually registered"
  []
  (registered-assets ::TransverseFluteTenutoVibrato))

Load all the samples into Overtone. This will automatically download the samples if they are not already present.

1
2

(def flute-samples
  (doall (map freesound-sample (keys FREESOUND-VIBRATO-FLUTE-SAMPLES))))

In order to play the samples we need to produce a mapping between the buffer that has the sound loaded and the midi pitch that buffer relates to.

First we convert a freesound-id into a midi note (pitch)

1
2

(defn- buffer->midi-note [buf]
  (-> buf :freesound-id FREESOUND-VIBRATO-FLUTE-SAMPLES name note))

Now we can generate the buffer id to midi note map.

1
2
3
4
5
6
7
8
9

(defn- note-index
  "Returns a map of midi-note values [0-127] to buffer ids."
  [buffers]
  (reduce (fn [index buf]
            (let [id (-> buf :id)
                  note (buffer->midi-note buf)]
              (assoc index note id)))
          {}
          buffers))

And we use this to set the buffers:

1
2
3
4
5
6
7
8
9
10

;Silent buffer used to ensure all 127 buffer spaces are filled 
(defonce ^:private silent-buffer (buffer 0))

(defonce index-buffer
  (let [tab (note-index flute-samples)
        buf (buffer 128)]
    (buffer-fill! buf (:id silent-buffer))
    (doseq [[idx val] tab]
      (buffer-set! buf idx val))
    buf))

Defining the Instrument

Now we have all our samples we define a instrument in Overtone using definst:

This allows us to specify a (huge) number of options on how our samples are played.

1
2
3
4
5
6
7

(definst sampled-flute-vibrato
  [note 60 level 1 rate 1 loop? 0 attack 0 decay 1 sustain 1 release 0.1 curve -4 gate 1]
  (let [buf (index:kr (:id index-buffer) note)
        env (env-gen (adsr attack decay sustain release level curve)
                     :gate gate
                     :action FREE)]
    (* env (scaled-play-buf 2 buf :level level :loop loop? :action FREE))))

There is quite a lot going on here. Lets focus on the configuration that effects how our sound samples are played.

Defining the Envelope Generator (env-gen).

An envelope generator (mapping to function env-gen) makes an audio signal that smoothly rises and falls. We are taking the recording of the real flute instrument as a digitized waveform, and then playing back its recordings at different speeds to produce different tones.

The most common form of envelope generator and the one we are using here is ADSR:

attack (A), decay (D), sustain (S) and release (R)

There are a number of other config settings in our example:

1
2
3
4
5
6
7
8
9
10
11
12

[
 note 60     ; Default pitch
 level 1     ; Volume
 rate 1      ; Playback rate 
 loop? 0     ; The note should loop after the first play
 attack 0    ; The way a sound is initiated. Fast attack (gunshot) vs Slow attack (closing a door slowly)
 decay 1     ; The time for notes to decay after the initial strike
 sustain 1   ; Once a sound has reached its peak, the length of time that the sound will sustain.
 release 0.1 ; The time for notes to decay after the key is released
 curve -4    ; Curve factor
 gate 1      ; note occurs when gate goes from nonpositive to positive. Note off occurs when it goes from positive to nonpositive
]

Look to the SuperCollider documentation on UGENs if you want to see all possible configuration options. Overtone is really a wrapper around the extremely powerful SuperCollider.

Grand finale

Finally we can start to play music in overtone using our flute:

1
2
3

(sampled-flute-vibrato 60) #=> pitch 60
(sampled-flute-vibrato 61) #=> pitch 61
(sampled-flute-vibrato 60) #=> pitch 60

Hear for yourself:

The Code in Full

Defining an instrument in Overtone:

Loading buffers 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

(ns overtone.samples.flute-vibrato
  (:use [overtone.core])
  (:require [clojure.string :as str]))

(defn- registered-samples
  "Fetch flute samples from the asset store if they have been manually
  registered"
  []
  (registered-assets ::TransverseFluteTenutoVibrato))

(def FREESOUND-VIBRATO-FLUTE-SAMPLES
  "Freesound ids for all samples in the Vibrato Transverse Flute pack"
  {154274 :C7  154273 :B6 154272 :A#6 154271 :A6 154270 :G#6 154269 :G6  154268 :F#6
   154267 :F6  154266 :E6 154265 :D#6 154264 :D6 154263 :C#6 154262 :C6  154261 :B5
   154260 :A#5 154259 :A5 154258 :G#5 154257 :G5 154256 :F#5 154255 :F5  154254 :E5
   154253 :D#5 154252 :D5 154251 :C#5 154250 :C5 154249 :B4  154248 :A#4 154247 :A4
   154246 :G#4 154245 :G4 154244 :F#4 154243 :E4 154242 :F4  154241 :D#4 154240 :D4
   154239 :C#4 154238 :C4})

(def FLUTE-SAMPLE-IDS (keys FREESOUND-VIBRATO-FLUTE-SAMPLES))

(def flute-samples
  (doall (map freesound-sample FLUTE-SAMPLE-IDS)))

(defn- buffer->midi-note [buf]
  (-> buf :freesound-id FREESOUND-VIBRATO-FLUTE-SAMPLES name note))

(defn- note-index
  "Returns a map of midi-note values [0-127] to buffer ids."
  [buffers]
  (reduce (fn [index buf]
            (let [id (-> buf :id)
                  note (buffer->midi-note buf)]
              (assoc index note id)))
          {}
          buffers))

;; Silent buffer used to fill in the gaps.
(defonce ^:private silent-buffer (buffer 0))

(defonce index-buffer
  (let [tab (note-index flute-samples)
        buf (buffer 128)]
    (buffer-fill! buf (:id silent-buffer))
    (doseq [[idx val] tab]
      (buffer-set! buf idx val))
    buf))
Defining the instrument 
1
2
3
4
5
6
7
8
9
10
11
12
13

(ns overtone.inst.sampled-flute
  (:use [overtone.core])
  (:require
   [overtone.samples.flute-vibrato :as vibrato]))

(definst sampled-flute-vibrato
  [note 60 level 1 rate 1 loop? 0
   attack 0 decay 1 sustain 1 release 0.1 curve -4 gate 1]
  (let [buf (index:kr (:id vibrato/index-buffer) note)
        env (env-gen (adsr attack decay sustain release level curve)
                     :gate gate
                     :action FREE)]
    (* env (scaled-play-buf 2 buf :level level :loop loop? :action FREE))))

Comments