Categories
android android-things

Android Things – Building a Distributed Piano with Nearby Connections API

In the previous post, we had a short introduction to some hardware basics for the Software Engineer. We learnt about some different components and how to create a blinking LED with Android Things. In this blog post, we will learn to build a distributed piano with Android Things, a Piezo Speaker and Google’s Nearby API.

Hardware setup for an Android Things Distributed Piano:

You will need:

In order to create the speaker component, set up your breadboard with the piezo by following the diagram below:

Distributed Piano Piezo Speaker Setup
Distributed piano piezo speaker setup

Attach the speakers to a PWM pin (red line in this diagram) and to a GND pin to complete the circuit.

If you remember from the previous blog post, we used the GPIO pin to send an on/off signal to the LED. Because the speaker requires us to send different notes to it, a simple on/off wouldn’t work in this case, which is why we use a PWM pin.

PWM (Pulse Width Modulation) is a technique used to encode a message into a pulsing signal. Its main use is to control the power supplied to electrical devices. PWM is used for things such as controlling servo motors or controlling a piezo.

Software Component

Once the hardware is set up, we can move onto creating the piano apps. The descriptions below will highlight the important parts, the full source code can be found here.

We will create two apps, one that will run on the Android Things compatible board and the other that will act as the companion piano app.

Android Things App

  1. Once we have created a project, make sure to include the dependencies for Android Things, Google Play Services and the speaker driver in your build.gradle. In your AndroidManifest.xml, include the permission to ACCESS_NETWORK_STATE  as we will need this whilst using the Nearby API. Define the SERVICE_ID and make sure your PianoActivity  is set to be the IOT_LAUNCHER.
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
       
        <application
            ...>
            <meta-data
                android:name="com.google.android.gms.nearby.connection.SERVICE_ID"
                android:value="@string/service_id" />
    
    
            <activity android:name=".PianoActivity">
                <intent-filter>
                    <action android:name="android.intent.action.MAIN" />
    
                    <category android:name="android.intent.category.LAUNCHER" />
                </intent-filter>
                <intent-filter>
                    <action android:name="android.intent.action.MAIN" />
    
                    <category android:name="android.intent.category.IOT_LAUNCHER" />
                    <category android:name="android.intent.category.DEFAULT" />
                </intent-filter>
            </activity>
    
        </application>
    compile 'com.google.android.things.contrib:driver-pwmspeaker:0.1'
    provided 'com.google.android.things:androidthings:0.1-devpreview'
    compile 'com.google.android.gms:play-services-nearby:10.0.1'
  2.  In the PianoActivity, we will broadcast that we are allowing “nearby connections” on this device. This code is based off the sample provided here for nearby connections.
      List<AppIdentifier> appIdentifierList = new ArrayList<>();
            appIdentifierList.add(new AppIdentifier(packageName));
            AppMetadata appMetadata = new AppMetadata(appIdentifierList);
            Nearby.Connections.startAdvertising(googleApiClient, serviceId, appMetadata, 0L, this)
                    .setResultCallback(new ResultCallback<Connections.StartAdvertisingResult>() {
                        @Override
                        public void onResult(@NonNull Connections.StartAdvertisingResult result) {
                            Log.d(TAG, "startAdvertising:onResult:" + result);
                            if (result.getStatus().isSuccess()) {
                                Log.d(TAG, "startAdvertising:onResult: SUCCESS");
                            } else {
                                Log.d(TAG, "startAdvertising:onResult: FAILURE ");
                                int statusCode = result.getStatus().getStatusCode();
                                if (statusCode == ConnectionsStatusCodes.STATUS_ALREADY_ADVERTISING) {
                                    Log.d(TAG, "STATUS_ALREADY_ADVERTISING");
                                } else {
                                    Log.d(TAG, "STATE_READY");
                                }
                            }
                        }
                    });
    @Override
        public void onConnectionRequest(final String endpointId, String deviceId, String endpointName, byte[] payload) {
            Nearby.Connections.acceptConnectionRequest(googleApiClient, endpointId, payload, this)
                    .setResultCallback(new ResultCallback<Status>() {
                        @Override
                        public void onResult(@NonNull Status status) {
                            if (status.isSuccess()) {
                                Log.d(TAG, "acceptConnectionRequest: SUCCESS");
                            } else {
                                Log.d(TAG, "acceptConnectionRequest: FAILURE");
                            }
                        }
                    });
        }
  3. On receiving a message from the nearby API, translate the message into a frequency that the speaker will play. This example is using the PWM speaker driver from here.
      @Override
        public void onMessageReceived(final String s, final byte[] bytes, final boolean b) {
            Log.d(TAG, "onMessageReceived");
            double frequency = toDouble(bytes);
            if (frequency == -1) {
                view.stopPlayingNote();
                return;
            }
            view.playNote(frequency);
        }
    @Override
    public void playNote(final double frequency) {
        try {
            speaker.play(frequency);
        } catch (IOException e) {
            throw new IllegalArgumentException("Piezo can't play note.", e);
        }
    }

    That is all that we require for the Android Things component.

Companion Piano Player App

  1. Create a module inside your current project for the companion app. This app will need to include the Nearby Connections API dependency too.
  2. In the layout xml file, we will include a keyboard view and a scrolling bar to scroll the keyboard.  I have extracted the KeyboardView  from the repository that can be found here.
    <?xml version="1.0" encoding="utf-8"?>
    <LinearLayout
        xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns:app="http://schemas.android.com/apk/res-auto"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:background="#ffffff"
        android:orientation="vertical">
    
        <za.co.riggaroo.androidthings.pianoplayer.keyboard.ScrollStripView
            android:id="@+id/scroll_strip"
            android:layout_width="match_parent"
            android:layout_height="30dp"
            />
    
        <za.co.riggaroo.androidthings.pianoplayer.keyboard.KeyboardView
            android:id="@+id/piano_view"
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            app:first_octave="1"
            app:octaves="4" />
    </LinearLayout>
    
  3. In the activity, create a GoogleApiClient and start listening for any connections that are advertising themselves. For simplicity, we will just connect to the first endpoint that is discovered.
        Nearby.Connections.startDiscovery(googleApiClient, serviceId, TIMEOUT_DISCOVER, this)
                    .setResultCallback(new ResultCallback<Status>() {
                        @Override
                        public void onResult(@NonNull Status status) {
                            if (!isViewAttached()) {
                                return;
                            }
                            if (status.isSuccess()) {
                                Log.d(TAG, "startDiscovery:onResult: SUCCESS");
                            } else {
                                Log.d(TAG, "startDiscovery:onResult: FAILURE");
    
                                int statusCode = status.getStatusCode();
                                if (statusCode == ConnectionsStatusCodes.STATUS_ALREADY_DISCOVERING) {
                                    Log.d(TAG, "STATUS_ALREADY_DISCOVERING");
                                }
                            }
                        }
                    });
    @Override
        public void onEndpointFound(final String endpointId, String deviceId, String serviceId, final String endpointName) {
            Log.d(TAG, "onEndpointFound:" + endpointId + ":" + endpointName);
    
            connectTo(endpointId, endpointName);
        }
        private void connectTo(String endpointId, final String endpointName) {
            Nearby.Connections.sendConnectionRequest(googleApiClient, null, endpointId, null,
                    new Connections.ConnectionResponseCallback() {
                        @Override
                        public void onConnectionResponse(String endpointId, Status status, byte[] bytes) {
                            if (!isViewAttached()) {
                                return;
                            }
                            if (status.isSuccess()) {
                                Log.d(TAG, "onConnectionResponse: " + endpointName + " SUCCESS");
                                view.showConnectedToMessage(endpointName);
                                otherEndpointId = endpointId;
                            } else {
                                Log.d(TAG, "onConnectionResponse: " + endpointName + " FAILURE");
                            }
                        }
                    }, this);
        }
    
  4. Obtain a reference to the KeyboardView and attach a listener which will trigger when the notes are pressed on and off.
            KeyboardView keyboardView = (KeyboardView) findViewById(R.id.piano_view);
            ScrollStripView scrollStrip = (ScrollStripView) findViewById(R.id.scroll_strip);
            scrollStrip.bindKeyboard(keyboardView);
            keyboardView.setMidiListener(new KeyBoardListener() {
                @Override
                public void onNoteOff(final int channel, final int note, final int velocity) {
                    presenter.noteStopped(note);
                }
    
                @Override
                public void onNoteOn(final int channel, final int note, final int velocity) {
                   presenter.notePlayed(note);
                }
            });
    
  5. When a note is pressed send a message via Nearby API with the corresponding note frequency to the connected Raspberry Pi. Send a “stop” signal, when the KeyboardView listener detects that the note has stopped playing.
    @Override
    public void notePlayed(final int noteNumber) {
        double frequency = getFrequencyForNote(noteNumber + NOTE_OFFSET);
        sendNote(frequency);
    }
    
    @Override
    public void noteStopped(final int note) {
        sendStop();
    }
    
    private void sendStop() {
        if (!googleApiClient.isConnected()) {
            view.showApiNotConnected();
            return;
        }
        Nearby.Connections.sendReliableMessage(googleApiClient, otherEndpointId, toByteArray(-1));
    }
    
    private void sendNote(final double frequency) {
        if (!googleApiClient.isConnected()) {
            view.showApiNotConnected();
            return;
        }
        Nearby.Connections.sendReliableMessage(googleApiClient, otherEndpointId, toByteArray(frequency));
    }

That is how you can make a distributed piano using Android Things. For the full source code check the github repository here.

Conclusion

By using Android Things and the Google Nearby API we could quite easily create a distributed piano. This post demonstrated how easily you can build your own Android Things Piano.

If you manage to play something awesome with this example, please share it with me on twitter @riggaroo. Check this Sia cover by Gautier Mechling:


If you want to get updates from this blog, be sure to subscribe below:

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.