Learning piano with a real piano and a microphone

You don't need a digital piano or any cables to use a learning app with a real piano. Many apps, Pianuvo included, can listen to an acoustic piano through the microphone on your phone, tablet or laptop and work out which notes you are playing. It works surprisingly well, but it is not magic. Knowing how it works and where its limits are will help you set it up well and understand the occasional missed note.

How an app hears a note

When you press a piano key, a hammer strikes the strings and they vibrate. The sound you hear is not one pure tone. It is a main pitch, called the fundamental, plus a series of quieter, higher tones called overtones or partials. Together they give the piano its sound.

A pitch-detection program listens for that pattern. It splits the incoming sound into its separate frequencies, looks for a fundamental with its family of overtones above it, and matches it to the nearest key. It also has to notice the moment a new note starts, so it can tell a fresh key press from a note that is still ringing.

All of this happens many times a second. A connected MIDI keyboard tells the app directly which key went down, so it is always a little more exact and a little faster than listening. But for learning and for scoring your playing, a microphone is usually good enough, and it means any piano will do.

What it does well, and what is harder

Single notes are easiest. One note at a time gives a clear fundamental and a clean set of overtones. Melodies played with one hand, which is most beginner music, are the best case.

The same three notes played one at a time (bar 1) and together as a chord (bar 2): the single notes are easier for a microphone to tell apart

Chords are harder. When several notes sound at once, all their overtones arrive mixed together, and some of them overlap. The program has to untangle which overtones belong to which note. Good detectors manage simple chords, but very fast chords or thick chords with many notes are more likely to lose a note.

Octaves are hardest. Two notes an octave apart overlap almost completely: in principle, every overtone of the upper note is also an overtone of the lower one. To a microphone, an octave can sound much like the lower note on its own.

Octaves: the upper note's overtones line up with the lower note's, so they are the hardest pairs for pitch detection to separate

Very high and very low notes are less reliable than the middle of the keyboard. The deepest bass notes have a weak fundamental, and the highest notes have few overtones to check. Beginner music sits mostly in the middle, so this rarely matters at first.

Long notes and the sustain pedal. A piano note keeps ringing after you strike it, and with the sustain pedal down, every note you play keeps ringing together. The program has to decide whether each sound is new or left over. If you play the same note again softly while it is still ringing loudly, it may be missed.

Repeating a note: release the key a moment before each repeat, and play each one clearly

Room noise

The microphone hears everything in the room, not just the piano. Talking, a television, music playing in another room, a barking dog and especially the app's own sound coming out of the speakers can all be heard. Sounds that resemble piano notes are the most likely to be mistaken for notes.

Practise somewhere reasonably quiet.
Turn the app's own sound down, or use wired headphones for it, so the microphone doesn't hear the app playing.
Avoid Bluetooth headphones for the app's sound if timing matters, as they add a delay.
Very echoey rooms can blur notes together. Soft furnishings, curtains and a rug all help.

Placing your device

Put the phone, tablet or laptop where it can hear the piano clearly, and close to it.

On an upright piano, the music stand is a good place. The strings are just behind the front panel, so a device on the stand is close to the sound.
On a grand piano, the music stand also works well.
On a digital piano played through its own speakers, put the device near the speakers.

Find where your device's microphone is (on many tablets and phones it is a small hole near an edge) and make sure a case, your sheet music or your hand doesn't cover it. Then play a few notes and watch whether the app reacts.

Test the whole range you will use with a low, a middle and a high note.

A quick placement test: play low C, middle C and high C, and check the app hears all three

Allowing the microphone

A web page can only use the microphone if you give it permission. The first time an app asks, your browser shows a prompt. Choose Allow. Browsers also only allow the microphone on secure pages, whose addresses start with https.

If you chose Block by mistake, you can change it in your browser's site settings, usually opened from the icon next to the address bar. On an iPad or iPhone using Safari, open the Settings app, go to Safari, then Microphone, and choose Allow. Then reload the page.

Tuning

Pitch detection compares what it hears with the pitches of a correctly tuned piano. Most pianos today are tuned so that the A above middle C sounds at 440 vibrations per second (440 hertz), and every other key is set in relation to it.

The A above middle C is the usual tuning reference: 440 hertz on a piano tuned to standard pitch

Acoustic pianos drift out of tune over time, particularly with changes in temperature and humidity. A piano that is slightly out of tune is usually fine, and a good detector allows for a piano that sits a little sharp or flat overall. But a piano that has not been tuned for many years can sit so far below standard pitch that notes start to be heard as the key next door. Strings that are out of tune with each other within one note, which makes it sound sour or wobbly, can confuse detection too.

If you suspect your piano is far out, a free tuner app can check: play the A above middle C and see how close it is to 440 hertz. If it is a long way off, the piano needs a visit from a piano tuner. Your ears will thank you too, since a well-tuned piano makes practice more pleasant.

Privacy

Letting an app use your microphone is a reasonable thing to think carefully about. Check each app's privacy policy to see what it does with the sound.

In Pianuvo, the privacy policy says that the sound is analysed in real time on your device, only while you are practising a piece, to work out which notes are being played. It is not recorded, stored, or sent to Pianuvo or any third party. The one exception is the Academy's optional recording feature, which saves a recording only when you start it yourself, and only on your device. You can turn microphone access off at any time in the app or in your browser or device settings.

Try it in Pianuvo

Choose Play my real piano when the app asks how you will play, or change it later in Settings. There you can test the microphone and adjust its sensitivity: lower if it reacts to talking or noise, higher if it misses quiet notes. Then try a one-hand piece such as Mary Had a Little Lamb, or start Unit 0 of the Academy. If you have a digital piano with a USB port, using a MIDI keyboard is the most exact option.