AI Piano Transcription: Notes Are Not a Finished Score

A piano recording goes into an AI system and a file of notes comes out. For a creator trying to recover an improvised idea, that could be a useful beginning. But “the notes have been detected” and “the score is ready to publish” are different claims. Between them sit questions about repeated notes, timing, musical structure and the instructions another player will actually read.

The original Onsets and Frames research offers a clear way to examine that gap. Its contribution was a method for transcribing piano performances, not a promise to turn every recording into finished sheet music. Understanding the distinction helps creators assess demonstrations without dismissing the underlying achievement.

What the 2018 system was designed to recover

Curtis Hawthorne and colleagues presented Onsets and Frames: Dual-Objective Piano Transcription at ISMIR 2018. The system combined predictions about note beginnings, or onsets, with predictions about which pitches were active across short time frames. During inference, a new note could start only when the onset detector agreed.

This addresses two related questions: has a note begun, and is that pitch still sounding? The paper evaluated frame-level and note-level measures, including a measure that also checked note endings. It used the MAPS piano dataset, training on synthesised pieces and testing on Disklavier piano recordings, with pieces kept separate across the splits.

That is evidence within a defined piano-transcription task. It does not establish the accuracy of a full-band transcriber, a singing transcriber or a notation editor. This article examines the original research, rather than presenting it as a new 2026 release or a description of every present-day transcription product.

One held pitch is not two repeated notes

The researchers’ Magenta explanation, published on 12 February 2018, describes converting solo piano recordings into a MIDI piano roll: a representation that can support synthesised playback or provide a starting point for sheet music. Its examples acknowledge transcription mistakes. The article also shows why requiring onset evidence can remove brief false activations and unwanted reactivations of a pitch.

Consider an original, invented teaching example. A pianist plays middle C, releases it and then strikes middle C again. A different performance contains one held middle C spanning roughly the same overall interval. The pitch name is the same throughout, but the performances contain different events.

A useful review would therefore ask whether the output contains two starts in the first case and one in the second. Merely finding middle C somewhere in the result would leave the central distinction unchecked. This example illustrates the question; no audio was recorded and no model was run for this article.

There is a second possible error to inspect. Suppose a held note is represented as several tiny notes separated by gaps. The output may contain activity around the correct pitch while giving another musician an incorrect account of the gesture. Counting matching pitch labels alone would not tell the reviewer whether the event had been preserved.

A performance representation still needs notation decisions

The MusicXML 4.0 notation tutorial distinguishes musical performance information in MIDI from the additional information needed for notation. Its examples include staff allocation, stem directions, beaming, accidentals and expressive markings. Sound-related information and the appearance of a score are related, but they are not interchangeable.

For a creator preparing a lesson, this suggests two separate approvals. First, does the recovered material give a satisfactory account of the recorded performance? Second, does the written version communicate the intended lesson? A file that can be played back is not evidence that both approvals have happened.

Imagine a fictional teacher who records a short piano exercise for a downloadable handout. The intention is a steady, simple rhythm, but one note in the demonstration is played slightly late. The teacher must decide whether the handout should document that particular performance or teach the intended regular pattern.

Those are different editorial briefs. A literal performance account might retain the delay. A teaching edition might regularise it after the teacher confirms the intended rhythm. The reviewer should record that choice rather than allow an unexplained conversion step to decide what the exercise means.

Review a small passage through the whole process

Before commissioning a long transcription, I would choose a short passage with a known purpose and ask a musically qualified reviewer to compare the original recording, recovered events and proposed notation. The passage should include something informative, such as a repeated pitch or a held note, rather than only the easiest phrase.

• Check the events: identify missing notes, extra notes and repeated attacks that have been merged or split.
• Check the timing: compare starts and endings, and distinguish a recovery error from an intentional editorial change.
• Check the reading task: ask whether another player can understand the proposed rhythm, grouping and distribution across staves.
• Check the handover: retain the recording, unedited output, reviewed version and a short account of the changes.

This is a proposed review process, not a measured success rate. It deliberately produces evidence about a particular passage and workflow. A clean demonstration on solo piano cannot stand in for a test on an ensemble recording with vocals and percussion.

If the supplier provides only a finished-looking page, ask whether the intermediate events and correction decisions are available. That makes it easier to locate a problem: was a second strike missed during transcription, or were two correctly recovered events merged while preparing the notation? The repair belongs at the stage where the information changed.

Describe the output at the level actually checked

An independent creator might reasonably publish a handout labelled as an edited teaching arrangement after musical review. A researcher might report note-detection results on a benchmark. A software supplier might demonstrate a transcription file. Each output can have value, but the descriptions should remain attached to the work actually assessed.

For business owners commissioning creative tools, the practical question is what remains for the human reviewer to approve. Include that work in the brief and the delivery schedule. Do not promise an immediately publishable score merely because a demonstration produces a recognisable tune.

MaryChuks.com’s article on separating constructions from deductions in AI geometry explores a similar boundary between stages of a system. Here, the boundary lies between recovering musical events and deciding how another person should read them.

The original Onsets and Frames work makes note beginnings an explicit part of piano transcription. For a creator, the broader lesson is equally concrete: preserve what was played, identify what was changed and review the score for its intended reader before calling the job finished.

Sources

Primary sources checked on 30 October 2026. The teaching examples and review process above are original illustrations; no transcription, software installation or benchmark test was performed.

• Hawthorne and colleagues: Onsets and Frames: Dual-Objective Piano Transcription — ISMIR 2018 conference paper.
• Magenta: Onsets and Frames — project explanation, 12 February 2018, with later 2018 update notices.
• MusicXML 4.0: Notation Basics — performance and notation representations.