NotePerformer is a piece of software which works with music notation software to improve the sound of those programs. It interprets notation and generates playback to best indicate how a real player might interpret and play that notation. Information on this software is hard to find online, but since it’s becoming increasingly popular and it’s already widely used, this encouraged me to put some more information out there. To do this I reached out to Arne Wallander, the inventor and developer of NotePerformer. All of the following information came from an email interview with him and I’ve tried to stick as closely to his wording as possible.

NotePerformer was released in 2013 by Wallander Instruments. Before this, Wallander Instruments had only released WIVI, a VST wind instrument package. NotePerformer is like a VST instrument, in that it’s a simulation of the sound of real instruments, but Wallander is quick to point out it’s divergent goals: “Noteperformer is very different from VST instruments, because the main purpose of the software isn’t just to produce audio, but to improve the quality and structure of the printed score, by guiding the composer towards making wise choices in orchestration. In many respects, NotePerformer isn't primarily an audio device, but it's a multi-faceted add-on to any notation program, which improves people's workflow, possibilities for proof-listening, loading times, and other aspects of the notation program.” These different goals are in essence what prompted the creation of the program. As Wallander writes, “NotePerformer is exactly the kind of software that I wished to have, when I started looking into VST instruments, almost 20 years ago. I never enjoyed editing MIDI by hand, so being able to work with a standardised abstract representation of your music (notation) which translates directly to a live orchestra, was always a dream of mine. I think the educational aspect of learning how to properly write music for a live orchestra is very important.”

VST’s did have an influence on NotePerformer, however. WIVI, Wallander Instrument’s first release, had a profound effect on it’s development, mostly acting as an example of what Wallander wanted to avoid: “WIVI was the exact opposite of NotePerformer, being a performance instrument, and it required lots of quality MIDI data to shine. In many ways it embodied the very things I wanted to get away from with NotePerformer. NotePerformer reuses many of the signal processing parts from WIVI, improves on it, but also fully automatises the aspect of performing the instruments, by doing so with algorithms. Also, I'm not a big fan of a million buttons and knobs. I prefer technology which does exactly what it's supposed to, from the very start. WIVI is a very complex software, with lots of confusing buttons and knobs that are overwhelming and redundant for 90% of the users. NotePerformer is essentially a GUI-less software, and I like that better, because I've always been very much a fan of creative limitation–the process of limiting your choices to drive creativity. I think NotePerformer embodies that very successfully.”

Since NotePerfomer isn’t just a sample library, but rather an adaptive program that creates new sounds based on the needs of the piece it’s ‘performing’, I was curious to learn more about its function. Wallander explained it as such: “In short, NotePerformer reads ahead one second into the score, and renders the performance of each note, within the context of the music and phrase it belongs to. NotePerformer also does lots of error-correction, such as correcting for idiosyncrasies of the notation program, and it simulates how the physical limitations of the underlying instrument affects the expression and performance of notes. There's a lot more to it. For example, NotePerformer deals with timing and balance much better than any complex multi-articulation sample library.” NotePerformer needed a frame of reference to be able to work the way it does, and as is explained in its promotional video (see top of this article) it was actually trained on existing sheet music and recordings of classical music. I wanted to know what this process looked like, and although this technology is proprietary and therefore it’s creation is something he couldn’t explain in depth, Wallander explained it wasn’t an automated process, “this step was very time-consuming and experimental, and an ongoing process. NotePerformer generates a performance from hundreds of performance rules. Many of them make little sense on their own, but they have a very high hit rate with orchestral recordings with the right combination of parameters.”

These features were in many ways unexplored territory, and it was bound to run into some unique problems, “Initially, figuring out how to get access to future events of the score was the greatest challenge. There's no standard technology for looking into the future of a score, during playback. The latency is a tradeoff solution which only partly solves that problem, as having only one second to go by is a challenge in itself. There's really not much information to go by, and very often NotePerformer needs to intelligently predict what's next. We could increase the latency, but to make better performance choices we'd have to increase the range to 5-10 seconds. That kind of delay would be unbearable for the user.”

Although this sounds impressive, I anticipate some composers committed to the use of sample libraries might not be convinced of the added value of using a program like this. I asked Wallander what he thought made using NotePerformer better than using a regular sample library; “The by far biggest benefit of NotePerformer is the proof-listening aspect. There are lots of high-quality sample libraries out there, where the individual samples surpass NotePerformer in audio quality, however… what you see is not what you hear. If you use those libraries as a frame of reference for your orchestration, your printed score is going to be very poor in quality. A live orchestra won't play the music the way you intended it to be played, and the balance between instruments will be way off target. It's very counterproductive to have playback which limits you or leads you in the wrong direction. Also, the structure and articulations in a sample library very often don't make sense, from an orchestral perspective. The most obvious example is how “staccato” has become a representation of any short note, but most scores that predate sample libraries use it very sparingly, if at all. It must be noted that none of these libraries were designed to be used on a notation program, but in a DAW, so they don't come to their full right in the more limited MIDI environment of a notation program. These libraries are all excellent when used in the environment for which they were built.”

I am already an avid user of NotePerfomer, and for any other user like me I wondered what plans there were for NotePerformer’s future. Although Wallander was hesitant, his answer left me with a lot to look forward to; “NotePerformer 3 is a very polished program, as is, and it's essentially at a stage where I'm almost afraid of touching the performance aspect of it, because it's so delicately tuned. Taking things to the next level will require a whole new approach to the AI aspect of NotePerformer, and possibly also the approach by which the sound is generated, and this development is going to take a lot of time. Years, possibly. But we do have a plan, and I hope and believe it will be worth the wait.”

• Xena Derks