Rob Tuley wrote:I think I'm missing something here....
Thanks for joining this discussion and giving me the chance to explain myself better. Thanks to Steinberg too for that!
Rob Tuley wrote:...but the way I see it, unless your proposal only does "display this fixed SVG image for x seconds and then switch to the next fixed image" you must have some semantic information about the images hidden somewhere.
My proposal is very simple. An SVG+MIDI format allows applications to be written that "display [and play] this fixed SVG image for x seconds and then switch to the next fixed image". But not only that: if a DAW could read such files, then it could change the temporal information inside the symbols and write the file back out.
Begin Edit 21.10.2016
Rob Tuley wrote:
For example what happens if you want to display the images on a device with an inappropriate resolution for their content? Either you just say "well, it's just too bad you can't actually read anything useful here" to the user, or you have split up or scroll the image somehow - but how?
: My first answer to this wasn't very good (sorry, nobody's perfect).
The best way to solve the screen resolution problem is for the authoring application to supply the score in various versions designed for the appropriate screen sizes. This is the way web sites are, or should be, handled for widely different screen sizes. The client application then just has to check the user's screen resolution, and select the appropriate score. Once the bars-per-system distribution is correct, it is trivial for an application to add scroll bars or change the display scale to fit a particular screen. Which score the app decides to use, and what it does with it, is up to the app.
Rob Tuley wrote:Scroll it vertically, bottom to top or top to bottom? Horizontally, right to left or left to right? Rotate it in a circle, clockwise or counterclockwise ? Semantics, semantics ....
All SVG elements can have class attributes that indicate which type of object the element represents. These can be abstract container classes ("system", "staff", "chord" etc.) or CWMN objects such as "slur", "tupletBracket" etc. The SVG namespace
mechanism is used to tell the client application which classes exist in the file, and how they nest. There can also be code in the file that lets the application know whether the notation is read top to bottom or right to left etc. Defining such things would be done in the process of defining the standard. Not all SVG+MIDI scores need to be written in CWMN, but those that are
might well use class names taken from MusicXML.
End Edit 21.10.2016
Rob Tuley wrote:
And on the MIDI side, somehow you have to define what patches to use from what sample libraries - or encapsulate the playback libraries in the file, and then define the semantics of exactly how a player should interpret MIDI commands like note velocity, channel control data, etc - the MIDI standard doesn't do that, except in the most vague terms (what exactly is a "synth drum" supposed to sound like, for example...)
Think of the MIDI in an SVG+MIDI file as being equivalent to a MIDI file. Its just that the MIDI info is written in plain text (like SVG), and is distributed around the file in various containers. Just as with MIDI files, its up to applications to decide when to send MIDI commands, and to which MIDI device they are going to be sent.
Begin Edit 24.10.2016
Rob Tuley wrote:
Or, you build something like the app you linked to, which only works in a couple of web browsers on one OS, and demands that the user install an exact list of other software components that it knows how to talk to (and by implication, any composer using it is constrained to realize their score using only what is available in those specific apps?)
: Here, my first answer was simply wrong! Sorry, I was writing in too much of a hurry.
First, Web apps
My app uses both the WebMIDI
API and the WebAudio
API. Support for these in the various browsers can be checked at http://caniuse.com/#feat=midi
Currently, the WebMIDI
API is only supported natively in Chrome, Opera and the Android browsers. More browsers than that already implement the WebAudio
The Jazz plugin can be used in Firefox and other browsers as a shim for the WebMIDI
If the WebMIDI
API is implemented somehow, then the app automatically recognises any (software or hardware) MIDI input or output devices attached to the computer. The user can direct the MIDI output to any output device attached to their machine.
Since not everyone has a (software or hardware) MIDI output device attached to their computer, I adapted an existing body of code for use as a resident soundfont synthesizer. This uses the WebAudio
API, resides on the web site next to the app, and does not have to be installed on the user's computer. Its a bit primitive (I'm not an audio expert), but demonstrates that the technology works in principle. Hopefully, someone more expert than me will improve it sometime. Someone at Steinberg maybe?
This all works (if it works at all) on any operating system
supported by the browser.
I designed this app for the web, not only because I want my scores to be playable there, but also so that I wouldn't have to write an SVG graphics renderer. However, web apps are not the only apps that could use SVG+MIDI files.
(such as Cubase) can use them too. There are actually lots
of applications for such files: The academic world could use them for adding arbitrary annotations or using coloured highlighting; different performances can be stored in the same score in order to learn/demonstrate/study different ways of playing the score; scores can be synchronized with existing recordings or videos in any format etc., etc.
Performance practice is learned by listening
in real time. Using such techniques, composers can provide performers with accurate temporal renderings of a new piece before
the first rehearsal thus reducing expensive rehearsal time. That's important, not only to those parts of the industry that use CWMN, but also to the learning of less common notations (e.g. classical Asian notations) and for the healthy development of new notations for New Music.
Music notation is, and always has been, an aide memoire
End Edit 24.10.2016
Begin Edit 21.10.2016
PaulWalmsley wrote:As you will appreciate, with very complex software bases It's Not Quite That Simple. Although you see the score scrolling in time with the music, that in no way implies that the playback code 'knows' about the drawn notation data in any way -- they are completely separate subsystems. However there are a handful of isolated interfaces that do convey some limited information between the two, such as the playback line.
I can't see your code, of course, but it sounds to me as if you are fighting a problem with your time paradigm. If tempo is a fundamental concept, and symbols have fixed meanings in a temporal ether, then temporal information is going to leak into the graphics and you are going to have to start using fictitious time layers...
Begin Edit 22.10.2016
Sorry, but your problem is still going through my mind. Here are some more thoughts:
Looking at the beginning of the video at
http://www.sonicstate.com/news/2016/09/ ... preadbury/
I see that the playback cursor takes up positions between the symbols
. This is, I think, a mistake. Dorico, like other contemporary CWMN editors, is trying to treat symbolic music notation as if it were space=time notation. Its a legacy problem in the time paradigm. CWMN developed in the 19th century under the assumption that there is a temporal ether (from which real performances are supposed to deviate). The continuously moving cursor represents that (non-existent) ether.
Things get much
simpler if all
the duration information is stored inside the chord symbols (separately from their graphics). If you do this, you can probably replace some large, complex chunks of code by a few simple calculations.
This means that the cursor should jump from symbol to symbol, waiting on each symbol until the next symbol is scheduled to begin. There is no time between
the symbols. That may look a bit odd at first, but I think your users would soon get used to the new behaviour, just as they have to the behaviour of digital clocks. (I'm old enough to remember the introduction of digital clocks, and how strange it looked to see the second hand jumping from marker to marker.)
End Edit 22.10.2016
As I said, I think its very important to consider the long term goals in order to get Dorico's architecture right. And its important to get the architecture right as early as possible.
For a start there's a many-to-many relationship between a drawn note and a played MIDI note...
My prototype SVG+MIDI format supports ornamented chord symbols. I see no problem there.
Also, it is, in principle, possible both to save different performances of a score in parallel layers of temporal info in the same file, and to have different scores (graphics) for identical temporal info. (See the following post.)
End Edit 21.10.2016
All the best,