Annotation and Representation Tools (Music)

Sheet music notation as constructed out of clefs, staves, bars and notes is a highly effective distillation of the idea of a musical work but is seriously limited in terms of what it can faithfully represent about the expressive qualities of a performance, e.g. timbre, changes in tempo and very specific interpretations to do with the onset and decay of notes. One of the key areas for musicologists interested in uses of ICT is the representation of music and what decisions need to be made to capture the essence of a performance or a recording to a scholarly and precise level of detail. This matter of degree is a core issue and encompasses debate about the amount of abstraction and reduction that is necessary to adequately represent the music without losing any of the key elements that make the relevant piece of music interesting, irrespective of whether that interest lies in the context of the historical, performance-related, technical or cultural domain. These decisions are central to the strategic policy of archives and repositories and are obviously bound up with universal and ongoing discussions about standards more generally, not just for musical resources but wherever digital material is being acquired and archived.

Potentially, the most useful long-term approach to the representation of musical resources in archives would be (unsurprisingly) to include as much data as possible in relation to both the recorded signal (acoustic or electronic) and the available complementary data that provides the context for the production of that signal. The first of these involves two main considerations: the sample rate at which the signal is recorded; and the compression ratio used by the format in which the file is saved. The compressed file acts as an index and surrogate for the original performance and once that performance is finished, the resultant recording is the best approximation that researchers have to reality; and it is therefore critical to many areas of scholarship that as little of the original information is discarded as possible. Regarding complementary data, the same principle applies but might be formulated in a slightly different way. Having the clearest audio source possible will undoubtedly be of service to subsequent generations of researchers; it is less clear however which questions future researchers will want to ask of the metadata and considerations of what to exclude will always need to be taken in light of emerging patterns of research and technological advances.

In practice, capturing information at the comprehensive level of detail that the comments above endorse is an expensive and substantial commitment, but there are nonetheless tools available which attempt to represent music in a scholarly way, some of which are based on XML approaches. The literature on these applications is extensive but a good summary of some of the packages that have been developed can be found on a website hosted by OASIS (the Organisation for the Advancement of Structured Information Pages). Out of all of the packages that have been developed, it might be instructive to look more closely at two examples, MusicXML and MEI (the Music Encoding Initiative).

MusicXML is commercially developed software and its purpose is to allow the interchange of musical data. The key function of this system, based on a set of 13 document type definitions (DTD’s), is to allow users a way of producing printed scores from intricately encoded XML documents encompassing a very wide menu of elements, attributes and entities that attempt to precisely annotate both the acoustic and descriptive components of a piece of music. The encoding system is interoperable with a wide range of other applications, including the market leading commercial notation packages Sibelius and Finale and also the widely used academic tools, MuseData and Humdrum.

MEI is the copyright of Perry Roland and University of Virginia and also relies on a very detailed set of DTD’s to facilitate precise description of musical works. [quote]It is designed to be comprehensive, that is, it provides ways to encode data from all the separate domains commonly associated with music, i.e. logical, visual, gestural (performance), and analytical.
Perry Roland ([/quote]

The design and development of MEI was influenced by the principles that guided the creation of the Text Encoding Initiative (TEI). Both approaches do not assume that the object of the encoding is by definition a specific type of entity (i.e. a particular type of written work or a particular facet of the musical process), but there is an underlying assumption that the original entity can be expressed in a written form. The scope and flexibility within MEI means that the music community is free to encode a wide range of materials using the system including critical editions and collections of works containing extensive amounts of music-related text; material which would usually lend itself to TEI encoding methods.

Where possible, MEI uses familiar terminology for elements and attributes, allowing the marked-up text to be both machine and humanly readable. This accessibility to users without advanced technical software knowledge is a critical factor for the widespread use of XML (and its related techniques) across all disciplines, but it is also extensible enough to enable MEI to cover imprecise or historical musical phenomena, such as the encoding of variant readings and support for mensural music and non-aligning bar lines. In combination with the fact that MEI files can be derived from MusicXML data and can be transformed into MIDI format (and can be also be converted to postscript and pdf renditions using XSLT), there are compelling reasons for putting further resources and effort into developing this system to accommodate even more sophisticated descriptions of musical data, appropriate for the purposes of scholarship

Despite the persistence of XML developers in their attempts to provide a de facto (or actual) standard for musical representation, aspects of that representational process continue to elude satisfactory definition. The principle problem for some researchers is that musical knowledge might be characterised as multiply hierarchical and may require the sort of overlapping and cross-category descriptions that XML is not primarily designed to accommodate, based as it is on tree-like structures. Geraint Wiggins has stated that using non-standard features of XML to try and cope with this complexity (e.g. milestones) should be considered bad engineering. He argues that this term would be equally applicable to a situation where one tried to implement whatever mechanism one had devised to compensate for XML’s shortcomings on top of an XML structure, in order to try and adequately represent the original data. In proposing an alternative model for musical description, which he broadly refers to as ‘knowledge representation’, he describes an approach that would seek to increase the scope for both expressive completeness and structural generality, two ideas that are normally more or less mutually inverse. His contention is that the minimal expressive power required to represent the multiple hierarchies of music is the directed graph (see fig.2) and that this should be used in conjunction with XML which would facilitate data interchange of syntactic descriptions of the graphs.

fig. 2 Graph diagramsfig. 2 Graph diagrams
Fig. 2 (taken from:

It must be allowed that the type of ‘knowledge representation’ language that Wiggins refers to may raise more questions than it answers but it is one of a number of attempts to lay out theoretical pre-requisites for a comprehensive representation system for music that accommodates both engineering and philosophical issues.

Another method of representation is to display digital images of printed musical scores and to enhance their usability with web and database techniques, as featured in the Online Chopin Variorum Project. Using CSS (cascading style sheets) and javascript, the website offers an unusual level of user interactivity in that bar-length details (generated on-the-fly) from a page of music can be dragged around the screen and superimposed over each other for comparison purposes. In the pilot phase of this project, research was also carried out into providing a third-party online annotation system and some progress was made in scoping out the problem. In the second phase of this project (end April 2008) further investigation into this area is scheduled. The same team is also working on a project to create a substantial archive of around 5000 digital images representing Chopin’s first editions, which will interlink with information taken from John Rink and Christophe Grabowski’s Annotated Catalogue of Chopin’s First Editions.

Syndicate content