New Sibelius to LilyPond conversion suite

[Kirill]

This new tool I wrote might be of interest to some:

http://www.sidorefa.com/sib2ly/

To my best knowledge, this is the most powerful
Sibelius to LilyPond converter to date.

The whole thing is, of course, free and open source.
Tell me what you think.

Best,

Kirill Sidorov

[Ewald Gutenkunst]

Hi,

Very interesting tool!
is there any way to use it on Mac?
The first step (the sib plugin) works already well.

~Ewald

[Johan Vromans]

Post by Ewald Gutenkunst
Very interesting tool!
is there any way to use it on Mac?
The first step (the sib plugin) works already well.

It's written in ruby, so...

-- Johan

[Hans Aberg]

Post by Johan Vromans

Post by Ewald Gutenkunst
Very interesting tool!
is there any way to use it on Mac?
The first step (the sib plugin) works already well.

It's written in ruby, so...

Ruby is a part of later Mac OS X; on 10.5.8:
$ which ruby
/usr/bin/ruby
$ ruby --version
ruby 1.8.6 (2008-08-11 patchlevel 287) [universal-darwin9.0]

Hans

[Ewald Gutenkunst]

Post by Hans Aberg
$ which ruby
/usr/bin/ruby
$ ruby --version
ruby 1.8.6 (2008-08-11 patchlevel 287) [universal-darwin9.0]
Hans

Thanks - with ruby it works well on my mac.

The pdf is made, but I have an error and warnings:

sib3.ly:273:47: Fehler: GUILE signalisierte einen Fehler für den hier beginnenden Ausdruck
\override VerticalAlignment #'max-stretch = #
ly:align-interface::calc-max-stretch
Warnung: Typprüfung für »max-stretch« gescheitert; Wert »#<unspecified>« muss vom Typ »number« sein


I think, such a nested constuct of slurs is not allowed in lilypond, (but possible in sib).You could use \( phrasing-slurs instead.

sib3.ly:97:54: Warnung: Legatobogen kann nicht beendet werden
<gis b>8( <gis b>8 <g b>8 <f g b>8( <e g c>8) <g c e>8 ) |%20


Perhaps, there is a problem with BarRests: these are doubled in a voice and therefor not in time with the other Staffs:

\relative c''{\oneVoice r8 |%1
R1*3/4R1*3/4 |%2
R1*3/4R1*3/4 |%3
R1*3/4R1*3/4 |%4
r4 r4 r8 cis8 |%5



\Ewald

[Kirill]

Ruby is a part of later Mac OS X; on 10.5.8: $ which ruby /usr/bin/ruby $ ruby

--version ruby 1.8.6 (2008-08-11 patchlevel 287) [universal-darwin9.0] Hans

Thanks - with ruby it works well on my mac.

Ewald,

Thank you for your report!
Could you please send me a (short fragment of) the .XML that causes the error.

Best,

Kirill Sidorov

[Kirill]

Post by Ewald Gutenkunst
Thanks - with ruby it works well on my mac.

Dear Ewald,

You report really helped. There was indeed a bug with phrasing
slurs -- fixed now.

The problem with double bar rests is more of a Sibelius bug (very rare),
whereby occasionally Sibelius would report more than one bar rest per
bar (but show only one).
Thanks to your noticing this rare eventuality, this has been fixed and
sib2ly now issues a warning in such cases and proceeds to ignore the extra
bar rests.

Sibelius has another "feature": very rarely the displayed key signature
mismatches the reported key signature (this happened in your example).
In such cases, the only option is to set the key signature again.

With these bugs fixed, your Schubert snippet looks like this:
Loading Image...

The fixes will be uploaded shortly.

[Kirill]

Post by Ewald Gutenkunst
Hi,
Very interesting tool!
is there any way to use it on Mac?
The first step (the sib plugin) works already well.
~Ewald

Yes, it should work on all platforms provided you have a Ruby interpreter.
I will update the instructions shortly.

[Kirill]

Post by Ewald Gutenkunst
Hi,
Very interesting tool!
is there any way to use it on Mac?
The first step (the sib plugin) works already well.
~Ewald

I have uploaded the SIB2LY suite in form of Ruby scripts.
These should be useable on any platform that is capable of running Ruby.
I am very curious to know if it runs ok on Macs and with Sibeliuses (Sibelii?!)
earlier than 6.

The usage is almost the same: ruby sib2ly.rb filename.xml

Please make sure you read the "Notes on using Ruby scripts" under "Usage".
The link to the Ruby source distribution is found on the homepage:

http://www.sidorefa.com/sib2ly/

Best,

Kirill Sidorov

[Johan Vromans]

Post by Kirill
http://www.sidorefa.com/sib2ly/

Must I conclude that this, too, requires Sibelius 6 or higher?

-- Johan

[Kirill]

Post by Johan Vromans

Post by Kirill
http://www.sidorefa.com/sib2ly/

Must I conclude that this, too, requires Sibelius 6 or higher?

I have no way of testing if it works for Sibelius earlier than 5.
However, I see no reason why it shouldn't work with Sibelius 5.
I have only tested it with Sibelius 6, though.

Try running the dump plugin in your earlier version of Sibelius. If it doesn't
throw an error, than in principle the second phase should be fine too.

[Johan Vromans]

Post by Kirill

Post by Johan Vromans
Must I conclude that this, too, requires Sibelius 6 or higher?

I have no way of testing if it works for Sibelius earlier than 5.
However, I see no reason why it shouldn't work with Sibelius 5.
I have only tested it with Sibelius 6, though.
Try running the dump plugin in your earlier version of Sibelius. If
it doesn't throw an error, than in principle the second phase should
be fine too.

I wrote the initial message since it does thow an error :(

Exporters/sib2lydump:processStaff:6-Cannot get field NumStaveLines

If I fake this, next is:

Exporters/sib2lydump:processStaff:6-Cannot get field HasBracket

If I fake this as well the dump seems to complete okay.

I can run the converter on Fedora Linux after installing the
appropriate ruby packages.

Two problems:

- For a particular score, it generates parts "sopr", "[Soprano]" and
"altII".
[Soprano] is an invalid name for (my version of) LilyPond.

- All staffs come out as RhythmicStaffs. After changing these to
ordinary Staffs LilyPond produces a decent score.
No lyrics and chords, though.

This is really very good stuff!

-- Johan

[Kirill]

Post by Johan Vromans
This is really very good stuff!

Johan,

Thank you for your report!

1) NumStaveLines has apparently been introduced only in Sib6. Using the value of
NumStaveLines the interpreter decides whether the staff is a normal staff or a
rythmic staff (hence all your staves are RhythmicStaffs). I guess I will have to
detect it differently for Sib5.
2) [] are of course invalid characters. This is easy to fix, though. The
converter tries to generate human-readable .ly source; in particular, it tries
to assigning meaningful names to identifiers by taking instrument names (as they
appear in the score) and generate identifiers by stripping illegal characters
from them; I forgot to include []'s into the illegal list.
3) Lyrics will be handled in the upcoming release; number 1 on my to-do list.
4) Chords ARE produced by the converter, at least from Sib6 scores.
If you are not seeing any chords, this is a critical bug. Could you please email
me a (short fragment of) .XML in which there supposed to be chords but none are
converted to .ly?

[Johan Vromans]

Post by Kirill
1) NumStaveLines has apparently been introduced only in Sib6. Using
the value of NumStaveLines the interpreter decides whether the staff
is a normal staff or a rythmic staff (hence all your staves are
RhythmicStaffs). I guess I will have to detect it differently for
Sib5.

For the time being, I'll just set it to 5.

Post by Kirill
3) Lyrics will be handled in the upcoming release; number 1 on my to-do list.

Great!

Post by Kirill
4) Chords ARE produced by the converter, at least from Sib6 scores.

There are chords in the .xml but not in the .ly.

Post by Kirill
If you are not seeing any chords, this is a critical bug. Could you
please email me a (short fragment of) .XML in which there supposed
to be chords but none are converted to .ly?

Done (via private email).

Thanks for the nifty tool!
The niftiest feature is probably that everything you want/need to
tweak can be done with convenient tools *after* making the dump.

-- Johan

[Kirill]

Hi all,

I apologise for multiple posting this morning.
I replied to earlier threads where Sib->Ly conversion was discussed,
without realising it was going straight to the mailing-list to annoy everyone.
Sorry!!

Best,

Kirill Sidorov

[Carl Sorensen]

Post by Johan Vromans

Post by Kirill
4) Chords ARE produced by the converter, at least from Sib6 scores.

There are chords in the .xml but not in the .ly.

This brings up an issue that has been concerning me in this whole thread.

I believe that a good xml dump from Sibelius is a wonderful tool, and I'm
delighted that Kirill has been working on it. Thanks, Kirill!

However, it seems to me to be less than ideal to have both Kirill's xml->ly
converter and the lilypond xml2ly that Reinhold has worked on. The
"official" xml2ly script has regression tests, and is well-maintained (now,
it wasn't always), and will work with any system that has LilyPond
installed, so it doesn't need additional tools.

It seems to me that the LilyPond community would be better served by
combining efforts on xml2ly instead of duplicating that functionality.

Now obviously, we can't (and don't really want to) tell any developer or
potential developer what to do. But I would like to invite Kirill to try
out xml2ly on the Sibelius dump, and then see if his time working on xml to
ly converters might be better spent improving xml2ly. Kirill, we'd love to
have you as part of the lilypond development team.

Thanks,

Carl

[Kirill]

Post by Carl Sorensen
This brings up an issue that has been concerning me in this whole thread.
I believe that a good xml dump from Sibelius is a wonderful tool, and I'm
delighted that Kirill has been working on it. Thanks, Kirill!
However, it seems to me to be less than ideal to have both Kirill's xml->ly
converter and the lilypond xml2ly that Reinhold has worked on.

As far as I know, xml2ly serves a somewhat different purpose. It is a MusicXML
to LilyPond converter, which is a very useful tool to have on its own, of course.
However, in order to convert Sibelius scores to LilyPond using xml2ly, one would
first need to translate from Sibelius to MusicXML before finally running xml2ly.

To the best of my knowledge, there is no one-to-one correspondence between the
representation of scores in Sibelius and the representation of scores in the
language of MusicXML; therefore the first step, Sibelius->MusicXML, would
already incur losses (I call it translation, rather than conversion). Moreover,
there is no one-to-one correspondence between MusicXML and the LilyPond syntax,
therefore the second phase of translation, MusicXML->LilyPond would introduce
further inaccuracies.

The best tool I have seen so far to generate MusicXML from Sibelius scores is
Dolet 1. It is frankly awful and costs $99.95 (
http://store.recordare.com/doletsib1.html ) The quality of the generated
MusicXML is such that it burns my eyes. (The Finale version is, perhaps, better
-- so, xml2ly might be more useful to Finale users)

To make things work via xml2ly, one would need to write:
1) A simple plugin for Sibelius to dump the score in some format preserving 100%
of what can be programmatically accessed using Sibelius ManuScript language.
Say, dump all the score elements into an .XML
2) Write an interpreter to translate this into MusicXML (losses begin here).
3) Then run, with further losses, xml2ly.

Seeing how things are, I decided to avoid the intermediate step 2, hence SIB2LY.

[Bertalan Fodor (LilyPondTool)]

Post by Kirill
1) A simple plugin for Sibelius to dump the score in some format preserving 100%
of what can be programmatically accessed using Sibelius ManuScript language.
Say, dump all the score elements into an .XML
2) Write an interpreter to translate this into MusicXML (losses begin here).
3) Then run, with further losses, xml2ly.
Seeing how things are, I decided to avoid the intermediate step 2, hence SIB2LY.

But why don't you dump directly to lilypond? Because Sibelius plugin
interface is not so good as writing in Ruby?

Bert

[Kirill]

Post by Bertalan Fodor (LilyPondTool)
But why don't you dump directly to lilypond? Because Sibelius plugin
interface is not so good as writing in Ruby?
Bert

That was my first naive attempt, to do all the work in a Sibelius plugin.
(That would have made things 100% portable, I thought)

However, it turned out later, translation from Sibelius to LilyPond is more
of an art than engineering, since there is no one-to-one correspondence
between the teo representations of music.
A very intelligent translator, not a mere converter, is required to get
decent results.
Initially (some may remember my earlier posts from 2006) I have tried to
implement all the translation intelligence in a Sibelius plugin (it was
fairly functional, and I still have it lying around but have long abandoned it).
When it grew to certain size it became simply impossible to maintain. There are
no development tools available in Sibelius, not for debugging,
not for anything.
Plus, it was ultra-slow for large score (Sibelius ManuScript has a very
slow interpreter).

Hence I decided to outsource all the logic to an external script.
At the time I was impressed by Ruby, so...

As far as users have reported, Ruby comes preinstalled on Mac OS X.
Most Linux users would have Ruby also, or can obtain it with a single
apt-get/emerge command.
For Windows users who don't want to install Ruby, I'm packaging the whole
thing into a standalone .EXE
So, I'm not too concerned about external dependencies.

Additionally, it agrees with the Unix philosophy of having small
independent tools to solve independent tasks.
As Johan mentioned,

Post by Bertalan Fodor (LilyPondTool)
The niftiest feature is probably that everything you want/need to
tweak can be done with convenient tools *after* making the dump.

At the moment, I think I will continue working in this direction (dumb plugin
for lossless export + intelligent scripts for translation).
I might consider writing a different output mechanism (but leaving the
translation logic intact) to produce MusicXML instead of immediately
producing .ly, but I am sceptical if it is needed.

Carl,
I promise I will maintain the project better, and will follow more robust
testing procedures (absolutely essential for this kind of tool).
I apologise for the temporary bugs, but the project is very-very young.
I'm sure in a few weeks it will be very useable.

[Johan Vromans]

Post by Carl Sorensen
However, it seems to me to be less than ideal to have both Kirill's
xml->ly converter and the lilypond xml2ly that Reinhold has worked
on.

Well, though they both have xml in their names, they process a
fundamentally different xml. LP's XML converter handles MusicXML while
Kirill's dumper produces an XML representation of the internal data of
Sibelius.

I doubt whether it is easy to change the dumper to produce MusicXML.
Dolet charges $200 for the MusicXML plugin for Sibelius, apparently
with reason.

Post by Carl Sorensen
Kirill, we'd love to have you as part of the lilypond development
team.

As far as I'm concerned, he already is. He's just working on different
things :)

But maybe it is feasible to translate Kirill's XML into MusicXML using
nifty XSLT? Then we have best of at least two worlds.

-- Johan

[Kieren MacMillan]

Hi all,

Post by Johan Vromans
But maybe it is feasible to translate Kirill's XML into MusicXML using
nifty XSLT? Then we have best of at least two worlds.

My XSLT-fu is pretty strong -- is this something that people will really use?

Cheers,
Kieren.

[Johan Vromans]

Post by Kieren MacMillan

Post by Johan Vromans
But maybe it is feasible to translate Kirill's XML into MusicXML using
nifty XSLT? Then we have best of at least two worlds.

My XSLT-fu is pretty strong -- is this something that people will really use?

Do you want an honest answer?

Not me.

My personal interest is to get minimal though sufficient information
from Sibelius to re-create the score in LilyPond with little effort
and --most important-- avoiding transcription errors.
That means: I need to have staffs (notes, rests, ties, slurs), lyrics,
and chords.

I have templates for the scores I want to make, and I cut/paste the
relevant parts into these templates. The rest is handwork.

-- Johan

[Carl Sorensen]

Post by Kirill
Carl,
I promise I will maintain the project better, and will follow more robust
testing procedures (absolutely essential for this kind of tool).
I apologise for the temporary bugs, but the project is very-very young.
I'm sure in a few weeks it will be very useable.

No problem.

My misunderstanding was with the xml vs. MusicXML difference.

If your exporter doesn't write MusicXML, then all of my comments are
irrelevant.

Please do go ahead; anything that can help export Sibelius scores is a good
deal!

Thanks,

Carl

[Kirill]

Update for sib2ly released (v1.01, 2 Feb 2010)

* Chord symbols now supported.
* sib2lydump.plg updated to work with Sibelius 5.
* Fixed a serious bug with phrasing slurs.
* Odd cases like multiple BarRests per bar are handled more gracefuly.
* Various other bug fixes.

Updated version can be downloaded from:

http://www.sidorefa.com/sib2ly

I am very grateful to all those who are helping with bug reports.

NOTE: If you are downloading the v1.01, you MUST update
sib2lydump.plg as well. There have been subtle but important
changes there.

Johan,
When you reported that chords do not work, I misunderstood you at first.
The chords as in <c e g> of course always worked.
The chord *symbols* as in "A#7" or "Bbsus4" are now implemented in version 1.01.
The example you sent me translates fine, so do chord symbols in the
example scores that come with Sibelius 5 (but see below).

In general:
1) Chord symbols must be typeset using "text.staff.space.chordsymbol"
style in Sibelius (the default).
2) Chords that match the following regexp, work:
^([A-H])(b|#|)(aug|maj|dim|sus2|sys4|sus|ma|mi|m|-|) <continued...>
(\^|[2-7]|9|11|13|)(\/[A-H](b|#|))?$
This should take care of the most commonly used ones. More complex ones
will be dealt with in the future. Others generate a warning and are,
for now, ignored.

Incidentally, what does F#m7(4) mean and how do we typeset (4) in lily?

[Johan Vromans]

Post by Kirill
Update for sib2ly released (v1.01, 2 Feb 2010)
* Chord symbols now supported.

Confirmed.

Post by Kirill
* sib2lydump.plg updated to work with Sibelius 5.

Confirmed.

I'm very impressed!

Post by Kirill
Johan,
When you reported that chords do not work, I misunderstood you at first.
The chords as in <c e g> of course always worked.

Oops, my fault. Being a guitar player, "chord" more naturally refers
to "C" than <c e g>.

Post by Kirill
The chord *symbols* as in "A#7" or "Bbsus4" are now implemented in
version 1.01. The example you sent me translates fine, so do chord
symbols in the example scores that come with Sibelius 5 (but see
below).

What I get now is that chord symbols come out twice. Once as a chord
symbol and once as a markup. E.g.,

<Bar BarNumber="2" Length="1024" >
<Text position="0" voicenumber="1" dx="24" dy="144" hidden="false"
Text="Bm7" StyleId="text.staff.space.chordsymbol"
TextWithFormattingAsString="Bm7" StyleAsText="Chord symbol"

Post by Kirill
</Text>

...
<Text position="512" voicenumber="1" dx="24" dy="144" hidden="false"
Text="E7" StyleId="text.staff.space.chordsymbol"
TextWithFormattingAsString="E7" StyleAsText="Chord symbol"

Post by Kirill
</Text>

produces

Soprano = {
...
r4^\markup {"Bm7"} e8^\markup {"E7"} fis8 gis8 a8 b8 c8 |%2

and

SopranoChords = {
\chords
...
b1*1/4:m7 e1*3/4:7 |%2

Post by Kirill
Incidentally, what does F#m7(4) mean

Most likely that a piano player tries to write guitar chords :)

Thanks, thanks, thanks (and keep going!)

-- Johan

[Michael Good]

Hi Kirill,

This is an interesting approach to Sibelius to LilyPond conversion. However, I
do agree with Carl's earlier message. It seems that a more generally useful
approach would be to improve MusicXML to LilyPond conversion.

The Dolet 5 for Sibelius and Dolet 5 for Finale converters both convert nearly
as much musical information to MusicXML as is possible via the program's plug-
in interface. Dolet 1 for Sibelius is many years old and designed for much
older versions of Sibelius that did not have such extensive plug-in
capabilities.

Obviously one can do much better than Dolet 1 with Sibelius 6, but I don't
think this extends to Dolet 5. I would be interested in hearing what you can
convert directly to your XML format from your plug-in that is not being
exported with the Dolet 5 for Sibelius plug-in. If it's available from the
Sibelius 6.1 plug-in interface, Dolet 5 should export it to the MusicXML file.

There will definitely be lossiness going from Sibelius to an intermediate
format, and from that intermediate format to LilyPond. But I think the
lossiness will be minimized if that intermediate format is MusicXML produced
by our latest Dolet plug-ins. The focus of the development could then be on
reducing the lossiness of MusicXML to LilyPond conversions. That would make
things work better for all programs - commercial programs like Finale or
Sibelius, or free programs like MuseScore.

Best regards,

Michael Good
Recordare LLC

[Kirill]

Post by Michael Good
Obviously one can do much better than Dolet 1 with Sibelius 6, but I don't
think this extends to Dolet 5.

Hi Michael,

Granted, your Dolet 5 is perhaps much more advanced than the earlier
versions. I'm sure you are doing a wonderful job there, and a very
useful one too.
I haven't tried Dolet 5 yet, though, as it costs $199.

However, I am such a militant believer in Free Software, that even if I
*did* start working on MusicXML->LilyPond converter, I would be
ethically compelled to first produce a free and open source exporter
from Sibelius to MusicXML, essentially duplicating the functionality of
Dolet and intruding on your territory. I do not want to do that.


Best,

Kirill Sidorov

[Martin Tarenskeen]

Post by Michael Good
There will definitely be lossiness going from Sibelius to an intermediate
format, and from that intermediate format to LilyPond. But I think the
lossiness will be minimized if that intermediate format is MusicXML produced
by our latest Dolet plug-ins. The focus of the development could then be on
reducing the lossiness of MusicXML to LilyPond conversions. That would make
things work better for all programs - commercial programs like Finale or
Sibelius, or free programs like MuseScore.

It would make things even much better if the Dolet plugins would be free
or at least cost much less than they do now. Then more people will start
using it, more bugs will be revealed, and feature requests will be
reported, more bugs will be fixed, and development speed will be increased
drastically. For example, how many people in this mailinglist actually
bought the Dolet plugin for Sibelius ?

Everyone has his own preferred music scoring tool. For me that is
Lilypond, for others this may be Sibelius, or Finale, or MuseScore.
MusicXML was and still is a great idea aiming to provide a way to exchange
scores between all these tools. I just don't hope MusicXML will be like
Esperanto: The Universal Language that practically no one really uses.

[Martin Tarenskeen]

Everyone will agree having not only musicxml2ly but also a Lilypond to
MusicXML converter would be cool.

I remember a similar discussion some time ago in the Mup mailing list. The
Mup developers from Arkkra Enterprises were saying it would probably be
less work to add musicxml export as an extra output option inside mup,
while a musicxml-to-mup converter would be easier to realize with a
separate application.

Could this also be true for Lilypond ? Would it be better/easier to have a
--musicxml output option ( just like --png --ps and --pdf ) instead of a
separate application that has to be written from scratch ? Maybe then the
conversion can re-use some of the lilypond parsing code that is already
available ?

I'm not a programmer ( or at least not a very good one ;-) ) but wanted to
post the suggestion anyway. Maybe someone will be inspired to make a start
with this difficult task ... :-)

[Graham Percival]

Post by Martin Tarenskeen
Could this also be true for Lilypond ? Would it be better/easier to have
a --musicxml output option ( just like --png --ps and --pdf ) instead of
a separate application that has to be written from scratch ? Maybe then
the conversion can re-use some of the lilypond parsing code that is
already available ?

Yes. We already know that. See the discussion on the issue
tracker. LilyPond developers have already examined the issue and
posted a roadmap of how a programmer would go about writing this.

The problem is that nobody is interested in doing it.

- Graham

[Reinhold Kainhofer]

Post by Martin Tarenskeen
Everyone will agree having not only musicxml2ly but also a Lilypond to
MusicXML converter would be cool.

Absolutely. The only problem is who will develop it? I don't have the time for
such a task, but rather create some more Urtext editions for my soon-to-be
founded music publishing house.

Post by Martin Tarenskeen
Could this also be true for Lilypond ? Would it be better/easier to have a
--musicxml output option ( just like --png --ps and --pdf ) instead of a
separate application that has to be written from scratch ?

Absolutely. If you want to write a separate ly->MusicXML application,you
basically have to duplicate how lilypond works.

Post by Martin Tarenskeen
Maybe then the
conversion can re-use some of the lilypond parsing code that is already
available ?

Of course. The only proble is that the lilypond structure is not ideally
suited for full MusicXML 2.0 export:
All other graphical output formats (eps, ps, pdf, png, svg) simply export some
graphical objects with a fixed position on a page, so at that stage the musical
information is no longer available, so MusicXML export has to hook in earlier.

The pure musical structure can probably be easily extracted from the music
stream (e.g. by an engraver, listening to all kinds of events), but at that
stage the page layout has not been done, so the great layout of lilypond
scores could not be exported directly.

On the other hand, the final graphical objects don't have any link to the music
object that caused them, so one would also have to add such a link to the
grobs.

The MusicXML export would then work in two stages in a lilypond run:
1) In an engraver create the xml tree for the pure musical content
2) at the same time, also listen to created graphical objects and add a
pointer to the corresponding xml node
3) After the layout is done, a MusicXML backend goes through the graphical
object and exports all positioning information (most of which is new in
MusicXML 2.0) to their linked xml nodes.

However, there is also a practical problem: How do you check the quality of
your export? There are so many things inthe MusicXML "specification" that are
left unclear, and the typical advice on the MusicXML mailing list is "Just
check what Finale does"... Unfortunately, the company behind Finale is not
interested at all in MusicXML functionality in lilypond (when I started
wokring on musicxml2ly, I asked them for a copy of finale or so, but their
response was that they are not interested and that I could buy a copy of finale
at their homepage). So, if you want to do serious work on MusicXML, you'll
have to buy a copy of finale for several hundred $$$ (well, Michael Good
suggested to buy one of the stripped-down version for "only" 100$ ......
However, these versions don't support MusicXML 2.0, so they don t really
help).

Cheers,
Reinhold

[Graham Percival]

Post by Reinhold Kainhofer
However, there is also a practical problem: How do you check the
quality of your export? There are so many things in the MusicXML
"specification" that are left unclear, and the typical advice on
the MusicXML mailing list is "Just check what Finale does"...

That's rubbish. Remind anybody of microsoft's "office docuement
standard" ?

MusicXML isn't a standard at all. If you have to say "umm, dunno,
look at what this other piece of software does", it's not a
standard. Period.
(unless you buy into microsoft's "whatever internet explorer does
is the standard" definition of a standard, which totally warps the
original intention of the term)

Post by Reinhold Kainhofer
So, if you want to do serious work on MusicXML, you'll have to
buy a copy of finale for several hundred $$$ (well, Michael Good
suggested to buy one of the stripped-down version for "only"
100$ ...... However, these versions don't support MusicXML 2.0,
so they don t really help).

Wow! That's pretty sleazy.

Is their company really *that* bad that they need to ask
open-source developers to buy several-hundred-dollar software just
to work on interaction between the two?


I see why nobody wants to work on musicxml export. I mean, if
even finale's company isn't interested in "playing nice", then why
on earth should other programmers jump through hoops to work on
musicxml?!?

Cheers,
- Graham

[Reinhold Kainhofer]

Post by Graham Percival
MusicXML isn't a standard at all. If you have to say "umm, dunno,
look at what this other piece of software does", it's not a
standard.

Just to make things clear: It's not so bad. It's just impossible to write a
full specification for musical content, since the notation is so complex. So,
any specification for a file format for music will invariably be incomplete.

Without a reference implementation, however, it's just not possible to write
consistent support of some more obscure features, since the details are often
not explained enough in the "specification".

And it's not that Michael's answer is typically "dunno", but rather that
recordare coded their finale and sibelius plugins in the way they intended the
MusicXML format to behave... If you ask a question on the mailing list, he
usually tries to answer it quite well.

The main problem, however is that there are no development tools and no
representative test cases (this is the main reason why I started the MusicXML
test suite, which is also part of lilypond's sourcetree).

Unfortunately, on the MusicXML mailing list, as I said, the main advice is to
use Finale as refence. And many questions about how aspecific MusicXML snippet
is supposed to be interpreted are also answered by "it is up to the
application how that should be interpreted, since some applications don't
support some particular concept". The problem with this answer is that it is
jumping over a crucial part of the explanation: What a particular snippet is
supposed to mean, regardless of whether an application supports that feature.
If you knowthe meaning behind someMusicXML code, one can always decide how
this can be transformed to/from a particular application's format. But that
vitual information is typically missing in the answers on the musicxml mailing
list...

Cheers,
Reinhold

[Martin Tarenskeen]

Personally I would already be quite happy if Lilypond would only be able
to export the most basic information like staffs, clefs, keys, notes, and
beamings. More detailed and/or complex details I would then add manually
in the software that I use to import the exported musicxml. Even that
would already be a big step forward and a time saver compared to exporting
score information via MIDI files and editing the result to get a readable
and musically correct score.

But the main problem remains: Lilypond developers are also very happy
Lilypond users, which could explain the lack of motivation to put a lot of
time and effort exporting to a format that only people who do NOT use
Lilypond really need ?

I'm a happy Lilypond user, and I don't really need MusicXML export, but it
would still be nice to have it some day in the future.

[Graham Percival]

Post by Martin Tarenskeen
But the main problem remains: Lilypond developers are also very happy
Lilypond users, which could explain the lack of motivation to put a lot
of time and effort exporting to a format that only people who do NOT use
Lilypond really need ?

That's not the problem. The problem is that you feel that
lilypond developers are your personal slaves.

LilyPond is open source. That means that you can see the workings
of the software. LilyPond is Free software. That means that you
can legally improve and distribute the software. Neither of those
means that people who have improved LilyPond in the past are under
ANY kind of obligation to you.

You're blaming lilypond developer for not doing a job that *you*
are not willing to do. That's not on.

Post by Martin Tarenskeen
I'm a happy Lilypond user, and I don't really need MusicXML export, but
it would still be nice to have it some day in the future.

If you want something done, then start working on it. Since
you're (apparently) not willing to work on this, you can hardly
blame us for having the same opinion on the matter.

- Graham

[Martin Tarenskeen]

Post by Graham Percival

Post by Martin Tarenskeen
But the main problem remains: Lilypond developers are also very happy
Lilypond users, which could explain the lack of motivation to put a lot
of time and effort exporting to a format that only people who do NOT use
Lilypond really need ?

That's not the problem. The problem is that you feel that
lilypond developers are your personal slaves.

Oh no! That's not what I was trying to say! I just mean that Lilypond is
such a great program - and implicitly I mean that the Lilypond developers
are great guys who put a lot of time and effort in it. And I meant to say
that with such a great program, the need to export to another format is
not very urgent.

Post by Graham Percival
You're blaming lilypond developer for not doing a job that *you*
are not willing to do. That's not on.

Again, my intention was NOT to blame anyone. My apologees if it sounded
like that.

Post by Graham Percival
If you want something done, then start working on it. Since
you're (apparently) not willing to work on this, you can hardly
blame us for having the same opinion on the matter.

I do realize users and developers in such an open-source project have a
shared responsibility. I have shown I am willing to help where I can - I
contributed to the Dutch translation, and contributed some modest lines in
the C sources to improve MIDI output for example. Also I did some more or
less succesfull attempts to improve the midi2ly utility. I might continue
these attempts after I did some more studying on Python programming.

But allow me to dream about complete MusicXML support from/to Lilypond
somewhere in the future :-)

[Jack Cooper]

Just out of curiosity...

Assuming for the moment that development interest might materialize if there were
money involved, approximately how much sponsorship dollars would be involved to
tackle this?

Jack

--
Jack Cooper, BerLen Music
www.berlenmusic.com
www.jack-cooper.com



----- Original Message ----

Sent: Mon, February 8, 2010 6:51:26 AM
Subject: Re: Lilypond to MusicXML (was: Re: New Sibelius to LilyPond conversion suite)

Post by Martin Tarenskeen
But the main problem remains: Lilypond developers are also very happy
Lilypond users, which could explain the lack of motivation to put a lot
of time and effort exporting to a format that only people who do NOT use
Lilypond really need ?

....

LilyPond is open source. That means that you can see the workings
of the software. LilyPond is Free software. That means that you
can legally improve and distribute the software. Neither of those
means that people who have improved LilyPond in the past are under
ANY kind of obligation to you.
You're blaming lilypond developer for not doing a job that *you*
are not willing to do. That's not on.

....

- Graham
_______________________________________________
lilypond-user mailing list
http://lists.gnu.org/mailman/listinfo/lilypond-user

[Stefan Thomas]

Dear community,
unfortunately, with my very old version of sibelius I can't run the plugin.
Ins't it possible to make a conversion into lilypond, without having
sibelius?
This would be a great thing!

[Michael Good]

Hi Martin and Kirill,

Martin, MusicXML is pervasive in the music preparation business, whether it is
preparation for print music, film, or TV. It is not yet used very widely as a
consumer format for sheet music, but this is starting to change. You can see
the list of sites offering MusicXML scores at:

http://www.recordare.com/xml/sites.html

The list of software supporting the MusicXML format - over 120 programs so
far - is available at:

http://www.recordare.com/xml/software.html

MusicXML itself is a free format, and as you can see is used widely in both
commercial and free software. We support developers in all areas. But to stay
in business, and keep developing the MusicXML format and its related software,
we have to charge for our plug-ins.

Kirill, I appreciate your position regarding free software, and your
frankness. One thing that I hope you will consider doing - once your current
Sibelius plug-in is completed - is to work on a converter from LilyPond to the
MusicXML format.

LilyPond may be free software, but it has more lock-in than nearly any current
commercial music notation product. Scores can only be imported into LilyPond
format. They can never be moved effectively from LilyPond to another notation
program.

MusicXML was invented to get rid of this type of proprietary format lock-in.
Most commercial products added MusicXML export years ago, either directly or
via plug-ins. It is unfortunate that LilyPond users cannot take advantage of
the same data freedom that Finale and Sibelius users can.

We hope in the future that LilyPond will be as supportive of the open MusicXML
format as most commercial software. If there is already a converter from
LilyPond to MusicXML that I have missed, please let me know. If my objections
are out of date, that would be excellent news!

Best regards,

Michael Good
Recordare LLC

[Patrick McCarty]

Post by Michael Good
LilyPond may be free software, but it has more lock-in than nearly any current
commercial music notation product. Scores can only be imported into LilyPond
format. They can never be moved effectively from LilyPond to another notation
program.
MusicXML was invented to get rid of this type of proprietary format lock-in.
Most commercial products added MusicXML export years ago, either directly or
via plug-ins. It is unfortunate that LilyPond users cannot take advantage of
the same data freedom that Finale and Sibelius users can.
We hope in the future that LilyPond will be as supportive of the open MusicXML
format as most commercial software. If there is already a converter from
LilyPond to MusicXML that I have missed, please let me know. If my objections
are out of date, that would be excellent news!

Hi,

There have been some discussions in the past about developing a
MusicXML backend for LilyPond, but so far (AFAIK) no one has started
working on it or knows how it should be implemented.

We have an enhancement request on our bug tracker for this issue:

http://code.google.com/p/lilypond/issues/detail?id=665

Thanks,
Patrick

[Graham Percival]

Post by Patrick McCarty

Post by Michael Good
It is unfortunate that LilyPond users cannot take advantage of
the same data freedom that Finale and Sibelius users can.

I agree that it is unfortunate that nobody has sent (GPL-licensed)
patches to address this. But hey, if nobody's interested in
musicXML, that's not our problem.

Post by Patrick McCarty
There have been some discussions in the past about developing a
MusicXML backend for LilyPond, but so far (AFAIK) no one has started
working on it or knows how it should be implemented.

I wouldn't say that. Han-Wen and Reinhold seem to have a good
handle on the issue. The problem is that nobody wants to work on
it.

*shrug*

LilyPond is open source, and it's never been easier to get started
with lilypond development. Whether or not somebody sends patches
is simply a matter of how interested they are.

Cheers,
- Graham

[Karl Hammar]

Michael Good:
...

Post by Michael Good
LilyPond may be free software, but it has more lock-in than nearly any current
commercial music notation product.

...

You might be able to claim that and get away with it, but look at [1]:

In economics, vendor lock-in, also known as proprietary lock-in,
or customer lock-in, makes a customer dependent on a vendor for
products and services, unable to use another vendor without
substantial switching costs. Lock-in costs which create barriers to
market entry may result in antitrust action against a monopoly.

Since with lilypond, there is no vendor and no services, there are
no lock-in possible.

On the technical side, lilypond has no lock-in since there are no
technical, or documentation or whatever barriers set up by the
lilypond community, to convert a lilypond input file to whatever you
wish.

Regards,
/Karl

[1] http://en.wikipedia.org/wiki/Vendor_lock-in

-----------------------------------------------------------------------
Karl Hammar Aspö Data ***@aspodata.se
Lilla Aspö 148 Networks
S-742 94 Östhammar [phone] Computers
Sweden [phone] Consulting
-----------------------------------------------------------------------

[Carl Sorensen]

Post by Michael Good
LilyPond may be free software, but it has more lock-in than nearly any current
commercial music notation product. Scores can only be imported into LilyPond
format. They can never be moved effectively from LilyPond to another notation
program.

This is true. But why should the LilyPond team develop converters from a
free format to a proprietary format? There's no financial incentive to do
so, and there's no moral incentive to do so.

Post by Michael Good
MusicXML was invented to get rid of this type of proprietary format lock-in.

LilyPond is not a proprietary format lock-in. It may be a lock-in, but it
is a lock-in to an open, free format.

It is also true that MusicXML is an open format. So I can see some moral
incentive to move from LilyPond to MusicXML. But there's not enough
developer resource on the LilyPond team to do all that we want to do with
LilyPond, let alone worry about getting things out of LilyPond and into
other formats.

The interest for doing this would come most likely from somebody who wants
to convert some kind of LilyPond archive (e.g. Mutopia) into another format.
Anybody who had that interest would certainly receive help from the LilyPond
development team.

Post by Michael Good
Most commercial products added MusicXML export years ago, either directly or
via plug-ins. It is unfortunate that LilyPond users cannot take advantage of
the same data freedom that Finale and Sibelius users can.

It is true that LilyPond users do not currently have access to a tool that
converts LilyPond input data files to MusicXML. But LilyPond users do have
access to all of the LilyPond sources, so they can modify the program at
will to meet their needs.

And everybody in the world has all the information available about LilyPond,
including source code, data formats, parsing trees, etc. So although a
MusicXML exporter has not been developed, the only barrier to doing so is
interest. Nobody has wanted to do so.

Post by Michael Good
We hope in the future that LilyPond will be as supportive of the open MusicXML
format as most commercial software. If there is already a converter from
LilyPond to MusicXML that I have missed, please let me know. If my objections
are out of date, that would be excellent news!

Would you consider it a net gain for the world if LilyPond were commercial
software, and used the revenues from sales to support the development of a
LilyPond to MusicXML converter? Personally, I would not. I would much
rather have data lock-in with an excellent open source engraving software
than have no open-source engraver but the ability to readily move data among
different proprietary music notation systems. Given my thoughts on this
matter, it's not likely that I'll take my limited LilyPond development time
and use it to develop a MusicXML converter. Instead, I'll try to improve
LilyPond itself.

Thanks for your comments, and thanks for participating in the discussion.

Carl

[Michael Good]

You can get started with a reasonable MusicXML import project for as little as
a US $10 investment in Finale NotePad. For a MusicXML export project, just
download a free Finale demo. Recordare does not make any money from either the
sales or downloads of MakeMusic products.

The Finale products offer the closest thing that MusicXML has to a reference
implementation. They may not be free or open source, but since you do not need
the full versions of the software, it is far from a major investment. If the
open source community wants a free and open source reference implementation,
the MuseScore, Zong, and LilyPond projects would be natural candidates for
this work.

Checking the musical results of a MusicXML exporter is neither difficult nor
costly. For best results it would be good to check with a free download of the
Sibelius demo as well as the Finale demo.

However, if you try to export LilyPond's formatting, checking the results does
become more complicated. It is only a select few programs like the Legato
Sheet Music Viewer (fka musicRAIN) that interpret and display full MusicXML
formatting, including note spacing. Those applications are indeed expensive.
But perhaps Legato would be interested in cooperating in this effort,
especially given the interest expressed in the issue tracker from one of their
potential customers.

Reinhold, if you feel an explanation on the MusicXML list is missing a crucial
element, please feel free to follow up! In some cases, I fear that what you
want to see from the format is at odds with MusicXML's use of selective
encoding. Selective encoding allows applications and scores to skip the export
of information that is not applicable to their needs. This design helps with
adoption of MusicXML export, but does place additional burdens on MusicXML
import.

Interchange format design is full of tradeoffs like this. We may not have made
the optimal tradeoff in every case. Given MusicXML's success, we seem to have
done well more often than not.

For those who are interested in more MusicXML discussions, feel free to join
the MusicXML mailing list. Signup is at:

http://www.recordare.com/lists#MusicXML

Reinhold has made excellent contributions to the MusicXML community with his
participation on the MusicXML list and with his test suite. We welcome more
contributions from the LilyPond community. But we would prefer it if you
refrain from calling our company "sleazy" and "bad".

Best regards,

Michael Good
Recordare LLC