A World Of Documents

Sylvain Wallez indirectly reminded me that I, too, have my 10-year anniversary in the Open Source world this year. On October 24, 2000, my first post went to the fop-dev mailing list (Apache FOP developer list), 11 months after the Apache FOP project had started.

I had started a new job back then which began with a project that should format invoices from the health sector to PostScript and print them on large-scale printers including automatic packaging. And for that XSL-FO looked like the right technology, but FOP had to learn how to use fonts other than the base 14 set (Helvetica, Times, Courier, Symbol and ZapfDingbats). Finishing some work started by someone else, my first contribution was initial support for custom Type 1 fonts.

Today, I’m probably the FOP contributor with the longest service record in the project.

I went through the hype phase (2001-2003) as well as the difficult time of the redesign after the famous 0.20.5 release which saw an almost complete exchange of the project team between 2003 and 2005. From 2005 on, we worked towards a stable FOP again that quickly surpassed the old 0.20.5 release for which soon there was almost no expertise around anymore. It’s unbelievable how many people still use 0.20.5 and say “we can’t upgrade”. Many of the problems they have are simply solved in the current release. Anyway, since this year’s 1.0 release, FOP finally doesn’t carry that undeserved taint of an 0.x software anymore. It was just a version number but not an indicator of FOP’s production readiness. The initial goal of a complete implementation of the XSL 1.0 standard was just unrealistic. There’s not even one commercial implementation on the market which achieved that goal. Many people came and went during all that time. FOP is what it is today through contributions from all of them.

Lately, AFP is a big topic where FOP took big leaps forward in the last 2-3 years. Color is also becoming more and more important as color printing gets cheaper and transpromo is gaining interest. Even in today’s world the number of pages printed still increases every year. Apache FOP is well positioned to cover the printed document as well as the electronic document. XSL is an established and accepted technology today. And Apache FOP is often the “entry drug” for people who start XSL. Many stay with it producing millions of pages each year. During the stagnation of the redesign phase, some FOP users turned to commercial implementations like RenderX or AntennaHouse. After 2005 I saw many switch back again after the major improvements because some of the commercial implementations couldn’t offer much more but had a higher TCO depending on the use case or because they offered unsatisfactory support and couldn’t influence the development direction enough. Apache FOP might face some difficult time ahead since the layout engine (the core) didn’t see considerable development lately. Some features like table markers, flow maps and auto-table layout would do FOP some good but it won’t be easy to implement.

Between 2000 and 2003, Apache FOP was part of my job and quickly became a hobby, too. I’ve developed Barcode4J (an open source barcode generator) during that time which plugs nicely into FOP. From 2004 to 2005, FOP was just a hobby as I worked part-time for a small company maintaining a model-driven application framework. In 2005 I went freelance offering commercial support, consulting and development services around Apache FOP. At times, FOP created up to 90% of my income. I did not get rich, of course, but the fun counts, too. It wasn’t always fun. Heated arguments over mostly non-features nibbled at my motivation which triggered me to reduce my involvement in FOP a bit. This year was dominated by two interesting projects that involved producing full-color PDFs, both Web-to-Print. One is Swiss Post’s DirectFactory, a web plattform where you can design and send postcards (or even chocolate wrappers now). The other was for a company that produces photo books, calendars and other gifts from within the browser. That finally got me back to a user role a bit. Before that I basically just hacked FOP without really using it myself. It’s interesting how the perspective shifts again when switching back to the user side. This is particularly important to me now that the EUR, GBP and USD are very weak against the CHF. My commercial support business gets increasingly more difficult because most of my clients are located outside Switzerland. And that means I have to shift my main focus to domestic clients but without leaving my past clients in a void.

I’d like to note something interesting in that context: In all my freelancer time since 2005, I’ve never picked up the phone to look around for projects. I was lucky that they always found me, often even at the right times. A large part of it, I’m sure, is my presence on the Apache FOP Users Mailing List. By answering questions, people recognize a recurring name at some point. When some users run into a problem they can’t solve easily, they often turn to me for a higher service level than is possible on the users mailing list. That said, I believe there is room for a good developer to get challenging tasks as part of a commercial support service for Apache FOP, especially if he’s not afraid of the complicated topic of paper layout and he’s not located in Switzerland which is currently just too expensive due to the currency weaknesses. Such an additional resource would be good for FOP.

Last year, I’ve started to develop an output management server which uses Apache FOP as its main formatter. Although a minimal version is now live working nicely for the DirectFactory, it turned out to be too big an endeavour for a one-man show besides end-user projects and the FOP commercial support. So I’ll be investing in a smaller product first that hopefully will create a little multiplicator for me which commercial support services simply can’t provide. To wrap up this admittedly lengthy post, I need to say that Open Source is a very important topic for me. When done right, it is such an effective way to pool resources and to create high-quality software. Getting rich with it (not that I need to be) is difficult, so the right way is probably somewhere in between commercial and open source products. My wish is that more users of Open Source software think about contributing something back to the many products they use. There are various ways to do that. Too often, people treat OSS as freeware which it simply isn’t.

This is something mainly for german-speaking people who are interested
in learning about document production with XSLT and XSL-FO. I’ll be
giving a two-day training on the basics of XSLT and XSL-FO during the
27./28. September 2010. The workshop is hosted and organized by Compart
Deutschland GmbH in Böblingen, Germany. More information is available on
their website (in German):
http://www.compart.com/en/seminar-workshop/events/EX-W1030en-xml-xsl-xsl-fo

Two days are quite short for that amount of information so the workshop
is quite intensive, but the feedback from past workshops was very
positive.

BTW, I’m happy to give this training (available in German or English) at
a location of your choosing (preferably within Europe). Let me know if
you are interested.

I’m currently writing a basic XSL training course which I’ll give the first time later this month. I decided to write the training materials using DocBook 5. Lots of XML examples, lots of SVG illustrations painted in Inkscape. I used the DocBook editing capability of my OxygenXML editor. Finally, I’ve customized the DocBook XSL stylesheets a bit, added syntax highlighting using XSLTHL and used Apache FOP (SVN Trunk) to create the PDF.

I must say: it was simply fun to work with this setup. DocBook is well documented. The documentation always had an option for all of my wishes. No wasted time, no crashes, no despair (I pity those who do this with MS Word). I can only recommend to do any larger documentation work like this.

With pleasure I learned today that IBM and Sun together launched http://odftoolkit.org/. It is at least partly seeded with code from Sun’s OpenOffice (namely the ODF Toolkit project). Initial components include ODFDOM (a low-level ODF API) and an ODF validator. Another very pleasant surprise is the publication of the source code under the liberal Apache License V2.0. Congratulations to the people involved!

This will make especially ODFDOM interesting for Apache FOP.  One of the missing puzzle pieces in FOP surely is an ODF output plug-in to replace the RTF format in the long run. Somehow I’ve got the feeling that we won’t have to wait all too long for that…

Two days with buzzing activity. Interesting talks. Lots of FOSS projects presenting themselves. Free food, drinks and admission. On Wednesday evening Google sponsored a social event with food, drinks and lightning talks. LiSoG and /ch/open offered delicious cocktails under free license on Thursday. Ever present was the organic FreeBeer by project21 (it comes at a price but the recipe is under the Creative Commons License Attribution Share-Alike).

It’s unbelievable what these guys put together. Sometimes I had the impression that there was more interest in the OpenExpo than in the larger TopSoft that was held right beside the OpenExpo in the same hall.

For the ASF, Andreas Hartmann presented Lenya, Christian Geisert presented OfBiz and I FOP. On Thursday, I held a presentation on FOP and I was happy to have an almost full room. Many people indicated that they didn’t know FOP, yet. So, this was a really good opportunity to do a little promotion.

I had a nice long talk with Andreas Vox from the Scribus team about layout. It was interesting to learn how Scribus does line-breaking compared to FOP.

More photos should be available on the OpenExpo site shortly along with the presentations (videos plus slides).

I’ve just noticed an interesting coincidence: There will be will be a PDF/A conference next year in Amsterdam. The event will take place April 10th and 11th at the Hilton Hotel in Amsterdam. This is at the same time as ApacheCon EU 2008 which takes place at the Moevenpick Hotel in Amsterdam from April 7th to April 11th 2008.

I guess it would be interesting to present Apache FOP at the PDF/A conference. For FOP users this could also be a reason to come to Amsterdam in that week. To a certain degree you can probably combine catching up on PDF/A with interesting discussions with FOP developers. I don’t know, yet, but hopefully Vincent gets one of his session proposals on FOP accepted which would be one more reason for FOP users to come to Amsterdam.

The first release (0.90alpha1) coming out of FOP’s redesign effort is exactly two years old today!

Between 2003-07-18 and 2005-11-22 (for more than two years), there was no release of Apache FOP. Many believed the project was dead. No wonder: the majority of the project team back then gradually went away for various reasons. There was no consensus about how the project should be continued. The “maintenance branch” constantly competed for attention with the necessary redesign effort which was started back in late 2001. The XSL standard lost its “new and shiny” status at some point which obviously didn’t help attracting a lot of creative minds with a lot of free time at their hands.

Today, XSL is a mature, if complex, technology which is widely adopted and treated as a commodity taken for granted. At least that’s often my impression. I mean, if we look at the size of the user base of Apache FOP compared to the number of the active committers, the difference is huge. Still, I’m very happy that we’ve been able to attract a sizeable group of new committers to Apache FOP once again over time.

From a personal perspective, it’s so good to see various companies and even some government institutions investing in FOP. Of course, the whole open source movement is still gaining momentum which helps here. Nowadays, I can even make my living by doing freelance work almost exclusively in the Apache XML Graphics ecosystem. My clients, in turn, get added value in various forms: new features, bugfixes, professional support with good reaction times, no vendor lock-in (independance from a single commercial provider), full access to the source code, sharing of development costs, no licensing costs etc. That’s a win-win situation for everyone involved.

I’m looking forward to the time ahead. There are still many things to do. One of the most important ones: Still not having jumped over the version 1.0 barrier harms the project: some companies don’t adopt 0.x software and others won’t invest in it. I hope we can finally, finally fix that in 2008, almost ten years after the XSL standard was started. And FOP was actually the first implementation of XSL-FO available.

Last week, we’ve conducted a user poll to get hard data for a decision to drop support for Java 1.3 for the Apache XML Graphics project (Batik & FOP). The results are quite clear: we’re safe to drop support for Java 1.3. That’s good news since it helps us concentrate on more important aspects of the project. The survey also revealed how many people are still on Java 1.4 and how many are already using Java 6.

We will now have to take a closer look at the question, if we should directly skip to Java 1.5 as some suggested. Personally, I’m a little hesitant, since I know a lot of people who are still stuck with Java 1.4 for the moment. They all want to migrate to at least 1.5, of course, but that requires careful preparation in complex environments. Better stick to Java 1.4 until at least summer 2008 when the end-of-life process at Sun ends for Java 1.4.

The W3C’s SVG working group has somewhat silently updated the SVG Print Working Draft last month. SVGP adds elements to embed multiple pages in a single SVG 1.2 file plus it adds color handling to match the additional requirements for printing. The draft seems a lot clearer and more useful now. It has been split into two parts (Primer and Language).

As I hinted in an earlier post, I feel that SVG Print may already be coming too late (although it has been in development for more 4 years now). Mars looks more interesting (a spec update is coming up, I heard). The packaging of binary resources (images, fonts etc.) in a ZIP file is very efficient and versatile while this is sort of an unsolved problem in SVG Print. The Primer hints at the use of Multiplex MIME (RFC 3391) which is possibly more difficult to handle than a ZIP archive. Over all SVG Print gives me the impression that it’s not much more than a research project by Canon’s research labs. Not that Adobe’s Mars is much more than that right now. In that regard, Microsoft’s XPS has probably evolved further. Too bad, Mars is still my favourite approach if I had to choose an alternative to PDF.

It didn’t suffice that Adobe Acrobat 7 used and expected a PDF/A namespace from an old specification draft (http://www.aiim.org/pdfa/ns/id.html). Now, it turns out that the official ISO document (ISO 19005-1:2005) had to be corrected, too. The document uses “http://www.aiim.org/pdfa/ns/id” as the PDF/A namespace. After a problem report today from a client trying out Acrobat 8 on a supposedly PDF/A-compliant file generated by Apache FOP, I found out that the PDF/A committee has published a corrigendum in which they are correcting the PDF/A namespace to “http://www.aiim.org/pdfa/ns/id/” (note the trailing slash). Furthermore, I think there’s an additional problem with metadata handling since the “Subject” is not mapped to dc:subject anymore, but to dc:description["x-default"]. Looks like I just got some more work to do…