Goodbye to Adobe FrameMaker, Hello AsciiDoctor

I am probably one of the longest time users of Adobe FrameMaker in the world. I started using it at version 2, sometime around 1990, and finished with it a few months ago. For most of this period it was the unbeatable desk top publishing solution for ‘serious’ documents and books (as opposed to magazine articles and the like). It was used by Sun Microsystems, IBM, and still I believe, HP for their technical publishing. The IMIA yearbook was historically published using it. Thousands of software and engineering companies around the world used it to publish manuals. It did a pretty good job of multi-channel publishing, at least to PDF and HTML, using WebWorks (now ePublisher), and was unbeatable for control over fonts, layout, cross references, and most other details of publishing.

Why did I stop using it?

Well. It’s still very expensive. It’s getting clunky to use. Bringing new users on-board means non-trivial training. It does DITA / XML, but many in the technical development world are losing interest in XML (I’ve always loathed it as one of IT’s worst invention, so no personal pain there). It isn’t available on Mac OSX or Linux.

Amazingly, the latest version (12) doesn’t even display properly on a modern hi-res laptop screen (Dell XPS) – like Adobe Acrobat, if you have a modern laptop with a slightly less modern second screen attached, the menus and UI in general is a complete mess. I spent some months trying to convince Adobe to look at this problem, to no avail. So much for Adobe knowing about graphics.

But the biggest issue is to do with being able to easily integrate multiple sources, including manually authored text, diagrams, UML diagrams from tools, tool-generated model / API documentation, bibliographies and so on. Today, a technical document consists of all kinds of content, which needs to be processed to create a document with seamless heading numbering, cross-references and so on, and then further processed into final output formats like HTML, PDF and DocBook. Having all the content stuck inside one tool, in either a proprietary format or else a difficult XML structure based on DITA or one of the other document XML standards, isn’t conducive to building content from multiple sources, especially where some of those sources are syntax files or excerpts.

Historically, the openEHR specifications have been published in FrameMaker, for nearly 15 years. Ten years ago and probably even by 2010, they were still the best quality technical specifications in the e-health domain, due to Frame’s publishing quality. But now, the problems listed above are starting to bite. In the end it was clear we needed to move to a new solution.

Colleagues pressed me to look at something called Asciidoc, which is one of the many Markdown kind of things around today. I wasn’t positive – I mostly hate markdown (and I’m not alone), because it’s non-standard and grossly simplifies publishing to a mindlessly childish level – headings, lists, links and code blocks. But eventually I gave in and started messing around with an implementation called AsciiDoctor.

First impressions were not good. On this page on the AsciiDoctor site are quotes like the following:

Then why, oh why, do we make it more difficult by burying the content in XML schemas like DocBook, allowing complex word processors to distract us or wasting time battling with finicky WYSIWYG editors?

Imagine if writing documentation was as simple as writing an email.

And also this:

When you’re in the writing (i.e., typing) phase, you want the words to flow onto the screen with minimal distractions and interruptions. Flow, not just time, is essential.

Most word processors excel at distracting you from writing. The result: you write less(ironic, huh?).

And:

You write an AsciiDoc document the same way you would write a normal text document. There are no markup tags or weird format notations. AsciiDoc files are designed to be viewed, edited and printed directly or translated to other presentation formats.

— Stuart Rackham, Creator of AsciiDoc

Hm. Most people who do any real document authoring and publishing with recognise the statements above as mostly nonsense. Good word processors don’t get in the way of writing. Maybe old versions of Word, or WordPerfect. Modern tools like Frame or Word 2013? Not at all. You can use templates that will make decent choices of layout, fonts, and most other formatting choices. So that’s not the problem.

Can writing a complex technical document really be just like writing an email, with no ‘weird format notations’ or tags? Not really. As soon as you want sophisticated content, like centred Figures with referenceable captions, headings that are numbered numerically in the main part of a book, but with letters in the Appendix section, and citations that are shareable across books, you’re going to need something special in the text to indicate these things.

And just consider the challenge of cross-referencing – there is no way to just write them directly into the text. Xrefs are always going to be some sort of technical markers, and tools are needed to find them and offer you the ones you need in the text you are currently working on.

So I have to admit to being quite cynical when embarking on the Asciidoctor journey. Imagine my surprise when it turned out that the tools worked very well, and enabled most of the features I have been used to from Frame for 20 years. To do this, there is sophisticated markup, numerous rules about formatting, and many macros. All within plain text. To get an idea of what a book looks like in Asciidoctor, have a look around this Git repo. Here’s an example of a document created as an Asciidoctor book. This document was created from multiple files (here and here), some of which are tool-generated from a UML tool.

If you have a careful look, you’ll see that there is in fact quite a lot of markup and macros:

Plus a ton of style-sheets for HTML and PDF output stages. No, this isn’t simple at all. Have a look at this user manual to get an idea of how simple it isn’t (lucky this manual is generally excellent isn’t it? You don’t need a manual this big to tell you how to write an email do you?)

The real power of Asciidoctor is that it supports the integration of multiple sources of content into multiple publishing targets. You can include Antlr4 files, generated UML diagrams and class definition tables and a lot of other things. Syntax, either inline or externally referenced will be colourised if there is a lexer for the syntax.

The tools are not perfect, but they are very good and getting better. PDF generation is somewhat weak right now, but the Asciidoctor developers are a) very responsive and b) very knowledgeable about ‘real publishing’, and working hard on it. I expect big things from the next release (at the time of writing we are using 1.5.2).

This set of graphics provides an idea of our current workflow; the README of this Git repo describes our experience using Asciidoctor for openEHR.

We have now converted all the openEHR specification documents to Asciidoctor format. What’s better now:

  • as a text format, Asciidoctor is treated by version control systems in a comprehensible way to human authors
  • syntax fragments are now properly and automatically colourised, a huge improvement over FrameMaker
  • separate grammar files can be included directly, and they get colourised as well. Very nice.
  • large books can be cut up into multiple files, in various directories
  • ‘variables’ can be defined and used, more easily than FrameMaker
  • including external files, including those generated by tools is much easier than with Word processor tools
  • did I mention the support? Try getting support from Adobe some time to really appreciate the quality of the Asciidoctor team.

What’s not so good:

  • defining and using cross-references is quite primitive right now. This needs tool support; hopefully someone will do some work on it.
  • Referenceable ‘markers’ can’t be defined like in FrameMaker yet, but I believe are in the works.
  • PDF output formatting is very rough, and file size is far from optimal. This is one area where it’s hard to compete with Adobe of course…
  • Management of citations is manual, and there is no way to have a shared bibliography that is processed into the correct bibliography for each using document, i.e. just the list of items actually referenced in the document. But this doesn’t come with Frame either…

A ‘neutral’ point is that editing is now done in the source form, so you need two screens to keep track of what the output looks like. This is a blessing and a curse. The upside is that you can use a text editor of choice, like Brackets, which knows about multiple files and even multiple books, and can do very nice regex and normal search and replace across the whole lot. The downside is… you’re hacking around in a lot of ‘markdown’, and always trying to remember the format rules for URLs, bullet lists, code blocks and so on – since all this varies across the numerous MarkDown variants. No, Markdown is not fun. But it’s liveable, and they say it will be standardised. Progressively generating output HTML for review while authoring is easy enough, especially with Brackets. So all in all, the authoring experience is tolerable.

I like the results we have for openEHR, and I that’s coming from an old-style stickler for quality publishing. I’m very happy we made this decision (certain colleagues and friends will be laughing behind their hands right now).

I now see Asciidoctor as an excellent proposition for general publishing of complex technical works, and would recommend it even for a ‘proper’ technical book. It’s been an interesting journey, and the future looks good.

Advertisements

About wolandscat

I currently work in e-health, and am senior architect of the openEHR.org specifications, designed for semantic interoperability of health information. I also designed the Archetype formalism and model used in openEHR. Outside of work, I am interested in guitar, travel, and philosophy.
This entry was posted in Computing, Health Informatics, openehr and tagged , , . Bookmark the permalink.

18 Responses to Goodbye to Adobe FrameMaker, Hello AsciiDoctor

  1. bjornna says:

    This is a really great summary on different aspects to consider when choosing publishing and documentation system. Good reading. Thanks!

  2. Sean Russell says:

    This is an outstanding write-up, and while my killer feature list is slightly different than yours, it overlaps in critical points. One item in my bucket list is to build tooling to produce trace-ability matrices across documents (requirements + design => matrix). Another is a tool that will take a diff of two versions of a document and produce a third document with track-changes style mark-up. There’s a long-standing ticket for the latter in the asciidoctor github with some interesting discussion, but no resolution as yet (https://github.com/asciidoctor/asciidoctor/issues/1030).

  3. Thomas Kern says:

    Since you already mentioned Brackets, you may want to try the Asciidoc Preview extension https://github.com/asciidoctor/brackets-asciidoc-preview, whihc should make it much more convenient. Simply install it from the Brackets Extension Manager.

    • wolandscat says:

      I did try it early on, but our docs are full of path variables that get set from the Asciidoctor command line, so Brackets doesn’t know about them. I also seem to remember it left html files in the work directory which could get accidentally committed.

      • Thomas Kern says:

        Good points. The HTML file is created when you use the “Export to Browser” feature. It’s kind of ugly to put this into the source directory, I agree. I’ll see if this can be changed.

        Do you mean that you define attributes containing paths? You could define all your attributes otherwise passed on the commandline within a separate file, say paths.adoc:

        :mypath: c:/tmp
        :anotherpath: d:/blabla

        At the top of your document, you just include this when you are in a browser environment:

        = Test

        ifdef::env-browser[]
        include::paths.adoc[]
        endif::env-browser[]

        This is {mypath} and {anotherpath}

        Make sure in the settings diaglog of the plugin that the “Safe mode” is either “Safe” (default) or even “Unsafe”. Otherwise ‘include’ does not work.

  4. wolandscat says:

    @ThomasKern Re: HTML location – can it be some sort of $TMP location? Re: paths – yes, we have most of them in files. I can do the remaining ones that way. And I had not realised you could do conditional directives, so that solves the obvious problem. Still learning! Thanks for the help, much appreciated.

  5. Shahryar says:

    I wrote a dissertation in Asciidoctor recently (http://bit.ly/1R1uy0r). I guess academic publications has publishing requirements above average, so it can be considered a proof of concept for Asciidoctor capacities. It went through Asciidoc -> Docbook -> Latex and Asciidoc -> HTML (and Asciidoc -> odf -> docx for journal submissions). Most of controlling things were communicated through “role”s in Asciidoc to my customization codes for DBLatex (DocBook to Latex) and Latex. I had to do a few regular expression things, and were in short of time to convert tables to nice Latex Tabulars so I embeded direct latex for that, used latex Gloassaries (instead of using same source in HTML), and used asciidocbib for bibliographies in HTML but generally it was a smooth process and I will continue with that.
    PS: the Asciidoc source of the dissertation and my toolchain things around Asciidoctor can be found here https://github.com/shahryareiv/asciimint .

    • wolandscat says:

      Funny – we both work in e-health, as you will see from the rest of my blog. But there’s only a very minimal amount of asciidoc source on your Git repo – I can’t see how you did your citations or multi-column layout for example (or maybe it’s all in DB.Latek, neither of which I use?)

      • Shahryar says:

        Yes! And of course OpenEHR is famous enough to be known by people in the field.

        The full source file of my dissertation is in the example folder (https://github.com/shahryareiv/asciimint/tree/master/example). At the moment, things are messy out there and I just put some files for possible references. But to summarize:

        I put the source file in its specific folder in src_text , all figures (including tikz) in src_figures, the cover (for Latex) in its specific folder in z_covers_templates, bibtex file in z_biblio_glossary and then run the makefile. The makefile calls asciidoctor to get a Docbook then calls DBLatex with my customization in z_compile_room/my_docbooklatex.xsl to make a latex file which is combined with the cover files and uses the layout specified in z_libs/tex_libs/my_tufte_v2.sty. The bibtex and glossaries are also get done in this stage. A similar process runs for the HTML version. The images are also converted to appropriate format for the final output.

        The citations are in the form of [cite:bibkey] in the source file. This should probably be changed to cite:[bibkey] in future. At the moment I use sed command to covert each [cite:bibkey] in the source file to +++bibkey+++ before converting to DocBook. The HTML route uses AsciidocBib to read and interpret each [cite:bibkey]. I hope all above things would be replaced by Asciidoctor in future.

        I also use “roles” in asciidoc to signal the right conversions. So for example if I put [.marginfigure] for an image block then I would have in my DocBook and I can convert that to \mymarginfigure{…} in the generated latex file and of course I have written some macro for \mymarginfigure in my Latex package that does the job.

        Hope things are now more clear.

  6. Shahryar says:

    Just noticed that WordPress hides the DocBook tags I mentioned in my previous comment! I was talking about \ and \

    • Shahryar says:

      Well! I could not escape those tags in the comment. Anyway you can find them in the files on github.

  7. David Gamba says:

    I am using Asciidoctor as well and migrated one of our projects from Framemaker too.
    Do you have any comments/experience with translations with asciidoc? That is an area I am still struggling with. Thanks!

    • wolandscat says:

      I’m afraid we don’t yet, and it is going to be an issue for us as well – we are publishing for an international audience.

      • David Gamba says:

        Looking forward to your blog post describing the solution you come up with 😉

      • wolandscat says:

        @David Gamba I will certainly do that, but probably not that soon. One problem is that translation is a problem much easier to deal with in structured text solutions like DITA where the text fragments are inside formally processable ‘boxes’, which is the opposite of Asciidoc, where the text structure is inferred by parsing a narrative form. In DITA you can translate complex documents pretty easily without having to repeat the formatting, but in Asciidoc, I have to say it’s not clear to me how translation without repeating formatting can be done yet. Maybe it just needs more thinking 😉

  8. Shahryar says:

    @wolandscat, If I got your concern correctly: AsciiDoc main point (at least for me) is its promise to keep matching with DocBook (well, a subset of DocBook). In this sense, transforms from Asciidoc to Docbook has the highest fidelity and an AsciiDoc document has a guaranteed DocBook alongside. I have never experienced DITA, but I guess, the same as for DocBook, the XSLT templates should be able to address a whole template-based formatting to anything including DITA or a presentation format.

    As another option, I guess some people prefer writing extensions and backends for Asciidoctor, traversing the Parsed Trees of the document in Ruby and/or templating with Slim. I have not done that way because of lacking Ruby skills (years ago it was not that common when I shifted to other roles in software industry) and of course my first priority was writing my dissertation. Also, my final target was Latex and DBLatex was mature enough hence just some customization could save me from reinventing the wheel. But if you are looking for a more serious thing (or reusing parts of your previous tool chain with DITA) then you may find this approach better.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s