xgridfit
Author: Peter S. Baker

Home Page Project Page Download CVS repository

# Managing large projects

## XInclude

Instructions for large fonts and font families may be split into parts so as to speed compilation for testing purposes, distribute work among members of a team, or create libraries of shared functions. Xgridfit's mechanism for splitting projects is XInclude, the W3C "XML Inclusions" specification. XInclude is extremely flexible and simple to use: it allows you to structure a complex project in a number of ways.

To use XInclude in an Xgridfit file requires just two steps. First, include the XInclude namespace declaration in the <xgridfit> element:

        <xgridfit xmlns="http://xgridfit.sourceforge.net/Xgridfit2"
                  xmlns:xi="http://www.w3.org/2001/XInclude">

Next, to merge part of another file into the file you're editing, add an <xi:include> element:

        <xi:include href="Junicode-Regular-Basic.xgf"
                    xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                    xpointer(/x:xgridfit/x:glyph)"/>

Here the `href` attribute (a URI) points to a file. The `xpointer` attribute points to the element or elements that we want to import into the current file; it is an XPointer (usually a simple XPath expression). In the present example, the XPointer expression pulls in all the <glyph> elements from the file `Junicode-Regular-Basic.xgf`.

It is necessary to specify the Xgridfit namespace for each <xi:include> element, and this makes the <xi:include> tag rather verbose; but much of this is boilerplate which can be copied from tag to tag. Indeed, the only variable part of the `xpointer` attribute is inside the final set of parentheses.

Several Xgridfit elements can take an `xml:id` attribute, and can be included by referencing that id: <xgridfit>, <pre-program>, <function>, <macro>, <glyph>. To fetch this <function> element

        <function name="myfunc" xml:id="jun-reg-myfunc">
           . . .
        </function>

use one of these <xi:include> elements:

        <xi:include href="Junicode-Regular-common.xgf" xpointer="jun-reg-myfunc"/>

        <xi:include href="Junicode-Regular-common.xgf#jun-reg-myfunc"/>

Notice that you do not need to worry about namespaces when you use the `xml:id` method.

Some XSLT engines can handle XInclude and others cannot. The XInclude capability in xsltproc (the engine invoked by the xgridfit shell script) is turned on by simply including the parameter `--xinclude` on the command line (this is done for you in the shell script). For some other engines, turning on XInclude capability is an arcane and difficult matter: you must check the documentation for your preferred engine. The `xgridfit` shell script uses `xmllint` when necessary to work around difficulties with XSLT engines and XML validators.

## Example of a large project

XInclude is flexible enough to permit you to structure your project in many different ways (you must just remember to avoid recursive includes). Here is a simple example involving the Junicode font (complete source code available at the download site).

Instructions for Junicode-Regular are compiled by running xgridfit against a master file, `Junicode-Regular.xgf`, which contains little besides <xi:include> elements:

        <?xml version="1.0"?>
        <xgridfit xmlns:xi="http://www.w3.org/2001/XInclude">
          <xi:include href="Junicode-Regular-common.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:infile)"/>
          <xi:include href="Junicode-Regular-common.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:outfile)"/>
          <xi:include href="Junicode-Regular-common.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:constant)"/>
          <xi:include href="Junicode-Regular-common.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(//x:control-value)"/>
          <xi:include href="Junicode-Regular-common.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(//x:function)"/>
          <xi:include href="Junicode-Regular-common.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(//x:macro)"/>
          <xi:include href="Junicode-Regular-common.xgf" xpointer="jun-reg-prep"/>
          <xi:include href="Junicode-Regular-Basic.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-Latin1.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-LatExtA.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-LatExtA-dotlessi.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-LatExtB.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-IPA.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-SpacMod.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-CombDiac.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                  xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-LatExtAdd.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-GenPunct.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-Greek.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-PUA.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
          <xi:include href="Junicode-Regular-NoEncode.xgf"
                      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                                xpointer(/x:xgridfit/x:glyph)"/>
        </xgridfit>

The file `Junicode-Regular-common.xgf` contains no <glyph> elements, but only global elements: <infile>, <outfile>, <constant>, <control-value>, <function>, <macro>, <pre-program>. These are included with the first seven xi:include elements (the <pre-program> is included by `xml:id`, just by way of illustration). The other files contain only <glyph> elements organized by Unicode range. These are included with a specific XPointer, `/x:gridfit/x:glyph`, rather than `//x:glyph`, to avoid pulling in <glyph> elements inside of <no-compile> elements. Each file validates against the Relax NG schema xgridfit.rnc.

When working with any particular Unicode range, most or all of the other <xi:include> elements can be commented out in the master file, speeding compilation. If more than one developer were working on this font, the file `Junicode-Regular-common.xgf` might be controlled by the lead developer so as to avoid conflicts introduced by individual contributors, who when necessary would submit patches on that file for approval.

## Shared functions and libraries

You very likely have some favorite functions that you insert in all your fonts. In addition, similar members of a font family are likely to share functions. For a library of shared functions, simply create an Xgridfit program file in which the only children of the <xgridfit> element are <function> elements. An Xgridfit program file can very easily merge the functions from this library anywhere in the sequence of your own functions:

        <function name="local-function">
          . . .
        </function>

        <xi:include href="My-Library.xgf#xpointer(//function)"/>

        <function name="another-local-function">
          . . .
        </function>

Alternatively, you can merge a single function from the library if you have supplied that function with an `xml:id` attribute:

        <function name="local-function">
          . . .
        </function>

        <xi:include href="My-Library.xgf#set-left-sidebearing"/>

        <function name="another-local-function">
          . . .
        </function>

Note that the included function must be called by its required `name` attribute and not by the `xml:id` attribute, which is needed only when a function is merged, before it gets compiled. The `name` and `xml:id` attributes should probably be the same when both are present.

## Separate compilation of source files

Though an Xgridfit file is destined to be included in a master file, it can also be compiled separately. This may be convenient when writing instructions for a specific range of glyphs. To prepare a file for separate compilation, simply use XInclude to pull in global elements from the common file. For example, one of the Junicode source files containing only glyphs has these XIncludes:

      <xi:include href="Junicode-Regular-common.xgf"
    	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
    			xpointer(/x:xgridfit/x:constant)"/>
      <xi:include href="Junicode-Regular-common.xgf"
    	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
    			xpointer(//x:control-value)"/>
      <xi:include href="Junicode-Regular-common.xgf"
    	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
    			xpointer(//x:function)"/>
      <xi:include href="Junicode-Regular-common.xgf"
    	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
    			xpointer(//x:pre-program)"/>
      <xi:include href="Junicode-Regular-common.xgf"
    	      xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
    			xpointer(//x:macro)"/>

If any glyph program in the file needs to refer to a glyph program in another file, this can also be pulled in via XInclude. <xi:include> elements for this purpose should be placed in a <no-compile> element, which makes them visible to all the programming in the file but prevents their being compiled. This is from `Junicode-Regular-Latin1.xgf`:

      <no-compile>
        <xi:include href="Junicode-Regular-Basic.xgf"
                    xpointer="xmlns(x=http://xgridfit.sourceforge.net/Xgridfit2)
                              xpointer(/x:xgridfit/x:glyph)"/>
        <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapAcute"/>
        <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapCircumflex"/>
        <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapDieresis"/>
        <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapGrave"/>
        <xi:include href="Junicode-Regular-NoEncode.xgf" xpointer="CapTilde"/>
        <xi:include href="Junicode-Regular-SpacMod.xgf" xpointer="circumflex"/>
        <xi:include href="Junicode-Regular-SpacMod.xgf" xpointer="tilde"/>
        <xi:include href="Junicode-Regular-LatExtA-dotlessi.xgf" xpointer="dotlessi"/>
      </no-compile>

Note that any of these included glyphs will be compiled if specified in a <glyph-select> element or with the `-g` command-line option.

Remember that you must either take care to avoid recursive XIncludes or comment out the XIncludes in a file before including elements from it it in another file.

Another strategy for speeding development is to create a quick-and-dirty Makefile for compiling and displaying a single file of glyph programs. The following example uses merge-mode, in which it is necessary first to compile the file containing the font's common elements (functions, control-values, etc.) and run that script. The Makefile compiles the common file only when it has changed.

    FONTNAME = MyFont
    RANGE = IPA

    COMMON = $(FONTNAME)-common
    COMMONSCRIPT = $(COMMON).py
    COMMONXGF = $(COMMON).xgf

    GLYPH = $(FONTNAME)-$(RANGE)
    GLYPHSCRIPT = $(GLYPH).py
    GLYPHXGF = $(GLYPH).xgf

    SOURCEFONT = $(FONTNAME).sfd

    OUTPUTFONT = $(FONTNAME).ttf

    INTERMEDIATEFONT = temporary.sfd

    $(COMMONSCRIPT) : $(COMMONXGF)
    	xgridfit -m -D -c yes -O $(COMMONSCRIPT) -o $(INTERMEDIATEFONT) \
    	    -i $(SOURCEFONT) $(COMMONXGF)

    $(GLYPHSCRIPT) : $(GLYPHXGF)
    	xgridfit -m -c no -O $(GLYPHSCRIPT) -o $(OUTPUTFONT) \
    	    -i $(INTERMEDIATEFONT) $(GLYPHXGF)

    $(INTERMEDIATEFONT) : $(COMMONSCRIPT) $(SOURCEFONT)
    	fontforge -script $(COMMONSCRIPT)

    $(OUTPUTFONT) : $(COMMONSCRIPT) $(GLYPHSCRIPT) $(INTERMEDIATEFONT)
    	fontforge -script $(GLYPHSCRIPT)

When you type `make MyFont.ttf`, this Makefile does the following:

- Compiles MyFont-common.xgf to make MyFont-common.py
- Compiles MyFont-IPA.xgf to make MyFont-IPA.py
- Runs the script MyFont-common.py against MyFont.sfd to create temporary.sfd
- Runs the script MyFont-IPA.py against temporary.sfd to make MyFont.ttf

This font contains all the glyphs in MyFont.sfd, but TrueType instructions only for the glyphs in the IPA range. To do the same with a file containing glyphs for another range, you need only change the `RANGE` variable near the top of the file.