Filed under XML

NioSax – Sax style xml parser for Java NIO

NioSax (pronounced ‘Neo-Sax’) provides a Java NIO friendly XML push parser similar in operation to SAX. Unlike SAX, with NioSax it is possible for the xml source to contain partial content (i.e. only part of the XML stream has been received over the network). When this occurs, instead of failing with an error, NioSax simply stops. As soon as your application receives more data you simply call the same instance of the parser again and it will resume parsing where it left off.

The public API consists of the classes within this package, although the bare minimum required for use are the NioSaxParser, NioSaxParserHandler and NioSaxSource classes.

To use NioSax you simply use NioSaxParserFactory to create a NioSaxParser, implement a SAX ContentHandler and finally create a NioSaxSource which references the content.

Then you can parse one or more ByteBuffer’s by updating the NioSaxSource with each buffer and pass it to the NioSaxParser.parse(NioSaxSource) method.

The only other two things you must to do with the parser is to ensure that you call NioSaxParser.startDocument() prior to any parsing, and call NioSaxParser.endDocument() once you are done with the parser so any resources used can be cleaned up.

Example

First in maven we need to add a dependency to NioSax. For details of the repository click on the ‘reteptools’ menu above. However you’ll need to add the following to your pom:

<dependency>
    <groupId>uk.org.retep</groupId>
    <artifactId>niosax</artifactId>
    <version>10.6</version>
</dependency>

Now we’ll create a parser:

import java.nio.ByteBuffer;
import uk.org.retep.niosax.NioSaxParser;
import uk.org.retep.niosax.NioSaxParserFactory;
import uk.org.retep.niosax.NioSaxParserHandler;
import uk.org.retep.niosax.NioSaxSource;

public class MyParser
{
    private NioSaxParser parser;
    private NioSaxParserHandler handler;
    private NioSaxSource source;

    public void start()
    {
        NioSaxParserFactory factory = NioSaxParserFactory.getInstance();

        parser = factory.newInstance();
        parser.setHandler( handler );
        source = new NioSaxSource();

        parser.startDocument();
    }
}

Next, when you receive data from some nio source and have the data in a ByteBuffer you need to pass it to the parser:

    public void parse( ByteBuffer buffer )
    {
        // flip the buffer so the parser starts at the beginning
        buffer.flip();

        // update the source (presuming the buffer has changed)
        source.setByteBuffer( buffer );

        // Parse the available content then compact
        parser.parse( source );
        source.compact();
    }

Finally we must call endDocument() to release any resources:

    public void close()
    {
        // releases any resources and notifies the handler the docment has completed
        parser.endDocument();
    }

Now all we need to is when we receive some data from an external source like a Socket, we pass the ByteBuffer to the parse method. This then passes it to the NioSax parser which in turn calls the ContentHandler as the parse progresses.

When it gets to the end of the available content, it compacts the buffer so that it can be reused.

Usually the buffer will now be empty, however if there was partial content (like only part of a Unicode character was present) then the parser would stop prior to that character and that character would remain in the buffer. The next packet received via nio would have the rest of that character and the parser would then continue where it left off.

This was originally posted early in 2009 but the post seemed to have vanished so this article is loosely based on the documentation for NioSax.

Implementing Builders with JAXB generated objects

While JAXB is a good method of mapping XML into Java objects, if your schema’s are large or complex, generating the required Object graph prior to marshaling into XML can be both tedious and error prone due to the large amount of ‘boiler plate’ code of creating an object, calling accessor methods to set the values etc.

A common pattern for easing this sort of thing is known as the Builder pattern where you either have a common object holding configuration and then it builds the objects for you, or a set of objects each configured with the information and when required then build the final Object graph.

Previously however, this involved writing these builder classes manually once you have generated the model from a schema, however retepTools provides a JAXB plugin which, with a little configuration within the bindings will automatically generate these builders for you.

First an example of a simple pair of objects to be generated by JAXB – this is an abridged version of the jabber:client namespace in XMPP:

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='jabber:client'
    xmlns='jabber:client'
    elementFormDefault='qualified'>

  <xs:element name='message'>
     <xs:complexType>
        <xs:sequence>
          <xs:choice minOccurs='0' maxOccurs='unbounded'>
            <xs:element ref='subject'/>
            <xs:element ref='body'/>
          </xs:choice>
        </xs:sequence>
        <xs:attribute name='from' type='xs:string' use='optional'/>
        <xs:attribute name='from' type='xs:string' use='optional'/>
     </xs:complexType>
  </xs:element>

  <xs:element name='body'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='nonEmptyString'>
          <xs:attribute ref='xml:lang' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='subject'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='nonEmptyString'>
          <xs:attribute ref='xml:lang' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:simpleType name='nonEmptyString'>
    <xs:restriction base='xs:string'>
      <xs:minLength value='1'/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

Now if run through JAXB this would generate three classes: Message, Body and Subject. With this default model you would have to do something like the following to create a valid Message:

Message message = new Message();
message.setFrom( "juliet@example.com/balcony" );
message.setTo( "romeo@example.net" );

Body body = new Body();
body.getBodyOrSubject().add( "Wherefore art thou, Romeo?" );
message.setBody( body );

Now with this simple example thats fine, but imagine something where your model is far more complex, or worse can contain arbitary namespaces as children such as in XMPP where Message can contain objects within any namespace?

This is where Builders become useful. Imagine the above being converted into this:

Message message = new MessageBuilder().
	setFrom( "juliet@example.com/balcony" ).
	setTo( "romeo@example.net" ).
	addBody( new BodyBuilder().
		 setValue( "Wherefore art thou, Romeo?") ).
	build();

Although at first this may look similar to the standard way, the builders break the code into more manageable chunks. The builders themselves can be reused, so if you are sending multiple messages then you could change the above code to:

MessageBuilder builder = new MessageBuilder().
	setFrom( "juliet@example.com/balcony" ).
	addBody( new BodyBuilder().
		 setValue( "Wherefore art thou, Romeo?") );

Message message1 = builder.setTo( "romeo@example.net" ).build();

Message message2 = builder.setTo( "tybalt@example.com/cousin" ).build();

As you can see, the use of builders becomes apparent, we created two Message objects with a single change between them in a lot less code.

On the next page we’ll configure maven to build these builders.

Using JAXB to marshall XMPP

Back in late 2008, one of the original design goals for the rewrite of my retepXMPP project was to use JAXB for handling the marshalling of XMPP Stanzas into POJO’s. The main ideas behind this was to standardise against the XMPP schemas available online and to make the addition of further protocols easier – mainly not to break existing code.

Now this was fine until I started testing against a couple of client libraries, certain messages were being ignored. It turned out that the XML generated by JAXB does not follow the rules defined in RFC 3920bis. The problem here was that JAXB was placing all of the namespaces together in the root.

For example here’s one of the examples from RFC3921bis as generated by JAXB:

  <iq from='juliet@example.com/balcony'
       id='rg1'
       type='get'  xmlns='jabber:client'  xmlns:ns1='jabber:iq:roster'>
    <ns1:query/>
  </iq>

Here’s what it should look like:

  <iq from='juliet@example.com/balcony'
       id='rg1'
       type='get'>
    <query xmlns='jabber:iq:roster'/>
  </iq>

There’s two differences here, but the main issue is where ‘jabber:iq:roster’ is declared in the root iq element and not against the query element it’s supposed to be – the other is the declaration of the ‘jabber:client’ namespace but that one is not a problem here.

Although both are strictly correct XML wise, it isn’t for XMPP and most parsers expect the stream to follow the rules.

So why is JAXB doing this? Well apparently its by design and, if you use the JAXB-RI like I do, there’s not much we can do to change this.

In JAXB 1 it actually declared the namespaces at the appropriate places so we would get the ‘jabber:iq:roster’ namespace against the query element as expected and the XMPP stanza would then conform correctly.

However in JAXB 2.0 they changed this to the current behaviour for performance reasons. Apparently with large documents looking ahead for what namespaces are present uses a lot of resources, so they declare them first.

Now I can see their point but for one small issue – this presumes that, when a document is being parsed then unmarshalled by JAXB it was originally produced by JAXB – what if it wasn’t generated by JAXB? In that case JAXB would still do the lookup anyhow.

Unfortunately when I took a look at the source for JAXB 2.2 this declaring of namespaces is done pretty close to the start of the marshalling process, so unless they add an option to disable it we have to live with this.

So how to fix this so we can still use JAXB but generate conforming XML?

Well I’ve got a solution but it’s not pretty. The solution simply involves breaking the marshalling/unmarshalling process up into separate units of work, one per namespace. We marshal the first object, then – if it has children, marshal them.

Now this involves additional work during the marshalling/unmarshalling process but it works. The downside is that we have to modify the schemas to do this.

Fortunately the XSF’s schemas are not concrete. As Peter Saint-Andre’s put it a few weeks ago on the muc mailing list, those schemas are ‘descriptive, not normative’. This means they are representative of what the XEP’s define, but they can be changed. In fact for some (like XEP-0045/MUC) the schemas don’t define the extension points used by other XEP’s. This means we can make some simple modifications to them to get this ‘hack’ to work.

So now I have this working pretty well – marshalling to xml runs smoothly. Unmarshalling still has a few problems but I’m being hit by a nice issue with Java’s generics but that’s going to be a later post.

Follow

Get every new post delivered to your Inbox.

Join 1,397 other followers