The main problem of XML, aside its verbosity, is the inability to create concise rules for the input or output document as a whole. Perhaps it’s a nice feature, because XML Schema should only be used to describe a data structure, not to infer business rules on it. Perhaps not. Nevertheless it’s not what I needed in one of the projects I’ve worked on.

My need was to actually check the business validity of such documents. This was used in a Web Service environment, a pretty stupid WS, which sole role was to fetch data from database and pack it into appropriate XML structures. Errors might occur in database’s views or in WS - as usual. They might be data multiplication or appearance of some elements while they shouldn’t. Resulting documents were correctly validated with the xml schema, but the result was simply wrong from the business point of view.

What I needed an XML formalization language, an ability to write rules that would assert some rules, report on not meeting stated rules. I was in need of a tool to write business rules to tame such XML entities.

The simplest way I found to solve this was to use Schematron! - “a language for making assertions about patterns found in XML documents”. This neat tool is a set of XSL templates, that you use in conjunction with a rule set on documents to check. As a result of the check you get another XML document with test assertions - whether failed or succeeded.

With Schematron you write a set of rules you expect the document to assert, than you use Schematron XSL template to produce XSL rules specific for your case. Now you only need to use newly generated XSL rules template on your XML document to check rules compliance. Easy, if not, check the diagram below.

How does it look?

  TouK Schematron test harness

    



  checking GetMigrationOffers
  
    Report date.
    Unique offers allowed.
    Each offer has to have an @abc attribute 
  
  
    Each offer has to have a tariff
    Each offer has to have a promotion
  



  checking GetAllPhones
  
    
        TACs should be unique. TAC: , 
        handsetId: 
        offerId: 
    
  

Here we see two rules, one named getMigrationOffers and the other getAllPhones. The rules - mainly their asserts seem pretty self explanatory, but for the sake of completeness I’ll describe the rules for getAllPhones.

There is one rule, which checks the uniqueness of tac elements. This rule tries to ensure that each handset should have a list of unique tac elements as its children. However there may appear tac elements of the same value in different handset elements.

Given an input XML in the form of:

   
      
         
            12028006
            20070705
            35535302
            01216100
            01216100
         
         
            12028006
            20070705
            35535302
            01216100
         
         
            12028006
            20070705
            35535302
            01216100
         
       
   

    
   
   
   
   
   
   
   
   
      
        TACs should be unique. TAC: 01216100, 
        handsetId: 95
        offerId: 103021
   
   
   
      
        TACs should be unique. TAC: 01216100, 
        handsetId: 95
        offerId: 103021
   
   
   
   
   
[...]

After running the validation, the report presents us with the result. It shows that there are actually non-unique tacs. Unfortunately the rule itself is not so optimal, as it is executed for each tac node. The better case would be to create a rule operating only on groups of tacs - having a rule for each handset’s tacs would be much better.

Performance consideration

As you may have seen, Schematron gives quite a potential, if it comes to rules building - maybe not the easiest to comprehend, since written with XPath, but good enough.

However, with all the XML processing involved in the process, it may take some considerable amount of time to execute such validations. For example, processing rules for file getMigrationOffers.xml takes about 2.296s - the file has 82 offer elements, which the rules operate on. But validating the other file, getAllPhones.xml takes 5.324s, with 3113 tac elements, and the rule iterating all of them.

This overhead is too much in most of the situations. That’s why this solution is rather not for use in normal execution pipeline - it would be unwise to put Schematron to check each request, thus entangle it into my Web Services normal flow.

What may be more desirable is to deploy a continuous integration server, with a project querying such Web Service and checking the rules in this manner.

Conclusion

So, what’s so great about having one XML generate another XML? Perhaps nothing, I think it would took just about a day to write some shell, python, <other text processing tool> that would perform equally (or even better). However, we loose technology homogeneity, and employ some other environments, not specific to our primary target platform, and that seems bad. Of course using some powerful text processing tool to impose the same rules might be much more efficient, thou less coherent.

What is your approach to such situations? Have you used Schematron or any other similar tool?

Code for this example is available on GitHub - http://github.com/zygm0nt/schematron-example.

DAY 1

It took place on 2nd-3rd of July, 2010, so this review is rather dated :) However, I’d like to keep this as reminder. I’ve arrived to the conference site, which was located in building of the Faculty of Mathematics and Computer Science a few minutes after the official start of the conference. I had been traveling from Warsaw the same day, and the only train that would not require me to get up at some night hour would arrive a bit too late. Oh well :)

I grabbed a tea and some biscuits and entered the series of lectures.

The first thing to listen to was a Welcome intro - quite nice one. Conducted by a guy from Cisco (AFAIK). He was talking about the opportunities for Kraków and how it will become a Polish Silicon Valley in near future, etc. Actually I don’t share his believes but the talk was ok.

Then came Dru Lavigne with some insight into BSD Certification program. Actually, does anybody use this? Come one. Do we really need another certification process? I for sure don’t see the need, especially for the BSD community. However the trend is good, may help popularize BSDs among enterprise leaders, because if something is certified, than it can be used in big enterprises, right? :)

Sławek Żak talked about NoSQL. Although the talk gave a bit of info about what the idea is and how does it compare to normal DBs, I did not find his presentation entertaining. In my opinion, there was not enough emphasis on the difference in usage for such databases. The talk about NoSQL I’d attended on Javarsovia was a lot better.

Next talk, presented by Attilio Rao was very, very technical. It was about “VFS/Vnode interface in FreeBSD”. It was rather an API presentation, and introduction on how to implement an FS in FreeBSD infrastructure, than a conference talk. This kind of presentation would be good suited for FreeBSD kernel developers not sysadmins.

Jakub Klama’s talk on the process of porting FreeBSD to Da Vinci embedded system was interesting. It had some photos of the board, tackled a few technical corners, but caught my attention. Well done!

Out guy among FreeBSD hackers - Paweł Jakub Dawidek - gave speech about HAST - High Availability STorage. In other words he implemented DRBD for FreeBSD. Sadly, for me this is just catching up with what Linux has in mainline since 2.6.33 (it was working very well even before that). It’s not so feature rich as DRBD, but the project is slowly maturing. Nevertheless, it’s good to finally have this on board.

Then an inconspicuous guy come onto the stage. Came from Bulgaria, named Nikolay Aleksandrov, that guy gave a talk titled Developing high speed FreeBSD. And the subject was astounding. He works for a major Bulgarian ISP and due to lack of cash to buy some serious networking gear, he wrote a FreeBSD extension that would sit in-between network adapter and the kernel and do all the hard work like routing, VLANs, and more. His goal was to make it lighting fast, and as far as his results showed, he succeeded. This talk was really amazing, he did what would normally take hundreds of thousands of dollars - in cash and skills - in his free time, or at least as a pet project.

DAY 2

Well, I’d skipped the first lecture of the day, because of laziness ;)

Had decide to pack myself and arrive to listen about what can freebsd borrow from AIX. Jan Srzednicki talked about some nice tools from the AIX world. He proposed that adding an educational, console-based tool for conducting basic (and even not so basic) tasks, would encourage people to learn the system. I think it would work. However the rest of his ideas weren’t good enough - at least not for me.

Next thing in line was The new USB stack. Interesting talk about new USB stack development, conducted by Hans Petter Selasky. This guy was really passionate about USB things ;-)

Martin Matuska presented his set of shell scripts that allow to create mfsBSD - an in-memory FreeBSD install. Since I’m already doing this kind of things with OpenBSD, the talk was entertaining.

Marcko Zec and Network stack virtualization. This was about extending FreeBSD to be able to create lots of compartmentalized environments with their own network stacks. As noted in the presentation: the solution still has problems with graceful shutdown of the stack. Still not stable enough - but very promising.

The closing presentation, given by Warner Losh (very knowledgeable guy behing bsdimp.blogspot.com) on the subject Using FreeBSD in a commercial settings. The talk was not what I’ve expected, but nevertheless was very interesting. It was about branching and merging back changes in case of using FreeBSD as a base for some commercial products. This could be easily applied to any other Open Source project. Warner described possible strategies for branching and performing merges, he noted also pros and cons of all the described solutions.

All in all, that was a fun time. Even thou I don’t use any BSD as my primary system at this time, and my BSD skills are a bit rusty, the talks were nice enough :) for a hobbist like me.

I took Arlanda Express, fast train that takes you directly from the airport to Stockholm. On board, you travel with 200km/h in good enough conditions. The trains takes you near the Central station in 20 minutes, which is great! The main venue I planned to see was Gamla Stan. All the attractions were also there, on the island.

The first noteworthy place on the road was Christina’s costume salon. Unfortunately closed on the day of my visit, however since it was recommended to me, and the place look interesting I plan on paying it another visit. This shop offers a lot of costumes, different sizes, themes. Website is: Shop’s page

Seems like just a usual gift shop, but has this unique feeling, this Norsk spirit. Shop offers many fantastic gifts and souvenirs, but unfortunately they don’t come cheap. Nevertheless, worth visiting and perhaps buying. Especially nice were little metal statues of Viking gods. Handfaste - The Viking Shop

Neat little workshop, that does metal signs on request, from your design. Very good quality. Actually there are quite a few of such workshops nearby, but this may serve as a reminder. Det “lille” Skyltmakerie

Awesome sci-fi, fantasy bookshop. Lots of stuff, comics, books, dvds, collectibles, etc. I strolled into it when going back to the airport, that’s why I had only around 5 minutes to look around. But the shop is definitely worth visiting. They offer English books among Swedish-lang ones, with 50:50 proportions in quantity. Science Fiction Bokhandeln

The darkest and best supplied metal shop I’ve ever seen - thou haven’t seen many of such stores. Again, stumbled on it on my way to the airport, so had only around 5 minutes to look around, scan through t-shirts, etc :D Sound Pollution

Some t-shirt shop, had also lots of underwear, dark costumes, etc. I think there may be more to it than I noticed.

Unfortunately hadn’t found this during my wandering. Will try next time: Roberta Settels shoes designs

Don’t know the name of this packed with designer items place, but it was warm and cosy :) Prices were high and the utensils neat. Bore marks of “Swedish design school”.

There were some additional preconditions:

• all given Enums are jaxb generated objects - but any standard Java Enum should work
• conversion was 1-to-1, no special conditions and processing

The example Enum for this problem looks like this one (copy&paste from jaxb generated source):

@XmlType(name ="ServiceType") 
@XmlEnum
public enum ServiceType {

    @XmlEnumValue("stationary")
    STATIONARY("stationary"),
    @XmlEnumValue("mobile")
    MOBILE("mobile");
    private final String value;

    ServiceType(String v) {
        value = v;
    }

    public String value() {
        return value;
    }

    public static ServiceType fromValue(String v) {
        for (ServiceType c: ServiceType.values()) {
            if (c.value.equals(v)) {
                return c;
            }
        }
        throw new IllegalArgumentException(v);
    }

}

“No big deal”, you say. I beg to differ. What I wanted to achieve was a simple construction which would look like this when used for another Enum (CommonEnumTypeHandler is the name of my generic converter):

public class ServiceTypeHandler extends CommonEnumTypeHandler { }

Unfortunately due to the fact, that Java does not have reified generics, which is described in multiple places, I had to stick with passing through a Class type of my enum. So it looks like this:

public class ServiceTypeHandler extends CommonEnumTypeHandler {

    public ServiceTypeHandler() {
        super(ServiceType.class);
    }
}

My final class has to look like this one below:

import java.sql.SQLException;

import com.ibatis.sqlmap.client.extensions.ParameterSetter;
import com.ibatis.sqlmap.client.extensions.ResultGetter;
import com.ibatis.sqlmap.client.extensions.TypeHandlerCallback;

public abstract class CommonEnumTypeHandler implements TypeHandlerCallback {

    Class enumClass;

    public CommonEnumTypeHandler(Class clazz) {
        this.enumClass = clazz;
    }

    public void setParameter(ParameterSetter ps, Object o) throws SQLException {
        if (o.getClass().isAssignableFrom(enumClass)) {
            ps.setString(((T) o).name().toUpperCase());
        } else
            throw new SQLException("Excpected " + enumClass + " object than: " + o);
    }

    public Object getResult(ResultGetter rs) throws SQLException {
        Object o = valueOf(rs.getString());
        if (o == null)
            throw new SQLException("Unknown parameter type: " + rs.getString());
        return o;
    }

    public Object valueOf(String s) {
        return Enum.valueOf(enumClass, s.toUpperCase());
    }
}