XML Pipelines: the Process Description
The world of XML is moving fast, in its twelve year old history has raised many technologies. Many ways to validate, transform or query the data carried by the XML. Actually the validations and the transformations are commonly used together in the processes today so commonly called XML Pipelines.
XML PipelineBy definition “XML Pipeline is formed when XML processes, especially XML transformations and XML validations, are connected together”, that means if we send the output of the validation as input to the transformation (and that is a common use case), we have just created XML Pipeline, more accurately the linear XML Pipeline. Well non-linear pipeline would be the one, where we decide between two process branches based on a condition, where we are looping, running parallel branches or handling errors on the fly.
XProc, W3C Recommended XML pipeline languageXProc is a XML language describing processes, using XSLT for transformations and the XSD, RelaxNG or the Schematron for validation. The greatest thing about XProc is that you describe processing of the XML document by XML language, using the W3C standards on the way.
There is a catchWell with the XProc, we describe the process by one language, validate by another language and transform by yet another language. By using the three different languages for describing even the simplest use case of the XML Pipelining, a huge space for flaws opens. If we want to make something as trivial as changing the name of one element, we have to make changes in three different languages.
An alternative exist, it’s called XDefinitionIf you remember my previous article called “XML Processing and Validation Merging Together” where I wrote briefly about an interesting XML technology called XDefinitions, you will find one language able to transform and validate XML. And because its ability to validate the multiple inputs, transform them, and validate output again, all written in only one highly readable language, the space for the errors is limited to unavoidable minimum.
XML Pipeline by XDefinition
Every XDefinition process can initiate another XDefinition process. This kind of scalability allows us to model linear or nonlinear XML pipelines by one language. If we want to change the structure, we will do it on one place only.
<xd:def xmlns:xd = "http://www.syntea.cz/xdef/2.0" xd:root = "Weather" >
<!-- declaration of variables -->
<xd:declaration>
String $source; /*URL to source data */
float $sum = 0; /*total sum of temperatures */
int $n = 0; /*number of measurements*/
</xd:declaration>
<!-- model of input data -->
<Weather date = "optional datetime('yyyy-M-d')">
<Measurement xd:script = "occurs 1..;"
wind = "required float()"
temperature = "required float(-99, +99);
onTrue {/* process value of temperature. */
$n++; /* increase number of measurements. */
$sum += parseFloat(getText()); /* add value to total sum. */
}"
time = "required datetime('HH:mm')" />
</Weather>
<!-- model of result -->
<html xmlns = "http://www.w3.org/1999/xhtml"
xd:script="create parseXml($source, '*');
/* Parse and validate input data with the Weather xdefinition. */">
<body>
<h1> create "Date: " + from("/Weather/@date") </h1>
<li xd:script="occurs +; create from('//Weather/Measurement')">
create "Time: " + from("@time") +
", wind: " + from("@wind") +
", temperature: " + from("@temperature");
</li>
<h3>
create $n == 0 ? "No data" /* $n is equal to 0 */ :
"Average temperature : " + ($sum/$n); /*print average*/
</h3>
</body>
</html>
</xd:def>
Input document
<?xml version="1.0" encoding="UTF-8"?>
<Weather date = "2005-05-11" >
<Measurement wind = "5.3" temperature ="13.0" time = "05:00" />
<Measurement wind = "7.2" temperature ="15.2" time = "11:00" />
<Measurement wind = "8.7" temperature ="18.1" time = "15:00" />
<Measurement wind = "3.9" temperature ="16.5" time = "20:00" />
</Weather>
Output document
<html xmlns="http://www.w3.org/1999/xhtml">
<body>
<h1>
Date: 2005-05-11
</h1>
<li>
Time: 05:00, wind: 5.3, temperature: 13.0
</li>
<li>
Time: 11:00, wind: 7.2, temperature: 15.2
</li>
<li>
Time: 15:00, wind: 8.7, temperature: 18.1
</li>
<li>
Time: 20:00, wind: 3.9, temperature: 16.5
</li>
<h3>
Average temperature : 15.7
</h3>
</body>
</html>
Try this example here online. As you can see XDefinition contains a model of the input document against which it’s being validated. You can even use variables from the validation in the transformation/composition as is $n and $sum (used in the example) that are global for every model of XDefinition. That is very important, because XDefinition can complete validation with non fatal errors. So your process doesn’t have to end with invalid but well formed input. Invalid document still contains the information value and you can work with it.
Why “composition”? Because the word composition describes the process better which is being done by the XDefinition. Nowadays the overwhelming majority of the data generated by every transformation process has XML output. If you take that for granted the whole XML Pipeline has to be well formed and the transformations are then compositions of smaller nodes. Any composition with the XDefinition is XML data modeling and your pipeline has a complete right to be called a XML Pipeline.
Resources
- http://www.syntea.cz/syntea_web/?page=xdef_guidepost&hl=en_US
- http://en.wikipedia.org/wiki/XML_pipeline
- http://www.w3.org/TR/xml-pipeline
| Attachment | Size |
|---|---|
| clanekxmlpipeswellEN.pdf | 69.36 KB |
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)
Comments
Christian Schli... replied on Thu, 2010/11/11 - 6:51am
Daniel Kec replied on Thu, 2010/11/11 - 7:54am
in response to:
Christian Schlichtherle
Andrew McVeigh replied on Thu, 2010/11/11 - 12:26pm
i had a look at xproc about a year ago when we had a requirement for an XSLT2 pipeline for fpml trade transformations. i just couldn't seem to find any implementations or any tools around it. it looked to be a moving target, in working draft state, and had been there for quite a while in various guises.
we were using Saxonica under .Net and in the end just made our own simple pipeline language with 10 different step types. worked a treat, but i always thought it was a shame the xproc thing hadn't resulted in anything concrete.
was i missing a mature implementation of xproc which was available? if not, what are the actual options -- xproc itself is large and making my own implementation is out of the question. is xdefinition a possible alternative and what are the step types? i.e. does it have things like choice, enrich etc. also, does an implementation exist?
Liezel Jane Jandayan replied on Thu, 2011/08/25 - 6:22am
Dinni Davidson replied on Fri, 2011/10/14 - 9:44am