Since years I am very unsatisfied with the state of generator tools, which generate (Java) classes from service contracts (WSDL, Swapper/OpenAPI etc.). Most of my concerns are on the one hand centered around the problems they cause with compile-time checks (i.e., static typing) and on the other hand they do not fully embrace compile-time checks where they would help developers. So it's kind of the worst of two worlds. But what would be the "best" API for creating message payloads look like?
Requirements
I have thought about what requirements I have for such tools and libraries and have come up with this list:- read defined payload in a way that statically checks that this data is available (),
- support data readers in validating incoming data,
- give you a compilation error if the service contract does not match your assumptions anymore,
- does not give you a compilation error if the service contract matches you assumptions but with a different data type hierarchy (but still the same data documents),
- supports easy signaling of error conditions to the caller,
- strongly supports (if not guarantees) that you create contract-compliant responses,
- allow streaming of messages being sent.
Existing Frameworks
We can now go through the list of available frameworks and libraries and see them all fail on one point or another. Let's pick some examples:
All JAXB-based technologies (like JAX-WS and Spring-WS) succeed on points 1, partly 2, partly 3, party 5. They partly fail on 2, 3,5 and fully fail on 4. This is due to the way that types in XML Schema are mapped to Java classes, and that only full payload validation of XML messages against the XML Schema is possible (or none). Thus, refactorings in the schema can break your code even if the schema-changes are fully backwards-compatible. The latter is also true for the OpenAPI generators for REST services.
On top of this, Spring-WS has a way of mapping exceptions to SOAP faults via EndpointExceptionResolvers, which is non-intuitive for me and unnecessarily separates the response payload preparation between the service class and the resolver. But JAX-WS isn't better: it parses the WSDL at run-time for every(!) instantiation of a service proxy, which in most cases is unncessary - especially if in principle all information of the WSDL is generated into the Java code. This leads to JAX-WS object caches which were not necessary if object instantiation was cheap. I would love to see code generators which are optimized for service-based messages (instead of full scale XML or JSON support) and address these issues to make the implementation of service providers and consumers easier and more robust.
Why all this pain?
But why do we need generators and such libraries anyways? Can't we just code something without it? Yes and no. I think that in general most people have experienced that a contract-first design leads to better solutions. It treats the service contract (and with it the shared knowledge between provide and consumer) as a first-class citizen and design artifact. However, the code must still be written and this would be an annoying, repetitve, and meaningless job to copy the contract structure to your programming language of choice. Thus, we need (lightweight) generators that take over that task and transfer the information contained in the contract to our code.
If such generators and the generated code are well thought out, they are a bless and can eliminate much work and make our development much faster and increase quality.
Envisioned Programming Model by Example
For the example I will use the following schema (I use XML schema but any other schema language would suffice the example):
<schema
xmlns="http://www.w3.org/2001/XMLSchema"
xmlns:tns="http:/example.digialsolutionarchitecture.com/service"
elementFormDefault="qualified"
targetNamespace="http:/example.digialsolutionarchitecture.com/service"
>
<element name="startMortgageCreationProcessRequest">
<complexType>
<sequence>
<element name="parcelIds" type="string" minOccurs="1" maxOccurs="unbounded" />
<element name="amount" type="tns:Money" minOccurs="1" maxOccurs="1" />
<element name="rank" type="unsignedInt" minOccurs="0" maxOccurs="1" />
</sequence>
</complexType>
</element>
<element name="startMortgageCreationProcessResponse">
<complexType>
<sequence>
<element name="caseId" type="string" minOccurs="1" maxOccurs="1" />
<element name="lrId" type="string" minOccurs="0" maxOccurs="1" />
<element name="parcels" minOccurs="0" maxOccurs="1">
<complexType>
<sequence>
<element name="parcel" minOccurs="1" maxOccurs="unbounded">
<complexType>
<attribute name="parcelNumber" type="string" use="required" />
</complexType>
</element>
</sequence>
</complexType>
</element>
</sequence>
</complexType>
</element>
<complexType name="Money">
<sequence>
<element name="amount" type="decimal" />
<element name="currency" type="string" />
</sequence>
</complexType>
</schema>
For me, the best API to read message data would look something like the following:
import com.digialsolutionarchitecture.example.service.StartMortgageCreationProcessRequest;
import com.digialsolutionarchitecture.example.service.StartMortgageCreationProcessResponse;
import com.digialsolutionarchitecture.example.service.StartMortgageCreationProcessRequest.Money;
public void startMortgageCreationProcess(
StartMortgageCreationProcessRequest request,
StartMortgageCreationProcessResponse response) {
try {
// evaluate request
List<String> parcelIds = request.getRequiredParcelIds();
Money amount = request.getRequiredAmount("Amount for Mortgage must be specified");
Integer rank = request.getOptionalRank();
// do something
...
// prepare response
...
} catch(ValidationFailed e) {
...
}
}
This API does look a lot like JAXB or any other (OpenAPI) data mapping. However, there are some important but less visible differences:
- There are two types of getters. One for required fields that validate that the returned value is indeed not null, and one for accessing optional values, which requires the business logic code to deal with missing values. If a schema indicates that an element is optional, only the optional getter is generated. For required elements, both getters are generated. This way the application can indicate its preconditions and easily validate the incoming message in an application-specific way. If the schema changes, e.g., a required element is changed to an optional element, compilation will fail because the getRequired method is then missing. Thus, the developer knows that he/she must add new logic to handle missing values.
- All getters may throw a ValidationFailed exception (I am trying out to strip unnecessary parts of identifiers - such as exceptions - as suggested by Kevlin Henney. I like it for exceptions, I think my code looks clearer, but my co-workers still hate it :-)): If element content does not conform to the schema or a getRequired finds no data, this exception is thrown. In this way, partial, reader-based validation is implemented, which is more lax than validating against the whole schema but in a developer-friendly way still gaurantees that the following logic can asume that data is set. getRequired methods are overloaded: one with and one without a validation message that is to be contained in the thrown ValidationFailed exception.
- Classes are not generated on the schema's type hierarchy but on the element hierarchy. This is a design decision which I would need to really try out in practice, because I am not so sure about the balance of its benefits and drawbacks. On the one hand, and this is the driver for this decision, the generated code becomes completed decoupled from the type hierarchy of the schema and thus is robust against schema type hierarchy refactorings, which is one important goal. However, on the other hand, a single type in the schema can be generated as multiple (Java) classes leading to redundancy and require more effort to map the externally visible schema types to internal business logic objects.
The envisioned service also needs to write data. I would opt for a fluent API resembling the element hierarchy:
import com.digialsolutionarchitecture.example.service.StartMortgageCreationProcessRequest;
import com.digialsolutionarchitecture.example.service.StartMortgageCreationProcessResponse;
public void startMortgageCreationProcess(
StartMortgageCreationProcessRequest request,
StartMortgageCreationProcessResponse response) {
try {
// evaluate request
...
// do something
String caseId = ...;
String lrId = ...;
String pn = ...;
...
// prepare response
response
.setRequiredCaseId(caseId)
.setOptionalLandRegister(lrId)
.addOptionalParcels()
.addRequiredParcel()
.setRequiredParcelNumber(pn);
.endParcel()
.endParcels()
.toXML();
} catch(ValidationFailed e) {
...
}
}
The fluent API does not require to explicitly reference any types and is thus robust against schema type hierarchy refactorings. Also, the API does use the required and optional naming conventions. setRequired methods will throw an exception if a null value is passed to them (in addition to the @NotNull annotation, which allows for static checking of the code). Thereby, the robustness principle (or Postel's law: "Be conservative in what you send, be liberal in what you accept") for protocol/API implementations is realized.
By using a fluent API, JSON and XML can be streamed. There is no need to map all internal business objects to data transfer objects and then serialize this new object graph, which furthermore eases the burden on garbage collection and peak heap consumption. Instead, business objects can be directly written to the destination format. One drawback of this method is that schema-constraints, which relate to several elements, cannot be enforced. For example, XML schema supports ID and IDREF types as well as allows for XPath identity checks for guaranteeing uniqueness of data fields. However, I have very rarely seen these features used in service contract specifications. As such, I deem this limitation not important.
There can be toXML and toJSON (and toWhatever) methods that would allow to use the same syntax to create JSON and/or XML documents. This way, XML schema could also be used to create JSON documents (and JSON Schema for XML creation) without problems.
However, as the attention-paying reader might have spotted, the presented message structure is very simple. Most importantly, the response message has no list, which poses a challenge and I have found no solution that guarantees schema-compliance at design-time by use of the compiler or standard static code analysis tools.
These are my current thoughts on how JSON and XML-based services should be implemented. I hope that you have found this line of thought interesting. Do you have any thoughts about this topic? If yes, please drop me an email or open a Twitter discussion. I hope that we can have an inspiring discussion!
|
<<< Previous Blog Post Digital Solution Anti-Patterns I: Expose the internal organizational structure to your customers |
Next Blog Post >>> Why and How to Profit from Software Engineering Research? |