2. General

Content of this chapter

General information about operation an handling of the API.

2.1. Authentication of WS Users

WS API 2.0 uses the Web Service Security Extension WSSE (http://www.oasis-open.org/committees/wss/documents/WSS-Username-02-0223-merged.pdf) for authentication. WSSE increase security by encrypting passwords. Authentication information is provided as an additional SOAP header with element “UsernameToken”. The UsernameToken element consists of four sub-elements:

The username (sub-element “Username”)
A random but unique value (sub-element “Nonce”)
The encrypted password (sub-element “Password”)
Time stamp of creation (sub-element “Created”)

For a period of time the “nonce” is unique. During that period it can only be used once. A new web service call requires a new nonce.

After this time, the nonce could be reused. To prevent an attacker replaying the web service call, there is an additional timestamp that makes the authentication data valid only for a very short time.

The password is encrypted by using the nonce value and the timestamp.

2.1.1. The timestamp

The timestamp indicates the creation time of the authentication data. In combination with the timestamp, the authentication information is valid only for a short time. After that time, the server rejects the web service call.

It's important, that the timestamp meets the following requirements:

The format is: year (4 digits), “-”, month (2 digits), “-”, day (2 digits), “T”, hour (2 digits, 24 hours), “:”, minutes (2 digits), “:”, seconds (2 digits), “Z”
Timezone is UTC

Example: 2012-03-06T10:55:17Z

2.1.2. Generation of the nonce value

The nonce value can be an arbitrary random value, but it must be ensured, that the value is unique. Most programming languages support the generation of an UUID, the Universally Unique Identifier. Among other things, the UUID is generated by the current timestamp, which helps to get a unique value.

Optionally, the nonce value can be transformed by hashing algorithms like MD5 or SHA1.

Finally, there are two requirements:

The length of the unencoded value must be a multiple of 4 bytes.
In the WSSE element the nonce value is base64-encoded.

The statement that an UUID is unique is only true for the same client, but it is possible, that two clients generate the same UUID. To prevent this each client should add its own (and unique) prefix or suffix.

Here is an example how to handle this in PHP:

$uuid =  uniqid( $prefix.'_', true);
$uuid_md5 = md5( $uuid);
$binary_nonce = substr( $uuid_md5, 0, 20);

This code generates a UUID by using a client-specific prefix, computes the MD5 hash of the UUID and trims the length of the hash down to 20 bytes.

Notes on security:

Because the nonce values is part of the authentication data it is recommended, that the nonce value is a random and non-predictable value.

2.1.3. Encryption of the password

The encrypted password (or password digest) is a simple base64-encoded SHA1 hash of the concatenated value of

the binary nonce value
the timestamp
and the plain-text password

In PHP this is simply done by

$rawDigest = $binary_nonce.$timestamp.$password;
$sha1 = sha1( $rawDigest, true);
$digest = base64_encode( sha1);

Appendix A - Tables shows a table of example authentication data that can be used to verify own implementations of the authentication algorithm.

2.1.4. SOAP Responses and SOAP Faults

After invoking a web service method, the server processes the request and returns the result. This is either a regular SOAP response or – in case of an error – a SOAP fault.

Here are two examples for retrieving the data of a mailing list. The web service method to call is „GetMailinglist“. The first example shows a successful invocation:

<SOAP-ENV:Envelope
    xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Header/>
  <SOAP-ENV:Body>
    <ns2:GetMailinglistResponse
      xmlns:ns2="http://agnitas.org/ws/schemas"
      xmlns:ns3="http://agnitas.com/ws/schemas">
      <ns2:id>12345</ns2:id>
      <ns2:shortname>Mailinglist</ns2:shortname>
      <ns2:description>Example</ns2:description>
    </ns2:GetMailinglistResponse>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

A mailing list with ID 12345 was requested. The response shows, that the mailing list exists, it is named „Mailinglist“ and has a description „Example“.

Here, we specified an invalid mailinglist ID:

<SOAP-ENV:Envelope
    xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Header/>
  <SOAP-ENV:Body>
    <SOAP-ENV:Fault>
      <faultcode>SOAP-ENV:Server</faultcode>
      <faultstring
        xml:lang="en-US">Unknown mailinglist ID</faultstring>
    </SOAP-ENV:Fault>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

This fault message shows, that something went wrong: The error was caused on the server and occurs, because an unknown mailinglist ID was used in the request.

The difference between the response of a successful call and an error is the content of the Body element: While a successful call returns a data structure that depends on the called web service, a SOAP fault returns a Fault element which has a fixed structure.

According to the specification of the web service protocol, there are four different values for faultcode. The important values are:

VersionMismatch: This indicates that the used version of the SOAP protocol is not supported.

Server: An error occurred on the server while processing the request.

Client: The client sent a malformed request.

In most cases, „Server“ indicates errors based on invalid data, like unknown IDs (see second example above). The element fault string provides more detailed information on the error.

Caution: Even if a certain web service method does not deliver a SOAP fault right now according to this documentation, we strongly recommend that your client code is able to handle a SOAP fault message because the same method of a later version of this web service API might provide a SOAP fault and your code should not fail because of that!

Frameworks for web services can be found for nearly every programming language. Amongst other things, these frameworks take care about the result and either return parsed data (in case of a successful call) or indicate a SOAP fault, so a developer of a web service client has not to deal with raw XML code. The most known web service frameworks are Spring-WS, Apache Axis 2 (both for the Java programming language) or the PHP SOAP client, that is included in PHP.

2.1.4.1. Bringing it all together

The authentication information is provided in the SOAP header <wsse:Security>:

<wsse:Security
      xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
      xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <wsse:UsernameToken
      xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
    <wsse:Username>user</wsse:Username>
    <wsse:Password
      Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">Vw4LvcBfBqUXBPkF3tt3wqwoaHs=</wsse:Password>
    <wsse:Nonce
      EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">N2RkYmMxODgxMTY0N2EzNWJiZjA=</wsse:Nonce>
    <wsu:Created
      xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2012-03-06T10:09:47Z</wsu:Created>
  </wsse:UsernameToken>
</wsse:Security>

2.1.5. PHP Client

We offer a basic PHP client (WSSESoapClient.php) providing authentication and the web service methods as open source software. We also provide a small code example of how to use a certain web service method (WS_use_example.php).

Both, client and sample code are available for download at SourceForge: http://sourceforge.net/projects/openemm/files/OpenEMM%20development/

To get a list of all available web service methods you may use this PHP code:

<?php

include("WSSESoapClient.php");

$wsdlURI=''; // Set your webservice URL here
$wsUser=''; // Set your webservice user name here
$wsPassword=''; // Set your webservice password here

$client = new WsseSoapClient($wsdlURI, $wsUser, $wsPassword);

var_dump( $client->__getFunctions());
var_dump( $client->ListMailinglists());

?>

2.2. Permissions

Access to webservices can be granted based on webservice methods or user roles.

2.3. Quota

The number of webservice calls is limited.The server returns a proper SOAP fault, when this limit has been reached.

2.4. The "{http://agnitas.org/ws/schemas}Map" data type

In several programming languages, use of the {http://agnitas.org/ws/schemas}Map data type can cause errors. This mostly concerns programming languages with weak standardization of types, such as PHP. Languages that exhibit strong standardization of types are not affected.

When calling these web services, the data types for keys and values must therefore be explicitly given.

PHP offers support for this by means of the SoapVar class. Correct registration of a recipient corresponds to

$profile = array(
  array(
    "key" => new SoapVar(
   "email", XSD_STRING, "string", "http://www.w3.org/2001/XMLSchema"),
    "value" => new SoapVar(
   "test@example.com", XSD_STRING, "string", "http://www.w3.org/2001/XMLSchema")
  ),
  array(
    "key" => new SoapVar(
   "mailtype", XSD_STRING, "string", "http://www.w3.org/2001/XMLSchema"),
    "value" => new SoapVar(
   0, XSD_STRING, "string", "http://www.w3.org/2001/XMLSchema")
  ),
  array(
    "key" => new SoapVar(
   "gender", XSD_STRING, "string", "http://www.w3.org/2001/XMLSchema"),
    "value" => new SoapVar(
   0, XSD_STRING, "string", "http://www.w3.org/2001/XMLSchema")
  )
);

2.5. Date format

If not stated otherwise, the date format is a ISO 8601 format with this pattern:

yyyy-mm-ddThh:MM:ssZ

where:

yyyy is year, 4 digits
mm is month, 2 digits
dd is day of month, 2 digit
hh is hour, 2 digits, 24h format
MM is minute, 2 digits
ss is second, 2 digits
T and Z are the characters "T" and "Z"

Time zone is always UTC.

Example: 2017-03-08T:13:44:15Z

2.6. Limitations in the bulk web service methods

Special Bulk web service methods permit the calling to a web service with more than one data record (for example AddSubscriberBulk). The number of records per call for this web service method is limited to a maximum of 1000 records. Exceeding this limit results in a Soap Fault.

2.7. Recommendations for the client

2.7.1. Timeout

Although webservices respond in a short time, longer response times may occur due to external influences (server load, amount of data, network bandwidth, ...). Therefore, a client-side timeout of 30 seconds or more is recommended.

Implementations of proper error handling strategies for the case of a timeout are recommended.

2.7.2. Error handling

Ffor the case of an error it is recommended to implement proper strategies for error handling in the client code. These depends on the webservice and use case. Some recommendations are:

Read-only webservices (like GetMailinglist) can be invoked again.
Webservices that alter data may require a check of the actual data before invoking it again. These depends on the webservice, use case and type of modification.
Invoking the webservice again should be delayed. The delay should be increased between consecutive calls ("exponential back-off").

2.8. Common error messages

2.8.1. Server responds with HTTP 404

Cause:

The requested webservice method is unknown.

This usually only occurs with web service clients that are not based on automatic generation of SOAP requests.

Solution:

If possible use a code generator or a framework that processes the WSDL document automatically. This is the recommended procedure.
Check the correct spelling of the webservice method. The name of the corresponding XML element is "<webservice method>Request" (like "<ListMailinglistsRequest" for "ListMailinglists")
Check usage of correct XML namespace for the webservice method.

2.8.2. Authentication of Username Password Token Failed

Cause:

Username or password in the authentication data are wrong.

Solution:

Check username and password.
Check, if password digest is correct for timestamp and nonce value.

2.8.3. Invalid/Repeated Nonce value for Username Token

Cause:

The nonce value speicified in the authentication data was already used in a previous SOAP request.

Solution:

Create a new nonce value for each SOAP request.

2.8.4. The creation time is ahead of the current time

Cause:

The timestamp of the authentication data is in the future.

Solution:

Check used timezone. Only UTC is allowed.
Check time of host. In common the timestamp is determined by the time of the host, that is running the SOAP client.

2.8.5. The creation time is older than currenttime - timestamp-freshness-limit - max-clock-skew

Cause:

The timestamp in the authentication data is outdated. It must not be older than 5 minutes.

Solution:

Create the timestamp directly before using it.
Check its timezone. Only UTC is allowed.
Check time of host. In common the timestamp is determined by the time of the host, that is running the SOAP client.

2.8.6. Access to endpoint denied

Cause:

The webservice user does not have the permission to access the requested webservice method.

Solution:

Use different webservice user with proper permissions or grant access to webservice method to user.

2.8.7. Mailing is not editable. Maybe scheduled or already sent.

Cause:

Client attempts to modify a mailing, which has already been sent or is scheduled for sending.

Solution:

Create a copy of delivered mailing. (Modifications on this copy won't affect the delivered mailing!)
Cancel planned delivery. The mailing can be modified and the modification will be taken into account for delivery.

2.8.8. Quota limit exceeded

Cause:

The maximum allowed number of webservices calls has been reached.

Solution:

Repeat webservice call at a later time.
Retrieve required information only.
Retrieve static data only once and cache this information in your client.
Use an procedure like exponential-backoff to retrieve states (for example: ImportStatus and ExportStatus).