August 11, 2023
In this article, I will take you on a journey of how to integrate directly with Zambian Mobile Network Operators (MNO) to be able to provide SMS service to Zambian citizens.
On the day when we were starting with this integration, everything was looking straightforward, however, after a few weeks reality became way more surprising, and we were struggling with this integration for a couple of months. To explain why it took more than I planned, I will try to break this topic into steps and explain what points you need to achieve to be able to send and receive SMS in Zambia for your end users.
Let's start with the context of our goal.
As a tech provider to NGO organizations, we have set up a goal to make the Talk To Loop platform available to every Zambian citizen. During the research, we learned that Zambians most trust and use in majority SMS as a default contact and communication channel.
Our requirements to achieve the given goal and make service for every user were as follows:
These are critical points that we set to achieve to reach minority groups in Zambia comfortably and free of charge, allow sharing their feedback about the service they receive from a local community and international NGO. At first, it looks straightforward and easy to achieve but trust me - it isn't ;) So let's dive into it!
During the early research phase, we found and considered multiple options to use telco aggregators, where Africa’s Talking is one of the options. They indeed offer fast integration, however, in our case, they do not cover integration with all major MNOs and apply to 2 out of 3 main telecommunication providers. A strong advantage is that they provide a sandbox environment that makes integration by itself less painful and is available via HTTP REST API. However, there are a few drawbacks that were feature killers for us to use. Firstly, we had to be available within 3 MNOs as mentioned just above, that is MTN, Zamtel and Airtel.
Secondly, we have very restricted requirements in the context of data security, and in that time (that is 2022), Africa’s Talking didn’t provide data retention, and within API we were not able to delete them from their servers. That is not acceptable within our policy. Also, unfortunately, they didn’t offer reverse-charge to provide toll-free service for the recipient of aid.
The last key element that blocked us to use 3rd party provider was the cost of integration and in the next step - long-term maintenance costs.
As already mentioned, there are a few other aggregators, however, they also share some of the issues above that are blockers for us to go with them.
Thus, to provide SMS service across the entire Zambia instead of using 3rd party SMS provider, we decided to make a direct integration with each Zambian telco provider. In Zambia, you can list 3 leading Mobile Network Operators: MTN, Zamtel and Airtel. To integrate directly with each of them, the first thing you need to do is sign up a contract.
Next, we wanted to provide service available within e.g. 4-digit number. To make it possible, we had to request a short code from ZICTA. ZICTA is a Zambia Information and Communications Technology Authority responsible for regulating the ICT industry in Zambia.Now we are reaching breaking, namely to obtain a short-code from ZICTA or to integrate with any MNO we had to sign up a contract with all of them, however, if your organization is not registered in Zambia, you won't be able to do anything. Your route ends sadly here.
In our case and thanks to our network, we are with Zambian Governance Foundation in partnership and Loop is hosted in Zambia by them. ZGF completed the setup and signed contracts with ZICTA, MTN, Zamtel, and Airtel, allowing us to obtain our own short code: 4343.
From now on, the formal phase was completed, and we could start the technical part of the integration.
Technical integration we scoped down to 3 individual integrations that we did with each MNO individually. Each of these 3 integrations in general is similar at a high level by definition and could be divided into 2 steps:
This PDU is used to test network connection and acknowledge that peer bind still has an active session.
TIP: We learned that the Zamtel connection works perfectly fine, whereas the MTN session is closed after a few seconds. That is why this param is mandatory to set, and this is what we did.
Here you can find a bit more information about standards and the length of SMS depending on the standard.
By default, each Mobile Network Operator, including MTN, Zamtel, and Airtel, is handled and delivered to the SMPP client (in this case, us) only if the SMS size is smaller than 255 octets. To handle up to 255 octets of SMS we use short_message PDU. However, nowadays nearly everyone owns a smartphone and the majority of the world uses Android devices. The problem is that e.g. Android automatically converts long SMS to MMS. So we had to collaborate with telco providers to convert received MMS into multipart SMS to send to us via SMPP using message_payload TLV.
Over time, using this connection, we have noticed that from time to time MNO side was breaking tunnels and the entire connection was down. This is a very tricky issue to identify and apply the right solution to allow in our case StrongSwan to negotiate a new IPSec tunnel connection. E.g. one MNO config to keep the tunnel alive has to be:
Where, on the other hand, other MNO config to keep the tunnel alive has to be:
As you can notice, these configs differ from each other, and we learned that it depends on MNO IPSec side configuration. What was very helpful to us to set it right was the official StrongSwan IPSec connection Wiki, and it may be helpful to you to dive into StrongSwan charon deamon.
This is crucial and for me, who manage the entire integration process, this was a completely new experience. Comparing Zambian engineers to the Western world taught me new culture and different approaches to work. It isn't easy to set up a meeting with these guys for some reason, however, when you finally make it happen people are super friendly and helpful! Thanks to the support, we could accomplish this integration, but you have to take a lot of patience to find and set up a slot!
Also in the context of long-term maintenance from time to time SMPP connection went down for some reason on the MNO side, and we had to define a process to request to act and re-activate the connection. This is very crucial to define such a process and your tech focal points to make rising issues smooth and efficient to keep your service alive, which we successfully achieved.
Depending on your need you may consider going with some 3rd party telco aggregator like Africa’s Talking, just check all pros and cons according to your requirements, especially in the context of data security, data retention, who has access to this data and where is it stored.
In our case due to legal and security requirements, there was one option - go directly with each MNO, that is MTN, Zamtel and Airtel. It was quite a wobbly road, I have to admit, where we were struggling with many different challenges. Starting from formal, contractual points on the road map, that thankfully our great ZGF partner was able to comply, through tech challenges. The tech part was quite demanding on us and thankfully my in-house engineers with a huge experience, knowledge, and open mind in combination with Zambian MNO support after months of extensive collaboration we completed and were able to release a new SMS service to the recipient of Aid.
If you are interested in such integration in Zambia or any other country or you want to exchange your experience, I will be more than happy to chat with you!
Marek Wrzosowski
