Runko Works

Case study

A performance-critical message aggregator, delivered as a fast PoC

An extra service for a client, delivered as a proof of concept in a very short time and faster than required by an order of magnitude.

Client
Telecom test and monitoring vendor, for its customer
Role
Lead developer and co-architect
Period
2021
Area
Platforms & data
Stack
Python · PostgreSQL · multithreading · microservices · REST · LDAP

Why it mattered

One of the vendor’s clients needed an additional service: take in a large stream of messages, store them, send mass messages on demand and enrich them with data from the client’s directory. It had to work fast, and the client needed to see it working soon.

The proof of concept arrived in a very short time and met the performance requirements with an order of magnitude to spare. The cooperation with the client continued.

How it works

  • Requirements and architecture worked out with the client, together with colleagues.
  • A stateless, multi-threaded application. Python 3 services whose workers keep nothing between two steps, so the load spreads over parallel threads.
  • PostgreSQL at the centre. What the service knows lives in one database: the messages that came in and the ones to send. The workers meet there and nowhere else.
  • Queries in batches of 10,000. For speed, the database is read and written ten thousand messages at a time instead of row by row.
  • Integration with the client’s REST API and its LDAP directory.