This content is part of the Essential Guide: The world of microservices and container technologies

How Runscope overcame microservices challenges

Learn how Runscope overcame a number of challenges surrounding deployment and service discovery work once the company began using microservices.

Microservices combine the benefits of SOA and DevOps to allow an enterprise to deliver faster updates and new features. But enterprises also face a number of key challenges in making this type of architecture work well. "Microservices are an evolution of SOA with DevOps and tools to solve problems," said Ryan Park, principal software engineer at Runscope, at the Fluent Conference in San Francisco.

A microservices approach allows Runscope to effortlessly push out 25 product deployments and 25 test deployments per day. The architecture also makes it possible to easily roll back changes if problems occur.

Runscope wrote its first code in January 2013, starting with its identity services. "We were lucky we got to do this green field," said John Sheehan, co-founder and CEO of Runscope. "A lot of this comes from the luxury of not having any legacy baggage." Conversely, organizations with legacy applications face several challenges in this sense.

Scalability as a fore thought

Building this kind of infrastructure and breaking services into pieces makes it possible to scale more easily when required. When a service is overwhelmed, more boxes are easily provisioned. Breaking the infrastructure into microservices also makes it possible to use the hardware more efficiently. This entails trying to avoid gaining monolithic applications. Park said, "We are able to put resources at the places that need them, and this has been beneficial to scaling out, versus constantly growing monolithic applications."

In this manner, Runscope's main application could be distributed across AWS and Rackspace. Good service boundaries allow Runscope to swap out data stores without affecting services. An added benefit is that this approach also provides a measure of network resiliency against faults or problems that occur across the Internet.

A microservice architecture has also helped Runscope to isolate breakage. "The lower criticality services don't cause system criticality," said Sheehan. In the past, if the email provider was down, it would throw an exception for other services. Now when one part dies, the controller spins up a new one and the rest of the system can continue to operate without worrying about any one piece being broken. One big exception is identity, which restricts users from being able to log in across the entire ecosystem of microservices.

Simplifying development

Microservices also reduces the cognitive overhead for shipping changes. With one large app, developers need to understand the entirety of the application. The lack of service boundaries in this monolithic approach can have far reaching effects for any change. With microservices, the developer only needs to understand the API contract.

A microservice approach also simplifies the programming constructs for working with data. Using traditional approaches, developers have to familiarize themselves with different databases like Postgres, SQL, Redis and the various protocols to support them.

The move towards Microservices has also made it easier to scale the team. First it put Runscope into a network mindset by building a lot of small services around HTTP, and forced developers to think about releasing new applications to support the architecture. App developers want to focus on adding user value, said Sheehan. Defining a uniform interface reduces what developers need to understand and makes it possible to work across programming languages.

Streamline service creation

Along the way, Runscope encountered challenges around service creation, service discovery and automated deployment and manufactured tools to address these. For instance, they built a service creation library called Smart Service that sits on top of Flask. The library includes a number of functions required for implementing services including a built-in health check, common logging and metrics, and simplified dependency management. The library is also language agnostic and supports integration using common standards like HTTP, syslog and statsD.

Microservices are an evolution of SOA with DevOps and tools to solve problems.
Ryan Park, principal software engineer at Runscope

Another benefit of Smart Service was that it could retry certain types of operations automatically such as Get, Put and Delete. If something failed on one host, a call could automatically be directed to another host without skipping a beat. But developers might not get all of the benefits of Smart Service if they write in Go or another language. As the growth of new languages meets a critical threshold, then enterprises should consider tooling for that language as well, said Park.

Make service consumption easy

Runscope was having to point to the service running the user identity service, which proved brittle. Python provides a great HTTP client library that is simple to use, said Park. But this still involves writing a lot of code to find the host a service lives on. So they built another tool called Smart Client on top of the Python request library. This helps to encapsulate the discovery logic for new microservices in a standard way.

Smart Client also helps when an application needs to make more than one service call to perform a unit of work. For example, calls for user ID and group ID can occur in parallel. The service call will not block until the application accesses a method in the response. This improves the responsiveness of code that needs to make a number of microservice calls. The application can start all of the calls together and then wait for the first response.

The other benefit of this design is that it makes it easier to code. Developers only need to use normal functions, rather than dealing with callbacks in order to get the benefits of parallel service calls.

Automate deployment for service discovery

Runscope also built an internal tool called Prometheus for deploying code branches. Prometheus automatically generates live updates, and provides a history to enable one click rollback. In addition, it allows Runscope to use existing tools like HTTP load balancers so they could bring up new services in parallel with older ones. Over time, Prometheus was broken up into a number of other services to help improve the deployment process. Now Prometheus is configured to talk to an internal separate cluster configuration service called Atlas.

 A lot of work was required on the API design to make sure it would work from the front end and across all points in the infrastructure. The configuration data sits behind Atlas in DynamicDB and Apache ZooKeeper, making it easy to see which hosts are running a given service.

But Zookeeper proved brittle and did not scale well enough to support thousands of processes and connections, said Park. The microservices making service calls don't try and contact hosts directly. They instead talk to HAProxy, which makes the call to the service. This approach allows the use of mature HTTP tools for load balancing. It also keeps Smart Client and the service discovery simple regardless of the process for trying to talk to a service. It is also resilient to disruptions in ZooKeeper. If ZooKeeper goes down, then HAProxy can insert new changes moving forward.

Every enterprise has different needs for implementing microservices. The specific features that Runscope implemented are not necessarily important for every company, said Park. But taking the time to build these features in advance can save a lot of development effort in the long run.

Next Steps

Should you be rethinking modular programming interfaces for microservices?


Dig Deeper on Enterprise application integration