Microservices show tremendous promise in enabling agility and improving resiliency. But a blind rush into a microservices architecture can also create more problems than it solves, said Shane Russell, a developer with the U.S. Digital Service, at the O'Reilly Fluent Conference in San Jose, Calif. His group started down the path of standardizing on a microservice approach for a Veterans Administration application when they discovered the constraints of microservices.
Russell's team was much too aggressive in breaking up a large Rails application into multiple deployable units from the start. This led to paying a large overhead in ways they had not anticipated. "Sometimes, the strategy that benefits organizations that are small don't benefit those that are big," Russell said.
After some initial research on microservices, Russell's team decided to take a microservice approach to new features to the VA's disability processing app. He found it was not a binary decision, but really involved a spectrum of choices regarding how monolithic or decentralized the architecture would be.
Make small things, but not necessarily small services
The starting application was built on Ruby on Rails and consisted of three pages that connected to a back-end legacy system. The team decided to pursue a microservices approach in creating a new feature to support a different workflow. "We did this because there is this deep-seated intuition that really good architecture involves making small things," Russell said.
Russell said creating small methods and small classes are well-established best practices for creating good application architectures. It followed that small services should also be a good thing, too. His team pursued a microservice approach for building a third service, as well. But when it came time to implement a fourth service as a microservice, the architecture was creating new problems. They found the monolithic architecture was easier to change in different ways than microservices and had less operational complexity.
Different kinds of change
Microservices do make some changes easier. Developers are free to adopt languages for implementing a specific service. Microservices also make it possible to leverage different data stores to get the best performance for a particular service. But a monolithic architecture makes it easier to make global changes that would otherwise affect every microservice.
For example, Russell's team wanted to change the format of application logs to add usernames. They had to adjust the code in all three services to make this change, and even now the logic is a little different across the services. They also wanted to take advantage of a new error-tracking system built on Sentry. But this transition required generating multiple API keys and deploying them in each service.
"Microservices make it easier to change one service without affecting others, but they make changes to the platform more difficult. And if you try and maintain a strong consistency between services, that will really frustrate you," Russell said.
Functional integration harder with microservices
Another big challenge was refactoring functional integration between features. "We had to do this once, and it is a massive pain. Small methods are amazing to refactor, and small classes [are] the same thing. Small services end up being a lot of pain, because you have to deploy the services at different times, and you have to make sure the APIs are stable. There is a lot of room for bugs."
With a monolithic application, developers only have to change the class name and API. With microservices, developers have to change the version number of the API and maintain multiple APIs for backward compatibility.
Monoliths can reduce operational complexity
Leading-edge microservice architecture needs to implement several architectural features to ensure failure states are handled correctly. One big source of pain is caching data to ensure consistency when errors occur. Russell observed that errors in an application like Netflix are not always critical. Users might be annoyed if the wrong movie pops up when they click on a link. But for the VA, an error might mean a veteran is not able to procure the medical services to which he is entitled.
Russell's team found they were doing a considerable amount of work to isolate errors that arose across different versions of the various microservices. This made it difficult to maintain a high level of test coverage. They also found they were doing considerable work on creating automated scripts for provisioning databases and caches across different versions of the microservice approach.
In the government realm, many things are not even possible to automate. Government systems require a paper-based procurement process for requesting domain-name-system entries, requesting certification and requesting a new IP address. As a result of this overhead, the payoff of a microservice approach was not big enough.
Weigh the benefits of different approaches
Russell acknowledged there are still a lot of benefits to a microservices-based architecture, like increased fault tolerance, independent deployments and rollbacks, forced modularity, and the ability to leverage different technology stacks. They can make it easier to break a large organization into more efficient teams.
Their monolithic application is slowly growing. Russell said the biggest pain is the build time is increasing about 10 minutes per year. At this rate, it may take about an hour for new builds in another two years. But then they might find ways to parallelize the build in the cloud when this becomes too excessive.
Another problem is a missing index or memory leak can bring down the whole app. There have a few instances where they had to roll everything back. Russell said this would not have been a problem with microservices. But even with a monolith, Russell's team is still writing code in a modular way using small methods and classes. Over time, they might be able to pull the most stable components out into microservices with more confidence.
Why microservices is for everyone
Challenges with .NET microservices architecture
Guide to microservices and containers