Recently I have been reading “Building Microservices Designing Fine Grained Systems” by Sam Newman. Indeed it is a must read book if you are trying to move toward microservices. This book is going to influence and reflect a lot at my work, so I was thinking I should definitely write a reflection blog. This is what I am attempting to do here. Obviously my point of view will differ in terms of many things, I will be explaining my take aways from the book and I will be trying to reflect my point of view and experience as well. Sam Newman is not a fanboy like me, so he expressed his caution while using docker, but I am a fanboy of docker/kubernetes so obviously I would have whole different perspective how docker makes managing orchestrating microservices easier or where and how docker/kubernetes fits in. Maybe I am going to write about it in another blog.
I agree with Sam Newman when he compared our software industry with fashion industry. We are easily dominated by tech trends. We tend to follow what other big organization is doing. But it has its reasons as well. This big organizations who are setting trends, they are doing a lot of experiments, and publishing their findings and some time publishing few open source tools along with it, as a biproduct of their work and smaller organization like us, most of the time we don’t get the luxury to experiment in that large scale, we don’t have the luxury to innovate, but we can easily become the consumer of those tools and we can easily take leverage of the knowledge that has been shared with us. So we take the easy route.
One of the key thing about microservices is that, every service needs to have some form of autonomy, it can depend on other service for data or information that is not within the boundary or context, but it has to have its own data source that it owns, where it can store and retrieve data that is within the boundary or the context of the service. It can take leverage the best programming language that matches with its goal, it can use the database that matches with its need. Independence of tooling is a great feature, it reduces the tech debts in a very positive way. That’s probably the main selling point of microservices.
At a service oriented architecture, maybe it can be a common practice to directly access database that other service is primarily using, the reason is very simple I need a data that the other service has, so I am accessing the data that I need. But what happens when the owner of the service change its schema for representing their data better? The other service that is consuming the data of that service locks up, the other services which were consuming the data won’t be able to consume the data that it absolutely needs. So maybe, we need less coupling in our services then what we are already got used when we were using service oriented architecture. The best way that we can think of, maybe we can do that using REST APIs, when we would need a data from other service we would request the data using API calls to the service who owns the data and we don’t know the internal details about how the data is being stored. We need the data not its internal representation. Less coupling allows us to change the internal representation of the things without impacting the expected result that it needs to produce. The import term here is, it allows us to improve and change without going through huge hassles and long meetings with stakeholders who runs other services that will be affected for the change in your internal representation. Not only database, usually the system that has been designed to make RPC calls to change or to collect resources of other services are also prone to this problem.
When it is microservice we are talking about, it is easy to imagine that there is a huge code base with millions of lines of code that does pretty much everything on its own, it is struggling with load and the codebase is getting out of hand and almost unmanageable and we don’t want that so we want to move out of it. We want to split this huge monolith into few manageable services with manageable loads. Now dividing a monolith is not always an easy task. Setting a boundary is often harder but every service needs to have a well defined boundary. When dividing monolith we can follow a simple rule of thumb that says code that changes together will stay together which is known as high cohesion and the codes that does not change together may split apart. Then we can further split it based on domain, functionality, special technical needs. For example maybe a facebook like friends network can leverage a graph structured database, maybe neo4j or graphql in that case it deserves to be separated as a different service that use the appropriate technical tools that it needs.
Service to service communication in microservice is tricky, many people are using trusted network based security where the idea is, we try out best (invests more) to secure our parameter/border, it is hard to get into the parameter but once you are inside the parameter we trust you with everything. For organization like this few extra header that defines username is enough. But there are some other organization they are not being able to guarantee with 100% accuracy that their parameters are secured, there are so many hacking techniques as well that allow them to breach the boundary pretending to be someone else. So maybe we need to maintain some form of security and privacy even when we are communicating inside our secured parameter, from one service to another service. All internal communication can be encrypted using some form of HTTPS which encrypts the message and verify that the message source and destinations are accurate. Some organizations are relying on JWT as it is very light weight but still every service needs to have the capability to verify that token. A shared token has to be shared between the services and it has to be done using secured fashion.
When deciding between orchestration and choreography, choreography should be the best pattern because otherwise if we try to orchestrate by one service, we are basically overpowering it compared to other services that eventually it will start becoming a god service that will keep dominating other services all the time. We don’t want service to service domination. We want equal distribution of load and responsibility and almost equal importance. Also it has its problems as well, because in a microservice world new services start to appear everyday and old services gets discarded everyday it becomes real hassle for the owner of the god service to track them and to add and remove those service calls to our god service. Rather than a service who is making rest api calls to notify other services to do certain things, we can fire events and any other service who might be interested on that event will subscribe to that event and play its part when the event is triggered. If new service comes in, it is easier for it as well to subscribe to that event. It can be implemented using kafka or maybe Amazon SQS.
In microservice world it is extremely important to isolate the bug, the bug from one service tends to cripple other services. So it often becomes difficult to identify the real bug in appropriate service. So we need to have proper testing mechanism that that will isolate a service while testing and at the time of testing other api calls that it makes to other services needed to be mocked so it needs to have proper setup for isolated testing. So we would servers that is capable to imitate the ideal behavior of other service. When testing, it would be wise to have another environment that has its own set of database, in that way we would be able to test how resource creation is responding on a particular service. We talked about isolated testing, but integration testings are also important. As in micro services, there are so many moving pieces, maybe a service is calling an api that has been removed or the url has changed for some reason, (maybe the owner of the service has forgot to inform other stakeholdsrs) in that case the service that is calling the api, it won’t be able to perform what it was supposed to do. So integration tests are important, it is going to test when integrated together it is still performing the way it should be performing.
Every service needs to get monitored and there has to be a set of action that would take place when a service is down. For example maybe there can be a sudden bug introduced in one service causing huge CPU or memory can make the service go offline. It can affect the other service that is running on that machine. It is usually recommended that one service is being hosted on one machine so that the failure on one service does not cascade the problem to another service. There also should a well placed rule or circuit breaker on when are we going to consider a service is down and when are we going to stop sending traffic to a service. Suppose one of are service is down, in that case another thing to note here is that is it going to be wise to show the whole system is down when there is only one service that is down out of maybe 100s of services. Instead of doing that we can replace a portion of website with something else. For example when the ordering service of an ecommerce is down. It won’t be wise to shut the whole website down, we can let user visit the site and rather than letting them click “place order” button we can show them “call our customer care to place order”. So if implemented wisely micro service can reduce our embarrassment. Netflix has implemented an opensource smart router called zulus that can open and close a feature to group of users while the rest of the users get that is default. It can come handy time to time as well. There are many tools out there to monitor servers, from opensource world we can use nagios, or we can use 3rd party monitoring tools like datadog, new relic, cloudwatch and the options are many.
When talking about monitoring, we also need to be conscious that sometime our app is going to pass all the health checks yet it will produce a lot of error. For such situations maybe we need to monitor the logs, maybe a centralized log aggregation and monitoring system (famous ELK stack) can be helpful. As we are dealing with microservice, I am sure we have so many servers and it won’t be wise to poke in every server to collect logs when there is crisis, a log aggregation platform is a necessity, that’s how I want to put it. Our log aggregation system needs to be intelligent to fire an alarm when it reaches a certain number of error messages. Depending on the severity of error message, maybe it is time to let the circuit breaker know that we don’t want any traffic on that service until it has been sorted out.
In microservices we are dealing with 100s and 1000s of services, so there will always be some sort of deployment going on, either it is on this service or that service. If DevOps culture is being adopted small changes will be incouraged to go on production. The philosophy is if a bug is introduced due to small change, it will be easier to spot and easier to fix compared to a huge chunk of code. As each services are interconnected, if a service go down even for a second other services gets affected so zero downtime is a must. We can use blue green deployment or cannery deployment strategy to achieve zero downtime thingy and it is absolutely important. If something goes wrong in a deployment there has to have a well defined strategy to rollback to previous build. It is better if it has a CI/CD pipeline attached with it. Chef, Puppet like server automation tools comes handy because it helps us to automate the monotonous tasks. When automation scripts are being written, it is being tested over and over again, it is faster and more reliable, same goes for rollback.
Micorservices makes security much easier, as we know from CIA triangle, we can’t have everything. Often when we want to ensure security, we will need to slow our systems down as a trade off. Now, as we are splitting our services, does all our services needs equal amount of security be ensured (read, does all our services needs get slow?). Often some services need more security than other. So microservices will give us opportunity to use the write security where security is needed, so it won’t slow the whole server down, when a specific service needs special security measure.
Sometime DNS management gets troublesome. It takes hours to get everything propagated across the servers. It can When we are dealing with interconnected services (and there are 100s of them) it will be really troublesome if server to server DNS records are not being reflected the same way, so we can have our own service discovery mechanism, we can have our own DNS server. We can take advantage of Zookeeper, Consul, Eureka and so on.
Scaling microservices can be challenging some time. It is said that people use 20% of the feature 80% of the time, in other word 80% of the feature are never been used. So when we have a heavy load, linearly increasing all the servers most of the time will be a wrong choice. So we need to setup more complex scaling techniques that needs to be implemented on each service separately. Our load balancers are getting smarter everyday so it won’t be too hard to implement.
Newman has pointed out few interesting things about implementation of microservices, one of the key point he has mentioned is about Conway’s Law. If the organization is not organized, and works independently in terms of team to department and collaboratively with other department like a microservice, it would be challenging for that organization to adopt microservices. So it makes me wonder maybe for smaller startup, maybe microservices are not the solution they should look for. Also when it comes to the ownership of service, in smaller organization maybe the same person or team is going to own all of the services. It is going to be a huge overhead as well.
Sadaf Noor 