A mobile-first CRM company operating at high scale was using an API compute cluster to handle massive traffic from over 2 billion devices, totaling 6,000 queries per second. The API cluster was substantial, comprising hundreds of instances and contributing around $10,000 each month to the AWS bill. As the company grew, controlling these costs became increasingly important. When engineers started investigating, they noticed that the API service was fairly well optimized and using the full capacity of the hardware. Despite their efforts to apply common cost-saving techniques such as autoscaling, they were not able to make a significant dent in the cloud bill.

Due to the focus on rapid development and easy testability during the 0-to-1 phase, there are often simple yet highly impactful opportunities hidden beneath the surface that are tricky to find. Our reverse engineering approach helps us to swiftly identify these opportunities so we can deliver high cost savings with precision. Using this approach, we were able to pinpoint a solution to reduce the excessive API traffic and, in turn, the size of the API cluster. We discovered that adding a simple cache and cursor management on the mobile app would allow it to function without excessively calling the API service.

The key to effective cost optimization is identifying solutions that are both impactful and simple to implement. The suggested change resulted in a substantial drop in API traffic and a 60-70% reduction in compute costs. This is also an example of hidden cost optimization opportunities that cannot be uncovered without the right expertise.