Migrating from Heroku PaaS to AWS
Context.
ADPList is a mentorship platform connecting mentors and mentees. The platform has been around since 2021 and has a network of 20k+ mentors and 160k+ mentees. The tech stack powering the platform is Django and PostgreSQL on the backend and a React App on the frontend. ADPList is hosted on Heroku PaaS. The ADPList platform hosts more than 30k+ mentoring sessions per month.
Problem Statement.
Fix application performance issues such as higher latency (> 8 seconds for 99th percentile) even during non-peak hours. (< 600 requests per minute)
Reduce infrastructure cost. Evaluate the feasibility of migrating from Heroku to AWS and execute it.
Implement centralized observability for all services.
Outcome/Impact.
300% improvement in application performance after our engagement. After migrating from Heroku to AWS, ADPList could handle 3x more traffic without increasing the costs.
We migrated all workloads to AWS to save more than 50% of the cost compared to Heroku.
We enabled centralized observability where all application logs and infrastructure metrics with dashboards are available in a single place.
Solution.
When ADPList approached us, they had an immediate problem at hand. The latency for the business critical APIs was high and caused poor user experience. Before anything else, we started focusing on solving that problem.
The entire stack for ADPList was running on Heroku. To improve performance, the ADPList team already implemented vertical infrastructure scaling. However, this resulted in a massive increase in costs. Apart from the costs, we realized that Heroku isn’t particularly suited well for microservices deployments at ADPList due to:
- Lack of control over infrastructure
- Limited observability for databases
- Inability to debug and correlate events across microservices
With these points in consideration, we decided to migrate the entire application stack to AWS.
Migrating to AWS
The tech stack for ADPList consisted of React frontend, backend API services written in Django, Redis as cache, and PostgreSQL as database.
On Heroku, each backend service was deployed as a standalone Heroku app with its own database (PostgreSQL) and cache (Redis).
We decided to migrate the workloads from Heroku to AWS in the following manner.
- Migrate stateless backend services to AWS
- Migrate database and cache to AWS
1. Migrate backend services to AWS
In this step, we set up AWS VPC using the Well-Architected framework best practices. We set up all backend services on AWS ECS. The backend services continued to connect to the database and cache running on Heroku.
We chose ECS with Fargate over its alternatives (EC2, EKS, etc.) after considering factors such as:
- the learning curve for the team
- ease of operations and control
- cost.
ECS with Fargate provides sufficient abstractions for our use case while keeping the learning curve easier than Kubernetes. It also provides pay-per-use infrastructure and very little operational overhead.
We used Github actions to set up CI/CD pipelines as it was more familiar with devs already. We were able to set up a working staging environment with CI/CD within three weeks. We then load-tested the application and found an interesting problem with how Nginx works with the Fargate. Nginx would cache older container IPs during a deployment or scaling event, which would cause errors. We used dynamic DNS resolution in Nginx to fix the issue.
Finally, after extensive testing, we updated the ADPList frontend to point to backend services deployed on AWS ECS. At this point, we stopped the infrastructure on Heroku for the backend services.
2. Migrate database and cache to AWS
For the Database, ADPList was using PostgreSQL, managed by Heroku. On AWS, we used RDS PostgreSQL as an equivalent. We ran a live replication between Heroku PostgreSQL and AWS RDS to minimize the database downtime. We used Bucardo for this. As a data replication tool, Bucardo works by replicating the data between a source and target database as long as the source and target schema are identical. So we set up live replication using Bucardo after restoring the schema on AWS RDS.
Once AWS RDS replication was running via Bucardo, we updated the configuration for backend services to use the database in AWS instead of Heroku. The switchover was completed without any business impact during non-peak traffic and was done with zero downtime.
After the switchover, we monitored the production environment for a couple of days and announced the retirement of Heroku resources. This entire exercise was completed by 1 engineer in 2 months. It resulted in an overall 50% monthly cost-saving after moving to AWS.
Here’s what Felix, CEO of ADPList says about our work.
“I am extremely pleased with One2N's exceptional expertise in orchestrating our migration from Heroku to AWS. Their proficiency allowed us to seamlessly handle 3x the traffic while ensuring our systems' reliability and performance. We were so impressed with their work that we've decided to engage their ongoing support for further enhancements.”
