Optimize flask/waitress Python app to avoid memory-related crash
Budget: $10 – $30 USD
We need help from an expert in optimizing our Python based flask/waitress app for stability.
App: https://github.com/CSS-Electronics/canedge-grafana-backend
Context: The app is deployed on e.g. a PC or AWS EC2 instance. It exposes an endpoint that can be connected to via Grafana dashboards. When a user makes a change in the dashboard, it sends a query to the backend, which fetches meta info and data from disk/S3, processes the data and sends it to Grafana for visualization.
The problem: Currently, the app is stable if users perform "1 update at a time" in Grafana. But if a user sends many requests (e.g. by 'spamming' changes to time periods in Grafana), it will trigger a queue of requests to the backend. In some cases, this results in the app getting killed - in particular when deployed on EC2 with 2 GB RAM. The problem seems to be that too many requests are processed in parallel, causing a memory issue.
Task: We want to retain the same loading speed performance as currently - but ensure that the backend app cannot crash due to 'session request spam'.
To solve this, we believe the flask/waitress settings should be modified. A simplistic method would be to limit the #sessions sent to the backend to 1 - this way out-of-memory crashes should not be possible. However, some of the requests sent to the backend are light-weight "meta info" requests, which should ideally be processed in parallel to ensure speed. Therefore, we assume a better solution would be to somehow restrict the parallel processing based on available memory.
The task is to create a fork of the project, deploy it on AWS EC2 t3.small as per our guide (and link it to Grafana cloud). As step 1, you should verify that you can produce a crash with the current code by spamming the #sessions (a screenshot of this should be sent to us). Next, you should modify the forked code to configure flask/waitress so that a crash cannot happen, but while retaining the same performance as before. Following this, you should validate that the crash is no longer possible. Once we have verified this on our side, the project is complete
We are looking for an expert in flask/waitress and we may have several more projects in this area.
App: https://github.com/CSS-Electronics/canedge-grafana-backend
Context: The app is deployed on e.g. a PC or AWS EC2 instance. It exposes an endpoint that can be connected to via Grafana dashboards. When a user makes a change in the dashboard, it sends a query to the backend, which fetches meta info and data from disk/S3, processes the data and sends it to Grafana for visualization.
The problem: Currently, the app is stable if users perform "1 update at a time" in Grafana. But if a user sends many requests (e.g. by 'spamming' changes to time periods in Grafana), it will trigger a queue of requests to the backend. In some cases, this results in the app getting killed - in particular when deployed on EC2 with 2 GB RAM. The problem seems to be that too many requests are processed in parallel, causing a memory issue.
Task: We want to retain the same loading speed performance as currently - but ensure that the backend app cannot crash due to 'session request spam'.
To solve this, we believe the flask/waitress settings should be modified. A simplistic method would be to limit the #sessions sent to the backend to 1 - this way out-of-memory crashes should not be possible. However, some of the requests sent to the backend are light-weight "meta info" requests, which should ideally be processed in parallel to ensure speed. Therefore, we assume a better solution would be to somehow restrict the parallel processing based on available memory.
The task is to create a fork of the project, deploy it on AWS EC2 t3.small as per our guide (and link it to Grafana cloud). As step 1, you should verify that you can produce a crash with the current code by spamming the #sessions (a screenshot of this should be sent to us). Next, you should modify the forked code to configure flask/waitress so that a crash cannot happen, but while retaining the same performance as before. Following this, you should validate that the crash is no longer possible. Once we have verified this on our side, the project is complete
We are looking for an expert in flask/waitress and we may have several more projects in this area.