Python scrappers refactoring and development

Background:

I have five scrappers based on python and BeautifulSoup.

3 are implemented as microservices in Flask (with similar template)
2 are part of Django project, but I have plan to migrate it to microservice as well
results of scrapping go to rabbitmq.

SCRAPPERS :

- must use proxy from POST parameter (proxy list coma separated) - mainly imlemented
- return proxy IP if blocked - to implementation
- use beautiful soup preferable

- implement endpoint to process single url and return result synchrously
returns the same JSON payload as standard - TODO

- implement small fixes of scrappers - TODO
- improve performance - TODO
- reduce size of docker container - TODO

- implement endpoint isUpToDate - check status of offer based on URL TODO
if 404 =>
if "archiwalne" => no

- adjust dockerfile to production environment - TODO
- prepare docker-compose to all scrappers run on production - TODO
- deploy over HTTPS and certificate on custom domains on new-VPS using Coolify (or similar)