Assistance with Web Scapping Flight data from a Travel Website with Anti-Bot Protection

Job ID: 39736408

Budget: $30 – $250 USD

Project Description
I need assistance in developing a robust web scraping script to extract flight price information from the Smiles website. The goal is to obtain accurate and up-to-date pricing data for analysis.
My current script is already developed, but it's encountering a persistent issue: the website's security tool (Akamai Bot Manager) is blocking API requests with an HTTP 406 - Not Acceptable error. This happens despite sending what appear to be the correct headers and cookies.
The Problem
* The Akamai security system detects that the requests are coming from a bot.
* The server requires dynamic session cookies (_abck, bm_sv) that are generated by complex JavaScript code executed in the browser.
* Solutions based on the Python requests library have failed, as they cannot replicate the full browser environment or generate these cookies with a valid signature.
What We've Already Tried
* Scraping with requests using hardcoded headers and cookies copied from a live browser session.
* We have confirmed that the 406 error occurs because the security cookie token is not being generated correctly.
Ideal Skills and Experience
* Proven experience in bypassing anti-bot systems like Akamai, Cloudflare, or Imperva.
* Proficiency with browser automation tools such as Playwright or Selenium, or specialized scraping APIs designed for these challenges (Scrapfly or ScrapingBee).
* Ability to handle sites with dynamic content and JavaScript.
* Capability to develop a fast and efficient script for multiple searches (e.g., by reusing a single browser session).
* Strong attention to detail to ensure data accuracy.
Note: It's crucial that candidates understand the nature of the Akamai problem and do not propose simple solutions that we have already tried.