Chrome Extension: Ai powered Web Scrapping Assistant -- 2
Budget: $250 – $750 USD
We need a Google Chrome extension that can use OpenAi (Codex. GPT3 etc.) or other Ai and Machine Learning tools to help our users do 3 main activities:
1- Find multiple CSS or xPath locators to ping-point a specific element in the source code of the page by entering the text value of that element, its type, its number of occurrences and specify that text value precedes/follows it. For example:
Target element text result is: https://cdn.images.com/thumbnail.png
Element type: URL
Number of occurrences in the page: 1
Preceded by element text value: Product Image
Followed by element text value: Price
And the Chrome extension will provide 1 or more possible CSS and or xPath locators that reply to the conditions/requirements above.
2- Find the REGEX (JavaScript) rule that can give the expected results. By doing the following:
1- User highlights visually highlight the area of interest in the page (like SelectoGadget extension does)
2- Extension captures the source code of the highlighted area as the string to which the REGEX will be applied
3- User enter the expected result: 89 (for instance this is the available stock on that specific product page)
4- The extension tries and proposes multiple REGEX rules that could give that result in a generic way (stock changes from a product to an other)
3- Find and propose JavaScript actions/code to apply different actions to a string/paragraph such as:
Delete a word of character(s): "Price $" (text field) so user can only keep the digits of the price
Replace a string by an other
Add text before or after a certain element/text
Other most common text manipulation actions (remove html, add/remove decimals, change date format etc.)
In all 3 cases we can have the user star by click and point the area where his target element is so to allow the extension to know which part of the source code to look into. In all 3 cases we shall parse the whole source code so we can build the full xPath and also be able to look into all the DOM elements (including at a later stage being able to deal with shadow DOMs).
This tool is targeted as non-tech users who cannot understand how a CSS locator can be built or how REGEX rules work. They need to be able to copy the different options given by the extension and paste them into our Web Scrapping tool to test/see if it works. Later we can study a deep linking option to feed that data directly into our software.
You can use open source existing project and adapt them. We strongly recommend using OpenAi APIs/tools or other similar services.
We are open to other ideas/techniques that could allow non-tech people to find or auto-generate the Css/xPath/Regex/JS code they need to target a specific element that they can see visually on the page but cannot recognize/reach-to in the source code.
1- Find multiple CSS or xPath locators to ping-point a specific element in the source code of the page by entering the text value of that element, its type, its number of occurrences and specify that text value precedes/follows it. For example:
Target element text result is: https://cdn.images.com/thumbnail.png
Element type: URL
Number of occurrences in the page: 1
Preceded by element text value: Product Image
Followed by element text value: Price
And the Chrome extension will provide 1 or more possible CSS and or xPath locators that reply to the conditions/requirements above.
2- Find the REGEX (JavaScript) rule that can give the expected results. By doing the following:
1- User highlights visually highlight the area of interest in the page (like SelectoGadget extension does)
2- Extension captures the source code of the highlighted area as the string to which the REGEX will be applied
3- User enter the expected result: 89 (for instance this is the available stock on that specific product page)
4- The extension tries and proposes multiple REGEX rules that could give that result in a generic way (stock changes from a product to an other)
3- Find and propose JavaScript actions/code to apply different actions to a string/paragraph such as:
Delete a word of character(s): "Price $" (text field) so user can only keep the digits of the price
Replace a string by an other
Add text before or after a certain element/text
Other most common text manipulation actions (remove html, add/remove decimals, change date format etc.)
In all 3 cases we can have the user star by click and point the area where his target element is so to allow the extension to know which part of the source code to look into. In all 3 cases we shall parse the whole source code so we can build the full xPath and also be able to look into all the DOM elements (including at a later stage being able to deal with shadow DOMs).
This tool is targeted as non-tech users who cannot understand how a CSS locator can be built or how REGEX rules work. They need to be able to copy the different options given by the extension and paste them into our Web Scrapping tool to test/see if it works. Later we can study a deep linking option to feed that data directly into our software.
You can use open source existing project and adapt them. We strongly recommend using OpenAi APIs/tools or other similar services.
We are open to other ideas/techniques that could allow non-tech people to find or auto-generate the Css/xPath/Regex/JS code they need to target a specific element that they can see visually on the page but cannot recognize/reach-to in the source code.