Automate Power Grid Data Collection (Script)
Budget: $10 – $30 USD
I’m working on a dashboard that displays technical data related to the U.S. power grid—specifically interconnection queue data from utilities and ISOs. I’ve already collected and cleaned a large portion of this data manually using Python and stored it in a structured database.
Now, I’m looking to automate the data collection process from some of the more complex utility websites. These pages contain structured tables (often paginated) but are sometimes behind basic protections or embedded in iframes, like:
• Southern Company OASIS: https://www.southerncompany.info/power-bi/transmission-policy-services/index.html
• OATI OASIS (e.g., Duke): http://www.oasis.oati.com/duk/index.html
The files are called generation interconnection
Your expertise in scraping protected or JS-rendered sites would be very helpful here. I’ll start with 2 of the more complex sites. I want the *scripts* of web scraping because i have 30 webpages with similar format that I want to do the same for.
Let me know if you’d be interested, and I can share further details.
Now, I’m looking to automate the data collection process from some of the more complex utility websites. These pages contain structured tables (often paginated) but are sometimes behind basic protections or embedded in iframes, like:
• Southern Company OASIS: https://www.southerncompany.info/power-bi/transmission-policy-services/index.html
• OATI OASIS (e.g., Duke): http://www.oasis.oati.com/duk/index.html
The files are called generation interconnection
Your expertise in scraping protected or JS-rendered sites would be very helpful here. I’ll start with 2 of the more complex sites. I want the *scripts* of web scraping because i have 30 webpages with similar format that I want to do the same for.
Let me know if you’d be interested, and I can share further details.