Database Management System and Website Scraper/Spider

Job ID: 32353205

Budget: $750 – $1,500 USD

I need a web based application to perform the following:

Database Management

- View database records on an ever expanding 8 million record relational database
- Store and manage photos associated with database records
- Search database records based on various database fields/taxonomies
- Interface with Python website scraping script
- Allow database record update via scraping individual website pages/records
- Programmed in Laravel
- Capable of running on shared host/VPS/Bare metal server
- Database management capable of creating new tables/records based on existing database records.

Admins: Can edit database records
Users: Can search database records and mark them as requiring update

Webscraping Script

- Coded in Python (this already exists, may need updating)
- Scrape individual website pages and multiple pages
- Scrape associated photos on individual pages
- Store results in database
- Have basic error handling capabilities
- Data logging errors during scraping
- Proxy management

Current database design is set but may need to be reviewed and changed based on project needs.
Related categories: PHP Python Web Scraping MySQL Elasticsearch