Database Management System and Website Scraper/Spider
Budget: $750 – $1,500 USD
I need a web based application to perform the following:
Database Management
- View database records on an ever expanding 8 million record relational database
- Store and manage photos associated with database records
- Search database records based on various database fields/taxonomies
- Interface with Python website scraping script
- Allow database record update via scraping individual website pages/records
- Programmed in Laravel
- Capable of running on shared host/VPS/Bare metal server
- Database management capable of creating new tables/records based on existing database records.
Admins: Can edit database records
Users: Can search database records and mark them as requiring update
Webscraping Script
- Coded in Python (this already exists, may need updating)
- Scrape individual website pages and multiple pages
- Scrape associated photos on individual pages
- Store results in database
- Have basic error handling capabilities
- Data logging errors during scraping
- Proxy management
Current database design is set but may need to be reviewed and changed based on project needs.
Database Management
- View database records on an ever expanding 8 million record relational database
- Store and manage photos associated with database records
- Search database records based on various database fields/taxonomies
- Interface with Python website scraping script
- Allow database record update via scraping individual website pages/records
- Programmed in Laravel
- Capable of running on shared host/VPS/Bare metal server
- Database management capable of creating new tables/records based on existing database records.
Admins: Can edit database records
Users: Can search database records and mark them as requiring update
Webscraping Script
- Coded in Python (this already exists, may need updating)
- Scrape individual website pages and multiple pages
- Scrape associated photos on individual pages
- Store results in database
- Have basic error handling capabilities
- Data logging errors during scraping
- Proxy management
Current database design is set but may need to be reviewed and changed based on project needs.