Need Help With MySQL Queries in Batches, Chunks, etc for Millions of Rows/Records
Budget: $15 – $25 USD
I need to format, insert, delete and update tons of rows in some big tables (that have millions of rows/records). And, I can't seem to get any of these queries (from the links below) to work, but I'd like someone that knows how to do queries like this.
So far, I've just been running my queries in the SQL tab in PHPMyAdmin with limits of 10,000 or 100,000 or 500,000 or 1,000,000 and waiting for it to finish. And, that's been succssful, but I keep having to go back in and see if the query is done and pasting it again to run it again. So, I need it to be more continuous.
A CRON Job might be good, but I often need to run different update queries, so I really want to just know how to do this with the batch method. I've read that using the PK (primary key) to batch the records to update in an update query is the fastest way. I can also run queries from Terminal (ie. Command Line) in my WHM if someone can teach me how to write the batch query (if that's even a fast way of doing it).
If someone can teach me how to do this stuff myself, then I will probably hire you to help me a few hours or more per day to help me format some of the data in my tables in preparation for a big project.
Here are some of the queries I think might help me if I knew how to run them correctly:
http://mysql.rjweb.org/doc.php/deletebig#deleting_in_chunks
https://stackoverflow.com/questions/51917248/how-to-update-millions-of-records-in-mysql/56220190
https://stackoverflow.com/questions/44635100/mysql-copy-value-from-one-large-table-to-another-large-table-millions-of-rows
So far, I've just been running my queries in the SQL tab in PHPMyAdmin with limits of 10,000 or 100,000 or 500,000 or 1,000,000 and waiting for it to finish. And, that's been succssful, but I keep having to go back in and see if the query is done and pasting it again to run it again. So, I need it to be more continuous.
A CRON Job might be good, but I often need to run different update queries, so I really want to just know how to do this with the batch method. I've read that using the PK (primary key) to batch the records to update in an update query is the fastest way. I can also run queries from Terminal (ie. Command Line) in my WHM if someone can teach me how to write the batch query (if that's even a fast way of doing it).
If someone can teach me how to do this stuff myself, then I will probably hire you to help me a few hours or more per day to help me format some of the data in my tables in preparation for a big project.
Here are some of the queries I think might help me if I knew how to run them correctly:
http://mysql.rjweb.org/doc.php/deletebig#deleting_in_chunks
https://stackoverflow.com/questions/51917248/how-to-update-millions-of-records-in-mysql/56220190
https://stackoverflow.com/questions/44635100/mysql-copy-value-from-one-large-table-to-another-large-table-millions-of-rows