Unclaimed Property Data Extraction -- 2

Retrieve fields
• Owner
• Last Known Address
• City
• State
• Zip Code
• Reported By
• Amount
• Property ID
Process:
Every 24 hours at 5pm
1. Navigate to: https://www.unclaimed-property.marylandcomptroller.gov/app/claim-search
1. Change display to 80
2. Clear daily postgresql table
3. Loop through AA to ZZ as last name
a. Collect all results
i. Navigate through all pages
b. Write results to master and daily postgresql table
i. On property ID already existing, ignore
4. Perform any captchas and rotate proxy as necessary
5. Anti join where master is not in daily
a. Populate Claimed in master as today where claimed not already set and property ID not in daily table
6. Join on ID between daily and master, check for any fields that are different between the two
a. Write exceptions into difference table with found_date as today
Database structure:
unclaimed_daily
• owner
• last_known_address
• city
• state
• zip_code
• reported_by
• amount
• property_id


unclaimed_master
• owner
• last_known_address
• city
• state
• zip_code
• reported_by
• amount
• property_id
• claimed
unclaimed_difference
• master_owner
• master_last_known_address
• master_city
• master_state
• master_zip_code
• master_reported_by
• master_amount
• master_property_id
• master_owner
• master_last_known_address
• master_city
• master_state
• master_zip_code
• master_reported_by
• master_amount
• master_property_id
• found_date