Large-Scale Multi-File Type Data Scraping
Budget: $15 – $25 USD
I have 5000+ files/URLs that need to be maintained in the sense the data in each are in various formats and styles, however I just need these specific fields or the equivalents when they exist which should be mostly always:
hospital_id
row_id
line_type
description
rev_code
local_code
code
ms_drg
apr_drg
eapg
hcpcs_cpt
modifiers
alt_hcpcs_cpt
thru
apc
icd
ndc
drug_hcpcs_multiplier
drug_quantity
drug_unit_of_measurement
drug_type_of_measurement
billing_class
setting
rate_category
payer_name
plan_name
standard_charge
standard_charge_percent
contracting_method
additional_generic_notes
additional_payer_specific_notes
You need to maintain these entries in a mysql database. You can use OpenAI or Claude or various other ways to help however some files are big so you'd have to think of how to send them in segments.
Example URLs:
https://www.virtua.org/-/media/Files/Virtua%20Enterprise/Virtua%20Corporate/Virtua/documents/Hospital%20Charges%20Information/Virtua%20Health%20Chargemaster-3,-d-,28,-d-,22.ashx
https://www.lmh.org/app/files/public/8979634e-c94d-4981-b5fb-0478a4550e65/48-6033703_LMH_standardcharges_2022.csv
https://resources.selectmedical.com/Pricing-Transparency/critical-illness-recovery/25-1897325_Select_Specialty_Hospital-Miami_standardcharges.csv
https://resources.selectmedical.com/Pricing-Transparency/critical-illness-recovery/64-0947034_Regency_Hospital_of_Meridian_standardcharges.csv
https://coc.ardenthealthservices.com/2022/oklahoma/2022_TulsaSpineandSpecialty_standardcharges.txt
https://app.peacehealth.org/FileRepository/Financial/StandardCharges/UDMC/93-0395583_Sacred_Heart_Medical_Center_University_District_StandardCharges.csv
https://www.ohiohealth.com/siteassets/patients-and-visitors/preparing-for-your-visit/out-of-pocket-estimates/34-0714456_mansfield_standardcharges.xlsx
https://www.claxtonhepburn.org/documents/CHMC-Standard-Charge-File-csv.csv
http://www.sthspecialtysurgery.com/media/22147/621772195_StThomasHospitalforSpecialtySurgery_standardcharges_2023_03_16_22.13.19.json
https://healthcare.ascension.org/-/media/project/ascension/healthcare/price-transparency-files/wi/390816857_ascension-se-wisconsin-hospital-st-joseph-campus_standardcharges.xlsx
https://uhsfilecdn.eskycity.net/ac/742829611_cornerstone-regional-hospital_standard-charges.xlsx
https://uvahealth.com/sites/default/files/2022-08/546001796_university-of-virginia-medical-center_standardcharges.txt
https://web.salemhealth.org/pricing/Data/SalemHospitalPricing.xlsx
https://encompasshealth.com/-/media/healthsouth/project/healthsouth/files/2023-price-transparency/04-2948290_fairlawn-rehabilitation-hospital-an-affiliate-of-encompass-health-_standardcharges.csv
https://www.losalamosmedicalcenter.com/sites/losalamos/assets/uploads/MRF%20-%20Los_Alamos_Medical_Center_Standard_Charges_05112021%20030390794.zip
https://www.aurorahealthcare.org/assets/documents/billing-insurance/pricing-transparency/391211629_aurora-medical-center-manitowoc-county_standardcharges.xml
https://uhsfilecdn.eskycity.net/bh/721539530-northstar-standard_charges.csv
https://assets.changehealthcare.com/Shop/PROD/static/CookevilleRegionMedicalCenter/62-6002166_CookevilleRegionMedicalCenter_standardcharges.csv.zip
https://wvumedicine.org/wp-content/uploads/2022/03/550525724_UnitedHospitalCenter_standardcharges.xlsx
https://encompasshealth.com/-/media/healthsouth/project/healthsouth/files/2023-price-transparency/943427408yuma-rehabilitation-hospital-an-affiliation-of-encompass-health-and-yuma-regional-standard.csv
https://wvumedicine.org/wp-content/uploads/2022/03/550643304_WestVirginiaUniversityHospitals_standardcharges.xlsx
https://sites.bmhcc.org/-/media/Pricing-Documents/MS/64-0772726_BMH-North-Mississipp_STANDARDCHARGES.zip
https://www.kch.com/Uploads/Public/Documents/charge-masters/621764613_Kosciusko_standardcharges.csv
https://www.chihealth.com/content/dam/chi-health/website/documents/cost-estimates/Charges_CSVreport_11_Jan_2021_NebraskaHeart.csv
https://encompasshealth.com/-/media/healthsouth/project/healthsouth/files/2023-price-transparency/63-1244181_encompass-health-rehabilitation-hospital-of-spring-hill_standardcharges.csv
https://www.kindredhospitals.com/docs/default-source/default-document-library/locations/transitional-care-hospitals/cms-price-transparency-files/2022-docs/030430242_sugar-land_standardcharges5f0e045edafd66f3a486ff0000603144.csv
https://apps.ohsu.edu/health/communication-preferences/resources/931176109_OregonHealthandScienceUniversity_StandardCharges.xlsx|https://www.ohsu.edu/sites/default/files/2023-03/931176109_OregonHealthandScienceUniversity_ProfessionalStandardCharges.xlsx
I only want to work with people who have done large scale scraping of data that has different names and such from thousands of files. Prove to me you have done it.
hospital_id
row_id
line_type
description
rev_code
local_code
code
ms_drg
apr_drg
eapg
hcpcs_cpt
modifiers
alt_hcpcs_cpt
thru
apc
icd
ndc
drug_hcpcs_multiplier
drug_quantity
drug_unit_of_measurement
drug_type_of_measurement
billing_class
setting
rate_category
payer_name
plan_name
standard_charge
standard_charge_percent
contracting_method
additional_generic_notes
additional_payer_specific_notes
You need to maintain these entries in a mysql database. You can use OpenAI or Claude or various other ways to help however some files are big so you'd have to think of how to send them in segments.
Example URLs:
https://www.virtua.org/-/media/Files/Virtua%20Enterprise/Virtua%20Corporate/Virtua/documents/Hospital%20Charges%20Information/Virtua%20Health%20Chargemaster-3,-d-,28,-d-,22.ashx
https://www.lmh.org/app/files/public/8979634e-c94d-4981-b5fb-0478a4550e65/48-6033703_LMH_standardcharges_2022.csv
https://resources.selectmedical.com/Pricing-Transparency/critical-illness-recovery/25-1897325_Select_Specialty_Hospital-Miami_standardcharges.csv
https://resources.selectmedical.com/Pricing-Transparency/critical-illness-recovery/64-0947034_Regency_Hospital_of_Meridian_standardcharges.csv
https://coc.ardenthealthservices.com/2022/oklahoma/2022_TulsaSpineandSpecialty_standardcharges.txt
https://app.peacehealth.org/FileRepository/Financial/StandardCharges/UDMC/93-0395583_Sacred_Heart_Medical_Center_University_District_StandardCharges.csv
https://www.ohiohealth.com/siteassets/patients-and-visitors/preparing-for-your-visit/out-of-pocket-estimates/34-0714456_mansfield_standardcharges.xlsx
https://www.claxtonhepburn.org/documents/CHMC-Standard-Charge-File-csv.csv
http://www.sthspecialtysurgery.com/media/22147/621772195_StThomasHospitalforSpecialtySurgery_standardcharges_2023_03_16_22.13.19.json
https://healthcare.ascension.org/-/media/project/ascension/healthcare/price-transparency-files/wi/390816857_ascension-se-wisconsin-hospital-st-joseph-campus_standardcharges.xlsx
https://uhsfilecdn.eskycity.net/ac/742829611_cornerstone-regional-hospital_standard-charges.xlsx
https://uvahealth.com/sites/default/files/2022-08/546001796_university-of-virginia-medical-center_standardcharges.txt
https://web.salemhealth.org/pricing/Data/SalemHospitalPricing.xlsx
https://encompasshealth.com/-/media/healthsouth/project/healthsouth/files/2023-price-transparency/04-2948290_fairlawn-rehabilitation-hospital-an-affiliate-of-encompass-health-_standardcharges.csv
https://www.losalamosmedicalcenter.com/sites/losalamos/assets/uploads/MRF%20-%20Los_Alamos_Medical_Center_Standard_Charges_05112021%20030390794.zip
https://www.aurorahealthcare.org/assets/documents/billing-insurance/pricing-transparency/391211629_aurora-medical-center-manitowoc-county_standardcharges.xml
https://uhsfilecdn.eskycity.net/bh/721539530-northstar-standard_charges.csv
https://assets.changehealthcare.com/Shop/PROD/static/CookevilleRegionMedicalCenter/62-6002166_CookevilleRegionMedicalCenter_standardcharges.csv.zip
https://wvumedicine.org/wp-content/uploads/2022/03/550525724_UnitedHospitalCenter_standardcharges.xlsx
https://encompasshealth.com/-/media/healthsouth/project/healthsouth/files/2023-price-transparency/943427408yuma-rehabilitation-hospital-an-affiliation-of-encompass-health-and-yuma-regional-standard.csv
https://wvumedicine.org/wp-content/uploads/2022/03/550643304_WestVirginiaUniversityHospitals_standardcharges.xlsx
https://sites.bmhcc.org/-/media/Pricing-Documents/MS/64-0772726_BMH-North-Mississipp_STANDARDCHARGES.zip
https://www.kch.com/Uploads/Public/Documents/charge-masters/621764613_Kosciusko_standardcharges.csv
https://www.chihealth.com/content/dam/chi-health/website/documents/cost-estimates/Charges_CSVreport_11_Jan_2021_NebraskaHeart.csv
https://encompasshealth.com/-/media/healthsouth/project/healthsouth/files/2023-price-transparency/63-1244181_encompass-health-rehabilitation-hospital-of-spring-hill_standardcharges.csv
https://www.kindredhospitals.com/docs/default-source/default-document-library/locations/transitional-care-hospitals/cms-price-transparency-files/2022-docs/030430242_sugar-land_standardcharges5f0e045edafd66f3a486ff0000603144.csv
https://apps.ohsu.edu/health/communication-preferences/resources/931176109_OregonHealthandScienceUniversity_StandardCharges.xlsx|https://www.ohsu.edu/sites/default/files/2023-03/931176109_OregonHealthandScienceUniversity_ProfessionalStandardCharges.xlsx
I only want to work with people who have done large scale scraping of data that has different names and such from thousands of files. Prove to me you have done it.