Intelligent Event Info Web Crawler
Budget: €30 – €250 EUR
Project Briefing for Freelancer (Web Scraping Task)
Scope and Goal
We are look for a smart crawler that can collect contact information of organizers and responsible persons for local public events such as:
City festivals (Stadtfeste)
Wine festivals (Weinfeste)
events (CSD / Christopher Street Day)
Public parades or local fairs
The target region includes Germany's federal states of North Rhine-Westphalia (NRW), Rhineland-Palatinate (Rheinland-Pfalz), and Hesse (Hessen).
The goal is to extract a comprehensive, structured dataset of event organizers and their contact persons, which includes names, roles, email addresses, phone numbers, and source URLs.
Keywords & Synonyms (Search Query Expansion)
To improve search accuracy, the crawler should search using different synonym clusters for the event types and contact terminology.
Event Type Keywords
"Stadtfest", "Altstadtfest", "Volksfest", "Kirmes", "Dorffest", "Lichterfest", "Sommerfest", "Stadt Fest" "Weihnachtsmarkt" "Wintermarkt" "Schützenfest"
"Weinfest", "Weintage", "Winzerfest", "Weinmarkt"
"CSD", "Christopher Street Day",
"Stadtveranstaltung", "Open Air", "Kulturveranstaltung"
Organizer Synonyms
Veranstalter
Organisationsteam
Ansprechpartner
Kontaktperson
Kontakt
Projektleitung
Orga-Team
Koordination
Contact Info Terms
E-Mail, Email, Mailadresse
Telefonnummer, Phone, Mobilnummer
Kontaktformular, Impressum
Google Search Logic (to be automated)
Use these operators to simulate searches in Google. The tool should loop through different cities and event types using these templates:
Search Query Templates:
text
Kopieren
Bearbeiten
"<EventKeyword>" AND (Veranstalter OR Ansprechpartner OR Kontakt) AND (E-Mail OR Telefon) AND "<City>" AND site:.de
"<City> <EventKeyword>" AND (Kontakt OR Organisationsteam) AND (Telefon OR E-Mail)
"<City> Kulturamt Ansprechpartner Veranstaltungen"
"<City> Stadtmarketing Kontakt Veranstaltungen"
"<City> CSD Veranstalter site:.de"
Use combinations of these keywords and iterate through all cities in NRW, Hesse, and Rheinland-Pfalz (links provided below).
Target Groups for Contact Extraction
The scraper should target pages containing contacts for:
Municipalities (Stadtverwaltungen / Kulturämter)
Tourism offices (Tourismusbüros)
City marketing departments (Stadtmarketing GmbHs)
Event agencies (Veranstaltungsagenturen)
Associations and Nonprofits (CSD e.V., Weinbauvereine)
Official event pages or microsites for specific events
Data Points to Extract per Match
Field Description
event_name e.g., "CSD Frankfurt" or "Weinfest Mainz"
event_type Based on the keyword cluster used
city From city loop or detected on the page
state NRW, Hessen, Rheinland-Pfalz
event_date (optional) If found on the same page
organizer_name e.g., "Kulturamt Bonn" or "CSD Koblenz e.V."
contact_person Ideally full name
position e.g., "Projektleitung", "Ansprechpartner", "Koordination"
email_address Either direct or general address
phone_number Prefer direct, fallback to office number
source_url URL where the data was found
Official City Lists (for location loop)
These are official city directories per federal state. Use them to generate your search combinations:
NRW (North Rhine-Westphalia)
Official list of cities/municipalities:
https://www.destatis.de/DE/Themen/Laender-Regionen/Regionales/Gemeindeverzeichnis/_inhalt.html
Choose "Nordrhein-Westfalen"
Alternative (Wikipedia - faster to parse):
https://en.wikipedia.org/wiki/List_of_towns_and_cities_in_North_Rhine-Westphalia
Rheinland-Pfalz
https://en.wikipedia.org/wiki/List_of_municipalities_in_Rhineland-Palatinate
(grouped by district)
Hessen
https://en.wikipedia.org/wiki/List_of_cities_in_Hesse
or
https://www.hessen.de/ueber-hessen/kommunen
Technical Notes
Crawling should respect robots.txt but can parse public contact pages and impressum pages if no direct info is found.
Results must be exportable in CSV and XLS format.
Please ensure deduplication and label entries where multiple contact persons are found.
Implement delay/throttling to avoid IP bans.
Optionally use Sitemap.xml or inurl:event, inurl:kontakt, inurl:csd, inurl:weinfest for faster page identification.
Deliverables
Clean, structured CSV or XLS file with all matched data
Documentation of which search rules or patterns were used
Source code or script (Python preferred) that can be reused or extended
Scope and Goal
We are look for a smart crawler that can collect contact information of organizers and responsible persons for local public events such as:
City festivals (Stadtfeste)
Wine festivals (Weinfeste)
events (CSD / Christopher Street Day)
Public parades or local fairs
The target region includes Germany's federal states of North Rhine-Westphalia (NRW), Rhineland-Palatinate (Rheinland-Pfalz), and Hesse (Hessen).
The goal is to extract a comprehensive, structured dataset of event organizers and their contact persons, which includes names, roles, email addresses, phone numbers, and source URLs.
Keywords & Synonyms (Search Query Expansion)
To improve search accuracy, the crawler should search using different synonym clusters for the event types and contact terminology.
Event Type Keywords
"Stadtfest", "Altstadtfest", "Volksfest", "Kirmes", "Dorffest", "Lichterfest", "Sommerfest", "Stadt Fest" "Weihnachtsmarkt" "Wintermarkt" "Schützenfest"
"Weinfest", "Weintage", "Winzerfest", "Weinmarkt"
"CSD", "Christopher Street Day",
"Stadtveranstaltung", "Open Air", "Kulturveranstaltung"
Organizer Synonyms
Veranstalter
Organisationsteam
Ansprechpartner
Kontaktperson
Kontakt
Projektleitung
Orga-Team
Koordination
Contact Info Terms
E-Mail, Email, Mailadresse
Telefonnummer, Phone, Mobilnummer
Kontaktformular, Impressum
Google Search Logic (to be automated)
Use these operators to simulate searches in Google. The tool should loop through different cities and event types using these templates:
Search Query Templates:
text
Kopieren
Bearbeiten
"<EventKeyword>" AND (Veranstalter OR Ansprechpartner OR Kontakt) AND (E-Mail OR Telefon) AND "<City>" AND site:.de
"<City> <EventKeyword>" AND (Kontakt OR Organisationsteam) AND (Telefon OR E-Mail)
"<City> Kulturamt Ansprechpartner Veranstaltungen"
"<City> Stadtmarketing Kontakt Veranstaltungen"
"<City> CSD Veranstalter site:.de"
Use combinations of these keywords and iterate through all cities in NRW, Hesse, and Rheinland-Pfalz (links provided below).
Target Groups for Contact Extraction
The scraper should target pages containing contacts for:
Municipalities (Stadtverwaltungen / Kulturämter)
Tourism offices (Tourismusbüros)
City marketing departments (Stadtmarketing GmbHs)
Event agencies (Veranstaltungsagenturen)
Associations and Nonprofits (CSD e.V., Weinbauvereine)
Official event pages or microsites for specific events
Data Points to Extract per Match
Field Description
event_name e.g., "CSD Frankfurt" or "Weinfest Mainz"
event_type Based on the keyword cluster used
city From city loop or detected on the page
state NRW, Hessen, Rheinland-Pfalz
event_date (optional) If found on the same page
organizer_name e.g., "Kulturamt Bonn" or "CSD Koblenz e.V."
contact_person Ideally full name
position e.g., "Projektleitung", "Ansprechpartner", "Koordination"
email_address Either direct or general address
phone_number Prefer direct, fallback to office number
source_url URL where the data was found
Official City Lists (for location loop)
These are official city directories per federal state. Use them to generate your search combinations:
NRW (North Rhine-Westphalia)
Official list of cities/municipalities:
https://www.destatis.de/DE/Themen/Laender-Regionen/Regionales/Gemeindeverzeichnis/_inhalt.html
Choose "Nordrhein-Westfalen"
Alternative (Wikipedia - faster to parse):
https://en.wikipedia.org/wiki/List_of_towns_and_cities_in_North_Rhine-Westphalia
Rheinland-Pfalz
https://en.wikipedia.org/wiki/List_of_municipalities_in_Rhineland-Palatinate
(grouped by district)
Hessen
https://en.wikipedia.org/wiki/List_of_cities_in_Hesse
or
https://www.hessen.de/ueber-hessen/kommunen
Technical Notes
Crawling should respect robots.txt but can parse public contact pages and impressum pages if no direct info is found.
Results must be exportable in CSV and XLS format.
Please ensure deduplication and label entries where multiple contact persons are found.
Implement delay/throttling to avoid IP bans.
Optionally use Sitemap.xml or inurl:event, inurl:kontakt, inurl:csd, inurl:weinfest for faster page identification.
Deliverables
Clean, structured CSV or XLS file with all matched data
Documentation of which search rules or patterns were used
Source code or script (Python preferred) that can be reused or extended