Data Extraction Specialist -- 2
Budget: $250 – $750 USD
We are seeking experienced developers or data analysts with expertise in working with government procurement APIs. The primary goal of this project is to extract both historical and active procurement data using APIs from various countries, with a strong emphasis on the European Union (TED APIs), Canada (Open Government API). Experience with other countries' procurement APIs is also highly valued.
Key Requirements:
Proven experience extracting data from government procurement APIs, specifically from the European Union (TED APIs) and Canada. Experience with other countries is a plus.
Strong skills in managing structured and nested data efficiently.
Focus on retrieving historical and active procurement data (detailed understanding of procurement contracts is not required).
To be precisely i want one time extraction the thing could happen is the volume of the data is big it's necessary more than one request. But it's not necessary have a api working for a rutinary request.
----------------------------
Task Details:
The APIs used follow the Open Contracting Data Standard (OCDS). Links to relevant documentation and standards are provided in the attached document.
Historical data should cover the period from 2018 onwards.
The extracted data should be delivered in CSV format.
We expect the code to be able to extract data from at least five different countries.
Some missing variables are acceptable, but prior review and approval from us will be required.
----------------------------
Data Structure:
Here is an example of the data structure we expect. The exact names of the variables are flexible; the focus is on capturing the required dimensions.
Historical Data:
procurement_id: Unique identifier for the procurement/tender.
publication_date: Date the procurement/tender was initially published.
status: Final status of the procurement (e.g., awarded, cancelled, expired).
article_id: Code of the purchased article.
article_description: Name/description of the purchased article.
buyer_id: Unique identifier of the buyer/government entity.
buyer_name: Name of the government department or agency that issued the procurement.
procurement_type: Type of procurement (e.g., framework agreement, direct contract).
amount: Final awarded amount for the procurement.
currency: Currency used in the procurement.
supplier_id: Unique identifier of the awarded supplier.
supplier_name: Name of the awarded supplier or vendor.
award_date: Date the contract was awarded.
url_award_notice: Link to the award notice or contract.
url_tender: Link to the original tender document or notice.
Active Data:
procurement_id: Unique identifier for the procurement/tender.
publication_date: Date the procurement/tender was published.
status: Current status of the procurement (e.g., open, under evaluation).
article_id: Code of the requested item.
article_description: Name/description of the requested item.
buyer_id: Unique identifier of the buyer/government entity.
buyer_name: Name of the government department or agency that issued the procurement.
submission_deadline: Deadline for suppliers to submit their proposals.
place_opening: Location where submissions will be opened (if applicable).
url_tender: Link to the tender document or procurement notice.
url_clarifications: Link to any clarifications or additional documents.
Documentation & Links:
European Union TED APIs: https://ted.europa.eu/en/simap/developers-corner-for-reusers#download-xml-notices
Canada Open Government API: https://buyandsell.gc.ca/for-businesses/selling-to-the-government-of-canada/find-opportunities
Additional API Documentation: https://api-docs.openprocurement.org/en/latest/------------------------------
What We Offer:
A short-term contract to extract and prepare this data for analysis.
After evaluating the performance and quality of the work, there is potential for long-term collaboration on a larger project.
What We Expect:
Complete and functional code that can extract the specified procurement data from at least five countries.
Flexibility for some missing data, but only after review and approval.
Delivery of the code after thorough testing.
Key Requirements:
Proven experience extracting data from government procurement APIs, specifically from the European Union (TED APIs) and Canada. Experience with other countries is a plus.
Strong skills in managing structured and nested data efficiently.
Focus on retrieving historical and active procurement data (detailed understanding of procurement contracts is not required).
To be precisely i want one time extraction the thing could happen is the volume of the data is big it's necessary more than one request. But it's not necessary have a api working for a rutinary request.
----------------------------
Task Details:
The APIs used follow the Open Contracting Data Standard (OCDS). Links to relevant documentation and standards are provided in the attached document.
Historical data should cover the period from 2018 onwards.
The extracted data should be delivered in CSV format.
We expect the code to be able to extract data from at least five different countries.
Some missing variables are acceptable, but prior review and approval from us will be required.
----------------------------
Data Structure:
Here is an example of the data structure we expect. The exact names of the variables are flexible; the focus is on capturing the required dimensions.
Historical Data:
procurement_id: Unique identifier for the procurement/tender.
publication_date: Date the procurement/tender was initially published.
status: Final status of the procurement (e.g., awarded, cancelled, expired).
article_id: Code of the purchased article.
article_description: Name/description of the purchased article.
buyer_id: Unique identifier of the buyer/government entity.
buyer_name: Name of the government department or agency that issued the procurement.
procurement_type: Type of procurement (e.g., framework agreement, direct contract).
amount: Final awarded amount for the procurement.
currency: Currency used in the procurement.
supplier_id: Unique identifier of the awarded supplier.
supplier_name: Name of the awarded supplier or vendor.
award_date: Date the contract was awarded.
url_award_notice: Link to the award notice or contract.
url_tender: Link to the original tender document or notice.
Active Data:
procurement_id: Unique identifier for the procurement/tender.
publication_date: Date the procurement/tender was published.
status: Current status of the procurement (e.g., open, under evaluation).
article_id: Code of the requested item.
article_description: Name/description of the requested item.
buyer_id: Unique identifier of the buyer/government entity.
buyer_name: Name of the government department or agency that issued the procurement.
submission_deadline: Deadline for suppliers to submit their proposals.
place_opening: Location where submissions will be opened (if applicable).
url_tender: Link to the tender document or procurement notice.
url_clarifications: Link to any clarifications or additional documents.
Documentation & Links:
European Union TED APIs: https://ted.europa.eu/en/simap/developers-corner-for-reusers#download-xml-notices
Canada Open Government API: https://buyandsell.gc.ca/for-businesses/selling-to-the-government-of-canada/find-opportunities
Additional API Documentation: https://api-docs.openprocurement.org/en/latest/------------------------------
What We Offer:
A short-term contract to extract and prepare this data for analysis.
After evaluating the performance and quality of the work, there is potential for long-term collaboration on a larger project.
What We Expect:
Complete and functional code that can extract the specified procurement data from at least five countries.
Flexibility for some missing data, but only after review and approval.
Delivery of the code after thorough testing.