CSV Data Distribution from an or more SFTP Server: Retrieve, Translate, and Transmit via API

Job ID: 39014853

Budget: €250 – €750 EUR

CSV Data Distribution from an SFTP Server: Retrieve, Translate, and Transmit via API

CSV files are stored in various source directories on one or more SFTP servers.
It must be configurable which information, with which status, is forwarded.
For example, the CSV contains "ok".
All entries that are not "OK" must be saved and stored as events.
These logs should be downloadable from the backend as a CSV file, containing all information and timestamps. No information from the CSV should be lost during the process.
It must be possible to filter the data, e.g., by date, status, carrier, or other columns.
The data needs to be retrieved, transmitted to a system via API, and logged.
It may be required to send messages from a specific SFTP folder in subdirectories using different API users. It should be configurable which SFTP folder and directory require which API user.
Additionally, there must be an option to configure when the archive and log directories should be cleared. It should be identifiable from the file when processing has occurred.
After the API is called and the information has been transmitted, the API must generate an event that is logged in the database and can be viewed and filtered via the backend.
A web interface must be developed to manage settings such as transfer intervals per ID and API, SFTP credentials, and to store additional API users.
Attached the API documentation.
via csv we get:
- order, parcel, GTIN, amount and status, user, timestamp, for example
Example will be send later.

Warnings and Escalations:

Send email notifications if the API has been unavailable for more than X minutes. This time should be configurable in the backend.
The email address to which the notifications should be sent must be configurable in the backend. It should also be possible to specify multiple email addresses.
An escalation option should also be available if the API is unavailable for Y minutes (also configurable).
The system must be capable of processing large data volumes.
APIs should only be called when necessary and when there is data to be transmitted.
Let me know if you need any further adjustments!
Related categories: PHP Python Linux Software Architecture MySQL