Automated File Monitoring and Secure Data Transmission Tool in Go (Golang)
Budget: €250 – €750 EUR
The aim of this project is to develop a tool capable of monitoring specified files for changes, extracting and converting data from these files, and then securely transmitting it to a server via a REST API. The tool should allow each monitoring task to be authenticated with its own API key.
Functional Requirements
File Monitoring:
The tool must detect changes to designated file paths. Any change should trigger an automated process to process the affected file.
The program should continuously monitor the configured files and detect changes (e.g., using CRC checksums).
The monitoring interval should be individually adjustable for each file.
Data Conversion:
Extracted data from the files must be converted into a specified JSON format required for forwarding to the REST API.
Once a change in one of the monitored files is detected, the program should automatically convert the affected file to the desired format.
Data Transmission:
The converted data must be sent to a server via a secured REST API connection. Authentication should be performed using specific API keys assigned to each monitoring task.
After successfully converting the file, the program should send the file to a configured server via a REST API (primary api.url.tld and secondary api.url1.tld as fallback).
The REST API parameters (such as URL, authentication methods, etc.) should also be configurable via the Systray menu.
The program should ensure that the file is successfully transmitted and implement retry attempts or error logging in case of failures.
Configuration Management:
The tool should offer an interface that allows users to add, edit, and remove monitoring tasks. Configuration settings should include the file path, monitoring interval, and associated API key.
The program should run as a Windows application in the system tray (Systray) without needing an open window.
A menu accessible via a Systray icon should allow users to add, remove, or edit files for monitoring.
The configuration data should be stored in an appropriate structure, such as a JSON file or a database, so that the program can load the configuration on startup.
Multi-Tasking:
The system should be capable of managing and executing up to 10 monitoring tasks simultaneously.
Auto-Start Integration:
The tool should be configured to automatically initialize and run upon system startup. This feature ensures seamless and uninterrupted monitoring of files without the need for manual user intervention.
Logging:
The program should create log files that document both successful and unsuccessful transmissions.
The logs should contain information such as the date and time of transmission, filename, transmission status, and any error messages.
These log files should be easily accessible and readable to facilitate monitoring and troubleshooting.
Logs should be deleted every 30 days, with this interval also being adjustable in the configuration.
Development Requirements
The tool should be modular in design for easy scalability and maintenance.
There should be a clear separation between the user interface and backend processes.
Data transmission must be encrypted and secure in accordance with current industry standards.
The user interface should be intuitive and user-friendly to facilitate the configuration and management of monitoring tasks.
Documentation and Support
Comprehensive documentation will be created, covering both technical implementation and user guidance.
Support includes technical assistance and troubleshooting to ensure smooth operation and maintenance of the tool.
Source codes have to be included. This documentation and support system will help users to effectively manage and utilize the tool with minimal difficulty.
Functional Requirements
File Monitoring:
The tool must detect changes to designated file paths. Any change should trigger an automated process to process the affected file.
The program should continuously monitor the configured files and detect changes (e.g., using CRC checksums).
The monitoring interval should be individually adjustable for each file.
Data Conversion:
Extracted data from the files must be converted into a specified JSON format required for forwarding to the REST API.
Once a change in one of the monitored files is detected, the program should automatically convert the affected file to the desired format.
Data Transmission:
The converted data must be sent to a server via a secured REST API connection. Authentication should be performed using specific API keys assigned to each monitoring task.
After successfully converting the file, the program should send the file to a configured server via a REST API (primary api.url.tld and secondary api.url1.tld as fallback).
The REST API parameters (such as URL, authentication methods, etc.) should also be configurable via the Systray menu.
The program should ensure that the file is successfully transmitted and implement retry attempts or error logging in case of failures.
Configuration Management:
The tool should offer an interface that allows users to add, edit, and remove monitoring tasks. Configuration settings should include the file path, monitoring interval, and associated API key.
The program should run as a Windows application in the system tray (Systray) without needing an open window.
A menu accessible via a Systray icon should allow users to add, remove, or edit files for monitoring.
The configuration data should be stored in an appropriate structure, such as a JSON file or a database, so that the program can load the configuration on startup.
Multi-Tasking:
The system should be capable of managing and executing up to 10 monitoring tasks simultaneously.
Auto-Start Integration:
The tool should be configured to automatically initialize and run upon system startup. This feature ensures seamless and uninterrupted monitoring of files without the need for manual user intervention.
Logging:
The program should create log files that document both successful and unsuccessful transmissions.
The logs should contain information such as the date and time of transmission, filename, transmission status, and any error messages.
These log files should be easily accessible and readable to facilitate monitoring and troubleshooting.
Logs should be deleted every 30 days, with this interval also being adjustable in the configuration.
Development Requirements
The tool should be modular in design for easy scalability and maintenance.
There should be a clear separation between the user interface and backend processes.
Data transmission must be encrypted and secure in accordance with current industry standards.
The user interface should be intuitive and user-friendly to facilitate the configuration and management of monitoring tasks.
Documentation and Support
Comprehensive documentation will be created, covering both technical implementation and user guidance.
Support includes technical assistance and troubleshooting to ensure smooth operation and maintenance of the tool.
Source codes have to be included. This documentation and support system will help users to effectively manage and utilize the tool with minimal difficulty.