Develop a Podman Compose File for Automated Logstash / Logstash Pipeline Setup
Budget: $30 – $250 USD
I am seeking an experienced engineer to create a Podman Compose file designed for a Logstash environment. The key objectives of this project are to automate the setup and configuration of multiple pipelines and ensure seamless file transfers from the host to the container.
Project Requirements:
Pipeline File Creation:
-Automatically generate 13 blank pipeline files as placeholders to verify the initial setup.
File Transfers:
-Transfer 13 pipeline configuration files from the host to the container, specifically to /usr/share/logstash/config.
-Move the logstash.yml configuration file from the host to /usr/share/logstash/pipeline.
-Transfer a JDBC driver from the host to /usr/share/logstash/config.
Java Keystore Setup:
-Configure a Java Keystore on the host with 5 variables (names can be placeholders).
-Transfer the Keystore file from the host to the container
-Validate Keystore file works successfully
Cron Automation for Pipelines:
-Set up a cron job to run each of the 13 pipelines hourly, starting at 02:00 AM, with subsequent pipelines running at one-hour intervals (e.g., pipeline 2 at 03:00 AM, pipeline 3 at 04:00 AM, etc.).
Data Volume Management:
-Mount a volume named logstash-data for persistent data storage.
Simplification of Deployment:
-Preferably, package the necessary files into a single tar archive to simplify extraction and execution within the container.
Deliverables:
A Podman Compose file that meets the above requirements.
Documentation on how the system is configured and instructions for future modifications.
Additional Notes:
Familiarity with Logstash, Docker/Podman, and cron job configuration is crucial.
Project Requirements:
Pipeline File Creation:
-Automatically generate 13 blank pipeline files as placeholders to verify the initial setup.
File Transfers:
-Transfer 13 pipeline configuration files from the host to the container, specifically to /usr/share/logstash/config.
-Move the logstash.yml configuration file from the host to /usr/share/logstash/pipeline.
-Transfer a JDBC driver from the host to /usr/share/logstash/config.
Java Keystore Setup:
-Configure a Java Keystore on the host with 5 variables (names can be placeholders).
-Transfer the Keystore file from the host to the container
-Validate Keystore file works successfully
Cron Automation for Pipelines:
-Set up a cron job to run each of the 13 pipelines hourly, starting at 02:00 AM, with subsequent pipelines running at one-hour intervals (e.g., pipeline 2 at 03:00 AM, pipeline 3 at 04:00 AM, etc.).
Data Volume Management:
-Mount a volume named logstash-data for persistent data storage.
Simplification of Deployment:
-Preferably, package the necessary files into a single tar archive to simplify extraction and execution within the container.
Deliverables:
A Podman Compose file that meets the above requirements.
Documentation on how the system is configured and instructions for future modifications.
Additional Notes:
Familiarity with Logstash, Docker/Podman, and cron job configuration is crucial.