Kafka expert to import csv and to call rest api
Budget: $10 – $150 USD
We require a highly skilled Kafka expert, who can access e-mails. this emails contain a csv, this csv itself contain data, which need to be converted into reasonable values. Afterwards this data is getting posted via an REST API to a different system. Both the csv and the converted data shall be accessible for debug purposes after being processed. A re-execution by an admin is also mandatory, if something went wrong during parsing or processing.
This is a test simple test task to find the right candidate. We will assign you later more work, if this implementation works well.
Key functionalities required:
- read e-mails via connector (event source)
- fetch the csv from the e-mail
- import the data into kafka
- convert/parse the data properly inside kafka
- ensure both csv, and parsed data are available for later debugging/re-execution
- post the parsed data to a REST api connector (event destination)
- provide a docker-compose to startup the kafka
- if coding => java/JDK17
- if framework => springboot (latest stable)
Strong expertise on Kafka and how to read data from one system, parse this data reliably with options to debug later and finally post data to the destination system
What is not required:
- HA solutions of kafka
- high load
=> the availability is a single node since (for this task) and the HA is therefore also not relevant
MS0:
- suggest connectors (IMAPTLS, REST-api via https)
- suggest the parsing/processing
- provide docker-compose for starting up kafka
MS1:
- implement the fetching via e-mail
- implement the csv parsing (if custom coding, preferably opencsv based)
- implement the persistency of both states (unparsed csv, and parsed data from csv)
MS2:
- implement the REST api caller
Budget?
will not be disclosed, place your best bid to get considered
What is next?
We will share you a NDA and afterwards a paid test task.
Payment?
- you estimate in a WBS (optimistic, expected, pessimistic, where optimistic < expected < pessimistic) after getting the task
- we discuss about clearances and effort
- we mutually agree to effort
- we assign you the task after mutually agreed
- you implement, test and deliver
- you provide also unit /integration tests to ensure code is always working
- you document each required step to make interaction of systems to work
- we test on our premise (you maybe help to fix issues)
- we pay
(basically the rules of freelancer)
Closed book vs open book?
We work only on open book.
Closed book means you are unwilling to define a WBS for the work and you add only a price tag to the task.
We are sorry we will not hire you in such a case!
Deliveries?
- in our on premise git (access will be granted to you)
- full sources
- multiple samples of body and responses of the api in test/resources which will be used as unit tests
- maven
- libs, need prior confirm and we prefer to use mostly latest stable versions
- JDK 17 (mostly LTS)
- your code is working on our end too
- your code passes our CI build (with provided maven pom.xml can be checked)
- you may require to help us here on setup
- docker compose to startup all required runtimes
- we require, depending on availability and on complexity 2-4 weeks test time per milestone. Usually clearly less
This is a test simple test task to find the right candidate. We will assign you later more work, if this implementation works well.
Key functionalities required:
- read e-mails via connector (event source)
- fetch the csv from the e-mail
- import the data into kafka
- convert/parse the data properly inside kafka
- ensure both csv, and parsed data are available for later debugging/re-execution
- post the parsed data to a REST api connector (event destination)
- provide a docker-compose to startup the kafka
- if coding => java/JDK17
- if framework => springboot (latest stable)
Strong expertise on Kafka and how to read data from one system, parse this data reliably with options to debug later and finally post data to the destination system
What is not required:
- HA solutions of kafka
- high load
=> the availability is a single node since (for this task) and the HA is therefore also not relevant
MS0:
- suggest connectors (IMAPTLS, REST-api via https)
- suggest the parsing/processing
- provide docker-compose for starting up kafka
MS1:
- implement the fetching via e-mail
- implement the csv parsing (if custom coding, preferably opencsv based)
- implement the persistency of both states (unparsed csv, and parsed data from csv)
MS2:
- implement the REST api caller
Budget?
will not be disclosed, place your best bid to get considered
What is next?
We will share you a NDA and afterwards a paid test task.
Payment?
- you estimate in a WBS (optimistic, expected, pessimistic, where optimistic < expected < pessimistic) after getting the task
- we discuss about clearances and effort
- we mutually agree to effort
- we assign you the task after mutually agreed
- you implement, test and deliver
- you provide also unit /integration tests to ensure code is always working
- you document each required step to make interaction of systems to work
- we test on our premise (you maybe help to fix issues)
- we pay
(basically the rules of freelancer)
Closed book vs open book?
We work only on open book.
Closed book means you are unwilling to define a WBS for the work and you add only a price tag to the task.
We are sorry we will not hire you in such a case!
Deliveries?
- in our on premise git (access will be granted to you)
- full sources
- multiple samples of body and responses of the api in test/resources which will be used as unit tests
- maven
- libs, need prior confirm and we prefer to use mostly latest stable versions
- JDK 17 (mostly LTS)
- your code is working on our end too
- your code passes our CI build (with provided maven pom.xml can be checked)
- you may require to help us here on setup
- docker compose to startup all required runtimes
- we require, depending on availability and on complexity 2-4 weeks test time per milestone. Usually clearly less