Need to review Apache Lucene writing code to achieve 20K EPS
Budget: ₹1,250 – ₹2,500 INR
Hello, We are using Docker, Spring Boot, Kafka and Apache Lucene to write the large number of data into file based structure. We are reading more than 20K packets from kafka and writing it into file using apache lucene, Currently we are facing issue with memory and garbage collector and theat. Heap memory get overflow because it is not able to release the memory as fast as its took into writing. So we need expert to work on this solution.
Currently, I'm facing a heap memory overflow issue when working with Docker, Spring Boot, Kafka, and Apache Lucene. The project involves reading over 20K packets from Kafka and writing them into a file-based structure using Apache Lucene. The heap memory is overflowing because it cannot release memory as quickly as it is being consumed.
Ideal Skills and Experience:
- Deep understanding of Java and its memory management
- Extensive experience with Apache Lucene, Docker, Spring Boot and Kafka
- Proficiency in optimizing JVM's garbage collector
- Ability to implement custom caching or buffering logic
I need an expert who can help me resolve this issue effectively. Please share your relevant experience when you bid.
Currently, I'm facing a heap memory overflow issue when working with Docker, Spring Boot, Kafka, and Apache Lucene. The project involves reading over 20K packets from Kafka and writing them into a file-based structure using Apache Lucene. The heap memory is overflowing because it cannot release memory as quickly as it is being consumed.
Ideal Skills and Experience:
- Deep understanding of Java and its memory management
- Extensive experience with Apache Lucene, Docker, Spring Boot and Kafka
- Proficiency in optimizing JVM's garbage collector
- Ability to implement custom caching or buffering logic
I need an expert who can help me resolve this issue effectively. Please share your relevant experience when you bid.