Log (Text) Files Indexer and searcher Python
Budget: $30 – $250 USD
The project is for my business, I have Log files with Large size ( Gigabytes of size) I want to build a search mechanism to search on the logs for a particular words after inverted indexing the logs and saving the inverted index in storage
Data structure and algorithms and Solid principles should be displayed ( Clean code and maintainable)
Requirements:
1-Functional Requirements
The system should be able to find and return from its storage, all files relevant to the keyword typed in the search query by a user And at what line the word occurs in the file and the content of that line.
The system should return the files relevant to the keyword without taking in consideration sensitivity of the letters.
The system should be able to index the files based on a specific pattern that the user enters.
The system will be a CLI based.
The system should be able while indexing to categorise the logs based on a particular scheme.
The system should be able to update the indexed file on a schedule and be able to read files in a chosen time span.
The System should be able to trigger an alert via cloud watch if a repeated error was found.
2-Non Functional Requirements
The system should have low latency. Whenever a user makes a query, we want the search results to appear as fast as possible.
The system should be scalable. It should be able to accommodate a growing number of files in its Data Source.
The system should be maintainable. it should be easy to change the way of indexing or the searching Abstraction should be applied in all the code
Data structure and algorithms and Solid principles should be displayed ( Clean code and maintainable)
Requirements:
1-Functional Requirements
The system should be able to find and return from its storage, all files relevant to the keyword typed in the search query by a user And at what line the word occurs in the file and the content of that line.
The system should return the files relevant to the keyword without taking in consideration sensitivity of the letters.
The system should be able to index the files based on a specific pattern that the user enters.
The system will be a CLI based.
The system should be able while indexing to categorise the logs based on a particular scheme.
The system should be able to update the indexed file on a schedule and be able to read files in a chosen time span.
The System should be able to trigger an alert via cloud watch if a repeated error was found.
2-Non Functional Requirements
The system should have low latency. Whenever a user makes a query, we want the search results to appear as fast as possible.
The system should be scalable. It should be able to accommodate a growing number of files in its Data Source.
The system should be maintainable. it should be easy to change the way of indexing or the searching Abstraction should be applied in all the code