Bulk Text File Splitting Program for Windows
Budget: €30 – €250 EUR
We need a simple program that runs on Windows that does a text file splitting job. The logic is simple (its just searching on a list of keywords to decide the split) but it needs to work with multiple large text files (some up to in the 100,000s of lines). We used to do this by Excel but the data sets got too big.
Almost for sure the code for this is in some library on the Internet or could be coded with AI coding methods as it is a simple task and easily described. The program we want to call “Text Spliter”. The program needs to split out lines of text if they contain a certain string.
What will make this job hard are the large number of lines in each of the files and the large amount of matching parameters than need to be searched on each line. We are happy with any modern language (C#, C++, Python + PyInstaller, etc.) as long as the final EXE is clean and self-contained. I think you will need some clever array / caching techniques to make this run a good speed.
If you are interested in this job please respond with
- Whether you are a stand alone programmer or part of a team
- Where abouts in the world you are and what hours you work
- What technology stack and environment and libraries and UI will be needed to run the program (its needs to be easy for non technical people).
- Relevant experience
** The Logic**
Inputs & UI
1) “Path” A windows path to a folder which is full of up to 200 large text files [make this a Browse button that opens windows Folder locator and when chosen writes the path into a text box]
2) “Searched Text” A text input box that can be really long (max 30000 terms, but typically 5000-10000). Each term on each line can be up to 200 characters long but typically will be 10-20 chars. The UI should show 40 lines on the screen but you can scroll down infinitely. In practise the search text will be cut and paste into here. Each Searched text item is on a single line. Not commas or semi colons.
3) “Split Files” Button – when this is pressed the program will start.
*** Functionality ***
1) When the user presses go the program needs to read in all the search terms into memory. The search terms will in 99% of cases be less than 16 characters long (UPC codes and other similar codes).
2) The program then needs to go to the path of the input folder and open the first text file (which may or may not have .txt extension). If it can be read as text, treat as text.
3) Display “Processing: “ and the file name and “Line: “ the line that is currently being read in that text file.
4) Work through each line in the text file and do a search for ANY of the Searched text items on each line. If any line contains any of the search terms at least once then SPLIT that line out and move it to the Output file. That whole line also needs to be removed from the original file and moved to the Output file (i.e that is the “split”)
5) As the Split lines are found they need to get written out to a new file. That file should be in a subfolder of the path in \OutputXXYY (where XXand YY are the time folder created). The name of the file should be the same as the input text file name appended with _SplitOut (and.txt if it wasn’t there already). This file will grow as lines that match are split out.
6) Eventually the file will be split into to. There should be no duplicates between the input and output file and the stripped out original and the split file should now have the same amount of lines combined as the original file.
** Deliverables ***
1) Compiled 64-bit Windows executable ready to run without extra installs.
2) Source code with clear comments so I can recompile if needed.
3) Brief README explaining build steps and any runtime dependencies.
**Example**
Text File
"The big hairy dog is very scary and he moves quick
The sneaky fox is brown and very quick
The white cow is slow and friendly and happy"
Searched Text
"quick
big"
Output
The Output (new) File name should now contain these two lines
"The big hairy dog is very scary and he moves quick
The sneaky fox is brown and very quick"
The original file should now just have this line in it
"The white cow is slow and friendly and happy"
Almost for sure the code for this is in some library on the Internet or could be coded with AI coding methods as it is a simple task and easily described. The program we want to call “Text Spliter”. The program needs to split out lines of text if they contain a certain string.
What will make this job hard are the large number of lines in each of the files and the large amount of matching parameters than need to be searched on each line. We are happy with any modern language (C#, C++, Python + PyInstaller, etc.) as long as the final EXE is clean and self-contained. I think you will need some clever array / caching techniques to make this run a good speed.
If you are interested in this job please respond with
- Whether you are a stand alone programmer or part of a team
- Where abouts in the world you are and what hours you work
- What technology stack and environment and libraries and UI will be needed to run the program (its needs to be easy for non technical people).
- Relevant experience
** The Logic**
Inputs & UI
1) “Path” A windows path to a folder which is full of up to 200 large text files [make this a Browse button that opens windows Folder locator and when chosen writes the path into a text box]
2) “Searched Text” A text input box that can be really long (max 30000 terms, but typically 5000-10000). Each term on each line can be up to 200 characters long but typically will be 10-20 chars. The UI should show 40 lines on the screen but you can scroll down infinitely. In practise the search text will be cut and paste into here. Each Searched text item is on a single line. Not commas or semi colons.
3) “Split Files” Button – when this is pressed the program will start.
*** Functionality ***
1) When the user presses go the program needs to read in all the search terms into memory. The search terms will in 99% of cases be less than 16 characters long (UPC codes and other similar codes).
2) The program then needs to go to the path of the input folder and open the first text file (which may or may not have .txt extension). If it can be read as text, treat as text.
3) Display “Processing: “ and the file name and “Line: “ the line that is currently being read in that text file.
4) Work through each line in the text file and do a search for ANY of the Searched text items on each line. If any line contains any of the search terms at least once then SPLIT that line out and move it to the Output file. That whole line also needs to be removed from the original file and moved to the Output file (i.e that is the “split”)
5) As the Split lines are found they need to get written out to a new file. That file should be in a subfolder of the path in \OutputXXYY (where XXand YY are the time folder created). The name of the file should be the same as the input text file name appended with _SplitOut (and.txt if it wasn’t there already). This file will grow as lines that match are split out.
6) Eventually the file will be split into to. There should be no duplicates between the input and output file and the stripped out original and the split file should now have the same amount of lines combined as the original file.
** Deliverables ***
1) Compiled 64-bit Windows executable ready to run without extra installs.
2) Source code with clear comments so I can recompile if needed.
3) Brief README explaining build steps and any runtime dependencies.
**Example**
Text File
"The big hairy dog is very scary and he moves quick
The sneaky fox is brown and very quick
The white cow is slow and friendly and happy"
Searched Text
"quick
big"
Output
The Output (new) File name should now contain these two lines
"The big hairy dog is very scary and he moves quick
The sneaky fox is brown and very quick"
The original file should now just have this line in it
"The white cow is slow and friendly and happy"