Batch Text Processing Automation

I need a reliable, repeatable way to run large batches of text data through a processing pipeline. The raw material typically lands in a folder as plain TXT or CSV files; once the script starts it should pick everything up, work through each file one after another, and write the processed results to a clearly named output directory.

Core expectations
• The workflow is fully automated: one command should launch the entire run.
• Processing steps are modular so I can easily switch individual stages on or off later.
• It must cope with thousands of lines per file without crashing or slowing to a crawl.
• Clear logging to show each file’s status and any errors that occur.
• Clean, well-commented source code plus a short README explaining setup and usage.

Preferred stack
Python is ideal—pandas for I/O, regex or NLTK/SpaCy for text handling—but if you have a faster or more elegant approach I’m open to it.

Deliverables
1. Source code (single repo or zipped folder)
2. README with setup instructions and an example command line call
3. A brief sample run demonstrating the output format on dummy data

Once I can drop a new batch in and run your tool with one line, the job is done.