Re-factor single threaded file hashing sample code to be multi-threaded with improved performance / Win32 C++ (Nov 2021)
Budget: £20 – £250 GBP
We are looking for an experienced software developer to re-factor a simple single-threaded sample program to be multi-threaded with improved performance. Depending on the quality of the code produced and the performance achieved, the project may then develop into a more fully featured application and a long term relationship with the right developer. We are looking for an elegant and creative solution.
The sample code is attached to this post.
Objective:
Achieve maximum search performance possible subject to the constraints below. Target performance >= 1GB/sec on modern hardware
Prerequisites:
1. Code to be written in C++ using VS 2017 or later using C++11 or later
2. Code designed to run on Windows 10 or later
3. Threads must use blocking unbuffered IO (like sample code). Other solutions will not be accepted.
4. Copyright in any winning solution will be assigned to us on payment
Specification:
The sample program reads a large binary file, calculates the MD5 hash value and writes the file to a second file. This process is inefficient because it is done using a single thread and the hash calculation must be done sequentially. The current approach is read-write-hash. Please re-factor the sample code to be multi-threaded. For testing purposes, we have limited the sample code to only use basic unbuffered blocking IO. Please do not change this aspect. It should not matter if the multi-threaded solution is efficient.
Possible solution:
1. Allocate a pool of N worker threads, each with a memory buffer
2. Use N-1 threads to read data chunks from input file and write to output file
3. Use final thread to calculate MD5 hash value
4. When a thread hash finished read-write-hash, re-cycle thread to work on another chunk
Notes:
1. Input file can be read in any order (but sequential would probably make more sense)
2. Output file must be written sequentially and match input file on completion
3. Hashing must be done sequentially. This cannot be avoided due to the MD5 calculation
4. Sample code includes a function to generate a 8GB sample binary file
5. A more advanced solution may allow the number of threads to be configured above a minimum number
We would welcome intelligent questions and discussion about the project. We will award the project to the developer who best describes a good solution. Payment is upon delivery of a working solution and source code.
The sample code is attached to this post.
Objective:
Achieve maximum search performance possible subject to the constraints below. Target performance >= 1GB/sec on modern hardware
Prerequisites:
1. Code to be written in C++ using VS 2017 or later using C++11 or later
2. Code designed to run on Windows 10 or later
3. Threads must use blocking unbuffered IO (like sample code). Other solutions will not be accepted.
4. Copyright in any winning solution will be assigned to us on payment
Specification:
The sample program reads a large binary file, calculates the MD5 hash value and writes the file to a second file. This process is inefficient because it is done using a single thread and the hash calculation must be done sequentially. The current approach is read-write-hash. Please re-factor the sample code to be multi-threaded. For testing purposes, we have limited the sample code to only use basic unbuffered blocking IO. Please do not change this aspect. It should not matter if the multi-threaded solution is efficient.
Possible solution:
1. Allocate a pool of N worker threads, each with a memory buffer
2. Use N-1 threads to read data chunks from input file and write to output file
3. Use final thread to calculate MD5 hash value
4. When a thread hash finished read-write-hash, re-cycle thread to work on another chunk
Notes:
1. Input file can be read in any order (but sequential would probably make more sense)
2. Output file must be written sequentially and match input file on completion
3. Hashing must be done sequentially. This cannot be avoided due to the MD5 calculation
4. Sample code includes a function to generate a 8GB sample binary file
5. A more advanced solution may allow the number of threads to be configured above a minimum number
We would welcome intelligent questions and discussion about the project. We will award the project to the developer who best describes a good solution. Payment is upon delivery of a working solution and source code.