C: Data search script
Budget: $99 – $100 USD
There is a project on C (not C++ or C#) that returns uint64_t array with 9216 units.
On the other side I have a file with more than 1000000 strings that also has a data with unsigned integers with different. length. From 0 to (2^64)-1.
So, I need to get best solution for searching 9216 uint returning from function in loaded data from file which contains 1000000 or more uints.
I have experimented with mySQL via python, but I got time search from 2.5 to 3 second on each iteration which is very slow. Maybe byte to byte comparison, or array to array or something will much faster.
Number of iterations with 9216 uint can be vary and it depends on other stuff from whole project.
Current project speed without comparison. Just with uploading result to file on disk is about 1085 iterations with 9216 uint lines per second. Using other solution with comparison developed on python and using fast ssd the speed is about 650 iterations with 9216 uint lines per second.
+++++++++++++
What I want at the end? I want avoid using disk which is bottleneck with using all in one algo that provides data search on the fly.
Project hard cap is $100.
That's it.
On the other side I have a file with more than 1000000 strings that also has a data with unsigned integers with different. length. From 0 to (2^64)-1.
So, I need to get best solution for searching 9216 uint returning from function in loaded data from file which contains 1000000 or more uints.
I have experimented with mySQL via python, but I got time search from 2.5 to 3 second on each iteration which is very slow. Maybe byte to byte comparison, or array to array or something will much faster.
Number of iterations with 9216 uint can be vary and it depends on other stuff from whole project.
Current project speed without comparison. Just with uploading result to file on disk is about 1085 iterations with 9216 uint lines per second. Using other solution with comparison developed on python and using fast ssd the speed is about 650 iterations with 9216 uint lines per second.
+++++++++++++
What I want at the end? I want avoid using disk which is bottleneck with using all in one algo that provides data search on the fly.
Project hard cap is $100.
That's it.