Python code find top most similar row pairs in matrix
Budget: $30 – $250 CAD
given numpy matrix with 900000 rows and 1000 columns
data = np.random.randn(num_vectors, num_features).astype('float32')
develop FAST, correct and reasonable for RAM python code to find top K (for example 3000) most similar rows and least similar rows.
for similarity use cosine similarity
compare with ground truth given by
from sklearn.metrics.pairwise import cosine_similarity
just idea
you may use
faiss or hnswlib or annoy or other python packages easy to install on both windows and unix
before application , tell me how long it takes for your python code to do this calualtions
time 2 days
data = np.random.randn(num_vectors, num_features).astype('float32')
develop FAST, correct and reasonable for RAM python code to find top K (for example 3000) most similar rows and least similar rows.
for similarity use cosine similarity
compare with ground truth given by
from sklearn.metrics.pairwise import cosine_similarity
just idea
you may use
faiss or hnswlib or annoy or other python packages easy to install on both windows and unix
before application , tell me how long it takes for your python code to do this calualtions
time 2 days