Production-Ready Video Recommendation Algorithm Enhancement

Senior Recommendation Algorithm Engineer (Fix & Harden Production System)

Project Overview

We have a production-grade video recommendation algorithm (short-form + long-form) built using:
    •    Node.js / Express
    •    Kafka, Redis (feature store)
    •    ClickHouse (real-time analytics)
    •    Milvus (vector embeddings)
    •    Contextual bandits + exploration logic

The system architecture is strong, but before public launch we need a senior-level expert to audit, fix, and harden the algorithm for real-world scale.

This is not a beginner project. We are looking for someone who has actually built or worked on recommender systems at scale.

⸻

What Needs to Be Fixed (Critical)

You will be responsible for:
    1.    Fixing Redis feature-store math
    •    Correct unstable online averages
    •    Ensure atomic, statistically valid updates
    2.    Algorithm simplification / correction
    •    Decide:
    •    Pure bandit v1 OR
    •    Proper ML integration (no fake ML layers)
    •    Remove misleading or non-functional components
    3.    Collaborative filtering optimization
    •    Replace expensive ClickHouse runtime queries
    •    Introduce precomputed or cached similarity logic
    4.    Cold-start & exploration improvements
    •    Ensure new users & new creators get fair exposure
    •    Prevent trending-only bias
    5.    Adversarial / gaming resistance
    •    Detect fake engagement, bot-like behavior
    •    Prevent score manipulation
    6.    Production safety
    •    Add kill-switches / feature flags
    •    Externalize ranking weights
    •    Safe fallback feed logic