Realtime GPS Data Processor

Job ID: 37572065

Budget: $10 – $200 USD

I'm searching for an expert who can speedily develop a lightweight processor in Golang. This processor will need to handle a massive amount of AVL data- one million+ entries every second- from a fleet in realtime.

OVERVIEW:
This Processor is designed to handle and processing a massive amount AVL real-time data, including longitude, latitude, altitude, speed, direction, accelerometer (CANbus Values"Based on availability") data for stationary movements, visible satellites, and polyline information.

KEY REQUIREMENTS:
- Competency in Golang (Lightweight Processor)
- The processor must be capable to simulating missing data, calibrate and remove noise
- GPS coordinates Calibration, noise removal and simulation (when is missing)
- Direction Calibration, noise removal and simulation (when is missing)
- Vehicle Movement Smooth Polyline on the Road Calibration, noise removal, road matching and simulation (when is missing)
- Adapt automatically to any data interval/frequency (ex: every 5 - 10 - 15 - 30 sec,...etc)
- Process must handle multiple AVL data frequency at the same time
- Traveling Distance Calibration, noise removal, simulation (when is missing)
- Speed Calibration, simulation and noise removal
- Stationary Vehicle Movement Calibration and noise removal
- The generated Data Must have a high accuracy
- Processed data must be consumed in real time
- Each processes data must include the AVL_ID - Timestamp
- Missing Ignition Simulated when is missing (Using Mobile_Device)
- CAN Bus/OBD data (Speed/Odometer/Movement) Based on availability

DEPENDENCY:
- AVL_ID or/and Mobile Device_ID
- Timestamp
- Ignition Status (Vehicle ON/OFF)
- GPS Coordinates
- Gyroscope Data (If available)
- G-sensor & Accelerometer
- OSRM (Open Street Map) (Free Service)
- CAN Bus/OBD Accuracy Validation (Based On availability)

TECHNICAL NEEDED:
- Kalman Filter for GPS and Accelerometer Data (Used for filtering noise and inaccuracies in the GPS and accelerometer data)
- Bezier Curve Algorithm (Used for smoothing the polylines on the road.)
- Linear Interpolation (Used for generating missing coordinates.)
- OSRM (OpenStreetMap Routing Machine): Used for exact vehicle road matching.
- InfluxDB. (The database where all processed and raw data could be stored for further analytics.

BEST PRACTICE:
- Stand alone Processor (Microservice Architecture)
- Event-Driven Architecture
- Stream Processing
- Data Sharding
- Efficient Algorithms and Libraries
- In-Memory Caches (e.g., Redis)
- Dynamic Load Balancing
- Rate Limiting and Back-Pressure Management
- Resource Monitoring and Alerting
- Database Optimization Techniques

The intended data source is AVL devices or/and Mobile Apps. Experience with these sources is preferable. The project timeline is aggressive, ideally less than 2 weeks. If you have proven experience in handling large, complex data sets in Golang within tight timeframes, I want to hear from you.