Build Google-cloud-native, 1 km-resolution precipitation forecasting MVP – Vertex AI + BigQuery – target ≥ 80 % skill vs ECMWF
Budget: $750 – $1,500 USD
I need an end-to-end, production-ready MVP that downloads publicly available satellite & weather data, trains a deep-learning model, and exposes a low-latency REST API delivering 1 km × 1 km daily precipitation forecasts out to 60 days.
Scope & Deliverables
Data pipeline (Apache Beam on Dataflow)
Ingest Sentinel-2, ERA5, IMERG (all free buckets)
Output: cloud-optimised GeoTIFF / Zarr cubes re-gridded to 1 km, stored in GCS & BigQuery partitioned by date.
Training pipeline (Vertex AI custom job)
Model: Temporal-Fusion-Transformer (PyTorch Lightning) – encoder 12 days, decoder 60 days.
Physics-aware loss (optionally embed ERA5 precipitation as residual).
Hyper-parameter search to beat ECMWF HRES RMSE by ≥ 15 % on held-out 2023 global data.
Target skill: ≥ 80 % relative MAE reduction vs. baseline climatology (we will verify with independent rain-gauge split).
Serving layer
FastAPI container, autoscaling Vertex AI Endpoint (GPU T4, min 0).
/predict – JSON in {“cube_id”:int, “past_seq”:[[float]]} → {“precip_mm_day”:[60 values]} latency < 200 ms p99.
/health + automatic CI/CD via Cloud Build.
Infrastructure-as-Code
Terraform script creates: GCS buckets, BigQuery datasets, Artifact Registry, Vertex AI resources, service accounts with least privilege.
Evaluation notebook
Pulls test set from BigQuery, computes MAE, RMSE, ROC-AUC for > 90th-percentile events; plots forecast vs. gauges & ECMWF.
README + one-line bootstrap
“make deploy” starts everything; “make train” kicks off Vertex job; “make eval” prints skill table.
Skills Required for this project,
-Python, PyTorch, Apache Beam, Google Cloud (Dataflow, BigQuery, Vertex AI, Cloud Build).
-Geospatial: xarray, rioxarray, rasterio, GDAL, Zarr.
-Machine-learning: time-series, transformer, hyper-parameter tuning, probabilistic forecasting.
Milestones & Payment
M1 30 % – Data pipeline live, cubes written, sample notebook shows 1 km maps.
M2 30 % – Model trained, checkpoint saved, evaluation notebook proves ≥ 80 % target skill.
M3 30 % – API deployed, Swagger docs, latency + uptime metrics shown.
M4 10 % – Terraform + full README; hand-off call complete.
Cloud Budget
We supply $500 cloud credits; keep GPU workloads on preemptible; all other steps CPU-only.
Evaluation Criteria
Success = independent test MAE at least 20 % lower than ECMWF HRES for same grid cells (we will audit with our withheld gauge set). Provide notebook evidence.
What I Will Provide
Google Cloud project ID & billing account.
List of exact data sources / URLs.
Sample 5-day training script to start from.
Slack channel for daily stand-ups.
Please reply with:
Link to similar Vertex AI + geospatial repo you built.
Estimate of hours & calendar time.
Fixed price (USD) split by milestone.
Short note on how you’ll hit ≥ 80 % skill (architecture or loss trick)
NB: I want this work to be completed within 7 weeks and payment will only be made when all work is completed and verified. Please make your work arrangements well
Scope & Deliverables
Data pipeline (Apache Beam on Dataflow)
Ingest Sentinel-2, ERA5, IMERG (all free buckets)
Output: cloud-optimised GeoTIFF / Zarr cubes re-gridded to 1 km, stored in GCS & BigQuery partitioned by date.
Training pipeline (Vertex AI custom job)
Model: Temporal-Fusion-Transformer (PyTorch Lightning) – encoder 12 days, decoder 60 days.
Physics-aware loss (optionally embed ERA5 precipitation as residual).
Hyper-parameter search to beat ECMWF HRES RMSE by ≥ 15 % on held-out 2023 global data.
Target skill: ≥ 80 % relative MAE reduction vs. baseline climatology (we will verify with independent rain-gauge split).
Serving layer
FastAPI container, autoscaling Vertex AI Endpoint (GPU T4, min 0).
/predict – JSON in {“cube_id”:int, “past_seq”:[[float]]} → {“precip_mm_day”:[60 values]} latency < 200 ms p99.
/health + automatic CI/CD via Cloud Build.
Infrastructure-as-Code
Terraform script creates: GCS buckets, BigQuery datasets, Artifact Registry, Vertex AI resources, service accounts with least privilege.
Evaluation notebook
Pulls test set from BigQuery, computes MAE, RMSE, ROC-AUC for > 90th-percentile events; plots forecast vs. gauges & ECMWF.
README + one-line bootstrap
“make deploy” starts everything; “make train” kicks off Vertex job; “make eval” prints skill table.
Skills Required for this project,
-Python, PyTorch, Apache Beam, Google Cloud (Dataflow, BigQuery, Vertex AI, Cloud Build).
-Geospatial: xarray, rioxarray, rasterio, GDAL, Zarr.
-Machine-learning: time-series, transformer, hyper-parameter tuning, probabilistic forecasting.
Milestones & Payment
M1 30 % – Data pipeline live, cubes written, sample notebook shows 1 km maps.
M2 30 % – Model trained, checkpoint saved, evaluation notebook proves ≥ 80 % target skill.
M3 30 % – API deployed, Swagger docs, latency + uptime metrics shown.
M4 10 % – Terraform + full README; hand-off call complete.
Cloud Budget
We supply $500 cloud credits; keep GPU workloads on preemptible; all other steps CPU-only.
Evaluation Criteria
Success = independent test MAE at least 20 % lower than ECMWF HRES for same grid cells (we will audit with our withheld gauge set). Provide notebook evidence.
What I Will Provide
Google Cloud project ID & billing account.
List of exact data sources / URLs.
Sample 5-day training script to start from.
Slack channel for daily stand-ups.
Please reply with:
Link to similar Vertex AI + geospatial repo you built.
Estimate of hours & calendar time.
Fixed price (USD) split by milestone.
Short note on how you’ll hit ≥ 80 % skill (architecture or loss trick)
NB: I want this work to be completed within 7 weeks and payment will only be made when all work is completed and verified. Please make your work arrangements well