Apache Spark projects — HireLancer archive

ID / Date Listing

Basic Apache Spark Setup

I’ve decided to move forward with Apache Spark and I’m looking for a straightforward, entry-level setup I can build on later. For now, I need: • A concise worki...

HL-39882440

Databricks Notebook with Dynamic Filters

I need a Databricks SQL notebook that connects directly to our existing Delta Lake tables and produces an easy-to-explore summary-statistics report. The key to...

HL-39875626

Türkçe Veri Düzenleme ve Temizlik 500 GB Turkish Text Corpus Cleaning for LLM Pretraining

Türkçe İlan Metni Başlık: 500 GB Türkçe Metin Korpusu Temizliği için Veri Mühendisi Projeye dair: Türkçe odaklı büyük bir dil modeli pre-train etmek için 500 GB...

HL-39817967

Advanced Scala & Databricks Training

I need an intensive, advanced-level workshop for a very small team (1–5 engineers) that zeroes in on two areas: sophisticated Scala programming techniques and t...

HL-39768277

Java spring batch and Spark scala Data Engineering based

Java Spring batch Lombok and Rest API get post is hugely recommended Also should be able to explain and demonstrate work while pair programming with me. JSON Sh...

HL-39761897

Big Data Architect (Databricks & AWS Specialist)

Data Architect – Big Data & Cloud (Databricks, AWS, Redshift, Glue) Contract Duration: 6 months to 1 year (Extendable) Experience Required: 10+ Years Total | 2–...

HL-39760134

Enterprise ETL Framework Enhancement

I'm seeking an experienced professional to support the enhancement of our enterprise ETL framework designed for data processing using big data technologies on A...

HL-39752741

Data Engineering Support Needed

I need an experienced data engineer to assist with an ongoing project. Key Responsibilities: - Integrate various data sources: databases, APIs, and files (CSV,...

HL-39748485

Data Science Internship Tasks Help -- 4

I’m midway through my internship at Devnest and have been handed seven separate challenges that must be wrapped up before the final review. The scope spans the...

HL-39719203

Senior Data Engineer Needed

I'm seeking an experienced Data Engineer to join my team. The ideal candidate should have extensive experience with Apache Spark, and be familiar with a variety...

HL-39682026

Data Engineer Autosys/Unix Pro for Long-Term Support

We are looking for a professional with Autosys and Unix experience to provide 1–2 hours per day of long-term support on an internal project. Project Overview: T...

HL-39659158

Apache Spark Contribution Mentor

I am seeking a highly experienced Apache Spark developer to assist and mentor my intermediate-level team in contributing to the official Apache Spark open-sourc...

HL-39630207

Spark Migration to Kubernetes Cluster

I need assistance migrating from Spark client mode to Spark cluster mode on Kubernetes for my office project, and upgrading to Spark 4.0. Currently, I'm using S...

HL-39603717

PySpark Log Table Creation

I need a PySpark script modification to store error logs in a Databricks Delta table instead of creating files in ADLS. Requirements: - Use Databricks as the da...

HL-39586041

Expert for Spatial Queries on MySQL

I need an expert to perform spatial queries on a MySQL database, specifically for polygon data. Key Requirements: - Proficiency in MySQL, Apache spark - Experie...

HL-39571955

Two Existing Public Datasets Combined in One

Combining 2 existing datasets, using a combination of programming tools and data manipulation techniques. Python with libraries like Pandas are well-suited for...

HL-39535673

Build an Apache Iceberg rewrite tool for version-1.4.3 (Backend)

I’m looking for a freelancer with strong experience in Apache Iceberg (specifically version 1.4.3), Java, and metadata handling to help develop a high-performan...

HL-39421167

Apache Spark developer needed to create notes on topics

I’m looking for a freelancer with deep expertise in Apache Spark to help create high-quality technical content.

HL-39416912

eBook Writer Needed – Apache Spark with Scala (Developer-Level Content with Code & Screenshots)

Description: I’m looking to hire an experienced Scala + Apache Spark developer to create a comprehensive and well-structured eBook based on a list of topics and...

HL-39381630

Azure Databricks Workflow Debugging

I'm encountering an 'Analysis Exception' error in my Azure Databricks workflow, specifically failing to merge fields 'sales_val'. My lead wants me to debug this...

HL-39353545

Spark Streaming & MLLib Project

I'm looking for an expert in Spark Streaming and MLLib to assist with a data analysis and reporting project. The data we'll be working with is boarding data, an...

HL-39338755

Spark Scala Java spring batch Data Engineering based

Design, code, test as a Data Engineer or Big Data developer Spark Scala Databricks, ETL on Azure AWS mandatory You should have working experience in previous pr...

HL-39304104

Data Engineer for GCP & Apache Spark

I'm looking for a seasoned Data Engineer with over 3 years of experience. The project involves deploying solutions on Google Cloud Platform and requires profici...

HL-39251432

Personal Data Engineering Mentor Required

Hi, I’m looking for a Data Engineering mentor or trainer who can personally guide me through all AWS-related tools used in modern data engineering workflows. My...

HL-39242766

Scala-Spark DataFrame Issue Troubleshooting

I am currently facing an issue with a Scala-Spark job. Instead of receiving expected column values, I'm getting column names. This issue is occurring while perf...

HL-39172872

Databricks Data Engineer for Delta Modelling

We are seeking highly skilled databricks data engineer who have experience in building data pipelines and data modelling in delta tables.Candidate should have a...

HL-39115554

Big Data Consultant Needed for Optimization

I'm in need of a seasoned Big Data Consultant with over 8 years of hands-on experience in optimized architecture. The ideal candidate will have a strong backgro...

HL-39108679

Expert Canva Designer for Software Engineering Presentation

I'm seeking a skilled Canva Presentation expert to develop slides for an educational project centered on Software Engineering, specifically covering Apache Spar...

HL-39045207

Data Engineer Required for JSON Cleanup

I'm in need of a skilled Data Engineer with a strong background in PySpark. The primary task will be cleaning and preprocessing my JSON files. The data inconsis...

HL-38959509

Exploratory data analysis inJupiter

I am looking for someone who can do Exploratory data analysis in Jupiter notebook in spark

HL-38949455

E-commerce Data Analysis with Apache Spark

I'm looking for a freelancer who can process an e-commerce dataset using the Apache Spark ML API in a Jupyter notebook. The main aim of this project is to gener...

HL-38941133

Senior Data Engineer

I am looking for an experienced Senior Data Engineer to join our team. The ideal candidate should have a strong background in data engineering, particularly wit...

HL-38746799

Install and Setup Apache Spark MLlib on Linux Ubuntu

I need an expert to set up Apache Spark MLlib on my local Linux Ubuntu server. TASK: once you are hired, you will need to install and setup Apache Spark MLlib o...

HL-38706308

Apache Spark Assignment

Link to task (ONLY TASK 1 required): https://hackmd.io/@pesu-bigdata/ryDbjmTkke#Task-1---Spark This document contains 2 tasks : Task 1 : This task is to be solv...

HL-38588113

Azure Synapse Analytics Spark Monitoring

I'm in need of an Azure Cloud Engineer with proven expertise in Azure Synapse Analytics and Spark monitoring. The primary focus of this role is on big data anal...

HL-38581870

Teach me Data Engineering and make me job ready

I'm aiming to become job-ready in the Data Engineering sphere, with a primary concentration on Data Pipeline Development. I require a comprehensive training pro...

HL-38526654

Spark Job Producing Incorrect Results or exiting with error code 1.

I'm facing an issue with my Spark job. The job runs, but it exits with an error. I'm processing structured data, specifically CSV and JSON files, from my local...

HL-38415790

Apache Spark Workflow for Real-time Analytics

I'm looking for an expert in Apache Spark to help me create a base workflow execution library. This library is primarily aimed at managing workflows which inher...

HL-38410946

Cloud Deployment: Airflow & Spark POC

I'm looking for an expert to help me deploy Airflow and Spark in the cloud, specifically on Azure using the Azure Kubernetes Service (AKS). The project involves...

HL-38299517

Remote Sr. Data Platform Engineer (Spark & Scala) in India for 12 months

Role Proficiency: Develop data-driven solutions for difficult business challenges. Utilizing analytical statistical and programming skills to collect analyze an...