Big Data Developer with Spark and Pytoch for creating Recommendations Engine
Budget: $2 – $8 USD
*Job Title: Big Data Developer*
*Job Description:*
We are seeking a skilled Big Data Developer to join our team and work on generating daily job recommendations for our platform. The ideal candidate will have experience with big data technologies and cloud services, particularly within the Oracle Cloud ecosystem.
*Responsibilities:*
- Develop and optimize PySpark applications for processing large datasets.
- Implement machine learning models using PyTorch for recommendation systems.
- Manage data storage and processing using Oracle Cloud Infrastructure (OCI) services.
- Load processed data into Elasticsearch and MySQL for efficient retrieval.
- Automate workflows using scheduling tools like Apache Airflow.
- Monitor and improve job performance and resource utilization.
*Technologies Used:*
- *Apache Spark*: For large-scale data processing.
- *PyTorch*: For building machine learning models.
- *Oracle Cloud Infrastructure (OCI)*: For managing big data environments.
- *Elasticsearch*: For indexing and searching recommendations.
- *MySQL*: For relational data storage.
- *Apache Airflow*: For workflow automation and scheduling.
*Qualifications:*
- Proven experience with Spark and PyTorch in a cloud environment.
- Familiarity with Oracle Cloud services, especially Big Data Service.
- Strong knowledge of Elasticsearch and MySQL databases.
- Experience with data processing and ETL pipelines.
- Excellent problem-solving skills and attention to detail.
Join us to leverage cutting-edge technologies in a dynamic environment, delivering impactful solutions for our users.
*Job Description:*
We are seeking a skilled Big Data Developer to join our team and work on generating daily job recommendations for our platform. The ideal candidate will have experience with big data technologies and cloud services, particularly within the Oracle Cloud ecosystem.
*Responsibilities:*
- Develop and optimize PySpark applications for processing large datasets.
- Implement machine learning models using PyTorch for recommendation systems.
- Manage data storage and processing using Oracle Cloud Infrastructure (OCI) services.
- Load processed data into Elasticsearch and MySQL for efficient retrieval.
- Automate workflows using scheduling tools like Apache Airflow.
- Monitor and improve job performance and resource utilization.
*Technologies Used:*
- *Apache Spark*: For large-scale data processing.
- *PyTorch*: For building machine learning models.
- *Oracle Cloud Infrastructure (OCI)*: For managing big data environments.
- *Elasticsearch*: For indexing and searching recommendations.
- *MySQL*: For relational data storage.
- *Apache Airflow*: For workflow automation and scheduling.
*Qualifications:*
- Proven experience with Spark and PyTorch in a cloud environment.
- Familiarity with Oracle Cloud services, especially Big Data Service.
- Strong knowledge of Elasticsearch and MySQL databases.
- Experience with data processing and ETL pipelines.
- Excellent problem-solving skills and attention to detail.
Join us to leverage cutting-edge technologies in a dynamic environment, delivering impactful solutions for our users.