Spark Scla

$ 2.00 — 8.00 USD

Expertise in designing and deployment of Hadoop Cluster and different analytical tools including Pig, Hive, HBase, Sqoop, Kafka Spark with Cloudera distribution.

Working on a live 20 nodes Hadoop cluster running on CDH4.4.

Working with highly unstructured and semi structured data of 40 TB in size (120 TB with replication factor of 3)
Managing external tables in Hive for optimized performance.

Very good understanding of Partitions and Bucketing in Hive
Developed Spark scripts using Scala as per the requirement using Spark 1.5 framework.

Using Spark API’s over Cloudera Hadoop Yarn to perform analytics on data used for Hive stored at HDFS.

Developed Scala Scripts, UDFs using both Data frames/SQL and RDD in Spark for data aggregation, queries and writing data back onto HDFS.

Exploring Spark to improve the performance and optimization of the existing algorithms in Hadoop using Spark context, Spark data frames, pair RDDs, double RDDs and Yarn.

Developed Spark code and Spark-SQL/Streaming for faster testing and processing of data.

Experience in deploying data from various sources into HDFS and facilitating report building on top of it as per the business requirement.

Performed transformations, cleaning, standardization and filtering of data using Spark Scala/Python and loaded the final required data to HDFS.

Load the data into Spark immutable RDDs and perform in-memory computation to generate quick and better response.

Analyzing how the data been processed by Informatica can be effectively processed using Spark and its API’s.

SQLHadoopScalaHiveSpark

Similar Freelance jobs:

Implantação Do Gateway De Pagamentos Getnet

Design, Media & ArchitectureSQLC# ProgrammingSoftware ArchitectureMySQLPayment Gateway IntegrationLocal JobResearch and DevelopmentDevelopmentVisual Studio Place Bid

Temos plataforma de e-commerce própria e gostaria de integrar o gateway de pagamentos da GetNet. Para integração da nossa API, 0 desenvolvedor terá que se basear na nossa documentação: https://protect2.fireeye.com/v1/url?k=698967fd-36125a5b-698baa44-ac1f6b1e02f6-a37a19790941a8d5&q=1&e=89f19824-e52a-467f-9fd6-728ca3b62687&u=https%3A%2F%2Fdevelopers.getnet.com.br%2Fapi Obs: Banco de Dados MySQL e código fonte desenvolvido utilizando Visual Studio 2013

Full Description of Implantação do gateway de pagamentos GetNet

Vpn App (v2ray+trojan+shadowsocksr) 5 O.s (ios+android+windows+mac+linux)

LinuxMobile App DevelopmentAndroidSQLMac OSC++ ProgrammingWindows ServerSoftware DevelopmentSwiftVirtual MachinesiOS Development Place Bid

smooth and fast VPN using protocols: V2ray-trojan-ssr vpn app with my own Ui & a basic backend panel using php+mysql i need app for (ios + android + windows + mac + linux ) = minimum $1500 / maximum $5000 for whole project, official programing language only ! not flutter-xcode-react,js...! Microsoft studio = windows version Android studio = android version Swift = ios//mac version C ,c++ = linux version i provide: Ui + panel template + servers + openSource vpn…

Full Description of VPN APP (V2ray+trojan+shadowsocksR) 5 O.S (ios+android+windows+mac+linux)

3 Basic Sql And Html Exercises (need It In The Next 24 Hours)

Data ProcessingData EntrySQLHTMLData Analysis Place Bid

You will find instructions in the PDF attached and then please resolve each Excel file according to the instructions inside each of it and leave the solution in a separate tab/sheet created in the same excel file. The solving deadline is June 23rd 6.00 PM (GMT+1)

Full Description of 3 basic SQL and HTML Exercises (need it…

I Need Scala Expert Right Now

JavaMobile Phones & ComputingSoftware ArchitectureGolangScala Place Bid

Hello. I need a scala expert who can start right now. thank you.

Full Description of I need Scala expert right now

Problem With Spark Ar Filter

InstagramSpark Place Bid

Need filter for instagram to be fixed. The image should be inside but instead it is cut off as seen in the image

Full Description of problem with Spark AR filter

Database Management Systems

SQLDatabase Programming Place Bid

Database Management Systems Database Management Systems

Full Description of Database Management Systems

Create An Mqtt Broker On Our Server Integrated With Postgre Sql

SQLMQTTInternet of Things (IoT) Place Bid

Please read this project very carefully, do not send me any private messages. Please send your bid titled 'I Can Create An MQTT Broker Integrated With Postgre sql' We have a project that is an Emergency help call system for the elderly. It combines unique hardware developed by us that is both an IP Phone and an MQTT Client. Our central database on which the Asterisk PBX operates is in Postgre sql. We want to create an MQTT Broker on…

Full Description of Create an MQTT Broker On Our Server Integrated…

I Need A Net Programmer

.NETSQLC# ProgrammingASP.NETMicrosoft SQL Server Place Bid

I want to create a form with aspnet,html5,c# using mssql2014 databse. This form function to to scan barcode and automatic save data barcode to database, and scan again (repetitive).

Full Description of I need a Net Programmer

Develop Accounting Module In Python

PythonSQLSoftware ArchitectureMySQLMicrosoft SQL Server Place Bid

fully accounting package in python & MS-SQL

Full Description of develop accounting module in python

Maintain Our Web Application

Engineering & Science.NETSQLASP.NETMicrosoft SQL Server Place Bid

We have a WEB Module to Administrate our Retail Management System. We need to modify and maintain the web application from JULY 2021 on wards. The WEB Application developed in asp.net using vb.net and RAD Grid. Backend is MS SQL Server.

Full Description of Maintain our WEB Application

Oracle, Sql

SQLOracleMySQLDatabase AdministrationMicrosoft SQL Server Place Bid

Hello, I have some issues on my sql project, need help to fix my inserts and 2 trigger.

Full Description of Oracle, SQL

Test Automation Engineer – Brussels (belgium) – 6 Months+ Freelance

SQLHP-UX Place Bid

Urgently Required Senior .NET Tester - Remote Work Job description Test Automation Engineer – BRUSSELS (BELGIUM) – 6 MONTHS+ FREELANCE I am looking for a Test Automation Engineer with experience in regression testing for a long-term freelance contract. This project would be fully automating their current manual testing methodologies. The project is the creation of a new pension software that will be delivered to the client’s customers. Role: - You will consult as a Test Automation Engineer for a company…

Full Description of Test Automation Engineer – BRUSSELS (BELGIUM) – 6…

Hcl Notes Db Development (message Data)

SQLLotus NotesDatabase Programming Place Bid

saving (Temperature data, performance data) machines data # import tool from SQL and scv # Live Stream Data every # data frequency up to 0.5 seconds # simply report # notes nomad

Full Description of HCL Notes DB development (message data)

App Building -- 2

Design, Media & ArchitectureMobile App DevelopmentiPhoneAndroidSQL Place Bid

app building for the online business

Full Description of app building -- 2

Oracle Apex Senior Developer

JavaSQLOracleDatabase ProgrammingOracle APEX Place Bid

Looking for a Senior Oracle Apex developer to help move a server-based system to the cloud. please apply if you are a senior and have done Apex work before. will need to see work example you have done.

Full Description of Oracle Apex Senior Developer