I want to connect with a Pyspark developer who can help me in writing pyspark code to ingest XML files dynamically to Hive tables

Job ID: 33279152

Budget: $250 – $750 SGD

Autocreation of hive tables:
get the attribute name,data type,size from xsd
mapping data type to hive compatibility data type
specify table partition column
create table structure script
connecting to hive and exeute the script

Create a XML parser using pyspark to
Parsing Manifest file:
Write a script to store data in table
Combining Manifest file with Master data:
generate metadata list of parameter reuire to run the loading
XML Validation:
Create a script to read XML/XSD from metadata table
Import XML packages
XML filename,formatutf-8,filesize validation
XML against XSD validation
XML validation Fail and sucess scenario

Write a wrapper shell script for job invocation

XML to table:
Read xml file and load to hive table
Related categories: Shell Script Hive PySpark