Đã Đóng

Data Acceleration

Health Care Service Corporation (HCSC) is a not-for-profit corporation health insurance company in the United States. The current scope of Data Accelerator project is data ingestion and data processing. In this project we are ingesting data from different data sources to the data lake by applying the required business transformation rules and later analyzing the data for faulty records.

This project was developed using Agile development methodology having Sprint of 3 weeks.

Role and Responsibilities:

As part of each Sprint, we were allocated respective target tables to which data should be loaded.

Fixed length/Delimited flat files were loaded into the Source tables.

According to the Mapping document we design the join criteria for the source tables.

One or more source tables to be joined, applying different business rules and loading to Temp table.

Writing Scala code to perform transformations joins on data frames and removing duplicates before loading to the target table.

Developing code involved writing shell script, HQL’s, Spark-Scala code.

Zena is used for job scheduling.

Kĩ năng: Apache Hadoop, Hive

Xem nhiều hơn: technical design document, technical design document web, cut files photoshop web design, david trossell bridgeworks, bridgeworks network, detailed design document, data processing health care, invoice billing project sql database design document, level technical design document, netvibes design document, technical system design document example, simple game design document, preparing low level design document project net, sow example design document, level design document, nmea data google kml files use, effective j2ee design document, example level design document web development, data warehouse technical design document, data description in software design document

Về Bên Thuê:
( 0 nhận xét ) Hyderabad, India

ID dự án: #26375393

3 freelancer đang chào giá trung bình ₹1083/giờ cho công việc này


Hi, I am a bigdata developer and a module lead in reputed MNC.i an into the IT industry for more then 12 years. I have tonnes of experience in developing projects using Java,Apache Spark,Hive,Kafka,Scoop,Pig,Scala,aws Thêm

₹1250 INR / giờ
(1 Nhận xét)

Hi, I have 10 years of IT experience and I have worked on SQL, Python, Shell scripting, Bigdata(Hadoop), Spark(Scala), Machine Learning models, NLP, Classification and Regression models. Since the last 7 years I am w Thêm

₹1200 INR / giờ
(0 Nhận xét)

Highly-motivated,dedicated,quick learner,deadline-committed,goal-driven hadoop developer with over 3+ years of experience. Proven track record of excellence. Some of my core skills include Spark,Scala,Hive,Sqoop,Shell- Thêm

₹800 INR / giờ
(0 Nhận xét)