Đã Đóng

Developing ETL pipeline using Azure, SSIS, Python ,SQL

I have to build an ETL pipeline of a data from a collaborating hospital data csv file.

Goal: Store the data in a cleaned and structured format into a database/file of choice. Write the code in Python or language of choice. Design a solution that can be scaled to TB of records.

Steps:

1. Make assumptions and justify them where things are unclear with comments in the code.

2. Write unit tests for all your functions.

3. Write data tests to ensure that the data is correct.

4. Remove Protected health information (PHI): Names, Addresses etc.

5. Clean data. Remove invalid values. Normalize it where reasonable.

6. Add a column that calculates the average of all three glucose measurement time points.

7. Add a column based on the average of all three glucose measurement time points that indicates whether it’s normal, prediabetes or diabetes.

8. Store data in a database or file format of choice.

Kĩ năng: Python, Lưu trữ dữ liệu, Quản trị cơ sở dữ liệu, ETL, MySQL

Xem nhiều hơn: books developing mobile applications using net35, ssis without sql 2000, developing prado apps using zend, developing online quiz using php mysql, using xcode ruby python php perl development, developing tabed menu using javascript, project developing online store using aspnet, tomcat version developing web services using jdk14 eclipse, etl project using sigma, developing data base using, using adwords api integrates sql, developing wap sites using aspnet, python etl pipeline, etl automation using python, azure devops python pipeline, python etl pipeline example, trigger azure data factory pipeline using rest api, modular image processing pipeline using opencv and python generators, etl pipeline python

Về Bên Thuê:
( 2 nhận xét ) DUBLIN, United States

ID dự án: #29444371

7 freelancer chào giá trung bình$120 cho công việc này

SqlDevelopment

I can qualitatively design and develop required ETL using MS SQL Server because I am Senior MS SQL/BI Developer with more than 10 years of exceptional professional experience.

$130 USD trong 3 ngày
(30 Nhận xét)
5.0
sreenivas2903

Hello i am expertise in sql queries and etl processing using ssis ping me if you are interested and give more information

$200 USD trong 3 ngày
(9 Nhận xét)
4.0
JohnnyZhu

Hi. I will suggest to use excel Power Query for data retrieval from files and the manipulation. Please chat for more detail. Johnny

$200 USD trong 7 ngày
(1 Nhận xét)
2.8
TheCrucial

Hi, I'm interested in Data Science. I worked SparkSql. I can deliver in 5 days. Working in coordination is my priority. I hope you contact me. Best Regards.

$170 USD trong 5 ngày
(1 Nhận xét)
2.6
Aoppp4

PYTHON JAVA PHP CSS HTML WOOCOMMERCE WORDPRESS CYBERSECURITY I'm a Linux Professional with over 5+ years of verifiable experience in the Web Hosting industry, I'm in the ideal position to offer a wide variety of Linux Thêm

$20 USD trong 7 ngày
(0 Nhận xét)
0.0
iammridulgupta

Hello, I am an experienced ETL /BI developer with around 5 years of experience working in data analysis and ETL development for retail and e-commerce clients. Delivered more than 50 dashboards and ETL solutions using S Thêm

$20 USD trong 4 ngày
(0 Nhận xét)
0.0
Mikeayus

Hello, I am a Microsoft Certified Data Analyst and Business Intelligence Developer and Trainer with over 3 years experience building enterprise data warehouses, data analytics and business intelligence models, reports Thêm

$100 USD trong 7 ngày
(0 Nhận xét)
0.0