
Đã đóng
Đã đăng vào
Thanh toán khi bàn giao
I need an experienced Python engineer who works confidently with AWS Glue to build and manage a small suite of data-integration jobs for a Hyderabad-based project. The core of the work is to design and automate Glue ETL pipelines that pull data from our production databases, catalog it accurately, and transform it into analytics-ready tables. Here is what I expect from the engagement: • Develop, test, and deploy Glue ETL jobs in Python. • Populate and maintain the Glue Data Catalog so new tables are discoverable and properly version-tracked. • Implement efficient transformation logic that cleans, enriches, and partitions data for downstream reporting. • Optimise job performance and cost by selecting the right worker types, job parameters, and database connections. Our source systems are relational databases, so experience configuring Glue connections, crawlers, and dynamic frames against JDBC endpoints is essential. Familiarity with ancillary AWS services—such as IAM for fine-grained permissions, S3 for staging, and CloudWatch for logging—will help keep the pipelines rock-solid. Deliverables will be accepted when: 1. All Glue jobs run from start to finish without manual intervention. 2. Transformed data lands in the specified S3 buckets and matches the target schema. 3. The Data Catalog reflects every table, column, and partition produced by the pipelines. 4. Code is organised in a Git repo with clear README and parameterisation for dev, test, and prod. I prefer someone already in Hyderabad so we can schedule occasional in-person white-boarding sessions, but day-to-day work can remain remote. If this sounds like a good fit, please share examples of previous Glue jobs or ETL code you have delivered.
Mã dự án: 40226885
11 đề xuất
Dự án từ xa
Hoạt động 24 ngày trước
Thiết lập ngân sách và thời gian
Nhận thanh toán cho công việc
Phác thảo đề xuất của bạn
Miễn phí đăng ký và cháo giá cho công việc
11 freelancer chào giá trung bình ₹7.968 INR cho công việc này

Hi there, I understand your need for an experienced Python engineer to develop and manage AWS Glue ETL pipelines tailored for your Hyderabad project. With over 7 years in cloud and DevOps, I specialize in designing automated, efficient Glue jobs and Data Catalog management to ensure seamless data integration and transformation. - Develop, test, and deploy robust Glue ETL Python jobs - Configure JDBC connections and Glue crawlers for relational data - Maintain Glue Data Catalog with accurate version tracking - Optimize job performance and costs via precise resource tuning - Organize code in Git with comprehensive README and environment parameterization **Skills:** ✅ AWS Glue ETL development & Python scripting ✅ JDBC endpoint integration & Glue Data Catalog management ✅ IAM, S3, CloudWatch for security and pipeline monitoring ✅ Git version control with dev/test/prod configurations ✅ Data transformation for analytics-ready output **Certificates:** ✅ Microsoft® Certified: MCSA | MCSE | MCT ✅ cPanel® & WHM Certified CWSA-2 I am confident I can deliver fully automated Glue jobs that meet your project standards and provide documentation for smooth handovers. Ready to discuss the next steps and timeline for your project. Could you share more details about the current data volume and the frequency of your Glue job runs? Best regards,
₹12.000 INR trong 14 ngày
4,6
4,6

Hi there, I am a strong fit for this scope because I have built and maintained AWS Glue ETL pipelines in Python that move relational data into analytics-ready S3 layers with full catalog management. I have developed Glue jobs using DynamicFrames and Spark transformations, configured JDBC connections and crawlers, managed IAM permissions, and optimized worker types and partition strategies for performance and cost control. I structure pipelines with clear separation between extraction, transformation, and load stages, parameterize environments for dev, test, and prod, and maintain clean Git repositories with documented deployment steps. I reduce risk by validating schema alignment against target models, implementing structured logging in CloudWatch, and testing full end-to-end runs to ensure jobs complete without manual intervention. I am available to start immediately and can align on milestones and in-person sessions in Hyderabad as needed. Regards Chirag
₹7.000 INR trong 7 ngày
4,4
4,4

With 7 years of experience in Python development and AWS Glue, I am the best fit for this Hyderabad-based project. I have the relevant skills to design, automate, and manage data-integration jobs using AWS Glue ETL pipelines. **How I will complete this project:** - Develop, test, and deploy Glue ETL jobs in Python. - Populate and maintain the Glue Data Catalog for discoverability and version tracking. - Implement transformation logic for cleaning, enriching, and partitioning data. - Optimise job performance and cost for efficient processing. **Tech stack I will use:** - AWS Glue for ETL pipelines. - Python for scripting. - Relational databases for source data. - IAM, S3, and CloudWatch for ancillary services. I have worked on similar solutions in the past, ensuring seamless data integration and transformation. I will configure Glue connections, crawlers, and dynamic frames against JDBC endpoints to meet the project requirements. I will ensure that all Glue jobs run without manual intervention, data lands in specified S3 buckets, and the Data Catalog reflects every table and partition. Code will be organised in a Git repo with clear documentation and parameterisation for different environments. If you are interested in collaborating with a seasoned Python engineer with AWS Glue expertise, I am ready to discuss further details and share my previous Glue job examples.
₹1.650 INR trong 7 ngày
2,0
2,0

With nearly a decade of experience under my belt, I have confidently built and managed a wide range of data-integration projects similar to yours. What sets me apart is my expertise with AWS Glue and Python, as well as the comprehensive understanding of using ancillary AWS services like IAM, S3, and CloudWatch for robust and efficient solutions. I’m also well-versed in organizing code in Git repositories with detailed README documentation and parameterization for various environments. Being based in Hyderabad, I perfectly fit your preference for in-person white-boarding sessions while being available for day-to-day remote collaboration within the project. This allows me to ensure that we deliver exactly what you envision with minimal need for manual interventions. Overall, my top priority is to design and automate Glue ETL pipelines that not only pull data from production databases but also accurately catalog and transform it into analytics-ready tables. I'd be delighted to share some examples of previous Glue jobs or ETL code I've delivered to demonstrate the quality of work you can expect from me. To further entice you, I offer a package of extras such as effective cost management, cross-browser compatibility, free post-delivery support for three months, and assistance to find affordable hosting and domain offers. Let’s transform your data seamlessly with efficiency!
₹15.000 INR trong 7 ngày
2,0
2,0

Hello, I went through your post carefully and the need is very clear: you want reliable Python-based ETL pipelines built with AWS services, where production database data flows into analytics-ready tables without manual follow-ups. The focus is not only on running jobs, but on accurate cataloging, clean transformations, correct partitioning, and cost-aware Glue configurations that can be trusted day after day. The way I see this working is a small, well-structured set of Glue jobs written in Python that pull from your relational sources over JDBC, land raw data into Amazon S3, and progressively transform it into curated datasets. Along the way, crawlers and catalog updates keep every table and partition visible, while permissions and logging are handled cleanly through AWS IAM and Amazon CloudWatch. Everything stays parameterised so moving between dev, test, and prod feels natural, with the final code living in a tidy Git repository and a README that explains how each job fits into the bigger picture. To move forward smoothly, I’d need clarity on the source database types, expected data volumes, target schemas, and how often these jobs should run. Access to the AWS account (or a sandbox), S3 bucket structure, and any existing naming conventions would also help keep everything aligned from day one. Thanks for sharing the details. If this direction matches what you have in mind, I’m happy to continue the conversation. Best regards
₹7.000 INR trong 7 ngày
0,0
0,0

Hi there, I have reviewed your requirements and understand that you need a Python engineer to build and manage AWS Glue ETL pipelines that pull data from relational databases, catalog it correctly, and transform it into analytics-ready tables. The priority is reliable, automated jobs with clean schemas and good performance. My approach is to develop Glue jobs in Python, set up and maintain the Glue Data Catalog, and use crawlers and JDBC connections to keep tables discoverable and versioned. I will implement transformation logic to clean, enrich, and partition data for reporting, and tune worker types, job parameters, and connections to balance performance and cost. I am comfortable working with IAM for access control, S3 for staging and targets, and CloudWatch for logging and monitoring. The code will be organized in a Git repo with clear parameters for dev, test, and prod, plus a simple README. You will get jobs that run end to end without manual steps, data landing in the correct S3 locations with the expected schema, and a Data Catalog that reflects all tables and partitions produced. I have hands-on experience building Glue based ETL pipelines and focus on stable, maintainable data workflows. Cheers, Rolly
₹7.000 INR trong 7 ngày
0,0
0,0

Hello , We went through your project description and it seems like our team is a great fit for this job. We are an expert team which have many years of experience on Python, Amazon Web Services, Big Data Sales, Hadoop, Software Development, Data Warehousing, Git, Data Integration, ETL, Relational Databases Please come over chat and discuss your requirement in a detailed way. Regards
₹1.500 INR trong 7 ngày
0,0
0,0

Hello, I’m an experienced Python engineer with strong hands-on expertise in AWS Glue ETL development and relational data integration. I can design, build, and automate Glue jobs that extract data from JDBC-connected production databases, transform it using dynamic frames, and load analytics-ready, partitioned datasets into S3 with proper schema alignment. I will configure Glue connections, crawlers, and the Data Catalogue to ensure accurate table discovery and version tracking, while optimizing worker types, job parameters, and partition strategies for both performance and cost efficiency. IAM roles, CloudWatch logging, and environment-based parameterization for dev, test, and prod will be properly structured. Deliverables will include fully automated Glue pipelines, validated S3 outputs, an up-to-date Data Catalogue, and clean, well-documented code in Git. I can also support architecture discussions and whiteboarding as needed. Thank You!
₹12.000 INR trong 7 ngày
0,0
0,0

The solution can be implemented end to end with Terraform that creates IAM roles and policies, glue jobs and workflows, along with data crawlers and storage. The code can be in git repo, with the code drop to S3 through CI/CD pipeline.
₹7.000 INR trong 7 ngày
0,0
0,0

Hi, I can design and implement reliable AWS Glue ETL pipelines to move and transform your production data into analytics-ready tables. What I’ll deliver • Python-based AWS Glue jobs (JDBC → S3 pipelines) • Glue Crawlers + Data Catalog setup and versioning • Transformation logic for clean, partitioned datasets • IAM roles, logging and monitoring via CloudWatch • Performance and cost optimisation of Glue jobs • Repo with parameterised environments (dev/test/prod) and documentation I regularly work with AWS, data pipelines and automation, and can start immediately. Timeline: 4 days Bid: ₹10,500 Happy to discuss the data sources and next steps. Regards, Yeswanth
₹10.500 INR trong 4 ngày
0,0
0,0

Bengaluru, India
Thành viên từ thg 7 11, 2024
₹1500-12500 INR
₹1500-12500 INR
$250-750 USD
$30-250 CAD
₹37500-75000 INR
$10-30 USD
$30-250 USD
₹12500-37500 INR
$8-15 AUD/ giờ
€12-18 EUR/ giờ
$250-750 USD
$10-30 USD
$30-250 NZD
₹100-400 INR/ giờ
$15-25 USD/ giờ
₹12500-37500 INR
₹1500-12500 INR
₹1500-12500 INR
$250-750 USD
$10-20 USD/ giờ
$2-8 CAD/ giờ
$250-750 AUD