The practice of designing and building systems for collecting, storing, and analyzing data at scale. It is a broad field with applications in just about every industry. Organizations have the ability to collect massive amounts of data, and they need the right people and technology to ensure it is in a highly usable state by the time it reaches data scientists and analysts.

rn

Often deal with the unknown by using more advanced data techniques to make predictions about the future. They might automate their own machine learning algorithms or design predictive modeling processes that can handle both structured and unstructured data. This role is generally considered a more advanced version of a data analyst. Some day-to-day tasks might include: Gathering, cleaning, and processing raw data; Designing predictive models and machine learning algorithms to mine big data sets; Developing tools and processes to monitor and analyze data accuracy; Building data visualization tools, dashboards, and reports; Writing programs to automate data collection and processing

Từ 1,197 nhận xét, các khách hàng đã đánh giá Data Engineers 4.85 trên 5 sao.
Thuê Data Engineers

Bộ lọc

Tìm kiếm gần đây của tôi
Lọc theo:
Ngân sách
đến
đến
đến
Loại
Nhiều kỹ năng
Ngôn ngữ
    Tình trạng công việc
    5 công việc được tìm thấy, giá theo USD
    VIdeo contents on Data Engineering 5 ngày left
    ĐÃ XÁC THỰC

    Hi, I am one of the core members of an EduTech startup and we are building course contents on various tech stacks including Data Analytics, Data Science & Data Engineering. On DS & DA, we already have enough creators, we need some senior Data Engineers to create end to end video courses for us.

    $1361 (Avg Bid)
    $1361 Giá đặt trung bình
    5 lượt đặt giá

    Need anyone who has hands-on Experience in spark and python. Minimum of 2 years experience. Should be able to code in pyspark.

    $318 (Avg Bid)
    $318 Giá đặt trung bình
    9 lượt đặt giá
    CANNOT SCALE BIG DATA PROCESSING 4 ngày left
    ĐÃ XÁC THỰC

    I built an ETL pipeline to process terabytes of data. To achieve that goal, I setup a Spark Cluster (Scala) and MinIO server for object data storage. I can process and save 200 gigabytes in roughly 30 minutes using 10 virtual machines, for Spark Processing. The issue I have is that I am not able to scale that Processing. Meaning if I double the number of spark virtual machines, this does not affect processing time. I need a Data Architect who has enough expertise to help me identify the bottleneck and fix the issue. ARCHITECTURE SUMMARY. • I use virtual machines set up on-premises using VMWare ESXi 6 • Physical machines (which host VMs) are on a 1 GB network. • There is no over commitment for vCPU nor RAM • Spark VMs. 16VCPU, 64 GB RAM • MinIO (Storage). 16vCPU, ...

    $213 (Avg Bid)
    $213 Giá đặt trung bình
    2 lượt đặt giá

    Hi there I need somone who can help me with list of things to be considered before moving to databricks with code base already running locally. I need help with list of things to be considered before moving to databricks. I guess person with data engineering background can help me Thanks

    $80 (Avg Bid)
    $80 Giá đặt trung bình
    16 lượt đặt giá
    Migration to python code base databricks 2 ngày left
    ĐÃ XÁC THỰC

    Hi there I need somone who can help me with list of things to be considered before moving to databricks with code base already running locally. I need help with list of things to be considered before moving to databricks. I guess person with data engineering background can help me Thanks

    $80 (Avg Bid)
    $80 Giá đặt trung bình
    17 lượt đặt giá

    Các bài viết khuyên đọc dành cho bạn

    Communication is the key to a successful project that generates great reviews.
    3 MIN READ
    Learn how to thrive and succeed as a digital nomad.
    3 MIN READ
    Freelancing while studying: make extra money & build your experience flexibly.
    4 MIN READ