Hello, I am a senior python developer and have rich experience with Airflow, Spark, Kubernetes, Docker and AWS/GCP.
- Schedule Spark jobs with Airflow
- Build DAG,
- Deploy spark on Kubernetes and Docker.
So I would like to help you with my experiences and skills.
Regards,
Maksym