Databricks Administrator

Avnet

Hybrid Actively hiring Posted this week
Tempe, Arizona Posted 3 days ago
View original post Log in to save

At a glance

AI generated

TL;DR

This senior-level position within a dynamic data engineering team focuses on developing and maintaining cloud-based data platforms using Databricks, Apache Spark, Delta Lake, and modern data engineering practices. The role involves designing and supporting Databricks workspaces, clusters, job workflows, and governance controls to ensure scalable, secure, and cost-effective solutions. Daily tasks include building and optimizing data pipelines with Apache Spark, PySpark, or Scala Spark, implementing data platform governance standards, and automating infrastructure provisioning using Terraform. The ideal candidate has hands-on experience with Databricks workspace administration, cluster management, job scheduling, Unity Catalog, and cloud platforms such as AWS, Azure, or GCP. Additionally, proficiency in SQL, version control systems like Git, and CI/CD tools is essential for contributing to the continuous improvement of data platform reliability and performance.

Skills

Databricks Apache_Spark Delta_Lake Terraform GitHub_Actions Azure_DevOps CI/CD Python Scala SQL Unity_Catalog Apache_Airflow MLflow Great_Expectations dbt AWS GCP Azure

What you'll do

  • Designs and manages Databricks workspaces and clusters for scalable data solutions.
  • Builds and optimizes data pipelines using Apache Spark, Delta Lake, and medallion architecture.
  • Implements governance standards including access controls and Unity Catalog in Databricks.
  • Automates infrastructure provisioning and platform configuration with Terraform or similar tools.
  • Collaborates on CI/CD processes for data workflows and platform deployments.

What we're looking for

  • 8+ years of experience with Databricks and Apache Spark
  • Expertise in designing and managing scalable data pipelines using Delta Lake and medallion architecture
  • Strong hands-on skills in cloud platforms (AWS, Azure, GCP) for data engineering
  • Experience with Unity Catalog, governance controls, and security policies in Databricks
  • Proficiency in automation tools like Terraform and CI/CD processes with GitHub Actions or similar
  • Knowledge of Spark performance tuning and cluster configurations for cost efficiency

Employer

About Avnet

Avnet is a global technology distributor and solutions provider connecting the world's top electronic component manufacturers with product designers and engineers through distribution and value-added services. Industry: Electronic Components Distribution

Avnet currently has 5 open roles on FindRole.

Most-posted roles

View all roles at Avnet