Data Administrator

Experience Required: 7+ YearsWork Location: Bangalore / Chennai / HyderabadRole Overview:We are seeking an experienced Databricks Administrator to manage, optimize, and maintain our Databricks platform across multiple environments. The ideal candidate will have a strong background in data engineering infrastructure, cluster management, security, governance, and automation within Databricks and related Azure/AWS ecosystems.Key Responsibilities:- Databricks Platform Administration: - Manage and maintain Databricks workspaces, clusters, jobs, pools, and libraries. - Monitor performance and optimize resource utilization across environments (dev, test, prod). - Implement autoscaling, cluster policies, and cost control measures. - Security & Access Management: - Configure and maintain user/group access via Azure AD or IAM integration. - Implement fine-grained access control using Unity Catalog and role-based permissions. - Ensure compliance with enterprise data security standards. - Automation & CI/CD: - Automate cluster provisioning, job deployment, and workspace management using Terraform, Databricks CLI, or REST APIs. - Integrate Databricks with CI/CD tools like Azure DevOps, Jenkins, or GitHub Actions. - Monitoring & Troubleshooting: - Proactively monitor cluster health, job performance, and data pipeline stability. - Diagnose and resolve system, configuration, and performance issues. - Implement alerting, logging, and audit trail monitoring. - Integration & Collaboration: - Work with Data Engineers, DevOps, and Cloud teams to ensure smooth data operations. - Support integration with Snowflake, Kafka, Azure Data Lake, or AWS S3. - Collaborate on optimizing Spark configurations and ETL performance. - Governance & Compliance: - Implement best practices for workspace and notebook governance. - Manage version control for notebooks and data pipelines. - Enforce tagging, cost allocation, and audit standards.Required Skills & Qualifications:- Bachelor’s degree in Computer Science, Engineering, or related field. - 7+ years of experience in Data Engineering or Platform Administration, with at least 3+ years in Databricks Administration. - Strong understanding of Apache Spark architecture and performance tuning. - Hands-on experience with Azure Databricks or AWS Databricks environments. - Proficiency with Terraform, Databricks CLI, REST APIs, and CI/CD pipelines. - Expertise in cluster configuration, job scheduling, and resource optimization. - Familiarity with Unity Catalog, Delta Lake, DBFS, and Lakehouse architecture. - Experience integrating Databricks with ADLS, S3, Snowflake, Synapse, or Redshift. - Strong problem-solving and scripting skills (Python, Shell, or PowerShell).

Apply Now