Lead Databricks Data Security Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead Databricks Data Security Engineer based in the United States. This role will lead the security architecture and implementation of access controls across a modern Databricks Lakehouse environment. You will serve as the senior technical owner for data access governance, translating security and compliance requirements into scalable controls. The position combines hands-on software engineering with architecture, automation, governance, and technical leadership. You will work closely with data engineering, platform engineering, identity teams, and enterprise stakeholders to establish secure data practices. Your work will shape how sensitive data is classified, accessed, monitored, and protected across the platform. The environment emphasizes infrastructure-as-code, automated security controls, and modern identity and federation patterns. This is an opportunity to build and mature a critical data security function while solving complex challenges at enterprise scale. Lead the architecture and implementation of role-based access control (RBAC) and attribute-based access control (ABAC) within Databricks Unity Catalog. Define scalable data access models, including data structures, permissions, tagging standards, dynamic views, row-level security, and column-level masking. Establish and evolve identity governance for the Databricks environment, including service principals, workload identities, SCIM, and integration with enterprise identity providers. Translate regulatory, contractual, security, and compliance requirements into practical technical controls and enforceable platform policies. Develop and maintain data classification frameworks and tagging structures that enable consistent and scalable access governance. Implement Databricks grants, catalogs, tags, access controls, and policies as code using Terraform and/or Databricks Asset Bundles. Build automation and controls that support version management, audit readiness, repeatability, and secure deployment practices. Partner with workspace administrators and platform engineers to align workspace, compute, secrets, networking, and identity controls with the overall data security architecture. Create reusable guardrails, templates, and workflows that enable data engineering teams to manage access efficiently while maintaining strong security standards. Provide technical leadership and guidance as the data security function evolves, balancing enterprise security requirements with usability and engineering efficiency. Use Python and SQL to automate security processes, monitoring, compliance activities, and operational workflows. Requirements: Bachelor’s degree in a relevant discipline with at least 8 years of experience as a Software Engineer or Data Engineer, with significant focus on data platform security and security automation; alternatively, a master’s degree with at least 6 years of relevant experience. At least 2 years of hands-on experience implementing Databricks Unity Catalog security capabilities. Demonstrated expertise designing and implementing RBAC and ABAC models at scale. Strong practical experience with Unity Catalog grants, dynamic views, row-level filtering, column-level filtering, and data masking. Fluency with identity and federation concepts, including SCIM, service principals, workload identities, and integration with enterprise identity providers such as Microsoft Entra ID. Experience applying infrastructure-as-code practices to Databricks security, access, tagging, and policy management, preferably using Terraform. Strong proficiency in Python and SQL for automation, monitoring, security operations, and compliance. Ability to translate complex security, regulatory, and contractual requirements into scalable technical solutions. Strong understanding of data classification, access governance, identity management, and security a