Architect (Unity Catalog & Hive Metastore)

Job Profile- Architect-1/ Data Engineer/ Migration Specialist Contract Duration- 6-7 months (ext) Mode of work- Remote Working Hours- UK ShiftJD- Qualifications & Skills - 9-10 years of hands-on experience indata platform engineering or migration projects - Strong proficiency inDatabricks, Unity Catalog, Hive Metastore, and UCX assessment tools - Cloud platform expertise ( Azure ) includingIAMand networking setup - Familiarity withBI/reportingtools integration and security policy governance - Excellent documentation and cross-functional collaboration skillsGood to have - Experience with cluster policy design and UC-enabled environments - Knowledge of ACL mapping strategies and data governance best practices - Ability to create migration dashboards or reports for leadership visibilityR&R- HMS Assessment & Migration Planning - Run UCX Assessment Tool to evaluate Hive Metastore (HMS) state, migration complexity, and readiness - Identify external and managed tables, BI dependencies, and environmental separation needs - Prepare and deliver a detailed migration plan and finalize a migration runbookPlatform Setup - Configure necessary cloud resources: storage accounts, IAM roles, and networking - Set up Unity Catalog Metastore(s) and assign workspace mappings - Establish and validate admin roles including Account, Metastore, and Workspace AdminsUnity Catalog Design & Cluster Enablement - Inventory and categorize data assets including catalogs, schemas, tables, views, and files - Design catalog/schema hierarchy aligned with business units or domains - Develop and test ACL/permission migration strategy and validate cluster policies for - UC enablementMigration Execution - Use UCX tools and upgrade notebooks to migrate tables and views from HMS to Unity Catalog - Migrate and validate ACLs and permissions using audit scripts - Update BI tool configurations and modify application connection strings as required - Transition and test job cluster policies for post-migration functionality.Post-Migration Validation & Handover - Verify data integrity, job executions, BI report accuracy, and access control mechanisms - Document rollback procedures and highlight unresolved findings - Conduct handover workshops and knowledge transfer sessions with operational stakeholders

Apply Now