Senior Lead Databricks Engineer

Job OverviewLocation: New Jersey (OnsiteRemote)Duration: 24+ MonthsExperience: 12+ Years Total, 7-8 Years in DatabricksCompensation: $75-80hourCompany IntroductionOur cutting-edge technology organization is seeking a highly skilled Senior Lead Databricks Engineer to drive our data infrastructure and analytics capabilities. We are looking for an exceptional talent who can architect robust, scalable data solutions that transform raw data into strategic insights.Job ResponsibilitiesDesign and implement complex data pipelines using Databricks platformDevelop high-performance Spark and PySpark processes for advanced data transformationIntegrate seamlessly with Azure or AWS cloud environmentsManage version-controlled repositories using GitGitHubMentor junior team members and drive technical excellenceCollaborate cross-functionally to deliver innovative data solutionsEnsure optimal performance, cost-efficiency, and reliability of data workflowsTroubleshoot and resolve complex data pipeline challengesRequired Technical Skills7-8 years of hands-on Databricks platform experienceAdvanced Apache Spark and PySpark expertiseProficient in Azure or AWS Cloud technologiesExpert-level Git and GitHub version controlStrong distributed computing and big data processing knowledgeExperience designing large-scale data architecturesRequired QualificationsBachelor's degree in Computer Science, Data Engineering, or related fieldMinimum 12+ years total professional experienceMinimum 7-8 years specialized Databricks experienceProven track record of designing enterprise-scale data solutionsStrong analytical and problem-solving capabilitiesExcellent communication and collaboration skillsPreferred QualificationsDatabricks CertificationAzure Data Engineer CertificationAWS Cloud CertificationsDelta Lake implementation experienceCICD and DevOps expertiseData security and compliance backgroundVisa RequirementsAll visa types accepted (Except CPTOPT)

Created: 2025-03-08