Senior Data Engineer
C&G Consulting Services, Inc - buffalo, NY
Apply NowJob Description
This job is a Hybrid role located in Jupiter FL. Open to Relocation. Job Description: Summary of Job FunctionsCollaborate effectively with Stakeholders, Project Managers, Software Engineers, Data Analysts, QA Analysts, DBAs, and Data EngineersBuild and maintain data pipelines based on functional and non-functional requirementsProactively seek out information and overcome obstacles to deliver projects efficientlyEnsure that data pipelines incorporate best practices related to performance, scaling, extensibility, fault tolerance, instrumentation, and maintainabilityEnsure that data pipelines are kept simple and not overly engineeredProduce and maintain design and operational documentationAnalyze complex data problems and engineer elegant solutionsStay abreast of emerging technologies and make relevant recommendationsUpgrade existing data models and pipelines leveraging newer features and techniquesWork in a Kanban environmentMentor less experienced data engineersParticipate in engineering standards and best practices evolutionParticipate in an on-call rotationLead investigations to troubleshoot pipeline issuesMinimum Requirements10+ years with hands-on data engineering and deep knowledge of data architecture fundamentals including:Extensive experience building ETLELT pipelines from a variety of data sourcesBroad experience with database and associated technologies for moving and storage of dataExperience evaluating and selecting tools and technologies based on fitness for purpose and stakeholder requirementsDeep knowledge and experience with concepts such as performance tuning, execution plan analysis, and index optimization in at least one database technology (e.g., T-SQL, PL-SQL, NoSQL, etc.)Extensive experience creating and maintaining a wide variety of database objects including, tables, views, functions, stored procedures, prehensive experience with data modeling indexes, temporal tables, CLR, etc.Deep understanding of the development of data pipelines using multiple external data sources and transport mechanismsStrong initiative, collaboration, accountability, impartiality, and communicationStrong analytical skills, a real passion for working with data and strong interest in solving data problemsStrong track record for judging core requirements and meeting deadlinesExperience managing master dataExperience writing C#, PowerShell, and PythonExperience with Git source controlExperience working in a Kanban SDLC and a strong understanding of traditional Kanban SDLC workflowsExperience with deploying changes through segregated Development, QA, UAT and Production SDLC stagesExperience owning mission-critical processesBachelor's degree in Computer Science, Information Systems, or related disciplinesAbility to work onsite in our Jupiter, FL officePreferred Skills and Previous ExperiencePython programming using libraries such as Pandas, Numpy, csv, Traceback, JSON, PyODBC, MathExperience with source code branching and pull requests code reviewsExperience with AWSExperience working with clinical andor consumer healthcare dataExperience with tools such as Red Gate, Grafana, OpsGenie and JAMSMSPhD in Computer Science, Information Systems, or related disciplines
Created: 2025-02-24