Senior Data Engineer Engineering · Charlotte ·

Company Overview:Is your CV ready If so, and you are confident this is the role for you, make sure to apply asap.Aidentified is an innovative company driven by a mission to transform how businesses discover and connect with impactful relationships. We leverage cutting-edge AI-powered technology to empower professionals, boosting efficiency and growth through the power of data and networking.Job Description:We are seeking an experienced Data Engineer to design, build, and maintain scalable and efficient data pipelines that support our business needs. In this role, you will work closely with data scientists, analysts, and other teams to ensure the integrity and reliability of data across the organization. Your contributions will be critical to shaping data-driven decisions and ensuring our data infrastructure is optimized for performance, scalability, and security.Key Responsibilities:Design, develop, and optimize scalable ETL pipelines using PySpark and SQL.Manage and process large datasets to ensure clean, accurate, and timely data delivery.Collaborate with data scientists, analysts, and stakeholders to gather requirements and deliver reliable data solutions.Implement and uphold best practices for data management, including data quality, governance, and security protocols.Integrate data from multiple sources and systems, ensuring seamless data flow.Monitor, troubleshoot, and enhance the performance and reliability of data pipelines.Optimize database queries and data models for performance, scalability, and maintainability.Develop and maintain Python and shell scripts for automating data processing and transformation tasks.Participate in code reviews, design discussions, and contribute to the continuous improvement of our data architecture.Required Skills and Qualifications:Bachelor's degree in Computer Science, Data Engineering, or a related field. Master's degree is a plus.5+ years of experience in data engineering or a similar role.Proficiency with database technologies such as SQL, NoSQL, and data warehousing solutions.Strong programming skills in Python, Java, or a comparable language.Experience with big data processing frameworks like Apache Spark or Hadoop.Familiarity with cloud platforms such as AWS, Google Cloud Platform, or Microsoft Azure.Strong problem-solving skills and the ability to work in a fast-paced, agile environment.Preferred Qualifications:Experience with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).Familiarity with big data technologies (e.g., Hadoop, Elasticsearch).Experience with data science tools like Jupyter notebooks.Experience with data pipeline orchestration tools (Airflow, Dagster, Luigi).Benefits and Perks:Competitive salary and potential equity opportunities.Comprehensive benefits package including health, dental, and vision insurance.401(k) plan.Flexible work environment, including hybrid work options.Join Us:If you are a forward-thinking Data Engineer with a passion for optimizing data processes, we would love to hear from you! Together, we can redefine how businesses harness the power of data for competitive advantage.We provide equal employment opportunities for all current employees and applicants for employment. This policy means that no one will be discriminated against based on race, religion, creed, color, national origin, nationality, citizenship, ancestry, sex, gender identity, age, marital status, physical or mental disability, affectional or sexual orientation, military or veteran status, genetic predisposing characteristics, or any other basis prohibited by law.#J-18808-Ljbffr

Created: 2024-11-06