Data Engineer

Tittle : Python Data engineer Location : Houston,TX ( In-Person interview)Contract W2No C2CAll VisaKey Responsibilities:Python API Development:o Develop and maintain RESTful APIs using Python/C# web frameworks such as FastAPI or Django.o Collaborate with front-end developers to integrate user-facing elements with server-side logic.Parallel Data Processing:o Utilize Pandas, NumPy, and other libraries to process large datasets efficiently.o Implement multithreading, multiprocessing, and asynchronous programming techniques.o Optimize data processing pipelines to handle millions of rows with minimal latency.Distributed Systems Design:o Design and implement distributed systems with a focus on scalability and reliability.o Understand and apply core concepts such as load balancing and task queues.o Use Docker to containerize applications and manage dependencies.o (Preferred) Experience with Kubernetes for container orchestration.Technical Communication:o Clearly articulate complex technical concepts to team members and stakeholders.o Document system designs, processes, and code effectively.o Collaborate with cross-functional teams to align on project goals and deliverablesMust-Have Qualifications:Experience in Python Web Frameworks:o Proficiency with FastAPI, Django, or similar frameworks.O C# codingo Understanding of RESTful API principles and best practices.Docker Knowledge:o Ability to create and manage Docker Files.o Experience with containerization for deployment and development workflows.Systems Design Understanding:o Basic knowledge of load balancing, task queues, and distributed system concepts.o Ability to design systems that are scalable and maintainable.Concurrent and Parallel Computing Skills:o Proficiency in multithreading and multiprocessing without relying solely on external libraries or frameworks.o Familiarity with asynchronous programming, particularly asyncIO in Python.Communication Skills:o Excellent technical communication abilities.o Experience collaborating in team environments and conveying complex ideas clearly.Preferred Qualifications:Education:o BS or MS in Computer ScienceAdvanced Data Processing Tools:o Experience with Polars, PySpark, or similar tools.o Handling of large-scale data processing tasks efficiently.Distributed Computing Experience:o Hands-on experience with distributed architectures in Docker.o Familiarity with concepts like task queuing, MapReduce, and saga patterns.Kubernetes Experience:o Knowledge of container orchestration using Kubernetes.o Experience deploying and managing applications in a Kubernetes cluster.Problem-Solving at Scale:o Demonstrated ability to solve complex problems using parallel or distributed computing.o Innovative thinking beyond single-threaded processes.Santhan@4804076925

Created: 2025-02-02