Data Engineer (Python, Pyspark, Spark SQL, Azure Databricks, Delta Tables, Azure Data Lake, Kafka) Job at Infinity Solutions, Remote

eG1wUFBtL2YrR0pVZEE3alZMZXUvZ2tjc3c9PQ==
  • Infinity Solutions
  • Remote

Job Description

About the Role and Job:

Role: Data Engineer (Python, Pyspark, Spark SQL, Azure Databricks, Delta Tables, Azure Data Lake, Kafka)

Position Title: Sr. Software Engineer

Location: Canada (Remote)

Hire Type: Fulltime

Job Description

  • 7+ years of hands-on experience in software development, with a strong emphasis on Python programming .
  • Deep expertise in PySpark for building and optimizing scalable data pipelines.
  • Proficient in designing and implementing real-time data streaming solutions using Spark SQL, DataFrames , and Datasets .
  • Solid experience with Azure Databricks , including working with Delta tables , workflows , and scripting in Python .
  • Familiar with Azure cloud services , especially Azure Data Lake , Azure Data Factory , and EventHub or Kafka for streaming data ingestion.
  • Skilled in writing complex SQL queries and building views involving joins across multiple tables.
  • Experience across the Software Development Lifecycle (SDLC) , with a strong grasp of Agile Scrum methodologies.
  • Proficient in tools like Jira (project management), Confluence (documentation), and GitHub (version control and CI/CD pipelines).

Soft Skills & Collaboration:

  • Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination .
  • Excellent problem-solving and code review skills.
  • Strong communication skills with the ability to understand business requirements and translate them into technical solutions.
  • Proactive and responsive during business hours for any ad-hoc calls or meetings .
  • Demonstrated ability to prioritize effectively , think creatively , and adapt quickly to new technologies and feedback .

Job Tags

Full time, Remote work,

Similar Jobs

Lids

Customization Illustrator Job at Lids

 ...media; ensures compliance with Lids policies and procedures Principle Duties and Responsibilities: Creates and prepares illustrated graphics for customization needs for use in stores and on web Illustrates and develops graphics for electronic media, including... 

ABARTA Coca-Cola Beverages, LLC

Order Builder (Loader) Job at ABARTA Coca-Cola Beverages, LLC

 ...Order Builder (Loader) Company: ABARTA Coca-Cola Beverages Job Location: 4900 Harrisburg, PA Other Potential Locations: Harrisburg, PA We are actively seeking enthusiastic team players who want to work with the world's most trusted and iconic brands. About... 

Sharp HealthCare

Care Coordinator - SCMG Utilization Management - Spectrum - Day - Full Time Job at Sharp HealthCare

**Facility:** Corporate Offices**City** San Diego**Department****Job Status**Regular**Shift**Day**FTE**1**Shift Start Time****Shift End Time**Other; Other**Hours** **:****Shift Start Time:**8 AM**Shift End Time:**5:30 PM**AWS Hours Requirement:**8/40 -...

Levi & Korsinsky, LLP

Derivative Litigation Associate Job at Levi & Korsinsky, LLP

 ...Overview We are seeking an associate attorney with 3 to 5 years of post-law school...  ...success in this role. This position is remote with the option to work from our New...  ...and trial preparation. Draft and review litigation documents, including complaints, discovery requests... 

Aaditus Technologies LLC

Data Engineer Job at Aaditus Technologies LLC

 ...looking for basically a data engineer Must be strong in the ETL/Database space Pentaho is preferred, but open to Spark/Databricks/etc Should be good with Python scripting (pandas, numpys, etc) Must be strong in SQL Data modeling experience is a...