let’s make something together

Give us a call or drop by anytime, we endeavour to answer all enquiries within 24 hours on business days.

Find us

PO Box 16122 Collins Street West
Victoria 8007 Australia

Email us

info@domain.com
example@domain.com

Phone support

Phone: + (066) 0760 0260
+ (057) 0760 0560

Data Engineer

  • By Weronika Drwila
  • 12 December 2023
  • 593 Views

Digital & Technology Team (D&T) is an integral division of our client company. That includes digitalizing and integrating our processes, ensuring best-in-class technology, and embedding a data-driven culture. By joining us you will work in one of the most dynamic and innovative teams and have a direct impact on building the future!
.

Your responsibilities would include:

1. Data Extraction, Transformation, and Storage:

  • structuring end-to-end processes of data extraction, transformation, and storage using serverless Azure services to deploy models and analytical solutions
  • handling activities such as quality assurance, data migration and integration, and solution deployment to ensure the business gets the best value
  • writing and maintaining ETL processes in Python, designing database systems, and developing tools for real-time and offline analytic processing.

.

2. Development and Maintenance:

  • implementing, maintaining, and further developing the functionality of Python packages for ETL processes, data lineage, operator inputs, including building logic
  • troubleshooting software and processes for data consistency and integrity
  • integrating large-scale data from a variety of sources for business partners to generate insights and make informed decisions.

.

3. Data Flow and Testing:

  • designing and implementing data flows to ensure efficient and reliable data movement
  • creating unit and integration tests for Python modules
  • participating in mission-critical processes of the data pipeline.

.

4. Technical Skills and Collaborative Environment:

  • providing technical support in understanding business problems and designing smart data products
  • collaborating within an agile culture, contributing to the team’s success
  • demonstrating excellent software engineering skills, including unit testing, integration testing, OOP, and the ability to write clean, efficient, documented, and scalable code.

.
You are a good candidate if you have:

• significant commercial experience in a similar position
• strong data analytics skills using Python
• excellent software engineering skills (including unit testing, integration testing, OOP)
• proficiency with Python and/or Scala (ideally both)
• experience working with large datasets through Spark and RDBMS
• solid knowledge of PySpark with the ability to apply it to write Spark applications as well as analyze data in a distributed environment
• very good SQL skills and ability to extract information from databases
• fluency in English for effective communication.

.
You are a perfect match if you also have:

• good working knowledge of Azure services
• experience in building and releasing Infrastructure as Code using tools like Terraform
• familiarity with versioning systems (e.g., Git), DevOps mode of working, and DevOps tools
• understanding of machine learning models, deployment, and monitoring
• strong interest in machine learning and a desire to develop skills in this field
• experience working in agile organizations
• being a team player with a proactive and collaborative approach.

.

Note: Prepare your CV in English (PDF), fill in the form, and apply! 🙂

.

Dotcommunity jest zarejestrowana w Rejestrze agencji zatrudnienia (KRAZ) pod numerem 9904.

    * - required