Data Engineer Team leader

Bigabid is a data science company bent on revolutionizing the app marketing industry.
It is a 2nd-generation user acquisition and re-engagement DSP optimized for app developers.

We use a wide range of state-of-the-art ML techniques to analyze TBs of data and profile hundreds of millions of users through multiple proprietary data sources.

We process TBs of new data every day (PBs overall).
This data is used to power our Machine Learning predictions, business critical metrics and analytics used to power our decision making.

Bigabid is a fast-growing startup founded in 2015 by an experienced team of serial entrepreneurs and backed by some of the most prominent angel investors in Israel.

We are looking for a talented leader that can help us design the next generation data architecture and tools for the data powering our company.
We are looking for individuals who can bring software engineering best practices into building both business and technical critical data pipelines.
Having excellent reliability for such a large and distributed cloud infrastructure is paramount to a data-driven company like Bigabid.

Your team's goal is to create amazing ground breaking tools to make the data scientists more productive and agile.

If you love working on complicated network pipelines, you understand the importance of reliable data and have felt the pain of big data inconsistencies, and you're the type who thinks of great solutions and want to bring them to life, BigaBid is your best challenge.

Responsibilities:

  • Evangelize operational best practices and continuously look for opportunities to automate and build tools to lower operational barriers, improve clarity on problematic areas, and improve reliability.
  • Lead and grow a team of top-talent senior data engineers.
  • Transform our data architecture for massive scale and high performance.
  • Develop and automate large scale, high-performance data processing systems to drive Bigabid business growth and improve the product experience.
  • Design the architecture and schemas for the tables and be the technical lead for architecting it to be extendable, testable, maintainable and debuggable.
  • Design data models for optimal storage and retrieval, and optimize the data architecture to meet critical product and business requirements.
  • Understand logging and how it impacts the rest of our data flow, architect logging best practices where needed.
  • Develop research tools for our researchers from the Data Science and Data Analysis teams.
  • Preserve quality over the research teams’ code.

REQUIREMENTS

  • 8+ years of relevant industry experience.
  • People leader and a technical leader.
  • Passion for building and motivating teams to reach their potential.
  • You are a strong partner and can drive cross-functional projects forward.
  • Experience building internal infrastructure that is shared across teams.
  • Working with data on the TBs scale.
  • Experience designing, building and operating robust distributed systems.
  • Strong skills of relational databases and query authoring (SQL).
  • Experience with Python is preferred, deep knowledge of PySpark – a plus.
  • Experience designing and deploying high performance systems with reliable monitoring and logging practices – a plus
  • Experience with Machine Learning – a plus
פרסם משרה
X