About the Role
To enable our high performance, Riskified has created robust and highly standardized processes for the entire model training and replacement process. This includes generating optimized training and validation data sets, model training and hyper-parameter tuning, model comparison methodologies, threshold setting and segmentation. While this process works very well, it is currently being performed mostly manually by using our internal model assembly pipeline.
The Automation team is responsible for automating the entire process, leveraging the existing platform so that we can run fully automated, periodic model retrains and replacements. The team will also be responsible for setting smart dynamic thresholds that can meet the business goals while reacting to real-time fraud waves. The team will work closely with data scientists in other teams who define the best practices around data set generation, algorithms to use, hyperparameter tuning and validation processes, implementing their recommendations at scale. The team will also collaborate closely with the operation teams, automating part of their work to meet their business objectives. You’ll work closely with the development teams to hand off your work for implementation in production. You will face issues of running the process at scale effectively, while keeping an eye out for the tradeoffs between model accuracy, time and costs. You’ll look to control for multiple hypothesis testing when retraining dozens of models automatically, implementing effective monitoring to be sure the entire process works flawlessly and iron out countless edge cases as part of a fully automated process.
As the team lead of a brand new team, you will have a unique opportunity to shape its work and focus. You’ll prioritize between automating existing processes thus reducing human effort and creating algorithms to better optimize overall performance. You’ll need a holistic view of Riskified’s challenges and have a keen understanding of the underlying goals of the department and company. You’ll lead the team, bringing measurable value to the department through your team's efforts.
What You'll Be Doing
Automate machine learning processes at scale, using best practices defined by other data science teams
Collaborate closely with research analysts in operations to create recommendations for smart segmentations and dynamic thresholds
Provide technical mentoring for team members
Explore potential improvements to optimize the tradeoffs per merchant by adjusting our thresholds automatically
At least 4 years experience as a data scientist/machine learning engineer in the industry
At least 2 years experience as a team lead of a data science/machine learning engineer team
M.Sc in exact sciences
Team player, amazing collaborator
Ability to write clean and concise code in Python and/or Scala
Experienced with data science and software development best-practices
Good communication skills, ability to clearly explain complex concepts
Creative thinker with a proven ability to tackle open problems and apply non-trivial solutions
Solid understanding of statistics and applied mathematics
Prior experience as a software developer or data engineer – advantage
To apply for this job please visit www.riskified.com.