SDS 437: Data Science at a World-Leading Hedge Fund

Claudia Perlich comes to Two Sigma from Dstillery, where she served as a chief scientist (2010 to 2017). In this role, she was responsible for the reliable estimation of targeting models (predictive modeling/ranking models using NB/logistic regression and others), the performance evaluation of systems/models and campaigns both in vitro and vivo, as well as the supervision of a real-time scoring engine that applies the models to identify the target segments of browsers. Since 2011, Claudia has also worked as an adjunct professor teaching Data Mining in the M.B.A. program at the New York University Stern School of Business. As a research staff member in the Data Analytics Research Group at the IBM Watson Research Center (2004 to 2010), she led teams that completed successfully in KDD Data Mining Competitions, designed and executed wallet/opportunity estimation models for IBM Sales using quantile regression, and worked on blog and twitter analysis tools for marketing. Claudia holds a Ph.D. in Information Systems from the New York University Stern School of Business (2004), an M.S. in Computer Science from Technical University Darmstadt, Germany (1998), an M.S. in Computer Science from the University of Colorado (1996), and a B.S. in Computer Science from Technical University Darmstadt, Germany (1995).

Claudia boasts that she’s managed to get at least 3 hours back in her day thanks to the lack of commute and has been exploring nature more than before. We agreed, however, that the R&D has taken a hit, which can be difficult for new hires who are trying to gel with a team remotely. Claudia works day-to-day at Two Sigma, one of the most successful hedge funds in the world. Ironically, she tried to avoid finance for as long as possible but made the shift from her role at an advertising company after speaking with members of the company who sold the culture to her.

More than half of the people working at Two Sigma work in data science, despite their titles. Strategic Data Science is a broadly mandated team that works at Two Sigma that functions as an R&D incubator within the company’s primary focus. They explore higher hanging fruits rather than direct pathways. Claudia’s work involves looking at hypothesis and fundamental economics to look at causes and predicting effects. This is called Alpha ideas, pre-modeling. Based on that, you come up with ways to characterize entities for the initial signals. They establish goal portfolios during the optimization stage before the last piece: execution. Something at the scale of Two Sigma means all eyes are on their behaviors and their choices have a market impact, so execution and the many ways shares can be bought and sold, is incredibly important. The variety of applicability of data science in these processes is huge. As far as day-to-day goes, Claudia does a regular standup meeting two a week and works with team members on their projects and roadblocks.

Claudia differentiates the needs of her team and the needs of Two Sigma at large when it comes to hiring. She focuses on bringing in people who can challenge the thought process and looking for diversity of thought. You, obviously, need coding skills and to be able to do data analysis, computing skills, and Claudia likes to see unusual professional backgrounds. Ultimately, it’s about finding skills that complement the team that already exists. As far as where the market is going, Claudia has noticed that many of her upcoming students are looking at the technology on a conceptional level. Real data intuition is something she has noticed has recently been lacking — the classic scientist behavior of thinking of the problem before the answer.

We shifted into Claudia’s competitive experience in data science. In 2007, Claudia got involved through IBM in competitions where she won 3 years in a row—the Netflix prize, a medical space question on breast cancer detection, and a telecommunication issue. She did not win by having the fanciest algorithm, what she found was a problem in the data set each time which she could utilize to get at her answer. After winning for three years, Claudia decided to get involved in running competitions and hackathons. How do you win competitions? You need a ton of time to get quality participation. You also need to be good at cleaning up data sets.

We ended by talking about Claudia’s decision to work in finance. The secret to her success is finding good bosses to work for. She realized she liked solving other peoples’ problems rather than focusing on her own. So having someone to work for whom she respects and relies on for judgment calls was incredibly important. Picking who you work for, rather than what you work on, is incredibly important. I can relate, as my Ph.D. topic didn’t interest me that much but my advisor did.

• Life and work during the pandemic [2:23]
• Claudia’s history with horses and riding [8:28]
• Claudia’s work at Two Sigma [12:00]
• Claudia’s role on a daily basis [20:51]
• Tools of the trade [30:27]
• What Claudia looks for when hiring [36:37]
• What skills do future hires need? [40:32]
• Claudia’s history with data science competitions [48:22]
• Why work in finance and at Two Sigma? [1:00:19]

• Two Sigma
• DataScienceGO Connect
• Ibis
• KDD
• KDnuggets
• The Hype Machine by Sinan Aral
• The Selfish Gene by Richard Dawkins

• LinkedIn

• LinkedIn
• Twitter

• Download The Transcript