Rachael Maltiel Swenson of eBay joined us as a guest to share advice for analytics-minded careerists.

Topics include:

  • how Rachael found her way to eBay via Bain Consulting, grad school, Expedia and Pinterest
  • what to keep in mind when applying for marketing analytics roles
  • why R and SQL are not the most valuable skills in an analytics career

For the full text transcript see below the fold:

Audio:

Play in new window || Download

Video:

Max: Welcome all! Max Mautner of the Accidental Engineer here.

Today we’re joined by Rachael Maltiel Swenson. Rachael is Head of Analytics of the Flex team at eBay, which is a cross-channel marketing analytics team.

Do you mind sharing for our audience a little bit about your current role there and what your team does?

Rachael: Yeah!

We setup digital marketing analytics in a pretty unusual way in that we have different teams that directly support the marketing channels for paid search, email, all that.

My team is the team that takes on broader projects–cross-channel projects–projects that tend to take a little bit more time. They might be technically challenging or something like that.

We’re the place where normally with analytics there’s these ideas like, “It would be really nice to look into this. Or we have to put out this burning fire over here so we don’t have time to look into that.”

So we’re the team that can look into those longer term ideas and are removed from the day to day fires. We end up with a lot of strategic innovative projects that tend to take a little bit longer.

Max: A lot of our audience is curious about how to get into careers in marketing analytics and the technical side of that, what skills they need to know.

We can go all the way back to maybe college: what led your path to being head of analytics of the Flex team?

Rachael: I can go all the way back. In college I majored in Math and Philosophy with no idea what I wanted to do, but those things were interesting so I studied them. When it was time to graduate I still had no idea what I wanted to do, and so I ended up taking a job at Bain & Company, strategy consulting. Which is a really good place to go when you have no idea what you wanna do!

You end up working on a lot of different cases or projects. Usually across different industries, really different types of business questions. But they’re all strategic business question and you solve them in really different ways. Sometimes it’s mathematical models, sometimes it’s interviewing people, sometimes it’s weeding through lots of old journals.

You see lots of different techniques, you see different industries, you see different types of questions, and so for me it was really good because I learned what I liked and what I didn’t like.

I knew that I liked marketing. I liked where business kind of touched people, and decision theory and people’s–what they liked and what they didn’t like. I didn’t wanna stay there, I did two years there and while I really liked the type of work, the culture, the hours, all that was a little challenging.

I thought that the direction that I wanted to go was answering these business questions. I recognized there was a lot of data and wanted to be able to analyze the data better. And so I went and got my Masters in Statistics. Actually, I went to get my PhD in Statistics and then I decided I didn’t want to do that.

Max: That’s a really interesting story I’ve heard many variations of: starting the PhD program which can be 4-7+ years.

Rachael: Five years would have been typical for statistics.

Max: A lot of people I know do not find the interest or will or desire to fulfill the four plus years’ experience. What was it that led you to reconsider completing the PhD program?

Rachael: I would sit in my classes and we would be learning these methodologies and techniques I’d be like, “Oh when I get a job someday I really hope I don’t have to do this at my job.”

I realized I was saying that about almost everything I was learning, and I was like, “I don’t know if finishing this is going to get me to exactly where I wanna be.” I liked the types of problems that statistics can be used to solve, but statistics grad school is about getting into the really nuanced rules and assumptions: “Are you sure you made the right assumptions and are your standard errors perfectly proper?”

And from being at Bain I recognized that a lot of companies don’t even know what’s your average or count—really, really basic things. And so I didn’t need to know how to do a hierarchical linear means model and make sure that I had the right assumptions for my standard errors to have an impact.

A linear regression was probably gonna have a lot of impact a lot of the time–a mean versus a median was gonna be able to have impact.

I liked the really applied problems and PhD-level academic statistics was getting very, very far from them. Super, super important for like medical studies. Medical studies need that really really nuanced perfection in standard errors because if you’re thinking like, “How is this like drug saving lives, whether or not the side affect is statistically significant?” Very, very important. But a lot of the day-to-day people decision type of stuff…you can make a lot of impact there with really simple stuff.

Max: One of the skills you were telling me about obtaining while you were doing the PhD program was R. And how R is a very popular tool in the statistical academic community as well as in the industry. Do you mind sharing for our audience what kind of struggles you dealt with in learning R? Did you have programming experience before you entered your Master’s program?

Rachael: Minimal, minimaI–I went to a liberal arts college so I studied a little bit of everything. I did take one or two computer science classes in undergrad. I understood the basic theories of computer science but hadn’t used R or anything like that at all. In grad school you learn R because if you want to complete your homework you have to use R.

In classes you would get basic examples of like, “And this is the line of code you need to write like a linear regression.” But then again of course you had to figure out how to do a lot more with it. I think some of the hard stuff with R is making your plots look fine. Like when you’re trying to move your legend up because it’s covering your “thing”…a lot of that stuff ended being really painful. But you start to just do one homework and you save it. And so then next week, when you have to do your next homework, you’re like, “Oh I remember I figured out how to move my legend” and you copy paste the code and bring it over.

In grad school we also had a statistical programming class. It was actually more focused on C++, although there was a little bit on R. They taught you some of the theories of creating functions, rather than just like writing your code to do something once. Although I have to admit I was taught a lot of stuff that I don’t necessarily always use in my day to day life, but I have some formal training.

Max: When your motivations changed around the PhD program and it dawned on you that the work you were doing on the PhD wasn’t what you wanted to be doing for the rest of your life or career: what series of steps did you take to find your way into a job in the private industry?

Rachael: I knew I was going to be graduating in June, I needed to find a job. My husband was still in grad school so we knew we were going to stay in Seattle.

I just went online and read all sorts of jobs postings. I wasn’t going with a very specific path, right? I wasn’t like, “I want a job in marketing analytics.” I was like, “I want a job.”

I knew I wanted something that required something quantitative–I had math from undergrad, I had my Master’s in Stats now. I wasn’t walking all the way away from that. But I also knew I wanted something where you solve business problems. Those two conditions if you’re just looking at job descriptions almost any job description will tell you some mathematical ability is helpful, and we’ll solve important impactful business problems.

Getting that first job when you’re coming out of school is hard. You don’t have the connections, you don’t have all this great work experience. I had some at least–I had Bain which was nice, versus just coming out of undergrad. So I just put my resume out into the black box of the internet.

I think I probably applied to over 16 different jobs. I probably only heard back from maybe three, and I got two job offers.

Max: That’s a pretty good conversion rate!

Rachael: It’s a pretty good conversion rate for the online black box of resumes :)

Max: Once you’d found your job at Expedia—how long were you there for again?

Rachael: I was at Expedia for three years. Yeah, variety of different roles over the years. I started in a cross-channel analytics role. It’s kind of similar to the role I have now, but it was just me versus a team. Cross-channel versus lots of strategic types of projects.

Max: I think a lot of our audience may not be familiar with cross channel analytics. For people who are not in marketing at all, do you mind breaking down what “cross channel” means?

Rachael: In digital marketing you think about the different channels which is just the different ways you can acquire a customer. Paid search (all the ads you click on Google), social (Facebook, Pinterest, Snapchat). Email marketing is a big one. Display which is just banner ads you see around the web.

And different companies will have different flavors of what works in their industry. Then “cross channel” is when you’re thinking about marketing across all of those things.

Oftentimes marketing is thought of in those channels distinctly. But you can imagine that it can be very impactful if you click on this Google ad about “cool basketball shoes” or something like that and you don’t actually transact. And then if we show you an email about “cool basketball shoes” (because we recognize where you are on a journey, and we try to keep talking to you). I will note that not very many companies are actually doing that type of streamlined stuff as well as they’d like to be.

But “cross channel” tends to be things that can have impact in lots of channels. If you’re thinking about the messaging. measurement, or creatives of your marketing then you can solve those things in ways that can work for lots of channels.

That tends to be the types of things that I’ve worked on as opposed to focusing on a specific channel.

Max: So the signals that buyers of Expedia might give off about where they are at on their journey of wanting to buy a travel package or airplane tickets are probably significantly different than the buying signals of an eBay customer.

How drastic of a change was that in role…I realize in between Expedia and now eBay, you worked at Pinterest. But what’s kind of the diversity of buying signals that people give off?

Rachael: So I’ve only been at eBay four months. So, I’m definitely still wrapping my head around it because three years of intuition that I had built up at Expedia rears its head sometimes at eBay. But one really basic thing is usually when you’re booking travel you don’t wake up one day and you’re like, “I think I might go to Hawaii” and then an hour later you book your trip to Hawaii, right?

Usually it’s more like, “Hmmm, we haven’t taken a vacation in a while. Maybe we should go someplace.” And you go home and talk to your significant other. Then you debate back and forth where you’re going. Then three months later maybe, you’re booking a trip, right? It’s a very long purchase cycle.

eBay sells all sorts of things. Clothes for example. It’s a little bit more realistic to think that you’ll wake up one day and you’re like, “I think I’d like some new shoes.” And then you buy them. The end you, and you move on with your life :)

From a digital marketing perspective this matters—one thing that is very common and very important in digital marketing is attribution. And its completely analytics-driven, it’s the math of who gets credit for what transaction, right? What tends to happen is you click a link in paid search, and then you click an email you get, and maybe you get an ad on Facebook, and you click all those things and then you purchase.

All three of those teams wants to say that they drove that purchase. But attribution is about figuring out who gets the credit.

There are a lot of people spending a lot of time thinking about this. At Expedia, our attribution covered a 30-day window because it is very realistic that you are shopping for travel over 30 days. At eBay, our logic is one day.

Max: Wow.

Rachael: It’s things that are being explored and thought about. I wouldn’t say that at eBay we think that we’re done and we can just move on with it. That’s actually something we’re thinking about right now is whether we’re correct or not.

But I believe that eBay and Expedia are very different for that time-to-purchase. But also, it’s one-size-fits-all, right? It might take me 30 days to book a trip, but it might take you a week, it might take somebody else three months, right? You have to come up with rules and logic and hope it works.

Max: This sounds like a really intractable problem, this attribution problem.

Is there any 20-year-off solution that will solve the problem across the industry?

Rachael: Some people will tell you that they will solve the problem across the industry now. There are third party solutions that you can bring in for attribution that use various mathematical models and they differ in how they do it.

There’s some that have much more models with lots of variables that capture all the different things with the premise that I can show you an ad in email and I can show you an ad on Facebook and if they use the same creative maybe I should think of those similarly. Maybe that information should be grouped together.

Companies do more like model things. Other companies do more like paths—like if you click display, and then email, and then I just click display and then purchase and you purchase. We can compare all the people like you or compare all people like me, and look at the difference in conversion rates and you can say that that email click…if your group is higher conversion then email has this lift in conversion. There are third party solutions that do that right now, but…

Max: You sound deeply skeptical about those offerings.

Rachael: No, I actually think there’s a lot of really good advanced models. It’s a question right now of build or buy. It’s a very common question in technology. What do you do in-house versus what do you out-sourced? Attribution is a hard one to outsource, because you’re giving away on your data, this other company is telling you who’s getting credit for what and you might not fully understand the logic. You probably don’t know all the underworking’s of the mathematical model. You probably don’t even get back fully granular data of click A drove transaction B.

I actually think a lot of those third-party solutions are probably pretty good, but it can be a hard thing to fully remove from the company.

Max: Because of organizational buy-in and bureaucratic concerns and legitimate ones, too?

Rachael: Yeah, like understanding what happened, right? Because okay you work with a third party, and 6 months later all of your paid search numbers go down.

What happened? How do we fix it? Well that’s really hard to figure out if you don’t understand how paid search is getting credit for the numbers that it has.

Max: Or you find out that they were calculating something wrong and you don’t have accountability in place. There’s not…you can have all of the SLAs in the world but there’s not the same effect as firing somebody I guess?

Build versus buy is a super interesting problem. Is that something that you’ve dealt with throughout your career post-grad school?

Rachael: Not a ton. I’m dealing with it a little bit right now, but not a ton. Companies do differ in what they build versus buy. Honestly, I think a lot of the analytics build versus buy questions, they’re honestly a little upper funnel of us. Because there’s a lot of build-versus-buy decisions in terms of how you track the data, how you log the data, how do you store the data. Which query tool do you use? Some of that stuff which affects us but the data engineering team is the one who makes a lot of those decisions.

There are options where you can bring in third party solutions to do various analytical things. Dashboard tools of course, attribution is one, there are third-party tools that do A/B testing. There are third party tools that do cohort models. I’ve always worked in big companies that have had big analytics teams, and I think the build-versus-buy question is very different when you’re a small company and you have two analysts.

Max: Oh, for sure.

Rachel: You’re trying to keep everything going. Then you probably should be buying a lot. eBay has around 60 to 70 digital marketing analysts.

Max: Wow.

Rachel: Which we are one team of analysts. I think there are probably 10 to 20 teams of our size across eBay. I think that puts us in a different position for build versus buy.

Max: Feel free to punt on this question, because I realize this might be non-public information but what tools do you guys buy?

Rachael: I don’t know everything, but common things for day-to-day analyst kind of things we use Hadoop and Teradata. I’ve used Hadoop and Teradata at previous jobs as very common query tools.

Dashboard stuff, there’s stuff all over the place. Tableau is certainly used with some frequency at eBay. But there’s less of a consistency this is our one dashboard tool versus what I’ve actually seen at other companies. Pinterest, we had our own internal dashboard tool, and at Expedia we used QlikView which is not quite as common as Tableau but I actually really liked it.

Those are some of the most common analytical tools. I think we’re only now starting to think about the next level. The cohort tools or the attribution tools, things like that. And every place I’ve worked has had a lot of internal stuff for A/B testing versus completely out sourcing.

Max: One of the things I’d love to hear you tell our audience is to reassure people who might be apprehensive about learning the tool kit that you’ve learned.

You learned R in Grad school. You also mentioned before we hit “record” about learning SQL, how relatively easy it is to learn. I think it would be great to hear about your experience learning SQL. I realize you use it a lot less now that you’re in a senior, managerial role. But for people who might be early in their careers, what was learning SQL like?

Rachael: Yeah. They don’t teach SQL in grad school. But you need SQL for any job where you’re going to be touching data. Before my interviews I did the really basic “SELECT * FROM __”. I’m like, “okay, I know what an “inner join” is versus an “outer join”—I’ve never done one but I know what it is from reading about it from Wikipedia”–that type of stuff. Then when I started my job I had to actually use it.

My manager told me that a really good analyst is not that they are the best at writing code, but it’s that they know how to get code which doesn’t necessarily mean writing it themselves.

He taught me to always ask around, “Does anybody else have an example query that I can work off of?” And so I learned a lot of SQL from other people sharing code to access the data tables that I was trying to access and I could figure how to modify it until I learned from seeing other people’s examples.

And I’ve always worked at companies where people are really happy to help each other. It wasn’t that long ago that they didn’t know SQL, or they didn’t know this specific left join trick or whatever. People were very happy to share and teach, but I learned a lot just by seeing other people’s queries. In my last two jobs, I started in those companies as a manager so I didn’t have as much time to learn the data. So, how did I learn the data? I asked people to share their queries with me. Then I could see what were the tables they were using, how did they access it, what were the fields called, where did they get their fields, and so I learned a lot just from seeing whatever everyone else did!

As a manager if I had somebody new joining my team that’s exactly what I would tell them to do. Often times I might try to give someone a first project where they are modifying something that somebody else did—that’s a great way to start learning.

Max: It sounds like analytics roles there’s an expectation that you do know SQL. But in your first analytics role out of grad school you didn’t know it as well as maybe some of your peers did. This is something where if you rely on your co workers and if you’re kind to your coworkers you can always get help!

Rachael: Basically, any analytics interview will ask a SQL question. I can’t actually remember if I had a SQL question in my interview at Expedia. But I’m sure if I did I was not amazing. I actually had spoken with my manager about it later because I think now it might be harder to get a job with the level of SQL that I had when I got that job. I think the industry has gotten a little more competitive. But his take on it, and the take when I’ve had people since then was, “Oh you know R or you know some other programing thing, SQL is really easy you can pick it up.”

I would say that, myself as a hiring manager, I’m looking for you know how to do something programming-wise. If it’s SQL, great! If it’s not and you have other strengths that may be okay.

Max: I think it’s a really amazing story that you put your resume out into the wilderness of the internet and ended up with a couple of offers.

For people who you’d advise as a hiring manager on the other side of the table: what do you recommend to analytics job seekers today?

Rachael: The first thing I’ll say is: when you don’t get a job it probably has almost nothing to do with you.

Which isn’t super helpful for getting a job but it’s at least helpful to take those rejections in stride. It is really hard to write a good job description. The way that we do job descriptions: I’ll just put eight bullet points and then here’s a paragraph, and, “Obviously now you understand everything that you’ll be doing for the next two years of your life.”

No, it doesn’t cover it at all. And every time I’ve hired I’ve had in my mind, “What am I kind of looking for to shape up my team?” That’s how I approach hiring, right? I’m trying to build a team and so I might be like, “I have three people on my team who are really strong technically—they have R, Python, their SQL’s really good but I think I could really use someone who can really help to tell the business story.”

I might want somebody who’s gonna be really good at creating PowerPoint slides, and telling the story, and honing in on the right questions to really solve what the business is looking for.

So I may get a whole bunch of resumes that are super strong technically, but they’re just not what I’m looking for right now. And people are not very good at indicating that nuance in a job posting. I can’t write a job description that’s like, “I already have a bunch of technical people on my team, so this next person I’m looking for is somebody who’s really good at telling a story.” You cannot write that on a job description.

Often times I have rejected people that were really, really great—they just weren’t what I needed to round out my team right then and there.

Anything beyond? Your resume should make really clear what your skills are, your resume should make clear what your impact is. I don’t just want tp know that you can query data, I want to know that you can drive business decisions and make things happen and that you know how to approach a problem.

Actually, I think analytics fits really well for the liberal arts background because it’s not just, “Can you come in and do ,ath?” It’s also, “Are you strategic and do you know how to solve business problems?” Okay, now put those two things together. That is what I think really makes the good analyst.

I often think that the good analyst is someone who can do all the technical things they have to do. The great analyst is someone who can tell a story and know how to approach a problem and knows how to prioritize stakeholder management.

The numbers…your business person isn’t gonna like these numbers, they’re really low. How are you going to approach that? I think those are the things that often sets someone apart. But you have to be able to do the math, right? I can’t have someone who only can tell a story.

Max: Of course. This has been fantastic. I think we’ve gotten a ton of information. Thank you for sharing your experiences. It’s been amazing!

Rachael: Sure!