In this episode I chat with Ben McMillan, a founding partner of IDX Insights, a firm offering “indexing as a service.”

Ben cut his teeth in manager analysis at a fund-of-hedge-funds and we spend a considerable amount of time discussing how this experience impacted his research in building hedge fund replication strategies.

As it turns out, a naive replication strategy is very easy to implement. A robust one, however, is deceptively difficult.  One of the most interesting insights I gleaned from this conversation is that the edge in replication may not be in applying more sophisticated math, but rather in the data sets applied.

We discussed where replication might work, where it doesn’t, and the dependent nature these liquid replicators have in crowd-sourcing their allocations from their less-than-liquid peers.

Finally, we discuss how these sorts of replicators might be further enhanced by replacing standard beta factors with more customized index solutions.

Ben is full of insights and this one runs long.  So let’s not waste any more time and let’s dive in.

Transcript

Corey Hoffstein  00:00

Ben, we’re gonna give it a little countdown. You’re ready to go.

Ben McMillan  00:02

Yep. Perfect.

Corey Hoffstein  00:03

All right, 321 Let’s jam. Hello and welcome, everyone. I’m Corey Hoffstein. And this is flirting with models, the podcast that pulls back the curtain to discover the human factor behind the quantitative strategy.

Narrator  00:25

Corey Hoffstein Is the co founder and chief investment officer of newfound research due to industry regulations, he will not discuss any of newfound researches funds on this podcast all opinions expressed by podcast participants are solely their own opinion and do not reflect the opinion of newfound research. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of newfound research may maintain positions and securities discussed in this podcast for more information is it think newfound.com.

Corey Hoffstein  00:56

In this episode, I chat with Ben McMillan, a founding partner of IDX insights, a firm offering custom indexing as a service, Ben cut his teeth in manager analysis at a fund of hedge funds. And we spend a considerable amount of time discussing how this experience impacted his research in building hedge fund replication strategies. As it turns out, a naive replication strategy is very easy to implement. A robust one, however, is deceptively difficult. One of the most interesting insights I gleaned from this conversation is that the edge in replication may not be in applying more sophisticated math, but rather in the datasets applied. We discuss where replication might work, where it doesn’t, and the dependent nature these liquid replicators have in crowdsourcing their allocations from their less than liquid peers. Finally, we discuss how these sorts of replicators might be further enhanced by replacing standard beta factors with more customized index solutions. Ben is full of insights, and this one runs long, so let’s not waste any more time. And let’s dive in. Ben, thank you for joining me today.

Ben McMillan  02:16

Thanks for having me. This is great.

Corey Hoffstein  02:18

So let’s start with a statement that I think is going to set the table for the whole podcast because you’ve had quite a career trajectory and arc. There’s a lot of stuff I’m really excited to talk about. But there’s really one focus point of this podcast that I’m really excited to dive into. So I want to set the table with a statement, which is you believe that many hedge fund strategies can be meaningfully and accurately replicated at the category level using highly liquid instruments like ETFs. And I just want to let that marinate for a second, because it’s somewhat of a controversial statement. There’s all sorts of replicator products out there that we’re going to talk about, but I think maybe those folks in the hedge fund community might take a little offense to that. So I think this is gonna be a fun one. And we’re definitely gonna dive into that topic. But let’s start the story. Start the narrative picking up your career in the mid 2000s. If I am correct, you were at that time working at a fund a hedge funds. And this is really where you cut your teeth doing manager due diligence and return analysis. And maybe, you know, reflecting on that initial statement we made there might have been pretty formative experience for you. So can you tell me a little bit about that experience? Maybe some of the things you learned while you were analyzing manager performance? And maybe some of the things that surprised you?

Ben McMillan  03:36

Yeah, no, absolutely. And I’ll even back up and tell you that I can guarantee a lot of people in the hedge fund community will will take issue with that statement, because I’ve lived in real time. But so I joined an asset manager called Mr. Watson in January, one of 2008, just as things were getting interesting, and it was they had a couple of different components to the business, which was appealing to me. They had a third party marketing arm, which had been around raising money for hedge funds since the mid 90s. They had a fund to funds group which had, I think, three different fund of funds, about 2 billion in assets. So they, you know, they weren’t huge by any stretch, but they were big enough, big enough, coupled with the fact that they had this kind of third party marketing arm that was early in the game, raising money for hedge funds. They had a lot of access to marketing managers. So I was hired, I was coming out of American Express, actually, where I was, I was working in the quantitative risk underwriting department. So the guy that hired me was a quant. He wanted to he had a lot of kind of skunkworks type projects in the mix, you know, over and above kind of the traditional manager due diligence, you know, attribution analysis. And so he wanted, you know, he was looking for a certain skill set to to bring on board and work under him. So, he hired me started, you know, like I said, beginning of 2008, which was the perfect time and, you know, the first thing he had was, he was a big data guy, so, I don’t mean big data and you know that arm wasn’t around back then. So big in the sense that he appreciated having his own his own database of managers scores, he actually developed his own kind of manager ranking score that ultimately ended up getting patented. And this was all in access, like access 2001 database. So the, you know, the the first job he had was like, just figure this out. So I moved it all the sequel, automated bunch of stuff, we had kind of nice reporting sitting on top of it. Because also, because of that, I was going into all the manager meetings, all the cap intro events, they had two full time, you know, guys doing doing manager due diligence reports, but I went on every single one of those meetings. So it was phenomenal, because it was just a ton of data points at a very interesting time. Interestingly, having been at AmEx, actually, in 2007, that was one of the bearer funds were blowing up. And I can say it now. But you know, we were sitting there, you know, American Express was unique, because they own their POS network, which meant they had a lot of data, proprietary data that, you know, the other credit card companies didn’t have. And so we were seeing things like Revolv rates skyrocket, even among the, you know, the Prime customers. And so there was a lot of, you know, by the time I came over, there was a lot of chatter, a lot of writing on the wall, that things might be going south, like I said, particularly after the bear funds blew up in 2007. So, obviously 2008 happened. Well, let me back up. One of the things that immediately struck me even before q4 2008, was, I was kind of expecting this was my first job on Wall Street, and I was I was really anxious to kind of pull back the curtain understand, like, alright, you know, what’s the secret like, I’m in the club. Now, guys, tell me, tell me where the Alpha comes from. And it was just a lot of, you know, a lot of the same story, every strategy was different, but the luxury equity guys were kind of all telling the same story, you know, bottoms up stock pickers, they were spent two years at a bank for covering the sector. You know, they look at it a different way than everybody else, which is what everybody else says, you know, the distressed guys all have the same story stat or convert Bhandar. It was difficult from that to kind of segregate who, you know, who really had kind of a persistent Alpha generating process. And so and even kind of powwow on with the other analysts and sitting on in the Investment Committee meetings, you know, as it just I was observing real time decision making process. And it was just, you know, there was a lot of kind of homogeneity within the hedge fund strategies, or so it appeared to me, and so q4 2008. Happens coming out of that. And, you know, not surprisingly, or perhaps surprisingly, hedge funds, you know, didn’t really perform that, well, certainly, long term equity funds, you know, they were, depending on what index, you look at, our funds were down between 20 and 30%, you know, for presumably, when they should have been adding a lot of value. The other kind of part of that, too, is, you know, coming out of Oh, eight, we started seeing all these gating provisions thrown up, you know, managers, you know, it makes sense, I get it in, you know, illiquid credit and distressed asset classes. But if you’re trading publicly traded equities, you may not like the bid that you’re getting on those stocks, but they’re liquid, you know, you can sell them. I remember, one manager, even in the credit space said that he’s like, you know, people are saying, there’s no bids for things out there. He’s like, Oh, there’s bids. He’s like, You mean, like what the bid is, but you know, it’s somebody’s on the other end, picking up that phone. And so, you know, we were coming out at kind of 2008 2009, like a lot of other fund to funds. You know, with kind of this this existential crisis between, you know, investors, we’re really becoming sensitive to fees starting to question the hedge fund value, Prop, the transparency, the liquidity, again, you know, this is public at this point, so I can say it, but we were that fund to fund one of the fund of funds was invested in Harbinger. And I remember it was we were sitting on the list, or Watson was, I believe, one of the funds on the creditor committee that after Phil Falcone borrowed $100 million from the fund to pay his personal taxes. You know, that was just kind of like one of the final straws where we’re talking to a lot other institutional investors, I remember as a cap intro talking to a very big pension, a guy from a very big pension fund. And he was just like, in complete disbelief that this, like, you know, and he’s like, Yes, I understand it was in the docks, like he didn’t do anything illegal. But, you know, that just kind of brought to the forefront. What, up until that point had kind of been an academic idea of the the call option for the hedge fund manager idea where, if you lose his money, he doesn’t really have any skin in the game, he still clips his 2% management fee. But he’s at certain points that are incented to take, you know, perhaps asymmetric or uncompensated risks, because it’s a call option. And so anyway, like, first half of 2009, this is all kind of rising to the top a lot of fun to funds, like I said, are questioning their own value prop there was even I remember being at a conference talking about and somebody’s questioning the value proposition of funds of fund of funds, you know, kind of the fund of funds cube type model. I just remember going back like we were talking about it was like almost a Dilbert strip. I was, you know, we were sitting there on the Investment Committee back at the office, like, at some point, this is just getting like, it’s getting ridiculous, like, you know, people are gonna say, enough is enough. And so, meanwhile, we had the guy that hired me, Mark Frieden and I had already been doing a lot of work on the attribution side, you know, really understanding is this hedge fund You know, aside from what their what their processes with the qualitative write up says, Are they adding value relative to their peer group? If so, how much? How do they do it? And one of the things that followed on from that was this idea of not just running factor attribution against a theory of factors, but but really starting to put constraints on what those factors were, and specifically look at things that we could trade basket manager had a broker dealer. And so this, you know, this was something we started really considering was the idea of all right, you know, if there’s a lot of beta, and that was one of the things. So let me circle back to the very beginning, when I was talking about, you know, a lot of the stories seem the same. We’ll fast forward, you know, by the end of 2009, early 2010, you know, we’ve done, we did a real deep dive, my full time job at that point became, you know, really trying to dig into the idea of, you know, replicate statistically replicating some of these risk return streams. And what we found is that there’s just a lot of systematic risk in these different hedge fund strategies. So hedge funds as an asset class, maybe not so much for obvious reasons. Long Short, equity funds are doing, you know, employing very different strategies, harvesting very different risk premium and credit funds, or stat ARB or things like that. But if you if you segment the strategies, well, and I’ll come back to that, because it wasn’t, you know, that wasn’t necessarily a trivial process, but we had our own database. If you really create these kind of homogenous groups of hedge fund strategies, you find that, you know, there’s a lot of the same underlying risks driving all the funds in that strategy. And this was, this was about the time that, you know, like I said, the idea of hedge fund replication was was really coming to the fore out of academia into kind of products on Wall Street. So I’m gonna go ahead and break my cardinal rule of podcasting here and just take a complete tangent after my first question, I may pull off the highway of my main main thread here. But, you know, there’s

Corey Hoffstein  11:52

a lot of discussion that I’ve been having both online and offline with folks about the applicability of a lot of the factor models and risk models that are available today. And you see it very frequently. Now, it’s a value proposition of a lot of asset managers to come in to an institution or an advisor and run some sort of factor, you know, risk decomposition, throw back a 30 page PDF, and talk to them about what their exposure is. And my frustration with that has been, the results are perhaps directionally correct. But never truly very precise that depending on how you run your analysis, the time series, you use the factor definitions you use, you can get very, very meaningfully different results. And so in your experience in running all this analysis, and attribution, how useful Did you find it? Where was the use in running that analysis? Was it in the actual precision of identifying the factor exposure one manager has versus another? Or was it just categorically being able to say, this is a value manager, this is a momentum manager.

Ben McMillan  13:02

That’s an excellent point. And I could if you just pulled off kind of the freeway of the mainline, I’m just gonna step on the gas because this was this, this was an up, you know, continues to be a huge source of irritation for me. And I’ll, I’ll start with the story, fast forwarding a little bit, we ended up at ended up at Vanek global. And we’ll talk about how we got there several years later, and I remember coming coming in there, and they had an external manager for IAQ products, so fun to have external managers as a mutual fund. And they kind of gave me this 30 page PDF, and said, Yeah, we had a group come in and two factor attribution on these different funds. And this long, short equity guys, you know, all upset because he said, it’s just, you know, total, totally fake. And so I was like, alright, well, let me you know, let me see what the attribution says, you know, they were kind of like, you know, you’re the guy with a degree in econometrics, like you tell us what’s going on. And so the first thing I looked at is like, there was no leverage constraint. So you know, it had this guy running it like, you know, this, they were running linear linear regressions. But they were inferring, like, two or three times leverage. The Swiss franc was in there as a factor. It was all over the place. And it was like, I was like, I don’t even care. They’re like, Yeah, but the t statistic, I was like, I don’t care what the T stats, you know, but the t statistic says, it’s like, there’s, you’ve got to put some sensibility. Constraints are filters in place, if you know, before you even run this, otherwise, the regression isn’t a genius, it’s not going to figure out and Intuit these things. You’ve got to put guardrails in place. And so back to kind of your earlier point, I’ve we were very, like I said, this was this was originally my formal training. And this is, you know, at American Express, we were running regressions for a living on huge datasets, and they were very focused on robustness. And they would just drill into you that, you know, you can’t, you can’t just hit f9 on, you know, a MATLAB code and look at the correlation coefficients and take it as gospel. You know, you’ve really got to make sure it’s fundamentally robust in the way that you set up the apparatus for the in this case, regression to make any inferences. And so what we found that Mr. Watson is we almost never ran individual managers against factors or if we did, we would include the peer group as a explanatory variable. So we would look at, you know, long short equity fund a, run a regression against our index of long short equity funds. And then we would include a handful of other sensitivities to understand to what degree on a risk adjusted basis where they may be tilting more towards the value or more towards growth or more towards a particular sector, relative to the, to the index, but that was crucial wasn’t, you know, including the kind of anchor explanatory variable of the other long, short funds, because, you know, then you’re just instilling, you know, a higher likelihood of robustness around the other factor sensitivities, but just taking an individual font, and, you know, running our aggression against that’s the other thing too, against a lot of factors, you know, there’s this tendency, and this is I don’t want to go into the huge segue here. But this is one of the things separately, I’ve kind of observed and a lot of, you will have to a lot of quants have in the last 10 years is, you know, computing power has gotten a lot cheaper. All these analytics packages are free, or very inexpensive now are in Python, you know, the pandas library, you know, replicates what my $250,000 SAS ETS package at American Express did 10 years ago. And so that enables a certain laziness, you know, it’s, it’s easy to just kind of, you know, hit f9, and see what the computer spits out. And if you, you know, it can be appealing if you don’t know the landmines to look for. So you, I 100% agree, and you absolutely hit on what I think is continues to be one of the biggest issues in Wall Street in particular, is is just the cheap data, cheap computing power has has kind of led to this lackadaisical quantitative approach. And so just to wrap that comment up, yeah, as it relates to kind of factor replication, our dependent variable, the thing we were replicating was never an individual fund. And there were some there was some products out there that attempted to do that, or they would take three or four funds, we spent a lot of time refining that dependent variable. So we would subscribe to the to the raw databases, manually classify, in this case, let’s let’s use long short equity funds as an example, we would go through we would look at their classifications, we would run a statistical analysis analysis, did they look like other long term equity managers? Interestingly, what we found and like I said, when I joined, you know, Mark had been doing this for 10 years already. And so he there was already a lot of data to kind of go back and look at things like style drift, manager, drift, just use classification, you know, fun, you know, can be misclassified, yet show no style drift or fun can be properly classified and show style drift. And what we found and again, I’m, I’m gonna cite statistics from washer equity space was that 20% of the funds that were classified, as long as you have equity, we would reclassify and, you know, 5%, were just incorrect. You know, it was it was a fat finger error, it was long, short credit, somehow it popped up and long short equity, but 15% were just too too different from the group. And by that I mean, there was a lot of, you know, like long short biotech is going to have, you know, a very different return premia are very different drivers of return than kind of your general Greenlight capitals of the world kind of general, longer biased, long, short managers. And so that was a large, that was a large part of, you know, we spent as much time really refining the dependent variable that we were going to drop into the regression, as we did, actually kind of developing the regression methodology, because we didn’t, you know, we didn’t want to go down the path of just throwing more complex math at the problem. And again, there were products that we’re looking at, you know, Kalman filters, and, you know, starting to integrate support vector machines, I mean, you could, it was easy, then it’s become even easier now, to just throw a lot more kind of computing complexity and power at the problem, we wanted to really make sure that we are developing a robust infrastructure such that a relatively simple linear regression could yield useful results. And that was, because I’ll wrap up by saying, and that was because, you know, we went into this exercise, you know, 2009, expecting to put monies in this in this in this strategy. Now, you you absolutely hit on on a hugely important point.

Corey Hoffstein  19:05

Well, I appreciate you getting us back on the highway there have read. And that’s, you know, it’s a really interesting topic that you bring up this, this is your edge in your computational power, or is your edge in your data. And to your point, in the era post 2008, the number of free open source computational tools, the ability to quickly churn through that data has really gone exponential. I mean, the in the Python landscape, you mentioned pandas, but things like sai kit, learn and TensorFlow and all this stuff, all of a sudden make machine learning techniques, one line of code and makes it that much easier to just sort of throw a handful of variables in a blender and hope something good comes out. So that’s something we definitely want to return to but back to sort of the main thread about replication or a little bit of the history here because I do find it interesting. It was in 2008, that Andrew Lowe first published sort of the semi I’ll work I guess you could call it on hedge fund replication. And if memory serves me correctly, it was a pretty simplistic model. It was, I think, just using 36 months of prior returns, and doing a very simple linear regression with some liquid factors he showed you could get some pretty meaningful replication. And the idea caught on pretty quickly. You mentioned a little bit in in your first answer, the zeitgeist of the 2008 era. But you saw this idea really took hold, I know that some banks started issuing products, privately that were actual implementations of Andrew’s idea. Can you tell us a little bit more about from your perspective, working at the funder hedge funds, what things were like in the evaluation of the hedge fund community, what the attitude was towards hedge funds, and maybe why these replicators were becoming to be so attractive to investors?

Ben McMillan  20:58

Yeah, what was interesting was what I saw anyway, was the reasons I mentioned previously, you know, post away, you know, funds the value prop of, of hedge funds in general, but I’m really going to focus on long term equity funds started to be questioned, and really like things like liquidity that was that was a recurring refrain, refrain I heard from other institutional investors was, you know, why is it take me four quarters to get out of this long short equity fund when they’re trading publicly traded equities. And so the other side of that too, and where I really started to see it, kick off was made off. And so I’ll never forget, this is, like I say, funny, but this is a really interesting story. Mark freed was sitting at the Bloomberg one day, and he called into one of the other partners offices, and he’s like, hey, they finally caught Bernie, and I’m like, What’s, who’s burning? And he’s like, Oh, it’s this guy that, you know, the, nobody really knew what was going on. But, you know, they were the rumor was, uh, he was front running his clients, he came in here, you know, one of his feeder funds came in here, and, you know, pitched us and he’s like, it’s interesting, Bloomberg saying was a Ponzi scheme. He’s like, I don’t know. But anyway, obviously, that all unfolded. Interestingly, I also went back to the original Access database that he had. And sure enough, one of the feeder funds was in there. With those, you know, the returns, I looked at the score, and it was interesting that it had been kicked out as a unreasonable outlier or something like that. But made off back to that point made up was when I really started seeing from the institutional investors that we dealt with the demand for SMEs go up transparency, liquidity, and that’s when, again, you know, I’m importing my own biases on this, but that’s when I really started to see kind of the replicator landscape take off, just from an interest level, and it was, you know, for better or for worse, it was, you know, it was a very disparate landscape. So, you know, there was the idea that, like you said, you know, low kind of kicked off the idea of statistical replication, I think Merrill was the one we went through that exercise very closely replicated the results from Lowe’s paper, you know, to within like, basis points, he kind of gave the whole recipe there, you know, we saw that I think it was Merrill had launched a note off of that paper to within, you know, basis points. And so the immediate thing we did was just take that model, and what was interesting was low, low use five fixed factors. So he just kind of said ahead of time, um, you know, we’re gonna identify a priori, you know, five factors, if I remember correctly, that, you know, we would each have economic justification for, you know, being the underlying drivers of returns for hedge funds. And those were static factors that are used to in this kind of rolling window approach against the hfri. W index. So this, this dependent variable of comprised of, you know, all hedge fund strategies, the results, were encouraging, kind of the obvious, and he mentions this in his paper, but the obvious next step would be alright, there’s, there’s no reason that you would want to try to replicate all hedge funds as an asset class. Or if you did, you would want to run the regression on different more homogenous dependent variables. And so that’s what we did, you know, we already have this database. So we just dropped in our long short equity index into that model. And, you know, the, our scores were higher. And it’s worth noting, too, that these are out of sample or squares because it was a rolling window analysis. That was one of the early criticisms or questions we got was Yeah, but this isn’t sample. And so, you know, lo talks about this Fung and che were two guys at Duke that had done a lot of early work on extending kind of the fama French model two, you know, seven and eight factor models for hedge fund returns. And so, you know, as a quant, we were, I was sitting there saying, Alright, this is appealing, because it is truly out of sample, you know, we also constrained the universe to ETFs, we did a stepwise process to allow the the constituents of ETFs to change from month to month. So we didn’t fix five factors, we broaden the landscape a little bit, and we started getting, you know, really compelling results, you know, out of sample our squares of 95% on our long short equity funds. And so that was interesting. I’m going to pivot a little bit though, in get back to your comment about the kind of zeitgeist of the area so, you know, meanwhile, you know, so the low paper kind of kicked off the idea of statistical replication, you know, using a top down regression approach. Meanwhile, I believe it was Mark Malick over a conquest we It’s just a well known managed futures, had kind of, you know, taken a different approach where he talks about that he published a paper called the beta of managed futures where he used some kind of momentum. I think he used average true range or, you know, one of the more kind of technical indicators in the Managed futures universe, but his point was, you know, trend following within the Manage futures landscape can be thought of as its as its own asset class or its own risk premia. And he kind of mechanically replicated, you know, a rule set that would harvest or seek to efficiently and effectively harvest that risk premium. So that was, that was kind of the other side of it was this statistical approach this mechanical approach, and then you had the 13, F replicators, which was a good idea in theory, but I remember we had, I saw one of the Bloomberg reporters at a conference, right, I was talking about the idea of hedge fund replication, he came up to me he’s like, What do you think about the new girl ETF that’s coming out? And I was, you know, we looked at the hadn’t been hadn’t launched yet. So we pulled the, you know, the file prospectus and looked at it. And, you know, it was, again, it was a great idea, but the problem was, they were looking at if I remember correctly, the top 100 hedge funds by AUM, and then just kind of pulling off the the long positions from the 13 F because shorts aren’t, aren’t required to be reported. And, you know, holding that holding that as a portfolio, and there was some, you know, rebalancing, I’m assuming it was quarterly rebalancing around the around the 13th cycle, but if you look at the hedge funds that they were replicating, you know, Renaissance was one of the top ones. And so, you know, I told this guy, I was like, you know, I don’t have any particular insights into Renaissance, you know, other than, you know, Simon’s is clearly a genius, but they themselves will tell you, I mean, the turnover in their book is like, I don’t know, 15 minutes or something like a quarterly, you know, a quarterly point in time snapshot of the stocks they hold, is going to have zero persistence or forward looking predictive ability, you know, in terms of the constellation their portfolio, you know, and then you think about other end of the spectrum, it’s like, you know, I think Angelo Gordon, the distressed fund was, was one of the top 100 by AUM, well, you know, they’re, they’re the 13, EPS is only going to be, you know, kind of stub post IPO type positions that they got stuck with, or it’s not going to be an accurate reflection of, of the risks that they’re intelligently underwriting. And so my point was, it was a perfect example of kind of a good idea, but what I would argue is a mediocre or subpar implementation. And so I mentioned that because all of those approaches kind of got lumped in as hedge fund replicators. And so there was a lot of early criticism where people were pointing to these things and saying, Hey, 13, you know, 13 apps can replicate what Seth Klarman is doing. Meanwhile, the statistical approaches can’t, you know, can’t replicate non publicly traded asset classes, like post reorg equities or things like that, we were kind of waving a flag saying all of that is accurate. However, if you really refine the universe, that you’re that you’re looking at, allow the the ETFs that you’re using as explanatory variables, and ultimately proxies to trade the strategy to, to, you know, vary a little bit, you can get, you know, really compelling results. And so that was just to finish this this up. That was when we kind of sat there and decided, alright, this is 2010. At this point, you know, let’s start putting firm money in this for the long short equity clone portfolio as it was, as it was called, and then just start, you know, start tracking this real time.

Corey Hoffstein  28:15

So you bring up a couple really interesting points. One of the guests I actually just recorded with a couple of weeks ago, sat in sort of the manager selection, and evaluation space and actually runs a 13 F replicating strategy now. And he echoed much of the same sentiments that you had, which was you can’t just blindly take 13 F’s without really understanding who the fund is and how they operate, because to your point, tells you nothing about the shorts that they hold that might be offsetting positions and tells you nothing about the derivatives that they hold, it might be a far more complex strategy, say than just a long only. So to him it was it wasn’t that 13 F’s weren’t necessarily applicable, it was you had to use them appropriately. And it sounds like that’s sort of where you’re going to the point of, you might be able to look at these common beta characteristics and do replication. But if you have more insight to the funds, it sounds like it allows you to perhaps have that much better evaluation. And it reminds me a little bit of, I think it was Catherine Kaminski, who wrote about the Managed futures space to sort of piggyback on your comment there where she looked at saying, Look, we know there’s all these managed futures funds, some trade quickly, some trade slowly, let’s make that a factor. And she mechanically made a fast minus slow factor. She also said there’s large funds in small funds and the large funds have to trade in different conch contracts. Because of their size. Let’s make that a factor. And obviously, the explanatory power went way up when you have these insights into how the industry operates. So I want to roll that into the paper that you wrote in 2011. The title of the paper was investable benchmarks and hedge fund liquidity. And you wrote this with Mark freed and mark your owl and it really extended Lowe’s work and InCorp worried at some of these ideas, so I was hoping you could again, sort of expand upon some of the problems you identified in Lowe’s original work, and maybe some of the solutions you were able to develop based upon your intuition and insights you had from doing all the manager evaluation and selection.

Ben McMillan  30:20

Yep. So the biggest one into his credit, you know, lo himself kind of identifies this as a area of ongoing work. And it was something like I said, because of the proprietary database that at that point, you know, had been running for a decade, we had a lot of rich insights into this was just simply how a how different hedge fund strategies are very different betas. But then even within each of those strategies, the similarity of managers is much stronger than people realize. So it’s kind of like two insights immediately rose to the top, which was, you know, launch with equity. And this is this is very intuitive. You know, it’s anybody that’s been in the hedge fund space is long, short, equity, hedge funds are very different than stat, our hedge funds, yes, both of them are market neutral, let’s say both of them are trading equities. But they’re, they’re trafficking in different timeframes are employing different approaches, they’re harvesting different factors. It’s a totally different game. And so what we had been doing at the hedge fund, like I said, when we were, you know, running our manager due diligence reports, we were benchmarking or running attribution for every manager against their respective benchmark, which was an internal benchmark we created. And like I said, that was, you know, it wasn’t just as simple as saying, Alright, let’s take the classifications, there was a lot of manual reclassifications, quantitative oversight style drift. And so, you know, we ended up you know, our long term equity benchmark, you know, looked different enough from kind of the off the shelf investment, or hfri benchmarks that there was power there and just kind of cleaning the data. And so that paper really did, there’s kind of three things in that paper, the first one was, you know, we didn’t want to go down the path of extending the math and running a lot of fancy, different regressions we’ve done that internally is an interesting exercise continues to be I think, in many respects, but what we wanted to do was introduce the idea of, you know, let’s, let’s keep the math relatively simple, and more importantly, robust. But let’s introduce a little more sophistication in the dependent variables. So let’s, let’s keep the model the same, but let’s point it at something that’s a little more thoughtfully created. And so we had, I think, I think 10 different internal hedge fund benchmarks that we manually created updated every month, to give you an idea of, you know, we didn’t even just look at long should equity hedge funds, we classified by long short equity, North America versus long term equity, em, versus sector focused. And so we were, you know, we wanted to make the dependent variables as homogenous as as reasonable. And then the other thing we did was like, as I said, we you know, we allowed the risk factors to a be different by hedge fund asset class. So, you know, the risk factors that we’d be looking at for convertible bond arbitrage would obviously be different from the long term equity funds, we incorporated a little bit of a stepwise process to allow there to be some variability in terms of, you know, looking at the different constellation of risk factors. And what we found was when you use that, you know, our dependent variable when you basically took Lowe’s model, and pointed it at what you know, we would argue are a little bit more thoughtfully constructed hedge fund indices, as opposed to kind of an off the shelf version, that’s when we saw the R squared just go massively up. And like I said, at the top of the list, I think on the first page or second page, in that paper, we have an exhibit, which which maps or shows the out of sample R squared for all the different hedge fund strategies. And so at the top of the list, not surprisingly, is long, short equity, North American hedge funds can you know, 95% r squared against a basket, a dynamic basket of you know, tradable ETFs, long and short. We constrain their aggression to use no leverage. So again, all those all those problems of kind of just the original, kind of hit the button and see what we get type factor, attribution reports, you know, we were developing this with the idea that we’ll actually at this point we had already been, we’d been running this paper came out and at the end of 2011, so we’d already been running money in this portfolio for a year. And I’ll come back to it. But we actually ended up getting a seat client, a big Hospital Foundation came in see that the strategy with 40 million. And by the time this paper came out, it was it was upwards of 100 million that we were running in the long short equity North American clone portfolio. And I’ll come back because a lot 2011 was an interesting year in the hedge fund space. And it led to another paper that we wrote. But the other thing, which was kind of interesting, and it gets it gets lost a little bit, but the ostensibly the point of the paper we were writing in 2011 was was it kind of presumed, okay, if you can realistically replicate certain hedge fund strategies, then what is the price of the option that you give the manager if you’re willing to accept less liquidity than the underlying So the premise being that alright, if I have access to, you know, hypothetically have have access to a basket of actual long term equity managers, which, you know, most institutional investors do, and you have access to a cone portfolio. Well, you know, one of them has daily liquidity, one of them doesn’t. And let’s assume that there’s no difference statistically or, or minimal difference statistically between the two portfolios. I’ll even acknowledge that there’s potentially a difference but let’s just price that enter let’s let’s account for that. But let’s isolate the the optionality that you’re giving the hedge fund manager by accepting less anything less than daily liquidity. And so we employed a bunch of options math to, to kind of price that out. And, you know, the number not surprisingly, was high, depending on you know, kind of where in the matrix you were, you wanted liquidity at what you’re giving up. But like I said, that was kind of secondary to the, you know, the fact you start getting a lot of interest in the fact that people were saying, wait a second, you know, the options applications, kind of interesting. But let’s go back to these Cohen portfolios, like, how are you doing this? Oh, by the way, you guys are actually managing real money doing this 2011 was an interesting year, because long term equity funds, depending on the index that you look at, we’re down, you know, between six and 8%, our clone portfolio, which, you know, for most of that year was over $40 million, I think was down, you know, 2025 basis points or something after after fees, which is also another big point is, you know, we were charging, I think 50 basis points or less than 50 basis points for that, for that product, it was running an SMA for this big Hospital Foundation, full transparency. You know, interestingly, I don’t want to go off in too much of a tangent, but they were huge. I mean, I think they had several billion dollars, just in Long, short equity funds. And they said they, they found the most utility, you know, they liked it for a lot of reasons, they can use it for cash, they kind of use it as an ATM machine. So that get, you know, they could access cash, you know, inter mom that could use as a placeholder after identifying a manager that they liked, but couldn’t yet send a subscription into, but they also used it as a real money benchmark to go back to their active guys and say, look, listen, you know, you guys, your your Tear Sheet saying that you have all this alpha relative to the s&p 500. However, you know, you don’t have any alpha or negative alpha relative to, you know, this Cohen portfolio, and they say, Yeah, but that’s not investable. And they say, Well, no, actually it is, we have $80 million. And we need to, you know, we need to either benchmark your your fees relative to this or have a conversation about fees. And that was something that was interesting to me, because I just, you know, that hadn’t really occurred to me as a, as a use for a big institutional investor. But, you know, it was it was one that they were, you know, it was utility that they were clearly clearly seeing over and above the, you know, just the, the mathematical appeal of the club portfolio itself. So, talk

Corey Hoffstein  37:09

to me a little bit about 2011. And, again, I’m going to take a tangent here, you know, you mentioned that sort of the average long short fund was down six 8%, the clone portfolio was down far less than that. But in theory, a clone portfolio should be sort of capturing the average, right? So it’s not the idea that necessarily that a clone portfolio is going to outperform ignoring fees, it should be getting the average. So where did that difference come from? And why should we look at that as a positive thing, necessarily? Shouldn’t we say, well, you sort of missed the benchmark there?

Ben McMillan  37:43

No. So that that is exactly the point. And that’s that is it? So post 2011, we kind of did a deep dive and said, alright, what happened here? Did we miss the mark? If, you know, clearly, we missed the mark. But was it unexpected, necessarily, and we had seen evidence of this type of behavior previously, and I’ll back up. And so one of the early criticisms we got or questions we got when we were kind of, you know, pitching this this product was, you know, people said, No, you know, hedge fund managers are way too dynamic for a monthly rebound. So, you know, again, as I mentioned, we stuck to the Andrew Loeb approach of, you know, rebalancing monthly, not only because that was kind of the right periodicity, but also that’s as frequently as we could get access to the dependent variable, you know, hedge funds only update monthly. So, you know, we couldn’t run it any more any more frequently, if we wanted to. But what we saw was that, you know, when we do those, those internal attributions for the fund of funds, you know, you can you can infer To what degree these correlation coefficients, you know, turnover over time. And so, you know, not surprisingly, stat ARB or market neutral is kind of a good example, those are, you know, typically higher frequency strategies. And so it ends up I mean, the big the big issue there, and we can talk about this later, but because their short short book was so large, from a top down statistical approach, it ended up just looking like a lot of cash. And, in fact, there were hedge funds. So that was a strategy that we would say, you know, is not suitable, at least not suitable for statistical replication, you know, maybe you can recreate it from the bottoms up mechanically, but, you know, running running a linear regression on a market neutral, a group of market neutral hedge funds, basically going to put in a bunch of cash, cash bond ETFs. Getting back to the long, short, long, short equity funds, you know, a lot of people were saying, Yeah, but, you know, hedge funds are way too dynamic. That’s what we’re paying for them for. And, you know, we were pointing to all this data saying, you know, actually, they’re not, you know, their value add, and again, at least in the long, short equity space, you know, they they’re very good stock pickers, and collectively, there’s really no stock alpha, but there is there is factor alpha, you know, they are kind of timing the factors in a way that adds value relative to what’s called an s&p 500. Over time, you know, they’re they are doing a good job, collectively, of kind of bundling these factors in a way that makes sense. And isn’t high turnover. These factor exposures, you know, yes, they’re dynamic. They migrate over time, but it’s over a period of months and quarters, you know, it’s not it’s not intra month. And so post 2011, we were looking at this and like I said, you know, we were running, we had the benefit of running this real time during 2011. So obviously, the Greek Euro crisis started to kick off in the back half of the year. Meanwhile, I was, you know, I was this was still at the fund of funds. And so I’m going on these major due diligence meetings, just because it’s a phenomenal source of real time data, particularly during an environment like this. And I’ll never forget, we were at a very big, still a very big, long short equity fund, and we were talking to one of the sector heads, and the guy said, for my entire career, all I’ve done is pick telecom stocks, he’s like, I know, every company, I, you know, I know the space. So well, he goes, now all of a sudden, every single day, me and my team are in a conference room on a whiteboard, trying to handicap the probability of whether or not Greece is going to default. And he’s like, that’s just not what we do. And that was that kind of kicked off, you know, the thought that, you know, Wow, these guys for any number of legitimate reasons, we’re dealing with risk factors, not just compensated risk factors, but also, you know, business risk factors. And this was something I credited, actually, you know, Mark curl, one of the one of the guys who was in business development, he said, you know, don’t forget, these guys are running businesses, so you get into the back half of the year, you know, if you’re already down, you know, X percent, there’s, there’s a very good reason to just go to cash, you’ve got a mortgage to pay, you got analysts to pay, you know, there’s all these non, you know, what we ended up calling these kind of uncompensated risk factors associated with any active manager, but in particular, LP funds, that, you know, investors are essentially, you know, funding or exposing themselves to knowingly or unknowingly, you know, some investors would would say, I don’t believe that, you know, that that’s meaningful, other investors would say, Yeah, but that’s the kind of price we pay. And so we were observing that real time, kind of an interesting corollary there is, we were getting to this big Hospital Foundation, which at that point, I think, was our, certainly our largest client, if not our only client, they had SMA as with a lot of their other, their other long term equity funds. And so these, you know, they were big enough that you know, even even the marquee names would give them separately managed accounts. And what was interesting there was not only did they have daily liquidity, which they insisted on, but they also had intra month transparency. And so there’s this whole game around, and Fung and che talked about this a little bit in some of the early academic literature. And again, we you know, we observed it real time, but there’s whole, this whole game around, you know, kind of reporting, reporting returns intra month, and there’s, if nothing else, just as a price discovery mechanism, the clone portfolio had a lot of value, because, you know, we would get these guys would call us, the Hospital Foundation would call us and say, Hey, where’s the clone month to date? In October, for example? And I’d say, you know, it’s down however many basis points and they’d call back in November during the rebound from getting my if I’m remembering correctly, and it’s a, you know, what’s the comb portfolio up? I go back and say, Alright, we rebalanced into, from out of these factors into these factors, the clone is up, you know, 300 basis points. And he’s like, okay, so, you know, the coin made it all back. And I said, Yeah, and, you know, meanwhile, he’s like, alright, well, none of these, all these managers took risk off, and, you know, missed the rebound. And so, you know, that was clearly that was clearly the factor that the clone, you know, for better or for worse, was insulated against. And the subject of the 2014 paper argued that, that factor, that that intra month, when you when you get out, the factor I’m defining here or attempting to define is, when you get outside the timeframe, specifically within call it an intra month timeframe, where on average, the luxury equity managers don’t add value from from rebalancing or managing the net exposure, by insulating yourself from that kind of sticking to your guns, for lack of a better phrase in a systematic loan portfolio, you’re effectively not paying that tax, and it is attacks over time. And that was the point of the 2014. Paper was, you know, to your exact point, we did the kind of this, this debrief, and you know, q1 of 2012 and said, you know, all right, I’m glad that we outperform the benchmarks by 800 basis points, but was it for the right reason? Is there a risk here that we’re missing? Could it go against us? And the net effect of this deep dive was, you know, the, the risk factor that we by design don’t have exposure to is this kind of, you know, what I’m gonna call intra month or higher frequency risk on risk off tactical management, that the long term equity managers just collectively don’t add value? And in fact, actually, it’s the, it’s even further than that they collectively detract value. And the thesis just to wrap it up, that’s kind of the thesis of that 2014 paper, you know, we attempted to quantify it, but the thesis of that 2014 paper was, there’s reasons for this, there’s, there’s these business reasons, and, you know, that was one of the benefits, I credit, the fund to funds experience was just simply having access to these guys, even if it wasn’t me, you know, the partners were all senior guys, you know, they they knew the PMs of these funds, you know, one example and I remember they said, you know, when John Paulson’s fund blew up and in a good way, you know, when he made that big, short subprime trade in 2007 You know, they were kind of laughing. We went into his office to do a manager due diligence thing, and I remember the one of the proprietor one of the founders of my fund came out He’s like, Yeah, he goes, I remember when that guy was only $200 million in a merger ARB Fund, which is when they first made their investment. He’s like, you know, he goes, I appreciate you made a lot of money in the short subprime trade he’s like, but that looks a lot different than merger. ARB doesn’t it? And he’s like, you’re the quant you tell me and I was like, Yeah, that’s a little bit of a departure. But anyway, to finally wrap this up, and not belabor the point, that was an important kind of anecdote that we were getting as the fund of funds was, you know, we were hearing these guys, like at this huge, long short equity fund, you know, actively tell us that, you know, they were kind of outside, they’re out over their skis and trying to navigate this kind of, you know, risk on risk off environment that was was typical during 2011.

Corey Hoffstein  45:36

What I find really, really interesting about the whole discussion of of replication is that in many ways, it is sort of a crowdsourcing endeavor, right? And there’s somewhat of a almost like circular nature of the dependent variable, where you’re rooting in theory for these active managers to do well, right? Because if you’re replicating what they’re doing, you would want them on average to do well, so that you can sort of track what they’re doing. And yet on the other hand, the whole argument for replicators is, there’s all these sort of perverse and potentially adverse incentives that these managers have, and oh, look, the the replicator is potentially able to avoid that. But if you’re the replicator puts them out of business. Well, that actually is one less manager that now the replicator can rely on. So how do you think about that relationship?

Ben McMillan  46:23

No. I mean, that’s you hit exactly on it. And this was kind of the you know, I don’t want to say joke, but this was, you know, in 2012 1314, this was kind of, you know, the thing that we kept coming up against both internally and externally was, we don’t want to put the launch of equity funds out of business, because, you know, we need them. If anything, we kind of refined that narrative and that clarity that we got from 2011 and said, launch equity funds, or, you know, our position was that individually launch equity funds, both individually and collectively add value, we would argue that, you know, there’s obviously more idiosyncratic risk at the individual level, even when you collect a basket of long term equity funds, you’re dealing with these kind of uncompensated, what I’m going to call business risk factors. But it was tricky. I mean, you know, we we ended up the for the final funds, business guys ended up retiring a third party marketing firm, you know, the different pieces got sold to different different groups. And so this hedge fund replication business ended up being bought by Vanek global the ETF shop. And that was one of the things they discussed is they’re like, Well, you know, are you guys worried about putting the hedge fund or, you know, the hedge fund business at a business? And the simple answer was, the incentives to be a hedge fund manager, at least back then the incentives to be a hedge fund manager, were still great enough. And everybody, you know, there’s always going to be a lot of group, you know, a lot of guys that think, you know, their approach is a little different, it’s, it’s, it’s worth taking a swing at, and that was kind of our position was that, you know, they’re always going to be there, collectively, you know, the crowdsourcing, you know, the, the statistical crowdsourcing approach works on this asset class. But, you know, the other part of it too, is we, you know, we were very clear to investors, this, the comb portfolio was never going to be up, you know, 30%, it’s never, you know, by definition, it’s always, you know, it’s gonna track or should track the, the indices. So, you know, if you’re a smaller endowment, or a smaller pension, or just a smaller institutional investor, and, you know, you’re only investing in one or two long term equity funds, you know, it probably, and you have access to them. And, you know, you don’t mind kind of the business risks that your underwriting, you know, it probably doesn’t make sense to just invest directly with, with, you know, managers, you could, you know, one could argue that you could find the, you know, the the guys that are going to do better than the rest of the group, we kind of pivoted our approach to go after the really big guys like this Hospital Foundation was kind of the perfect client, you know, they, they were so big that they, they needed to benchmark part of their active portfolio in these clone portfolios. And we kind of landed on that as kind of the perfect case study of, you know, how these fit in the ecosystem, you know, in a in a way that’s kind of preserves the homeostasis and isn’t too disruptive.

Corey Hoffstein  48:54

So we spent a lot of time talking about the long short space, specifically the equity, long short space and some success you had there and replication. Were there other areas of the hedge fund world that you found that replication worked really well for? Or maybe more interestingly, the ones where it really didn’t seem to work? Well, in areas you don’t think replication can be effective?

Ben McMillan  49:17

Yeah, so that mean, kind of the first obvious one was was like stat arbor or market neutral. And that’s where I think it was I remember I got a call this was a Banach had bought us out and kind of the middle of 2012. And I think it was like, in my first month there, I got a call from from a good friend of mine, who’s the CIO of a big French family office in Manhattan. He said, Hey, this group just launched in a market neutral, replicator ETF, so it’s a you know, ETF of ETFs. He goes, I think they’re using your same approach. We knew who it was, we’d already done a deep dive. He goes, What do you think about it? And I said, you know, I kind of dug up the research we did on market neutral from the two dozen 11 paper and I said, you know, the problem there is just, you know, mathematically it’s going to be difficult for any linear approach to kinda disaggregate really the factors in the long side versus the factors on the short side. And so the net result, at least what we found is, you know, it ends up looking a lot like cash or cash plus kind of levered cash plus type instruments, you know, maybe maybe some short vol effect in there. And sure enough, at this point, you know, the ETF fact sheet was like 80%, TLT, big ol sh y, like all these all these cash ETFs. And I told him, I was like, you know, I think that’s probably what’s happening here. And, you know, that was a good example of an approach where we, you know, we found the top down statistical approach didn’t work wasn’t warranted, launching a product offer that was, you know, I would argue, kind of mediocre at best or certainly suboptimal. And so that part, you know, part of the issue there from the business side, is that contributed to the, what I would say, maybe, you know, the stigma against hedge fund replication was there was, you know, even if you weren’t a quant, you know, this guy, for example, was he’s a up and down Graham Dodd value guy didn’t know what r squared one was, but, you know, intuitively had an idea is like, you know, long should equity, I kind of understand that he’s like this, he’s like, it just feels like something’s off. And so, you know, from the business perspective, the rush to get product out there to kind of address these strategies were just simply, you know, wasn’t warranted, you know, didn’t really do the quote unquote, replication industry any good. And back to your your point. So market neutral was one for sure. market neutral stat ARB where it just the statistical approach anyway, it didn’t really hold any water. And I remember, I think in that paper that 2011 paper for, for market neutral, we ended up getting like, you know, 50 50% r squared. And so, you know, people people would always ask for some reason, that was the one they migrated to, they’re like, Well, what about this? Can you replicate this, and we turned down, you know, we turned down money, because we just said, you have to treat that as an artifact. You know, first of all, when you can, you know, statistically when, you know, when you dip down into, you know, as r squared goes down, kind of the confidence interval around it goes up, I was like, but then, you know, also you look under the hood, and it’s really just kind of levered cash or short vol as like, you’re better off actually going direct. And so market neutral is one. And that was from a statistical perspective. The other side, which is kind of interesting now, is there were strategies for which statistically, we could, we could get good inferences into, you know, what they were doing, but the factors weren’t necessarily tradable. So credit distress was kind of an obvious one. And that was that was another big focus for for people. Everybody wanted to hack the access to that space, so to speak. And you know, if you remember back, you know, at this point, 2012 ish, I don’t know how many smart beta ETFs were out there. But I’d be surprised if it was more than a couple 100 At most, you know, and a lot of them are different iterations of kind of, you know, the, the plain vanilla, but it was growing rapidly in fixed income ETFs, in general, were starting to take off. And in fact, you know, we were seeing that real time at Vanek, you know, they had gotten into that space a couple years prior, but you know, the Aum and those assets was really starting to take off. And I remember in particular, we had a conversation with an institutional investor who’s really interested in a credit clone portfolio. Ultimately, we turned it down because we just didn’t we didn’t have the confidence that we could transact we, you know, we could explain the risk factors, there were different kinds of bond indices, but we weren’t confident in the ability to access those risk factors in a liquid fashion. But I did point to an ETF that was relatively new that that time that was a Vanek ETF, it was Angel was ETF and it was a high a high income fallen angels, the risk factor, and it’s the idea is that if I think it was based off of a Bank of America index, but it was there was a lot of research in the in the fixed income space that bonds that were previously investment grade and had had been subsequently traded down into distress, had a more durable kind of high yield risk premia associated with them, it was basically just think of it as a kind of a better way to access the high yield risk premium. We did a deep dive, we loved it, we started, we really tried to make it work within the kind of the credit clone portfolio problem was there wasn’t enough other of those types of ETFs. But we we pointed this institutional investor kind of to that and said, you know, listen, this industry obviously, is, you know, more of these products are coming to market. And so the number, the number of strategies that we’ll be able to be clones by hedge funds, is only going to increase over time. That’s where the, you know, the smart beta landscape to, you know, to borrow a term from Andrew Lowe is really democratized the migration of alpha into beta, you know, different types of smart beta. And so, yeah, to answer your question, you know, kind of a, the big ones that didn’t work or market neutral stat or for statistical reasons, but then at that point to distressed and credit for more kind of market access reasons.

Corey Hoffstein  54:39

So smart beta is a nice, nice pivot in the evolution of smart bid is a nice pivot into where I wanted to head next, which was a paper that you wrote, actually in 2014. And this is something you referenced a little bit earlier in the conversation sort of danced around a little bit. I want to I want to touch right on it because there’s this common perception in the industry, at least a common phrase is just sort of wait for the crisis, that that’s when active managers are really going to shine. And it’s this sort of anti beta movement of, you can go passive, but you’re really going to regret that in the crisis. And you wrote a paper called, when does active management add value. And the conclusions of the paper really pretty much fly in the face of that conventional wisdom, that it really wasn’t a tremendous amount of value. In fact, perhaps even negative value being added during these crisis environments, particularly among the long short cohort that you were looking at? Can you talk a little bit about the paper and maybe sort of walk us through the main thrust of it?

Ben McMillan  55:39

Yep. And what I can, I’ll even set the stage, because you hit on a key point, which we started to observe right around that time, which was, you know, in 2008, launch equity funds in particular, but hedge funds in general clearly didn’t live up to their billing. And so investors start saying, wait a second, we’re tying up our money. We’re paying you 220. And, you know, yeah, you’re down 25%, which, you know, is better than the s&p. But, you know, come on, that’s not what we signed up for, you know, kind of collectively, again, what we saw was, you know, all the hedge fund managers were pointing and saying, Yeah, but look at 2000, you know, we did add value, you got to, you know, take a long term horizon, I think, I think at that point was when Buffett kind of made his bed with another fund to funds about the s&p is going to outperform hedge funds for the next 10 years. And so, you know, then 2011 happened. And that was where institutionally, you know, as you pointed out, at this point, the kind of the active to passive migration that started picking up steam, also, at least from where I was sitting kind of the, the idea of these these various behavioral theories for, or rather against active management, were starting to take hold, you know, obviously, you know, Kahneman had won the Nobel Prize years prior, but that was starting to come to the fore in the in the investing community, and 2011 happened. And like I said, you know, we were running a large amount, and I think, over $100 million in this clone portfolio, which outperformed by, you know, 800 basis points, we, you know, we did the deep dive immediately afterwards, because we asked the exact same questions you did is alright, you know, was this expected was, or did something come off the rails was, was this an unintended risk exposure that, you know, that, yes, worked to our friend, por favor this time, but could have easily gone against us. And so that was the kind of setting the table for the 2014 paper, because we were getting a lot of these questions from from both hedge fund managers, as well as institutional investors. And so all we did in the 2014 paper was, take that exact clone portfolio, you know, we didn’t want to go down the rabbit hole of trying to improve it, we literally just took the same model we were running, which at that point, you know, had several years of a Gibbs compliant track record, we generated the back test went back, I think, to late 90s. And we just plotted, you know, one of the things we found early on was, you know, over a market cycle, the R squared was was very high between the, in this case, the long short equity club portfolio and long term equity managers. So, the Alpha was was zero or close to zero, maybe it was a little bit negative, but the alpha over time vary greatly. And so that was what we wanted to drill into was, was why was the alpha, you know, negative in 2011? Why was it negative in 2008? And the first thing was to just quantify it, you know, what did it what did it look like, and so that’s what we found, you know, the first big insight of the paper was that, you know, the Alpha was kind of hovering around zero, we looked at the rolling 12 month, alpha of the active managers relative to the club portfolio, again, entirely relegated to launch equity, hedge funds, rolling, you know, rolling 12 month, Alpha was, you know, plus or minus, call it 1%. But then all of a sudden, you get 2008, you get 2011, where the active managers just fall out of bed. And, you know, we were, we were very quick to point out that, you know, yes, this is we’re looking at 10, to 15 years of monthly data points, it’s only, you know, we’re only looking at a sample size of kind of two events, we’ve observed this, but it’s, you know, it’s the best we’ve got it is it is still robust. And, and moreover, it also kind of, you know, anecdotally fits what we were seeing, which is back to that guy at the, you know, multibillion dollar launch of equity shop, talking about trying to handicap, whether or not Greece is going to leave the euro, that kind of frenetic what we classify in the paper as risk on risk off environment, where, you know, that the managers are getting outside of there, specifically, the launch of equity managers are getting outside of their comfort zone and trying to, you know, manage the net exposure and a timeframe that they’re just simply, you know, not not equipped to do and in the background, you know, all these kind of behavioral reasons. And again, I want to bring, I want to re emphasize the, you know, the kind of the business risk, and that’s something that, you know, the institutional sales guys would always talk about is, you know, in the back half of the year, if they’re down and they’re trailing, you know, whatever their benchmark is by a couple 100 basis points, they can come back from that if they just basically, you know, take exposure to zero sit in cash for the rest of the year, and kind of reset the following year. That’s not what investors are signing up for. They want kind of dedicated exposure to various risk premium in a thoughtful fashion and that’s that’s what the clone does while insulating from You know, the the ability to make these kind of intra month, you know, risk on risk off decisions. And so that was the point of the 2014 paper. And specifically, what we pointed out was, you know, the, the alpha of the active managers relative to the current portfolio is roughly zero or slightly negative over time, but it’s it’s very negative, or very pronounced in environments when the VIX spikes. So, again, 2011 being the perfect example. And so I remember I was on a panel at a conference in Phoenix, I think 2015, where we were talking about hedge fund replication. And the guy next to me was the guy at Credit Suisse who’s running their hedge fund replication product, which I think was an Etn at that time, and he and I were, you know, kind of friends in the industry, we traded notes, we’d grab coffee together and things like that. And there’s the afterwards there’s this guy, you know, he and I were kind of preaching the same thing, guys, look, these these Cohen portfolios work, but you’ve got to be thoughtful, you’ve got to do it the right way. There’s caveats with this, and that, and afterwards, it’s, you know, there’s this consultant in the in the audience that was just like, you know, not having any of it, he’s like this, either, we might as well been trying to pitch him like, you know, Voodoo, or something. He’s just like, and he was citing individual hedge funds, and it was a smaller consultant, you know, clearly, you know, had his focus list of for lunch with equity managers that, you know, always added alpha. And he just, you know, he just wasn’t having any of it. But that was a, I mentioned that, because that was kind of a good example, you know, back to your point about the at the very beginning to comment you made about, you know, this would irritate a lot of people in the hedge fund community. And I was like, Yes, I can absolutely attest to that. Because it was still largely an uphill battle, even in 2014 20 2015, to kind of convince people that there was real utility to these con portfolios, we were very clear that we’re not saying this is going to completely remove the need for active managers, but it’s at a minimum, there’s a lot of utility as a compliment.

Corey Hoffstein  1:01:50

I think of these clone portfolios a little bit as in replication, including in general is a little bit of a trivial and entirely non trivial problem. So I guess what I mean by that is the idea and the math behind it is pretty trivial. And you can get a pretty sophisticated model up and running pretty quickly. In light of our sort of earlier comments we made. I mean, I last summer, when when long shorts were struggling. rhodo commentary, I think, was called like attack of the clone because I’m a complete nerd. But I just spent an afternoon hacking together a CalMAN filter to track a long short index and got pretty close replication. Now, I had to use complete hindsight, you know, for example, like one of my factors was the NASDAQ simply because I knew in the late 90s of ton of long, short managers were way overweight tech, and that was my way of sort of hacking that in does not mean that that would work particularly well out of sample, right that I was sort of able to get this great replication in sample not so great out of sample. There are tons of replicating products out there today, available, obviously from the banks and available in ETF format, as we mentioned, as someone who has spent a tremendous amount of their career, building these types of products. What are some of the ways in which you would evaluate someone else’s replicator? What are the questions you would ask? What are the things you would look for? What are sort of the red flags and screens you would think about in determining whether you think a replicator is going to be successful going forward?

Ben McMillan  1:03:23

That’s an excellent question. Because it’s, you know, obviously, like you mentioned, the proliferation not only in the Smart beta landscape, but in the replicator landscape has has just gone exponential since 2012, or 2014. And, you know, the big ones for us is, is it economically defensible? So you know, back to that, the CIO at the family offices said, Hey, take a look at the market, neutral replicator, you know, already without even digging into the numbers, there’s a really good reason that in this case, at least, a statistical replication shouldn’t work that well, you know, same thing for, you know, 13 apps on high turnover, you know, stat ARB funds, you know, there’s, it’s lacking that fundamental premise. And I think that’s crucial. Not even in the replication landscape, but smart beta, just anything. I think you’ve written about this, there’s kind of a U curve of complexity and utility or inverted U curve, depending on how you orient the y axis where it’s if you think of is kind of similar to the Laffer curve, where, you know, at 0% taxes and 100% taxes, tax revenue is zero. Well, you know, if you swap out tax revenue and taxes for you know, sophisticated complexity, you know, at 0% complexity and you know, maximum complexity, complexity, I would argue the utility to the investor is zero, or minimal, let’s say and one of the one of the big problems with the rushed from active to passive is there, there has been a tendency, I think, to throw the baby out with the bathwater. And now the problem I’m seeing institutionally is, you know, not just institutionally but retail as well. You know, now people are avoiding any kind of sophistication. It’s it’s all black box. We were fighting a little bit of this in 2012 1314 15. And like I said, it did Help within the replicator subset that there were these kind of mediocre implementations that, you know, didn’t didn’t do the collective group any justice. But to answer your question directly, the first thing we would look for is just robustness. You know, did it pass the kind of common sense smell test? Did it, did it pass the statistical tests, this was one of the things that the Credit Suisse guy would kind of take notes on all the time, because there were, at least in the academic literature, there was, you know, there’s kind of a an arms race to get more sophisticated, and a lot of it was warranted, actually. So like, common filter is a perfect example. Like, we did some early works, it just has a lot of utility. Ultimately, it was a product design exercise, we said, you know, what the linear regression does close enough, it gets us 95% of the way there, there’s fewer moving parts. So it’s an easier sell, it’s probably a little bit more robust from a lack of assumptions perspective, particularly if we’re going to be running to an a million dollars. But that doesn’t mean that there’s not utility in that that additional complexity, you just, you know, you need to quantify it. And that’s something you know, that even back to your point about the NASDAQ, you know, I would argue, there’s, there’s an economic justification for having that in their current portfolio, even if there was no evidence retroactively. So I would look at that and say, All right, that’s, you know, that’s not data fitting, that’s there’s a strong economic justification for having, you know, a high beta sector, particularly because even anecdotally, back then, you know, basically everybody in late 90s, everybody, except for Warren Buffett was overweight tech. And so even though the dyed in the wool value, guys were migrating to tech, and you know, figuring out how to classify pets.com as a value stock and things like that, it can be justified, you know, if you if you had said, I got these great in sample results in the Swiss franc, you know, that’s where it doesn’t, it doesn’t pass, you know, the, you may pass this statistical test, but not the common sense test. And so that’s in terms of evaluating replicator products, and even just, you know, products and smart beta in general, I would say at the top of my list is just kind of the common sense, you know, before you even dig into the map and how it’s constructed. Just the common sense, is there an economic justification for why this should work? And that’s what you know, AQR talks about this research affiliates, everybody talks about this, but it’s, you know, it’s easier said than done. And it also it also highlights, I mean, you mentioned the bank platforms, which I think in many ways, kind of do good jobs at democratizing, at least for institutional clients, you know, democratizing access to factors in the same way that smart beta ETFs have done it for more retail clients. And the example I give and again, this is straight out of, I think, an Andrew Lowe paper, where he talks about merger ARB, you know, back in the 70s, there was, you know, only three guys doing merger arbitrage. Nobody knew what it was, and there was these huge spreads associated with it, you know, fast forward to 2009. And there’s a merger ARB, I think Credit Suisse actually has a merger ARB ETF for ETN that mechanically replicates access to that factor. And so there’s been a lot of good in that in what I’m going to call technology in general, and I’m putting the ETF Etn wrapper in that in terms of democratizing the the factor landscape. But the two big issues are, it makes complexity a lot easier to inadvertently incorporate unnecessarily and in an uncompensated way. And then the flip side of that is, you know, people have kind of put their fingers up and said, Nope, anything, even thing, even remotely smelling of quad is a black box, it’s a data mine, don’t want to touch it, I just want the cheapest ETFs. You know, it’s all about it’s all about cost. And that’s, you know, I don’t necessarily want to pivot too much. But I would argue, actually, at least from where I sit today, that’s the biggest battle we’re seeing is the act that the passive pendulum, in many ways has swung too far. And that it’s kind of oriented people to just seek the low cost provider at the expense of everything, even if it’s really thoughtful, you know, complexity, the right way, or sophistication, I’ll say,

Corey Hoffstein  1:08:44

well, it strikes me that there might be a bit of a balance. So one of the things I was thinking about when we initially started talking about replicators, was if I believe that you can give me a really good replicator for long, short beta, is there an opportunity for me to then look at the individual factors you’re using and change them? You know, for example, if there’s a factor in there that looks at value versus growth? Is there a way for me to say you want it my preferred value factor is enterprise value to EBIT? Ah, not price to book, or this is the way I wanted to find growth, or if you’re buying a certain type of basket of stocks to represent a sector, this is my preferred weighting methodology and almost take your cloned beta and then make it cloned Smart beta to a certain degree. Is that something you’ve looked into at

Ben McMillan  1:09:33

all? Absolutely. 100% and you hit you hit exactly on kind of the pivot of, you know, kind of not only what we’re focused on now, but that we started seeing with this first big institutional investor, and it was simply that, you know, back in 2011, there just weren’t as many smart beta ETFs you know, there’s there was, you know, maybe 10 or a dozen value ETFs and you know, now there’s, there’s several dozen and they’ve there’s a lot of different flavors, I think I think actually two sigma had a good research piece where they evaluated different specifications of value different weighting schemes and just simply showed the disparity in any given year determined, depending on how you identified value. And so one of the early conversations we had with this big institutional investor is, you know, we were using a very liquid off the shelf value ETF. And we said, Listen, there’s, you know, this, this, this because it’s liquid, and it’s cheap and transparent, we have confidence, especially at size, you know, the last thing we wanted to run into was, you know, trying to become the biggest weight and a smaller value ETF where we had to go to the the APS and try and get creates and things like that. So we opted for liquidity above all else, but we said to them, we could synthetically replicate a more sophisticated or a different specification of value, depending on your guys’s preference, or, you know, our thoughts. And ultimately, we didn’t end up going that route. But it’s I actually think it’s the highest and best use of that implementation is exactly as you’ve identified, there’s kind of two layers to it, the, you know, the replication methodologies is, you know, really a good kind of default asset allocation scheme, let’s say, and then depending on, you know, investor preferences, you can manufacture better or smarter exposures than is available on the off the shelf, you know, smart beta ETF landscape to kind of have, as you said, kind of a smarter, better, clean portfolio.

Corey Hoffstein  1:11:26

I know that’s something you have really thrown yourself fully into this idea of building your own indices indexing as a service, you launched your own firm IDX insights in 2017. Can you expand on the idea a little why it fits the market today? What type of clients you think this sort of custom indexing is useful for? And how it works?

Ben McMillan  1:11:49

Yep. So the Genesis is exactly, you know, the point that you made is, you know, we we started identifying, you know, back then in the early days that there was, you know, there’s a lot more that can be done in terms of indexing or creating exposures that weren’t available in ETF landscape. Part of the segue there, too, as I started to appreciate having been inside Vanek, you know, which is first and foremost, an ETF shop, that got me an access to a lot of other ETF providers, and, you know, provided a really good look under the hood of just the business of ETFs. In general. And it is difficult, you know, if a client if an institutional or retail client wants a specific tilt on value, or a specific take on value, you know, if there’s not an ETF that addresses that, then they’re out of luck, unless they have the ability to kind of do this direct indexing or this indexing as a service. Now, fast forward technology, again, has really helped kind of disintermediate debt. So what I mean by that is, so that was kind of the the idea for IDX. And it started really, as a consulting entity, and then we morphed into, you know, more of a full blown kind of solution focused on this custom indexing, indexing as a service where, you know, we’ll go to clients and you know, exactly the same fashion, as you said, and said, Look, here’s, you know, don’t think of an index as kind of an off the shelf, take it or leave it option, which is currently what you have with ETF landscape, we like to think of it as a menu of options. So if you think about value, there’s, there’s multiple ways you can define value, you can look at one signal, you can look at multiple signals, you can weight them by value, you can weight them by market cap, you can weight them equally, you can increase the number of definitions of value, both in terms of quantity, as well as sophisticated sophistication, you know, you don’t necessarily have to go full tilt, you know, Piotroski F score, but you can, there’s somewhere in between. And one of the early engagements I had was with actually a company that was looking at was a insurance company, and they had an asset management arm where they had a very specific definition, or they had a very specific exposure they wanted, it was kind of a blend of European dividend value focus. And these were entirely bottoms up stock pickers. And so they came to me with, you know, like a 300 megabyte Excel file with like, 36 different rules or screens for value, and they said, We want you to turn this into an index. Now we’re back and I was like, you know, guys, it doesn’t really work like that, like, first of all, you know, the value we provide isn’t just, you know, outsource quantitative services, it’s to help you measure in justify that, you know, the complexity, you know, help you build the index, even if you’re coming to us with the recipe, so to speak, we’re helping you, you know, build it with the highest quality ingredients in the most robust fashion, you know, I want you to be able to cook the same dish, with maximum quality all the time, under any environment, we must have gone back and forth over months, starting with the simplest version of their rules, and then layering, you know, basically creating dozens of indices that would incorporate one more degree of quality or sorry, one more degree of complexity, and kind of take it to them and say, decreasing returns to complexity, not surprisingly, and to your point about the you know, kind of the the Laffer curve of complexity, you know, the goal isn’t to obviously, create the best back test. It’s all you know, it’s also not to create the most complex For the simplest product, it’s to maximize the kind of the Sharpe ratio of the complexity or of the degrees of freedom. And so that’s where to, you know, to answer your question directly, that’s where I think, kind of, you know, custom indexing, indexing as a service, as we’re calling it is, is really poised to do well in is in this kind of subset of investor that’s big enough, you know, not necessarily, you know, huge institutions, but you know, even just bigger, more sophisticated RAs, that have the ability to consume signals directly, they don’t necessarily need a 40 act product to invest, they have the sophistication to understand, you know, why, you know, book to value is different than price earnings, they understand the idea of, of, you know, the trade off between complexity and degrees of freedom and robustness. And the last point is, again, technologies really made it easy to kind of deliver these these custom indices to this group, because they themselves, in many cases, kind of have their own trade desks, or even, they don’t need prime brokers anymore, like we did, you know, 2010 for the big institutional clients, you know, even, you know, even the Charles Schwab, the IB is, you know, the trading platforms that custodians have really gotten good at allowing these guys to take in signals as an example. And, and, you know, incorporate them directly to their model portfolios, run them run accounts pair pursue. And so that’s kind of we had an article a while back, right, I talked about kind of the technology disintermediating now, what I believe to be the ETF landscape, and as this picks up steam ETFs themselves, they’re never gonna go away, but but they’re going to become increasingly obsolete for a certain type of investor who’s who, you know, understands that they want something that’s not off the shelf, and has the ability to take signals and turn it into product.

Corey Hoffstein  1:16:41

I really liked this idea of Sharpe ratio plotted against degrees of freedom. Talk to me about how you think about quantifying the degrees of freedom of of a strategy, like when you’re looking at building, say, a value index, or a quality index, what do you what constitutes a degree of freedom to you?

Ben McMillan  1:17:00

So at this? I mean, it’s an excellent question. And I guess, in many ways, it kind of depends on the strategy, I’ll give you two examples. But at the simplest level, there’s kind of two ways I think about it, you know, at the simplest level, it’s if there’s another rule, if you’re adding a measure, or you’re adding a rule, if there’s another if then or another screen going into place, you know, we think about that as a degree of freedom. So if if somebody wants to move away from just looking at, you know, price to book as the as a single measure of value and incorporate something else, like, you know, price earnings or price to sales, probably the inverse, that represents a degree of freedom. Now, I would argue that that’s a well spent degree of freedom, because you’re, you’re, you’re effectively buying robustness. And this is something you’ve talked about, and really done a good job illustrating is, you know, kind of the fragility of of single, single signals. And actually, a good example is when I was at Vanek, we were evaluating an external index provider that had a long flat commodity index. And so it was if I remember correctly, it was a signal sick one look back, it was a Moving Average crossover strategy, where they were looking at the five day moving average by I believe the 250 moving average, applied to the the 20 most liquid futures products. Very simple, very robust, you know, the evidence in favor of you know, so, Yan Vanek asked me to take a look at it. And I was like, you know, moment, you know, AQR has written at nauseam about the utility of momentum, particularly for a volatile asset class like commodities, Mark Malik, again, I was like, it makes a lot of sense. However, at a minimum, we’d want to understand the risk isn’t necessarily yes, you’re diversifying across instrument? And and yes, you’re setting up a rule that’s effectively designed to kind of truncate that left tail risk. But how do we know that you know, five by by 250 is kind of the magic number. And this isn’t an optimization exercise. But I want to make sure I didn’t inadvertently pick the best iteration of that. And so what I did was go through and evaluated the Sharpe ratio, I created a 3d plot really pretty heatmap, where Sharpe ratio was on the z axis. And then the fast Moving Average was on 1x, one Y axis, and the slow Moving Average was on the other. So you had every combination of zero, you know, or call it five to 50 days on the quote unquote, fast Moving Average. And I think we looked at 50 to 250, on the slow Moving Average. And what we plotted was the Sharpe ratio. And there’s a million different ways you can look at this, you know, on the z axis, we kind of swapped out, you know, a bunch of different metrics that we are interested in. But Sharpe ratio is a good example. We we plotted this topology, and so it was a really nice heat map. And what you saw, you know, what I kind of, you know, wrote in for the internal memo is, again, the objective, you know, if we were to run a data mining exercise, we would want to pick the, the exact, tippy top of the global maximum of this topology. We’re not in that game. What we do want to do is we want to quantify the degree that there’s potentially signal risk of only relying on the There’s one little point on this on this Cartesian plane of, you know, five day moving average by 200, day moving average. And, you know, it happened to be a little bit lower on the on the plane. So, arguably, you could do better with just by simply moving to different specifications. But at a minimum, the point I made was, you probably want to diversify, you probably want multi know, what we’re seeing is that the irrespective of what combination you pick, it outperforms simple buy and hold, you know, not just on on, you know, absolute returns, but you know, risk adjusted downside risk all of that. So why not diversify away from that single, you know, that that single signal and again, so that’s, I think that’s a really good illustration, and I always use as an example for for clients and prospective clients, when we go through this exercise of testing all these different iterations, because it really even for non non quantitatively oriented people, they kind of instantly get, and I just did this exercise for bigger ra rags showed a redacted version of this memo, and they kind of instantly get Okay, I got it, I don’t necessarily, you know, I don’t necessarily want to try and pick the global maximum, but I also don’t want to have my only exposure be this one signal. And so to answer your question, you know, a little bit more directly, it’s those kinds of exercises where we go through and like I said, you know, with this, this other client that had the very complex idea about European dividend value stocks, where we, we, you know, anytime there’s a new rule or, or new screen or new something in the, in the recipe, so to speak, you know, we just want to test kind of every iteration, that’s reasonable to understand, what is the benefit from that, that additional rule that additional, you know, loss of a degree of freedom? And then also, what is the risk? Did we inadvertently pick, you know, a local maximum, or did we inadvertently pick, you know, the one spot that performs well, but if you shifted even a little bit, you know, everything goes off the rails. And so I would say, you know, ironically, or maybe not, but in the in the kind of custom indexing or indexing as a service area, I would argue that exercise is probably really the value add, it’s not that we’ve got the the secret sauce for defining value, it’s that our value prop is set up to give the right kind of investor kind of the most comprehensive view of the exposure they’re getting, and then ultimately build it and deliver it for them.

Corey Hoffstein  1:22:15

So this idea is near and dear to my heart, because it’s been a business line of mine at New founds, for the last decade, that I didn’t have as good a name as indexing as a service. But it’s definitely something I’ve dealt with. And I wanted to get your thoughts on how you think about dealing with the balance of the clients wants with the client, how the client thinks about building the portfolio, their philosophies and the portfolio, how you manage that with the evidence that you see in a given research strategy and trying to nudge them perhaps towards a more robust implementation. My experience has been, even once you get it live, there’s always this sort of tinkerers dilemma, which is the client keeps coming back and asking you to explore, you know, these minor changes that you might know, don’t have a big impact, but you need to sort of run the research just for their, for their benefit to educate them. And at the end of the day, what makes it so interesting is you’re not just a calculating agent for this index, right? You’re not just running it on an ongoing basis, you’re not just administrating it, you’re you’re helping build it. So how do you really balance that it’s not just what the client wants, it’s making sure that you feel like the index is built with integrity as well,

Ben McMillan  1:23:28

you have hit on exactly exactly the source of I’m not going to say our biggest dilemma, but it is our biggest kind of time sucks also not the right word. But you know, it, it’s the biggest kind of thrust of this business is managing the expectations, even managing their own their own risk tolerance. I mean, one thing we’ve started toying around with, and a very simple, you’ve read about this extensively, and I think it adds a lot of merit to this process is kind of asking questions at the forefront of the investor, even simple questions, like, you know, not the, you know, not the kind of condom on, you know, would you if I give you, you know, two from coin flips, and one had a, you know, 78% chance, and that ended up but even more simple than that, just saying, you know, for what your, for this index, would you, and you got to be careful here, I’m going to caveat it, you know, you obviously, you don’t want to lead them into a data mining exercise, but it’s been helpful for us to help understand how they think about their own objective function, because humans are, you know, are phenomenal liars to themselves. So, you know, we’ll ask the question, you know, 2008 versus 2013 As an example, you know, 2013, the s&p was way up 2008 The s&p was way down, what would be your preferences for, you know, whatever a tactical sector product in those years because, you know, there’s a lot of different implementations, you know, all of which are robust and built with integrity, but they have different risk or you know, they have maybe this maybe they mean at the same time Sharpe ratio, but they have different characteristics from year to year and like it or not, you know, that matters for a lot of these guys. And so, you know, Would you be okay being, you know, flat in 2008, you know, arguably with 30% Alpha, but only being up 1% in 2013, when the SP was ripping, and almost invariably what we find anyways, you know, people are at least the folks we’re talking to they’re, they’re much more in index hugging mode, you know, I say index, broad based index hugging mode, and they would like to admit, or probably even realize, and so a lot of it, I don’t have a good answer for you, but a lot of it is just kind of poking at the trend trying to triangulate in on, you know, to what degree are they susceptible to tinkering. And then, you know, building the product that they say they want, but also really forcing them to come to terms with it, if that’s, you know, really what they want. Because, you know, everybody’s a CAD, you know, I’d love to perform in 2018, it’s like, alright, well, what if what if the option was, and again, this is, this is a real exercise, we happen to have, you know, to two indices, that kind of traffic in the same asset class, they’re, they’re constructed a little bit differently. And so, you know, one of them is going to be down, you know, call it 15, one of them would have been down and call it 2,015%, in 2008. But it also would have been up more than 20%, in 2013, when the s&p was up, you know, 30%. And so that, that’s very different, even if it has the same alpha over a market cycle cycle as the, you know, the one that was flat and both Oh, wait, and third 2013, we’ve found, like I said, you’ve you’ve written about this a lot, I think it’s a really good way to frame the conversation with clients, and very, hopefully, very quickly suss out, you know, what, what risks do they really want, because part of, you know, a large risk for them is, is, you know, the tracking error risk, whether or not they admit it, you know, a lot of people say they their absolute return focus, but you know, at the end of the day, relative performance matters a lot more, and they can justify being down less than 2008, eight, you know, a lot easier than they can justify missing out in 2013. But it’s tough. I mean, it’s the tinkerers dilemma, as you said, is, is huge, in way, candidly, way more prevalent than I would have thought,

Corey Hoffstein  1:27:06

I want to bring this conversation full circle, we spent the entire beginning of the conversation talking about hedge fund replication, and sort of these liquid alternative beta products, we discussed how active managers often add value at the complete wrong time. And now we’re sort of shifting into a conversation of becoming systematic, active managers. Why is, you know, this idea of indexing as a service necessarily different and not going to fall into the same issues that the clone portfolios were meant to fix in the first place? How do you sort of differentiate these two and say, this is okay, where we think the more other active managers might have issues, we still think indexing as a service is not going to have the issues that the clone portfolios were meant to fix?

Ben McMillan  1:27:55

Yeah, I mean, I think that’s an excellent question, because it is you want to make sure, you know, we don’t fall prey to the, you know, do as I say, not as I do kind of syndrome. And I think I think it boils down to the fact that, you know, we’ve we’ve identified, and I’m not just saying, you know, you know, we locally, but you know, kind of the industry collectively has identified a lot of issues with active management with the LP structure, in particular, even ETFs. We haven’t touched on this, but it’s, you know, relatively sort of, you know, obvious, you know, that the the massive passive flows into ETFs, you know, are ultimately creating their own uncompensated risk factor, potentially, you know, most obviously, in the in the fixed income landscape, but there’s, he’s kind of known, the uncompensated risk factors, be a business be the, you know, kind of exogenous non market risks that are prevalent in the Clone strategies prevalent in any active strategy. And I think all we can do is just kind of be thoughtful about mitigating those risks. And I, I do think kind of the, you know, the really the business is you work collectively in, you know, there’s not, there’s not a lot of folks doing this, but but it is, I think, just a thoughtful next step in terms of saying, Okay, here’s, here’s what previous iterations got wrong, you know, with active managers, you know, with any active kind of discretionary, non systematic process, you’re going to be importing a lot of these behavioral errors, which, again, at this point, are a lot more obvious and a lot more accepted than they were 10, or certainly 20 years ago, you’ve got the kind of vehicle issues, you know, certainly with LPs, but also increasingly with ETFs not only from a lack of customer customization standpoint, but also just from, you know, the the ETF wrapper becoming its own potential choke point. And so, you know, direct indexing, or indexing as a service is just saying, you know, look, we’re not we’re not trying to do something totally revolutionary, but we are in many ways, just trying to kind of replicate what some of the bank platforms have, have already successfully done for their, you know, they’re big institutional clients, and, you know, deliver it in a in a thoughtful, robust fashion to a broader audience. And I think ultimately, you hit on it, I think it is, I think kind of the prop probably what I would argue is one of the defining characteristics of this business, you know, indexing as a service is more of the bespoke coaching, not just saying, alright, we can create anything you want. But, you know, our job is to kind of present you the menu of options and help you choose the, you know, the robust one that fits your risk preferences. And you know, after having forced you to kind of think through your risk preferences. So it is I think it is a subtle, but an important distinction.

Corey Hoffstein  1:30:25

Last question for you. Totally unrelated to anything we’ve been asking, but it’s the last question of the season for everyone. And the question is this, if you had to liquidate all of your investments today, and only invest in one thing for the rest of your life, now that can be an asset class, it can be an investment strategy can be really whatever you want, but you can only invest in one thing, what would it be? And why?

Ben McMillan  1:30:51

That’s an excellent question. And it’s, you know, it’s one I’ve thought about, and I’ve used kind of as an illustrative tool when I, you know, talk to certain folks and say, Look, this, you know, this is kind of a set and forget it, if I had to, you know, put everything into one thing for the next 100 years, this would be it. And, you know, in many ways, I kind of, I’m gonna, I’m gonna copy what Ray Dalio said, and one of his early research reports about the idea of, you know, his risk parity, you know, setting aside leverage for a second, there’s obvious issues with that, but, you know, it’s, it’s difficult really to improve upon, you know, a multi asset class approach, and I would even say, ETFs are fine, you know, if it’s, if it’s long enough horizon, you can, I’d be less concerned about the, you know, kind of the wrapper risks, but it would be, you know, multi asset class, I would have, you know, some kind of, I wouldn’t try and get overly cute with the allocation among asset classes, I, you know, just probably do it on a very simple risk weighted basis, maybe have a momentum filter, because the again, as we all know, as you’ve heard about that, you know, the math of just simply avoiding large losses, you know, really does compound over time. And if it’s truly set in, forget it, you’d want some kind of systematic risk off lever embedded in there, you know, and I don’t know that it would be much more complex than that. So you can kind of think of it as a risk parity approach, you know, unlevered, across, you know, any asset class I could reasonably get access to, with, you know, a momentum filter, and to go to cash, let’s say, and, and, you know, true, prolonged risk off environments, you know, oh, eight, early 70s, you know, 2000 type scenarios, I don’t, I don’t know that I would get overly clever. And then of course, there’s the obligatory 50 basis point allocation to crypto, but that’s a whole nother. There’s your upside. Yeah, exactly.

Corey Hoffstein  1:32:32

Well, then, this has been a lot of fun. I hope we, in this conversation, have ruffled some feathers, I think your research is really, really fascinating. I definitely recommend anyone listening to go and find the papers, and we’ll link to them in the show notes. But if people do want to find more of you more, your writing more your research, get in touch with you, what’s the best way for them to find you.

Ben McMillan  1:32:51

So, you know, our websites, probably the best right now, just IDX insights.com. We’ve got a ton of research up there. And, you know, we’re pretty proactive in engaging with folks, you know, just even if it’s not directly related to us, kind of, you know, doing whatever we can to help so, this is this has been phenomenal. By the way. I’m gonna say it’s been a long, long time follower of your blog and your writing. So I’m thrilled that we finally get to do this. So thanks for having me.