6 min read

Can Numerai Succeed Where Others Failed? A Look at Crowdsourced AI in Quant Trading

Published September 26, 2024 1:57 PM

How will AI trading platforms fare in an environment that faces increasing regulation ? | Credit: Hameem Sarwar / Find.co.

Key Takeaways

In 2020, the first hedge fund to crowdsource algorithm development, Quantopian, bowed out.
However, Numerai and Quantiacs continue to operate a similar model.
Can they succeed where Quantopian failed?

In 2020, the demise of Quantopian, which was the first-ever hedge fund to crowdsource the development of its trading algorithms, marked a major symbolic defeat for the idea it embodied.

However, a new generation of crowdsourced quant platforms like Numerai have risen from the ashes of their predecessor. But can they succeed where Quantopian failed?

The Rise and Fall of Quantopian

Launched in 2011 by John Fawcett, Quantopian aimed to democratize algorithmic trading by providing retail investors, data scientists, and developers access to data and tools typically reserved for institutional traders.

The idea was simple. Quantopian provided historical trading data for free so that users could build and backtest models to determine how well they would have performed during different periods and market conditions.

The company then licensed the best-performing algorithms for use by its hedge fund via a revenue-sharing model, where users who created successful models could earn a percentage of the profits.

By 2016, however, it was increasingly clear that Quantopian was struggling to make money.

As the company’s researchers acknowledged at the time, commonly used backtest evaluation metrics, “offer little value in predicting out-of-sample performance”. In other words, models that looked successful when applied to test data failed to deliver in a live market environment.

The IP Issue

Part of the problem Quantopian faced is inherent to the crowdsourced quant paradigm.

To encourage participation, platforms only license model outputs while users retain intellectual property ownership. But this means investors can’t evaluate or interpret algorithms’ underlying logic.

Up against heavyweight quantitative trading outfits like Two Sigma and D.E. Shaw, which built and deployed their own models, Quantopian’s user-generated algorithms simply couldn’t compete.

Nevertheless, other decentralized data science initiatives have filled the gap.

Today, developers can license their models through CrunchDAO which acts as a middleman connecting predictive AI buyers and sellers. Meanwhile, Numerai and Quantiacs continue to carry the torch for crowd-sourced hedge funds.

Does Numerai Differ From Quantopian?

Numerai’s innovation lies in its meta-model, which aggregates predictions from a decentralized network of data scientists.

By combining different models and inputs, Numerai’s algorithms make investment decisions that are less prone to error. It also means the hedge fund owns its underlying model, even though individual contributors retain their intellectual property.

Users are still rewarded based on the quality of their predictions. But because the hedge fund has greater insight into the combined output of multiple predictions, the black box problem that plagued Quantopian hasn’t been such an issue (at least, not yet).

Another distinguishing feature of Numerai is that it rewards contributors in Numeraire (NMR), a cryptocurrency designed to solve one of the fundamental problems with crowdsourcing in general–that such systems are vulnerable to Sybil-type manipulation.

As Numerai founder Richard Craib explained in 2018, “You might have someone make a thousand accounts, try to get lucky, and not be submitting the model they really believe in.”

Under the tokenomic model, however, contributors must stake NMR when they submit their models, ensuring they have no incentive to enter sub-optimal algorithms.

Quantiacs

Another approach to crowdsourced algorithmic trading is taken by Quantiacs. Like Numerai, Quantiacs runs tournaments to identify the most accurate predictive models.

Entrants can each submit up to fifty models, and the results are evaluated during a live contest. Quantiacs then allocates $2 million to the top seven models from each contest period. Once each algorithm is deployed, its developers are paid 10% of any profits generated during the period it runs.

Expanding to New Markets

One sign of the continued appeal of crowd-sourced quant platforms is their steady expansion into new markets.

Quantiacs, for example, started out focusing exclusively on the futures market, but today, it runs five distinct tournaments covering futures, stocks, and crypto.

In a similar development, Numerai Crypto launched as a parallel tournament for predicting movements in the crypto market earlier this year.

Unlike the hedge fund’s data-driven investment strategies, Numerai Crypto makes the meta-model’s predictions publicly available, while contributors can still stake NMR to earn rewards.

Numerai said it wouldn’t use the predictions generated to trade, although it added that “observing it will be fascinating.”

Hedge Fund Performance

The success or failure of crowdsourced approaches to quantitative trading can only be measured through financial performance.

Quantopian ultimately bowed out because the hedge fund was making a loss. But are today’s alternatives profitable?

Unfortunately, assessing their exact performance is difficult because neither fund has published the necessary data.

Concerningly, Numerai stopped publicly reporting its hedge fund performance in 2023. Prior to this, the fund had occasionally shared performance updates with its community of data scientists, but those updates are no longer readily available on the website or other public channels.

In an interview , Craib said the reason Numerai doesn’t disclose the information is due to regulatory restrictions on marketing hedge funds.

The initiative has not shown any outward signs of stress, however, posts on the Numerai forum raised concerns that the fund “suffered serious losses” in October 2023.

Judging Quantiacs’s success is a little easier as the firm has a leaderboard tracking the returns generated by top algorithms. Based on this information, the crowdsourced strategy has been delivering positive returns so far. For example, among models entered in the current tournament ending Oct. 31, those that were allocated capital are all in the green, with the highest allocation generating returns of 3.90% to date.

A New Approach?

For any investment vehicle, strategies that work one day may fail the next, and adapting is necessary for long-term success.

While Numerai Crypto introduced a whole new asset class, the upcoming V5 update, “Atlas”, expands the universe of stocks included in the dataset. It also restructures the data available to participants in the tournament.

A post on the Numerai forum said that most models will become more accurate once they are retrained on the new data. However, it’s still too early to tell how Numerai or other AI trading platforms will fare in an environment that is still so nascent but facing increasing regulation.

Was this Article helpful? Yes No