How This Year's Research Influences Next Years Startup Trends

Which recent developments in AI research are most likely to influence startups in 2024?

Last week, the leading researchers in the world of Artificial Intelligence convened in New Orleans for the largest AI conference of the year: NeurIPS. While the conference is geared towards academic research, it also provides a peek into what categories the leading AI researchers are most excited about for next year.

While exciting research trends don’t necessarily turn into good businesses, they are always a helpful datapoint, particularly in an industry like AI which is still very much dependent on research breakthroughs to drive business progress.

At this year’s conference LLM and generative (e.g., diffusion-based) companies continued to occupy the majority of mindshare. However, some of the new categories that attracted the most attention from researchers included: generative video / generative 3D assets, AI in the physical sciences (especially biology), AI agents, alternatives to the Transformers architecture, and most notably, the continued debate between “open vs. closed” AI.

Based on what we saw at NeurIPS, I’ve pulled together a list of five categories / themes that I think are poised to break out in 2024:

Generative videos and generative 3D assets

Over the past year, a massive amount of 2D content has been generated by the likes of Stable Diffusion, Dall-E 2, Midjourney, and a slew of other diffusion models. To-date, diffusion models have struggled to generate 3D assets or videos, particularly any that can show movement without distorting the objects themselves.

As massive amounts of research talent has poured into the generative 3D and video spaces, I’m expecting to see better and better outputs hit the market sometime next year, ultimately driven by a movement away from diffusion models. As better models come online, I’d expect commercial traction to quickly follow suit: 3D and video content capture human interest much moreso than images alone, and as such are likely going to be easier to monetize (e.g., for animation, advertisements, social media content, etc.)

3D generative “worlds” / a revitalization of the metaverse

Beyond just 3D object development, there is increasing research excitement around the development of entire generative worlds (whether 1:1 digital twins of the earth, or net-new generated worlds). In practice, these will most likely resemble simulations on totally unprecedented scale (cue the simulation hypothesis returning to the public forum).

While this sounds more and more like a rebranding of the metaverse, the recent breakthroughs in AI technology should make 3D worlds much more compelling than ever before. I’d expect the earliest use cases to emerge in the Media and Gaming segments first, but the technology could eventually proliferate across almost everything that we do. A crappy recreation of the real world was never a compelling metaverse narrative, but being able to generate your own net-new worlds on the fly could be the “iPhone moment” VR / AR has been waiting for.

Synthetic data for new modalities

As LLMs have proliferated over the last year, much of the progress in language AI has been derived from the availability of the massive amount of free training data available on the internet. As more data modalities emerge (e.g., 3D images, video, audio), the reality is that there isn’t nearly enough free data on the internet to train large foundation models on other types of data.

Data has long been identified as the key input to modern AI. We got lucky with all the free text on the internet, but we haven’t been as lucky with other types of datasets. This year at NeurIPS, there were several interesting early projects starting to emerge around multimodal data generation, but everything still feels very early. Synthetic data has long struggled to really hit it’s stride as a category in the past, but I’m expecting several interesting companies to emerge to attempt to solve what is going to very quickly become the core bottleneck of modern AI.

AI agents

While the term “agent” became a buzzword in the world of AI in 2023, it’s hard to nail down a specific definition of what it means. Broadly speaking, I think of AI Agents as models that will actually go out and execute on tasks for you, without needing human intervention / humans in the loop (e.g., a real life Jarvis or Cortana). Unfortunately, most “agent” products in market today still don’t actually work, particularly in enterprise settings.

This year at NeurIPS, there was more research dedicated to solving these problems than ever before. Specifically, more and more credible research has begun to emerge around teaching Agents to use software tools, to interact with external API’s, to read web pages, and more. Truly automating human workflows is an incredibly hard problem, but represents one of the largest opportunities in all of AI. I’m expecting massive improvements in agent quality in 2024, with some very verticalized offerings to start to clear the “Enterprise Readiness” bar on discrete tasks for the first time.

Open source AI

If nothing else, the “open vs. closed” debate in AI was the single most talked about point at NeurIPS this year. While I think having to “pick a side” is really a false dichotomy (it’s not clear to me that “open source” and “safe AI” are correlated in any way), there’s really no doubt that Open Source is playing a bigger and bigger role in AI than ever before.

While I still expect the closed source model providers to stay far ahead of open source alternatives in 2024, I think the open source world has lots of opportunity when it comes to developing datasets, frameworks, tooling, and more. The excitement around the open source AI community has captivated AI researchers globally, and their contributions will likely drive down the barriers of how hard it is to build with AI in coming years.

As the technical bar to build with AI comes further and further down and more and more people are able to leverage it, we expect to see massive growth in the ways that AI solutions proliferate our every day lives. If nothing else, there’s never been a more exciting time than to be building in AI.