All Models Great and Small: It’s 2024, You Don’t Have to Use GPT-4 for Everything Anymore


Last year, enterprises and hobbyists alike experimented with the most recent easy-to-use, broadly-available, general-purpose foundation models to solve any task at hand. In our experience, 99% of those enterprises and hobbyists were (and are) experimenting with GPT-3.5 and GPT-4. This year, as we move from Notebook demo GenAI-based applications into full production applications, those same users are realizing that this stuff is expensive, it’s slow, and that smaller, open-source, fine-tuned models can compete with or entirely outperform the bigger models. In 2023, you didn’t need to prove positive ROI on a GenAI application; in 2024, you absolutely do. It’s time to think about model selection.

In this talk, we cover the pros, the cons, and the open questions around selecting a bleeding edge foundation model over one of many task-specific, smaller models. We’ll cover the cloud versus edge discussion that is happening across all enterprises, touching on latency vs accuracy tradeoffs and then diving deeper into how that intersects with fine-tuning task--specific models for your particular use case. A spoiler: we’ll conclude with a recommendation that sometimes the expensive OpenAI/Anthropic solution is worth it, but that (a personal prediction!) the Mistral, Nomic, Hugging Face, AI2, etc offering will rear its head to an astounding degree in the coming year.


John Dickerson is Arthur's co-founder and Chief Scientist as well as an Associate Professor of Computer Science at the University of Maryland. He works at the intersection of machine learning and economics, with a focus on designing incentives that promote ""good"" participation in complex systems.

His research centers on solving practical economic problems using techniques from computer science, stochastic optimization, and ML. He received his PhD in computer science from Carnegie Mellon University (SCS CSD PhD '16). Learn more about John here.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google