Abstract: 

Large Language Models (LLMs) are notoriously very powerful at generating content following instructions. This means that, with the right prompts, they can be used to answer questions and solve tasks for us. However, for more complex tasks, they need to be integrated into more sophisticated systems that allow them to 'reason' about a problem, plan next steps and use external tools. These systems are referred to as LLM agents.

Although potentially very powerful, LLM agents have three main problems: higher latency, reliability and security, as they can often result in unpredictable interaction loops leading to unexpected outcomes and loss of control over the system. This is particularly worrying when, as in our case, the LLM is given the power to interrogate internal company data. But there is another way. Instead of giving the LLM full autonomy, we can require the agent to solve a problem by following a set of pre-defined tasks, with simpler and more restricted interactions. In this way, we can do a trade-off between generalisability and control.

In this talk, I will share my experience of building and deploying a simple and constrained LLM-driven agent in production for a UK fintech company. More specifically, we will:

- Explain the problem we are trying to solve and understand why we can use an LLM agent to solve it.
- Understand the core logic of our LLM agent, highlighting the choices made to allow multiple methods to query the company's internal data in a controlled way.
- Finally, the benefits and limitations of this approach will be discussed.

Session Outline:

- What an LLM agent is
- What building simple and constrained agents mean
- When they are preferred over complex and unconstrained agents

Bio: 

Dario is a data scientist with a Ph.D. in physics and over 8 years of industry experience in data science and machine learning. He specialises in end-to-end data science processes, including the development, deployment, and integration of machine learning applications into business operations. Previously, he led a team of data scientists at Moltiply, a prominent provider of credit-related outsourcing services in Italy. Currently, he serves as a Lead Data Scientist at Faculty, a leading provider of applied AI solutions based in the UK, where he oversees large-scale machine learning and LLM projects in the Consumer and Services business unit.