Abstract: Reinforcement learning is a machine learning approach concerned with solving dynamic optimization problems in an almost model-free way by maximizing a reward function in state and action spaces. This property makes it an exciting area of research for financial problems. This session demonstrates the application of reinforcement learning to create a financial model-free solution to the asset allocation problem, learning to solve the problem using time series and deep neural networks. We demonstrate this on daily data for the top 24 stocks in the US equities universe with daily rebalancing. We use a deep reinforcement model on US stocks using different deep learning architectures. We use Long Short Term Memory networks, Convolutional Neural Networks, and Recurrent Neural Networks and compare them with more traditional portfolio management approaches. The Deep Reinforcement Learning approach shows better results than traditional approaches using a simple reward function and only being given the time series of stocks.
Session Outline:
Lesson 1: Asset Allocation
Asset allocation, where the goal is to obtain the weights of the assets that maximize the rewards in a given state of the market considering risk and transaction costs, is a problem easily framed using a reinforcement learning framework. Investors and financial researchers have been working with approaches like mean-variance optimization, minimum variance, risk parity, equally weighted, and several other methods backed by machine learning. We demonstrate the history and application of these techniques.
Lesson 2: Reinforcement Learning
Reinforcement learning is a machine learning approach concerned with solving dynamic optimization problems in an almost model-free way by maximizing a reward function in state and action spaces. This property makes it an exciting area of research for financial problems. We discuss RL infrastructure, types of RL, types of networks and design structure of the problem of applying RL for asset allocation.
Lesson 3: Example: RL for Asset Allocation
We apple RL for solving the asset allocation problem. The Deep Reinforcement Learning approach shows better results than traditional approaches using a simple reward function and only being given the time series of stocks. In Finance, no training to test error generalization results come guaranteed. We can say that the modeling framework can deal with time series prediction and asset allocation, including transaction costs.
Bio: Sonam Srivastava is the founder of Wright Research, an India-based Robo-advisor, where she creates data-driven portfolios out of her deep passion for quant finance.
Wright Research is a wealth creator in the digital space that uses scientific data-driven methods to tactically extract opportunities across assets in the public markets to grow clients’ wealth. Wright functions as SEBI registered Robo advisor and is among the most popular advisors among millennial investors with more than 30000 clients and 125 crore+ in assets. Wright Research has delivered a 90% + outperformance over the index in the last 2.5 years.
She has 10+ years of experience in investment research and portfolio management, working on systematic strategies, long-short strategies, and algorithmic trading. She started her career in the field with Mumbai-based Forefront Capital, which got acquired by Edelweiss. At Edelweiss, she worked as an algorithm designer at Edelweiss's institutional equity broking desk. After that, she worked at HSBC Europe as a quant building factor-driven portfolio solutions. Before starting Wright Research, she also worked at Qplum, doing portfolio management at the artificial intelligence-driven Robo-advisor.
She graduated from IIT Kanpur and has a master’s in financial engineering from Worldquant University. She is a globally recognized researcher and works as a visiting faculty as AI in Finance Institute New York and BSE Institute Limited.
