Zack Li
CTO at Nexa AI
Zack Li is the Co-founder and CTO of NexaAI, a Silicon Valley-based Series A round startup specializing in on-device AI model training and AI infrastructure development. As co-author of the Octopus series models and creator of the functional token, which ranked second on Huggingface among 600K open-sourced AI models in April 2024, Zack is at the forefront of AI innovation. He also designed and leads Nexa SDK, an open-source AI inference toolkit. Prior to NexaAI, Zack developed on-device expertise at Google and Amazon Lab126, focusing on NLP model training, model compression and deployment. He graduated from Stanford and served as president of the Stanford Chinese Entrepreneurs Organization, a global network of over 10,000 members that connects top VCs and founders in Silicon Valley.
All Sessions by Zack Li
On-Device Multimodal Model development and Inference Acceleration
<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Beginner</span> </span>In large language models, traditional tokenization methods have been effective for handling language-related tasks, but they face significant challenges when applied to function calling, often resulting in inaccuracies and hallucinations. To overcome this, we have introduced a novel approach in the Octopus model by using functional tokens, transforming function calling into a language completion task. By treating functions as distinct tokens, the Octopus model enhances accuracy and reliability in function execution, allowing seamless integration of function calling capabilities into language models. Additionally, the challenge of running AI models efficiently on-device, especially with respect to power consumption and processing speed, remains a key hurdle. Our solution addresses this by incorporating advanced quantization and optimization techniques that make on-device AI both faster and more energy-efficient. In this session, we will discuss general quantization technology and local model inference for various modalities such as image generation, text generation, image understanding, and automatic speech recognition. Attendees will learn how these innovations can improve function calling accuracy and significantly boost the efficiency of on-device AI applications, making cutting-edge AI accessible across a range of industries and devices.
On-Device Multimodal Model development and Inference Acceleration
<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Beginner</span> </span>In large language models, traditional tokenization methods have been effective for handling language-related tasks, but they face significant challenges when applied to function calling, often resulting in inaccuracies and hallucinations. To overcome this, we have introduced a novel approach in the Octopus model by using functional tokens, transforming function calling into a language completion task. By treating functions as distinct tokens, the Octopus model enhances accuracy and reliability in function execution, allowing seamless integration of function calling capabilities into language models. Additionally, the challenge of running AI models efficiently on-device, especially with respect to power consumption and processing speed, remains a key hurdle. Our solution addresses this by incorporating advanced quantization and optimization techniques that make on-device AI both faster and more energy-efficient. In this session, we will discuss general quantization technology and local model inference for various modalities such as image generation, text generation, image understanding, and automatic speech recognition. Attendees will learn how these innovations can improve function calling accuracy and significantly boost the efficiency of on-device AI applications, making cutting-edge AI accessible across a range of industries and devices.
