Zack Li

Zack Li

CTO at Nexa AI

Zack Li is the Co-founder and CTO of NexaAI, a Silicon Valley-based Series A round startup specializing in on-device AI model training and AI infrastructure development. As co-author of the Octopus series models and creator of the functional token, which ranked second on Huggingface among 600K open-sourced AI models in April 2024, Zack is at the forefront of AI innovation. He also designed and leads Nexa SDK, an open-source AI inference toolkit. Prior to NexaAI, Zack developed on-device expertise at Google and Amazon Lab126, focusing on NLP model training, model compression and deployment. He graduated from Stanford and served as president of the Stanford Chinese Entrepreneurs Organization, a global network of over 10,000 members that connects top VCs and founders in Silicon Valley.

All Sessions by Zack Li

Day 3 10/31/2024
3:05 pm - 4:05 pm

On-Device Multimodal Model development and Inference Acceleration

<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Beginner</span> </span>

In large language models, traditional tokenization methods have been effective for handling language-related tasks, but they face significant challenges when applied to function calling, often resulting in inaccuracies and hallucinations. To overcome this, we have introduced a novel approach in the Octopus model by using functional tokens, transforming function calling into a language completion task. By treating functions as distinct tokens, the Octopus model enhances accuracy and reliability in function execution, allowing seamless integration of function calling capabilities into language models. Additionally, the challenge of running AI models efficiently on-device, especially with respect to power consumption and processing speed, remains a key hurdle. Our solution addresses this by incorporating advanced quantization and optimization techniques that make on-device AI both faster and more energy-efficient. In this session, we will discuss general quantization technology and local model inference for various modalities such as image generation, text generation, image understanding, and automatic speech recognition. Attendees will learn how these innovations can improve function calling accuracy and significantly boost the efficiency of on-device AI applications, making cutting-edge AI accessible across a range of industries and devices.

Day 3 10/31/2024
3:05 pm - 4:05 pm

On-Device Multimodal Model development and Inference Acceleration

<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Beginner</span> </span>

In large language models, traditional tokenization methods have been effective for handling language-related tasks, but they face significant challenges when applied to function calling, often resulting in inaccuracies and hallucinations. To overcome this, we have introduced a novel approach in the Octopus model by using functional tokens, transforming function calling into a language completion task. By treating functions as distinct tokens, the Octopus model enhances accuracy and reliability in function execution, allowing seamless integration of function calling capabilities into language models. Additionally, the challenge of running AI models efficiently on-device, especially with respect to power consumption and processing speed, remains a key hurdle. Our solution addresses this by incorporating advanced quantization and optimization techniques that make on-device AI both faster and more energy-efficient. In this session, we will discuss general quantization technology and local model inference for various modalities such as image generation, text generation, image understanding, and automatic speech recognition. Attendees will learn how these innovations can improve function calling accuracy and significantly boost the efficiency of on-device AI applications, making cutting-edge AI accessible across a range of industries and devices.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google