Comparative Analysis of NPU Optimized Software Framework

EDGE AI POD

The future of AI isn't just in massive cloud servers—it's already sitting in your pocket. In this eye-opening presentation, Yeon-seok, CEO and co-founder of JTIC AI, reveals how his company is revolutionizing the AI landscape by tapping into the underutilized Mobile Processing Units (MPUs) that have been standard in smartphones since 2017.
While tech giants pour billions into cloud infrastructure, JTIC AI has identified a critical opportunity: leveraging the powerful AI processors already in billions of devices worldwide. This approach delivers not just cost savings, but crucial advantages including offline functionality, enhanced data security, and real-time responsiveness—without depending on internet connectivity.
The technical journey involves three essential components: hardware utilization, model optimization, and runtime software. Yeon-seok breaks down sophisticated model optimization techniques like pruning, quantization, and knowledge distillation that make complex AI models deployable to mobile devices. However, the biggest challenge isn't hardware capability but software fragmentation. Unlike the GPU market dominated by NVIDIA and CUDA, mobile devices operate in a fragmented ecosystem where Apple, Qualcomm, MediaTek, and others maintain incompatible software stacks—creating significant barriers for AI engineers.
JTIC AI's innovative solution is an end-to-end automated pipeline that handles everything from model optimization to device-specific benchmarking. Their system can determine which runtime will deliver optimal performance for specific models on specific devices—something that's impossible to predict without comprehensive testing. With this approach, developers can deploy sophisticated AI across the mobile ecosystem without wrestling with manufacturer-specific implementations.
Ready to unlock the AI capabilities already sitting in your users' pockets? Discover how on-device AI can transform your applications with better privacy, offline functionality, and faster response times—all while reducing your cloud infrastructure costs.

Send us a text

Support the show

Learn more about the EDGE AI FOUNDATION - edgeaifoundation.org

章

1. Introduction to JTIC AI (00:00:00)

2. Background and Benefits of On-device AI (00:00:51)

3. Key Elements: Model Optimization Techniques (00:01:28)

4. Target Devices and MPU Limitations (00:03:15)

5. Software Frameworks and Compatibility Issues (00:05:26)

6. Benchmarking for Optimal Performance (00:09:36)

7. End-to-End Pipeline Solution (00:11:16)

8. Benefits and Conclusion (00:12:29)