Processing Large Data Volumes using PK Chunking & Hyperbatch with Daniel Peter

Technology Flows : Salesforce Architecture Podcast

In this episode I will be speaking with Daniel Peter (@danieljpeter) about processing large volumes of data on Salesforce.

Daniel is Lead Application Developer at Kenandy, an ISV who had built an ERP solution on the Salesforce Platform.

Daniel’s first hand experience of how the Salesforce multi-tenant database behaves has lead him to develop techniques for processing tens of millions of records.

He will describe the techniques which he has refined to ensure SOQL queries are executed with consistent reliability and not fall foul of the most common exceptions relating to row selection, which are:

• Non-selective query
• Too many query rows returned
• Query time out during execution

Daniel will explain how the Batch Apex query locator can be used to implement a technique called PK chunking which allows fine-grained control of the number of rows to be processed in each batch which largely overcomes the 3 common exceptions.

Daniel has even gone as far as experimenting with parallel execution through his Hyperbatch open source project which you can download from GitHub.

Whether your Salesforce database contains tens of thousands or rows or or if you’re up into the 10 of millions Daniel’s tips on working with multi-tenancy are a real eye opener as to what is possible when you design for scale from the outset.

Please enjoy!

Please leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris

Recorded in June 2017

This podcast interview was first published by Technologyflows.com

© TechnologyFlows