I got used to work on a project that is implemented over Dynamics CRM online 2015. One of the main limitations that you will be exposed to in any dynamics CRM online project is the two-minute time limit for the Plugin to finish execution, which will be a problem if you have huge data and huge number of records. Plugins that needs more than 2 minutes to execute will throw a timeout exceptions.
So here are some solutions that I got used to work on to make the plugin execute in less than 2 minutes:
1) Microsoft says: If you intend to register your plug-ins for synchronous execution, we recommend that you design them to complete their operation in less than 10 seconds. It’s best to minimize processing time in plug-ins to maintain interactivity of the client applications that are connected to the same organization service that executes the plug-in.
So, try to slice your plugin in more than one plugin that each executes in less than 10 seconds.
2) If your plugin is executed Asynchronously, move it to a scheduled task (job), by creating a console application that is triggered every (x) minutes; x: can is determined based on the business requirement; and put it in the Windows task scheduler.
In this way, you will get rid of too many plugins, but make sure that you have a good memory in the server you are using to host your scheduled jobs. AZURE is a good solution. :)
3) If you want to process huge data in your plugin or any scheduled job, make sure you select only the needed columns and DON'T use ColumnSet(true), and use the offline methodology, which is: Retrieve the records and store them in server memory and process them offline without go and forth to CRM for every record you want to process, but remember you need a good server memory to store that large data. And use the "Using" statement to automatically dispose the CRM connection.