In Nucache when we are loading in all data that we page over the data as to not cause an SQL timeout #7970
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is similar to the problem mentioned in v7 preview here #7574
When a site cold boots or rebuilds nucache from scratch by querying the DB, if there's a ton of data in the database this could lead to an Sql timeout exception. When we read this data in we use a db reader (cursor) so we don't load all of the data into memory at once (to preserve memory) but this means the connection may stay open longer. Because of that, we should be paging over the data using a db reader. This PR updates this behavior to read in by pages of size 500.
When in PureLive and a content type is changed, the content/media nucache needs to rebuild from the database data which is also a factor in when sql timeouts can occur. This paging will alleviate that issue but i've also added a Parellel call to process both the content and the media in parallel if the user's machine supports it which could go a little quicker as well.
Testing
These queries will execute in various circumstances - like a cold boot when nucache files don't exist, or when content types are changed. So to test just have PureLive mode on (default) and add a property to a doc type, this will rebuild all of nucache, both media and content with these queries.