numpartitions

Number of datastore partitions

Syntax

n = numpartitions(ds)

n = numpartitions(ds,pool)

Description

n = numpartitions(ds) returns the default number of partitions for datastore ds.

n = numpartitions(ds,pool) returns a number of partitions to parallelize datastore access over the parallel pool specified by pool. To parallelize datastore access, you must have Parallel Computing Toolbox™ installed.

Examples

Number of Partitions

Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.

ds = datastore('mapredout.mat');

Get the default number of partitions.

n = numpartitions(ds)

n = 1

By default, there is only one partition in ds because it contains only one small file.

Partition the datastore and return the datastore corresponding to the first part.

subds = partition(ds,n,1);

Read the data in subds.

while hasdata(subds)
    data = read(subds);
end

Number of Partitions for Parallel Datastore Access

Get a number of partitions to parallelize datastore access over the current parallel pool. You must have Parallel Computing Toolbox installed.

Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.

ds = datastore('mapredout.mat');

Get a number of partitions to parallelize datastore access over the current parallel pool.

n = numpartitions(ds, gcp);

Partition the datastore and read the data in each part.

parfor ii=1:n
    subds = partition(ds,n,ii);
    while hasdata(subds)
        data = read(subds);
    end
end

Input Arguments

ds — Input datastore

datastore

Input datastore. You can use the datastore function to create a datastore object from your data.

pool — Parallel pool

parallel pool object

Parallel pool object.

Example: gcp

Extended Capabilities

Thread-Based Environment

Run code in the background using MATLAB® backgroundPool or accelerate code with Parallel Computing Toolbox™ ThreadPool.

Usage notes and limitations:

• In a thread-based environment, you can use numpartitions only with the following datastores:

  • ImageDatastore objects

  • CombinedDatastore or TransformedDatastore objects you create from ImageDatastore objects by using combine or transform

  You can use numpartitions with other datastores if you have Parallel Computing Toolbox. To do so, run the function using a process-backed parallel pool instead of using backgroundPool or ThreadPool (use either ProcessPool or ClusterPool).

For more information, see Run MATLAB Functions in Thread-Based Environment.