Construction¶

Local¶

Constructing local bolt arrays is nearly identical to using NumPy’s constructors.

Spark¶

Constructing bolt arrays in Spark is similar, except the constructors must be provided with a SparkContext. This is normally provided when running Spark interactively, or created at the beginning of a Spark job. In addition, you can specify which axes will be distributed. Briefly, arrays are represented using a subset of axes as the keys. So a five dimensional array specified with axis=(0, 1) would be represented as key,value pairs where the keys are two-tuples and the values are three-dimensional arrays. Bolt is designed so that its methods are invariant to the choice of distributed axes, but the choice will affect performance of many operations.

Examples¶

Comparing local and distributed constructors

>>> a = blt.ones((2, 3, 4))
>>> a.shape
(2, 3, 4)
>>> a.mode
local

>>> a = blt.ones((2, 3, 4), sc)
>>> a.shape
(2, 3, 4)
>>> a.mode
spark

Comparing different axis choices

>>> x = np.arange(2 * 3 * 4).reshape(2, 3, 4)
>>> blt.array(x, sc, axis=(0, 1)).shape
(2, 3, 4)
>>> blt.array(x, sc, axis=(0, 1, 2)).shape
(2, 3, 4)

>>> blt.ones((2, 3, 4), sc, axis=(0, 1)).shape
(2, 3, 4)
>>> blt.zeros((2, 3, 4), sc, axis=(0,)).shape
(2, 3, 4)

Detailed API¶

Local

class bolt.local.construct.ConstructLocal[source]¶

static array(a, dtype=None, order='C')[source]¶

Create a local bolt array.

Parameters:	a : array-like An array, any object exposing the array interface, an object whose __array__ method returns an array, or any (nested) sequence. dtype : data-type, optional, default=None The desired data-type for the array. If None, will be determined from the data. (see numpy) order : {‘C’, ‘F’, ‘A’}, optional, default=’C’ The order of the array. (see numpy)
Returns:	BoltArrayLocal

static concatenate(arrays, axis=0)[source]¶

Join a sequence of arrays together.

Parameters:	arrays : tuple A sequence of array-like e.g. (a1, a2, ...) axis : int, optional, default=0 The axis along which the arrays will be joined.
Returns:	BoltArrayLocal

static ones(shape, dtype=<type 'numpy.float64'>, order='C')[source]¶

Create a local bolt array of ones.

Parameters:	shape : tuple Dimensions of the desired array dtype : data-type, optional, default=float64 The desired data-type for the array. (see numpy) order : {‘C’, ‘F’, ‘A’}, optional, default=’C’ The order of the array. (see numpy)
Returns:	BoltArrayLocal

static zeros(shape, dtype=<type 'numpy.float64'>, order='C')[source]¶

Create a local bolt array of zeros.

Parameters:	shape : tuple Dimensions of the desired array. dtype : data-type, optional, default=float64 The desired data-type for the array. (see numpy) order : {‘C’, ‘F’, ‘A’}, optional, default=’C’ The order of the array. (see numpy)
Returns:	BoltArrayLocal

Spark

class bolt.spark.construct.ConstructSpark[source]¶

static array(a, context=None, axis=(0, ), dtype=None)[source]¶

Create a spark bolt array from a local array.

Parameters:

a : array-like An array, any object exposing the array interface, an object whose __array__ method returns an array, or any (nested) sequence. context : SparkContext A context running Spark. (see pyspark) axis : tuple, optional, default=(0,) Which axes to distribute the array along. The resulting distributed object will use keys to represent these axes, with the remaining axes represented by values. dtype : data-type, optional, default=None The desired data-type for the array. If None, will be determined from the data. (see numpy)

Returns:

BoltArraySpark

static concatenate(arrays, axis=0)[source]¶

Join two bolt arrays together, at least one of which is in spark.

Parameters:	arrays : tuple A pair of arrays. At least one must be a spark array, the other can be a local bolt array, a local numpy array, or an array-like. axis : int, optional, default=0 The axis along which the arrays will be joined.
Returns:	BoltArraySpark

static ones(shape, context=None, axis=(0, ), dtype=<type 'numpy.float64'>)[source]¶

Create a spark bolt array of ones.

Parameters:

shape : tuple The desired shape of the array. context : SparkContext A context running Spark. (see pyspark) axis : tuple, optional, default=(0,) Which axes to distribute the array along. The resulting distributed object will use keys to represent these axes, with the remaining axes represented by values. dtype : data-type, optional, default=float64 The desired data-type for the array. If None, will be determined from the data. (see numpy)

Returns:

BoltArraySpark

static zeros(shape, context=None, axis=(0, ), dtype=<type 'numpy.float64'>)[source]¶

Create a spark bolt array of zeros.

Parameters:

shape : tuple The desired shape of the array. context : SparkContext A context running Spark. (see pyspark) axis : tuple, optional, default=(0,) Which axes to distribute the array along. The resulting distributed object will use keys to represent these axes, with the remaining axes represented by values. dtype : data-type, optional, default=float64 The desired data-type for the array. If None, will be determined from the data. (see numpy)

Returns:

BoltArraySpark