Skip to content

Abstract Dataloader Specifications

Abstract Dataloader Specifications.

The implementations here provide "duck type" protocol definitions of key data loading primitives. In order to implement the specification, users simply need to "fill in" the methods described here for the types which they wish to implement.

Type Parameters

ADL specification protocol types are defined as generics, which are parameterized by other types. These type parameters are documented by a Type Parameters section where applicable.

Composition Rules

ADL protocols which can be composed together are documented by a Composition Rules section.

abstract_dataloader.spec.Metadata

Bases: Protocol

Sensor metadata.

All sensor metadata is expected to be held in memory during training, so great effort should be taken to minimize its memory usage. Any additional information which is not strictly necessary for book-keeping, or which takes more than negligible space, should be loaded as data instead.

Note

This can be a @dataclass, typing.NamedTuple, or a fully custom type - it just has to expose a timestamps attribute.

Attributes:

Name Type Description
timestamps Float[ndarray, N]

measurement timestamps, in seconds. Nominally in epoch time; must be consistent within each trace (but not necessarily across traces). Suggested type: float64, which gives precision of <1us.
Source code in src/abstract_dataloader/spec.py 
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
@runtime_checkable
class Metadata(Protocol):
    """Sensor metadata.

    All sensor metadata is expected to be held in memory during training, so
    great effort should be taken to minimize its memory usage. Any additional
    information which is not strictly necessary for book-keeping, or which
    takes more than negligible space, should be loaded as data instead.

    !!! note

        This can be a `@dataclass`, [`typing.NamedTuple`][typing.NamedTuple],
        or a fully custom type - it just has to expose a `timestamps`
        attribute.

    Attributes:
        timestamps: measurement timestamps, in seconds. Nominally in epoch
            time; must be consistent within each trace (but not necessarily
            across traces). Suggested type: `float64,` which gives precision of
            <1us.
    """

    timestamps: Float[np.ndarray, "N"]

abstract_dataloader.spec.Sensor

Bases: Protocol, Generic[TSample, TMetadata]

A sensor, consisting of a synchronous time-series of measurements.

This protocol is parameterized by generic TSample and TMetadata types, which can encode the expected data type of this sensor. For example:

class Point2D(TypedDict):
    x: float
    y: float

def point_transform(point_sensor: Sensor[Point2D, Any]) -> T:
    ...

This encodes an argument, point_sensor, which expected to be a sensor that reads data with type Point2D, but does not specify a metadata type.

Type Parameters
  • TSample: sample data type which this Sensor returns. As a convention, we suggest returning "batched" data by default, i.e. with a leading singleton axis.
  • TMetadata: metadata type associated with this sensor; must implement Metadata.

Attributes:

Name Type Description
metadata TMetadata

sensor metadata, including timestamp information.
Source code in src/abstract_dataloader/spec.py 
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
@runtime_checkable
class Sensor(Protocol, Generic[TSample, TMetadata]):
    """A sensor, consisting of a synchronous time-series of measurements.

    This protocol is parameterized by generic `TSample` and `TMetadata` types,
    which can encode the expected data type of this sensor. For example:

    ```python
    class Point2D(TypedDict):
        x: float
        y: float

    def point_transform(point_sensor: Sensor[Point2D, Any]) -> T:
        ...
    ```

    This encodes an argument, `point_sensor`, which expected to be a sensor
    that reads data with type `Point2D`, but does not specify a metadata type.

    Type Parameters:
        - `TSample`: sample data type which this `Sensor` returns. As a
            convention, we suggest returning "batched" data by default, i.e.
            with a leading singleton axis.
        - `TMetadata`: metadata type associated with this sensor; must
            implement [`Metadata`][^.].

    Attributes:
        metadata: sensor metadata, including timestamp information.
    """

    metadata: TMetadata

    def stream(self) -> Iterator[TSample]:
        """Stream values recorded by this sensor.

        Returns:
            An iterator yielding successive samples.
        """
        ...

    def __getitem__(self, index: int | np.integer) -> TSample:
        """Fetch measurements from this sensor, by index.

        Args:
            index: sample index, in the sensor scope.

        Returns:
            Loaded sample.
        """
        ...

    def __len__(self) -> int:
        """Total number of measurements."""
        ...

__getitem__ 

__getitem__(index: int | integer) -> TSample

Fetch measurements from this sensor, by index.

Parameters:

Name Type Description Default
index int | integer

sample index, in the sensor scope.

 required

Returns:

Type Description
TSample

Loaded sample.
Source code in src/abstract_dataloader/spec.py 
110
111
112
113
114
115
116
117
118
119
def __getitem__(self, index: int | np.integer) -> TSample:
    """Fetch measurements from this sensor, by index.

    Args:
        index: sample index, in the sensor scope.

    Returns:
        Loaded sample.
    """
    ...

__len__ 

__len__() -> int

Total number of measurements.

Source code in src/abstract_dataloader/spec.py 
121
122
123
def __len__(self) -> int:
    """Total number of measurements."""
    ...

stream 

stream() -> Iterator[TSample]

Stream values recorded by this sensor.

Returns:

Type Description
Iterator[TSample]

An iterator yielding successive samples.
Source code in src/abstract_dataloader/spec.py 
102
103
104
105
106
107
108
def stream(self) -> Iterator[TSample]:
    """Stream values recorded by this sensor.

    Returns:
        An iterator yielding successive samples.
    """
    ...

abstract_dataloader.spec.Synchronization

Bases: Protocol

Synchronization protocol for asynchronous time-series.

Defines a rule for creating matching sensor index tuples which correspond to some kind of global index.

Source code in src/abstract_dataloader/spec.py 
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
@runtime_checkable
class Synchronization(Protocol):
    """Synchronization protocol for asynchronous time-series.

    Defines a rule for creating matching sensor index tuples which correspond
    to some kind of global index.
    """

    def __call__(
        self, timestamps: dict[str, Float[np.ndarray, "_N"]]
    ) -> dict[str, Integer[np.ndarray, "M"]]:
        """Apply synchronization protocol.

        Args:
            timestamps: sensor timestamps. Each key denotes a different sensor
                name, and the value denotes the timestamps for that sensor.

        Returns:
            A dictionary, where keys correspond to each sensor, and values
                correspond to the indices which map global indices to sensor
                indices, i.e. `global[sensor, i] = sensor[sync[sensor][i]]`.
        """
        ...

__call__ 

__call__(timestamps: dict[str, Float[ndarray, _N]]) -> dict[str, Integer[ndarray, M]]

Apply synchronization protocol.

Parameters:

Name Type Description Default
timestamps dict[str, Float[ndarray, _N]]

sensor timestamps. Each key denotes a different sensor name, and the value denotes the timestamps for that sensor.

 required

Returns:

Type Description
dict[str, Integer[ndarray, M]]

A dictionary, where keys correspond to each sensor, and values correspond to the indices which map global indices to sensor indices, i.e. global[sensor, i] = sensor[sync[sensor][i]].
Source code in src/abstract_dataloader/spec.py 
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
def __call__(
    self, timestamps: dict[str, Float[np.ndarray, "_N"]]
) -> dict[str, Integer[np.ndarray, "M"]]:
    """Apply synchronization protocol.

    Args:
        timestamps: sensor timestamps. Each key denotes a different sensor
            name, and the value denotes the timestamps for that sensor.

    Returns:
        A dictionary, where keys correspond to each sensor, and values
            correspond to the indices which map global indices to sensor
            indices, i.e. `global[sensor, i] = sensor[sync[sensor][i]]`.
    """
    ...

abstract_dataloader.spec.Trace

Bases: Protocol, Generic[TSample]

A trace, consisting of multiple simultaneously-recording sensors.

This protocol is parameterized by a generic Sample type, which can encode the expected data type of this trace.

Type Parameters
  • Sample: sample data type which this Trace returns. As a convention, we suggest returning "batched" data by default, i.e. with a leading singleton axis.
Source code in src/abstract_dataloader/spec.py 
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
@runtime_checkable
class Trace(Protocol, Generic[TSample]):
    """A trace, consisting of multiple simultaneously-recording sensors.

    This protocol is parameterized by a generic `Sample` type, which can encode
    the expected data type of this trace.

    Type Parameters:
        - `Sample`: sample data type which this `Trace` returns. As a
            convention, we suggest returning "batched" data by default, i.e.
            with a leading singleton axis.
    """

    @overload
    def __getitem__(self, index: str) -> Sensor: ...

    @overload
    def __getitem__(self, index: int | np.integer) -> TSample: ...

    def __getitem__(self, index: int | np.integer | str) -> TSample | Sensor:
        """Get item from global index (or fetch a sensor by name).

        !!! info

            For the user's convenience, traces can be indexed by a `str` sensor
            name, returning that [`Sensor`][^^.]. While we are generally wary
            of requiring "quality of life" features, we include this since a
            simple `isinstance(index, str)` check suffices to implement this
            feature.

        Args:
            index: sample index, or sensor name.

        Returns:
            Loaded sample if `index` is an integer type, or the appropriate
            [`Sensor`][^^.] if `index` is a `str`.
        """
        ...

    def __len__(self) -> int:
        """Total number of sensor-tuple samples."""
        ...

__getitem__ 

__getitem__(index: str) -> Sensor

__getitem__(index: int | integer) -> TSample

__getitem__(index: int | integer | str) -> TSample | Sensor

Get item from global index (or fetch a sensor by name).

Info

For the user's convenience, traces can be indexed by a str sensor name, returning that Sensor. While we are generally wary of requiring "quality of life" features, we include this since a simple isinstance(index, str) check suffices to implement this feature.

Parameters:

Name Type Description Default
index int | integer | str

sample index, or sensor name.

 required

Returns:

Type Description
TSample | Sensor

Loaded sample if index is an integer type, or the appropriate
TSample | Sensor

Sensor if index is a str.
Source code in src/abstract_dataloader/spec.py 
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
def __getitem__(self, index: int | np.integer | str) -> TSample | Sensor:
    """Get item from global index (or fetch a sensor by name).

    !!! info

        For the user's convenience, traces can be indexed by a `str` sensor
        name, returning that [`Sensor`][^^.]. While we are generally wary
        of requiring "quality of life" features, we include this since a
        simple `isinstance(index, str)` check suffices to implement this
        feature.

    Args:
        index: sample index, or sensor name.

    Returns:
        Loaded sample if `index` is an integer type, or the appropriate
        [`Sensor`][^^.] if `index` is a `str`.
    """
    ...

__len__ 

__len__() -> int

Total number of sensor-tuple samples.

Source code in src/abstract_dataloader/spec.py 
190
191
192
def __len__(self) -> int:
    """Total number of sensor-tuple samples."""
    ...

abstract_dataloader.spec.Dataset

Bases: Protocol, Generic[TSample]

A dataset, consisting of multiple traces concatenated together.

Trace subtypes Dataset

Due to the type signatures, a Trace is actually a subtype of Dataset. This means that a dataset which implements a collection of traces can also take a collection of datasets!

Type Parameters
  • TSample: sample data type which this Dataset returns. As a convention, we suggest returning "batched" data by default, i.e. with a leading singleton axis.
Source code in src/abstract_dataloader/spec.py 
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
@runtime_checkable
class Dataset(Protocol, Generic[TSample]):
    """A dataset, consisting of multiple traces concatenated together.

    !!! note "[`Trace`][^.] subtypes [`Dataset`][^.]"

        Due to the type signatures, a [`Trace`][^.] is actually a subtype of
        [`Dataset`][^.]. This means that a dataset which implements a
        collection of traces can also take a collection of datasets!

    Type Parameters:
        - `TSample`: sample data type which this `Dataset` returns. As a
            convention, we suggest returning "batched" data by default, i.e.
            with a leading singleton axis.
    """

    def __getitem__(self, index: int | np.integer) -> TSample:
        """Fetch item from this dataset by global index.

        Args:
            index: sample index.

        Returns:
            Loaded sample.
        """
        ...

    def __len__(self) -> int:
        """Total number of samples in this dataset."""
        ...

__getitem__ 

__getitem__(index: int | integer) -> TSample

Fetch item from this dataset by global index.

Parameters:

Name Type Description Default
index int | integer

sample index.

 required

Returns:

Type Description
TSample

Loaded sample.
Source code in src/abstract_dataloader/spec.py 
211
212
213
214
215
216
217
218
219
220
def __getitem__(self, index: int | np.integer) -> TSample:
    """Fetch item from this dataset by global index.

    Args:
        index: sample index.

    Returns:
        Loaded sample.
    """
    ...

__len__ 

__len__() -> int

Total number of samples in this dataset.

Source code in src/abstract_dataloader/spec.py 
222
223
224
def __len__(self) -> int:
    """Total number of samples in this dataset."""
    ...

abstract_dataloader.spec.Transform

Bases: Protocol, Generic[TRaw, TTransformed]

Sample or batch-wise transform.

Note

This protocol is a suggestively-named equivalent to Callable[[TRaw], TTransformed] or Callable[[Any], Any].

Composition Rules
  • Transform can be freely composed, as long as each transform's TTransformed matches the next transform's TRaw; this composition is implemented by abstract.Transform.
  • Composed Transforms result in another Transform: 
    Transform[T2, T3] (.) Transform[T1, T2] = Transform[T1, T3].
Type Parameters
  • TRaw: Input data type.
  • TTransformed: Output data type.
Source code in src/abstract_dataloader/spec.py 
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
@runtime_checkable
class Transform(Protocol, Generic[TRaw, TTransformed]):
    """Sample or batch-wise transform.

    !!! note

        This protocol is a suggestively-named equivalent to
        `Callable[[TRaw], TTransformed]` or `Callable[[Any], Any]`.

    Composition Rules:
        - `Transform` can be freely composed, as long as each transform's
          `TTransformed` matches the next transform's `TRaw`; this composition
          is implemented by [`abstract.Transform`][abstract_dataloader.].
        - Composed `Transform`s result in another `Transform`:
          ```
          Transform[T2, T3] (.) Transform[T1, T2] = Transform[T1, T3].
          ```

    Type Parameters:
        - `TRaw`: Input data type.
        - `TTransformed`: Output data type.
    """

    def __call__(self, data: TRaw) -> TTransformed:
        """Apply transform to a single sample.

        Args:
            data: A single `TRaw` data sample.

        Returns:
            A single `TTransformed` data sample.
        """
        ...

__call__ 

__call__(data: TRaw) -> TTransformed

Apply transform to a single sample.

Parameters:

Name Type Description Default
data TRaw

A single TRaw data sample.

 required

Returns:

Type Description
TTransformed

A single TTransformed data sample.
Source code in src/abstract_dataloader/spec.py 
256
257
258
259
260
261
262
263
264
265
def __call__(self, data: TRaw) -> TTransformed:
    """Apply transform to a single sample.

    Args:
        data: A single `TRaw` data sample.

    Returns:
        A single `TTransformed` data sample.
    """
    ...

abstract_dataloader.spec.Collate

Bases: Protocol, Generic[TTransformed, TCollated]

Data collation.

Note

This protocol is a equivalent to Callable[[Sequence[TTransformed]], TCollated]. Collate can also be viewed as a special case of Transform, where the input type TRaw must be a Sequence[...].

Composition Rules
  • Collate can only be composed in parallel, and can never be sequentially composed.
Type Parameters
  • TTransformed: Input data type.
  • TCollated: Output data type.
Source code in src/abstract_dataloader/spec.py 
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
@runtime_checkable
class Collate(Protocol, Generic[TTransformed, TCollated]):
    """Data collation.

    !!! note

        This protocol is a equivalent to
        `Callable[[Sequence[TTransformed]], TCollated]`. `Collate` can also
        be viewed as a special case of `Transform`, where the input type
        `TRaw` must be a `Sequence[...]`.

    Composition Rules:
        - `Collate` can only be composed in parallel, and can never be
          sequentially composed.

    Type Parameters:
        - `TTransformed`: Input data type.
        - `TCollated`: Output data type.
    """

    def __call__(self, data: Sequence[TTransformed]) -> TCollated:
        """Collate a set of samples.

        Args:
            data: A set of `TTransformed` samples.

        Returns:
            A `TCollated` batch.
        """
        ...

__call__ 

__call__(data: Sequence[TTransformed]) -> TCollated

Collate a set of samples.

Parameters:

Name Type Description Default
data Sequence[TTransformed]

A set of TTransformed samples.

 required

Returns:

Type Description
TCollated

A TCollated batch.
Source code in src/abstract_dataloader/spec.py 
288
289
290
291
292
293
294
295
296
297
def __call__(self, data: Sequence[TTransformed]) -> TCollated:
    """Collate a set of samples.

    Args:
        data: A set of `TTransformed` samples.

    Returns:
        A `TCollated` batch.
    """
    ...

abstract_dataloader.spec.Pipeline

Bases: Protocol, Generic[TRaw, TTransformed, TCollated, TProcessed]

Dataloader transform pipeline.

This protocol is parameterized by four type variables which encode the different data formats at each stage in the pipeline. This forms a Raw -> Transformed -> Collated -> Processed pipeline with three transforms:

  • sample: a sample to sample transform; can be sequentially assembled from one or more Transforms.
  • collate: a list-of-samples to batch transform. Can use exactly one Collate.
  • batch: a batch to batch transform; can be sequentially assembled from one or more Transforms.
Composition Rules
Type Parameters
  • TRaw: Input data format.
  • TTransformed: Data after the first transform step.
  • TCollated: Data after the second collate step.
  • TProcessed: Output data format.
Source code in src/abstract_dataloader/spec.py 
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
@runtime_checkable
class Pipeline(
    Protocol, Generic[TRaw, TTransformed, TCollated, TProcessed]
):
    """Dataloader transform pipeline.

    This protocol is parameterized by four type variables which encode the
    different data formats at each stage in the pipeline. This forms a
    `Raw -> Transformed -> Collated -> Processed` pipeline with three
    transforms:

    - [`sample`][.]: a sample to sample transform; can be sequentially
      assembled from one or more [`Transform`][^.]s.
    - [`collate`][.]: a list-of-samples to batch transform. Can use exactly one
      [`Collate`][^.].
    - [`batch`][.]: a batch to batch transform; can be sequentially assembled
      from one or more [`Transform`][^.]s.

    Composition Rules:
        - A full `Pipeline` can be sequentially pre-composed and/or
          post-composed with one or more [`Transform`][^.]s; this is
          implemented by [`generic.ComposedPipeline`][abstract_dataloader.].
        - `Pipeline`s can always be composed in parallel; this is implemented
          by [`generic.ParallelPipelines`][abstract_dataloader.], with a
          pytorch [`nn.Module`][torch.]-compatible version in
          [`torch.ParallelPipelines`][abstract_dataloader.].

    Type Parameters:
        - `TRaw`: Input data format.
        - `TTransformed`: Data after the first `transform` step.
        - `TCollated`: Data after the second `collate` step.
        - `TProcessed`: Output data format.
    """

    def sample(self, data: TRaw) -> TTransformed:
        """Transform single samples.

        - Operates on single samples, nominally on the CPU-side of a
          dataloader.
        - This method is both sequentially and parallel composable.

        Args:
            data: A single `TRaw` data sample.

        Returns:
            A single `TTransformed` data sample.
        """
        ...

    def collate(self, data: Sequence[TTransformed]) -> TCollated:
        """Collate a list of data samples into a GPU-ready batch.

        - Operates on the CPU-side of the dataloader, and is responsible for
          aggregating individual samples into a batch (but not transferring to
          the GPU).
        - Analogous to the `collate_fn` of a
          [pytorch dataloader](https://pytorch.org/docs/stable/data.html).
        - This method is not sequentially composable.

        Args:
            data: A sequence of `TTransformed` data samples.

        Returns:
            A `TCollated` collection of the input sequence.
        """
        ...

    def batch(self, data: TCollated) -> TProcessed:
        """Transform data batch.

        - Operates on a batch of data, nominally on the GPU-side of a
          dataloader.
        - This method is both sequentially and parallel composable.

        !!! info "Implementation as `torch.nn.Module`"

            If these `Transforms` require GPU state, it may be helpful to
            implement it as a `torch.nn.Module`. In this case, `batch`
            should redirect to `__call__`, which in turn redirects to
            [`nn.Module.forward`][torch.] in order to handle any registered
            pytorch hooks.

        Args:
            data: A `TCollated` batch of data, nominally already sent to the
                GPU.

        Returns:
            The `TProcessed` output, ready for the downstream model.
        """
        ...

batch 

batch(data: TCollated) -> TProcessed

Transform data batch.

  • Operates on a batch of data, nominally on the GPU-side of a dataloader.
  • This method is both sequentially and parallel composable.

Implementation as torch.nn.Module

If these Transforms require GPU state, it may be helpful to implement it as a torch.nn.Module. In this case, batch should redirect to __call__, which in turn redirects to nn.Module.forward in order to handle any registered pytorch hooks.

Parameters:

Name Type Description Default
data TCollated

A TCollated batch of data, nominally already sent to the GPU.

 required

Returns:

Type Description
TProcessed

The TProcessed output, ready for the downstream model.
Source code in src/abstract_dataloader/spec.py 
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
def batch(self, data: TCollated) -> TProcessed:
    """Transform data batch.

    - Operates on a batch of data, nominally on the GPU-side of a
      dataloader.
    - This method is both sequentially and parallel composable.

    !!! info "Implementation as `torch.nn.Module`"

        If these `Transforms` require GPU state, it may be helpful to
        implement it as a `torch.nn.Module`. In this case, `batch`
        should redirect to `__call__`, which in turn redirects to
        [`nn.Module.forward`][torch.] in order to handle any registered
        pytorch hooks.

    Args:
        data: A `TCollated` batch of data, nominally already sent to the
            GPU.

    Returns:
        The `TProcessed` output, ready for the downstream model.
    """
    ...

collate 

collate(data: Sequence[TTransformed]) -> TCollated

Collate a list of data samples into a GPU-ready batch.

  • Operates on the CPU-side of the dataloader, and is responsible for aggregating individual samples into a batch (but not transferring to the GPU).
  • Analogous to the collate_fn of a pytorch dataloader.
  • This method is not sequentially composable.

Parameters:

Name Type Description Default
data Sequence[TTransformed]

A sequence of TTransformed data samples.

 required

Returns:

Type Description
TCollated

A TCollated collection of the input sequence.
Source code in src/abstract_dataloader/spec.py 
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
def collate(self, data: Sequence[TTransformed]) -> TCollated:
    """Collate a list of data samples into a GPU-ready batch.

    - Operates on the CPU-side of the dataloader, and is responsible for
      aggregating individual samples into a batch (but not transferring to
      the GPU).
    - Analogous to the `collate_fn` of a
      [pytorch dataloader](https://pytorch.org/docs/stable/data.html).
    - This method is not sequentially composable.

    Args:
        data: A sequence of `TTransformed` data samples.

    Returns:
        A `TCollated` collection of the input sequence.
    """
    ...

sample 

sample(data: TRaw) -> TTransformed

Transform single samples.

  • Operates on single samples, nominally on the CPU-side of a dataloader.
  • This method is both sequentially and parallel composable.

Parameters:

Name Type Description Default
data TRaw

A single TRaw data sample.

 required

Returns:

Type Description
TTransformed

A single TTransformed data sample.
Source code in src/abstract_dataloader/spec.py 
334
335
336
337
338
339
340
341
342
343
344
345
346
347
def sample(self, data: TRaw) -> TTransformed:
    """Transform single samples.

    - Operates on single samples, nominally on the CPU-side of a
      dataloader.
    - This method is both sequentially and parallel composable.

    Args:
        data: A single `TRaw` data sample.

    Returns:
        A single `TTransformed` data sample.
    """
    ...