Input Data

Wiki ▸ Input Data

Supported Data Types

Flink places some restrictions on the type of elements that can be in a DataStream. See Flink's documentation for more information.

flink-htm supports all composite types such as Tuples, Scala Case Classes, and Java POJOs.

Field Encoders

In HTM theory, input data is encoded into a uniform representation called a Sparse Distributed Representation (SDR). When defining your network model (see Models), add an encoder for each field in your input type. Fields without a configured encoder are ignored.

See the HTM.java Encoder API documentation.

Scala API

The flink-htm Scala API provides a convenient DSL for configuring encoders. A companion object is provided for most encoders of HTM.java (see issue #19).

To use, import the Scala DSL then apply a MultiEncoder to a set of field encoders.

import org.numenta.nupic.encoders.scala._
...
val encoder = MultiEncoder(
    DateEncoder().name("timestamp").timeOfDay(21, 9.5),
    ScalarEncoder().name("consumption").n(50).w(21).maxVal(100.0).resolution(0.1).clipInput(true)
)
val network = ...
network.setEncoder(encoder)

See the examples for more information.