Skip to content

Commit

Permalink
[infra] remove separate javadoc build
Browse files Browse the repository at this point in the history
  • Loading branch information
je-ik committed Oct 17, 2023
1 parent f9088ae commit 01f9c59
Show file tree
Hide file tree
Showing 8 changed files with 11 additions and 124 deletions.
40 changes: 0 additions & 40 deletions .github/workflows/javadocs.yml

This file was deleted.

40 changes: 0 additions & 40 deletions .github/workflows/mvn.sh

This file was deleted.

31 changes: 0 additions & 31 deletions .github/workflows/upload-javadocs.sh

This file was deleted.

4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,9 +346,7 @@ First, let's introduce some glossary:
```

## Online Java docs
* [Latest](https://proxima.datadriven.cz/javadoc/)
* [0.13-SNAPSHOT](https://proxima.datadriven.cz/javadoc/0.13-SNAPSHOT/index.html)
* [0.12.0](https://proxima.datadriven.cz/javadoc/0.12.0/index.html)
* [Latest](https://datadrivencz.github.io/proxima-platform/apidoc/)

## Build notes
CI is run only against changed modules (and its dependents) in pull requests. To completely rebuild the whole project in a PR push a commit with commit message 'rebuild'. After the build, you can squash and remove the commit.
10 changes: 5 additions & 5 deletions docs/src/content/book/datamodel.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ A _key_ is unique string identification of any particular instance of an entity.

Here we see two instances of a particular entity with attribute `Attribute1` and `Attribute2` with type `int`.

The type and serialization of all attributes is defined by a _scheme_ and associated [ValueSerializer](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/scheme/ValueSerializer.html). Let's assume we have an eshop which sells some goods. A typical eshop will have a database of products and users and will want to track behavior of users on the website in order to provide some level of personification. Given such use case, we would define entities in HOCON configuration as follows:
The type and serialization of all attributes is defined by a _scheme_ and associated [ValueSerializer](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/scheme/ValueSerializer.html). Let's assume we have an eshop which sells some goods. A typical eshop will have a database of products and users and will want to track behavior of users on the website in order to provide some level of personification. Given such use case, we would define entities in HOCON configuration as follows:

```
entities {
Expand Down Expand Up @@ -79,9 +79,9 @@ Each _attribute_ is of two possible types:

##### Scalar attributes

`Details` and `preferences` are examples of a _scalar attribute_ of entity _user_. Such attribute may be present or missing (be null), but if present it can have only single value of given type. The type of the attribute is given by its _scheme_. _Scheme_ is [URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) which points to a [ValueSerializerFactory](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/scheme/ValueSerializerFactory.html), which creates instances of [ValueSerializer](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/scheme/ValueSerializer.html). This serializer is then used whenever the platform needs to convert the object representing the attribute's value to bytes and back.
`Details` and `preferences` are examples of a _scalar attribute_ of entity _user_. Such attribute may be present or missing (be null), but if present it can have only single value of given type. The type of the attribute is given by its _scheme_. _Scheme_ is [URI](https://en.wikipedia.org/wiki/Uniform_Resource_Identifier) which points to a [ValueSerializerFactory](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/scheme/ValueSerializerFactory.html), which creates instances of [ValueSerializer](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/scheme/ValueSerializer.html). This serializer is then used whenever the platform needs to convert the object representing the attribute's value to bytes and back.

The scheme `proto:` used in the example above declares that the attribute will be serialized using [ProtoValueSerializer](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/scheme/proto/ProtoSerializerFactory.ProtoValueSerializer.html) and hold a corresponding class that was generated using `protoc` (and extends [Message](https://www.javadoc.io/doc/com.google.protobuf/protobuf-java/latest/com/google/protobuf/Message.html)). For details refer to [protocol buffers](https://developers.google.com/protocol-buffers) documentation.
The scheme `proto:` used in the example above declares that the attribute will be serialized using [ProtoValueSerializer](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/scheme/proto/ProtoSerializerFactory.ProtoValueSerializer.html) and hold a corresponding class that was generated using `protoc` (and extends [Message](https://www.javadoc.io/doc/com.google.protobuf/protobuf-java/latest/com/google/protobuf/Message.html)). For details refer to [protocol buffers](https://developers.google.com/protocol-buffers) documentation.


##### Wildcard attributes
Expand All @@ -95,12 +95,12 @@ We will see examples of all these uses throughout this book.

### StreamElement

The platform handles all data as _data streams_ consisting of _upserts_ and _deletions_ of data. Each _upsert_ or _delete_ is an immutable event describing that a new data element was added, updated or removed. Every [StreamElement](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/storage/StreamElement.html) consists of the following parts:
The platform handles all data as _data streams_ consisting of _upserts_ and _deletions_ of data. Each _upsert_ or _delete_ is an immutable event describing that a new data element was added, updated or removed. Every [StreamElement](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/storage/StreamElement.html) consists of the following parts:

| entity | attribute | key | timestamp | value | delete wildcard flag |
|-----------|-------------|-------|------------|-------|----------------------|

Entity is represented by [EntityDescriptor](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/repository/EntityDescriptor.html), attribute is represented by its name (which is especially needed for wildcard attributes, because name of the attribute does not represent a specific instance of the attribute) and [AttributeDescriptor](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/repository/AttributeDescriptor.html).
Entity is represented by [EntityDescriptor](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/repository/EntityDescriptor.html), attribute is represented by its name (which is especially needed for wildcard attributes, because name of the attribute does not represent a specific instance of the attribute) and [AttributeDescriptor](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/repository/AttributeDescriptor.html).

Key, timestamp and value are the key of the entity (representing a "row"), timestamp is epoch timestamp in millis representing the instant at which the particular change (upsert or delete) happened and value is the new updated value (for upserts) or null (for deletes).

Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/book/generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ This adds `cz.o2.proxima:compiler-maven-plugin` as a plugin to the build and inv

If you cannot use the maven plugin, you can incorporate generation of the model into your build using direct invocation of the generator.

The generator can be invoked from `cz.o2.proxima:proxima-compiler-java-cli` artifact using class [ModelGenerator](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/generator/ModelGenerator.html).
The generator can be invoked from `cz.o2.proxima:proxima-compiler-java-cli` artifact using class [ModelGenerator](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/generator/ModelGenerator.html).

Downloading the artifact and running the class, we get:
```shell
Expand Down
4 changes: 2 additions & 2 deletions docs/src/content/book/storages.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ We will describe the key properties of these three type below.
A commit log is a type of write-once-read-many storage. The key properties are, that it consists of immutable records, that can only be appended and have associated timestamp. Such records are called _events_. A commit log is thus an **unordered** sequence of events and thus forms a _partitioned event stream_.

Proxima platform further restricts this to _partitioned event stream of upserts_.
This is due to the requirement that every event in the event stream refers to a particular _(key, attribute)_ pair, thus every event describes a change of value (or insertion of new value) of particular attribute of particular entity at given timestamp. These proparties were defined in [Data model]({{< relref "/book/datamodel" >}}) and are constituents of **StreamElement** (see [javadoc](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/storage/StreamElement.html)), a commit log is therefore a stream of StreamElements.
This is due to the requirement that every event in the event stream refers to a particular _(key, attribute)_ pair, thus every event describes a change of value (or insertion of new value) of particular attribute of particular entity at given timestamp. These proparties were defined in [Data model]({{< relref "/book/datamodel" >}}) and are constituents of **StreamElement** (see [javadoc](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/storage/StreamElement.html)), a commit log is therefore a stream of StreamElements.

A commit log, being a stream of upserts, can be reduced to table, via the _table-stream duality_ (see [Data model]({{< relref "/book/datamodel" >}}). We can create a table from the stream of upserts by _applying_ the upserts to the initial state of the table. Provided the commit log implementation (e.g. [Apache Kafka](https://kafka.apache.org/)) is able to hold all most recent updates to any key-attributes, then we can reconstruct a table from the stream by reading all the updates and applying each individual update to initially empty table. A commit log that has this property is called a _state commit log_, because it is able to hold the state (and modifications over time) of attributes.

Expand Down Expand Up @@ -135,6 +135,6 @@ A replica attribute family might also specify an optional `filter`, which will t
}
```

The filter must implement [StorageFilter](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/storage/StorageFilter.html) interface.
The filter must implement [StorageFilter](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/storage/StorageFilter.html) interface.

With this high-level description of entities and attribute families, we are ready to see how will this definition by actually used to access the data and to ensure eventual consistency between primary and replica storages. Let's first see the latter described in [Replication]({{< relref "/book/replication" >}}).
4 changes: 2 additions & 2 deletions docs/src/content/book/transformations.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,6 @@ transformations {
}
```

This definition declares a transformation of attribute `data` of entity `event` using a user-defined transformation implemented in `cz.o2.proxima.example.EventDataToUserHistory` (see [here](https://github.com/datadrivencz/proxima-platform/blob/master/example/model/src/main/java/cz/o2/proxima/example/EventDataToUserHistory.java)). The transformation is implementation of [ElementWiseTransfomation](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/transform/ElementWiseTransformation.html), which receives input `StreamElement` for updates to the input attribute(s) and emits transformed elements through [Collector](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/transform/ElementWiseTransformation.Collector.html). The `apply` method returns number of outputs that will be written through the collector, because the collection might be asynchronous.
This definition declares a transformation of attribute `data` of entity `event` using a user-defined transformation implemented in `cz.o2.proxima.example.EventDataToUserHistory` (see [here](https://github.com/datadrivencz/proxima-platform/blob/master/example/model/src/main/java/cz/o2/proxima/example/EventDataToUserHistory.java)). The transformation is implementation of [ElementWiseTransfomation](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/transform/ElementWiseTransformation.html), which receives input `StreamElement` for updates to the input attribute(s) and emits transformed elements through [Collector](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/transform/ElementWiseTransformation.Collector.html). The `apply` method returns number of outputs that will be written through the collector, because the collection might be asynchronous.

As in the case of attribute families (see [here]({{< relref "/book/storages#selecting-what-to-replicate" >}})), transformations might be invoked for all updates (upserts) to the source attribute(s), or the definition can contain optional `filter` implementation of [StorageFilter](https://proxima.datadriven.cz/javadoc/latest/cz/o2/proxima/core/storage/StorageFilter.html) interface.
As in the case of attribute families (see [here]({{< relref "/book/storages#selecting-what-to-replicate" >}})), transformations might be invoked for all updates (upserts) to the source attribute(s), or the definition can contain optional `filter` implementation of [StorageFilter](https://datadrivencz.github.io/proxima-platform/apidocs/cz/o2/proxima/core/storage/StorageFilter.html) interface.

0 comments on commit 01f9c59

Please sign in to comment.