# I/O API

## Inputs

Every module can have zero or more inputs, as well as outputs. Inputs take data from other modules, outputs push data to
subsequent modules. Inputs and outputs preferrably have one of the predefined types, but can in general take on any
custom flatbuffer defined type.

**The predefined types are**

- Event
- Frame
- IMU
- Trigger
- BoundingBox

### Input definition

Inputs can **only** be defined in the `initInputs` static function. The number of inputs as well as their types and
names must be fixed for every module, and can not be changed at runtime. However, to take on a variable number of input
signals, inputs can me marked as optional. In that case, the code has to check at runtime if an input is connected.

Inputs get defined in the `static void initInputs(dv::InputDefinitionList &in)` function. The argument is a modifiabable
container that offers functions to add inputs of different types:

```C++
in.addEventInput(const std::string &name, bool optional = false)
in.addFrameInput(const std::string &name, bool optional = false)
in.addIMUInput(const std::string &name, bool optional = false)
in.addTriggerInput(const std::string &name, bool optional = false)
in.addBoundingBoxInput(const std::string &name, bool optional = false)
```

To add an input for a custom flatbuffer type, use the following function. The type identifier is the four character type
identifier string.

```C++
in.addInput(const std::string &name, const std::string &typeIdentifier, bool optional = false)
```

### Inputs at runtime

Inputs can be accessed in constructor, `run` and `configUpdate` functions. Inheriting from `dv::ModuleBase` provides an
object called `inputs`.

To retrieve an input with a known name and type, call the respective function

```C++
const auto input = inputs.getEventInput(const std::string &name);
const auto input = inputs.getFrameInput(const std::string &name);
const auto input = inputs.getIMUInput(const std::string &name);
const auto input = inputs.getTriggerInput(const std::string &name);
const auto input = inputs.getBoundingBoxInput(const std::string &name);
```

To retrieve an input for a custom flatbuffer type `<T>`, call

```C++
const auto input = inputs.getInput<T>(const std::string &name);
```

### Input APIs

Not all input types share the same information. Therefore, the APIs for the different input types differ slightly.

`````{tabs}
````{group-tab} Events

#### Data

```C++
const auto inEvents = input.events();
```

Returns an (random access) iterable data structure of the latest events to arrive at the input. One can convenently loop
over the arrays with `for (const auto &event : input.events()) {}`.

The returned value can also be implicitly converted to a `const dv::EventStore`. It can directly be passed to any
function that takes a `const dv::EventStore&`

#### Meta data

```C++
int width = input.sizeX();
int height = input.sizeY();
cv::Size size = input.size();
```

Return the width / height of the input data. The maximum x-coordinate of an event at this input is `input.sizeX() - 1`.
This value is depends on what module / camera is attached to the input at runtime. E.g. In case of a low-resolution
*DVS128* it would be 128, in case of a high-resolution *DVS346* it would be 346. The same holds for the y-coordinate.
````

````{group-tab} Frames

#### Frame Data

```C++
const auto frame = input.frame()
```

#### Individual Frame Meta Data

The following methods for accessing the frame contents are supported:

- `const auto &pixels = frame.pixels();` A reference to a vector type for raw pixel access
- `cv::Mat *matPtr = frame.getMatPointer();` A pointer to an *OpenCV* `Mat` representing this frame

Each frame has additional metadata, that can differ for each frame

- `long timestamp = frame.timestamp();` The timestamp of the frame
- `long timestamp = frame.timestampStartOfFrame();` The timestamp of the start of frame
- `long timestamp = frame.timestampEndOfFrame();` The timestamp of the end of frame
- `long timestamp = frame.timestampStartOfExposure();` The timestamp ofthe start of the exposure
- `long timestamp = frame.timestampEndOfExposure();` The timestamp ofthe end of the exposure
- `dv::FrameFormat format = frame.format();` The format of the frame. Options are *GRAY*, *BGR*, *BGRA*
- `int width = frame.sizeX();` The width of this specific frame. The width can be smaller or equal to the input width,
  but never larger.
- `int height = frame.sizeY();` The height of this specific frame. The height can be smaller or equal to the input
  height, but never larger.

#### Input Meta Data

```C++
int width = input.sizeX();
int height = input.sizeY();
cv::Size size = input.size();

```

Returns the maximum dimensions of the input. Individual frames on the input can be smaller than this, but never bigger.
````

````{group-tab} IMU

#### Data

```C++
input.data()
```

Retrieved the newest data on the this input. The returned data type is (random access) iterable. A convient way to
iterate through the newest data is to call `for (const auto &sample : input.data()) {}`
````

````{group-tab} Triggers

#### Data

```C++
input.data()
```

Retrieved the newest data on the this input. The returned data type is (random access) iterable. A convient way to
iterate through the newest data is to call `for (const auto &sample : input.data()) {}`
````

````{group-tab} Bounding Box

#### Data

```C++
input.data()
```

Retrieved the newest data on the this input. The returned data type is (random access) iterable. A convient way to
iterate through the newest data is to call `for (const auto &boundingBox : input.data()) {}`

#### Input Meta Data

```C++
int width = input.sizeX();
int height = input.sizeY();
cv::Size size = input.size();

```

Returns the maximum dimensions of the input. A bounding box on this input is always in respect to these dimensions.
````
`````

#### Common for all inputs

The following functions are available for all input types, including custom flatbuffer types.

```C++
input.isConnected()
```

For optional inputs, this function returns true iff the input is connected to an output in the current runtime.

```C++
input.getOriginDescription()
```

Returns a string describing the origin of the data. In most cases, this string will give some information about the
original creator of the data, like serial number of the camera. However, this is not guaranteed, as every module the
data passes through is allowed to alter the string it passes down in any way.

```C++
input.data()
```

**Advanced use** Retrieved the newest data on the this input. The returned data type is equivalent to a shared pointer
type. Dereferencing the return value gives the flatbuffer type `<T>` of the latest data.

```C++
input.infoNode()
```

**Advanced use** Returns the underlying config trees info node about the output connected to this input. This info node
provides information about dimensions etc about the input. Convenience functions such as the event inputs `sizeX` etc.
are based on this information. An input of a custom flatbuffer type can have arbitrary meta information about the
connection, which an be obtained from the info node. For example, to get an integer with key *sizeX* from the info node,
call `input.infoNode().getInt("sizeX")`

## Outputs

### Output definition

Outputs can only be defined in the static `initOutputs` function. The number, names and types of outputs have to be
constant during runtime.

Outputs are defined in the `static void initOutputs(dv::OutputDefinitionList &out)` function. The argument is a
modifiabable container that offers functions to add outputs of different types:

```C++
out.addEventOutput(const std::string &name)
out.addFrameOutput(const std::string &name)
out.addIMUOutput(const std::string &name)
out.addTriggerOutput(const std::string &name)
out.addBoundingBoxOutput(const std::string &name)
```

To add an output of a custom flatbuffer type, use the generic function with your custom types four-character type
identifier.

```C++
out.addOutput(const std::string &name, const std::string &typeIdentifier)
```

### Output setup

Upon initialization of your module, **outputs have to be set up in the constructor**. Setting up outputs means assigning
them required meta information such as dimensions and source identifiers. In most cases, one wants to set up an output
in terms of an input. For example, a filter module takes events of a certain dimension and wants to emit events with the
same dimension. This is why setup takes place at initialization time, rather than static.

#### Event, Frame and Bounding Box outputs

```C++
output.setup(int sizeX, int sizeY, const std::string &originDescription)
```

Sets up the output with width `sizeX` and height `sizeY` and the supplied origin description. It is advisable to copy
the origin description from an input, as the purpose of this field is to keep track of the original creator of the data.

```C++
output.setup(const RuntimeInput &input)
```

Sets up the output with the same parameters as a compatible input. Event, Frame and Bounding Box inputs are compatible
with each other. For example, setting up a frame output with the same parameters as an Event input would look like
`outputs.getFrameOutput("frames").setup(inputs.getEventInput("events"));`

#### Other outputs

```C++
output.setup(const std::string &originDescription)
```

Sets up the output with the given origin description. It is advisable to copy the origin description from an input, as
the purpose of this field is to keep track of the original creator of the data.

**Advanced use** This is sufficient for the provided IMU and Trigger types. For a custom flatbuffer type, you may
require additional information to be present in the output info config node. To set up these custom fields, put them in
the output info node obtained by `output.infoNode()`

### Send data to outputs

Data can be sent to outputs at any time in the run function. One can send as many data packets as needed (or none). Any
sending of data has to happen on the thread where the run function runs on.

Sending data is easy. Depending on the datatype, we expose different convenience functions to use.

#### Events, IMU, Trigger

To send data to an output, simply stream the data to the desired output. In case of events, IMU, and trigger data, the
piped data elements are first appended to an out packet. They only get sent out when piping in `dv::commit`.

```C++
outputs.getEventOutput("events") << event1 << event2 << dv::commit;
```

Alternatively, one can specifically obtain the the output container, append to it and commit it.

```C++
auto outEvents = outputs.getEventOutput("events").events();
outEvents.push_back(event1);
outEvents.push_back(event2);
outEvents.commit();
```

After committing, the output container gets reassigned and can be used again immediately after.

For IMU and Trigger outputs, one uses the `data()` function instead of the `events()` function. (The `data()` function
is also available for events and functually identical).

```C++
auto outIMU = outputs.getIMUOutput("imu").data();
outIMU.push_back(imu1);
outIMU.push_back(imu2);
outIMU.commit();
```

The output containers for events, IMU and trigger also have stream operator support, as

```C++
auto outTrigger = outputs.getTriggerOutput("trigger").data();
outTrigger << trigger1 << trigger2;
outTrigger.commit();
```

All methods are equivalent in performance.

#### Frames

##### OpenCV Frames

To send an *OpenCV* frame to an output, simply use the stream operator on the output.

```C++
outputs.getFrameOutput("frames") << myFrame << dv::commit;
```

*OpenCV* frames do not have a notion of timestamps. By default, all frames sent out this way would have a timestamp of
0\. To make sure the frames on the output have timestamps, one can stream in the timestamp first.

```C++
outputs.getFrameOutput("frames") << timestamp << myFrame << dv::commit;
```

Alternatively, one can get the frame at the output, and use that to commit the *OpenCV* matrix manually. This is used,
when one wants to set additional parameters such as exposure times.

```C++
auto outFrame = outputs.getFrameOutput("frames").frame();
outFrame.setTimestamp(timestamp);
outFrame.setTimestampStartOfExposure(timeStartOfExposure);
outFrame.setTimestampEndOfExposure(timeEndOfExposure);
outFrame.setMat(openCVFrame);
outFrame.commit()
```

##### Non-OpenCV Frames

```C++
// Get current output frame
auto outFrame = outputs.getFrameOutput("frames").frame();
// Setting the format and size makes the pixel buffer allocate the right amount of storage
outFrame.setFormat(dv::FrameFormat::BGR);
outFrame.setSize(640, 480);
// Set the timestamp
outFrame.setTimestamp(timestamp);
// .pixels gives a writable, vector compatible buffer with enough storage.
// You can write the image in there with any method
std::copy(myBuffer, outFrame.pixels());
// As soon as all the data is ready, calling commit sends the data out
outFrame.commit();
```

Non *OpenCV* frames are bit harder to send out. The idea is to generate an empty array on the output, by setting the
format as well as the size. Then write the image data in the buffer obtained by `pixels()`. As soon as all the data is
ready, call `commit()` to send the frame out.

**Note** Set the size and format before writing any data to the pixel array. After calling commit, one has to set the
size and format again for the next frame, which gives a new array.
