Logisland REST API¶
The Logisland REST API for third party applications.
| maxdepth: | 3 |
|---|
Introduction¶
Logisland makes available a standard RESTful API definition to interoperate with any third party application implementing it.
The API should be implemented by a third party application and logisland will regularly poll this endpoint in order to:
- Ask for configuration changes to be triggered.
- Report the latest configuration applied (to ease up resynchronization and business continuity).
Both flows can hence be resumed by the following sequence diagram:
Usage¶
In terms of API, two degrees of freedom are possible:
Dataflow:
A dataflow is a set of services and streams allowing a data flowing from one or more sources, being transformed and reach one or more destinations (sinks).
Act at dataflow level if you want to:
- Add/Remove any streaming endpoint
- Change any active stream configuration (e.g. kafka topic)
- Create/Remote/Modify any service
Pipeline:
A pipeline is a processing chain acting on a data flowing point-to-point.
The api gives you the possibility to have a finer-grained control of what is going of any stream pipeline without perturbing the stream itself. This means that the processor chain will be dynamically reconfigured without the need of stopping the stream and reconfigure the whole dataflow.
Act at pipeline level if you want to:
- Add/Remove processors in the pipeline
- Change any processor configuration
Hint
As a general rule, the changes will be triggered if the lastUpdated field of the object you are going to modify is fresher than the one known by logisland.
API Specification¶
This section resumes the Rest API specification. More details are available on the swagger spec.
Operations¶
GET /dataflows/{dataflowName}¶
Summary¶
Retrieves the configuration for a specified dataflow
Description¶
A dataflow is a set of services and streams allowing a data flowing from one or more sources, being transformed by a pipeline and reach one or more destinations (sinks).Logisland will call this endpoint to know which configuration should be run.
This endpoint also supports HTTP caching (Last-Updated, If-Modified-Since) as per RFC 7232, section 3.3
Parameters¶
| delim: |
|
|---|
Responses¶
200¶
Return the dataflow configuration. On logisland side, the following will happen: - At dataflow level:
Fully reconfigure a dataflow (stop and then start) if nothing is running (initial state) or if lastUpdated is fresher than the one of the already running dataflow.
In this case be aware that old stream and services will be destroyed and new ones will be created.
Do nothing otherwise (keep running the active dataflow)
At pipeline level:
- The processor chain will be fully reconfigured if and only if the pipeline lastUpdated is fresher than the lastUpdated known by the system.
In any case the stream is never stopped.
Type: Versioned extended inline
Example:
{
"lastModified": "2015-01-01T15:00:00.000Z",
"modificationReason": "somestring",
"services": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
}
],
"streams": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring",
"pipeline": {
"lastModified": "2015-01-01T15:00:00.000Z",
"modificationReason": "somestring",
"processors": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
}
]
}
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring",
"pipeline": {
"lastModified": "2015-01-01T15:00:00.000Z",
"modificationReason": "somestring",
"processors": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
}
]
}
}
]
}
304¶
Nothing has been modified since the last call.
In this case the body content will be completely ignored (hence the server can answer with an empty body to save network and resources).
404¶
Not found (the server probably does not handle this dataflow)
default¶
Unexpected error
POST /dataflows/{dataflowName}¶
Summary¶
Push the configuration of running dataflows.
Description¶
In order to ensure business continuity, Logisland will contact the third party application in order to push a snapshot of the current configuration.The endpoint will be called: - On a regular basis (according to logisland configuration). - Each time the a dataflow or a pipeline configuration change has been applied.
This service can be seen as well as a liveness ping.
Parameters¶
| delim: |
|
|---|
Request¶
Body¶
A streaming pipeline.
Versioned extended inline
Inline schema:
| delim: |
|
|---|
{
"lastModified": "2015-01-01T15:00:00.000Z",
"modificationReason": "somestring",
"services": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
}
],
"streams": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring",
"pipeline": {
"lastModified": "2015-01-01T15:00:00.000Z",
"modificationReason": "somestring",
"processors": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
}
]
}
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring",
"pipeline": {
"lastModified": "2015-01-01T15:00:00.000Z",
"modificationReason": "somestring",
"processors": [
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
},
{
"component": "somestring",
"config": [
{
"key": "somestring",
"type": "string",
"value": "somestring"
},
{
"key": "somestring",
"type": "string",
"value": "somestring"
}
],
"documentation": "somestring",
"name": "somestring"
}
]
}
}
]
}
Data Structures¶
Component Model Structure¶
| delim: |
|
|---|
DataFlow Model Structure¶
A streaming pipeline.
Versioned extended inline
Inline schema:
| delim: |
|
|---|
Pipeline Model Structure¶
Tracks stream processing pipeline configuration
Inline schema:
| delim: |
|
|---|
Property Model Structure¶
| delim: |
|
|---|
Stream Model Structure¶
Inline schema:
| delim: |
|
|---|
Versioned Model Structure¶
a versioned component
| delim: |
|
|---|