Common Interface

To exchange data between your component and Keboola, use

• a predefined set of input and output folders for tables and files,
• a configuration file,
• environment variables and return values.

Optionally, you can use

• logging,
• manifest files for working with table and file meta-data,
• the OAuth part of the configuration file, and
• actions for quick synchronous tasks,
• artifacts to pass extra files between jobs (preview feature).

Sensitive configuration values are protected by encryption, and third-party access by OAuth2 authorization.

To quickly get the picture, have a look a random sample data folder.

Our Python library PyPi (opens in a new tab) provides a convenient way how to interact with the Common Interface. Learn more in the documentation (opens in a new tab).

Component Limits

Even though you can define your own limits for your component, all components are also subject to the following service limits:

• Both memory and swap sizes are set to an equal value
• Docker overlay2 (opens in a new tab) size is set to 10 GB

The size allocated for overlay2 is consumed by memory swapping, and all other operations in the component (for instance, ad hoc module installations); only input and output folders (/data/) and /tmp/ are excluded. As the swap size cannot be larger than the allocated disk space, we cannot safely increase the memory limit over 8 GB.

If you need more than 8 GB of memory/swap or larger disk space, get in touch with us to discuss possible solutions.