Plugins

Overview

stoQ is a highly flexible framework because of its ability to leverage plugins for each layer of operations. One of the biggest benefits to this approach is that it ensures the user is able to quickly and easily pivot to and from different technologies in their stack, without having to drastically alter workflow. To better understand the stoQ workflow and how each of the below plugins are used, check out the stoQ workflow section.

For a full listing of all publicly available plugins, check out the stoQ public plugins repository.

Configuration

Plugins may be provided configuration options in one of four ways. In order of precendece:

  • From the command line
  • Upon instantiation of Stoq()
  • Defined in stoq.cfg
  • Defined in the plugin’s .stoq configuration file

Command Line

When running stoq from the command line, simply add --plugin-opts to your arguments followed by the desired plugin options. The syntax for plugin options is:

plugin_name:option=value

For example, if we want to tell the plugin dirmon to monitor the directory /tmp/monitor for new files by setting the option source_dir, the syntax would be:

dirmon:source_dir=/tmp/monitor

Instantiation

When using stoQ as a framework, plugin options may be defined when instantiating Stoq using the plugin_opts argument:

>>> from stoq import Stoq
>>> plugin_options = {'dirmon': {'source_dir': '/tmp/monitor'}}
>>> s = Stoq(plugin_opts=plugin_options)

stoq.cfg

The recommended location for storing static plugin configuration options is in stoq.cfg. The reason for this if all plugin options defined in the plugin’s .stoq file will be overwritten when the plugin is upgraded.

To define plugin options in stoq.cfg simply add a section header of the plugin name, then define the plugin options:

[dirmon]
source_dir = /tmp/monitor

Plugin .stoq configuration file

Each plugin must have a .stoq configuration file. The configuration file resides in the same directory as the plugin module. The plugin’s configuration file allows for configuring a plugin with default or static settings. The configuration file is a standard YAML file and is parsed using the configparser module. The following is an example plugin configuration file with all required fields:

[Core]
Name = example_plugin
Module = example_plugin

[Documentation]
Author = PUNCH Cyber
Version = 0.1
Website = https://github.com/PUNCH-Cyber/stoq-plugins-public
Description = Example stoQ Plugin
  • Core
    • Name: The plugin name that stoQ will use when calling the plugin. This must be unique.
    • Module: The python module that contains the plugin (without the .py extension).
  • Documentation
    • Author: Author of the plugin
    • Version: Plugin version
    • Website: Website where the plugin can be found
    • Description: Description of the plugins utility

Additionally, some optional settings may be defined:

[options]
min_stoq_version = 2.0.0
  • options
    • min_stoq_version: Minimum version of stoQ required to work properly. If the version of stoQ is less than the version defined, a warning will be raised.

Note

Plugin options must be under the [options] section header to be accessible via the other plugin configuration options.

Warning

Plugin configuration options may be overwritten when a plugin is upgraded. Upgrading plugins is a destructive operation. This will overwrite/remove all data within the plugins directory, to include the plugin configuration file. It is highly recommended that the plugin directory be backed up regularly to ensure important information is not lost, or plugin configuration options be defined in stoq.cfg.

Multiclass Plugins

Plugins that are of more than one plugin class are called Multiclass Plugins. Multiclass plugins help to simplify and centralize plugin code. Development is nearly identical to creating a regular plugin.In order to create a Multiclass plugin, the plugin must be a subclass of one or more plugin class.

In this example, we will create a Multiclass plugin that is a worker as well as a dispatcher plugin. We simply need to subclass our plugin class with WorkerPlugin and DispatcherPlugin and ensure the scan (required for worker plugins) and get_dispatches (required for dispatcher plugins) methods exist.:

from typing import Optional
from stoq.plugins import DispatcherPlugin, WorkerPlugin

class MultiClassPlugin(WorkerPlugin, DispatcherPlugin):
    def scan(
        self, payload: Payload, request_meta: RequestMeta
    ) -> Optional[WorkerResponse]:
        # do worker plugin stuff here
        return

    def get_dispatches(
        self, payload: Payload, request_meta: RequestMeta
    ) -> Optional[DispatcherResponse]:
        # do dispatcher plugin stuff here
        return