Documenting the Dropbox Connector¶

Documentation of a Connector¶

To explain the use of the Documentation Generator tool, we use the Dropbox Connector of the Connector SDK as an example of how to document a connector using the Documentation Generator.

The Dropbox Connector is a publicly available Cloud Studio connector that is described in the Connector SDK tutorial.

This example assumes that you have downloaded and installed the Documentation Generator and are familiar with Cloud Studio, its connectors, and the general structure of the documentation of a Cloud Studio connector, as can be seen at individual Cloud Studio Connectors.

In this example, we look at the structure of the documentation that needs to be generated, the structure of the connector documentation template YAML file, and the steps to go through to create the documentation.

Documentation Directory Structure¶

At Jitterbit, we document connectors following a convention and pattern based on the structure of a connector itself. This is a summary of the directory and file structure that is followed:

• <connector_name>: Directory of the connector

  • docs: Directory of documentation of the connector.

    • assets: Directory of images and any non-Markdown files

      • <connector_name>-activities.png: Design component palette showing an endpoint and all of its activities
      • <connector_name>-configuration.png: Connection configuration screen
      • <connector_name>-<a-activity>-1.png: Step 1, the A Activity configuration screen
        . . .
      • <connector_name>-<a-activity>-n.png: Step n, the A Activity configuration screen
      • connector_name-<a-activity>-operation.png: The A Activity operation example
      • connectivity-tab.png: Connectivity tab of the design component palette
      • refresh-icon.png: Icon image file of the refresh icon
      • variable-icon.png: Icon image file of the variable icon

    • <a>-activity.md: Describes the A Activity of the connector, used for a page such as https://example.com/<connector+name>+<a>+activity

    • <b>-activity.md: Describes the B Activity of the connector
      . . .
    • <z>-activity.md: Describes the Z Activity of the connector
    • connection.md: Describes the connector’s connection configuration
    • index.md: Main index page for the connector

  • README.md: Overall README of the connector for display at GitHub.com. This page is not displayed publicly but is used at GitHub.com when the files are displayed.

For an example implementation, see any connector such as the Amazon Redshift connector.

The purpose of this example is to create all of these files for the Dropbox connector by using the Documentation Generator and the Dropbox Connector’s adapter.json.

Connector Adapter JSON File¶

All connectors used in Cloud Studio are described by a file, named by convention as adapter.json. This file is typically located in the ui directory of the root of the connector source code. A connector can use a JSON file with a different name that serves the same purpose. If so, it will be identified in the manifest for the connector. Use that file as a subsitute.

If you do not have a copy of the files of the Dropbox Connector, download its adapter.json file from its GitHub repository. That is the only file required for this example.

Initial Documentation Generator Configuration¶

The first time you use the Documentation Generator you must specify the location of where you are building the documentation. This is referred to as the “source directory for connectors documentation.”

The directory you specify can be either a new location or an existing directory. It is the location where the generation directory will be created. By convention, the Jitterbit documentation uses the location of our existing connector documentation.

If an existing connector documentation source directory is used (one that contains directories named after each of the different connectors), the Documentation Generator will be able to copy Markdown and image files back and forth between the existing and generated documentation directories. This can speed up editing and review of the documentation prior to submission to source control.

Loading the Dropbox Adapter JSON File¶

Having set the source directory directory, you can now load an adapter.json. If you load the Dropbox connector adapter.json, and it was successfully read, the connector name Dropbox will appear in section 1.0 of the Documentation Generator:

Creating the Template YAML File¶

With the adapter.json loaded, you can now create the Connector Documentation Template YAML file. It will be placed in the generation/<connector_name>-connector directory, and will be named following the pattern <connector_name>-connector_documentation_template.yaml.

In the case of the Dropbox connector, that will be generation/dropbox-connector/dropbox-connector_documentation_template.yaml.

The YAML file will automatically be loaded and displayed in section 2.0 with the name of the connector.

The full path to the template can be seen if you hover the cursor over the file name:

You can edit the template file by clicking Edit Documentation Template, which will open the file in the application you have configured as your default YAML file editor.

Editing the Documentation Template¶

This section describes the contents of the YAML file and how to edit it.

As described in Template YAML File, the template is composed of two portions. The top portion (referred to as the edit template) is to contain any changes that can and should be made. The bottom portion (referred to as the base template) has additional settings that have been gleaned from the connector but should not in the normal course of usage be edited.

When a template is first created, the activity_sort_order and activity_types sections have entries that require adjusting, as these can be set only after looking at the connector images. The connection image shows the order of the activities, and the last step of each activity shows if there are data schemas in the requests and responses returned by the connector.

Also, many entries will have PLACEHOLDER_TEXT appended to indicate that the text was generated from using a placeholder for that entry obtained from the adaptor.json. Thus, it needs review and editing by you. Check for all instances of PLACEHOLDER_TEXT and both remove them and edit their entries.

At this stage, edit only the endpoint_connection_fields, substitutions, activity_sort_order, and activity_types sections, as the text of the activities will be need to be adjusted based on the settings of the activity_types.

Edit each of these sections and then click Regenerate to regenerate the template based on the new settings:

This will adjust the text in each activity as appropriate, such as source or target, and the construction of the ACTIVITY_OPERATION_USE_CASE substitution.

Using Snippets¶

Snippets can be used in cases where there is complicated or repetitive text that would be difficult to add to the template file.

A snippet is entered by beginning the entry with the snippet notation:

--8<--

followed with the name or path of a file. If the path does not begin with snippets, the snippet will be looked for relative to the snippets/connectors/<connector-name>/ directory.

These are valid snippets:

--8<-- copy_data-schema.md

--8<-- "snippets/connectors/dropbox-connector/copy_data-schema.md"

Snippets can be used in all fields. There are some extra fields that are included with the intent of adding text at certain specific locations of the documentation:

The entries used here can be either text or snippets. Note that any snippet is assumed to be flush to the left margin. For example: the text of the ACTIVITY_DATA_SCHEMAS field is added after the data schema step, and is always indented on the left margin in the final presentation. However, any text (either directly or as a snippet) is not to include that indent but should be flush with the left margin. Appropriate spaces will be added to each line of the snippet to make it the correct indentation.

Completing the Template and Generating Markdown Files¶

With the template regenerated, you can now edit each activity and its steps, adding appropriate entries and searching for and removing all instances of PLACEHOLDER_TEXT.

When finished editing, click Generate Markdown from Documentation Template to load the template file and generate Markdown and image placeholders files into a directory of the generation directory.

The directory will be based on the name of the connector, as specified in the documentation_directory field of the template, and will be of the form generation/<connector-name>, such as generation/dropbox-connector.

As the template is reloaded by the Documentation Generator before generation, it’s important to save the template and any edits before generation.

Include Configuration Images determines if the documentation generation includes generating placeholder images for each step and the connection. If this box is not selected, the documentation will be generated without reference to images.

Copying Files¶

To make viewing of the final documentation easier, you can copy the final images into the generated documentation directory. If the images are already in the final directory (in a documentation_directory/<connector-name> directory), you can use Copy Images of Current Connector to Generation Directory to place copies in the generated directory and make the images visible when previewing the documentation.

Conversely, you can manually place the image files in the generation/documentation_directory and then use Copy Generated Documentation into Connectors Documentation Source Directory with the option of Copy Images from Generation Directory to make copies of the generated images in the documentation_directory.

This last control is what is generally used to install the final reviewed Markdown files and images into the correct location prior to submission. As that is typically how you would use this, by default Copy Images from Generation Directory is already selected.

Note that copying is additive only. If you have a file (either Markdown or an image file) in a <connector-name> directory in the documentation directory that does not exist in the matching generated/<connector-name> directory of the documentation directory, using these controls will not remove those files. You will need to manually remove them.

For More Information¶

Though this tool is unsupported, we are happy to receive any feedback, questions, or concerns. You can reach the Jitterbit Documentation Team at documentation@jitterbit.com.