How APIs/Plugins Are Run
This documentation isn’t up to date with the latest version of Gatsby.
- mention how multiple configurations are merged
- the node creation flow in the diagram is no longer correct
onCreateNodeare handled differently than described
You can help by making a PR to update this documentation.
For most sites, plugins take up the majority of the build time. So what’s really happening when APIs are called?
Note: this section only explains how
gatsby-node plugins are run. Not browser or ssr plugins
Early in the bootstrap phase, you load all the configured plugins (and internal plugins) for the site. These are saved into redux under the
flattenedPlugins namespace. Each plugin in redux contains the following fields:
- resolve: absolute path to the plugin’s directory
- id: String concatenation of ‘Plugin ’ and the name of the plugin. E.g.
- name: The name of the plugin. E.g.
- version: The version as per the package.json. Or if it is a site plugin, one is generated from the file’s hash
- pluginOptions: Plugin options as specified in gatsby-config.js
- nodeAPIs: A list of node APIs that this plugin implements. E.g.
[ 'sourceNodes', ...]
- browserAPIs: List of browser APIs that this plugin implements
- ssrAPIs: List of SSR APIs that this plugin implements
In addition, you also create a lookup from api to the plugins that implement it and save this to redux as
api-to-plugins. This is implemented in load-plugins/validate.js
Some API calls can take a while to finish. So every time an API is run, you create an object called apiRunInstance to track it. It contains the following notable fields:
- id: Unique identifier generated based on type of API
- api: The API you’re running. E.g.
- args: Any arguments passed to
api-runner-node. E.g. a node object
- pluginSource: optional name of the plugin that initiated the original call
- resolve: promise resolve callback to be called when the API has finished running
- startTime: time that the API run was started
- span: opentracing span for tracing builds
- traceId: optional args.traceId provided if API will result in further API calls (see below)
Immediately place this object into an
apisRunningById Map, where you track its execution.
Next, filter all
flattenedPlugins down to those that implement the API you’re trying to run. For each plugin, you require its
gatsby-node.js and call its exported API function. E.g. if API was
sourceNodes, it would result in a call to
API implementations are passed a variety of useful actions and other interesting functions/objects. These arguments are created each time a plugin is run for an API, which allows us to rebind actions with default information.
All actions take 3 arguments:
- The core information required by the action. E.g. for createNode, you must pass a node
- The plugin that is calling this action. E.g.
createNodeuses this to assign the owner of the new node
- An object with misc action options:
Passing the plugin and action options on every single action call would be extremely painful for plugin/site authors. Since you know the plugin, traceId and parentSpan when you’re running your API, you can rebind injected actions so these arguments are already provided. This is done in the doubleBind step.
Each plugin is run inside a map-series promise, which allows them to be executed concurrently. Once all plugins have finished running, you remove them from apisRunningById and fire a
API_RUNNING_QUEUE_EMPTY event. This in turn, results in any dirty pages being recreated, as well as their queries. Finally, the results are returned.
The majority of API calls result in one or more implementing plugins being called. You then wait for them all to complete, and return. But some plugins (e.g. sourceNodes) result in calls to actions that themselves call APIs. You need some way of tracing whether an API call originated from another API call, so that you can wait on all child calls to complete. The mechanism for this is the
The traceID is passed as an argument to the original API runner. E.g
You keep track of the number of API calls with this traceId in the apisRunningByTraceId Map. On this first invocation, it will be set to
Using the action rebinding mentioned above, the traceId is passed through to all action calls via the
After reducing the Action, a global event is emitted which includes the action information
CREATE_PAGEevents, you need to call the
onCreatePageAPIs respectively. The plugin-runner takes care of this. It also passes on the traceId from the Action back into the API call.
You’re back in
api-runner-node.jsand can tie this new API call back to its original. So you increment the value of apisRunningByTraceId for this traceId.
Now, whenever an API finishes running (when all its implementing plugins have finished), you decrement
apisRunningByTraceId[traceId]. If the original API call included the
waitForCascadingActionsoption, then you wait until
apisRunningByTraceId[traceId]== 0 before resolving.