Skip to content
This repository has been archived by the owner on Dec 15, 2021. It is now read-only.

Commit

Permalink
Browse files Browse the repository at this point in the history
Merge pull request #296 from recena/JENKINS-31153
[JENKINS-31153] Rename Workflow to Pipeline
  • Loading branch information
jglick committed Jan 15, 2016
2 parents d16b80e + 2ca9e66 commit 118b89e
Show file tree
Hide file tree
Showing 55 changed files with 171 additions and 172 deletions.
48 changes: 24 additions & 24 deletions COMPATIBILITY.md
@@ -1,6 +1,6 @@
# Plugin Compatibility with Workflow
# Plugin Compatibility with Pipeline

For architectural reasons, plugins providing various extensions of interest to builds cannot be made automatically compatible with Workflow.
For architectural reasons, plugins providing various extensions of interest to builds cannot be made automatically compatible with Pipeline.
Typically they require use of some newer APIs, large or small (see the bottom of this document for details).
This document captures the ongoing status of plugins known to be compatible or incompatible.

Expand Down Expand Up @@ -97,11 +97,11 @@ Entries list the class name serving as the entry point to the relevant functiona
## Miscellaneous

- [X] `rebuild`: supported as of 1.24
- [X] `parameterized-trigger` (to support a workflow as downstream): supported as of 2.28
- [X] `parameterized-trigger` (to support a pipeline as downstream): supported as of 2.28
- [X] `build-token-root`: supported as of 1.2
- [ ] `build-failure-analyzer`: [JENKINS-27123](https://issues.jenkins-ci.org/browse/JENKINS-27123)
- [ ] `shelve-project`: [JENKINS-26432](https://issues.jenkins-ci.org/browse/JENKINS-26432)
- [X] `job-dsl`: Workflow creation supported as of 1.29
- [X] `job-dsl`: Pipeline creation supported as of 1.29
- [X] `zentimestamp`: basic compatibility in 4.2
- [X] `claim`: scheduled to be supported as of 2.8
- [X] `ListSubversionTagsParameterValue` (`subversion`): supported as of 2.5.6
Expand All @@ -118,25 +118,25 @@ Entries list the class name serving as the entry point to the relevant functiona

## Custom steps

For cases when a first-class Workflow step (rather than an adaptation of functionality applicable to freestyle projects) makes sense.
For cases when a first-class Pipeline step (rather than an adaptation of functionality applicable to freestyle projects) makes sense.

- [X] `docker-workflow`: DSL based on `docker` global variable
- [X] `credentials-binding`: `withCredentials` step as of 1.3
- [X] `ssh-agent`: `sshagent` step as of 1.8
- [X] `parallel-test-executor`: `splitTests` step since 1.6
- [ ] `gerrit-trigger`: [JENKINS-26102](https://issues.jenkins-ci.org/browse/JENKINS-26102), [JENKINS-26103](https://issues.jenkins-ci.org/browse/JENKINS-26103)
- [X] `mailer`: `mail` step in Workflow 1.2
- [X] `mailer`: `mail` step in Pipeline 1.2
- [ ] `artifactory`: [JENKINS-30121](https://issues.jenkins-ci.org/browse/JENKINS-30121)
- [X] `email-ext`: `emailext` step scheduled for 2.41

# Plugin Developer Guide

If you are maintaining (or creating) a plugin and wish its features to work smoothly with Workflow, there are a number of special considerations.
If you are maintaining (or creating) a plugin and wish its features to work smoothly with Pipeline, there are a number of special considerations.

## Extension points accessible via metastep

Several common types of plugin features (`@Extension`s) can be invoked from a Workflow script without any special plugin dependencies so long as you use newer Jenkins core APIs.
Then there is “metastep” in Workflow (`step`, `checkout`, `wrap`) which loads the extension by class name and calls it.
Several common types of plugin features (`@Extension`s) can be invoked from a Pipeline script without any special plugin dependencies so long as you use newer Jenkins core APIs.
Then there is “metastep” in Pipeline (`step`, `checkout`, `wrap`) which loads the extension by class name and calls it.

### General guidelines

Expand Down Expand Up @@ -166,7 +166,7 @@ If you need a `Node` where the build is running to replace `getBuiltOn`, you can

It is a good idea to replace a lengthy `@DataBoundConstructor` with a short one taking just truly mandatory parameters (such as a server location).
For all optional parameters, create a public setter marked with `@DataBoundSetter` (with any non-null default value set in the constructor or field initializer).
This allows most parameters to be left at their default values in a workflow script, not to mention simplifying ongoing code maintenance because it is much easier to introduce new options this way.
This allows most parameters to be left at their default values in a Pipeline script, not to mention simplifying ongoing code maintenance because it is much easier to introduce new options this way.

For Java-level compatibility, leave any previous constructors in place, but mark them `@Deprecated`.
Also remove `@DataBoundConstructor` from them (there can be only one).
Expand Down Expand Up @@ -196,7 +196,7 @@ public @CheckForNull String getStuff() {
```

If you want a nonblank default, it is a little more complicated.
If you do not care about XStream hygiene, for example because the `Describable` is a Workflow `Step` (or is only being used as part of one):
If you do not care about XStream hygiene, for example because the `Describable` is a Pipeline `Step` (or is only being used as part of one):

```xml
<f:entry field="stuff" title="${%Stuff}">
Expand Down Expand Up @@ -240,14 +240,14 @@ None of these considerations apply to mandatory parameters with no default, whic

See the [user documentation](scm-step/README.md) for background. The `checkout` metastep uses an `SCM`.

As the author of an SCM plugin, there are some changes you should make to ensure your plugin can be used from workflows.
As the author of an SCM plugin, there are some changes you should make to ensure your plugin can be used from pipelines.
You can use `mercurial-plugin` as a relatively straightforward code example.

#### Basic update

Make sure your Jenkins baseline is at least 1.568 (or 1.580.1, the next LTS).
Check your plugin for compilation warnings relating to `hudson.scm.*` classes to see outstanding changes you need to make.
Most importantly, various methods in `SCM` which formerly took an `AbstractBuild` now take a more generic `Run` (i.e., potentially a workflow build) plus a `FilePath` (i.e., a workspace).
Most importantly, various methods in `SCM` which formerly took an `AbstractBuild` now take a more generic `Run` (i.e., potentially a Pipeline build) plus a `FilePath` (i.e., a workspace).
Use the specified workspace rather than the former `build.getWorkspace()`, which only worked for traditional projects with a single workspace.
Similarly, some methods formerly taking `AbstractProject` now take the more generic `Job`.
Be sure to use `@Override` wherever possible to make sure you are using the right overloads.
Expand All @@ -262,7 +262,7 @@ Typically you will unconditionally return `true`.
#### Checkout key

You should override the new `getKey`.
This allows a workflow job to match up checkouts from build to build so it knows how to look for changes.
This allows a Pipeline job to match up checkouts from build to build so it knows how to look for changes.

#### Browser selection

Expand All @@ -272,15 +272,15 @@ You may override the new `guessBrowser`, so that scripts do not need to specify

If you have a commit trigger, generally an `UnprotectedRootAction` which schedules builds, it will need a few changes.
Use `SCMTriggerItem` rather than the deprecated `SCMedItem`; use `SCMTriggerItem.SCMTriggerItems.asSCMTriggerItem` rather than checking `instanceof`.
Its `getSCMs` method can be used to enumerate configured SCMs, which in the case of a workflow will be those run in the last build.
Its `getSCMs` method can be used to enumerate configured SCMs, which in the case of a pipeline will be those run in the last build.
Use its `getSCMTrigger` method to look for a configured trigger (for example to check `isIgnorePostCommitHooks`).

Ideally you will already be integrated with the `scm-api` plugin and implementing `SCMSource`; if not, now is a good time to try it.
In the future workflows may take advantage of this API to support automatic creation of subprojects for each detected branch.
In the future pipelines may take advantage of this API to support automatic creation of subprojects for each detected branch.

#### Explicit integration

If you want to provide a smoother experience for workflow users than is possible via the generic `scm` step,
If you want to provide a smoother experience for Pipeline users than is possible via the generic `scm` step,
you can add a (perhaps optional) dependency on `workflow-scm-step` to your plugin.
Define a `SCMStep` using `SCMStepDescriptor` and you can define a friendly, script-oriented syntax.
You still need to make the aforementioned changes, since at the end you are just preconfiguring an `SCM`.
Expand All @@ -289,14 +289,14 @@ You still need to make the aforementioned changes, since at the end you are just

See the [user documentation](basic-steps/CORE-STEPS.md) for background. The metastep is `step`.

To add support for use of a `Builder` or `Publisher` from a workflow, depend on Jenkins 1.577+, typically 1.580.1 ([tips](#basic-update)).
To add support for use of a `Builder` or `Publisher` from a pipeline, depend on Jenkins 1.577+, typically 1.580.1 ([tips](#basic-update)).
Then implement `SimpleBuildStep`, following the guidelines in [its Javadoc](http://javadoc.jenkins-ci.org/jenkins/tasks/SimpleBuildStep.html).
Also prefer `@DataBoundSetter`s to a sprawling `@DataBoundConstructor` ([tips](#constructor-vs-setters)).

#### Mandatory workspace context

Note that a `SimpleBuildStep` is designed to work also in a freestyle project, and thus assumes that a `FilePath workspace` is available (as well as some associated services, like a `Launcher`).
That is always true in a freestyle build, but is a potential limitation for use from a Workflow build.
That is always true in a freestyle build, but is a potential limitation for use from a Pipeline build.
For example, you might legitimately want to take some action outside the context of any workspace:

```groovy
Expand Down Expand Up @@ -342,16 +342,16 @@ Do not necessarily need any special integration, but are encouraged to use `Once

## Custom steps

Plugins can also implement custom Workflow steps with specialized behavior.
Plugins can also implement custom Pipeline steps with specialized behavior.
See [here](step-api/README.md) for more.

## Historical background

Traditional Jenkins `Job`s are defined in a fairly deep type hierarchy: `FreestyleProject``Project``AbstractProject``Job``AbstractItem``Item`.
(As well as paired `Run` types: `FreestyleBuild`, etc.)
In older versions of Jenkins, much of the interesting implementation was in `AbstractProject` (or `AbstractBuild`), which was packed full of assorted features not present in `Job` (or `Run`).
Some of these features were also needed by Workflow, like having a programmatic way to start a build (optionally with parameters), or lazy-load build records, or integrate with SCM triggers.
Others were not applicable to Workflow, like declaring a single SCM and a single workspace per build, or being tied to a specific label, or running a linear sequence of build steps within the scope of a single Java method call.
Some of these features were also needed by Pipeline, like having a programmatic way to start a build (optionally with parameters), or lazy-load build records, or integrate with SCM triggers.
Others were not applicable to Pipeline, like declaring a single SCM and a single workspace per build, or being tied to a specific label, or running a linear sequence of build steps within the scope of a single Java method call.

`WorkflowJob` directly extends `Job` since it cannot act like an `AbstractProject`.
Therefore some refactoring was needed, to make the relevant features available to other `Job` types without code or API duplication.
Expand All @@ -362,8 +362,8 @@ Each encapsulates a set of related functionality originally tied to `AbstractPro
* `SCMTriggerItem` integrates with `SCMTrigger`, including a definition of which SCM or SCMs a job is using, and how it should perform polling. It also allows various plugins to interoperate with the Multiple SCMs plugin without needing an explicit dependency. Supersedes and deprecates `SCMedItem`.
* `LazyBuildMixIn` handles the plumbing of lazy-loading build records (a system introduced in Jenkins 1.485).

For Workflow compatibility, plugins formerly referring to `AbstractProject`/`AbstractBuild` will generally need to start dealing with `Job`/`Run` but may also need to refer to `ParameterizedJobMixIn` and/or `SCMTriggerItem`.
For Pipeline compatibility, plugins formerly referring to `AbstractProject`/`AbstractBuild` will generally need to start dealing with `Job`/`Run` but may also need to refer to `ParameterizedJobMixIn` and/or `SCMTriggerItem`.
(`LazyBuildMixIn` is rarely needed from outside code, as the methods defined in `Job`/`Run` suffice for typical purposes.)

Future improvements to Workflow may well require yet more implementation code to be extracted from `AbstractProject`/`AbstractBuild`.
Future improvements to Pipeline may well require yet more implementation code to be extracted from `AbstractProject`/`AbstractBuild`.
The main constraint is the need to retain binary compatibility.
34 changes: 17 additions & 17 deletions README.md
Expand Up @@ -2,36 +2,36 @@

Building continuous delivery pipelines and similarly complex tasks in Jenkins using freestyle projects and traditional plugins can be awkward.
You need to mix Parameterized Trigger, Copy Artifact, Promoted Builds, Conditional Build Step, and more just to express what should be a simple script.
The Workflow plugin suite attempts to make it possible to directly write that script, what people often call a _workflow_ (sometimes abbreviated _flow_), while integrating with Jenkins features like slaves and publishers.
The Pipeline plugin (formerly known as Workflow) suite attempts to make it possible to directly write that script, what people often call a _pipeline_, while integrating with Jenkins features like slaves and publishers.

# Features

## Scripted control flow

Your whole workflow is a single Groovy script using an embedded DSL, possibly quite short and legible; there is no need to jump between multiple job configuration screens to see what is going on.
Your whole pipeline is a single Groovy script using an embedded DSL, possibly quite short and legible; there is no need to jump between multiple job configuration screens to see what is going on.
Conditions, loops, variables, parallel tasks, and so on are defined using regular language constructs.
At any point you can insert a shell/batch script to do “real work” (compilation, etc.).

## Useful steps

Standard DSL functions (“steps”) let you run external processes, grab slave nodes and workspaces, perform SCM checkouts, build other projects (workflow or freestyle), wait for external conditions, and so on.
Standard DSL functions (“steps”) let you run external processes, grab slave nodes and workspaces, perform SCM checkouts, build other projects (pipeline or freestyle), wait for external conditions, and so on.
Plugins can add further steps.

## Pause and resume execution

If Jenkins is restarted (intentionally, or because of a crash) while your workflow is running, when it comes back up, execution is resumed where it left off.
If Jenkins is restarted (intentionally, or because of a crash) while your Pipeline is running, when it comes back up, execution is resumed where it left off.
This applies to external processes (shell scripts) so long as the slave can be reattached, and losing the slave connection temporarily is not fatal either.

Flows can pause in the middle and wait for a human to approve something, or enter some information.
Pipelines can pause in the middle and wait for a human to approve something, or enter some information.
Executors need not be consumed while the flow is waiting.

## Pipeline stages

Workflows can be divided into sequential stages, not only for labeling but to throttle concurrency.
Pipelines can be divided into sequential stages, not only for labeling but to throttle concurrency.

# Getting started

Read the [tutorial](TUTORIAL.md) to get started writing workflows.
Read the [tutorial](TUTORIAL.md) to get started writing pipelines.

There is also a [DZone Refcard](https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow).

Expand All @@ -43,20 +43,20 @@ Releases are available on the Jenkins update center.
You need to be running a sufficiently recent Jenkins release: an LTS in the 1.580.x line or newer (currently 1.596.x for the latest updates), or a weekly release.
See the [changelog](CHANGES.md) for news.

For OSS Jenkins users, install _Workflow: Aggregator_ (its dependencies will be pulled in automatically).
For OSS Jenkins users, install _Pipeline_ (its dependencies will be pulled in automatically).
You will need to restart Jenkins to complete installation.

CloudBees Jenkins Enterprise users get Workflow automatically as of the 14.11 (1.580.1.1) release.
Otherwise install _CloudBees Workflow: Aggregator_ from the update center.
CloudBees Jenkins Enterprise users get Pipeline automatically as of the 14.11 (1.580.1.1) release.
Otherwise install _CloudBees Pipeline_ from the update center.
Again dependencies will be pulled in automatically, including all the OSS plugins.

# News & questions

* [Changelog](CHANGES.md)
* [jenkins-workflow tag](http://stackoverflow.com/tags/jenkins-workflow) on StackOverflow
* [JIRA](https://issues.jenkins-ci.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+JENKINS+AND+resolution+%3D+Unresolved+AND+%28component+%3D+workflow-plugin+OR+labels+in+%28workflow%29%29+ORDER+BY+component+ASC,+key+DESC&mode=hide) (file issues in the `workflow-plugin` component, or other components with the `workflow` label)
* [User list discussions](https://groups.google.com/forum/#!topicsearchin/jenkinsci-users/workflow) (mention `workflow` in the subject)
* [#JenkinsWorkflow](https://twitter.com/hashtag/JenkinsWorkflow) on Twitter
* [User list discussions](https://groups.google.com/forum/#!topicsearchin/jenkinsci-users/pipeline) (mention `pipeline` in the subject)
* [#JenkinsPipeline](https://twitter.com/hashtag/JenkinsPipeline) on Twitter

# Demo

Expand Down Expand Up @@ -119,14 +119,14 @@ The snapshot Docker demo is mainly useful for verifying the effect of ongoing ch

While the implementation is divided into a number of plugins, for ease of prototyping they are all kept in one repository using snapshot dependencies.

* `step-api` defines a generic build step interface (not specific to flows) that many plugins could in the future depend on.
* `step-api` defines a generic build step interface (not specific to pipelines) that many plugins could in the future depend on.
* `basic-steps` add some generic step implementations. There is [more documentation there](basic-steps/CORE-STEPS.md).
* `api` defines the essential aspects of flows and their executions. In particular, the engine running a flow is extensible and so could in the future support visual orchestration languages.
* `support` adds general implementations of some internals needed by flows, such as storing state.
* `job` provides the actual job type and top-level UI for defining and running flows.
* `api` defines the essential aspects of pipelines and their executions. In particular, the engine running a flow is extensible and so could in the future support visual orchestration languages.
* `support` adds general implementations of some internals needed by pipelines, such as storing state.
* `job` provides the actual job type and top-level UI for defining and running pipelines.
* `durable-task-step` uses the `durable-task` plugin to define a shell script step that can survive restarts.
* `scm-step` adds SCM-related steps. There is [more documentation there](scm-step/README.md).
* `cps` is the flow engine implementation based on the Groovy language, and supporting long-running flows using a _continuation passing style_ transformation of the script.
* `cps` is the flow engine implementation based on the Groovy language, and supporting long-running pipelines using a _continuation passing style_ transformation of the script.
* `cps-global-lib` adds a Git-backed repository for Groovy libraries available to scripts.
* `stm` is a simple engine implementation using a _state transition machine_, less intended for end users than as a reference for how engines can work. Currently only partly implemented.
* `aggregator` is a placeholder plugin allowing you to `mvn hpi:run` and see everything working together, as well as holding integration tests.

0 comments on commit 118b89e

Please sign in to comment.