Skip to content

Commit

Permalink
[FIXED JENKINS-27097] use ISO-8601 in changelog
Browse files Browse the repository at this point in the history
Use an explicit format string (to mimic raw format)
to enforce ISO-8601 date format.
Side benefit is this format string should not be
impacted if a future/exotic git CLI do introduce
some subtle changes in format=raw output.
  • Loading branch information
ndeloof committed Feb 24, 2015
1 parent 130748a commit de49c9b
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 3 deletions.
Expand Up @@ -699,7 +699,11 @@ public void prune(RemoteConfig repository) throws GitException, InterruptedExcep
*/
public ChangelogCommand changelog() {
return new ChangelogCommand() {

/** Equivalent to the git-log raw format but using ISO 8601 date format - also prevent to depend on git CLI future changes */
public static final String RAW = "commit %H%ntree %T%nparent %P%nauthor %aN <%aE> %ai%ncommitter %cN <%cE> %ci%n%n%w(76,4,4)%s%n%n%b";
final List<String> revs = new ArrayList<String>();

Integer n = null;
Writer out = null;

Expand Down Expand Up @@ -736,7 +740,8 @@ public void abort() {
}

public void execute() throws GitException, InterruptedException {
ArgumentListBuilder args = new ArgumentListBuilder(gitExe, "whatchanged", "--no-abbrev", "-M", "--pretty=raw");
ArgumentListBuilder args = new ArgumentListBuilder(gitExe, "whatchanged", "--no-abbrev", "-M");
args.add("--format="+RAW);
if (n!=null)
args.add("-n").add(n);
for (String rev : this.revs)
Expand Down
10 changes: 8 additions & 2 deletions src/main/java/org/jenkinsci/plugins/gitclient/JGitAPIImpl.java
Expand Up @@ -43,6 +43,7 @@

import javax.annotation.Nullable;

import org.apache.commons.lang.time.FastDateFormat;
import org.eclipse.jgit.api.AddNoteCommand;
import org.eclipse.jgit.api.CommitCommand;
import org.eclipse.jgit.api.CreateBranchCommand.SetupUpstreamMode;
Expand Down Expand Up @@ -1158,6 +1159,8 @@ private String statusOf(DiffEntry d) {
}
}

public static final String ISO_8601 = "yyyy-MM-dd'T'HH:mm:ssZ";

/**
* Formats a commit into the raw format.
*
Expand All @@ -1176,8 +1179,11 @@ void format(RevCommit commit, @Nullable RevCommit parent, PrintWriter pw) throws
pw.printf("tree %s\n", commit.getTree().name());
for (RevCommit p : commit.getParents())
pw.printf("parent %s\n",p.name());
pw.printf("author %s\n", commit.getAuthorIdent().toExternalString());
pw.printf("committer %s\n", commit.getCommitterIdent().toExternalString());
FastDateFormat iso = FastDateFormat.getInstance(ISO_8601);
PersonIdent a = commit.getAuthorIdent();
pw.printf("author %s <%s> %s\n", a.getName(), a.getEmailAddress(), iso.format(a.getWhen()));
PersonIdent c = commit.getCommitterIdent();
pw.printf("committer %s <%s> %s\n", c.getName(), c.getEmailAddress(), iso.format(c.getWhen()));

// indent commit messages by 4 chars
String msg = commit.getFullMessage();
Expand Down

10 comments on commit de49c9b

@MarkEWaite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deployed a pre-release version of the git client plugin onto my work server which included this change. I think this change may be what caused one of my previously functioning jobs to fail with the stack trace:

FATAL: String index out of range: -1
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
        at java.lang.String.substring(String.java:1911)
        at hudson.plugins.git.GitChangeSet.isoDateFormat(GitChangeSet.java:179)
        at hudson.plugins.git.GitChangeSet.parseCommit(GitChangeSet.java:118)
        at hudson.plugins.git.GitChangeSet.<init>(GitChangeSet.java:89)
        at hudson.plugins.git.GitChangeLogParser.parseCommit(GitChangeLogParser.java:73)
        at hudson.plugins.git.GitChangeLogParser.parse(GitChangeLogParser.java:57)
        at hudson.plugins.git.GitChangeLogParser.parse(GitChangeLogParser.java:44)
        at hudson.plugins.git.GitChangeLogParser.parse(GitChangeLogParser.java:25)
        at hudson.scm.ChangeLogParser.parse(ChangeLogParser.java:57)
        at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:894)
        at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:862)
        at hudson.model.AbstractBuild.getCulprits(AbstractBuild.java:349)
        at hudson.model.AbstractBuild.getCulprits(AbstractBuild.java:346)
        at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:687)
        at hudson.model.Run.execute(Run.java:1770)
        at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
        at hudson.model.ResourceController.execute(ResourceController.java:89)
        at hudson.model.Executor.run(Executor.java:240)

I will investigate further as time allows. My first attempt will be to revert this change on a private branch, just to see if that resolves the issue. If it does, then I may investigate further to write a test which exercises the failure. If I'm successful at writing the failing test, then I might try fixing the problem.

@ndeloof
Copy link
Contributor Author

@ndeloof ndeloof commented on de49c9b Mar 3, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MarkEWaite ok, thanks for investigating on this. Can you also please run git log with the same format string and paste here to compare the actual output with expected ISO format ?

@MarkEWaite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will attempt it. My initial test was to run the same job again.

The second time I ran the job (a multi-configuration job), it moved the error message from the flyweight executor's log to the logs in the jobs in the various configurations. The error message disappeared from the flyweight job.

The third time I ran the job, it resolved the error message from most of the configurations. A few (one or two) failed, but most succeeded.

The fourth time I ran the job, the error message was resolved in all configurations.

I'll attempt to print changelog entries from all the changes in the repository I was using, in case the problem is somehow specific to the changelog of that repository.

@MarkEWaite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a custom build based on git plugin 2.3.5 (see this branch - knowing that I'll delete that branch in a week or two...)

With that change applied to my running Jenkins, GitChangeSet code reported:

Exception avoided on isoDateFormat('2015-03-03T09:22:42-0700')
Exception avoided on isoDateFormat('2015-03-03T09:22:42-0700')

I think that means the check for ISO 8601 format in the git plugin needs to do more than a length check before deciding to skip the parsing and formatting.

@ndeloof
Copy link
Contributor Author

@ndeloof ndeloof commented on de49c9b Mar 6, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the standard indeed both defines +hhmi and +hh:mi, also Z could be used by people using UTC

I suggest we use:

private String isoDateFormat(String s) {
        if (NumberUtils.isDigits(s)) {
            // legacy mode
            int i = s.indexOf(' ');
            long time = Long.parseLong(s.substring(0, i));
            return FastDateFormat.getInstance(ISO_8601).format(new Date(time * 1000)) + s.substring(i);
        } else {
            // already in ISO format
            return s;
        }
}

@MarkEWaite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're going in the right direction with that proposed change, but I don't think it will handle the case that exists in the current code. In the current code, the pattern is assumed to be a sequence of digits followed by a space followed by what I believe is assumed to be time zone information.

The NumberUtils.isDigits(s) test will cause that sequence of digits, a space, and time zone, to now return the string without formatting it as an ISO-8601 date format.

How about:

private String isoDateFormat(String s) {
    int spaceIndex = s.indexOf(' ');
    if (spaceIndex > 0 && NumberUtils.isDigits(s.substring(0, spaceIndex))) {
        // legacy mode
        long time = Long.parseLong(s.substring(0, spaceIndex));
        return FastDateFormat.getInstance(ISO_8601).format(new Date(time * 1000)) + s.substring(spaceIndex);
    } else {
        // already in ISO format
        return s;
    }
}

I need more time to think about that proposed rework, and more time to write tests around the various conditions which can exist. There is already at least one bug reporting that the date information returned by one of the API calls is incorrect. This seems like the chance to write several tests around that bug and confirm that it is fixed and remains fixed.

@ndeloof
Copy link
Contributor Author

@ndeloof ndeloof commented on de49c9b Mar 6, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this assumes timezone will always be set. I suggest :

    String date = s;
    String timezone = "";
    int spaceIndex = s.indexOf(' ');
    if (spaceIndex > 0) {
        date = s.substring(0, spaceIndex);
        timezone = s.substring(spaceIndex+1);
    }
    if (NumberUtils.isDigits(date)) {
        // legacy mode
        long time = Long.parseLong(date);
        return FastDateFormat.getInstance(ISO_8601).format(new Date(time * 1000)) + timezone;
    } else {
        // already in ISO format
        return s;
    }
}

@MarkEWaite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. I like that way of expressing the solution very much.

I'd really like tests around that code before we make a change, but the problem may already exist in the interaction between git-plugin 2.3.5 and git-client-plugin 1.16.1. We may not have time to wait for me to write tests of the various combinations of date and time zone. I've seen this code break my builds in several different cases, on different Jenkins servers. I suspect we don't have very long before other users will encounter the same problem I encountered.

Will you have time to write some tests of that code?

@ndeloof
Copy link
Contributor Author

@ndeloof ndeloof commented on de49c9b Mar 6, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do this early next week.

@MarkEWaite
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Please sign in to comment.