Skip to content

Commit

Permalink
[FIXED JENKINS-19392] Simpler and more efficient to use String.intern…
Browse files Browse the repository at this point in the history
… to compact fingerprint data.

This should work across builds and even across jobs, and does not force us to load other builds.
  • Loading branch information
jglick committed Dec 22, 2014
1 parent 47fa17b commit 2f59ffc
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 38 deletions.
3 changes: 3 additions & 0 deletions changelog.html
Expand Up @@ -55,6 +55,9 @@
<!-- Record your changes in the trunk here. -->
<div id="trunk" style="display:none"><!--=TRUNK-BEGIN=-->
<ul class=image>
<li class=bug>
Fingerprint compaction aggravated lazy-loading performance issues.
(<a href="https://issues.jenkins-ci.org/browse/JENKINS-19392">issue 19392</a>)
<li class=bug>
Possible unreleased workspace lock if SCM polling fails during setup.
(<a href="https://issues.jenkins-ci.org/browse/JENKINS-23568">issue 23568</a>)
Expand Down
46 changes: 8 additions & 38 deletions core/src/main/java/hudson/tasks/Fingerprinter.java
Expand Up @@ -294,8 +294,7 @@ public static final class FingerprintAction implements RunAction2 {

public FingerprintAction(Run build, Map<String, String> record) {
this.build = build;
this.record = PackedMap.of(record);
compact();
this.record = compact(record);
}

@Deprecated
Expand All @@ -306,9 +305,8 @@ public FingerprintAction(AbstractBuild build, Map<String, String> record) {
public void add(Map<String,String> moreRecords) {
Map<String,String> r = new HashMap<String, String>(record);
r.putAll(moreRecords);
record = PackedMap.of(r);
record = compact(r);
ref = null;
compact();
}

public String getIconFileName() {
Expand Down Expand Up @@ -341,48 +339,20 @@ public Map<String,String> getRecords() {

@Override public void onLoad(Run<?,?> r) {
build = r;
compact();
record = compact(record);
}

@Override public void onAttached(Run<?,?> r) {
// for historical reasons this setup is done in the constructor instead
}

private void compact() {
// share data structure with nearby builds, but to keep lazy loading efficient,
// don't go back the history forever.
if (rand.nextInt(2)!=0) {
Run pb = build.getPreviousBuild();
if (pb!=null) {
FingerprintAction a = pb.getAction(FingerprintAction.class);
if (a!=null)
compact(a);
}
}
}

/**
* Reuse string instances from another {@link FingerprintAction} to reduce memory footprint.
*/
protected void compact(FingerprintAction a) {
Map<String,String> intern = new HashMap<String, String>(); // string intern map
for (Entry<String, String> e : a.record.entrySet()) {
intern.put(e.getKey(),e.getKey());
intern.put(e.getValue(),e.getValue());
}

Map<String,String> b = new HashMap<String, String>();
/** Share data structure with other builds, mainly those of the same job. */
private PackedMap<String,String> compact(Map<String,String> record) {
Map<String,String> b = new HashMap<String,String>();
for (Entry<String,String> e : record.entrySet()) {
String k = intern.get(e.getKey());
if (k==null) k = e.getKey();

String v = intern.get(e.getValue());
if (v==null) v = e.getValue();

b.put(k,v);
b.put(e.getKey().intern(), e.getValue().intern());
}

record = PackedMap.of(b);
return PackedMap.of(b);
}

/**
Expand Down

4 comments on commit 2f59ffc

@daniel-beck
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this result in perm gen memory issues?

@jglick
Copy link
Member Author

@jglick jglick commented on 2f59ffc Dec 23, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Java 6/7 you mean (Java 8 is fine)—perhaps. I suspect you would run low on heap and start ejecting Runs well before then, which would allow Strings from the fingerprint maps to be collected.

I actually initially wrote this to keep a custom intern set but decided to go with the somewhat simpler approach of String.intern and let the JVM manage it. If you think it worthwhile I can resurrect the original code. It has the downside that pseudo-interned Strings cannot ever be collected, so on Java 8 it would be strictly worse than the current code.

@daniel-beck
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know the details well enough to judge this. Maybe something to keep in mind when the issue actually comes up on real installs. Jenkins only requires Java 6 after all, and Java 8 isn't even in some of the default Linux package repos.

@jglick
Copy link
Member Author

@jglick jglick commented on 2f59ffc Dec 23, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option is to just delete the compaction altogether (or at least on Java 7-). With the introduction of lazy-loading the intent is that most of the Runs will not actually be in memory, so the original purpose of this code becomes less relevant.

Please sign in to comment.