Office hours this week: Git plugin refactoring

This Wednesday's Jenkins office hours is all about the Git plugin refactoring that's going on.

Git plugin is one of the most popular plugins out there, and it's been around for quite some time. Combine that with the fact that there are so many different ways to use Git, it was inevitable that Git plugin became quite a capable but complex plugin over time. It has more than a dozen options and switches, and it was becoming harder to use and harder to maintain.

As early as 2010, some of us have already been saying that we should refactor this plugin, but none of us have managed. The good news is, I finally started tackling this problem last month while I was in London, and I've made a steadily progress since then and I'm ready for a wider review.

So we'll spend this Wednesday going over the changes.

Reducing the # of threads in Jenkins: SSH slaves

As the usage of Jenkins expands, we started seeing users who run multiple 100s of slaves on one master, and thus it became a lot more important for us to scale well to even larger number of slaves.

While I was looking at the thread dump of a large system, I started noticing that there are a large number of threads lying around pumping InputStream and writing to another OutputStream. On Linux, each thread occupies 2MB just for its stack size, so if we can eliminate some of them, it'd be a good saving.

So this morning, I tackled one source of such waste.

Jenkins has the ability to launch slave on a remote server via SSH for the longest time, and to simplify this, we've been using a pure-Java implementation of SSH client.

To cut the long story short, I was able to eliminate two pump threads per every SSH connection. Furthermore, when it runs on the upcoming Jenkins 1.521, it'll save one more thread per every SSH connection. So if you have 100 slaves connected through SSH, this alone saves up 600MB of memory. That's pretty good for a few hours work!

Jenkins User Conference Israel - Coming June 6

Your favorite Butler will visit Israel on June 6 for the Jenkins User Conference Israel. More than 200 people have already registered to hobnob with other Jenkins users and eat like kings.

The agenda is up here. You’ll find a great list of speakers from Israel, Europe and the US to compliment a plethora of treats. There will be an ice cream break, fruits break, beer break and special chef lunch.

Continuous Information - Jenkins Newsletter vol. 4

Volume 4 of Continuous Information came out last night. It contains insights and highlights from founder Kohsuke, the latest growth stats, upcoming event info, Jenkins resources, and more.

Highlights:

  • Jenkins has nearly 20,000 more active installations than it had last June, up from 43,500 to more than 61,000
  • Nearly 100 plugins have been added since late last Fall when we did the last Jenkins survey. Now there are more than 730 plugins
  • Bay Area JUC (Oct 23), JUC Israel (Jun 6) and several other Jenkins events around the world have open registration
  • Latest, greatest Jenkins improvements include a new LTS based on 1.509, more context menu improvements, and master/slave data transfer performance improvement
  • There's also a Security advisory out recommending upgrade to at least 1.502
  • A plethora of Jenkins and Continuous Delivery resources

Giving Back to the Community: 3 Ways to Keep Jenkins Growing

With more than 600 plugins, Jenkins has a vibrant community and we're dependent on YOU to keep it that way. Here are 3 ways you can give back to the community to ensure that everyone benefits and Jenkins keeps growing...

Giving it back to the community #1: vendor+community=win

Jenkins is becoming ubiquitous enough that tool vendors and service providers often find their users asking them to provide Jenkins plugins. The challenge for these companies is that they don’t necessarily possess the necessary Jenkins expertise to do one.

Here at the Jenkins project, what we are trying to do is to work with these people to deliver a plugin. It gets the job done a whole lot more quickly if the vendor brings in their expertise on their tool/services and we bring in our expertise on Jenkins plugin development.

For example, we recently worked with SOASTA to help them open-source the plugin they developed in house, then help them add a whole bunch of new functionalities. By open-sourcing a plugin in the Jenkins project, vendors win as the community helps fix bugs and improve plugins. The Jenkins project wins by building relationship with vendors. And finally the users win by having more integrations.