Submitted by kohsuke on Wed, 2014-04-09 17:18
NIO-based Java Web Start (JNLP) slave handling is coming to 1.560. This will help you run a large number of JNLP slaves more efficiently. A connected JNLP slave used to occupy one thread on the master, but now it occupies none. Combined with the earlier change that eliminated threads from idle executors, now you can connect thousands of slaves.
All you have to do is to use the latest slave.jar from Jenkins 1.560. No other changes are necessary on users' part.
A bulk of this is implemented in remoting 2.38, and a good part of it was implemented about a year ago on the airplane on the way to Europe.
I recently had an opportunity to visit a big Jenkins user on site, and one of the things they've told me is that building projects in the Maven job type is substantially slower than doing the same with the freestyle project type.
This is partly expected, because this job type does more for you. For example, it automatically archives your build artifacts, fingerprints all the relevant information, and so on. These are good things, and naturally, it cost time.
But the slow down they are seeing was substantial, and this is a complaint I've heard from others as well. So I started looking into it.
With a help of artificial delay induced to my network interface and several custom scripts to probe into the running processes, I was able to understand what was going on and make some good improvements.
If you have slaves that connect through Java Web Start (such as slaves installed as Windows services), we have a good news for you.
In case of a connection loss, this type of slaves has been designed to automatically attempt to reconnect to the master. This makes sense because you want these slaves to remain online all the time, even if your janitor trips over the ethernet cable. Unfortunately, it also means that over the time, these slaves accumulate gunk, such as mutated static states, any left-over threads or memory leaks, or native libraries that are loaded into JVM.
(This is a guest post by Stephen Connolly)
Every developer, at some stage, will be handed a project to maintain that somebody else was responsible for. If you are lucky, the developer will not have left the organization yet and you get a brief Knowledge Transfer as the developer packs up their desk before heading on to their new job. If you are unlucky, you don't even get given the details of where the source code is hiding.
Now begins the detective work, as you try to figure out how to build and release the project, set up Jenkins jobs to build the project and run the tests…
It doesn't have to be this way, you know!
What if I told you there was a file sitting at the top level that told you exactly how to build the project and do the important things? You'd be interested, wouldn't you?
When I tell you it's the README file? “But that's all lies. Nobody keeps that up to date. Argh!!!”
But what if Jenkins reads the README file and uses it for the build definition? Now you not only have a CI system ensuring that the build definition is correct, but you have less work to do setting up the job.
What if, because the build definition is now in Source Control, you can have Jenkins create jobs for each branch with ease?