You are here

Buildbot

Where are bee packages located, and how are they built?

Buildbot is located on Kyle's server under my account. These are instructions provided by Seb, the original jallib buildbot master. I have taken the responsibility. I'll write up more and edit it when I can.

Bee packages are generated here: ~/jallib/slave/bee/build/tools/distrib/
 
Buildbot config: ~/jallib/master/master.cfg
 
You'll see a line like this:

weekly = Nightly("weekly build (bee)",["jallib-bee"],minute=0,hour=6,dayOfWeek=6)

this is the weekly "bee" builder": runs every sunday, at 6am (so uploaded here in europe at 14/15pm.

(and btw, you also have: 

scheduler = Scheduler(name="blamer", branch=None,treeStableTimer=8*60*60,builderNames=["jallib-standard"])

This is other builder, it runs each there were a change in SVN, and since this nothing has changed (no commit) for 8 hours (avoid building stuff for nothing when people commit several time))

Few lines later:

bee_factory = factory.BuildFactory()

bee_factory.addStep(SVN(svnurl=svnurl))

 

bee_factory.addStep(package_step)

So: bee will update its SVN in jallib/slave/bee/build on the slave side (SVN step) and perform package_step. This is one declared as:

package_step = Package(

                command="cd tools && make bee VERSION=bee",

                env={'LANG' : 'C',

                        'JALLIB_JALV2' : JALLIB_JALV2,

                        'JALLIB_PYTHON' : JALLIB_PYTHON,

                        'PYTHONPATH' : PYTHONPATH,

                        #'UPLOAD_USER' : ''

                        #'UPLOAD_PASSWD' : ''

                        },

                timeout=7200,

 

                )

Most important stuff:  cd tools && make bee VERSION=bee. if you're under linux, you can do this on your computer, it will generate a bee package. So buildbot just trigger a makefile target, that's all.

Note: you also have:

c['status'].append(JallibMailNotifier(fromaddr="build[at]sirloon[dot]net",

                                     extraRecipients=["jallib[at]googlegroups[dot]com"],

                                     ##extraRecipients=["sebastien.lelong[at]gmail[dot]com"],

 

                                     sendToInterestedUsers=False,addLogs=True))

This is what sends email to the group (I just manually modified the template in mailnotifier module). The important thing here is if you're testing stuff, put your email address only, to avoid spamming the group.

Any changes here need to restart the master. you can "make stop && make start", or if you're lazy like me, use my scripts :)

 

stop_buildbot.sh

 

start_buildbot.sh

This will handle all masters and all slaves. I highly recommend this.

So, that's for the master. the slave is configured elsewhere, it will link to the master and receive orders, like "build bee package". See: ~/jallib/slave/buildbot.tac

This file is specific to twisted (this is python network framework buildbot is running on), you should never have to change it, this is very generic. Except the path "basedir = r'/home/mattschinkel/jallib/slave'". When I moved buildbot to your home, I change it.

So this declares a slave, it will connect to the master, the master will trigger builds and sends commands. Since there 2 main builders (standard and bee), you'll see "standard" and "bee" directory. These are the roots of builder. In each, there's a "build" directory, and then the SVN root. Each builder has its own SVN tree so they are independent and can run at the same time. Each commit will get updated here. NEVER change a file locally in a builder's SVN, you may face merge issue (conflicts) when someone change also this file, and, most importantly, buildbot will build something different from what is in the SVN (that's not the goal :)).

Note: if you need cleaning, you can delete "bee" and "standard" directory. stop/start all masters/slaves (with my script), the builders will checkout a fresh SVN tree.

When you need to update compiler version, just install it in your $HOME/bin directory then change this line in master.cfg:

 

JALLIB_JALV2 = '/home/mattschinkel/bin/jalv24q2'

Also remember master's environnment is pushed to slave's environment. You'll see stuff like 

                env={'LANG' : 'C',

                        'JALLIB_JALV2' : JALLIB_JALV2,

 

                        'JALLIB_PYTHON' : JALLIB_PYTHON},

Every scripts launch from Makefile expect these variables, that's how it's specified from the master. (you won't have to change this, that's just for information)

Finally, the package step, which calls "make bee" was responsible for uploading to googlecode. I commented a line in Makefile since it wasn't working anymore. Careful with the passwords when you deal with that: buildbot logs a lot of stuff, like command lines, environment variables, etc... Check attachment.

Talking about log: "make log" (or open twistd.log file...) in master or slave directory will tell you the truth. Logs are very well written, self-explained. If you don't understand something, logs will really help.

Well, I guess this is quite a lot of information :) I let you have a look and get back to me if you need more.

2. yes it have crontab, just "crontab -e" to edit your own. I have set some for you:

@reboot /home/mattschinkel/bin/start_buildbot.sh

00 22 * * * rm -f /tmp/jallib_*

 

01 22 * * * rm -f /tmp/ditaot*

rm commands is just here to clean to temp files. the @reboot is here to relaunch buildbot is the server reboot.