25 Commits

Author SHA1 Message Date
Joshua Hesketh
62721fbb68 Update config.yaml to define jobs rather than plugins
Pay off some technical debt and make config.yaml define jobs rather than
plugins.

Change-Id: Ib4aa17649c02ab246d31515c8073230e8259cc4e
2014-12-02 13:24:30 +11:00
Matthew Oliver
2328b0ce0c Add the debug_log file to setup_logging
The debug_log file from the configuration was not being passed
to the setup_logging function. Causing no logs to be written
to to the log file.

This change, passes it file path to setup_logging.

Change-Id: Id84a6efdbd7ced98cb7774d09cf7068f988bf16e
2014-04-17 09:36:08 +10:00
Joshua Hesketh
96adb287f8 Allow jobs to shutdown turbo-hipster
This lets jobs turn off and exit turbo-hipster once they are done.
This is useful for when using nodepool or when a job leaves the
environment dirty and we can't run more jobs on this worker.

Change-Id: I823be4196a5bf9ca92a14d9caf26163398a9434c
2014-04-03 10:42:49 +11:00
Joshua Hesketh
5cd8ea3ce3 Add in tests for ZuulManager and ZuulClient
Also tidy up unnecessary fakes.

Change-Id: I165667faf61faa7fc06c10925db462e4650bda6a
2014-03-28 15:53:57 +11:00
Matthew Oliver
49c9063147 Break up Turbo-Hipster configuration
Seeing as the power of Turbo-Hipster is in its pluggable design,
this change allows the configuration to be spit up allowing
each plug in to maintain it's own part of the configuration
including overwriting any default configuration.

There is a new configuration parameter 'conf_d', which is
mapped to a directory:

    "conf_d": "/etc/turbo-hipster/conf.d"

worker_server.py grabs all files inside this directory and
attempts to load them. If it fails, the error is logged.

The motivating factor for this change for me, is that it allows
the deployment of turbo-hipster via puppet much simpler.
The base TH puppet class will create the config.json, whereas
there puppet TH plug in classes can pop extra configuration into
the conf_d directory.

Change-Id: Ied20b46d4caa642d130097f3fe019df9c0ec5851
2014-03-25 14:37:40 +11:00
Joshua Hesketh
9cd2f933a4 Register worker functions once we are connected
This also re-registers once a server appears ensuring reconnection is
properly set up.

Change-Id: I9b915482494d1b273991cb8bdd1b6ad963cf25f2
2014-03-05 16:49:26 +11:00
Joshua Hesketh
1d8c4e5375 Thread the server object
Instead of the server being the main process allow it to be threaded
to make testing and importing easier.

Change-Id: I5950b29188ed41a280f467f06bb4d3eb1f65b12a
2014-03-05 14:30:45 +11:00
Michael Still
89e4c6dd0d Add unit tests for execute_to_log.
Change-Id: I828f35d099c6cba5527c2f0d6240854ae9c39153
2014-01-10 14:09:10 +11:00
Joshua Hesketh
6f8133341b Move commands into cmd
Change-Id: Ieb51e4af8ce88b725d8bf9c2753757261e39fea6
2014-01-06 17:30:19 +08:00
Joshua Hesketh
b39f8840d6 Make sure we wait for a gearman server
Change-Id: I4cc5f5b6fb50a77116aeede4a450bd8281a5a5a9
2013-11-21 13:12:00 +11:00
Joshua Hesketh
3baaa0d160 Update log format
Change-Id: I5c8673ef713da43802837093753491ca31d07dd2
2013-11-21 11:05:37 +11:00
Joshua Hesketh
6eb5fdcd8d Add a bunch of debugging info
Change-Id: Idda0e7e79fa29aa458947356ddaed9d47d0a1d46
2013-11-21 10:37:27 +11:00
Joshua Hesketh
8326907163 Fix config setup order
Change-Id: I8cb584e6918797d9c7b197d629db7dad08073059
2013-11-21 10:37:27 +11:00
Joshua Hesketh
4343b956c5 Refactor th to have one gearman worker thread
Change-Id: I78f95a67b80ce0627b4a3bbb20578f3d16028714
2013-11-21 10:37:27 +11:00
Joshua Hesketh
527966b2e4 Have a different worker name for each thread.
This makes each thread create their own git repo so threads don't
compete and cause race conditions

Change-Id: Ifaf1faf153d9b26a0ff302b00603573b04205b04
2013-11-21 10:37:27 +11:00
Joshua Hesketh
1217693a01 Fix pep8 issues 2013-09-17 13:34:46 +10:00
Joshua Hesketh
05d0158c6c work on documentation 2013-09-09 15:16:08 +10:00
Joshua Hesketh
3b74fcdebe fix module import 2013-09-06 16:19:52 +10:00
Joshua Hesketh
4acd716827 fix typos 2013-09-06 16:05:37 +10:00
Joshua Hesketh
a74f49da7b refactor how datasets are stored 2013-09-06 15:52:49 +10:00
Joshua Hesketh
e58ecb27ab rearrange config installation 2013-09-06 12:39:46 +10:00
Joshua Hesketh
ac445b5a3e Fix module path 2013-08-14 12:55:22 +10:00
Joshua Hesketh
39a0feef7b add in copyright 2013-07-31 12:00:53 +10:00
Joshua Hesketh
6b6700b0a9 Move plugin listing into config 2013-07-26 16:47:32 +10:00
Joshua Hesketh
1ab465f923 restructure project slightly 2013-07-26 13:57:28 +10:00