2008-01-22 Dan Williams * client/client.py - Bump user cert buffer up to 20000 bytes 2008-01-07 Michael Schwendt * server/Builder.py - Fix exceptions on target_list access: it must be a list always * etc/ - killproc with explicit pidfile arg from sysconfig file 2008-01-06 Michael Schwendt * etc/plague-builder.init, etc/plague-server.init - killproc with delay 5 as clean shutdown takes some time * www/job.psp - Fix the "Result" that overlapped with double-quotes. 2007-12-26 Michael Schwendt * client/client.py, server/UserInterface.py - Fix update_builders traceback for non-admin users. - s/PLAUGE/PLAGUE/ for typo in env var. 2007-11-13 Michael Schwendt * etc/ - Update initscripts for LSB. - Move logfile and pidfile definition into sysconfig scripts. * server/PackageJob.py - Run cvs pkg checkout and make srpm with umask 0022. * builder/BuilderMock.py - Apply the race condition fix in _status_prepping() which has been fixed in Fedora Extras buildsys in September. 2007-11-12 Michael Schwendt * server/BuildMaster.py - Fix SQLite traceback in ctor. ProgrammingError: SQLite objects created in a thread can only be used in that same thread... * server/Config.py, builder/Config.py - Fix "no config" exception handler - Set default server UI port to 8887 to match client/client.py - Set active builder ports to match server config defaults 2007-11-11 Michael Schwendt * client/client.py - Fix pause command. 2007-09-30 Ville Skyttä * server/PackageJob.py - Fix rpmUtils imports * www/config.psp - Fix inaccurate down_message comment 2007-09-18 Michael Schwendt * etc/plague-builder.init, etc/plague-server.init - Fix usage help output 2007-05-20 Ville Skyttä * common/AuthedXMLRPCServer.py - Fix SimpleXMLRPCDispatcher initialization for Python 2.5 2007-04-27 Dan Williams Shad L. Lords * server/BuildMaster.py - (_write_archjob_status_to_db): update builder address too 2006-12-17 Dan Williams Toshio Kuratomi * common/XMLRPCServerProxy.py - Fix transport class initialization for Python 2.5 (RH #214687) 2006-12-06 Dan Williams Patch from Oliver Falk * server/Config.py builder/Config.py - Fix namespace of ConfigError when config file cannot be opened 2006-10-31 Dan Williams * common/ssl-client-test.py common/ssl-server-test.py - Two utilities to test the functionality of SSLConnection class and the SSL server/client classes 2006-10-31 Dan Williams * server/PackageJob.py - Fixes for older python versions that don't have yum.Errors.* but yum-utils' repomd module instead - Fixes for Yum 3.x logging changes 2006-10-31 Dan Williams Patches by Joe Todaro * server/PackageJob.py - (_kill_all_archjobs): don't traceback when killing jobs in depsolve_wait, where the archjob key is filled in, but the archjob itself hasn't been created yet because it's not yet building 2006-10-25 Dan Williams * www/template/main.psp - Fix cross-site scripting vulnerability by stripping the provided email address quite harshly 2006-07-25 Dan Williams Patch from Dennis Gilmore * Sun Niagra architecture support 2006-07-22 Dan Williams * server/PackageJob.py - Fix yum configfile creation during depsolve for non-base architectures (reported by Joe Todaro ) 2006-07-15 Dan Williams * server/Config.py - Fix optional arches (reported by Joe Todaro ) 2006-06-26 Dan Williams * builder/Builder.py - Add a configure option for setting maximum number of concurrent jobs if the autoconfigured number is inappropriate; "max_jobs" in the [General] section 2006-06-21 Dan Williams * builder/BuilderMock.py - Fix BuilderMock subclasses not correctly setting their arch_command variable (Dennis Gilmore) 2006-06-02 Dan Williams Patch from Michael Schwendt * server/Repo.py - (_update_repo): use lockfiles for repo directories to allow coordination with external scripts that manipulate repodirs 2006-06-02 Dan Williams Suggest by Michael Schwendt: * server/Repo.py - Change occurances of BuildJob/buildjob -> PackageJob/packagejob 2006-05-29 Dan Williams * Make most everything work now; both Passive and Active builders will download/upload the SRPM from the server. The SRPM's URL is no longer passed to the builder in the initial job request, but comes along later once the SPRM is actually available to the builder (which might be after an upload to the builder in the case of Passive builders). * Pylint cleanups * Clean up logging on builders by consolidating newline handling is one place * Use the Targets command to pass back to the server any upload URL the builder might have (for passive builders) 2006-05-23 Dan Williams * common/Commands.py - Add a "JobSPRM" command for notifying builders that the job SRPM is available. Passive builders need to upload the SRPM so we have to split the srpm out of the NewJob command to give the server time to upload to a passive builder. 2006-05-20 Dan Williams * server/Builder.py builder/Builder.py - Make passive builders work more - Consolidate some code into base Builder object in the server - Don't hang a job if it fails before we've had a chance to contact the builder after starting it 2006-05-16 Dan Williams * Make passive builders work somewhat more; there are still some bugs in command processing though. 2006-05-16 Dan Williams * common/XMLRPCServerProxy.py - Add cancellation ability to requests 2006-05-15 Dan Williams * builder/Builder.py builder/BuilderMock.py - (untested) Make passive builders work on the builder side 2006-05-14 Dan Williams * Rework archjob handling. They are now processed from the owning PackageJob object, and only one is spawned for the lifetime of the PackageJob for each architecture (previously one was spawned for each individual build job on the builder). Archjob UIDs are generated on the server now rather than the builder. * Correctly handle builders going away before they've had a chance to notify the server that they have started an archjob. Previously, these jobs would just be lost. This is the real reason for the rework of the archjob handling above. * On the builder, don't use die() to end jobs that have failed; do it properly and upload files to the server if we weren't killed * Deal with active builders whose hostnames can't be resolved when the server starts * Kill any existing jobs on builders when the server starts up * Consolidate and simplify logging functions in PackageJob * More pylint-induced cleanups 2006-05-12 Dan Williams * builder/Builder.py - Handle job kill command 2006-05-12 Dan Williams * common/Commands.py - pylint cleanups - Add the KillJob command 2006-05-11 Dan Williams * builder/Builder.py - Make Builder::cleanup() work again - Sleep in ActiveBuilder::stop() until the thread is actually stopped 2006-05-10 Dan Williams * builder/Builder.py - (Builder::__init__): print out build arches on start * builder/main.py - pylint cleanups - (drop_privs): use Exceptions rather than return; and ensure that, if we aren't running as root, that we are running as who the user configured us to run as in the config file 2006-05-09 Dan Williams * builder/Builder.py - Handle the JobFiles request * common/Commands.py - Check arguments on PlgCommandJobFiles * server/ArchJob.py - pylint cleanups - Rework result files download code; archjob now requests result files from the builder, which knows how to handle builder-type specific result files operations - get_upload_dir() renamed to get_result_files_dir() * server/Builder.py - pylint cleanups - Move dispatching of common commands into the base Builder class - (_decompose_job_files_ack): new function; extract and return info from a JobFilesAck command - (_handle_job_files_ack): new function; handle a JobFilesAck in the builder-type specific manner. For Active builders, we don't have to do much since the file was uploaded to us by the builder itself * server/BuilderManager.py - (any_prepping_builders): fix usage of builder.alive() -> builder.available() * server/PackageJob.py - Small cleanup of result files bits 2006-05-09 Dan Williams * builder/Builder.py - (upload_files): fix extraction of server address, and pass the jobid to the server so it knows what the result files are for * builder/BuilderMock.py - Fix name of ul_callback() * common/FileTransfer.py - Trap operational errors * server/ArchJob.py - (get_upload_dir): new function; return the directory into which result files should be written * server/Builder.py - (get_archjob): new function; return archjob object for a particular jobid - (_handle_building_jobs): correct scoping of bits that get status for certain jobs - Increase the required active builder contact interval slightly - (request): reset the unavailable count every time the builder contacts us * server/BuilderManager.py - (__init__): Enable the HTTP POST handler when there are active builders - (upload_callback): handle incoming files by writing them to the correct location based on their jobid 2006-05-08 Dan Williams * builder/Builder.py - pylint cleanups - make SRPM downloads and uploads work * builder/BuilderMock.py - pyling cleanups - remove log() - make download and upload work * server/Builder.py - Don't send multiple UnlockRepo commands - Actually let archjobs have some processing time 2006-05-08 Dan Williams * server/PackageJob.py - (_request_one_arch_job): Use the correct fileserver port * server/ArchJob.py - (_status_repo_unlock): fix unknown variable, should be self.job * common/URLopener.py - import socket to fix an unknown type * common/FileUploader.py - (__init__): don't need to pass url in as each subclass keeps track of the url itself * common/FileTransfer.py - (__init__): don't keep track of url as each subclass needs to do this separately; Fix indentation of self._certs to make sure it gets set even if it's None * common/FileDownloader.py - (__init__): don't need to pass url in as each subclass keeps track of the url itself; in our case it's actually a list of urls. Make sure self._files is defined, and has at least one item too. * common/Commands.py - (deserialize_command_stream): don't let unknown commands into the deserialized command stream; they show up as None items in the returned list. - (PlgCommand.deserialize): deserialize PlgCommandUnlockRepo commands too 2006-05-08 Dan Williams * common/Commands.py - Add JobFiles and JobFilesAck commands 2006-05-08 Dan Williams * common/HTTPServer.py - For test code, print out received files, and put them in /tmp/server_dir rather than just in 2006-05-08 Dan Williams * common/FileTransfer.py - Refactor common code from FileDownload.py and FileUpload.py into FileTransfer.py * common/FileDownload.py - pylint cleanups - Refactor for FileTransfer.py - Allow multiple transfers for one FileDownloader object * common/FileUploader.py - pylint cleanups - Refactor for FileTransfer.py - Allow multiple uploads for one FileUploader object 2006-05-03 Dan Williams More builder/server comm rework; - Add job status command - implement repo unlock command - fix bugs 2006-04-27 Dan Williams Commit partial rework of builder<->server communcation. 2006-03-27 Dan Williams * common/HTTPServer.py - Add extended authentication - Consolidate POST handling and authentication handling into PlgBaseHTTPServer class 2006-03-26 Dan Williams * common/HTTPSURLopener.py - Generalize and move to URLopener.py * common/URLopener.py - Add bits to allow file uploads - Support both HTTP and HTTPS * common/FileUploader.py - new file; easily upload files * common/FileDownloader.py - Convert to URLopener * common/HTTPServer.py - Make test code work with jobid of 0 2006-03-26 Dan Williams * common/HTTPServer.py - Add ability to process HTTP POST requests, mostly for file uploads 2006-03-24 Dan Williams * server/DBManager.py - Close database connection on delete for PostgreSQL and MySQL. Fixes leaking MySQL connection with Python 2.3 and MySQL-python 1.0.0, as seen on RHEL4. #rh182441# * server/BuildMaster.py - Delete the database context when on stop * server/User.py * server/UserInterface.py - Clean up deletion of database contexts and cursors 2006-03-23 Dan Williams * server/Repo.py - Consolidate depsolve_again and depsolve_immediate locks into one lock to avoid deadlocks 2006-03-23 Dan Williams * server/PackageJob.py - (_arch_deps_solved): fix cachedir and logfile paths; turns out yum already adds the installroot to these. Furthermore, return an error when we can't find a suitable yum config file for the arch 2006-03-20 Dan Williams * common/FileDownloader.py - Pass error message to download callback * builder/builder.py server/ArchJob.py - Print out download errors, if any 2006-03-20 Dan Williams * server/Config.py - Change config file format for builders. Each builder is now listed on a different line like this: builder1 = 20 https://127.0.0.1:8888 Each builder's tag (what's before the = sign) must start with 'builder'. The number right after the = is the builder's weight, or preference. Higher numbers mean the builder is more likely to be used. * server/BuilderManager.py - Use builder weights when picking which builder to use for a job * server/Builder.py - Add accessort for builder weight 2006-03-13 Dan Williams * builder/builder.py - (main): Fix usage of handle_request when serve_forever was intended 2006-03-12 Dan Williams Fix up kmod support. * server/Config.py - Fix splitting of arches for Additonal Package Arches - Use wildcard matching (*, ?, [...], [!...]) for package names in Additional Package Arches so we can support kmods easily * server/PackageJob.py - Add lots of comments in arch_handling() so people can tell what's going on 2006-03-12 Dan Williams * server/BuildMaster.py server/DBManager.py server/UserInterface.py - Prefix 'epoch', 'version', and 'release' database fields with 'rpm_', since MySQL 5 added 'release' as a reserved word. Bastards. 2006-03-12 Dan Williams * builder/builder.py - Add a SIGTERM handler for clean shutdown - Clean up lifecycle management * common/HTTPServer.py - Clean up lifecycle management of PlgHTTPServerManager objects * common/SSLCommon.py - Clean up lifecycle management of PlgBaseServer objects - Initialize correct parent object in PlgBaseSSLServer.__init__ * etc/plague-builder.init etc/plague-server.init - Cleanups * server/main.py - Add a SIGTERM handler for clean shutdown - Clean up lifecycle management 2006-02-26 Dan Williams * client/client.py - Handle more errors, and print them more nicely 2006-02-19 Dan Williams * builder/builder.py - Kill jobs in 'downloaded' state after 30 minutes, because it's likely there was an error returning the archjob's UID to the server. In this case, the server has already gone on and assigned the job to another builder. - (run): Move execution of the status function outside of the try/except block so we don't mask AttributeErrors in the code 2006-02-19 Dan Williams * server/Builder.py - Fix mistaken occurances of self.address -> self._address - Print a message when errors occur starting new jobs on builders 2006-02-16 Dan Williams * server/ArchJob.py - Fix a few print statements that were using an unknown variable to get the builder's address from - Handle socket timeout errors 2006-02-16 Dan Williams * server/ArchJob.py - (process): don't trap AttributeErrors triggered by running the status worker function, only trap them when getting the status worker function. Also don't set state to 'failed' when an internal error occurs, since that's not a valid status. - (internal_failure): new function * server/PackageJob.py - (_stage_building): trap and report internal failures too 2006-02-15 Dan Williams Patch from Andreas Thienemann #rh180555# * etc/plague-server.init etc/plague-builder.init - Correct usage of pidof so it kills plague processes executing under Python 2006-02-15 Dan Williams * common/SSLConnection.py - Don't use pyOpenSSL's sendall() call, since it simply calls SSL_write() in a loop. Instead, we simulate the sendall ourselves, allowing us to honor socket timeouts and to not peg the CPU when we get WantReadError/WantWriteError exceptions due to non-blocking IO - Clean up the recv/sendall functions somewhat too - Add some cosmetic inter-function spacing - Increase WantReadError/WantWriteError sleeps from 0.1s -> 0.2s - Return valid socket error message on timeout 2006-02-15 Dan Williams * common/XMLRPCServerProxy.py - Fix up some of the test code 2006-02-15 Dan Williams * server/main.py - remove pidfile on exit 2006-01-24 Dan Williams * common/ExecUtils.py - (exec_with_redirect): by default, have child start a new process group * builder/builder.py - (_handle_death): kill child's entire process group, not just the child itself 2006-01-23 Dan Williams * common/SSLConnection.py - Revert previous commit, causing too much trouble 2006-01-22 Dan Williams * server/Builder.py - Move most of the building_jobs() logic into another thread to combat hanging issues 2006-01-22 Dan Williams * common/SSLConnection.py - Be smarter about SSL stuff by using pending() where possible in a vain effort not to hang as much. Cribbed from rhn code by misa@redhat 2005-12-01 Dan Williams Patch from Rudolf Kastl #rh174573#: * etc/plague-server.init etc/plague-builder.init - Add nls support to initscripts - initscript cleanups 2005-11-29 Dan Williams * Add an option in the server config file to disable depsolving 2005-11-29 Dan Williams * Add a traceback server that listens on a Unix socket and writes backtraces for all threads to it. Disabled by default. 2005-11-28 Dan Williams Reported by #rh174379#: * server/UserInterface.py - Accept '.' as valid package name character 2005-11-26 Dan Williams * First cut of depsolving code. You need to add a 'mock_configs_dir' option in your plague-server.cfg in the Directories section that should usually point to /etc/mock 2005-11-25 Dan Williams * server/DBManager.py utils/plague-user-migration.py - Postgres requires single-quotes, not double - Postgres wants "True" for boolean fields rather that "1", so add coversion functions for all database types to do the right thing 2005-11-24 Dan Williams * client/client.py server/Builder.py www/template/main.psp - Use timeouts for XMLRPC clients 2005-11-24 Dan Williams * common/AuthedXMLRPMServer.py common/SSLCommon.py common/SSLConnection.py common/XMLRPCServerProxy.py - Implement socket timeouts for XMLRPC client & server 2005-11-20 Dan Williams Patches from Jeff Sheltren * server/User.py server/main.py server/DBManager.py utils/user-manager.py - Store users using same db engine as the job database * utils/plague-user-migration.py - Script to move user info from sqlite to postgres/mysql 2005-11-19 Dan Williams * www/* - Add time columns to main, failed, and success pages showing relative times jobs started or finished 2005-11-18 Dan Williams * common/SSLConnection.py - Fix behavior when socket timeout is set - Increase default socket timeout - Fix position of refcount decrease in close() so we actually close the socket now * common/SSLCommon.py - Add optional socket timeout parameter for PlgHTTPSConnection * common/HTTPSURLopener.py - Pass along a socket timeout, if specified * common/FileDownloader.py - Specify a socket timeout for download connections 2005-11-18 Dan Williams * server/ArchJob.py - Log failed downloads before retrying - Retry downloads from builder up to 5 times, not 3 * builder/builder.py - Retry downloads from server up to 5 times, not 3 2005-11-15 Dan Williams * server/Builder.py - Suspend builders on hard errors like running out of disk space or file descriptors or whatever 2005-11-15 Dan Williams * server/ArchJob.py server/PackageJob.py - Log kill requests 2005-11-14 Dan Williams * common/ExecUtils.py - Close files opened in parent after exec() of child 2005-11-13 Dan Williams * common/ExecUtils.py - On python 2.3 and lower, unblock signals before execing processes. Requires py_pthread_sigmask module. 2005-11-12 Dan Williams * server/PackageJob.py - (die): immediately kill jobs by starting up a controller thread when the job is in 'waiting' stage 2005-11-12 Dan Williams * server/ArchJob.py - Lock access to self._downloads - Make more stuff private to ArchJob - Use variables for download dict to reduce possibility of typing mistakes 2005-11-01 Dan Williams * builder/builder.py - (dl_callback): retry failed downloads from the build server up to 3 times before failing the job 2005-11-01 Dan Williams Patch from Alexandr Kanevskiy * builder/builder.py - Remove unused imports - Clean up usage of work_dir and other previous global variables now that they are local to the BuilderMock class 2005-11-01 Dan Williams * server/ArchJob.py - (get_files): fix up download status dict to work with new download-retry stuff - (set_download_status): remove, unused * server/Repo.py - (_update_repo): remove ignore_errors argument from shutil.copy(), since that's incorrect for that function 2005-10-31 Dan Williams * www/* - Add a 'down_message' option to config.psp and show it, if one exists, on all pages 2005-10-31 Dan Williams Patch from Alexandr Kanevskiy * builder/builder.py - (_status_prepping): catch mock failure case earlier 2005-10-27 Dan Williams Patch from Jeff Sheltren * server/main.py - Honor config file location passed in on command line 2005-10-25 Dan Williams * server/PackageJob.py - (_stage_prep): fail the job if we can't access the SRPM here. Can happen if the server restarts and requeues waiting jobs, but the SRPM isn't around anymore for some reason. 2005-10-25 Dan Williams * server/ArchJob.py - Retry finished job downloads from builders up to 3 times before failing the job entirely 2005-10-25 Dan Williams * builder/builder.py - (_handle_death): don't forget about a job when it gets killed, but doesn't have an mock subprocess (for example, if the job is in the 'downloaded' state) 2005-10-25 Dan Williams * server/PackageJob.py - (repo_add_callback): trap failures during repo addition of job * server/Repo.py - (_update_repo): recognize failures and notify the PackageJob of them so the job can fail cleanly rather than half-copying stuff to the repo 2005-10-24 Dan Williams Patch from Jeff Sheltren * server/DBManager.py server/Config.py - Add MySQL support 2005-10-19 Dan Williams * server/builder.py - Make the server's builder-tracking thread more resistant to builder errors 2005-10-19 Dan Williams * client/client.py - Don't traceback when the 'allow_uploads' config options is not present - Correctly print target in job detail command without a traceback 2005-10-17 Dan Williams * server/DBManager.py - Increase 'username' and 'status' field sizes in the jobdb 2005-10-05 Dan Williams * builder/builder.py - Fix race condition in which files would not be downloaded to the build server 2005-09-16 Dan Williams * builder/builder.py common/FileDownloader.py server/ArchJob.py - Rename FileNameError -> FileNameException * builder/builder.py - Clean up exception handling on job creation so that we don't end up calling start() before Thread.__init__() got called. 2005-09-14 Dan Williams * server/PackageJob.py server/Config.py - Rename "scratch" targets to "testing" targets to make their purpose clearer - Don't delete packages from the repodir if their target is a testing target 2005-09-14 Dan Williams * utils/distro-rebuild.py - Optionally add a changelog entry to rebuilt specfile 2005-09-13 Dan Williams * builder/builder.py - Try to tighten up child process handling to not leave defunct mock processes around 2005-09-10 Dan Williams * utils/distro-rebuild.py - Don't delete the rebuilt SRPM until after everything has been queued 2005-09-10 Dan Williams * server/BuilderManager.py - Fix python < 2.4 issues with sequence sort() call 2005-09-09 Dan Williams * server/UserInterface.py - Allow '+' in CVS tags 2005-09-09 Dan Williams * Add support for PostgreSQL as the database backend. Databases are now abstracted and support for others, like MySQL, need to be added to server/DBManager.py. Should be really easy though. NOTE: sqlite support might be broken right now due to SQL statement quoting changes 2005-09-08 Dan Williams * server/PackageJob.py server/ArchJob.py - Fail job if downloads from builder fail 2005-09-07 Dan Williams * utils/distro-rebuild.py - Script that takes a directory of SRPMs, finds the newest ones, adds a ".1" to their Release, then queues them up using plague 2005-09-01 Dan Williams * client/client.py server/Repo.py server/UserInterface.py - Implement package upload functionality using 'scp' as part of the 'build' command 2005-09-01 Dan Williams * www/builders.psp - Better unavailable builders display 2005-08-31 Dan Williams * server/BuilderManager.py - Load-balance builders somewhat 2005-08-31 Dan Williams * www/ - Fix up web frontend for latest server code 2005-08-31 Dan Williams * builder/builder.py - Reset self._die so we don't kill cleanup processes 2005-08-31 Dan Williams * server/UserInterface.py - Fix more target-related stuff 2005-08-31 Dan Williams * builder/builder.py server/Builder.py - Expose "num slots" values to build server 2005-08-30 Dan Williams * builder/builder.py common/ExecUtils.py - To execute mock, we now fork() and execv() the mock process so that we can more reliably gather its output - Correctly clean up the mock root directory - Rename builder.log -> job.log so it can't get confused with build.log from mock 2005-08-30 Dan Williams * builder/builder.py - all self.log() -> self._log() * server/Builder.py - Fix condition where finished jobs wouldn't get noticed by the server * server/PackageJob.py - Fix CVS commands 2005-08-29 Dan Williams * More misc cleanups 2005-08-29 Dan Williams * Make more stuff actually work 2005-08-29 Dan Williams * server/Config.py - Ensure duplicate user_aliases and duplicate target strings cannot be loaded - Move user target alias resolution into the Config stuff to keep it centralized * server/UserInterface.py - Move target user alias resolution to Config stuff - Consolidate 'enqueue' and 'enqueue_srpm' commands into one 'enqueue' command to enqueue packages that detects what you are trying to enqueue by looking at the source - Work with new target dict stuff 2005-08-25 Dan Williams * Initial commit of reworked stuff: - Each target gets separate config files on builder and server - Builders now run multiple jobs per builder instance - Config files now ConfigParser based - Target specifications are richer and require distro, target, and repo names - Builder's supported arches are autodetermined - Various database fields renamed and/or removed IT DOESN'T WORK YET 2005-08-24 Dan Williams * common/BaseConfig.py client/client.py - Rewrite the client to make the code less stupid, more easily followed, and to validate more input options - Also fixes RH #166692 2005-08-15 Dan Williams * server/BuildMaster.py - Clear out old job info when requeueing a job 2005-08-15 Dan Williams * server/ArchJob.py server/PackageJob.py - Let "scratch" target jobs continue building even if one or more arches has failed 2005-08-15 Dan Williams * client/client.py server/UserInterface.py - Implement API versioning, and check the API version in the client before doing anything else 2005-08-14 Dan Williams * Switch to new server config file format using ConfigParser - Server config files are no longer python scripts - Server now takes a mandatory -c option for config file location - Server will write out a default config file at the specified location if none exists - Each target has its own config file, by default in /etc/plague/targets/ - CVSROOT and CVS_RSH are now target-specific, not server-wide - Additional Package Arches are now target-specific, and reside in each target's config file rather than server-wide * "Scratch" targets are now supported; ie, they are targets that do not contribute packages to a repository. Use the option "scratch=yes" in the target's config file in the "General" section to make the target a scratch target. * Repository Scripts: you may now specify a script that is run after packages are copied to a repository. Use the "repo_script" option in each target's config file to specify a script for that target/repo. The script must exit with result 0 on success, and > 0 on error. The script is given one argument, which is the name of the target/repo which has just copied packages. If the script fails, those listed in "admin_emails" in the server config file will be mailed with the script's output. The script is also killed if it takes longer than 1 hour to complete, since packages cannot be built for the target while the script is running. 2005-08-11 Dan Williams * Tag plague-0_3_1_RELEASE 2005-08-11 Dan Williams * plague.spec - Clean up to Fedora Extras standards in preparation for package submission * builder/builder.py - Don't ignore mock output during the 'cleanup' stage - Make the process() function less byzantine by moving each step to a separate function and using getattr() to call them, like in PackageJob.py on the server * www/* - Clean up error handling so it actually works all the time intead of returning Error 500 Internal Server Error when, for example, the build server isn't running Patch from Ville Skyttä * www/template/head.psp www/indiv.psp - Use HTTP GET instead of POST to stop nags about form data resubmission 2005-08-08 Dan Williams * Tag plague-0_3_RELEASE 2005-08-08 Dan Williams * README plague.spec utils/Makefile utils/certhelper.py - Make creation of SSL certificates not suck 2005-08-07 Dan Williams Play nicely when running more than one builder instance on the same machine. * builder/builder.py - Try to delete actual mock work dir rather than just the buildroot * client/client.py - Prettier printing of job detail command output - Show builder port numbers * server/ArchJob.py - Save builder port number * server/BuildMaster.py - Add a builder port # to the archjobs table, and save builder port # when writing archjob status to the DB * server/Builder.py - (to_dict): return builder port # too * server/BuilderManager.py - Don't exit when adding new builders that have wrong connection protocol, just ignore them * server/UserInterface.py - Retrieve and return builder port numbers too * www/builders.psp - Differentiate builders and their jobs by port # - Only show building archjobs, not 'downloading' or 'downloaded' ones * www/job.psp - Show builder port numbers 2005-08-05 Dan Williams * Rework builder tracking code to always keep Builder objects around, and to mark them as active/unavailable rather than dropping unavailable builders. Unavailable builders are pinged every 5 minutes to see if they are alive or not. Admins can still manually ping builders. * Remove lots of locking code in between the Builders, ArchJobs, and the BuildMaster since it was only relevant when pyOpenSSL still sucked. Communication with the builder's XMLRPC server only happens from each Builder object's thread now. * Consolidate job-killing code in both the ArchJob and the PackageJob 2005-08-05 Dan Williams * builder/builder.py - (_mock_done): hand-wavy attempt to ensure the mock child process is always reaped * client/client.py - (enqueue, enqueue_srpm): Print out returned job UID - (detail_job): remove bogus email argument * server/BuildMaster.py - (create_job_request): new function to consolidate job request creation - (enqueue, enqueue_srpm): consolidate to one enqueue() function since the database really makes no distinction between CVS and SRPM jobs - (_start_new_jobs): return UID to requester when we become aware of it * server/UserInterface.py - (_wait_for_uid): new function, waits max 3 seconds for the BuildMaster to return the new job's UID - (enqueue, enqueue_srpm): use create_job_request(), and wait for the job's UID to be returned from the BuildMaster, which we then pass back to the client - (detail_job): remove bogus email argument - (_kill_job): warn user if the job is no longer building or waiting to be added to the repo. We can't kill jobs if they are no longer building. * www/job.psp - detail_job()'s email argument got removed, remove it here to 2005-08-04 Dan Williams * server/UserInterface.py - Accept asc/desc ordering in addition to field order. You now do e.x.: args['orderby'] = ['package asc', 'endtime desc'] * www/indiv.psp - Stick in-progress jobs at the top of the list * www/template/head.psp - Keep user's email address through the success/failed pages 2005-08-04 Dan Williams * Attempts to reduce network bandwidth usage between client & server - for listing jobs, 'status' item in the arg list is now a list itself which is combined using OR. - New 'orderby' arg for listing jobs - Fixes for both of the above in the web interface, the client, and the server * Small fixes to the web interface to display 'result' as well as 'status' 2005-08-04 Dan Williams * server/PackageJob.py - Clean up job files when the job is done. We now only keep logs and 1 SRPM in 'server_work_dir', the RPMs get copied to the repo and then deleted from server_work_dir 2005-08-04 Seth Vidal * add in success and failed pages for www report 2005-08-02 Dan Williams Patch from Josh Boyer * client/client.py - Fix up usage printout 2005-08-01 Dan Williams * Implement 'finished' state, allow users to move failed/needsign jobs to the 'finished' state * Random cleanups of database access code to make it more readable * Simplify user permissions * server/PackageJob.py - Each state/stage now has a private method called "_stage_" that gets called when the job is in that stage. This simplifies the job's process routine quite a bit - Jobs now record a 'result', either 'success', 'failed', or 'killed' 2005-08-01 Dan Williams * common/AuthedXMLRPCServer.py common/SSLConnection.py server/Builder.py - Simulate socket timeouts with select() 2005-07-29 Dan Williams * server/BuilderManager.py server/BuildMaster.py server/UserInterface.py client/client.py - Add admin-only pause/unpause commands to the server 2005-07-29 Dan Williams * www/job.psp - Line up arch jobs - Only show build times for successful jobs 2005-07-29 Seth Vidal * www/index.psp - despam email addresses 2005-07-29 Seth Vidal * server/UserInterface.py - fix resolved target to succeed if a match comes from the client_target_map also simplify the function a bit. 2005-07-28 Jeremy Katz * plague.spec: Create builder user and work dir in -build package * builder/CONFIG.py: Set builder cert by hostname. Listen on all interfaces. 2005-07-27 Jeremy Katz * builder/builder.py (PPCArch.__init__): Use setarch for ppc32 2005-07-26 Dan Williams - www/config.psp www/templates/main.psp - Move config-type stuff for SSL to config.psp 2005-07-26 Dan Williams - Throttle CVS checkouts to 5 jobs at a time - Validate package and cvs_tag inputs to enqueue functions since they get passed directly to commands.getstatusoutput() 2005-07-25 Dan Williams * client/client.py server/UserInterface.py server/BuildMaster.py - Add a "requeue" command to restart a failed or killed job 2005-07-25 Dan Williams * server/CONFIG.py - Move stuff around, two new options for admin_emails and success_emails * server/BuildMaster.py server/PackageJob.py - Set endtime when all builds complete or fail, to not include all the repo time in the job's build time * server/PackageJob.py server/UserInterface.py server/BuilderManager.py server/EmailUtils.py - Notify admins when a builder times out - Consolidate email sending into one place - Notify success_emails when a build job succeeds 2005-07-22 Dan Williams * server/UserInterface.py - Fix traceback in job detail code * www/* - Add an individual job view - Cleanups for Safari 2005-07-22 Dan Williams * builder/builder.py - Fix traceback when killing jobs on shutdown of the builder * client/client.py - Add a job detail command * server/ArchJob.py - Store job start and end times in the database * server/BuildMaster.py - Remove JobsQuery class, no longer used - Store more info in the job database to support web front end features - Don't try to restart jobs in the 'initialize' state since they'll get restarted anyway * server/PackageJob.py - Generalize log URL construction so it can be accessed from the user interface - Pass more job info to the BuildMaster to be written to the DB - Don't use tempfile.mkdtemp(), it seems to have issues - Return 30 lines of log rather than 20 * server/User.py - Grab a new connection to the database on every access, so that user addition/modification can happen when the server is running * server/UserInterface.py - Add a "job detail" interface that returns information about a single specific job * server/main.py - Greatly reduce buffer size for the logfile, now it actually gets written out in a timely fashion 2005-07-22 Dan Williams * Web front-end cleanup - Use Diana Fong's new design. Delicious! 2005-07-21 Dan Williams * builder/CONFIG.py builder/builder.py - You must now specify a target/arch -> mock buildroot name mapping in the config file * server/UserInterface.py server/CONFIG.py - Add client target alias support so people don't have to know the exact target name - Add rowlimit support to job listing 2005-07-21 Dan Williams * common/FileDownloader.py server/PackageJob.py - Remove usage of os.chdir() since it's process-wide - Lock usage of tempfile.mkdtemp() since it seems to have threading issues 2005-07-21 Dan Williams * www/* - Add the web front-end 2005-07-20 Dan Williams * builder/builder.py server/main.py - Don't use lighttpd any more - Fix some issues with non-SSL build systems (Oliver Falk) * client/client.py server/UserInterface.py - Return job lists as dicts for easier use - Return archjobs embedded in the job list rather than a separate array 2005-07-18 Dan Williams * server/UserInterface.py client/client.py - Rework argument handling and passing so that listing jobs is more flexible. Can now list jobs by UID as well. 2005-07-18 Dan Williams * builder/builder.py common/ArchUtils.py server/PackageJob.py - Allow the server and builder to actually build "sub" arches like i486/sparcv9/etc, which broke after the the builder changes on 2005-07-16 2005-07-18 Dan Williams * builder/builder.py - Don't start a cleanup over top of another cleanup when we're told to kill the job * server/BuildMaster.py server/Repo.py - Clear repo additions array after each repo update, fixes problem where jobs would switch back from 'needsign' -> 'repodone' after the job was already complete 2005-07-18 Dan Williams * server/PackageJob.py server/BuilderManager.py - Restart orphaned archjobs immediately rather than sticking them at the back of the build queue 2005-07-18 Dan Williams * builder/builder.py common/FileDownloader.py server/ArchJob.py - Allow '+' character in RPM names - Clean up file name handling in URLs 2005-07-16 Dan Williams * builder/builder.py server/Builder.py server/BuilderManager.py - Make the builder aware of what mock targets & arches it can actually build, and expose that information to the build server - On the server, make sure that we ask the builder to only build for arches that it supports for the target we request (This makes noarch jobs work correctly everywhere) 2005-07-16 Dan Williams * server/PackageJob.py - Include last 20 lines of relevant log file for failed build jobs 2005-07-14 Dan Williams * etc/plague-builder.init etc/plague-builder.config - Allow multiple builders to be started/stopped at one time. plague-builder.config must be copied to /etc/sysconfig/plague-builder and have the CONFIGS variable defined to be a space-separated list of builder config files, with the whole list enclosed in quotes 2005-07-14 Dan Williams * server/PackageJob.py - Include log URL in email notifications 2005-07-14 Dan Williams * builder/builder.py - Make the builder not hang if you interrupt it in the middle of a time.sleep() - Fix misdeclaration of log() which caused a traceback in some instances * server/Builder.py - Kill any job the builder is running when the server starts up * server/BuildMaster.py - Restart interrupted jobs when the server starts up - Remove old query queue code that's no longer used * server/PackageJob.py server/UserInterface.py - is_build_job_stage_valid -> is_package_job_stage_valid 2005-07-13 Dan Williams * builder/builder.py builder/CONFIG.py etc/plague-builder.config etc/plague-builder.init - Move arches and hostname into the CONFIG.py file, since its already instance-specific. Change the initscripts to reference the config file, which gets passed to the builder on the command line now. 2005-07-13 Dan Williams * common/AuthedXMLRPCServer.py common/SSLCommon.py common/SSLConnection.py - Play better with Python 2.2 2005-07-12 Dan Williams * Warn user and exit if server or builder is already running * Add aliasing/mapping of plague targets to CVS targets, to allow for different CVS directory names than what our targets are 2005-07-10 Dan Williams Patch from Ignacio Vazquez-Abrams * Add initscript/daemon support for the server 2005-07-10 Dan Williams * server/PackageJob.py server/BuildMaster.py - Reduce total threadcount by breaking PackageJob runs into two threads, so that no thread executes while the package is waiting for a builder 2005-07-09 Dan Williams Patch from Ignacio Vazquez-Abrams * Add initscript/daemonize support for the builder 2005-07-09 Dan Williams * Rework the authorization framework so we don't need to subclass/override subclass so much stuff * Make the specfile Python version independent 2005-07-08 Dan Williams * Pass archjob information along to the client, and move query into UserInterface's thread 2005-07-07 Dan Williams * builder/builder.py - Clean up the buildroot no matter what * Make archjobs write their status to the database too 2005-07-07 Dan Williams * server/PackageJob.py - Make some prep error emails more informative 2005-07-07 Dan Williams * server/PackageJob.py - Use Python exceptions for prep and build errors - Clean up a bunch of stuff - Specify dest file when copying files to the repo * server/Repo.py - Ensure that the destination file is actually the file rather than a path, since shutil.copy() just copies the file over top of the directory in some instances, corrupting the repo 2005-07-06 Dan Williams * Convert M2Crypto code to pyOpenSSL so that stuff actually works 2005-07-05 Dan Williams * Rework a bunch of stuff so the build server doesn't use quite as much CPU, also split out stuff from server/client_manager.py * Generalize the common/ classes to provide both SSL and non-SSL facilities, renaming a lot of those files in the process * Fix non-SSL builder/server and client/server communication Note: At this time, SSL may be broken. 2005-07-01 Dan Williams * We now use lighttpd as the fileserver in both the server and the builder. It's a lot more robust than the python implementation, but the python implementation code still exists and can be used. * The server key and certificate must now be in the same file. You can 'cat' them together with 'cat key.pem cert.pem > key_and_cert.pem' * Initialize m2crypto threading in buildservery.py and builder.py * Change some config options for the builder that said 'client' to 'builder' * Add some testing code to SimpleHTTPSServer.py and FileDownloader.py 2005-06-29 Dan Williams * Add the ability to search jobs on current job status from plague-client 2005-06-29 Dan Williams * Copy finished SRPMs to the correct directories in the repo 2005-06-29 Dan Williams * Implement job kill functionality 2005-06-28 Dan Williams * README - Fix up readme for current CVS Also fix up some makefiles to create the /etc/plague/[server|builder]/certs directories. 2005-06-26 Dan Williams * Move stuff around. The client that package maintainers will use to submit jobs is now in client/, and the actual build daemon has moved to builder/ 2005-06-26 Dan Williams * plague.spec Makefile - Make RPM builds using the specfile work 2005-06-26 Dan Williams * client/buildclient.py common/SSLCommon.py common/SimpleHTTPSServer.py common/SimpleSSLXMLRPCServer.py - Set a much lower socket read timeout than the default, which is 600 seconds. We don't want clients blocking the server for too long, and non-blocking sockets are a bit too complicated at this time. 2005-06-26 Dan Williams * utils/package-builder.py - Report server errors * Fix paths in config files 2005-06-26 Dan Williams * We have a name: "(The) Plague". We come to eat your packages. * Add Makefiles, install common stuff in python site-packages, and convert files to use config from /etc 2005-06-25 Dan Williams * server/BuildMaster.py server/UserInterface.py - Make queries work again 2005-06-25 Dan Williams * server/client_manager.py - Lock access to build client queries because they could be called from different threads 2005-06-25 Dan Williams * client/buildclient.py - Break out some functions from process() to make code easier to follow - Use unique extensions on buildroots so more than one build client can run at the same time on the same machine - Execute mock after the build to clean up the buildroot before deleting the buildroot directory Requires latest mock from CVS 2005-06-24 Dan Williams * Rework job database handling. Only 1 thread now has access to the job database, the BuildMaster thread. This should fix issues with database locking, timeouts, etc. sqlite doesn't really like multiple threads, plus it doesn't have row locking, only table locking. So, when things want the BuildMaster to do something, they queue up a request and the BuildMaster gets around to it. Once consequence of this change is that job UIDs are not known until the job gets added to the database, so we can no longer return the job's UID to the client enqueueing the job. * BuildClients are also in their own thread now so they don't block the BuildMaster. There are interesting cases where clients can block the server while the server is reading data from the client, or if the client tracebacks in the middle of an SSL connection. This should help keep the server more robust. Operational latencies are also reduced by this change and the one to the BuildMaster. 2005-06-24 Dan Williams * common/CommonErrors.py - Try to trap actual error strings. Does this actually work? 2005-06-24 Dan Williams * client/buildclient.py - Remove some debug strings - Fix undefined variable usage 2005-06-24 Dan Williams * server/Repo.py - Deal with repo locking during createrepo stages * Ensure repositories can run createrepo without being accessed by clients during the operation. Since createrepo runs can take a long time, during that time clients cannot try to install their buildroots or do any operations against the repository, otherwise they may fail randomly with yum errors. So we lock access to the repository on a 2-level lock: when a build job is done, it asks the repo to copy its finished RPMs, and the repo enters lock level 1. Level 1 prevents new build jobs from entering their 'prep' state. When all currently running jobs have finished their prep state, and the repo is in lock level 1, the repo promotes to lock leve 2 and is able to run createrepo after copying any new RPMs into the repo. When this is done, all waiting clients are released into their 'prep' states. This requires an absolute latest mock from CVS. 2005-06-24 Dan Williams * server/BuildMaster.py server/client_manager.py - Trap more errors 2005-06-21 Dan Williams * common/FileDownloader.py - Trap more errors, simplify code a bit 2005-06-17 Dan Williams * common/CommonErrors.py - Oops, "e" isn't an error, its the data. Treat it as such. * server/client_manager.py - Trap more errors in BuildClient._update_cur_job() 2005-06-17 Dan Williams * common/CommonErrors.py server/client_manager.py common/SSLCommon.py common/FileDownloader.py * server/BuildJob.py - better check for unspawned jobs - Add a '/' to SRPM URLs that clients download to protect against CONFIG.py errors - Mark the repo as invalid after we copy RPMs to it * server/BuildMaster.py - Only run createrepo when the repository has actually changed * server/client_manager.py - Fix bug that caused jobs to simultaneously get started on all clients that supported an architecture 2005-06-17 Dan Williams * common/SSLCommon.py - Trap some more SSLErrors 2005-06-17 Dan Williams * server/client_manager.py server/BuildJob.py - Fix name clash of builder_gone 2005-06-17 Dan Williams * utils/package-builder.py - Simply arguments a bit, same command now for CVS or SRPM builds. Script assumes that if the cvs/srpm argument ends with .src.rpm and exists locally, that its an SRPM build. * utils/repoconv.py - Copy debuginfo RPMs too 2005-06-17 Dan Williams * utils/repoconv.py - Converts between flat directories of RPMs, and the buildserver repo format of /name/v-r/arch/package * server/client_manager.py - Accept error 61 (connection refused) and increment unavailable_count when we get it 2005-06-16 Dan Williams * common/SSLCommon.py - Override SSL.SSLServer's handle_error() method since it doesn't take the right number of arguments * common/SimpleSSLXMLRPCServer.py - Add proxy lcl_resolve_dotted_attribute() function to deal with different versions of python * server/BuildJob.py - email_to -> self.username in email_result() * server/UserInterface.py - email_to -> email in email_result() - Fix typo that broke simple SRPM building (cvs_tag -> srpm_file) * server/buildserver.py - Deal with m2crypto 0.09 X509 cert handling of 'emailAddress'/'Email' * utils/package-builder.py - Deal with m2crypto 0.09 X509 cert handling of 'emailAddress'/'Email' - Fix simple SRPM building, our local enqueue_srpm() function was calling just enqueue() on the server 2005-06-15 Dan Williams * client/buildclient.py client/CONFIG.py - Fix up the XMLRPC server so its port is configurable, to support more than one build client running on the same machine 2005-06-15 Dan Williams * server/UserInterface.py - Add a list_clients call that returns a list of currentl active build clients * server/client_manager.py - Return list of active build clients * utils/package-builder.py - Hook up list_clients call, and refactor client printing code into new function use by both list_clients and update_clients 2005-06-15 Dan Williams * server/BuildJob.py - Fix up build failure email to include status on each sub-job - Fix email_result since we use email addresses as usernames * server/UserInterface.py - Fix email_result since we use email addresses as usernames 2005-06-14 Dan Williams * client/buildclient.py - Write mock output to a log on failure * server/buildmaster.py -> server/BuildMaster.py * server/BuildJob.py server/UserInterface.py server/buildserver.py - Fix for buildmaster.py -> BuildMaster.py * server/BuildJob.py server/BuildMaster.py - BuildMaster object now has a 'createrepo' method, which BuildJobs call when they need to update the repo. BuildMaster.createrepo() is locked so that we never run two createrepos at the same time * server/CONFIG.py - Clarify some options * server/UserInterface.py - Remove list_waiting_jobs and list_building_jobs, we now have a more flexible "list_jobs" interface that will be extended to allow more search parameters - In update_clients(), return new clients that we've found * server/client_manager.py - Print out clients we find when we start up - In update_clients(), return new clients that we've found * utils/package-builder.py - Implement most of remaining functionality: - hook up enqueue and enqueue_srpm commands - Add a list_own_jobs command - grab user's email address from certificate or ~/.package-builder - Allow non-SSL connections to build server 2005-06-14 Dan Williams * common/SimpleSSLXMLRPCServer.py - Fix for usage on Python 2.2.x, which doesn't have SimpleXMLRPCDispatcher 2005-06-14 Dan Williams * server/CONFIG.py - New options "ssl_frontend" and "ssl_buildclients". Only ssl_frontend is used at this time. Setting it to True requires use of SSL to submit build jobs and get server status. Setting it to False allows use of xmlrpclib connections over HTTP, no certificates involved. * server/User.py - Add guest member to User class * server/buildserver.py - Move UserInterface class to UserInterface.py - Re-enable the non-SSL XMLRPC server code - Switch XMLRPC servers between SSL/non-SSL depending on CONFIG's ssl_frontend value - VerifiableSSLXMLRPCServer -> AuthenticatedSSLXMLRPCServer * server/UserInterface.py - Generalize UserInterface into a base class - Implement UserInterfaceSSLAuth to handle SSL authenticated requests - Implement UserInterfaceNoAuth to handle non-SSL requests 2005-06-13 Dan Williams * client/buildclient.py common/FileDownloader.py common/HTTPSURLopener.py common/SSLCommon.py common/SSLXMLRPCServerProxy.py common/SimpleHTTPSServer.py common/SimpleSSLXMLRPCServer.py server/client_manager.py - Clean up cert handling by stuffing cert file paths into a dict * common/SSLCommon.py - QuietSSLServer: new class to quiet certain SSL errors we don't care about * common/SimpleSSLXMLRPCServer.py - Add an authorization framework. The actual request gets authorized in SimpleSSLXMLRPCServer, but since we need to pass the authorization object back up to the Instance class, we need to override a bunch of functions in superclasses just to pass the object through. Subclasses of SimpleSSLXMLRPCServer don't need to use authorization, they just don't set an auth callback. If authorization is in use, the subclass is asked to authorize the connection, and passes back an arbitrary authorization object, which gets passed to each Handler method. * server/buildmaster.py server/buildserver.py - Change buildmaster_db -> jobdb * server/buildserver.py - XMLRPCBuildMaster -> UserInterface - Attach authorization to all public XMLRPC methods - VerifiableSSLXMLRPCServer: new class to handle request authorization - Add "guest" user support. If the user's cert checks out but they are not in the user database, they have read-only permission to the server * server/User.py - New User object: encapsulates all permissions that a user might have. - New UserAuthentication object: looks up user authentication info in a user database and creates User objects * utils/user-manager.py - Local user administration tool * utils/package-builder.py - New CLI front-end to the build system that support XMLRPC over SSL * utils/certs/fedora-upload-ca.cert - Fedora CVS Upload CA certificate, required to validate users 2005-06-13 Dan Williams * common/HTTPSURLopener.py common/SSLXMLRPCServerProxy.py common/SimpleHTTPSServer.py common/SimpleSSLXMLRPCServer.py common/SSLCommon.py - Refactor _initSSLContext() usage into SSLCommon.py * common/SimpleSSLXMLRPCServer.py - Route all requests through new class VerifiableSimpleXMLRPCRequestHander which has the ability to callback the server for client certificate verification. 2005-06-12 Dan Williams * Require a "hostname" argument to the build server. But we get to remove it as a config option from CONFIG.py then, which is good :) * Implement Additional Package Architectures support. This is essentially an look-aside file listing packages and other arches these packages should build on, but that a whole distribution isn't built on. Examples include OpenSSL on SPARC, where sparcv8 and sparcv9 optimized builds are produced, but not an entire sparcv9 distro. See the "addl_pkg_arches" directory for an example file, and the CONFIG.py file for more information. * Simplify build client tracking on the server. There is now only 1 instance of a BuildClient (formerly BuildClientInstance) class per build client. Previously there was one instance for each arch the client supported. Since at this time we don't allow more than one build job per client, its un-necessary to track anything per-client-per-arch. * Return the build job UID when queuing a job * Start jobs on clients immediately when creating them, instead of putting the server-side tracking object in 'initialize' state first and then starting the job on the client in the next process() iteration. * Do some locking when the BuildClient class gets the job the client is currently running. Because BuildJob instances run in their own threads, the BuildJob could be starting a new job on the BuildClient while the BuildClient instance is getting status. This could result in a race condition and potentially two jobs thinking they have started concurrently. * Add a "Getting Started" section to the README file 2005-06-09 Dan Williams * client/buildclient.py - Add sparc32 and sparc64 bits 2005-06-09 Dan Williams * common/FileDownloader.py - Don't traceback on refused or dropped connections * server/buildjob.py -> server/BuildJob.py - BuildJob objects are now threads - Clean up console output a lot - Delete arch-specific jobs that don't actually start * server/buildmaster.py - Clean up console output * server/buildserver.py - Clean up 'enqueue' command return messages * server/client_manager.py - Try to tighten up exception handling around socket operations - Don't print so much stuff to the console - Prettier printing of downloaded file lists from client 2005-06-09 Dan Williams * client/buildclient.py - Fix up some log messages to be shorter 2005-06-08 Dan Williams * Fix SRPM only mode: - server/buildjob.py: In 'prep' stage, make sure we know where the SRPM is, and don't try to delete the checkout dir when in SRPM-only mode (since there is no checkout dir) 2005-06-08 Dan Williams * Convert all client/server communication to SSL. You will now need certificates (see the README file for how to set them all up) to get them to talk to each other. Now requires m2crypto module as well. 2005-06-08 Dan Williams * Refactor FileDownload.py and FileServer.py, moving them into a shared directory common/. Fix client & server to point to these modules and implement callbacks in each that are required by FileDownload.py. Since at this time we don't install anything to site-packages, the build-client and build-server scripts are necessary to properly set up PYTHONPATH to find stuff in the common/ directory. 2005-06-07 Dan Williams * Rework much of the build system to support HTTP transfer of SRPMs to build clients, and of logs & RPMs back to the build server. Simple SRPM building is broken right now but will be fixed soon. 2005-05-22 Dan Williams * client/fileserver.py - Add a simple HTTP server for package download to build server. Its integration into the archwelder is not yet complete. 2005-05-12 Dan Williams * Add an SRPM build mode that short-circuits CVS checkouts to build an SRPM queued by the user. This mode CANNOT be enabled at the same time as CVS checkout mode, since its less secure. It should only be used for local building and development. To enable this mode, set the 'use_srpm_not_cvs' config option to True. enqueue_srpm(, , , ) 2005-05-12 Dan Williams * Add ability to drop ArchWelders and the jobs they are currently building, and to find them again when kicked to do so. To do this, config file for the server got changed to CONFIG so we can reload(CONFIG) and get new builders on the fly 2005-05-11 Dan Williams * Add README file explaining stuff about build system architecture