2000-05-23 Preston Landers * Added the standard GPL boilerplate comment headers to some source files. * 'dry run' option also works for --refresh (no local server .def files will get overwritten with pagecast -n -r.) * Created @pcupdateurl@ sed-replacement string This is currently defined in cf/Makefile.defs as http://pagecast.sourceforge.net But I wanted to centralize its definition. * Updated FAQ with clarification about --refresh overwriting server files & server naming. * INSTALL whoopsie, says to edit cf/pagecast.conf not etc/pagecast.conf. Fixed in CVS. 2000-05-22 Preston Landers * implemented the 'dry-run' option -n --nosend [Except for the 'dry run' message at the end, it will *look* like it's contacting hosts, but there will be no network traffic] * thread check speed up a little bit, should make it subjectively a little faster without burdening the CPU * a missing self.G was causing bug 106204, now fixed in CVS * update.py basically works now, use pagecast --refresh * long options introduced * More oopsies in 2.0.0 release: * Mail doesn't work currently * fixed bug where stats weren't initialized * should update mail server interaction to handle 'host not found' more gracefully * began work on update.py 2000-05-21 Preston Landers * Streamlined pagecast-2.0.0.tar.gz released; no make install. various other little fixlets and stuff, doc updates, TODO stuff updates (for 2.1) * If you're having problem with the Makefiles (/cf/Makefile.defs not found) try editing the top-level makefile. Change the export... line to this: export PAGECAST := /path/to/pagecast-2.0.0 I still haven't got a good compatible solution to this problem that works with older Makes. * Various server updates. Would like to build an auto-distribute system, perhaps using Sourceforge. That will probably be in 2.1. * This ChangeLog hasn't been kept up to date (although there hasn't been much activity. Getting ready to really, really, release 2.0. 2000-03-02 Preston Landers * Added a Python version check, current minimum version is 1.5.2 (Thanks to Fredrik Lundh ) This version is required because of bugs in SMTP, ConfigParser, and urllib in earlier versions. 2000-02-23 Preston Landers * I've made a few changes in the past week or two without documenting them here. To the best of my recollection, they are as follows: * GNU Make is required. Changed default target to install rooted in the 'installed' dir of the source tree and create a link to the executable. I tried to get it to work with BSD Make but I couldn't get a few things to work; maybe I'll try it later. BSD users can use GNU Make (gmake) until then. Makefiles simplified and configure script no longer neccesary. * Various documentation updates all over the place. * French language file added. * Several German oriented servers added, a couple of them seem broken. * Added "Prohibited" server group list * Added "broken" group * Fixed some server definitions, notably Excite. * Multiple groups can be specified on the command line separated by commas * Preparing for Beta 2 release. Want to have .deb of beta 2 availible. 2000-02-03 Preston Landers * Too many changes to list individually. Makefile support, but not ./configure script yet or packages... all in good time. * Be sure to edit cf/Makefile.defs to set important target installation * Lots of little bug fixes and cleanups 2000-01-19 Preston Landers * Updated README and FAQ, including new information on how to define new servers. * Improved configuration file handling, can read from multiple sources. This is in preparation for possible ./configuration support. 2000-01-17 Preston Landers * Back after a long break. * Added .def files for all the search engines that were present in released 1.1.1 (except Canada.com, see below) and corrected various bits of data in the definitions. * Implemented verbosity in the printing function. Incrememt verbosity with -v command line option (can use more than one) * Took out the silly security check * Improved results handling, should catch and mark virtually all errors * Have to drop Canada.com for the beta. They have a second "are you sure" page which confuses pagecast. I may add support for this engine in the future. It shouldn't be too hard and I encourage any of you to do it. * Fixed various little bugs and things * Added SMTP support. I think BCC/CC handling may be broken 1999-09-08 Preston Landers * Logging: cleaned up all logging. Now there is no longer a subclass of Lumberjack; it is used directly. Print log events via the pprint function of the Global (G) class. Thinking about translatable strings. * Groups control refined. Configuration is now read from the file groups.conf and servers are explicitely assigned groups from there. Thanks Klaus for the ideas. groups selectable from the command line via -ggroup1 -ggroup2 * Translatable strings support. All the strings that the user sees in normal mode should be translatable. Documentation on how to translate to other languages forthcoming. Many of the debug strings are not yet translateable but it should be very straightforward to do so. Very simple shell script to change the link to the current language: "chlang english" to use the english.lang definitions. * Lots of miscelaneous cleanups * Meta groups? I worked a little sample and then deleted it; I'm thinking 'groups of groups' but not sure yet how it would best work. 1999-09-05 Preston Landers * generic.py (GenericHTTP.submitURL): Thread control fine tuned. MaxThreads now works. * defineable email 'key's in GenericHTTP modules. (Before, only one key ["email"] was used.) * Groups seem to be working. Basically groups work like this. Each search engine .def file can define whatever groups it feels it belongs to. However, that search engine will be activated if and only if *at least one* of those groups is marked "on" in the pagecast.conf file. * Use of unique names "encouraged" by the template.def.py file. Since the ServerRegister is a Python dictionary (keyed by the server name) it is imperative to have a unique name for each server. * ServerRegister is a simple dictionary used to control which servers are availible and running and so on. It's simply a Python dictionary keyed by a string (the server's name) and whose value is the Server Thread object itself. * GenericMail servers (Infoseek) temporarily disabled until full mail/SMTP rewrite is in effect. * Created working server definitions for all 'known' servers * Need to write much documentation * Squashed bug when fewer threads than the MaxThreads were started. * Released 2.0 alpha 2 -- updated docs a little, especially TODO 1999-09-03 Preston Landers * submitengine.py (PagecastLog.__init__): StatusLog deprecated; use Status * Major overhaul to how individual servers objects are configured and instantiated. All .def files in a certain directory ("./servers/") are executed as mini-Python scripts. The format for the .def file is relatively simple and will be documented and refined soon. * Class system abstracted by one level for Search Engine (interaction) objects. Same general interface is used with any type of interaction object. Basically there is one type of server thead with a very simple interface to its interaction object (the SEO). IT's up that SEO to configure itself appropriately and execute. * Primitive support for Server Groups added. Group control (and hence individual server control) needs to be implemented. All servers in servers/*.def are automatically run (currently.) Also, some security checks NEED to be made. * bug in forming urllib request (params?) * VERY crusty 2.0 alpha 1 development release 1999-08-30 Preston Landers * Proxy support enhanced; uses urllib exclusively instead of httplib, support for redirects integrated * G is now passed to all classes' __init__ so it's an instance of each submitengine object (rather than being a global instance of the whole module); this means a program that calls submitengine can have multiple separate instances of it. 1999-08-29 Preston Landers * 2.0 work begun; TODO list prioritized * Switched to httplib/urllib (instead of old requests.py) * Proxy support * Excite definition tweaked 1999-08-28 Preston Landers * Debug -- added -d (or -v) command line debugging option that will print extra informative statements * Documentation updated -- added information about how to define your own search engines. Also update TODO, NEWS, and web page * Release -- Pagecast 1.1.1 packaged. This will be the last 1.1 release probably. 1999-08-25 Preston Landers * Underscores removed in all .conf files. This was confusing older versions of ConfigParser.py * Definitions -- added 3 new definitions courtesy David Emmerson 1999-07-23 Preston Landers * Release -- Pagecast 1.1.0 packaged up. README, the web page, and other documentation updated. Added NEWS and TODO files. * Trivia -- Pagecast now consists of 1623 lines of Python code. * Mail interface -- had to go back to using tmp files; I thought it worked the way I was doing it, but more extensive testing revealed otherwise. Someday I'll have to clean that up a little. * README updated -- installation procedure changed slightly. The pagecast archive expands into a pagecast-1.1.0 directory, and you need to symlink that to pagecast. 1999-07-22 Preston Landers * Config option -- response_subject in the [Mail] section defines the subject line of responses Pagecast sends out. * ControlThread eliminated altogether. submitengine made a class instead of a function and does what ControlThread used to do. One less thread means less memory and CPU usage, which is a good thing. Still need to do one more cleanup in future versions to make it completely object-safe (fix the G global variables mess.) * GTK+ -- preliminary GTK+ support. All the app does is run the program in a graphical console; nothing special. This may or may not be improved in the future depending on interest, but probably not by me; I have more interesting projects to work on. This does require PyGTK. 1999-07-20 Preston Landers * Preparing for 1.1.0 release; more doc strings, etc. README and other docs updated. * Pretty_Link -- link checking added in both strict and non-strict flavors. See README for details and pagecast.conf for some examples. * AltaVista -- added a new failure string: "page is no longer valid" for when AltaVista can't connect to the URL. * Kernel oopses -- might be caused by compiling Python with GCC 2.7.2.3. Solution is to use EGCS (or possibly later GCC?) to compile Python. * SearchEngineObject -- Fixed bug which causes some errors not to be counted. * Lumberjack -- hook interface rewritten to be cleaner and simpler (for the user.) Hooks can now modify log object text items. They should return None if this is not desired. Lumberjack is now nearly "module-ready" to be used in other projects. This was the original intent all along; Lumberjack was part of another project then I cut-n-pasted it into Pagecast. Now it's getting back to its object oriented roots. * Lumberjack -- added a test suite 1999-07-19 Preston Landers * ControlThread was written incorrectly (all the action was in start() instead of run()). My bad. This was fixed. Not sure if this will fix the Linux 2.0.36 problems; awaiting reports from users. * #!/usr/bin/env python used now for ease-of-use. Procmail must therefore invoke the python interpreter directly but that's not a big deal. Changed the procmailrc to reflect this. * FAQ document created with two entries. * BCC/CC: added (blind) carbon copy capabilities in mail replies * Bug: seems to be a bug in counting the number of error hits. Maybe need to double-check success and fail codes for the servers? * Hooks added to the logging facility; this should make integration with a GUI interface easier. * Many code cleanups. Lots of comment strings added. Should be easier for future maintainers to identify what's going on where. * Meta Tags functions moved to separate file for readability / maintainability * Logging classes moved to separate file (so it can be used in other projects.) Log code cleaned up and made more extensible. 1999-07-04 Preston Landers * 1.0.1 release -- also known as brown paper bag release. * Google changed its interface slightly (to foil robots like Pagecast?) Updated servers.conf to reflect this. * Updated the README and other documentation. * Fixed the mail-handling code (was trying to read from a file instead of stdin). Thanks to Joshua Levitsky for pointing this out. Also fixed the procmailrc. 1999-07-01 Preston Landers * First release.