Well, it's been almost 2 years since my last update, so I guess it's time for another one. I have been making lots
of changes lately to improve my Search Engine Optimization, and advertising income to help pay for the site. A lot
of the changes have been made based on suggestions from a very helpful affiliate manager at one of my affiliate
sites. There are many obvious changes, and a few less obvious ones:
Removed the google banner ad at the top of my home page.
Added rotating ads to my affiliates
Started replacing most of my Perl based pages with Web 2.0 pages and Perl XHRs
Major performance changes on database side to allow autocompletes to finish in our lifetime - still needs
Improved the Autocomplete search boxes and their landing pages
Actually created a landing page for the subject search
landing page show's matches for you search
matches are updateded in real-time as you change the query
matches tell you a little information about the match, including the number of servers I know have a
Added "Quick Searches" which cover about 90% of what people search for
Added a list of free usenet trials
Started removing links that nobody but google and yahoo use
Due to a freak hardware problem, I lost all of the data in my database. I was able to recover enough data from an
old backup to get the site back operational, but all statistics, such as votes, have been reset.
Major changes in the backend that provides better connection between some of my sites.
Clicking on the now goes to the re-vamped webnews gateway (upgrades still in
Thumbnails from my Usenet Pictures site are now
inserted into messages read in most binaries groups.
Removed most of my banner ads since I am not making much money on them anyway. I am just keeping top ad(s)
on each page.
Trying to dream up an all new look and feel for the site. It has been lime green, black and white long
enough. Please send suggestions, even another site that you think looks
"kewl" would be helpful. I just want something new.
Replaced a very old server that was in charge of my dns stuff with a newer machine that won't lock up at
Lots of DNS problems occurred with several of my domains in August. Some caused by the above machine,
others caused by an issue with my registrar.
First of all, I apologize for the long time since I last made any useful changes to the site. My wife died
last year from cancer, and I haven't really felt like messing with this stuff for a while.
I am making cosmetic changes to the site again. I have had a lot of complaints from users that say they
have difficulties reading some of the pages for various reasons, so I have modified the CSS to use more standard
fonts and font sizes. I am using suggestions I have found on: anybrowser.com.
I have made a bunch of changes for performance recently including a new database server, parallel web
servers behind a load balancer and analyzing most of my SQL to improve index selectivity.
Because of my wife's illness, I have never gotten any further with the drill-down logic in the search page.
This will be high on my list of things to finish in the coming days (weeks? months?)
Systems are all operating pretty well, and load has not gotten too high on the high-speed line yet, so I
guess I am happy.
I have gotten a new machine to replace my web server I will start transitioning all of my sites to it
shortly. New machine has much faster processor/disk/ram than the current box, but I will still be limited by the
Database machine. Just trying to eliminate one more possible bottleneck.
I have changed the search to not limit your results based on the number of groups that match your searches.
However, I sill only show the first 100 matches so if your search matches a couple hundred groups you won't get
I am beginning to work on the drill-down logic for the search page. It should be available in the next few
We're Home! As of this morning, I have moved all of my machines back into my
house. I have a new high-speed line, and all the addresses are fixed. You should start seeing my server list
getting updated more frequently now that I now longer have the bandwidth restrictions I have been working under.
New database server has been installed, I just need to do some regression/burn-in testing on the machine,
and then I will transfer the databases to it. This new machine seems to be about 3-4 times faster than the current
one according to my initial tests.
Lost a hard drive Yesterday I lost a hard drive on my database server. I have
finally gotten it swapped out, but I had to reload from a database backup and have lost approximately 28 hours of
activity and updates. Everything should be normal again in a few hours.
I am moving I have had many problems with my servers at their current location,
and I have decided to install a High-Speed line at home so I can more easily maintain the systems. This will be
occurring before the end of July, so there will be an outage for part of 1 day. I plan to purchase a new database
server and parallel it here at the house to reduce my down time.
I have made changes to my data gathering software which should result in more accurate article counts. I am
using some of the information gathered by the news reader and a little
additional information to correct the article counts in groups that people are reading.
New Server Page I have added a new server page here. This version allows you to better pick
and choose what information you want to see/sort by in the server pages. I will be doing the same thing in the
groups page shortly.
Massive problems with the INN server on my news machine have resulted in extended down times over the past
week or two. I am working on it, but it doesn't look good at the present. Unfortunately, the news server is also
my database server and I am running out of free memory as I continue to increase the number/size of articles I
Hierarchies are no longer restricted (for those that have not already noticed). I do some short-cutting
however, to improve search performance (comp.* gets a '^' prepended, etc). So if you expect a group and are not
seeing it, you can try to put the right hierarchy (de.comp.*).
I still filter a lot of group names because they are not syntactically correct. For example alt.ABCD is
wrong because it contains capital letters. I also block nuisance names, such as groups with 9 or more parts (ie.
It's gone!!!! Okay, I hated the popup as much as anyone else around here. So
when I added it to my popup killer, I decided it was time for it to go.
They're gone!!! I finally got rid of the "arch" on my tables. I recently got
some complaints, and when I tested it, the link didn't read news. Guess you will all have to live with my reader
from now one.
I am making some more changes to my service. I will probably get rid of the "hierarchy" restrictions in the
next few days, and add some nice "drilldown" features to allow for more flexible reporting.
Still no statistics, but I am switching to a new log analyzer, and I should have some in a couple of days.
Major upgrade to primary WEB/DNS machine done in the past 24-hours. Didn't plan on it, but I lost my old
1GB boot disk and had to rebuild the system. I figured while I was at it, I might as well pick up a newer version
of the OS.
No new statistics right now, I have gone to using multiple web servers and my statistics data isn't merging
I have added "poor mans" load balancing using multiple web servers and multiple DNS entries with matching
names. It is working real well and both servers are lightly loaded most of the time.
Upgraded to PostgreSQL 7.2 - this new version does some real nice things for my performance. Also, it is no
longer necessary to shutdown the server around 5am for maintenance with this version.
Occasionally, I was having a problem with "Too many clients". With the upgrade of the Database Server and
multiple web servers, I have also doubled the allowed number of sessions.
I am adding a new affiliate program with thenewsgroups.com. They
have a similar offering to my current affiliate usenet-access.com
but claim to pay me more money per new user I send them. I am going to start alternating them and see who performs
As usual, I have talked thenewsgroups.com into giving me an
account so I can add them to my index. You all can see my results for them within about 48 hours.
I have installed a new database server at my T1 site. It is now sitting there about 95% idle all day long.
This should correct the long response times I have been seeing.
My news reader is working very well now. It allows reading news from any server (including commercial
ones), and has a really neat killfile feature. Try it out if you like, or need, a web based news reader.
I have modified a lot of my result screens based on suggestions from various users to make the results
easier to use and work with.
I have re-structured the commercial page to break up
all the commercial usenet news stuff I have into different categories.
If you can see this, then my site conversion to use PostgreSQL is complete.
I decided to change the site to use PostgreSQL because MySQL has been documented (time and again), to scale
poorly with too many users.
Also as part of this conversion, I have converted everything to Perl. With mod_perl running, I get at least
as good performance as PHP, and using DBI I can write my SQL code once and reuse it just by changing the DBD
I have begun converting this entire site to use CSS1/2. This means that for those of you using older
browsers, the site might become a lot less colorful. However, I am trying to make sure that even without CSS, the
site still is at least as usable as before.
I have put a brand spanking new Motherboard with DMA100 capabilities and a 40GB DMA100 Hard drive into the
Database server. While doing this, I discovered that the server's 866 PIII was jumper selected to run at 600MHz.
Performance of the database should once again jump.
For those of you that have been helping me Beta Test my search
engine. Thanks, I have learned a lot about necessary performance changes and implemented most of them. It is
still not blinding fast doing a search, but it continues to improve.
A special thanks to those of you using my search engine to search for content that I didn't want it to
index. I have increased my content filtering a hundred times since you all started hitting it.
Finally hit 1 million hits in a month, and the machine actually survived it .
I have now replaced the web server. I took out the 200MHz Pentium that has been my web server since about
1996, and replaced it with a 466 Celeron. I haven´t gotten paged because the server load was too high since.
I have been looking for some more appropriate affiliate links for my site, and am getting setup for several
right now with various commercial news providers, plus some sites that sell utilities for news reading. Look for
them in the next couple of weeks.
Moved some of my services to another machine to reduce the load on the server again. Performance is
improving, but I will need to get one more machine for backend support to make further improvements.
Added a new advertisement from valueclick which seems to be working pretty well. Thanks for those of you
that have filled out the popup.
For those of you that hate popups (like me), let me know. I might
be convinced to eliminate it if enough people scream.
I have been getting a lot of emails (automated by me and from users), that the index is running slow and
getting "Too many connection" errors.
I seem to have tracked the problem to my recent upgrade from PHP3 to PHP4. Don´t understand why it is
doing it exactly, but apparently this version of PHP4 doesn´t always reap its children in a system call,
which leaves hung connections to the db.
I have upgraded from PHP4.0.4pl1 to PHP4.0.5 to see if it fixes the problem. Holler if you see anything
Updated Mysql version to a more recent release. Performance seems much better now.
Fixed SELECT code for newspage.php3 to sort new (unvoted for) pages as if they had zero votes instead of a
NULL vote value. This places new pages in the middle of the results instead of hidden at the end.
Made similar changes to newssrch.php3. This was resulting in servers that had possibly good groups not even
getting a chance to be voted on because they were considered to have less votes than servers that had negative
I have added some more hierarchies to the search engine. I am now indexing groups in: de
Traffic has really been growing lately. I have bumped my machine up again in performance. The database
machine is now operating at 866MHz with a 133MHz memory bus.
I upgraded the system memory to 256Meg PC133s at the same time
I also upgraded the hard drives that the database are stored on in January. I have dual UDMA66 IDE drives
on separate IDE busses in a striped configuration using vinum
Yes it screams .
I am now working on tuning the kernel for the increased memory available. My processor is now averaging
about 85% idle all day long, because I am getting a boatload of wait I/O. But I have only a 30MB cache and 173MB
inactive memory so I have a few ideas where improvements might be made.
I have moved the database to a mysql database server on the same machine as the PostGreSQL server I use for
my search engine. I am now able to index at least half of all working
servers every night due to the improved performance. Guess it´s time to retire that old 266 P-Pro
I have added some more hierarchies to the search engine. I am now indexing groups in: 3dfx, borland,
inprise, sybase, webring
Scorecard is starting to get quite a lot of hits (including, I notice) from people I am harvesting. No
backlash so far, so I guess that they might appreciate it. I am thinking about providing a service to them to tell
them what servers are no longer working for them.
Site has begun to throw out pages saying that the database is too busy. I have increased the number of
available connections, but this is not helping. I may move it to a different database machine, and/or a different
database package such as PostgreSQL.
Welcome back to school (and for us older people, congratulations on surviving another summer alone with
your children ).
Stats were way up in August, but seem to be returning to "normal" for September. Must have been everybody
getting back from summer vacation at once.
Site seems to be operating at or above my expectations for performance (considering the number of hits), so
I haven´t tweaked a darn thing this week.
I have put up my scorecard page. You should go
have a look, a few of the stats are quite interesting.
Guess that´s it. I have a couple of new things to add when I get a few extra minutes/days, I will see
what I can do. At present my TODO list looks like this:
Utility that can connect to a web-based reader and get some stats.
Page that ranks some of the commercial web-based news readers
My own web-based reader (for fun or profit )
Program to try to start harvesting my own newsfeed for open servers. I have the data, why don´t I
start using it.
Try to get more of the commercial services signed up to be indexed and ranked.
If you can think of any other ideas for new pages/features, give me a holler.
Okay, first off, here are the statistics for the last 4 months. There is probably nothing in life as
satisfying as procrastinating .
I have been making a lot of changes to improve performance of the search engine and results pages.
Changed software to only keep track of groups on servers that actually contain articles. Listing a bunch of
empty servers because they have created the group is a waste of everyones time.
During my changes, I accidentally reset the status flags for all the servers in my database. This has
caused a lot of old servers to show up as being active when they don´t actually work. This should be cleared
up in a couple of more days. If not, I will manually fix the remaining servers next week some time.
I am either looking for new web-based news reading software that I can host myself using either Perl or
PHP3 or a combination of the 2, or a new partner that would like to pay me for sending them traffic. I
haven´t received any checks since the start of the new year from my current partner.
In the interim, I went ahead and modified the redirect code to correctly connect to a server AND
news group on my current newsreader partner.
I have noticed that a significant number of the servers listed on the sites I am harvesting from are broken
in some way. I have started tracking the statistics for this and will put up a scorecard page RSN.
Its fixed. Thanks to all of you for your emails about the problems with the system, I thought it was
working until I heard from you. I apologize to everyone for the problems lately. I made a change to help improve
performance of the harvester, and somehow I ended up indexing ALL groups. This caused the database to grow
dramatically, and then something got corrupted. Anyway, it all seems to be okay now, so have fun. I will try to
post a statistics update later today or tomorrow.
Update 2000/04/17 (tax day)
First, the statistics for February and March
Although this looks like a serious fall off in use during March, it is actually not. My logs have a "hole"
from part of the 2nd through part of the 10th. I don´t know where they went, but it is roughly 9 days of
logs that are missing (1/3 for the month) with only a 20-25% falloff, so this actually indicates an upswing 8).
I have once again dropped the banner ads for usenetserver.com. My link to a signup page has not worked
since I reinstated them on my pages, and they have not included my site as a referral on their main signup page,
so I have found another company to work with. I have changed my affiliation and now have links to Usenet-access.com. So, if you are thinking of
paying for UseNet News, please do it through this link, I can always use the extra cash to help run this site.
A big apology to everyone for not keeping this page updated more frequently. I have just finished several
large contracts and should have a little more time to keep this page updated.
Alright, first I would like to start off with a statistics update (it has been 5 months since the last and
things have changed a lot.)
Many changes have gone into the harvesting software to help prevent even more false positives on posting. I
used to accept a "200 (posting ok)" message at any time from a server. The problem is that a lot of
servers are defective and send a "201 (no posting)" when you connect, but then send the "200" message
when you request "mode reader". Since I got the "200" message after the "201" I accepted it and said posting
was allowed, which was wrong because they had already said "NO". With the latest changes, I get much better
results (although I lost over 50% of my postable servers).
Made a change to improve download statistics for how long it takes to get an active file from a site.
Unfortunately, sites with very small active files don´t send enough data to counteract the effects of the
slow process involved in establishing a connection and requesting the active file be sent. To fix this, I have
changed my timer to start after the first line of the active file is received which gives much better results and
does not penalize small servers as much.
Major modifications to the newssrch page. It now caches the results and will NOT try to create new results
until after the next time the harvesting software runs. Since I do not harvest during the day this should give a
significant increase in the number of pages I can serve each day during the busy 12:00-18:59 CST time slot.
The new search software also has the ability to know if another person is already searching for the same
information. If this happens, all subsequent searches will pause until the results from the first searcher are
made available and will then send the cached results to the visitor. This can cause a apparent delay while doing a
search, but it reduces the load on my server when 7 people all look for the word "pedo" at the same time. It also
is ONLY an apparent delay. The search is faster for the initial searcher because of the decreased load, and
subsequent searchers only have to wait for the remaining time for the results to be made available, so everyone
My statistics have dropped off significantly and consistently from month to month since I blocked all of
the pedophile groups I could identify. Although I do miss the amount of traffic (and resultant income from
impressions), I can´t say that I regret my decision, and I do continue to monitor for those groups and block
I actually got a couple of affiliate checks recently (which helps me keep this site up). Even though my
hits/page views are down, my actual click-throughs have gone up. Apparently the visitors I lost weren´t
clicking on banners anyway.
Modified the harvesting script to avoid running during the 12:00 to 18:59 CST time period. This is when I
get most of my hits and queries should be faster if there are no locks on the database caused by data inserts from
The new server is rock solid so far. I have been very pleased with both its reliability and responsiveness
to heavier loads.
My stats were down some from July, but that isn´t too surprising with 5 full days of down time and
bad performance when it was working.
I have several people using my list of servers on the FAQ as their own personal bulletin board. I have now
modified my software to deactivate pages which have no news servers listed.
I have shortened the amount of time my script is allowed to run to only 10 minutes per hour. At 15 minutes
per hour it was nearly done with the days work by 2:00 am, so I am trying to "smooth" the bandwidth.
The new database server is up and very fast. Everyone have fun. All limits have been removed (at least for
this weekend) to get an idea of the actual load this server can take.
First, I would like to give a little statistics update for July (it has been a while). These statistics
include approximately 13 operational days at new location.
Added count of number of servers listed on each harvested page
Setup new database server machine at new office site. After I finish converting my software/web pages, all
limits will be removed and we'll see how well she runs.
In order to avoid violating my current lease I have implemented a spin lock on the search engine. This will
only allow 4 searches at a time and should help reduce my CPU overhead plus reduce my bandwidth usage.
I am so confused . Why does moving your computer 15 miles west make
it stop working? I think I have figured out why it is not updating the list of servers and should have them all
reprocessed by tomorrow. Sorry (AGAIN) for the inconvenience.
Thank God for backups. After I moved my server to my new location with the T-1 I lost one drive and then
later the other one. Not sure if it is bad power here or what.
UPS coming in tomorrow.
Moved my server to a T-1. I am not sure how well it will stand up to the potential load, but we can give it
Began doing some cleanup to improve the way this site looks in Opera and WebTV.
I have added a navigational box to all pages to make my and hopefully your lives easier.
Added the index for the commercial servers (currently only 1 ). I
also sent email to several of the other companies providing this service to see if they would like to participate.
More spell checking and grammar checks for most of the pages.
I have made several changes over the past several days and forgot to change this file (sorry).
Added new searches which categorize servers by how fast they send me an active file update, and how many
new articles/second they have received since the last time I visited them. These should give people a good general
idea of what access speeds are like from Missouri, USA to servers, and also how well connected/up-to-date servers
All code is now in place to start a mailing list of new servers as they are added to my list, and old
servers which have closed down. If anyone is interested in this, let me know and I will set it up.
Back from vacation. I recolored all the pages to black text on white background because I have had several
people complain about reading some of my information.
I rearranged the entry page to make it a little shorter and hopefully to improve the readability. I noticed
I tended to ramble in a few areas.
I have updated my list of web pages to harvest - thanks to Brian Kraft.
I have added the capability for people to post web pages and servers that they have found directly to my
Setup the necessary database for a daily/weekly newsletter of new servers to be added after my vacation.
Added performance statistics for my own download times when getting the active lists from servers, etc.
This data will be used to add new reports after vacation based on how "quick" and "active" servers are.
I have been making major changes to different portions of the site. I added links to USENETSERVER on the opening page because they have an affiliate program that
I have joined. I have also added a list of many (most?) of the commercial servers that are available on the
internet. I hope to add a search engine that only indexes the commercial sites and ranks them by number of
articles, performance, &etc.
The other changes I have been making are in the databases and indexing software. There shouldn´t be
anything real obvious yet, but look for some new features coming up soon.
I have made changes to the entry page to make it more readable (I hope) on WebTV. If these changes make it
worse I hope that some of my WebTV users will let me know.
During my changes to modify how my news server news.maxbaud.net is handled by the
index I made a mistake. It was one of those "well everything is working perfect... oh yeah I need to change one more
thing" things. Forgot to test my 2 line change and I apologize, I will try to do better next time.
Housekeeping has now gone into effect. I am looking for stupid things, like typos etc. I have
also decided to show the most popular, fastest, most complete and most current sites and groups in separate pages.
Got rid of that damned annoying voting box. It now opens your news reader and then clicks through to the voting page
in the original browser window. This should be much less stressful.
I decided that too much of my entry page was being taken up by the list of site changes. To help reduce load time I
have now moved the updates to /change.htm.
I hope this will help (once again) to reduce the bandwidth requirements of my site.
I need to get different software. I can't get my final monthly logs
until after 18:00 on the first of the month. I have been going nuts wanting to post these all week. Last month my
site received over 492,000 hits, performed almost 299,000 page views and had over 78,000 sessions, it also sent
over 3.2GB worth of files. Not bad for a 33.6 modem, huh?
I haven't heard any feed back about the new voting system. What does everyone think? Is it just a waste of
bandwidth or a valuable service? Does anyone have any suggestions of other features that could be added to the
voting procedure, or better questions? Let me hear from you.
I am very pleased with the performance of my new site and greatly appreciate all your terrific feed back. I have
only had a few complaints and hundreds of compliments. This latest change to the site is to help get rid of those
few complaints I am still seeing. I have now added a voting feature to the site so you can see what others have
thought about a particular news server or group on a news server. For more details go to the FAQ.
After one full month of running with my new page formats, my statistics have changed dramatically. I had 226,000
page views in April, and sent out 2.5GB of data. I also appreciate the fact that many of you have clicked on my
banner advertising. Lastly, it has come to my attention that there were MANY sites out there which
had direct links to my search engine, with a query string and everything. By looking at my logs (and following the
referals) I discovered that most of these were making very inefficient and CPU intensive searches (such as "pre"). I
have decided to block this for now and see how things go. I appologize if this is an inconvenience for anyone.
I am running this site across a 33.6K modem connection to my ISP. The link is used for:
Web site hosting
News server searching
Needless to say, it can get fairly busy. So, if you are having performance problems, please try to be understanding.
With the latest changes, my page views in March rose to over 130,000 per month, and I sent 1.3GB of data to users
like you .