Newzbots.com

Newzbots was born out of a love for football.  I initially called it “Football News Network” and every 5 minutes it did screen scrapes on about 10 different football news sites. It then took the links and titles and displayed them up on a single page. Like the Drudge Report or Feedly. Only it was for sites that I had an interest in.

I had additionally setup a script to go out and pull the data from each article and, through a home grown algorithm, determine what team the article was being written about. Additionally a separate script would figure out what NFL player the article was about. I also had a collection of NFL players’ twitter accounts and would tweet the article to them.

This worked really well and several high profile NFL players actually retweeted or tweeted back. Unfortunately, there was no way to tell if articles were positive or negative, so after a few negative articles were tweeted out, I turned off this feature.

After a couple of years of the website only doing only football news, I added news from MLB, NHL, NBA and Hollywood. I also changed it from Football News Network, to Newzbots. At this time, I also switched from screen scraping to pulling the data from RSS feeds. Still screen scraping, but much easier to gleen data from and less likely to break if a website changed it’s layout.

Once all of these new areas and RSS feeds were added, I found that the machine running the script to figure out what team the articles were about was not fast enough to parse every single article that was being written.  Especially since I started pulling data from over 500 RSS feeds. So I had to turn that script off as well.

I also wrote an iOS app that essentially showed the articles in a slightly better way on the iPhone.

In early 2018, the machine that I had been running the scripts and website on began to die, so I migrated it all up to a server at InMotionHosting. I also did a complete changeover of the website from it’s original layout to using Bootstrap, jQuery and PHP. This made the webpages more responsive and work better for the different screen sizes available to the users of the website.

The scripts to get new articles was stil being run on my home machine. I would simple grab the data and add it to the database on the website. However, I found that the script was doing way to many DB calls.  Because of this, I created an API that allowed me to push new articles to the site without having to continuously conect to the database over and over and over.

The site currently updates every 10 minutes and the web page is dynamic, in that it refreshes itself every 20 minutes. It also keeps track of the latest articles you haven’t seen so that when you come back to it, you’ll know what’s new to you.

I don’t have any future plans for the site, aside from maintaining the data pulls from the current sites and possibly adding some more feeds to the current groups. There is the outside chance of adding College Football into the mix, but it all depends on my free time.