TiVo Replacement, etc. continued

So now that you have read this post, and you have SABnzbd in place, (actually before you use it) you'll need a usenet provider. Most likely, this will not be free (another departure from P2P). However...

Many ISPs provide some type of Usenet access with your account. For instance, I use Comcast, and they give me a 1GB/month account through Giganews, which I access at nntp://newsgroups.comcast.net. You may think a Gig sounds like a lot to download, but consider that a 30 minute show well compressed into XviD is usually around 350MB. That gives you just less than two shows a month to download (my account my be more than that, but it's only marginally more).

The things to consider when choosing a usenet provider are:

1. Retention: How long after someone posts something to usenet can you retrieve it through this service.
2. Connection speed: Can you connect to this server at your internet connection's full capacity?
3. Completion: Once you find what you want to download, will this provider have the whole thing or just parts of it?

There are quite a few different providers out there. I'll discuss only the most popular few.

I use Newshosting and have been quite happy with them for several years. Their website is simple, and my experience with them has been quite good. I would guess that their downtime averages only a few hours a year. Now for the three things I mentioned above:

1. They have good retention (I get around 75 days with a $14.95/mo account)
2. I have a 6Mbps connection and I get that speed (sometimes with spikes much higher) consistently.
3. I have not had any problems with this at all, when posts are less than 60 days old or so.

There are three plans with Newshosting: For $10/month you get what they call NH10: 10GB per month with advertised 70+ day retention. For $14.95/month (NH45) you get unlimited downloads, advertised 45 day retention with a limit of 8 concurrent connections. I have this type of account and I'm pretty sure the 45 day retention limit is only in headers - if you get an NZB file that's older than that, you can still get what it refers to (I've always been able to). Finally, for $19.95/month you get the whole shebang - advertised 70+ day retention, unlimited downloads and unlimited concurrent connections (though I have a hard time imagining how more than 8 connectios would be useful).

Again, I've been thrilled with newshosting and would absolutely recommend them to anyone looking for a usenet provider. To sign up, click here.

Another provider that I've used and liked is Easynews. They might be the most popular provider out there. They have similar statistics to newshosting in the three categories I mentioned, except that their retention is less. From their site: "Retention is one to two weeks in all binary groups, three or more in heavy binary groups." For downloading TV shows, and for people who keep on top of things, this shouldn't be a problem. I remember getting much better retention than that from them. I'm guessing they advertise low retention because they are legally bound to provide at least what they advertise.

One significant difference from Newshosting to Easynews is the type of account you can have. Easynews does not appear to have an unlimited account. In stead, they offer one account type, and that is as follows: $9.95 gets you either 20GB of downloads or 1 month of service, whichever comes first. Then if/when you exceed the 20GB limit, they charge the $9.95 again and your month/download limit starts over. They also explain that NNTP downloads are discounted by 25%. This requires some explanation...

From easynews, you can download directly from their website, over http, or you can connect with a news browser (this is essentially what SABnzbd is). Apparently with a news browser there is a bit of overhead, causing what you've downloaded to be higher than what you've gotten. Thus Easynews gives the discount I mentioned. So If all you use is NNTP through them, you should be able to get 26.66GB downloaded per $9.95.

But wait! There's still more to consider... If you use your computer on the World Computing Grid (a distributed computing program) for at least 15 days/month, you can get 2 free GB of downloads. Also, there is a survey you can take to earn another 2 free GB.

Easynews prides themselves on connection flexibility and privacy. They have many different routes you can use to get to their servers, and they support the NNTPS protocol (secure NNTP) as well as HTTPS. Furthermore, they advertise that they don't log downloads and when you post to usenet (another discussion entirely) they remove any possibly identifying headers. If privacy/anonymity are important to you, Easynews may just be the way to go.

After reading up on them, it is apparent that they have many more features than Newshosting. The main advantage Newshosting has over them is the unlimited download accounts. If you'd like to sign up for Easynews, click here.

The last provider that I'll be discussing here is Giganews. However, it's quite late already, and that post will have to wait for another time.

TiVo replacement, Usenet automation

I recently helped a friend of mine set up his system at home the way mine is set up for downloading and watching TV shows (some specifics of his were a bit different, but the same stuff applies). This has made my life an incredible amount easier, so I thought I'd share with those that want to know.

Intro: (read if you want to know why I care about SABnzbd)

I have three kids, ages 7, 5 and 10 months. I also enjoy a minute amount of prime-time television, much of which is not kid-friendly. Also, TV's "prime-time" is my prime-time to spend with my family. I'm home from work, dinner's over, and I can finally be with my family. The last thing I want to do is spend this precious time staring at the TV.

Several years ago, as file-sharing became popular, I discovered a wonderful thing called Usenet. It's been around since before the internet (for academic discussions mostly) but has transformed in the last 10-15 years into a place where people share their binary files. Just as with all the file-sharing networks, they share music files, pictures, movies, programs, and (drumroll, please) TV shows!!!

Of course, you can also find much of the same content on peer-to-peer (P2P) networks, like kazaa or bitTorrent, but the availability is questionable and I've never been happy with how much work it takes for my computer to retrieve the files from P2P networks.

On Usenet, you simply retrieve what you're there to get. There's a lot more technical explanation that could be given at this point, but it's not really relevant to the point of this post. Simply getting, though, is a benefit because you are not serving back what you've retrieved (potentially sidestepping legal issues), and you don't have to wait for other people to have what you want. If it's on your usenet server, you simply grab it. Also, you usually grab it as fast as your connection is capable of.

At this point I feel it necessary to make the following disclaimer: Usenet and P2P networks are absolutely loaded with unsavory material. Searching through either one, you are almost certain to see filenames for things that you shouldn't want to see. Be prepared.

Another disclaimer: A vast majority of the material one can obtain through P2P or usenet is copyright material. In most cases, it is not legal to download or use. There are gray areas, however. I push my use into a gray area by saying that what I'm downloading could not be purchased today, and is not something that I am going to keep. It's as if my friend recorded it and gave me a copy. Legal? I don't know, but that's how I justify it for myself. End of moral debate.

Through my many years of usenet downloading, I've used several different tools. When I started, I had newsgroups I would watch and get headers for. The first show I consistently downloaded was Smallville (in my defense, Smallville had a really good start. Really!). So I would download all the headers for alt.binaries.tv.smallville. Then I would look for the headers that indicated where the episode I wanted was. Next I would download all the parts (standard posting format is split rar files) for the file, verify their integrity (using either .par or .par2 parity files), repair them if necessary (again with parity files) and finally extract the file (tv show) I wanted to obtain.

This method was not too bad, and as a result, I was able to watch my favorite show on the night it aired, even though I couldn't see it when it aired.

As time passed, I found better and better tools for automating this process, and today I feel that I have the best set of tools yet for the job. That is the purpose of this post, though I suppose I was fairly prolific in getting to the point.

This brings us to SABnzbd. I'm sure it stands for something. I have no idea what that is. SABnzbd is a wonderful little program that runs in the background on your computer and can watch for what you want, download it, verify it's integrity, extract it and delete all but what you really want. And this is all in the background!

Speaking of background, I need to explain what an NZB file is. As I began looking for more obscure shows (SciFi's Taken comes to mind) I had to start looking in bigger and bigger groups. There is a group called alt.binaries.multimedia that most often contains well over 2,000,000 headers. If you want to find something in that group, you download all 2,000,000+ headers, then look through them for what you want. Well this is a rediculously long process. So someone came up with the idea of NZB files. An NZB file is simply an xml file that contains "pointers" to usenet postings. So if a new episode of The Office gets posted somewhere on usenet, rather than search everywhere for it, I might obtain an NZB file pointing to all I need to get that episode. So that's what an NZB file is. I'll explain later how to get them.

Back to SABnzbd. You might notice the NZB there in the middle of the name. This program only works with these files. You can't download headers with it.

Now, with all that said, it's finally time to set it up. First you'll need to download it from http://sourceforge.net/projects/sabnzbd/. When it's extracted, you'll have a series of folders along with some files. Before running it the first time, open up SABnzbd.ini. It's fairly well documented with comments, so I'd recommend taking a look at all that's there and changing whatever you need to or want to change. Note that in the folder config section it will refer to the current folder as . and use / to specify a subdirectory, rather than \.

Much of what's there can be configured from the web interface, but I prefer having most of it in place before I start the program. Now that that's done, go ahead and double click SABnzbd.exe. Most likely you'll see a DOS box pop up and go away fairly quickly. I thought this was rather strange the first time it happened to me, but it turns out that all the interface is in a local web page. This page can be addressed (if you've left all the defaults in the config file) at http://localhost:8080/sabnzbd.

So that about wraps it up for configuring SABnzbd. I hope to write one or two more blogs this weekend about this. I still need to discuss news servers and nzb file providers. I suppose this is kinda backwards, so I hope you'll forgive me!

Yet another blog...

Any of you that know me know that I also maintain a blog at http://blog.dunninteractive.com. Why, then, am I starting another one?

Well, this my intention with this blog is to write down some of the more nerdy things I've learned, discovered or created. My other blog is more for family and friends to get a glimpse into my every day life.

Some of the things I hope to discuss here include practical applications of vbscript files (yes, the .vbs files that were so famous for being viruses several years ago), automating and using your own online "TiVo" setup, .Net programming, HTML and friends, MS Office and so on.

I would love to see this turn into a place that is very useful to those that are just a bit less nerdy than I am, and to those who are more nerdy, but don't know some stuff that I do.

Here's hoping I'm more faithful in writing this one than I am in writing my other one!

Bob