Sorry we’ve been so quiet lately. We didn’t know when we be able to diagnose let alone fix things, and then, when we started thinking about shutting down, we wanted to be sure before we said anything.

We lost the hunger

The main problem we’ve been facing is that none of us were passionate about Newshutch anymore. Yes, the move to Joyent has been problematic¹, but we’re glad it was because it forced us to face facts.

I sincerely believe that scrappy upstarts can upset and displace entrenched competitors. If we’ve learned nothing else from the last ten years it’s that creative destruction is alive and well.²

However, the key ingredient for the underdog is passion. We started working on Newshutch in early 2006 because we were scratching an itch. Like email clients, RSS clients make more sense as web applications. At the time the only big web based reader was Bloglines, and it remains unbelievably hideous to this day. These days however, to meaningfully compete with Google Reader, AND Bloglines, AND who knows how many other RSS readers, we’d have to do nothing else but work on Newshutch, and frankly, it wouldn’t be worth it. The problem of developing a usable web based RSS reader has been reasonably solved, so we’d rather pour ourselves into new projects where we could make a bigger impact.

No regrets

We thought it would be emotionally hard to shut down what we’d worked so hard on, but instead it’s liberating. Now we’re free to find new things to work on that will be even better than Newshutch.

One of the things that bothers me about Newshutch and other readers is that they?re great at hitting you with tons of information, but I’d rather be hit with the right information. Better still, I’d rather have tools to help me create things. Whatever we work on next will place a higher value on creation than consumption.

Another reason shutting down feels right is that it doesn’t feel like failure. We set out to make a newsreader that didn’t suck and we succeeded. Even better, we learned more about technology, design, project management, customer service, and business in one year than we would have in ten years at a normal job. At a typical job you’ve got an army of system administrators, marketers, and project managers, to back you up and nobody takes the heat when things go wrong. When you’re responsible for a live application from top to bottom you get smarter faster.

Even if you know you’ll fail (and you should always assume that you will) you should still try to build a product for yourself. Looking back it was almost hubris to attempt to build a newsreader (they’re very resource intensive), in Rails (Doug was a Java guy), with just two people, but we’re glad we did. We may have failed but what we’ve learned more than offsets any sting of failure.

What’s next?

Newshutch will shut down on November 10th. Before then, we’ll begin shutting down pieces of functionality until only OPML feed list export and account management remain. That should be enough time for you to migrate to other feed readers.³ The Newshutch blog will remain up indefinitely and we hope that you’ll stay subscribed to it because we’ve got a few more posts coming on the joys and pitfalls of building a web service. Also, we’ll announce future projects on the Newshutch blog (and at nathanbowers.com). Micah has already launched a new personal productivity service called Making the Chain, so be sure to check it out.

Thanks

Thanks to everyone who complained about, contributed to, commented on, and used Newshutch. That’s another hidden benefit to starting your own web service, you hear from all sorts of people, and usually they’re pretty cool. We’d still love to hear from you, so if any of you have ideas about what we should do with our codebase, or what you’d like to see us work on next, fire away in the comments.

1. Joyent is a fine company with good service, but they weren’t right for us. This issue deserves its own post, but the bottom line is that Newshutch was too resource intensive for the three Accelerator slices we had, but to get enough Accelerators we’d end up paying more than we did for dedicated hosting. We weren’t in the Joyent sweet spot. If you start small with Joyent and scale up organically, that works fine. If you’re Twitter and have huge scaling needs and budget to match, also fine. If you are already up and running somwhere else, think long and hard before pulling up stakes to move to a new environment. Risking a move like that was a horrible decision on our part, but on the other hand I’m glad we did it because it forced us to make a decision about our future.
2. See also: The entertainment industry vs. Apple, Detroit vs. Japan, IBM vs. Microsoft, Yahoo vs. Google, The Pony Express vs. the telegraph, etc…
3. I still don’t really like any of the other readers out there. Google Reader seems to have the best functionality, but it’s just so damn ugly! Content viewed in a feed reader should be even easier to read than the feed publishers’ website, but every other web based feed reader botches design and typography so badly that it’s invariably worse. If I was Google I’d drive a cargo container of cash to 37signals’ or Dan Cederholm’s offices and say “Congrats, you’re the new EVP of UI at Google.” Since that probably won’t happen, your best bet is probably to use Google Reader with Jon Hicks’ theme.

We added some things and fixed some things:

• Added “test driving” for new users. Now you don’t have to sign up to start using Newshutch, though you will have to sign up to make sure your account stays alive.
• Fixed the annoying Safari scrolling slowness. It turns out Safari has a hard time when there are many items with CSS semi transparency. The “mark read” buttons on each post were the problem.
• Moved the instructions for hotkeys and adding feeds to a new header link, “Tips and Hotkeys”.
• There were other fixes and backend updates too numerous to mention here.
• We will now be accepting advertising, so we’re pitching it on the “welcome” page. We promise to keep any ads we accept neat and dignified. If anyone has any comments or suggestions for how we handle advertising, let us know.

Update: I forgot to mention that under the “manage” tab you can now rename feeds and see if they are stale.

New stuff:

• Added an option to automatically mark items as read. This is a cookie based option, so you’ll need to set it on each computer/browser you use.
• Added a “mark read” link to the top of unread entries (many users requested this).
• Moved the option to view flagged entries to the main tab area (many, many users requested this).
• Moved the “mark everything read” button outside of the left area scroll when you have independent scrolling on.

Bug fixes:

• IE was having trouble with AJAX refreshes of new, unread items because of a conflict with the Javascript for visual effects that happens when items update. This is fixed.
• Fixed parser issues that cause sometimes very bad repeats of “read” items. This might mean that you’ll see one last set of duplicates of items you’ve already read as the parser catches up with the our new way of seeing if feed items are new.
• Fixed issue where mixed case feeds were being forced to lower case, making them impossible to add.
• Fixed parser issue that was causing feeds not to be updated. Basically the parser was choking on several blog feeds from Weblogs, Inc.
• Made several parser, backend, and other fixes.

Safari Users

Be sure to clear your cache and reload (Safari can be overly cache happy).

We have a lingering and annoying bug in Safari. When Safari receives content via AJAX, it gets the encoding wrong and things like proper quotation marks get mangled. We thought we could fix it by using the browser filters Rails plugin, but it didn’t work. This issue and other possible fixes were mentioned in the forums. Anyone know what the deal is with Safari?

Thanks to everyone who wrote in with suggestions and bug reports!

New features

• Flagging.
• Scrolling panes work better.
• Now we prevent adding duplicate feeds.

Fixes

• Feeds in any language and encoding should work. Old entries parsed before this update will still have invalid characters though.
• Mixed case URLs don’t mess up when adding feeds.
• Add feed bookmarklet now prompts for login when needed.
• Other bug fixes and parsing fixes.

Newshutch has been not so quick lately, and by not so quick I mean that even simple queries would peg our database CPU at 100%. Well, that unusable mess should be all over now.

The only thing that still lags is when you pull up a feed with a lot of long unread items, but overall things are much better.

New features

• Independently scrolling panes option: Right now this just uses cookies so you’ll need to set this option for each new computer or browser you use. Someday we’ll make this option persist with your login.
• Collapsible categories: Right now these are just static open/close buttons to help you keep your place while you’re reading. If you close the window everything will open again. Like scrolling panes, we’ll eventually make these persistent.
• Better Printing: Feeds printed decently before, but I made some changes. I also forgot to ever mention it to users. It’s too bad that people expect web pages to print badly unless there is an explicit “printer friendly” page.
• Design changes: I moved some things around and added some things. I’ll leave it as a “Where’s Waldo” exercise for you

Enhancements/Fixes

• SPEED! I’m sitting here watching top report the DB CPU at 97% idle. Sweet. Is it just me or is watching a computer “breathe” via top hypnotic?
  Doug worked hard to get those queries (and a ton of other bugs) under control; somebody buy that man a beer!
• Mark everything read button is back (finally).
• Fixed favicon downloads. This was broken for a while so when new feeds were added they would be missing favicons. No need to re-add feeds though, if your favicons are available, we’ll get them for you.
• Fixed some parsing bugs. The annoying bug that prevented some feeds from showing their full content is fixed. We also fixed some other bugs related to OPML import/export.
• Better Opera support. It’s not perfect, but at least it renders now.
• Fixed Mac Firefox scrolling/ghosting issue. For anyone who cares, Mac Firefox didn’t like that I had set an overflow CSS property on the feed entries.
• There were other numerous bug fixes and tweaks too mind numbing to discuss here.

Things that aren’t fully baked

• Non latin characters in non-utf-8 feeds still don’t work.
• The keep unread checkbox doesn’t stay checked (woops).
• Flagging didn’t quite make it in this round.
• Various bugs related to parsing or importing troublesome feeds.

As always, thanks to everyone for their support and comments. Keep telling us what we can do better!

Finally! Thanks to everyone for being patient. Now we can stop putting out fires and start making things better. We should have never tried to get away with running our database and web server on a single server, and we learned that lesson the hard way. Now we’re running the web server off the older server and the database has the new beefy dual processor box.

Everyone should be able to see a big difference in responsiveness. Let me know how it works for you guys.

Hi all,
Sorry about all of the ups and downs today but it looks like we’re good for the night.
The most current update addresses the serious case of the slowness you’ve most certainly noticed. We also have a new fancy-grade beefy database server on the way that will also make a major difference.

Most of the changes in this release are on the back-end but here are some UI and functionality changes/bug fixes we sneaked in:

• OPML import handles categories better and no longer barfs Rails stack traces
• Feed list animation and rendering bugs have been fixed
• Email address are actually parsed correctly now in the new user page
• Blog and Forum links now the top of the page

Because of the way we’re caching the feed list, we currently cannot change the unread counts based on what is selected in the “Show entries from” drop-down. However, when you select a feed, the entries you see are limited by the selected value the same as before. Getting the unread counts to change again is near the top of the list of things we’re addressing in the next rollout. The main page loads around a billion times faster now so the trade-off was an easy choice.
The “mark everything as read” link has also been removed because of database killing concerns, but will also be back in a future update.

Please let us know if you have any problems!

Newshutch has been public for 6 days. We’ve gotten lots of high quality feedback. Many thanks to everyone who’s commented, told their friends, and used the application. When you build an application you develop blind spots; that’s why there’s no substitute for feedback from real users.

Officialy we just rolled out version 1.0.2, not that any of you should care about the version number. The best thing about web applications is that you don’t have to burn a million CDs, pack them into boxes with manuals, ship them to Office Depot, and pray that everyone buys upgrades. Instead, the next time you go to Newshutch it’s a little better and you didn’t have to install anything or (horror!) submit to a Windows Update.

New features in this version

• Browser button to add feeds: This is fantastic. Now you don’t have to go through the pain of copying and pasting web addresses into Newshutch. Now you just click “Add to Newshutch” in your browser toolbar and that’s all.
  
  [Update] I neglected to mention that for the browser bookmarklet to work, you need to stay logged in to Newshutch via the “Remember Me” checkbox. We’ll fix this in an upcoming revision.
  
• Keep unread: While this remains checked, the entry will remain “unread”.
  
• Filter entries based on freshness: I use this when I want to dig through “read” items, but I don’t want to see all the older entries.
  

Enhancements

• When showing both read and unread entries, the unread count appears instead of the number of all entries in a feed.
  
• There are also a number of back end enhancements that make the application more snappy and reliable.