Crosscut Status

New Features: Favorites, User Feeds

I just released a couple of new features which support the goal of increasing our readers' ability to track and share their news sources.

Favorites:
The first new feature, favorites, gives registered users a way to save articles, blog posts, photos, or clicker items for later access. Your favorites are displayed on your profile page and the stream of everyone's favorites can be seen on the recent favorites page.

Though that is handy for Crosscut content, one of our core beliefs is that the web is wider than any one site so on the favorites help page there is a bookmarklet (with instructions) which saves any web page as a Crosscut favorite. So, use the "add to favorite" links on Crosscut to save local content or use the bookmarklet to save news from around the web.

User feeds:
As we add new ways for engaged readers to use the news engine underlying Crosscut, we want to encourage that information to make its way out to the rest of the web. As of today, every Crosscut user has a web feed for their actions. This feed includes all comments and favorites, as well as blog posts and articles for authors.

As examples, here are a few of our authors' feeds:
Lisa Albers: http://crosscut.com/account/lisa_albers/feed/
Knute Berger: http://crosscut.com/account/knute_berger/feed/
Ted Van Dyk: http://crosscut.com/account/tedvandyk/feed/

As always, let me know if you have ideas or suggestions about how we can support you on Crosscut.

Site briefly inaccessible

This morning we had a node failure in the web app tier which resulted in several minutes of "server not accessible" messages for our readers. I've switched in a replacement node and now I am performing an autopsy on the old node.

Current status: all systems go

The Switch

This morning Chuck Taylor and I are going to be bringing up a new Yesler server on EC2, pouring all of the data from the current Crosscut.com over into it, and then switching DNS over so that everyone will be pointed to the new server instead of the old one.

There won't be a lot of visible change, but the entire foundation of the site will be replaced, including the editorial tools, the ad server, and the engine which renders the HTML to your browser.

This post will be updated as things progress, so if things go awry check here for the details.

8:16AM - Taking the final snapshot of the database and files from the old servers and pouring them into Yesler.
8:40AM - Pour complete and checked. Chuck is creating a front page layout and I'm importing ads.
10:24AM - All data is in the new system, the front page is ready, the ads are loaded, and now we're just doing a final check before switching DNS.
10:50AM - DNS has been switched and the old server will automatically redirect you to the new IP#. So, until your DNS records update you'll see a site URL like so: http://75.101.144.159/. After DNS has fully propagated you'll just see the same old "crosscut.com". It can take some remote DNS services as many as 24 hours to get the updates, but almost everyone in the Pacific NW should be updated within the next few hours.
11:28AM - At this point we're just watching the logs and server statistics to make sure that everything is running smoothly. We're seeing a lot of search crawlers going through our new URL structures, which is exciting (well, to nerds anyway).

At this point I'm going to call the switch complete and any new status updates will have their own post on this blog.

Crosscut.com is dead. Long live Crosscut.com!

Introducing Yesler: A Brief Technical Overview

Though we're preparing to completely change out the Crosscut.com server software and hardware, I'd like to take a minute to describe the new system, which we call Yesler.

The reasons to switch to a new platform are many: increase capacity, increase development speed, reduce editorial workload, enable cheap experimentation, and boost developer happiness. All of these reasons support our central goal of better serving our readers and the wider web. As newspapers nationwide are cutting staff and losing revenue, it is more important than ever that online news startups like Crosscut stay lean, nimble and innovative.

For hosting, we're moving off of a Spry VPS and onto the Amazon cloud. I've had a great experience with Amazon's EC2 and S3 services in previous projects and since that time the AWS team has released static IP#s and NAS to make it the clear choice for dynamic service like Crosscut, especially as we move into media types other than text and still imagery. It also helps that the Amazon Web Services team is within yelling distance of the Crosscut offices and holds office hours at StartPad.

Each of our servers in the cloud is running a solid, field tested combination of open source technologies which are called LAMP (Linux, Apache, MySQL, and Python) in the web development world. Large sites like Wikipedia, Yahoo, and Facebook run on LAMP so it's proven to be quite reliable.

Our custom software is based on the Django web framework, which was originally created for use in a newsroom and so has been well suited for our needs. The basic platform now supported by the Django Foundation and has a very active developer community which just finalized Version 1.0. The timing is perfect for us because though the Django has been in production use for years it's seen a lot of polish for this first big release. Django is written in Python, which is a clean, modern programming language in common use on the web.

Fronting the Django services for dynamic content is the Apache web server/Memcached duo which handles load balancing, static image serving, and other, more esoteric methods of quickening our users' browsing experience.

Nestled between Django and Apache is our code, Yesler, which defines the model, logic, and display for an elegant and nimble publishing system which will be the foundation for future technical work. In the next few days we'll switch over to using Yesler on the main servers of Crosscut, and then in the following weeks we'll roll out new features and improvements using the new capabilities which Yesler provides.

Introducing Sawmill, Crosscut's beta-test site

We've launched a parallel Web site called Sawmill where we will test technological improvements and experiment with design, all with your feedback. You can find it at sawmill.crosscut.com.

This parallel universe is running on all new technology. Right now it looks like the regular Crosscut (though it's not necessarily up to the minute, news-wise). But in the future you'll see changes and improvements on this test site before we move them to the regular Crosscut.com. First step is to test a new framework and a new platform, the Amazon cloud. Next week, we hope to have the main site running on this new framework, too.

Some background:

The regular site is now running on a content management system we've nicknamed Saratron, which was developed and customized for us by Crosscut's launch Web developer, Shawn Sutherland. It's based on free, open-source technologies — the PHP programming language, the MySQL database, and the Linux operating system. It's running on two virtual servers we rent. It has served us well for almost two years, but we're ready to rapidly expand.

The new system is also based on free, open-source technologies — the Python programming language, the Django Web framework, MySQL, and Linux. The modular foundation of Django makes it easier to modify functionality and appearance. Django is an increasingly popular technology in use by news Web sites including The Washington Post.

I'll let CTO Trevor Smith expound on the technical details, but in language we lay people can understand, we've essentially put a new engine in an older car. It's a behind-the-scenes switch that will enable us to make visible, rapid improvements as we move forward.

After a shakedown of the new technology on Sawmill, we'll switch the present Crosscut site to the cloud, too, possibly next week. Then we'll test all improvements and changes on Sawmill going forward, right before you very eyes. Sawmill won't be the most reliable place to get Crosscut content — it's our sandbox, might not always be running, and might not have the latest articles — but if you want to help us develop new tools for journalism, we invite you to drop in on Sawmill. And, of course, we welcome your feedback.

Crosscut.init()

Welcome to the status and technical discussion blog for Crosscut, the online news source for Seattle and the Northwest. Here you will find information about what the technical team is doing behind the scenes, status updates on visible changes to the site, and information during maintenance or unscheduled downtime.