Adapted from an earlier post on paulgillin.com.

All of a sudden, “curation” is one of the hottest words in the Web 2.0 world. That’s because it’s an idea that addresses a problem humans have never confronted before: too much information. In the process, it’s creating some compelling new ways to derive value from content.

Amount of data published in 2010 depicted as iPads stacked on the playing field of Wembley Stadium

Content curation is about filtering the stuff that people really need from out of all the noise around it. In the same way that museum curators choose which items from a collection to put on display, content curators select and publish information that’s of interest to a particular audience.

This function is becoming more and more critical as the volume of information on the Internet explodes. It’s projected that the amount of digital information that will be created in 2010 could fill 75 billion 16 GB Apple iPads (fun infographic here). Yet, as influencer relations expert Katie Paine points out, 90% of it is crap. As more and more crappy content pervades the Internet, the value of curation should grow.

The problem is that curation is labor-intensive. Someone has to sift through all that source information to decide what to keep and what to throw away, and human decision-making isn’t easy to automate. Keyword filtering has all kinds of shortcomings and RSS feeds, while useful in many contexts, are basically headline services.

We’ve recently been working with a startup that’s developed an innovative technology that vastly improves the speed and quality of content curation. CIThread has spent the last 15 months building an inference engine that uses artificial intelligence principles to give curators a kind of intelligent assistant. The company is attacking the labor problem by making curators (or you can call them “editors”) more productive rather than trying to replace them.

Full disclosure: We have received a small equity stake and a referral incentive from CIThread as compensation for our advice. Other than that, the pay has amounted to a couple of free lunches. We make no money unless this idea is as good as we think it is.

CIThread (the name stands for “Collective Intelligence Threading” and yeah, they know they have to change it) essentially learns from choices that an editor or curator makes and applies that learning to delivering better source material.

The curator starts by presenting the engine with a basic set of keywords. CIThread scours the Web for relevant content, much like a search engine does. Then the curator combs through the results to make decisions about what to publish, what to promote and what to throw away.

As those decisions are made, the engine analyzes the content to identify patterns. It then applies that insight to delivering a better quality of source content. In effect, it learns to “think” like the curator. CIThread can be linked to popular content management systems to make it possible to automatically publish content to a website and even syndicate to Twitter and Facebook without leaving the curation dashboard.

That’s what happens on the back end, but there’s intelligence on the audience side, too. CIThread can also tie in to Web analytics engines to fold audience behavior into its decision-making. For example, the curator can set the engine to overweight content that generates a lot of views or clicks into its decisions and to deliver more source material just like it to the curator. All of these factors can be controlled via a dashboard.

Shhhhh!

CIThread is still pretty early stage. It has some  test customers, but none can yet be identified. Here’s a general description of what one of them is doing, though.

This company owns a portfolio of properties throughout the US and uses localized websites as both a marketing and customer service tool. Each site contains frequently updated news about the region, but the portfolio is administered centrally for cost and quality reasons.

Using CIThread, individual editors can now maintain literally dozens of these websites at once. The more the engine learns about their preferences, the more sites they can support. That’s one of the coolest features of inference engines: they get smarter the more they’re used.

The technical brain behind CIThread is Mike Matchett, an MIT-educated developer with a background in computational linguistics and machine learning. The CEO is Tom Riddle (no relation to Lord Voldemort), a serial entrepreneur with a background in data communications, storage and enterprise software.

The two founders started out targeting professional publishers, and that’s a pretty safe bet. But we think the opportunity is much bigger. Nearly any company or organization today can develop unique value for its constituents by delivering curated content. Using tools like CIThread, they can do it more quickly and productively than by training humans. They can also capture the knowledge of their editors so that experience doesn’t walk out the door due to resignation or layoff.

If you want to hear more, e-mail curious@cithread.com or visit the website.

Since we first wrote this, a couple of other tools have come to our attention that attack this same curation task. Curata has an engine that scours the Web for content and auto-posts it to blogs and social network sites. The company has a shipping product and real customers. Curata is positioning its service as more of a lead generation tool than an editorial productivity aid. See the two-minute video below.

CurationStation looks a lot like Curata. It’s a low-cost service that filters content based upon keywords and publishes automatically to multiple destinations. The $2.99 signup incentive is attractive, but set a reminder on your calendar, because it turns into a $279 monthly fee after the first 30 days. If anyone has experience with either of these products, or is aware of other solutions, please comment.

‘When my students come back to visit, they carry the exhaustion of a person who’s been working for a decade, not a couple of years,’ says Duy Linh Tu of the Columbia University Graduate School of Journalism. ‘I worry about burnout.'”

He’s talking about the pressure of the new online newsroom. It used to be that daily deadlines were considered intense, but in today’s hyper-competitive environment, many reporters are expected to file several times a day. “Young journalists who once dreamed of trotting the globe in pursuit of a story are instead shackled to their computers,” writes The New York Times.

Some staffers at The Politico start their work days before dawn. Editors walk the aisles asking who’s broken a scoop that day, and reporters may wake up to find an e-mail sent at 5 a.m. asking why they were beaten on a story. The pressure is on to file something – anything – that a reader hasn’t seen before.

The Politico knows that the new competitive environment doesn’t tolerate delay.  “Everybody in the audience is his or her own editor based on where they want to move their mouse or their finger on the iPad,” says Politico’s editor in chief, John F. Harris. Perhaps it’s not surprising that the Politico has lost about 20% of its news staff this year. But where are they gonna go? The website’s results-fueled journalism is becoming the norm.

The Christian Science Monitor sends a daily e-mail telling its reporters which stories had the highest view count the previous day. Gawker Media displays the top 10 most viewed stories, along with reporters’ bylines, on a monitor in its offices. Some news outlets even compensate their staff based on traffic. And then there are search-driven word factories like Associated Content and Demand Media that assign stories based upon search popularity and pay by the page view.  Search marketing expert Mike Moran calls these outfits “content chop shops” that cheapen quality by elevating search visibility. But you can’t argue with success. Yahoo bought Associated Content for $90 million and Demand Media is reportedly hoping to be the first $1 billion IPO in nearly a decade.

The good news is that some media properties are hot again. The bad news is that they’re places where few people can apparently stand to work (See also Search-Driven News).

Miscellany

A.H. Belo reported a narrower second-quarter loss, but what stole the headlines on the earnings call was the rising importance of circulation revenue, which now accounts for nearly 30% of the company’s sales. In fact, circulation revenue was up 66% in the quarter, largely due to price increases at the Dallas Morning News. Executives crowed that the Dallas paper is now the third most-expensive in the country, behind only The New York Times and the Boston Globe. The prices are a function of “the quantity and quality of what we put in the newspaper,” said Belo CEO Robert Decherd. They’re also a function of what the dwindling ranks of elderly print readers are willing to pay. Belo also reported that it has $60 million in the bank and is increasing is earnings before interest, depreciation, taxes and amortization (EBITDA), even though revenues continue to decline. The company’s strategy appears to reflect that of many of its competitors: milk the print cow while you can, cut costs and hope to get traction in new markets. That’ll work for a little while longer.

The Democrat-controlled Federal Communications Commission surprised everyone this week by choosing to defend rules adopted under the George W. Bush administration that loosed restrictions on media cross-ownership. In a filing with the US Appeals Court, the FCC supported the 2007 ruling by a Republican-dominated FCC that made it easier for media companies to own multiple media outlets in the same marketing. The agency had been widely expected to take the first chance it had to reverse that decision in the name of restoring more competition to the market. FCC Chairman Julius Genachowski issued a statement that we read three or four times and still couldn’t understand. Perhaps the FCC has decided that owning multiple local media properties doesn’t matter for much when all are tanking at about the same speed. Fellow commissioner Michael Copps attacked the FCC’s decision and vowed to move the strengthening of cross-ownership rules “to the commission’s front burner where it deserves to be.”

And Finally

Pulitzer-winning editorial cartoonist Steve Breen decided to satirize the Gulf oil spill by drawing some of his cartoons using oil instead of ink. The process turned out to be a lot more involved than you might think. Breen flew from San Diego to New Orleans on his own dime and then drove to Pensacola, FL to find tar balls of sufficient viscosity to work with. He then diluted the tar with various solvents until he hit upon gasoline as the perfect element to soften the tar enough to work with. The result is a striking sepia tone, with which Breen has skewered not only BP but also America’s obsession with oil. Here’s Breen’s page on the San Diego Union-Tribune site. Click on the image at right to see a gallery.

The Times of London set up a paywall on July 2 and has lost 66%, 84%, 90% or 93% of its online traffic as a result, according to the rival Guardian. The Guardian apparently can’t figure out which figure to believe, so it lays them all out in a tedious and self-indulgent exercise that is probably of interest only to management at the Guardian.

“New paywall costs the Times 66% of its internet readership” says the July 18 headline, which then helpfully points out in the subhead that that means that 33% of the audience is still there. Two days later, though, the very same Guardian trumpets, “Times loses almost 90% of online readership,” a decline it characterizes as “massive.” We marvel at what a difference two days can make.

The Guardian then presents a convoluted analysis of comparative data that suggests that the Times’ website traffic has fallen anywhere from 84% or 93% since it began charging £2 a week for online access. The paper also presents various scenarios for calculating the Times’ share of overall traffic to UK newspaper sites and debates what the impact on the paper’s bottom line will be.

The nut graph, however, makes it clear that this is a non-story: “The figures are…unlikely to surprise some executives at the Times: the Sunday Times‘s editor, John Witherow, predicted in May that ‘perhaps more than 90%’ of pre-registration readers were likely to be lost once the registration-only service was implemented.”

So what is the story here? The Times got exactly what it was expecting. Its financial people have presumably run the numbers and decided that they’re ready to take the traffic hit. In fact, the Guardian even quotes Rupert Murdoch saying that paywalls could generate “significant revenues” for his newspapers.

Let’s give the Times credit for setting up a real paywall. Even Google can’t penetrate this sucker. Clicking through to any section or story from the home page is pointless without a credit card in hand. Murdoch is putting his money where his mouth is. He has pledged to take all his newspapers to a paid-access model, and the Times’ experiment is bold, regardless of the outcome. Unlike subscribers to the Wall Street Journal or the Financial Times, the readers of the London Times have no compelling financial interest in the content. In the crowded UK news market, they also have plenty of alternatives from which to choose. If the Times can make its paywall work, it will give a lift to the rest of this beleaguered industry. Although probably not to the Guardian.

In case you missed it, the perpetually poverty-stricken Journal Register Co. is doing some pretty gutsy stuff. The company, which was delisted from the NASDAQ New York Stock Exchange two years ago, has a new CEO who’s interested in reinventing publishing. John Paton (right) has a blog and a Twitter Account. He also has the admiration of Jeff Jarvis, who doesn’t confer praise lightly.

What got Jarvis so excited was a July 4 experiment in which the company’s 18 dailies published using nothing but free, web-based tools. They called this the Ben Franklin Project in recognition of both the country’s birthday and Journal Register’s liberation from ancient proprietary production systems.

More importantly, the company changed the way it reported the news for that day. Readers were actively involved at the front of the process in directing the reporting staff and looking virtually over reporter’s shoulders as stories were prepared. “The Ben Franklin Project is the beginning of a new era of an open and transparent newsgathering process,” wrote Paton on his blog. This is a company worth watching again.

MediaShift has an excerpt from journalism educator Alfred Hermida about rethinking the role of the journalist in the participatory age. While Hermida doesn’t break a lot of new ground, he crystallizes some concepts we’ve been talking about here for some time, namely that the evolving role of the journalist is as aggregator and authenticator rather than original reporter. Quoting Tom Rosenstiel, Hermida describes the still-important role of the journalist as “a sense-maker to derive meaning, a navigator to help orient audiences and a community leader to engage audiences.”

He also quotes from an article by BBC World Service director Peter Horrocks that calls for an end to “Fortress journalism.” Horrocks writes, “In the fortress world, the consumption of journalism was through clearly defined products and platforms… but in the blended world of Internet journalism all those products are available within a single platform and mental space… the reader may never be aware from which fortress the information has come.”

In the world Horrocks describes, the audience pulls together its own newspaper, woven from bits and pieces assembled from various online sources. The consequence of this is that media organizations can’t afford to reinvent the wheel anymore. Each needs to focus on what it does best and pool efforts rather than duplicate them. So maybe 90 of those 100 journalists who currently attend a Presidential press conference can spend their time out in the field assessing reaction and gathering analysis rather than listening to the same thing. What a concept.

Miscellany

Advertiser optimism continues to grow. Advertiser Perceptions Inc. (API) reports that 32% of ad executives now expect to increase their ad spending over the next 12-months. That’s the largest percentage increase since API began asking ad execs about their intentions in 2007. A year ago, the figure was -5%. The 1,412 ad executives who were surveyed continue to be pessimistic about magazine and national newspaper advertising, with intentions to increase spending down 10% and 32% respectively. But even those sentiments are greatly improved over the -26%/-46% plans of a year ago. The biggest winners are digital and mobile media, with more than 60% of ad executives planning to increase spending there.

Give Tribune Co. credit for trying to diversify its revenue stream. The bankrupt company is dedicating 10 people to a new consulting business that will sell knowledge of social media and Internet advertising to small and mid-sized businesses. The new venture is called 435 Digital Services, a nod to Tribune Co.’s headquarter address at 435 N. Michigan Ave.

The Denver Post is going after a local political site, saying that Colorado Pols is stealing its copyrighted material. The political site, which generates marginal revenue, allegedly lifted between three and eight paragraphs of news articles from the Post and other publications. Colorado Pols says it doesn’t need the Post. “There’s thousands of other outlets out there,” says founder Jason Bane. Post owner Media News is one of those media companies that wants to raise the perceived value of its content. The company has confirmed that it will begin testing online pay models this summer at its newspapers in Chico, Calif., and York, Pa.

Speaking of pay walls, Time magazine now has one. Secure in its role as the only newsweekly left standing, the venerable but mostly irrelevant magazine is requiring readers who want to read online versions of its print article to subscribe to either the print or the iPad edition. They can then see the same stuff that’s in the magazine on a screen. Online-only content will continue to be free.

Circ Labs, the University of Missouri-backed startup that is developing a tool that learns from a user’s online behavior and delivers recommendations for content, has launched a prototype service prior to general release. The prototype installs a Firefox add-in that enables the browser to recommend an article and to read similar articles suggested by the algorithm. Users can share content with each other and be notified of new content as it becomes available.

To test, go to gocirculate.com and create an account. The confirmation page contains a link to the toolbar software. You can then browse and add pages to the knowledge base. We were able to install the menu bar, but couldn’t log onto the site for some reason, and Circ Labs provides no means to recover a password. We guess that’s why they’re calling this a test.

Buried in a lightweight study of the Internet habits of young women is this nugget: “Nearly half — 48% — of all respondents now claim to get more news through Facebook than from traditional news outlets.”  This number comes from Lightspeed Research and Oxygen Media, which surveyed the habits of 1,504 U.S. adults who use social media. The researchers also claim that 39% of women between the ages of 18 and 34 now describe themselves as Facebook addicts, and that a third of young women check Facebook before going to the bathroom in the morning. We supposed one needs one’s priorities.

Variety’s website has adopted DailyMe’s behavioral tracking and recommendation technology called Newstogram.  Newstogram generates data on user’s interests to deliver visitors content, advertisements and e-commerce opportunities tailored specifically to them, based on their specific interests and behavior. DailyMe started life as a customized news service for consumers but has morphed into a customization engine that publishers can serve up to their visitors. Readers get filtered news and publishers get better insight into what motivates readers.

And Finally…

Roy Rivenburg is still at it. The jokester who dreamed up Not the LA Times two years ago continues to tweak the nose of the West Coast’s most self-important newspaper. A recent story has Times editors arguing over whether it’s better to start articles with the time or the weather. The inspiration is this page of formulaic opening sentences extracted from the real newspaper. “If I don’t find out the time of day in the first sentence, I stop reading,” says one subscriber.