Digital Libraries: Googles and Our Discontents

Readers here will find a very nice summary of an April 4, 2009, "Library 2.0" conference at Yale University. There my New Haven Review colleague and Yale instructor, Donald Brown nicely summarizes points made by the conference attendees. However, it is not surprising to see some of the issues glossed over by those who spoke (or perhaps simply not recorded by Don).

Don also offers his own points regarding the threats posed by Google and its ilk to libraries, per se, all worth the reading. However, I thought a few points ought also be made regarding the seemingly onerous burden placed upon us by Library 2.0.

Let me take on some of the issues Don mentions in the spirit of Plato...

Don: "Michael Zimmer of Univ of Wisconsin-Milwaukee…offered a few caveats to the collective zeitgeist of online über alles with the notion, picked up from Neil Postman, of technology as always offering a Faustian bargain….Zimmer had reason to wonder if ‘Library 2.0’—the library as modeled on Google, essentially—will continue to provide a ‘safe harbor for anonymous inquiry.’ Not simply ‘who owns the content’ of what we post—but who owns the documentation, who gets to data-mine, and so forth."

Bennett: "Agreed, this problem of privacy of our inquiries is a serious problem. On the other hand, "datamining" is, in a very real sense, what every scholar already does. The question is not the activity but the nature of the data and the purposes of the mining. I’m actually grateful when Amazon recommends books to me of related interest based on the buying habits of previous customers. In a pre-Internet world, there was no way to have gotten that type of information."

Don: "Frank Pasquale, Visiting Professor at Yale Law School, spoke of the possible consequences of putting all our searches for information in the hands of ‘proprietary black box algorithms subject to manipulation.’ Wikipedia is always the first or second entry in any Google search. The first ten are apparently all anyone looks at. Everything that gets buried by the algorithm is as good as not there. This is not how research is conducted."

Bennett: "Actually, in some ways, this is how research is conducted. Before there was Google, there were—and still are—all sorts of 'shortcuts' in the research process: encyclopedias, reference guides, anthologies, textbooks, library catalogs (once card, now digital). I'm not certain what 'subject to manipulation' refers to here since there is always some form of sorting and re-sorting of data that occurs in secondary tools. Google—indeed, any search engine, is just one of these. The fault lies less in Google than in ourselves and our research habits."


Don: "[T]he notion of open access to all information, via the internet, of complete ‘transparency’ of provider and user, was more or less the mantra of the day. But what the Faustian bargain came to seem finally was not with the technology itself, but with giants such as Google and Amazon as the Big Brothers playing Mephistopheles, offering us the interconnected, easy access world of our dreams, but a world where we sacrifice something of our own intellectual curiosity, restlessness, and desire to see outside or beyond that black box algorithm that makes things so easily manageable for us."

Bennett: "Kind of covered that already, Brutus."

Don: " Wolpert pointed out that what made the MIT professors move for Open Access was their realization that, in the world of electronic text, libraries only ‘lease’ access to online work, rather than owning it like all those printed copies they store in perpetuity. If something happens to the provider or to the lease, all that material is no longer available. And now the publishing world seems poised to turn over all electronic control of out-of-print materials to Google to broker for us, and to disseminate to us according to its lights."

Bennett: "Wolpert has probably overstated the case. Libraries don’t lease all of their online content. Some of it is in fact purchased through “digital archive” arrangements, where libraries own digital copies of their books in perpetuity. These typically affect collections of historical books (not from Google but other major library vendors) and certain journal collections. The fundamental problem is not lack of ownership; it’s the ability of libraries to host the data on their own. And while I'm not one defend Google, on this front, again, I think the case is overstated since I believe Google turns over digital copies of books from any library to that library. (E.g., Harvard gets digital copies of its books alone; Stanford gets theirs, etc.) The problem is not ownership of that data: it's what happens when the current distributor of it goes bust or ceases offering that service. Now all the libraries, whom hopefully did get digital copies of their books, have to figure out a new way to get all that content up on the Web and strung together."

Options, Embargoes, and Exemptions in Commercial Microfilm Publishing

As I’ve described elsewhere, commercial microfilm products largely comprise content from libraries or print publishers (such as newspapers or periodicals). Just a day ago, I had a phone call with an institutional contact about a proposed microfilm venture. In the course of our conversation, I reviewed several key issues affecting products of this nature. A summary of some of these should illuminate the available options and trade-offs for institutions and publishers about to engage with one another in a microfilm publishing arrangement.

ROFOs and ROFRs

Prior to the mid-1980s, and sometimes even later, micropublishers rarely included digital rights in their microfilm contracts. In the last decade, they no longer overlook this opportunity, especially those with the financial wherewithal to go digital or hopeful of eventually getting there. As a consequence, commercial microfilm publishing contracts commonly include provisions concerning the possibility of a digital edition. The simplest approach to addressing a digital version of a microfilm product is to include a “digital option.” Options come in various forms and follow standard legal practice. My contracts often included a right of first offer (ROFO), which grants the publisher the right to bid first in good faith on a digital project in the event that the content owner opts or a third party offers to digitize the content, whether for free or for fee. A variant is the right of first refusal (ROFR). This tends to be more restrictive because in this arrangement the content owner must give the publisher the right to match or meet any possible offer, especially from a third party. In the current climate, library sources find ROFOs more acceptable because they want the latitude to publish their content for free and not be subject to the restrictions (and obligations) implied in an ROFR.

Cannibalization

In my own experience, library sources have rarely posted serious objections to granting a digital option. After all, the provision is just an option and not a binding agreement. More important to content owners, especially libraries, are restrictions on the digitization of the data. For the publisher, these restrictions have a simple purpose: to prevent the “cannibalization” of the microfilm product by the availability of a digital version. (Cannibalization is the common term for lost sales by dissemination of an alternate edition or version of the same content.) However, many libraries—especially research libraries, which are often the main sources for scholarly microfilm sets—have public and scholarly missions that emphasize expanded access to knowledge and, therefore, their holdings. (I am perhaps too bold here, but this argument, which I have heard frequently seems disingenuous in the extreme. Many of the institutions claiming this free knowledge mandate are private and do not grant on-site free access to their holdings to unaffiliated individuals, so why the rush to give it all away free to the teeming masses?)

The expanded access mission of these institutions generally translates into “open access” (read “free”) digital versions of their collections. Where once upon a time funding agencies favored microfilm preservation, providing grant money for either equipping micrographics departments within libraries or paying vendors under the direction of the grantee, today those same funds have been aggressively financing the provision of digital equipment and services. So here we have a fundamental conflict of interests. The microfilm publisher would, of course, see any open source digital version of the content from the prospective product as anathema. On the other hand, libraries see opportunities to upload digital versions of their holdings —much simplified by provision of a microfilm version of their content from which to work—as critical to their mission to augment access and, equally important, raise the visibility of the collection and its host library.

Embargoes & Exemptions

The result for most microfilm contracts is the arrangement of a limited “embargo” on the digitization of content from the microfilm, especially by the source institution and for free. Few institutions, however, will tolerate a complete embargo. As a consequence, it is not uncommon to include an exemption of up to ten percent of the content for any form of dissemination by the source institution. (Indeed, for publishers, this allowance may favor the collection’s distribution in microfilm by serving up a digital teaser for the more complete collection.) Outside of that exemption, embargoes vary. With the Library of Congress, I negotiated a 36-month embargo on digital conversion and dissemination from the date of publication of the microfilm edition of the content. Columbia University’s Rare Book and Manuscript Library, on the other hand, opted for a yearly addition of ten percent to the first-year exemption, so that by year ten, the entire collection would be available for digitization and dissemination without restriction. Note that embargoes are especially important to the publisher if it decides—as some do today—to create the microfilm collection from a digital imaging process. (Yes, as backwards as this seems, there are a number of good reasons for creating microfilm collections in just this manner.) Since digital images, along with gratis copies of the microfilm, will be supplied to the source institution, the embargo is critical to the publisher. On the other hand, the gratis provision of a complete set of digital images represents a strong temptation to the library.

For the publisher, of course, provision of an exemption does not necessarily result in digitization and dissemination of the content by the content. First it is possible that the publisher may “go digital” with the product before the embargo period finishes. Second, and more likely, the content owner will not have digitized the content by the time the selling arc of the product has ended (usually 3 to 5 years) owing to a lack of funds, interest, staff, or what have you.

Nonetheless, content owners often do not want to abdicate the ability to digitize, should a grant funder or a more generous library director come waltzing through the door. Libraries want these exemptions and, in an ideal world, to limit the embargo as much as is reasonable. In the course of negotiation, content owners can always just stand firm on promoting its open access interests—seeking large exemptions or miniscule embargoes. But I recommend the trading on negotiable portions of the contract. For example, the source institutions might trade a greater exemption or shortened embargo for reduced royalty payments or some other cost-savings concession that sweetens the bottom line for the publisher. This give-and-take will at least provide some form of compensation for potential sacrificed revenues as a result of cannibalization. On the other hand, for content owners that may care little about exemptions or embargoes—they really have no internal plans to digitize the content—they may adopt the opposite tactic. In exchange for a reduced exemption or longer embargo, they prefer a higher royalty, larger advance, or a guaranteed yearly minimum.

As I’ve recommended to content owners with whom I’ve negotiated, publishing contracts are not written in stone. They actually comprise a set of moveable parts that may be added, revised, or discarded, and since each of these parts represent a set of obligations—if you will—they each have opportunity costs that can be traded. In brief, contracts are “barterable,” and should be treated accordingly in that way, especially for commercial microfilm products, which are technically “print-on-demand” products with notoriously long lifespans.