By Lisa Schiff, Technical Lead for CDL Publishing Services

The California Digital Library (CDL) is pleased to announce a new release of its search and display technology, the eXtensible Text Framework (XTF) Version 2.1.   XTF is an open source, highly flexible software application that supports the search, browse and display of heterogeneous digital content.  XTF offers efficient and practical methods for creating customized end-user interfaces for distinct digital content collections.

Highlights from the 2.1 release include:

• Extensive interface improvements, including new search forms, built-in faceted browsing, and new look and feel.
• Increased support for document and information exchange formats.
• XHTML and OAI-PMH output
• NLM article format indexing and output
• Microsoft Word indexing
• Streamlined XSLT stylesheets for simpler deployment and adaptation.
• Updated documentation that has been moved to the XTF project wiki, allowing XTF implementers to share solutions with entire user community.
• "Freeform" Boolean query language, offered as an experimental feature.
• Backward compatibility with existing XTF implementations.

A complete list of changes is available on the XTF Project page on SourceForge, where the distribution (including documentation) can also be downloaded.

Since the first deployment of XTF in 2005, the development strategy has been to build and maintain an indexing and display technology that is not only customizable, but also draws upon tested components already in use by the digital library and search communities - in particular the Lucene text search engine, Java, XML, and XSLT.  By coordinating these pieces in a single platform that can be used to create multiple unique applications, CDL has succeeded in dramatically reducing the investment in infrastructure, staff training and development for new digital content projects.

XTF offers a suite of customizable features that support diverse intellectual access to content.  Interfaces can be designed to support the distinct tools and presentations that are useful and meaningful to specific audiences.  In addition, XTF offers the following core features:

• Easy to deploy: Drops directly in to a Java application server such as Tomcat or Resin; has been tested on Solaris, Mac, Linux, and Windows operating systems.
• Easy to configure: Can create indexes on any XML element or attribute; entire presentation layer is customizable via XSLT.
• Robust: Optimized to perform well on large documents (e.g., a single text that exceeds 10MB of encoded text); scales to perform well on collections of millions of documents; provides full Unicode support.
• Extensible:
• Works well with a variety of authentication systems (e.g., IP address lists, LDAP, Shibboleth).
• Provides an interface for external data lookups to support thesaurus-based term expansion, recommender systems, etc.
• Can power other digital library services (e.g., XTF contains an OAI-PMH data provider that allows others to harvest metadata, and an SRU interface that exposes searches to federated search engines).
• Can be deployed as separate, modular pieces of a third-party system (e.g., the module that displays snippets of matching text).
• Powerful for the end user:
• Spell checking of queries/li>
• Faceted displays for browsing
• Dynamically updated browse lists
• Session-based bookbags

These basic features can be tuned and modified.  For instance, the same bookbag feature that allows users to store links to entire books, can also store links to citable elements of an object, such as a note or other reference.

A sampling of XTF-based applications both within and outside of the CDL include:

• Mark Twain Project Online (http://www.marktwainproject.org), developed by the Mark Twain Papers Project, the CDL and the University of California Press.
• Calisphere (http://www.calisphere.org/), a curated collection of primary sources keyed to the curriculum standards of California’s K-12 community, developed by the CDL.
• The Encyclopedia of Chicago (http://www.encyclopedia.chicagohistory.org/), developed by the Chicago History Museum, The Newberry Library, and Northwestern University.
• The Chymistry of Isaac Newton (http://webapp1.dlib.indiana.edu/newton/) and The Swinburne Project (http://webapp1.dlib.indiana.edu/swinburne/www/swinburne/), Indiana University.
• Finding Aides at the New York Public Library (http://labs.nypl.org/2007/10/30/extensible-text-framework-xtf/).
• EECS Technical Reports (http://sunsite2.berkeley.edu:8088/xtf/servlet/org.cdlib.xtf.crossQuery.CrossQuery?rmode=btr), UC Berkeley.

The "CDL Guidelines for Digital Objects, Version 2.0" (CDL GDO) has been updated to include specifications for use of the METS File <file> element.  You can find the updated version at http://www.cdlib.org/inside/diglib/guidelines/ .

The revision applies to Sections 2.1, 2.2.2, 3.1, and 3.2.4 only:

• To support the orderly transmission and ingest of digital objects, the CDL recommends the inclusion of checksum (MD5, SHA-1, or CRC32) and byte size values in the METS File <file> element.  Note that this information is preferred, but not required.
• The subheadings within Sections 2.1 and 3.1 have been relabeled, and are now consistently based on METS element names.

Please contact the CDL at oacops@ucop.edu if you have any questions.

The "CDL Guidelines for Digital Objects, Version 2.0" (CDL GDO) has been updated to reflect modified requirements for METS unique identifiers.  You can find the updated version at http://www.cdlib.org/inside/diglib/guidelines/ .

The revision applies to Section 3.1 only, and pertains to objects submitted for the CDL’s "Enhanced Service Level”.  This service level encompasses the presentation of digital assets via CDL websites. It is also sufficient for increased preservation services in the UC Libraries Digital Preservation Repository.

The METS top-level <mets> tag must contain an OBJID attribute containing an ARK identifier for the digital object.  Previously, the CDL GDO indicated that the OBJID attribute could contain a unique local identifier in lieu of an ARK identifier.  CDL systems do not support this scenario, however, for objects submitted for the Enhanced Service Level only.

Please contact the CDL at http://www.cdlib.org/inside/feedback/ if you have any questions.

Position title: Manager, Infrastructure and Applications Support  (Req. 20070286)
Position location: Oakland, California
Closing date: July 19, 2007

Reporting to the University Librarian and Executive Director, the Manager of Infrastructure and Applications Support is responsible for the technical design, implementation, maintenance, and operations of the common technology enterprise services that support all program and service areas. The Manager is responsible for the Computing and Storage Resource Center comprising a distributed network of CDL-owned resources at three physical locations and for managing the overall integration architecture for computing systems, database management systems, storage systems and network infrastructure. The Manager also provides support to application developers in the CDL’s program areas for the development, staging and production environments, and for collaboration tools supporting the work of CDL and its partners.

HOW TO APPLY:  For a complete job description or to apply for this position, visit the University of California Office of the President employment web site at:  https://jobs.ucop.edu/applicants/Central?quickFind=51793; it is also listed on the CDL web site at http://www.cdlib.org/glance/jobs.html.

UC Office of the President job postings: http://jobs.ucop.edu
Job requisition number: 20070286

Visit the California Digital Library web site at: www.cdlib.org

The University of California is an Equal Opportunity /Affirmative Action employer.

Beginning in 2007, CDLINFO was reformatted in order to help CDL’s latest news reach users speedily via RSS feeds for either all items that appear in any month or for items in specific areas only. See Subscribe to CDLINFO RSS for more information. CDLINFO will still be sent to subscribers via email, as usual; however, it will arrive monthly instead of biweekly.

If you would like to contribute to CDLINFO, please send email to Robin Davis-White at CDLINFO-SUBMISSIONS-L-Request@UCOP.EDU. If you would like to contact the CDL Web Production Team, email Eric Satzman at esatzman@ucop.edu.

http://www.cdlib.org/inside/diglib/guidelines/

Digital materials of ever-increasing variety and complexity are seen to be worth collecting and preserving by memory organizations — libraries, archives, museums, etc. Materials include objects converted into digital form from existing collections such as manuscripts, maps, visual images, and sound files, as well as “born digital” materials such as web sites.

In order for the CDL to provide effective preservation and access services, these materials need to be represented in a uniform manner. The CDL GDO provides specifications for all new digital objects prepared by institutions for submission to the CDL. It is based upon and supersedes the “CDL Digital Object Standard, Version 1.0″ (May 2001) and the “OAC Best Practice Guidelines for Digital Objects, Version 1.1″ (January 2004).

The CDL GDO includes the following features:

• Establishes “sliding scale” requirements, i.e., the more a digital object conforms to the guidelines, the more preservation and access services can be provided for it.
• Provides specifications for preparing digital objects, comprising metadata and content files (e.g., digital images, text) packaged using the Metadata Encoding and Transmission Standard (METS) format.
• Includes updated recommendations for digital image files.

A draft version of the guidelines was prepared from the fall of 2004 through the winter 2005. Feedback received from CDL contributing institutions was incorporated into this final version of the guidelines.