It is hard to capture in a blog post this year’s code4lib and do it justice. However, I will can honestly say it was one of the best conference experiences I’ve ever had. This year’s conference had a larger and more gender diverse audience. There was a fabulous preconference, great presentations, and helpful breakouts. Probably the thing I liked the best was just getting a chance to sit down, play with code, and have other people help me figure things out. The moment that captured the spirit of the conference for me was sitting with Terry Resse and Erik Hatcher talking about and playing with Solr. I learned a lot and got a tangible idea about how I was going to integrate a Solr-based application into my site. When I started the conference I felt overwhelmed with trying to get Solr running and building a usable application. By the end of the conference, my level of confidence was much greater. A great deal of that just had to do with working on stuff and having people answer questions and solve problems. It is a great feeling to come away with and one of the things that I value most about the code4lib community. I’m already looking forward to next year’s code4lib in Portland, OR!

• iStumbler
• Bonjour Browser

From Ed Summers Atom Publishing Protocol Primer

Atom protocol RFC 4287

REST (Representational State Transfer)

• Application state and functionality are divided into resources
• Resources are uniquely addressability using a universal syntax (ie. URIs)
• All resources share a uniform interface for the transfer of state between client and resources (HTTP)
• Live HTTP Headers

• Firefox plugin for looking at these
• GET
• POST
• PUT – update
• DELETE

Atom + REST = APP (Atom Publishing Protocol)

A Few Particulars

• service
• workspace
• collection (feed)
• resource (entries and media entries)

Service Document

• Sits on the server and defines what collection are available and what you can do with them

Google Data Uses Atom Publishing Protocol

• Google Base
• Blogger
• Google Calendar
• Google Code Search
• Google Notebook

Create an Atom Server yourself

• Help interoperability on the web
• Linked to Web Service API building

WebDAV is similar

Idea: UW Bibliography

• don’t know where people are publishing and need to capture this information
• Directory
• Citation
• Citation Management
• Archival rights (Sherpa/RoMEO)

What is it?

• 12,000 + Citation
• 2,700 Ready to Archive
• 9 Departments
• 257 People

Find items that are ready for deposit into Institutional Repositories

Written in Rails

How to use the system

1. Import Directory Server data
2. Add People
3. Add Papers
4. Collect the archival data from Sherpa
• want to know if you can upload and archive the publisher’s copy
5. Flag publishers that let you upload and archive the publisher’s copy
6. Use OpenURL to get electronic copy and save PDF to server

Features

• Tag cloud of terms
• List of people
• Find an Expert
• RSS feeds

• People
• Groups

To Do List (it is not perfect)

1. Sherpa/RoMEO
• don’t have have complete coverage
• Can’t disambiguate what copy (publisher, pre-print, etc) can be archived
2. Our de-duplication is jacked
3. Name collisions

Get the code
Google Code Project – http://code.google.com/p/bibapp

Concept

• Input – user search
• Output – A list of related library subjects
• Subject recommendation engine

Motivation

• Search log analysis
• What kinds of queries are people inserting into our search tool
• Lots of topical subject queries

• connect people to subject portal in a search environment
• Locally developed subject classification
• 100 subject notes in 12 top-level categories
• Subject influence by the university cirriculum

Smart Subject Applications

• Quick Search Integration
• Find article module
• Recommends a set of topical databases

• simple format for syndicating search results

How it Works

1. Harvesting available institutional data
• Course catalog
• Course Descriptions
2. Faculty Publication Repository
• Article citations
3. Create text extract representation for each academic depart on campus
4. Index the text extracts
5. Retrieval interface queries indices
6. Retrieval algorithm crosswalks academic departments to Library Subject Recommendations

Technology Used

• SWISH-E
• PHP

Strengths

• Application and collection independent
• Subject recommendations can be integrated in any library search application
• Encourages broadened, serendipitous resource discovery

Weaknesses

• False Positives
• Zero Hits

Future Plans

• Deploy new users of Smart Subject tool
• Database adviser

• article tables of contents data
• backlog of course descriptions

http://www.lib.ncsu.edu/dli/projects/smartsubjects

Look for holding availability

CatalogWS

• REST web API for dynamically querying information from the NCSU Libraries Catalog

Motivations

• Initial impetus – 2 requests

• Can we have RSS feeds for the catalog?
• Can we integrate catalog results into library

Using the service

http://www.lib.ncsu.edu/catalogws/?

Parameters

• service
• query
• output
• count
• offset
• sort
• style

XML Response

• Defined with Relax NG Schema
• Data from search results page

• Search information
• Results
• Facets

Services on top of XML

• RSS
• OpenSearch

• role=subset
• Extended OpenSearch parameters to createa face parameter for use in the Open Search URL template

CatalogWS project site

ILS agnostic web portal for students and faculty to perform research activities

Create 1 single interface for all library resource to minimize interface learning curve

Develop in-house a “framework” to combine all of our resources

• Most resources are in XML
• Digital Library: METS
• MetaLib XServer: XML
• Catalog: MARCXML
• Library Website: XHTML

Data Store

• Native XML stores allows for easy storage of complex data
• No need to develop a complete relational database and covert data – too messay
• No need to normalize data
• Just import!

Native XML Database- Could it be that simple?

eXist – Open Source

• Still in infancy-ish stages
• Platform independant

• Java Backend
• API: REST, SOAP

Berkeley DB XML – Open Source

• Proven capabilities
• Support for a wide range of platforms
• Good performance
• Decent help support
• Commercial backing
• No full-text extensions
• No inherent directories

Commercial Options

• MarkLogic

• Enticing Discounts for .edu and non-profits
• Commercial Support
• Much more complex to administrator
• Speed

Scalability Testing

• eXist not meant for searching, more for browse and fetch
• DBXML Sleepycat – rework queries and modified indexes to make these respond in 30-60 seconds

Converted MARCXML to custom format because MARCXML not helpful (elements all have the same names)

• dbxml – 1.6 -1.7 second response

Query Optimization

• This is an important step since we are dealing with infant technology
• dbxml has a query plan generator
• eXist will soon have a query plan generator and a new query optimizer

Implementation

Create a web portal using a Native XML Database

Performance

• The good

• .9 seconds

• More advanced queries can get as high as 12-15 seconds

• What happens when 10-50 simultaneous users search with advanced queries

Need to develop a lots os search query translation algorithms to missing Full Text Extension

So the answer is NOT YET!

It’s a Sun Shiny Day

• Apache SOLR to the rescue!
• SOLR implements Lucene index on XML documents
• SOLR is platform independent

• Runs as a java web app
• Interface via REST

• XML database use XQueries

Easy Implementation

• XSL Stylesheet to covert MARCXML to SOLR XML

• Coverated 492,000 in 2.5 hours

• 3 hours
• Andrew showed the final product which uses Solr in the Lightning Talks

Other options

• Nux
• XTF
• Cheshire

• We have given away our collections
• We don’t build or own the tools that manage them
• We provide complex, poorly-marketed systems
• We function like a monopoly service when our competition is thriving right under our nose.

Memory work – preserve and provide access to our culture’s memories

5-3-1 rule

• Pick 5 issues you believe are important
• Focus on 3
• Now make 1 happen

Five Things we can fix

• Digital preservation
• Standards adoption
• The sucky state of most library software
• Third-party library hegemony
• Scholarly awareness of key issues in LibraryLand

3

• Digital preservation
• The sucky state of most library software
• Scholarly awareness of key issues in LibraryLand

1

• The sucky state of most library software

4 Nifty Happenings with Library Software

• Evergreen
• Umlaut
• Scriblio
• The solr search engine you are all going to bolt on top of your geezy old ILS middleware as soon as you get home from code4lib

Renaissance of librarian built software

• Begins to restore the balance of power
• Reinstates the direction of our profession
• Puts emphasis back on the library as memory organization
• Sends the message that we means business

Other outcomes

• Creative decoupling of components
• Interesting re-use of tools, such as WordPress
• Re-socialization of librarian artisans

My big 1 today: Evergreen

• Evergreen is big… really big
• Timing is perfect: an era of worrisome consolidation, even as … paradoxically…
• The centrality of the ILS is weakening

• in a better position to take risks, change software

Useful over-generalization

• Nobody cares about open source
• Nobody cares about standards
• Nobody cares about usability
• Nobody cares about Evergreen
• The ARL body count continues to drive too many decisions
• IT directors do not have the resources to take on underfunded mandates
• Most libraries cannot provide developer time

It is all about the experience they don’t care about the software

How directors see the world

• How much does it cost, and what are we getting for the money?
• What are other directors doing?
• What problems will it create?
• Why would I spend time/money on this rather than X?
• Is this thing fully baked?

Need to answer these questions ahead of time. Otherwise, you don’t have a prayer of getting resources/support.

What do directors “know” about open source

• One guy in a garage… probably in a torn tee-shirt

• One care accident away from orphan software
• No support model
• Cheesy “make-do” quality
• Arcane and developer-oriented
• Nobody else is doing it

Open source software projects have a huge problems with marketing their “products”.

There is no such thing as free software!

5 Strategies

• The riveting lead
• The elevator talk
• The pre-visit background investigation

Search Analytics for Your Site – Lou Rosenfeld

]]> http://www.librarywebchic.net/wordpress/2007/02/28/hurry-up-please-its-time-karen-schneider-keynote/feed/ 2 http://www.librarywebchic.net/wordpress/2007/02/28/code4lib-day-1/ http://www.librarywebchic.net/wordpress/2007/02/28/code4lib-day-1/#comments Thu, 01 Mar 2007 02:42:45 +0000 Karen http://www.librarywebchic.net/wordpress/2007/02/28/code4lib-day-1/ My first full day at code4lib started off with a bang. I ran into Nicole Engard who is the recipient of this year’s the Code4Lib Scholarship for Women. It is really nice to see Nicole is here as well as so many other awesome tech women.

I also talked to Terry Reese about LibraryFind and where it is going. Terry also has some other interesting projects he is working on. In particular, he’ll be doing a Lightning talk tomorrow that I’m looking forward to. I’m psyched about getting to LibraryFind running at UH, when we get a chance.

Overall the first day of the conference was awesome. I gave a Lightning talk on AWStats and how we’ve created some custom reports. I got to talk to lots of different people and see great presentations. For those who couldn’t make it, parts of code4lib were videoed and these will be put up later. I hope that you are able to enjoy the conference vicariously through the blog posts and video.

]]> http://www.librarywebchic.net/wordpress/2007/02/28/code4lib-day-1/feed/ 2