WorldCat Registry and Yahoo Pipes

2009 April 13

Ever since I discovered the WorldCat Registry web services, I’ve been drawn to the idea of creating mashup maps of libraries using the data in the registry. When you do a registry search you get back a list of libraries and some basic information, like their address. Therefore it didn’t seem like it would be too difficult to show the libraries in a particular town or zip code on a map. After all geographic location is one way the registry let’s you search. Being more of a hack and slash coder, I wanted to find an easy, simple way to do this that others could understand. Enter Yahoo Pipes, which I have a love/hate relationship with. I love how easy it should be to create mashups with Pipes. I hate the fact that anytime I do something new with it I need to get my stress ball out. My map mashup project has been no exception. Part of the problem is the XML which OCLC outputs. (They need to look at other web standards for XML and see if their content matches with any of these.) After some serious massaging of the data with Pipes to get to the right group of nodes. I was able to easily get the address info: street, city, state, zip, country. Problem is that I’m doing something wrong because I can’t get Pipes to build me a map from this. Grr!! I can’t seem to get my head wrapped around the Location Builder and Location Extractor widgets, which should be helping me (I think). I’m wondering if I need latitude and longitude. I don’t think so, most maps services are smart enough to find things based on street address. I’m wondering if the stupid random capitalization in OCLC’s data is causing issues.

At this point, its time to take a step back. I did this when I was working on the WorldCat Search API Pipe and it eventually led to success. However, if anyone has advice and tips about Yahoo Pipes and Maps, please feel free to comment. Also I’ve published my Pipe in its current incarnation, which means those of you who are feeling geeky can clone it and try to make it work right!! Suggestions and solutions are welcome in the comments.

Update 4/13/2008 8:25pm

Thanks to Tony Hirst my Pipe is now working. Downside, it doesn’t show all the libraries on the map at once :(

I’m still working on getting it to take parameters. Ie so people could put in a City, State and see all the libraries. Some of that is deciphering a bit of how the urls which request info from the registry are built. (Can we say fun with url encoding!?) Will update folks as I make progress.

4 Responses leave one →
  1. 2009 April 13

    This immediately reminded me of a blog post that came through my reader a while back: http://ouseful.wordpress.com/2008/10/14/data-scraping-wikipedia-with-google-spreadsheets/
    It’s a tutorial of sorts for taking structured data, piping it, and coming out the other end with a workable map.

    I think as long as you use similar mappings (not sure about the y sub-node) to the tut, you can get the kml feed and use it anywhere.

    Hope this helps.

  2. 2009 April 13
    hapdaniel permalink

    You can use a Location Builder within a Loop module (drag the Location Builder from the sidebar into a Loop module that is already on the canvas). For the location select item.ad:zip and set the results to be for item.y:location ( you will need to type that in). Pipes will build the necessary geo information based on the contents of y:location. Please remember that Pipes uses caching, so it could take 30 minutes for those changes to show up.

  3. 2009 April 13

    This pipe does the geo-coding:
    http://pipes.yahoo.com/pipes/pipe.info?_id=d833d1319417f906b304785ff0c6a04e

    Email me if you need any more hints…

  4. 2009 April 14
    Joanna White permalink

    Hi Karen, so glad to see that you got WorldCat Registry working with YahooPipes now! Are you are still running into issues with xml? Registry xml tags are explained in more detail at http://www.worldcat.org/affiliate/tools?atype=regsearch. The upper and lower case in library names or addresses should not be causing issues, or at least we have not heard of any. We are currently doing cleanup on some older profiles that are not conforming to the schema. Not sure if that is what you are running into. Don’t hesitate to contact me directly if we can help out offline.

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS