THE INTERNET -> YahooPipes ->Processing
This is a quick how-to on how you can write a custom yahoo pipes to scrape data from various websites and rss feeds then dynamically load them into a processing application for representation.
Basically you can use yahoo pipes to do what it’s good at, collect, scrape and collate data from around the internet then return have it return only what data your interested in to a processing application in a nice standardised re-usable way though a xml file.
First example (simple):
We have yahoo pipes collect all known earthquakes over the last 7 days from a rss feed provided by usgs, we then use the “location extractor” provided by yahoo pipes to extract the earthquakes location to lat and longs and we also use a simple regex command to extract the earthquakes magnitude from the earthquakes description. All of this is then returned to a processing script where the location and magnitude of each earthquake is visualised.
The Yahoo Pipe: here
http://pipes.yahoo.com/pipes/pipe.info?_id=a135e8a8730f782fb817d792298562a4
The processing Patch: here
Example 2: the average twit
This example searches for conversations about specific topics on twitter and builds up the average icon of those users talking about that topic. because we are not including icons more than once these faces tend towards individuals that twitter about a specific topic a lot, this means that more obscure topics tend to be among specific individuals resulting in a icon heavily weighted to the icons of a few users and more ubiquitous topics produce greater less defined icons.
The Yahoo Pipe: here
http://pipes.yahoo.com/pipes/pipe.info?_id=rJkmdk2b3RGn_ShI1ZzWFw
The processing Patch: here
requires the proXML library for processing from here
TrackBack URI