Hi dave, just caught your Morning Coffee notes with your thoughts on

the Google API and making it into infrastructure.  It totally echos

many things I've thought, and the easiest expression of my parallel

thinking I found off the top of the 'Net is in a comment thread about

the Skype API, where they ask the dev. community for input:


http://lennpryor.blogs.com/lenn/2005/05/i_need_your_inp.html#comment-6184160


Here're my thoughts regarding the points you make in your podcast

(which was awesome, BTW!)  I have personally stopped development on

one project, and not started others, due to the so closed as to be

stupid access policies of the google and yahoo search api's.  The

project in question was a greasemoneky script to detect plagarized

text in blog comment posts.  It was pretty short, as it was just

shoving each paragraph through the search engine of the moment that I

had wired up to it.


Google, yahoo, altavista, msn...didn't matter, they all would either

lock you out after not very many screen scraped or api searchs.  This

thing does a query per paragraph.  All that it's primitive heuristic

needed was the number of hits for the search term, but would anyone

give me Just That? No, I had to screen scrape the one freaking INT I

needed out of an entire page of text, as it was not in any of the xml

api's.  Javascript is pretty easy to tweak, so on each page reload I

was doing lots of queries on a thread of any length.  The actual

logic/heuristic never got improved because the search engines just

won't play ball with developers.  Altavista is the only one that does

block me on bandwidth limits, but they have no api and their results,

frankly, suck.


I hit the max on every engine very quickly, either do to api limits or

screen scrape bandwidth use.  Oh how retarded, and I'm pretty sure

it's because all of the search engine companies get most of their

revenue from ads.  Apple, MS or IBM, who don't should do a totally

open one like you're talking about.  Or hell maybe I'll start a

project to do a P2P distributed search engine with an api that's a

superset of the union of the google, yahoo, ebay and amazon api's (oh

yeah, amazon and ebay api's suck for developers for mostly the same

reasons as googles!)  Bet some folks might be down for some code

jamming on that! :-)


What the hell gives with that, I've been wanting to build 3D VR stores

of ebay for YEARS, but just can't get the level of open-ness in the

API that I need.  Amazon too on this one. Do they not understand that

this will Send Them Lots of Business?  The more developers that use

their backend, the more customers they get sent!

As you said, new flow is a market signal, as economists would say,

notifying you of new opportunities!


Completely run with the 'how can we both/all make money?'  Wasn't that

part of the dream of component based software, that we could all be

cottage programmers competing to make the best widget that did X,

where X is whatever we're currently interested in? Wasn't that part of

the promise/dream of OO in the 80's-90's?


Why isn't XML-RPC, SOAP or REST the internet version of this vision?


But not all of yahoo's revenue is from ads, my wife subscribes to

Launchcast for instance (their 'custom channel' A.I. is spooky good at

finding new things you'll like after you've trained it Long Enough!)

and they have email subscribers and suchnot too.  What percent of

their revenue is from search engine ads specifically, since I'm

certain that that is where their fear comes from.  Losing those

eyeballs to some app. that uses their api.  Google are almost 100%

ads, so of course they are evil to us web developers.  Don't be evil

my ass!


IBM have always been the best at returning calls/emails, from where I,

a mostly anonymous, custom software mercenary see things (while

whereing that hat, at least).  NeXT was awesome to developers, until

they got bought by apple, then it went back to the Apple Way with lots

of $$$ required for any goodies.  The balance is a bit better, but

they don't give you anywhere near the free tools/toys at a free level

of developer membership as MS or even IBM these days!


Anyway, thus ends a rant saying 'hear, hear!' to your MCN, in spades!