From the leader in this week’s Economist:
“Financial progress is about learning to deal with strangers in more complex ways.”
s/Financial/Technical/ and it applies just as well. What else are we doing in tech, if not figuring out ways for strangers to deal with each-other? Sometimes we focus on designing safeguards, like firewalls or spam filters, and sometimes we focus on creating opportunities, like social networks or source code repositories.
Late in 1963, shortly before he was assassinated, U.S. President John F. Kennedy asked Canadian Prime Minister Lester B. Pearson for his opinion on how the U.S. should cope with escalating unrest in Vietnam.
Peason: “Get out.”
JFK: “That’s a stupid answer. Everyone knows that. The question is how do we get out?”
How, indeed? As JFK had finally come to understand, military conflicts, justified or not, are like a Chinese finger trap: it’s easy for a political leader to order the troops in, but very tricky to pull them back out (just ask the British about Northern Ireland, the Russians about Chechnya, or even Pearson’s Canadian successors about southern Afghanistan).
Good luck to President Clinton, President McCain, or President Obama (alphabetical order) in January 2009 — they’re all smart and well-intentioned people, but they’re going to find that the trap has already been pulled very tight, and there’s not much room left to wiggle free.
In the search logs for OurAirports, I noticed a series of searches for URLs:
http://www.feliciano.de/Webgalerie/bilder/Italy/une/yiwul/ http://www.unduetretoccaate.it/codice/aseje/wocobo/ http://www.altaiseer-eg.com/ar/articles/jed/umut/
At first, I thought they might be a kind of link spam — some sites display recent searches — but when I checked one of the URLs, I found something totally unexpected:
<?php echo md5("just_a_test");?>
They’re all the same. This is almost certainly related to passwords: is there a known flaw in a PHP content-management system like Drupal, or in the PHP API for a search engine like Lucene, where this would do some damage, or is it just a test probing for weaknesses? Is the PHP code supposed to be served up literally like that, or should I be seeing the MD5 instead?
Some people compare blogs (and mainstream media) to an echo chamber, constantly repeating and amplifying the same messages, but the echoes usually die out quickly. Not so, today, when I found this story on the planenews.com aviation news feed:
21 Feared Dead in Munich Crash.
About twenty one of the 44 passengers and crew of the British European Airways airliner which crashed yesterday near Munich carrying the Manchester United football team and many journalists are feared dead. About eight others are in hospital, seriously injured. Frank Swift, the former international goalkeeper, who had become a journalist, died in hospital.
I didn’t hear about any crash yesterday, but according to the Wikipedia article on Manchester United, there was a crash near Munich on 6 February 1958 that killed eight of the team’s players. In fact, when you follow the full story link in the posting, there is a story about the crash. The phrase “From the archive” is hidden in the deckline, but the dateline is “Saturday February 2, 2008″ (probably automatically updated by the site). There’s nothing else in the online version to indicate that this is an archived story from 7 February 1958, though a Brit would probably know that British European Airways ceased operations in 1974.
This is an easy mistake to make trying to keep up a blog of current events, and I don’t mean to suggest that the maintainer is stupid, or that I couldn’t do the same thing — in fact, next December, watch this spot for postings about an air attack on Perl Harbor.
The Wikipedia article about me was vandalized yesterday (vandalized version) by someone from the IP address 24.225.66.95, which seems to be in or near Raleigh, North Carolina.
What should I do?
I’m leaning towards #5, though I’m disappointed that kids these days seem to have forgotten how to swear properly: “a freaking loser”???
I forgot that I’d enabled Google analytics for the XML 2007 web site. Even though the conference is long over, I though it would be interesting to look and see what some of the trends were from September 2007 to January 2008 (keeping in mind that these stats apply to the kind of web users interested in a tech conference, not to the web at large).
Despite the halo effect from the iPod and the widespread use of Mac notebooks among speakers, MacOS still hasn’t managed to make much of a dent in the visitor logs:
If MacOS can’t beat Linux on the desktop, I don’t know if it has a bright future.
Firefox is still #2 behind MSIE, but for this crowd, the gap is small:
If you’re designing or maintaining a web site with a tech audience, you’d better be testing on Firefox as well as MSIE.
I know that web designers like big layouts, but the sad fact remains that 1024×768 is still the most common resolution (and remember that the browser window may be much smaller than the screen):
A long tail of resolutions follows, but it’s worth noting that the classic 800×600 has only 1.96%. Better news comes from colour depth, where almost everyone has 16bpp or better:
Search engines, referrers, and direct access were all important traffic sources:
Blogs did show up among the referring sites, but the biggest traffic producers were traditional links from partner organizations (other conferences, IDEAlliance itself, etc.) — these were also the stickiest, since most people coming from these links went on to read more than one page.
As far as search engines go, I was surprised to find that nothing really matters but Google (assuming that Google Analytics isn’t biasing the numbers):
I knew that Yahoo! and MSN were behind in search, but I had no idea just how bad it was (at least in the tech crowd). More than half of the people who found the site via a search engine went on to read more than one page.
The top search phrases were rather dull and predictable:
And so on through a very long tail. Individual speakers’ names start appearing soon, but none with more than 10 searches. I trolled through the low-frequency search phrases for something funny (and maybe risque), but all I came up with was the number “736″, which resulted in three visits. I gave up trying to find the site in the Google results for that number. Does anyone really search for a single three-digit integer, and if so, how many pages of results will that person scroll through?
I’m using a single dedicated server to host ourairports.com, megginson.com, and a couple of minor domains. OurAirports is a database-heavy application using (currently) a MySQL v.5 database hosted on the same server. I’ll offload the database to a separate server if traffic keeps increasing, but as long as I’m getting compliments from tech people for my fast response times (mainly thanks to MySQL’s built-in query caching), there’s no point paying for extra hardware.
My ISP set up the server for me last summer with a bare-bones Ubuntu distro, then I installed the extra packages I needed using aptitude over ssh. Since then, I’ve done many Ubuntu in-place upgrades, rolled out hundreds of changes and upgrades to the web apps and dozens to the database schema (some very significant), and upgraded WordPress n-teen times. Check this out:
$ uptime 13:08:31 up 175 days, 10:02, 1 user, load average: 0.23, 0.06, 0.02
That’s right — since my ISP first set up the server with a basic Ubuntu system, I’ve never had to restart it. In fact, if Apache and mod_php (PHP5) had ‘uptime’ commands, they’d show almost the same amount of time, since I restarted them only to make configuration changes in the first few days of setting up the server (unless apt stopped them to install a newer version during one of my upgrades). I’ve restarted MySQL more recently, but again, only to experiment with configuration changes (especially for fulltext).
Using reliable old technologies like Linux, Apache, MySQL, and PHP doesn’t win any cool points, but it certainly makes maintaining a web server and its applications easy. I can go on vacation, for example, without worrying about being able to get online to fix or restart my server every couple of days. I don’t have to stay up until 3:00 am on Sunday night so that I can take the server offline to roll out new software versions or bug fixes (aptitude installs any security fixes in place). I spend lots of time with my family. I go to my kids’ school concerts. I learned banjo and mandolin (why not, since I have the free time?).
And yes, my PHP web app is easy to maintain and extend, because I designed it to be that way (I can often implement, test and roll out new features in a matter of minutes, even when they require database schema changes) — it’s the developer, not the programming language, that determines the quality and maintainability of an app. A lot of newbies use PHP, so there’s a lot of bad PHP out there, but the same can be said for any language, even Ruby.
Image: Thomas Penn, second proprietor of Pennsylvania, not as nice as his dad William.
Almost a year ago, I wrote that Open data matters more than Open Source — it doesn’t matter (to you, the end user) whether a web site is using Open Source software or not, if they still keep your data locked up.
Here’s a nasty example: Robert Scoble has just had his Facebook account disabled for running a script to try to scrape his personal information off the site (since Facebook doesn’t provide him with any other way to get it).
I understand that Facebook needs to protect against malicious bots — and they might decide to restore his account once they know what Robert was actually trying to do (though for now all traces of him have vanished) — but do we really want to have hope for the good will of social sites and beg for our own data every time we want it? Are web site owners the new version of the Proprietors in the early American colonies, who can grant rights as favours when they see fit?
Update: I read that the school concert went ahead, with Frosty the Snowman replacing the modified Silver Bells as the token non-religious song on the programme (Frosty makes no reference to any religious holidays).
Both of my children attended Elmdale Public School here in Ottawa from junior kindergarten to grade six. Now, my kids’ alma mater has triggered a nation-wide moral panic by changing the line “it’s Christmas time in the city” to “it’s festive time in the city” in the song Silver Bells for a grade-two and -three concert.
I’ve already gone on record saying that it’s OK to wish me Merry Christmas — I’m as proud of my Christian background as some of my friends and neighbours are of their Jewish, Muslim, Sikh, and Hindu backgrounds — but that’s not what this was all about. The primary choir was already singing songs about Christmas and Hanukkah, and the choir leaders decided to add an additional song that was non-religious. I think that the existing non-religious songs Jingle Bells or Winter Wonderland would have been fine, but they decided to take Silver Bells — an otherwise secular pop song about shopping downtown in a city — and replace the word “Christmas”. Silly? Probably. An attack on Christmas or Christianity? Hardly.
Here are some people who might need help understanding the idea of Christmas and Christianity:
According to the Christian New Testament, Jesus didn’t have anything good to say about people like this — he far preferred the company of prostitutes and tax collectors to the religious self-righteous. If you are religious (any religion), pray, meditate, or just hope that their hearts can still be opened this season.
A bit over a year ago, I ran into an unusual problem — for several days, I stopped receiving messages from a customer (in the middle of an important project), then I discovered the messages all hidden deep in my (gmail-hosted) spam box. Everything from that domain was suddenly being flagged as spam.
What happened? This customer had a large mailing list that they used for announcements, etc. My guess is that they sent out an announcement, a lot of other gmail-users flagged it as spam, and whatever weighting algorithm gmail uses tipped it over so that the messages were no longer considered legit by default. I was able to train gmail not to treat those messages as spam (for me, specifically), but it took a week or two before I could trust that some of them weren’t being sent to the spam box.
Hard-core spammers have always had to deal with this kind of thing, and they spend a lot of time trying to figure out a way around it. What’s happening now, though, is that companies with legit (or semi-legit) e-mail lists are also starting to get into trouble, because web-mail makes it possible for hundreds or thousands of people to get together and all vote your e-mail to be undesirable.
That this isn’t a legal thing. It doesn’t matter at all if your e-mail list is opt-in or opt-out, if the “Send me announcements” checkbox was checked by default or not, or if the recipient originally clicked 10 screens of disclaimers before buying your product/signing up for your service. If they don’t like the e-mail you’re sending them, they’ll just click “Spam”, even if you had a legal right to send it; and if enough of them do it, the e-mail value of your domain fast approaches nil.
You’d better make sure that your mass e-mails have stuff that people actually want to read:
And so on.
This new collaboration is an unexpected side-effect of the shift from desktop e-mail clients to web mail, and it would be foolish for companies not to pay attention. If you consider your domain name to be a valuable part of your corporate identity, don’t piss it away by sending out poorly-targeted mass e-mails, because no matter what prior permission you have, people now can … and will … punish you. After all, it takes only a single mouse click.
This might be of interest:
Dear AWS Developers,
This is a short note to let a subset of our most active developers know about an upcoming limited beta of our newest web service: Amazon SimpleDB, which is a web service for running queries on structured data in real time. This service works in close conjunction with Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), collectively providing the ability to store, process and query data sets in the cloud.
Traditionally, this type of functionality has been accomplished with a clustered relational database that requires a sizable upfront investment, brings more complexity than is typically needed, and often requires a DBA to maintain and administer. In contrast, Amazon SimpleDB is easy to use and provides the core functionality of a database - real-time lookup and simple querying of structured data - without the operational complexity.
Were excited about this upcoming service and wanted to let you know about it as soon as possible. We anticipate beginning the limited beta in the next few weeks. In the meantime, you can read more about the service, and sign up to be notified when the limited beta program opens and a spot becomes available for you. To do so, simply click the “Sign Up For This Web Service” button on the web site below and we will record your contact information.
It’s not SQL, or even SQL-like, though, supporting only the operators “=, !=, <, > <=, >=, STARTS-WITH, AND, OR, NOT, INTERSECTION AND UNION”. I’m no relational expert, but I don’t think Codd would have been impressed. A distributed database is one of the big missing pieces from Amazon’s services, but I’m not sure if this will be it.
While I posted a lot before the XML 2007 conference, I didn’t really have time to post anything during it. This is the third time I’ve chaired a medium-sized tech conference, and while it’s busy all year, it’s insane while the actual conference is in progress — any free time is for rest or nourishment, not blogging. The conference ended yesterday, and I haven’t really had time to reflect on and digest the experience, but here are some things that stick out right now from the memories swirling in my head.
It was cold this time, colder than XML 2006 last December in Boston. I brought suitable clothing with me, so I was still able to get outside for some long walks around the city, but with wind chills dropping well below -10degC, a lot of visitors probably weren’t equipped and had to spend most of the conference indoors. That’s not as bad as it sounds, since the hotel is connected to, literally, kilometers of indoor shopping, with pedestrian bridges across major streets. Unfortunately, they probably didn’t make it the 2.5 blocks outside to have banana or cranberry pancakes for breakfast at Charlie’s Sandwich Shoppe.
We also had snow — only a dusting of about three inches in Boston the night before the conference started, but since Boston’s not a snow city like Buffalo, Syracuse, Ottawa, Montreal, etc., I wasn’t sure if they’d have the roads clear and people would make it to the opening plenary. Fortunately, while many schools closed outside of Boston, the downtown kept moving as usual. We had over 200 people in a packed ballroom, and there was good audience participation.
We had excellent 45- and 90-minute presentations during the day, of course, but some of the conference’s greatest energy was generated in the informal evening sessions. John Boyer’s XForms evening on Monday, and Ken Holman’s Standards and Specs Lightning Rounds on Tuesday both played to packed rooms with noisy audiences.
Both featured rapid-fire sequences of short presentations (15 minutes each for Monday, six minutes and twenty seconds each on Tuesday) giving the people there a chance to hear lots of different speakers and ideas in a short period of time. If you weren’t there on Tuesday night, you’ll find it hard to believe that an evening of standards talks could be exciting, but it was.
XML is no longer mainly about a circle of people who know each-other and meet at conferences and committee meetings a few times a year. With so many new people, the change sometimes comes out as a lack of consideration — for example, speakers who wouldn’t bother showing up to hear the other presentation in their sessions (also leaving the session chair wondering if there would be a second speaker) — but overall, I think that people were back to being a bit more considerate than I’ve seen in the recent past. I saw fewer session chairs and speakers sitting in front of the room checking their e-mail while someone else was giving a presentation, had less trouble with people talking loudly in the hall outside while presentations were in session, and audiences tended to be friendly and supportive. Almost every presentation had a decent-sized audience ready to ask questions, and with all the work the speakers put in, they deserved at least that.
I experienced an example of consideration above and beyond any call of duty, when one person took time out from a serious family crisis to track me down, call me in my hotel room, and let me know about a change affecting the conference. You know who you are — thanks, and our thoughts are with you.
I love Boston. It’s a lot like Toronto, where I lived for six happy years and first learned how to love big cities properly (from near but not right in downtown, moving around on foot or public transit, and dealing with small stores and local merchants who get to know your name), but with its own special treats, like the antiquated trains (?) on the Green Line, a decent subway system that actually goes to the airport, truly fanatical baseball fans, and the Charles River. I’ve made a good number of visits to Boston over the past two years getting these conferences together, and I’ve come to feel very comfortable in the city.
Next year, I leave the conference to new people, and the conference leaves Boston for Arlington, VA. I hope that my life brings me back in contact with both the conference and Boston in the future, and I wish the best of luck (and stamina) to next year’s organizers.
I leave Ottawa tomorrow to fly down to Boston and get ready for XML 2007, which begins Monday morning. If the weather holds, I plan to head down in my own Piper Warrior, but I have a refundable airline ticket just to be safe.
The planning committee has been working on this conference for a full year now, and we have ended up with a very strong list of speakers representing companies big and small, both vendors and users. Take a look!
Update: the site shopping cart is broken, and doesn’t properly remove items from the total owing — too bad.
Here’s one easy way: via TechCrunch, Deutsche Grammophon, the gold standard in renaissance/ baroque/ classical/ romantic/ orchestral/ opera/ etc. music (often confusingly referred to collectively as “classical”, roughly equivalent calling all popular music since 1890 “rap”), will start selling their catalogue as unprotected MP3s at midnight German time tonight (6:00 pm in New York City) at their new site dgwebshop.com.
As a teenager in the late 1970s, I used to visit the House of Sound in Kingston (Canada), where they had thousands of DG records — probably most of the catalogue — packed in tight on on shelves lining a wall of the store. I couldn’t always afford them, but I loved being able just to pull them out and take a look at the covers of the different famous recordings. These days, the so-called classical music section of any but a couple of specialized stores in big cities like New York or London have maybe one or two rows of worthless classical-pop compilations hidden behind the DVDs of TV series nobody watched in the 1980s — no wonder people don’t shop at record stores any more.
We tech types have been claiming for a while that music companies could make more money selling unprotected digital music, so here’s the test. I plan to give them a lot of my own money if the site actually works, though I should note a couple of caveats:
I hope the site can handle the traffic. Rock on, Deutsche Grammophon!
The XML 2007 conference (Boston, 3–5 December) now supports both personal schedules and recommendations, thanks to Edd Dumbill and Expectnation.
If you’re planning to attend the conference (and I hope you are), you can create a free account (or use your OpenID), then visit the Conference schedule page. If you’re logged in, you’ll see a star in the top right corner of each slot. If you click on the star, it will turn yellow, and the presentation will be added to your personal schedule.
The web site uses people’s schedules to set up recommendations. For example, at the time I’m writing this, you can visit the page for Stewart Taylor’s and Adam Lee’s presentation XML and XPath in the Wild (analyzing the XML and XPath actually found on the web and in Open Source projects) and get a list of these recommendations for other presentations to attend:
You don’t have to be registered for the conference to do this — feel free just to poke around and see if you can put together a personal schedule that tempts you to come to balmy Boston in December.
(Update: I left out a lot of reviewers because of a last-minute editing keystroke gone horribly wrong — with luck, everyone’s listed now.)
The speakers for XML 2007 (Boston, 3–5 December) are listed on this web page — it’s well worth a quick skim, to see the range of people and companies represented this year.
However, beyond the IDEAlliance staff and and contractors, and the speakers, there are a lot of other people (all volunteers) who help to make a conference like this happen. I’d like to use this posting to thank them publicly, and hope desperately that I haven’t left anyone out. The order of names in each section is not significant.
For North America, it’s 1-800-816-2237.
The phone number is not available anywhere on Acer’s support sites (supposedly, it’s hidden somewhere in the Windows XP control panel, but you can’t get that with a broken computer), so I thought a nice, Google-friendly posting might be in order to help anyone else looking at a broken Acer notebook.
Probably because the number’s so well hidden, I got through instantly to a human being, who (a) seemed to be based in North America rather than overseas, and (b) knew what he was talking about, or at least was able to get answers.
I’ve recently been highlighting some of the presentations from the upcoming XML 2007 conference (Boston, 3–5 December). For the others, I’ve written a little bit about what I (and other members of the planning committee) found interesting and relevant about the proposal. In this case, though, I’m comfortable letting the title speak for itself:
A Lightweight Approach to Building the Department of Defense’s Semantic Web: Can Mashups Bring the “Wild, Wild Web” to the Warfighter? (Mary Ann Malloy and Rosamaria Morales, MITRE Corporation)
The track at XML 2007 (Boston, 3–5 December) that I often refer to simply as the “Document track” is actually called “Documents and Publishing.” That’s an important distinction, because publishing — even using a text-y format like XML — doesn’t have to involve producing a document of any kind, print or online.
An excellent illustration of that point is Matthew Browning’s and Robin Doran’s presentation, BBC iPlayer Content production: The Evolution of an XML Tool-Chain. Matthew and Robin talk about how, once the team stopped thinking about their information in relational database terms and started thinking in XML terms (RelaxNG, to be specific), it became a lot easier to manage changes and improvements to the project, which lets viewers watch shows they’ve missed from the last seven days of BBC television. It worked so well, in fact, that two other projects became redundant.
That’s not to say that it was necessarily an easy transition, but in the BBC’s case, at least, the rewards far outstripped the cost. That’s precisely the point of our 2007 conference them, “XML in Practice” — we’ve selected many papers that emphasize what did and didn’t work in real, large-scale projects, rather than focusing on new specs and prototypes, to help people make the same kinds of decisions in their own projects. XML won’t always make sense — in some cases, it will never make sense — but learning from others’ experience is a much better guide than speculation.
Earlier postings:
I didn’t have time to look at the OpenSocial API yesterday, so I’m continuing today looking at the data format for the last major area, persistence data.
My first impression of the persistence data API is that it doesn’t belong in v.1 of OpenSocial — unlike the member/friends and activities APIs, it doesn’t seem to be solving a core problem for social-site app writers (I have no way to get at a friends list except through the site’s API, but I can store my own data, thanks). I can see only two reasons that it’s here, neither of them very admirable:
I’ll give Google the benefit of a doubt and assume that it’s a vision thing, but that’s still very unhealthy — specs should solve the real problems of the present, not the speculative problems of the future, especially bare-bones v.1 specs like this.
Now that that’s out of my system, let’s take a look at what you get back from a URL like http://{DOMAIN}/feeds/apps/{appId}/persistence/global (and its many variants). From the spec, here’s what you get when you request a single piece of information from the API:
<entry xmlns='http://www.w3.org/2005/Atom'> <title type="text">somekey</title> <content type="text">somevalue</content> </entry>
Or, in non-XML terms,
$globals{'somekey'} = 'somevalue'
That comes from a URL like http://{DOMAIN}/feeds/apps/{appId}/persistence/global/somekey which requests a single value. Using the first URL mentioned gets you a feed of name=value pairs, sort-of like an associative array:
<?xml version='1.0' encoding='UTF-8'?>
<feed xmlns='http://www.w3.org/2005/Atom'>
<id>http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global</id>
<updated>2007-10-30T20:53:20.086Z</updated>
<title>Persistence</title>
<link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml'
href='http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global'/>
<link rel='http://schemas.google.com/g/2005#post' type='application/atom+xml'
href='http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global'/>
<link rel='self' type='application/atom+xml'
href='http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global'/>
<generator version='1.0' uri='/feeds'>Orkut</generator>
<entry>
<id>http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global/somekey</id>
<title>somekey</title>
<content>somevalue</content>
<link rel='self' type='application/atom+xml'
href='http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global/somekey'/>
<link rel='edit' type='application/atom+xml'
href='http://sandbox.orkut.com:80/feeds/apps/02864641990088926753/persistence/global/somekey'/>
</entry>
</feed>
There’s only one entry in the spec’s example, but there could be a lot more. Basically, this is the equivalent of something like
$globals = { 'somekey' => 'somevalue' }
The comparison isn’t quite fair, because there are also some links explaining what you can do to modify this information, etc., but it still seems like a lot of markup for not much value (pun intended). I wonder if this would be a good place to use JSON instead of Atom+XML? After all, the serious apps will be doing their own data storage anyway, and the client-only apps will probably use a JavaScript API that hides the Atom from the developer.
As hinted at, at least, in my URL posting, there are several different data scopes:
It seems like a reasonable division of scope, especially since the app can’t get anything out that it didn’t put in.
I do believe that, eventually, many web apps will be about to outsource storage as a service instead of having to maintain their own databases and database clusters — in fact, Amazon’s S3 and its competitors already provide precisely this service, though they might not be optimized for a lot of name=value look ups. I’m surprised though, that this could be considered a key feature of a social app spec, when so much else was left out.