Getting SugarSync to work with USB drives on a Mac

IMPORTANT NOTE: SEE WARNING AT END

Cloud based file syncing tools like Dropbox and SugarSync are great ways to share files across multiple computers.  I pay for extra storage on SugarSync, and it is the primary tool I use.  (I have Dropbox at the free level, but mostly for sharing files with other people and with apps that can use it as a sort of lingua franca of syncing).

I have a lot of disk storage in external USB drives, and out of the box, on the Mac, SugarSync refuses to sync external drives.  Since the USB drives are not being used as transient storage, in my case that really doesn't make much sense.

So this is a trick I use to fool SugarSync into syncing them.  It relies on a capability the Mac has in its OS that is not usually understood by users:  Symbolic Links, aka "Make Alias".  Here's how I use it to move a directory off of my hard disk:

• First, I create and sync a directory on my internal disk.  Let's call it "Bar", and it's in the directory "Foo".  
• When SugarSync isn't doing anything (all files are in sync), I quit SugarSync.  
• I copy "Foo" to my external USB drive. (It has to be the parent directory of the directory being synced).
• I rename the "Foo" on my internal drive to "Foo SAVED"
• I right click on "Foo" on my external drive
• Pick "Make Alias" from the pop-up menu.  That creates "Foo Alias"
• Drag "Foo Alias" back to the same directory "Foo" was in on the internal drive
• Rename "Foo Alias" back to "Foo"
• Restart SugarSync

One last thing: SugarSync doesn't detect changes as I edit files.  Instead, I have to restart the app and it picks up all changes that I have made since the last restart.  This works for me where I have a primary computer I do all the work on and just need to snapshot things on a regular, but not continuous, basis.

That's it.  After I am completely confident that the new "Foo/Bar" on my external drive is syncing properly, I will delete "Foo SAVED".

WARNING:

Needless to say, this is not supported by SugarSync.  It could break in any update.  It could just not work for you.  It could delete all the files in the affected directory.  You are on your on with this.  The best I can say is "It works for me, in my configuration, as far as I know".

The Strange Economics of O'Reilly Books

I'm a big fan of the O'Reilly books -- probably because I've been buying computer books for a long, long time and O'Reilly has consistently delivered good products.

But something bothers me about them: the pricing on the O'Reilly web site is not just a little higher, but a lot higher than what's on Amazon.  This applies not just to physical books, but to e-books as well.

The difference is so dramatic that it is better to buy the Kindle version of an e-book and use the $4.99 upgrade  on the O'Reilly web site to upgrade to the full, unlocked version of the e-book than it is to just buy the e-book from them.

I'm sure it's the nature of the business for publishers -- you cannot undercut your retailers -- but it doesn't make a lot of sense on the Internet these days.  Somebody needs to figure out how to solve this. Because until then, Amazon is going to keep getting a piece of my O'Reilly ebook purchases for no good reason -- I'd rather buy the book from the publisher direct and let them use the money to acquire new & improved content.

So much for the Internet and disintermediation...

What could an Apple TV set really do?

I keep seeing reports of the new Apple TV, and they keep telling a story purporting to say why it will be incredible but then only offering mundane details (like this one, The Apple Television Is Coming, And This Is Why It's Going To Be Revolutionary).  And they argue about issues like fees for various packages on cable TV and whether they can unbundle.  Which is a side show.

If Apple really has a revolutionary television set, it needs to offer one feature: It needs to offer advertising, during shows, which is targeted to the user, not to the geography of the cable system or the demographic of all the views in general.  It needs to show me ads that are relevant to me.

Think about all the Google Adsense ads you see around the web.  No matter where you go, you see the same set of advertising, because the ads are aimed at you specifically.

Apple has the opportunity to partner with content creators to offer advertisers the ability to buy specific demographics and they can do so in an auction format like Google uses.  This means that content creators will make more money, cable systems will make more money for their coop advertising, and Apple will take a dime out of every dollar.

All that being said, there needs to be a critical mass of viewers out there to make this worth while.  So the features of the TV have to be compelling.  At the size and prices we're talking about for the TVs in the articles, the buyers aren't really concerned about saving a few bucks on their cable packages that unbundling offers.  So it will have to be something else.  That, I will leave to the rest of the Internet to predict.

But given that the money in this will come from an ecosystem, not just from product sales, it would not be unreasonable to suggest that either (a) the last revision of the Apple TV puck was really getting the hardware ready to do this or, (b) there will be a new Apple TV puck that offers all these features to allow any TV to become more like an Apple TV (perhaps without Siri or some other nice but not essential feature).

If I'm right, you read it here first!

Twitter, meet Pub Sub, or How to be the next Twitter

I've been experimenting with the Twitter API lately, getting a feel for not only its published capabilities but the nuances of what you can achieve with it.  If you want to see where I've gone with that, feel free to drop in on socialseer.com from time to time -- as I learn new tricks or come up with new ideas, the web site may go in a completely different direction.

Most of the time, I think about the humans at the other end of the tweet.  Where are they? Are they male or female? Are they happy or sad? Who are they talking about? What are they talking about?  These are all interesting questions, and interesting questions to try to get a computer to answer.  I have code that can answer some of these precisely, guess at some of them, and some are still beyond my grasp.

But even with incomplete and sometimes questionably accurate information, a wealth of analysis can be done.  You can see a blog post here about the reverberations of Rush Limbaugh's "slut" comment and, again, my socialseer.com website has many other examples.

Sometimes, one of my posts will get a lot of hits -- mostly when the topic is on something controversial or in the news.  And perhaps it's a bit of vanity, but I try to cater to that in the posts.  Deep down, though, I'm really looking at how I can extract more interesting information from Twitter and how that information can be used. That probably puts me in a very distinct and small minority -- I don't care about the events as much as how the events can be observed through the Twitter lens.

If you're with me so far, we're about to go a bit deeper and nerdier...

The starting point for this analysis is a mechanical access to the information in Twitter.  Twitter offers a reasonably decent API (reasonably, because it's full of quirks, inconsistencies, strange failures, and limits you have to deal with).  And for the most part, they offer it to any or all to play with.  That's kind and generous of them, and I am grateful to them for that.

At the lowest level, Twitter turns out to be a (quirky, etc.) implementation of what software engineers call "publish and subscribe" -- just normally done by humans.  I write a Tweet and send it, and those of you who follow me on Twitter will see it in your timeline.

It turns in this case, Bianca is not interested in just any Tom, Dick, or Harry

Whether you read the Tweet or not is another question, if you have, say, Tweetdeck running my Tweet shows up on in your open application; message received as far as the software is concerned.

Twitter also allows me to publish messages and to tag them (via hashcodes) for others to subscribe to, although that mechanism is much weaker (there is no formal following of hashcodes in Twitter for example, but must applications allow you to open a persistent and updating search).

Net net, Twitter is using classic approach to integration between software applications and adapting it for communication between humans.

You're saying, at this point, OK I concede you're right, but so what?

One so what is that it gives us a different way to look at Twitter and describe its function a bit more rigorously.  That, I realize, is not a great benefit.

The other so what is that it gives an insight into what's missing from Twitter or what else you could do with it.  So what ( :-) ) are those insights?

Let's start with a kind of hack which is being used to enrich the content of Tweets, such as how a lot of Twitter clients show pictures that are referenced by URLs in the tweet.  The picture is not really part of the Tweet, but it appears to the recipient to be part of it.

Twitter on the Mac opens a small window with
the photo  from a Tweet without bothering you with the mechanics of where the photo's stored

The problem with this kind of solution is two-fold.  First, it is implemented in an informal and ad-hoc way.  Twitpic works, but if I want to build my own sort of Twitpic, it probably won't work.  Second, the picture is completely independent of the core Twitter system, so it could be deleted or modified in some fashion after the Tweet is sent.  Probably with Twitpic you cannot do that, but I bet some enterprising hacker is looking for a way to send a Tweet out with a lolcat, get it retweeted a million times, and then replace the picture with an advert for credit cards.

Still with me? Wondering why?  Ah, well, I'm finally getting around to my real point now...

Even with the current Twitter  we could -- albeit in a way that violates the TOS -- use it to communicate things other than (im)pure English (or German or Chinese or).  We could, for example, send complete messages structured for computers to consume, not humans, at least not directly.

Let me give you a simple example.  Suppose you're a band and you want to tell your fans where and when your next concert is.  You could send a snippet of <140 character prose "we'll be at Bob's Bar in Orlando next Friday at 8pm".  You could send something with greater detail: "we'll be at Bob's Bar, 1234 John Young Pkwy, Orlando 39999 at 8:00pm through 10:00pm".  Or you could send the kind of calendar invite many of us send around in email.  Twitter could display the invite in human readable format, a map of the location, and the date and time in a way that when we click it, it could go right into our calendar.

More examples beyond that -- well, start thinking -- there's a business model for a start up in a lot of them!

But even beyond that, you could (well, I could) imagine a new Twitter, one which is not meant for direct human consumption at all.  One that is purely meant for communication between applications.  You might still limit the message size, but might allow something a bit more reasonable -- say 1K -- that would allow for a decent sized JSON payload.  As a business model, you could sell access -- say a small per message fee to publish, but none to subscribe -- with perhaps a free tier of a limited amount of messages (100 per day?) per user to encourage adoption.  Or charge people for messages above a certain size.  Some messages could be ultimately intended for direct human consumption -- activity stream kind of things -- but some could be just for communication between applications.

The key thing would be that if we all agreed to use this system, it wouldn't matter what our underlying applications were.  And so long as each message type had a uniquely identifying marker, a million different message types could coexist without problem.  In sense, it's like the notion of routing web service calls over messaging middleware, something that was incorporated in things like the SOAP specifications but just about never seen in the wild.

Some obvious categories of messages:

• Document update notifications: I would always have the most recent version of a presentation, or a contract, or a PDF.  This would be a much more refined way of doing sharing than, for example, shared folders in Dropbox, and a much more independent mechanism than Google Docs.
• Schedule updates: It's just painful to receive an email every time a meeting gets changed, especially when it's a useless message like "adding Bob".  If we're all on the same calendaring system, we can do this now, but when we aren't, the routing of meeting invites and updates via email gets messy.
• Bridging activity streams: People are trying to build walled gardens around activity streams, like Salesforce's Chatter.  Either you're in (and pay) or your out.  But if I don't know my subscribers, how would I know what system they are using and reach them there?
• Software updates: you could have a generic demon that subscribes to update notifications from your application vendors, and then launches the application to update.  It's annoying, on the Mac, that Apple has its updater, Microsoft has it updater, Adobe has its -- and they all want to run all the time on their own.

What else? Again, it's up to your imagination to decide.  And your business accumen to bring to market.

The key is to have a single pub-sub service like a Twitter that's tuned for mechanical consumption.  If you like the idea, and think we should give it a go, maybe something small & open source, let me know at charles@mcguinness.us.  If you want to just take this idea and run with it, go for it.  All I ask is for friends and family shares when you go public :-) You never know, something like this could be ten times larger than Twitter in the apps world we live in.

A Day in the life of Rush Limbaugh on Twitter

Rush Limbaugh's been a pretty hot topic lately, and he's certainly been a popular discussion on Social Media.

In the 24 hours preceding 8pm EDT (which is midnight, GMT), there's been roughly 14,000 tweets that include "limbaugh" in the contents, or about 10 a minute.  Keep in mind that it's a weekend, and Rush hasn't said anything on the air since Friday ...


Sentiment

Some breakdown of the sentiment in those tweets (click on chart to enlarge):

52% of the tweets registered as weakly or strongly negative, while 21% registered as weakly or strongly positive.  That's no surprise.  No matter who's talking about Rush or their position, they're probably not happy about what's going on. See the comments at the end about limits of sentiment analysis.


Hash Tags

One thing that is always interesting is the use of hash tags (#winning).  These are unstructured and uncontrolled, and so it is purely convention that is adopted by Twitter users.  For the posts that talk about Rush, the following are the top twenty hashtags:

HashTag	Count	Percent
#p2	824	9%
#limbaugh	585	6%
#stoprush	583	6%
#tcot	544	6%
#boycottrush	536	6%
#gop	262	3%
#taxpayerfunded	245	3%
#flushrush	232	3%
#rush	194	2%
#snl	186	2%
#waronwomen	182	2%
#gamechange	158	2%
#cnn	147	2%
#fem2	126	1%
#tlot	109	1%
#topprog	107	1%
#ows	102	1%
#rushlimbaugh	94	1%
#teaparty	93	1%
#news	82	1%

#p2 is the has tag for "Progressives on Twitter".  I was surprised to see it as the most popular tag.  #tcot is "Top Conservatives on Twitter". #fem2 is for feminists. #tlot is "Top Libertarians on Twitter".  What's interesting is that there is no hashtag which is reaching critical mass.  If you were to search for just #stoprush, for example, you would get only a tiny fraction of the posts about Limbaugh.

Twitter Users

There are no clear "top posters"; the most frequent poster is "Miaminonymous", who appears to just retweet everything, with 131 posts.

The top 25 people mentioned in tweets are:

User ID	count	% of Mentions
@thinkprogress	480	4.0%
@hipstermermaid	320	2.7%
@limbaugh	310	2.6%
@huffingtonpost	191	1.6%
@credomobile	182	1.5%
@politico	167	1.4%
@addthis	158	1.3%
@shoq	153	1.3%
@superguts	152	1.3%
@denisleary	152	1.3%
@billmaher	133	1.1%
@youtube	131	1.1%
@politicususa	121	1.0%
@tmorello	118	1.0%
@theblaze	117	1.0%
@cdibona	114	1.0%
@mediaite	112	0.9%
@anonyops	112	0.9%
@sandrafluke	102	0.9%
@thedailybeast	95	0.8%
@rushlimbaugh	92	0.8%
@krystalball1	80	0.7%
@boingboing	76	0.6%
@stoprush	70	0.6%
@thedailyedge	70	0.6%

Interestingly, @limbaugh is not the twitter account Rush uses, @rushlimbaugh is.  


Comments

What I make of this

Rush Limbaugh is still a hot topic in social media.  I would expect that tomorrow, Monday, if there is significant news related to Rush (more advertisers pull out or he says something controversial) we will see a spike in topics.  On the other hand, if something else rises to the top of the news cycle, we may see Rush take a breather on Twitter.  It is interesting that the conversation is still scattered -- huge numbers of disconnected users and hash tags.   There is not one conversation taking place; there are thousands.

Limits on Sentiment Analysis


Sentiment analysis is a mechanical assessment of the sentiment, positive or negative, in a tweet.  It does not necessarily indicate approval (if positive) or disapproval (if negative) of a particular subject.  Consider some contrived examples: "I hate the constant criticism of Rush" is negative, while "I am so happy that Rush is losing advertisers.  I love the ones who are quitting" is very positive.  The sentiment generally tells us whether the statement is happy and upbeat or negative and downbeat.  In large numbers, it is a crude assessment of a topic like I am using it in this post.

Rush Fire

It is tempting to look at the recent dust up around Rush Limbaugh’s insulting name-calling of a law school student as merely another instance in a right vs. left battle in this country. Many times, commentators will mention that people on the left are equally capable of inflammatory comments as they go on to skewer Rush.  Some on the right, sensing it is a partisan issue, have defended Rush, even some women.

Although I have a strong opinion on the subject from both a political as well as a propriety view, there's something that a lot of people are missing about this incident: the rising political power of social media and, just as importantly, the dominating use of social media by women.

In a nut shell, Rush used fighting words against a woman, and women are fighting back and winning.

Proflowers Facebook page

~6K Likes, ~8K comments

And the tools they are using are social media.  But, strangely, the traditional media is blind to that.  Take the first article I linked to in this blog post, from "The Week".  Where does it mention the intense conversation taking place on Facebook? The websites devoted to tracking the advertisers to Rush's show and barraging them with demands to stop advertising?  Nowhere.  If you were to just read the article, you would think that the advertisers acted out of moral indignity, not in response to pressure from the (female) public.

Nearly 300,000 people Liked Planned Parenthood,

about a third of those who like Rush.  You might be

tempted to think the Social Media audience is male

and conservative.  That is a dangerous assumption.

But let's look at the ways social media worked to rally a response to Rush Limbaugh.  First is the obvious channel of Facebook.  There are all sorts of friend to friend sharing of outrage, although it's hard for anyone to see much of that given the general privacy settings on Facebook.  But we can see some of the effects nonetheless.

Rush is getting a lot of traffic on his Facebook page, pro and con:

Nearly 20k comments on Rush's Page

And there's a whole bunch of Boycott Rush groups on Facebook, including this one:

This is pushing people to write to advertisers, asking them to drop support for Rush's show.  For example, take a look at Lifelock's page on Facebook:

And 2000 more on another post

You might say this is a tempest in a Facebook teapot, but have a look at what's going on out on the Internet. One example is "boycottrush.org"which redirects to "leftaction.com".  Let's look at its traffic (courtesy of Alexa):

It has gone from being nothing to being somewhere around the 14,000th most visited site on the web.  That seems insignificant until you drill down and discover that about 1.5 million visitors have stopped by this page since the start of the Rush fire. (see footnotes)  This is a lot of visitors.

And these visitors are mostly female and aged 45 and up.  These are not the people most associate with savvy internet users, but that assumption is clearly wrong (as an aside, these are the people who tend to vote).  Perhaps not too surprisingly, many of these are women who are veterans of the women's liberation movement.  They obviously haven't lost their fervor, and they're taking it online.

This group is well organized, maintaining a web-collaboration based spreadsheet in of advertisers (former and current) with contact information which they are keeping current with every show:

And Rush has been a popular topic on Twitter all of a sudden, to no surprise:

Sadly, there is no sentiment tracking that would help us differentiate the pro-rush vs. anti-rush tweets.  But given the other activity on the internet, it's safe assumption that a lot of it is anti.

I also find it amusing that there seems to be a bit of a gender divide in the analysis of what's going on. My observation is that male reporters seem more blind to the social media power that is being brought to bear and are quicker to focus on the partisan or free-speech aspects of the situation rather than the misogyny that women respond to.  It would appear that if it's not your ox that's being gored it's easier to be dispassionate.

Coming on the heels of the Susan G. Komen situation, it's clear that women are starting to find that they have a real power they can exercise through social media.  (For us men, it would be best if we pay attention to it!)

You might think that this topic is far afield from the core of enterprise technology, but keep this in mind.  Rush Limbaugh is not just a man, but a very large business undertaking.  It is easy to say it's just him, but the problems that cropped up have affected his business and his scores of partners.  Any business which is  in the public eye can have a problem like this.

Rush himself seems to have gone days before he noticed the firestorm he started, and as a result made a situation much worse before he issued his apology.  Had he been more aware, and apologized sooner, it would have better diffused the situation.  It seems to me that a company (and Rush is a company) that notices a mistake quickly and reacts to it looks much better than one that lets things drag on.  The former seems like a company that cares, while the latter seems only like a company that didn't realize it had been caught until too late.

The advertisers have adopted a different strategy, which is to lay low mostly.  Regardless of whether they stay or go, their communications tend to be as succinct and under the radar as possible.  Everyone knows the risks of offending one side or the other, and many seem intent on saying as little as possible, probably hoping it will all blow over eventually.  Whether you can hide in this era of social media is an open question; perhaps for the partners they can, but certainly Rush cannot.

In the end, neither Rush nor most of his advertisers were attune to the signals that were coming from the social media. Your CEO may not call a college student a slut in front of millions of people, but it's just as easy to unwittingly commit some other faux pas that triggers a massive response.  And if you are an advertiser, you have to contend with the perception that your advertising is an endorsement of the person you are sponsoring; now you have to worry about partner reputations as well.

To be fair, most uninvolved people (read: men) didn't pay a whole lot of attention to social media initially.  But the situation shows that businesses cannot afford to turn a deaf ear to the voices being raised on the internet. The only questions left now are how to listen and how to respond.


Footnotes:

Estimate of 1.5 million visitors to leftaction.com:

http://www.internetworldstats.com/emarketing.htm estimates total internet population at 2.3 billion users 

http://www.alexa.com/siteinfo/leftaction.com# estimates daily traffic since the start of the Rush fire on average of .014% of the total internet population.  This also gives demographics of the visitors. 

That works out to about 300K daily visitors, or 1.5 million for the 5 days of the Rush fire tracked by Alexa at the time of this writing.

Windows 8 First Impressions

I've downloaded the Windows 8 Consumer Preview, and have some initial thoughts to share about Win 8 and the new Metro interface. I'll start by saying why it's hard for me, or any one really, to do a decent job of a review, and why we should all be skeptical of any review at this point.  Then I'll move on to trying to ignore my own advice in the previous sentence...

So let's start ...

When you look at a piece of software, you have to envision living with it.  You want to do your work, play your games, surf the net, whatever it is you do with a computer.  And when you have a new, and incomplete, version of software, you can't do that for real.  And so you guess. At least I do.  And odds are I'll guess wrong.  Worse, we all have our biases.  In my case, I was a loyal windows users right up to Windows XP.  After that, we started to switch to Macs at our house.  The last version of Windows we bought was Vista, which was a disaster.  After that, we pretty much are all Mac.

I still use Windows on a daily basis, but it has been demoted to a virtual machine to be summoned for running windows only software.  So, as you can see, I'm not your best windows fanboy to review Metro and Windows 8.

Still, I was excited enough by what I saw to download and install it -- so I'm willing to let it win me over.

So let's have a look at Windows 8...

The key thing about Windows 8 is the Metro UI, as you see it in the first screen shot.  The idea is to move to a more tablet like experience.  And the Metro UI does that, with its large square, easy to hit with a finger icons.  Of course, only a subset of the apps are there, but a right mouse click brings up an icon to see all your applications:

Which leads to:

Not quite the old start menu, but not really far from it.

Metro Apps like to run full screen:

Which is very tablet like.

But you can still run traditional windows (small w) applications as well

Interestingly, the windows desktop is, in some sense, a full screen Metro application.  It is treated like that.  So we you go to switch tasks, the desktop appears as a whole:

Except when you alt-tab, then the desktop programs are distinct:

Looking at it from the viewpoint of a desktop system, there's a couple of things that are interesting.  Both Apple and Microsoft seem enamored of the full screen application.  It's taken from the tablet paradigm for sure.  And, I suppose if you have a smallish laptop display, full screen is useful in situations.

What's amusing (or alarming, you pick) is that this move to an app at a time goes against the entire development history of Microsoft.  MS/DOS was, of course, an app at a time full-screen operating system!  And yet very quickly, as memory capacity grew on PCs, people started trying to find ways to multi-task.  At first there were a variety of add-on multi-taskers to MS/DOS.  Eventually, Microsoft Windows came out, although the first versions were not that good at multi-tasking.

Then, as multi-tasking became more reliable, the holy grail for Microsoft was desktop integration.  You probably don't remember a time when you couldn't copy from Excel and past into Word, but trust me, that was a big deal when it happened.  The idea that you could have a spreadsheet and word processor running at the same time and seamlessly move data back and forth was amazing.  You may want to think that Microsoft's monopolistic tendencies crushed its office competitors like Lotus and Wordperfect, but it was that little trick of integration that made Office dominant.

With Metro, we're not really throwing that away, but it feels like it.  Think about how you would insert a small table you built in Excel into a PowerPoint presentation: you would have two windows open, PowerPoint and Excel.  You'd select the table in Excel, copy it, click on the PowerPoint window, and past.  You do that sort of thing all the time without thinking about it.

But with Metro, assuming you had a Metro Word and Metro Powerpoint, the navigation between applications is more disjoint.  You have to either execute weird mouse moves or alt-tab to get between the applications.  It won't seem smooth.

But, you might ask, what if you aren't really running multiple applications at once? What if you are just logged into Facebook and whiling away the hours?  Well, I admit, Metro is probably great for that.

And that leads to my observation: Metro -- like iOS to be fair -- is designed for the consumption of content, not the creation.  It's like there's two worlds of Windows 8: The creation side (traditional windowing usage) and the consumption side (Metro).  That is a strange dichotomy.

To me, it feels like Microsoft glued two operation systems together -- Windows 8 desktop and Metro tablet, and decided to call it a single operating system.  This is very Microsoft like; when Windows 3.x came out, you had to start MS/DOS and then boot Windows, but you always knew that you had an MS/DOS world (to run your legacy DOS applications) and a Windows world.  It's the same thing, just twenty years later: You have a legacy Windows world and a new Metro world.

If you compare this to how Apple is approaching the integration of iOS and OSX (no longer Mac OSX as of Mountain Lion), there is no separate world of "OSX" apps and "iOS" apps on the desktop.  You can launch and manage the same applications in both the traditional way as well as via the new iOS-like ways.  You can run the same applications in a windowed mode, or in full screen.  The power is in the hands of the user; the transition is up to the user to make or not make at his or her own speed.

I had hoped to find Metro and Windows to my liking.  I had hoped to find something new and useful and a real advance of the desktop paradigm.  I had hoped to find a reason to want to bring Windows back into my daily life beyond just for a few Windows only applications.  But what I find is a forced, awkward, and disconnected experience.  Here's hoping that it improves -- a lot -- by GA.

Real numbers for smart phones

I read a lot about mobile platforms, and there seems to be a constant battle for bragging rights between the iOS and Android platforms.   There's a lot of talk about how activation rates for Android phones are so much higher than for iOS, and therefore that iOS devices have a relatively smaller market share.

The follow on argument from the Apple boosters is that, well, Apple is smaller but they have all the profits.

But is Apple really the smaller vendor?  I was browsing through a report from the Pew Research center (see the report here), and it said that of smart phone users surveyed:

• 20% of cell owners now describe their phone as an Android device, up from 15% in May 2011
• 19% of cell owners now describe their phone as an iPhone, up from 10% in May 2011
• 6% of cell owners now describe the phone as a Blackberry, down from 10% in May 2011

Statistically, then, Android and iOS are tied. (Note that 45% of all cell phone owners say they own a smartphone, so there's nobody missing from those percentages).

What's even more interesting is that Android users are up 5% in about a year, but iPhone users are up 9%.  That seems terribly at odds with the conventional wisdom about adoption (like at business insider).

I have a couple of thoughts on the subject:

• iOS has a good head start
• Apple has a stronger support for keeping 1 and 2 generation old devices up to date with software.  If you have a 3GS, you are still able to get iOS 5.  Android, on the other hand, tends to orphan devices fairly quickly.  So people may stay on an iOS device longer than people on Android devices, resulting in a longer in-service life for iPhones.
• Similarly, the fact that a few years old iPhone is still able to run current software means that it has value in the used market.  An old and out of date Android device is not going to look attractive compared to getting a new one from the carrier.
• How many people get an Android and switch to iPhone later?

Because of these, I suspect there's a lot of retired yet fairly modern Android phones out there.  And that's why a random survey of what people actually use shows the iPhone still holding even with Android.

Recovering from a hacked wordpress website

It was a dark and stormy night ...

The infection bootstrap file

It was late last night, and I wanted to make a slight change to one of my websites before heading to bed.  But when I brought up the home page of my site, I was startled when the normal front page was replaced by a glaring red Google warning that my site was serving up malware!

So much for going to bed anytime soon.  I've had this happen to me before, years ago: a long stagnant and nearly forgotten web site of mine used a third party plug-in which got hacked and pushed me onto the Google blacklist.  That was no fun to undo, and I wanted to clean up the current mess quickly enough to avoid being branded a malware site.

As a bit of background, the website in question runs on Dreamhost.com's hosting service and is built with Wordpress.  I've been a Dreamhost customer for about 10 years now and, while they've probably had more than their fair share of problems in the past, the combination of features, flexibility and price have kept me a reasonably loyal customer.

I SSH'd into my account to look around.  I discovered that the hack, as far as hacks go, was pretty simple.  Just about every PHP file (as far as I know, every single PHP file) on my account had one line of script added to it.  This script redirected people's browsers to a malware site.  That was interesting, because it meant that somebody was able to sweep through every file I owned and rewrite it.  It wasn't a site-specific attack -- all the sites I hosted, even a bunch of dead ones, were affected.  Somebody was able to either (a) run code on my account (which seemed most likely case -- that's how I'd do it), perhaps by having the viral payload in each PHP file try to infect other files on my account or, (b) had some FTP based program that kept running through and fetching files to infect them (seems unlikely and harder to implement).

As an aside, can I just say how much I hate PHP?  It's too-stupid to see a variable without a superfluous dollar sign.  It has a bastardized object model.  And it's got a security model that says as long as the file ends in .php  it gets run as code.  Code signing? Meta files to separate executable content from data? Nope.  If you can figure out how to upload a rogue .php file to a web site, you can run it with full permissions on the server.  That's just a constant invitation to disaster.  Like the one that happened to me.

Anyway, there I was, staring at a massive corrupting of all my web sites that used PHP.  The only thing that appeared to save me is the infection was additive rather than replacing code -- it just stuck a line of code at the beginning of each PHP file:

What got put in front of every PHP file

Still, I had two big tasks ahead: (1) stop any future damage and (2) remove all the current damage.  Mind you, it was not phrased like that in my mind.  It was more like "oh shit I got to fix this fast and figure out how to stop it from happening again." Neither turned out to be real fast, unfortunately.

Let me start with the second task, undoing the damage.  The damage consists of prepending the PHP files with an encoded script, most often in base-64 but sometimes just using hex escapes to obscure the evil logic.  I'm not sure why they bothered, though, other than to look cool. It sure did stand out when I looked: a more stealthy injection would be harder to find intermingled with all the other code in the PHP files.  But the consistent nature of the hack made it easy to spot.  So easy, in fact, somebody has written a fixit script to remove the changes.  So backing out the changes was no problem. (note that it removed the infection from my files, but did not find and remove the additional files placed in my web site.  Read below about those).

The first task was the real challenge and what took all the time.  I removed all the hacks to my files, but within a half hour they were all back again.  This meant that the infection process was still underway or had been retriggered. Not pleasant.

The term "fog of war" come to mind describing my state.  Somehow, I was able to uninflect files only to see them re-infected, but I had no idea how it was being done.  The following possibilities seemed to exist:

• Someone still able to log into my account
• Someone had super user access to the server
• Wordpress had some sort of flaw in it that allowed people to upload or modify PHP files
• Wordpress had some sort of flaw that allowed people to modify the database where they could insert code there (that would then infect the files)
• I missed some infected file that was being run and reinfected my sites
• Something else?? Some other hack that I had closed but leaked a user id or password to the server or database??

Immediately I took all my web sites off line -- I simply moved the contents of the websites into a separate directory and put a place holder index.html file into the root directory of each web site.  That bought me time.

I started by thinking it might be some sort of wordpress exploit.  I had turned on allowing people to register user ids in one of my blogs shortly before the mess had happened, so I guessed it might be some sort of SQL injection problem.  I proceeded to export my posts from wordpress, and then did a scorched earth reinstall.  I then re-uploaded my posts, rebuilt my customizations to the templates and put the web site back online.

And a half hour later, it had been hacked again.  Ok, that wasn't it.  Maybe an initial exploit opened the door for a subsequent one.  Or maybe it's that my hosting provider, Dreamhost, had been penetrated themselves and my assailant had root access. Or? It was still foggy.

To close the door on the first one, I went through a round of password resetting.  I changed the FTP/SSH password.  I changed the database password.  And then I cleaned the hacked PHP files and reopened the web site.

Again, before too long, the PHP files were getting hacked yet again.  This was frustrating.

I was starting to think that my hosting provider was hacked and there was nothing I could do. But I tried one more thing in desperation.  If I had my own server (say Amazon EC2), I would be able to exert a lot of control over the virtual machine -- reboot it, for example.  But with a shared hosting provider, I cannot.  But there's a small, almost hidden command in the control panel for Dreamhost websites, which terminates all existing FTP connections: what if someone had kept open an FTP connection? It seemed crazy, but I had all connections terminated.  I also went into my sites and had all existing cookies invalidated, so someone who had a login to my web sites would be forced off.  (If you're a wordpress user, look at the file wp-config.php and the section about changing the salts).

After that, I went through another round of changing passwords from the database on down.  I'm a Mac person, and the program 1password is invaluable in generating and maintaining a set of secure passwords.

I finally finished up very late last night, and today it appears that it is holding.

But there's one more thing ...

This morning, the hosting provider, Dreamhost, provided me a list of potentially hacked files in my account.  There were a whole bunch of other infected files I had missed, but whose infection patterns looked different. The first set -- the set I had seen and cleaned -- started with a PHP eval statement, but the second set used a bunch of hex escapes (see the first illustration in this entry).  My searches using grep didn't look for them, and the fixit script didn't see them either.  So I just deleted them all.  They were not infected files, they were the starter files to initiate an infection.  (They were all named either r.php, rr.php, or footlib.php -- the last of which was a small boot strap program that allowed someone to peruse the files in my account via the web.  It appeared to be the source of the initial infection).

I'm going to stay on top of my web sites for a while until I'm comfortable the threat has passed.  I'm not sure how the infection got in, so I'm not sure what to look out for.  Dreamhost had been hacked into itself in January, and I'm thinking that an infection script was put into one of my web sites where it laid dormant until last night.

All this is no fun.  If I wasn't familiar with Linux shell commands and PHP, I wouldn't know what to do. I guess host my blogs at Tumblr?  On the plus side, I've learned a lot about how wordpress works.  And did I mention how much I dislike PHP?

Molly Wood goes all Apple ...

I heard this in a pod cast, and I thought it was interesting.  Molly Wood of CNet has burned through several Android phones and found them each to fail her in one way or another.  So now she's gone to an iPhone 4S, adding it to her MacBook Air, iPad, and iPod.

Why I love the new (beta) messages program from Apple

If you're an all apple family like we are, the introduction of the iMessages protocol was great.  That protocol allows text messages get sent over the internet instead of SMS.

The new Messages program fills an important gap -- the ability to send and receive messages on a laptop/desktop computer.  For most people, this is a nice to have, but for me, this is fantastic.  Because I'm on planes a lot, there's a lot of time where my phone is not connected to anything.  But since all planes (at least all Delta planes) offer WiFi, I am on my MacBook or iPad constantly during a flight.  And if I want to text with my wife while I'm up in the air, now I can.

OK, maybe some of you won't think this is a great advancement, but for me, it's a real plus!

The dubious logic of Apple's iBooks textbooks

I keep looking at Apple's announcements about ePub textbooks, and I just can't figure out how it works.

Sure technically, I can, but from a business perspective, it's really hard to see how this will take off.

Let's see if I have the story right so far:

• Apple is announcing a new iBook 2 application that has more multi-media and embedded logic capabilities, so textbooks can come alive for the readers.
  
  Bringing a more app-like experience to books is a great thing.
  
  
• Apple has released a free authoring program to support creation of this material.
  
  Good authoring tools are important.  When I see that the publishing format is HTML5, I think that there's a big dearth of good HTML5 authoring tools.  So anything that adds to the mix is great.
  
  
• Apple is insists that the books are priced $15 or less.  To support this, electronic textbooks, unlike physical ones, will be owned by the student and cannot be given away, transfered, or re-used.
  
  The idea is something like noting that a paper textbook has lifespan of a few years, but costs $75, so a $15 eBook that has to be re-bought every year is the same as a 5 year life-span on your paper text book.  Except that the eBook is always up to date, whereas your 4 year-old textbook may be out of date.  That seems like a better book at the same price.
  
  
• Apple insists that books be sold only through their store.
  
  They built the tools, they should benefit from them.  No problem there.

On the surface, then this all seems great.  Better books at the same cost, and we save a forest or two in the process. 

But under the surface, there are some real problems with this.

First, it's definitely an "I'm from the vendor and you can save money by buying my newest kit" kind of proposal.  True, the iPad is a wonderful device. But the lifespan of an iPad is probably the same as a textbook or less, especially in the hands of the K-12 market.  That $500 is the cost of 5 to 7 textbooks (given the $75 price point McGraw-Hill used as a comparison).  So is this still going to be the same cost as paper books?

Let's do some math.

Imagine that there are 5 textbooks needed for any given grade. ( I'm kind of far removed from K-12, so forgive me if they need only 2 or 10.) That's $375 of textbooks, but they last on average 5 years, so that's $75 a year per student for paper textbook purchases.  That's one side of the equation: $75/PSPY (per student per year).

On the other side, the iPad costs $500.  I would expect the average life to be about 4 years at best (either because the device gets destroyed, the battery dies, or it no longer runs the latest iOS and becomes useless), and you can figure on a $100 repair midstream (new battery, cracked glass, etc.), so that's $150 per student per year for iPads.  And, you still have to buy 5 text books at $15 each, so you're back to $75 per student per year for content.  Grand total is now $225 per student per year, or a 200% price increase to go eTextbooks.   That's the other side: $225 PSPY 

Since $75 does not equal, $225, where is that extra $150 going to come from?

It's easy to predict what will happen.  Either (a) A school will get a grant to buy a bunch of iPads and then not have money to repair or replace them over time and they will be abandoned, or (b) the costs of this will be dumped on the parents -- you can choose the eBook track of instruction if you keep your child equipped with an iPad, or you can choose the paper book track if you do not.  That's just going to further divide the haves from the haves not.  Buy your kid an iPad, they can go to the iPad classes.

The situation would be a lot easier with Kindle Fires or Nooks, since these devices are substantially cheaper.  But Apple doesn't sell cheap kit.  Perhaps, in the upcoming iPad 3 announcement, they announce that they're going to keep selling the iPad 2 but at a reduced price, say $300.  You still end up with an extra $100 PSPY in costs that have to be eaten by somebody. (Note that the iPod touch is $200, so if they try to move the iPad 2 below $300 they have to start squishing the iPod prices too).

Next, let's talk about content.  

Other than a few showcase efforts, how committed are textbook manufacturers going to be in creating spiffy interactive content for their textbooks when they still have to create dead-tree versions of the same books?  It would be nice to think the market will drive them towards richer content, but the decisions in textbooks are political, and I have to think publishers will turn their eBooks into shovelware.

Put another way, textbook publishers can already publish their textbooks as iPad apps in the app store.  Where are they with that?  Nowhere.  So a slightly better editing program and a PR campaign is going to change that?

I hate to think that I'm the guy saying that iTunes music sales aren't going to do anything at the dawn of the iPod.  But there are a lot of differences here.  First, the record industry already had its content in digital format (from CDs), so the conversion to iTunes format was completely automated.  Second, there is no real difference in the experience delivered by the downloadable music.  It's not like they all had to gain video or games or dorian mode explorer software. It was the same product as before.  No one was disappointed to discover the downloads had roughly the same quality and features as the previous format.

Finally, there's a philosophical issue.  

The approach as I understand it is fundamentally at odds with the new cloud-based world.  If you use dropbox or similar -- or even iCloud -- you're used to the idea that wherever you are, so is your data.  I read Amazon Kindle books.  The same book is on my iPad, my laptop -- heck, even my phone.  And as I make progress through it on one device, the rest are in sync.  If I am at my desk, I would much rather read a book on my 23 inch monitor than on my iPad.  This is how cloud works.  Apple knows this: see the commercials for iCloud? Change my presentation in one place, and it's updated the other.  Take pictures here, see them there.  But this is not how the new text books work -- you have to read it on the iPad only.  And it's locked to that iPad.

It's this last issue that makes this a real head-scratcher.  It's like the last 5 years or so of technology advances hasn't happened.

If the goal of Apple is to start to introduce electronic textbooks into the marketplace and see if they can make for better education, that's great:  it's a noble experiment.  But if the expectation is that this is going to have a radical impact soon, I think that is not realistic.  Look how long and hard Bill Gates has tried to make education better.  And that's with him throwing tons of his money at the problem. As much as I respect Steve Jobs, this is not a an area that lends itself to easy solutions.  If it did, our schools would be the best in the world.

A year from now, I expect this will all be moved to the back burner as Apple tries to figure out what went wrong.  Or they will realize the real market for this is college, where you can insist on students having an iPad -- and they find success there.  We'll see...

It was an extremely hefty bill for a drink from Apple. What I did about it.

Over the holidays, a bunch of us were gathered around the table, laptops open, enjoying the holidays when a drink spilt into my keyboard.  I cannot tell a lie, it was my fault.

After doing the customary recovery attempts -- turn computer off, flip upside down, let drain and dry over night, I discovered my computer still worked fine, but had developed a few keyboard quirks.  A few keys didn't type at all, others would generate two key presses, and some keys just kind of auto repeated endlessly. To see if the damage was limited to the keyboard,  I plugged an external keyboard into it and it worked fine.  The computer still ran, my iPhone still synced to iTunes.  It all ran just fine.

With a resigned heart, I knew the keyboard needed to be replaced.  Were this a desktop, it would be simple -- unplug old keyboard, plug new keyboard in. Kick yourself for being clumsy, but it's something you can fix yourself. But with a laptop, it is not so simple, and so I knew my best course of action was to take it to the Genius Bar in Altamonte Springs, FL.

I visited the Apple store a few days later, and awaited my appointment with a "Genius".  I explained the situation, and his first comment was "this is going to be expensive."  I knew a new keyboard was going to be expensive, and I was prepared for that.  I had priced them online, and the going price was around $250.  But I had just missed some important foreshadowing.  As we went over the situation, he commented about if the logic board was damaged it, alone, was going to be $800.  Of course, I knew the logic board was working fine, so I ignored the comment.  I had tested the computer with an external keyboard and it all worked.

He took the laptop in the back and was gone so long I assumed he had gone on break in the interim.  My level of anxiety was growing.  I went over his comment in my mind -- why did he assume, without any knowledge, that the logic board needed to be replaced? Is this a problem with the design of the Airs -- anything spilt on the keyboard goes right to the logic board? That would be bad if it were true.

When he finally returned, he told me he had bad news: both the keyboard and yes the logic board needed to be replaced.

I stared at him.

"You're kidding me," I told him, "the logic board is working fine.  It's just the keyboard that's not working."

He assured me that the logic board was covered in goo and it was only a matter of time before it was dead.

I challenged him: but it's obviously working now!  His response was that the goo on the logic board could, in time, do further damage (something about acids) and that keyboard problems pointed to logic board damage.  I did not believe him.

I asked him, point blank, if, even understanding what he said, I still wished for them only to replace the keyboard: would they do that?

No.  Logic board replacement or nothing.

At that point, a scenario flashed in my mind.  It's a car repair scenario, where the shady auto mechanic tells you that you need to fix things in your car that are not broken just so they can charge you a higher bill. In my mind, Apple had just become the shady auto mechanic, asking to replace a pricey part that gave no signs of being broken.  And to be clear, they had offered no proof that it was broken.  Just the word of some anonymous tech in the back; the computer had be returned to me completely closed back up.  I was at a disadvantage, because I could not inspect the logic board myself.

I took stock of the situation.  The computer was working fine, save for the keyboard.  They were offering me a my-way-or-the-highway expensive repair that they asked me to accept, on faith, contrary to the facts that were evident to me, was absolutely needed.

I came to one opinion:  I was being duped by Apple.

I was mad.  I declined to pay for what I felt was surely an unnecessary expensive repair and left.

Of course, that left me with a "Now what, smarty pants?" kind of problem.

I've seen the videos on the making of a MacBook Air on Apple's web site -- precision manufacturing, miniature assembly, etc.  Apple may be, in my mind, the analog of the shady mechanic overselling repairs, but that makes the MacBook Air the analog of high end cars, which are much more complicated than their predecessors and competition.

I went home and did some more research.  Apple sure doesn't encourage a DIY spirit of repair.  In fact, they invented their very own kind of screwdriver, the "Pentalobe", just so you cannot open your own computer! My first step was ordering a custom "Apple Tool" just so I could take the back off the thing.

I formulated a plan.  First, I would just look inside to assess the damage.  Was the logic board really covered in goo as the "Genius" implied?  Could it be cleaned?  Maybe I could prevent the further damage the "Genius" worried about. Could the keyboard be cleaned? In days of old, flushing a keyboard with water or alcohol could clear up mis-functioning keys. Worst case, I would replace the keyboard.  And if it really was as bad as they said, I would have to either suck it up and have them replace the parts (or replace them myself), or just treat the beast as a desktop computer (since the external keyboard worked fine) for as long as it cared to go on living.

It took a while to get the right screwdriver (heh, Apple has two different Pentalobe kind of screws, and I got the driver for the wrong one first). After the right tools arrived, I followed the instructions on the iFixit.com website to disassemble my MacBook Air.  I must confess some trepidation.  This was not a toaster I was taking apart, this was a $1500+ computer.  And if I broke something else, there would be no one to back me up.  Apple had already told me there was nothing they could do for me, as I had chosen the "highway". I was totally on my own with this.  On the other hand, if the Apple repair truly was $1000, I might as well upgrade the computer, so I figured my risk wasn't the entire $1500.

Disassembly went smoothly -- mostly --  and I saw that the computer design was sufficiently modular that it wasn't as hard as I worried.  Just very time consuming.  A few steps required some puzzling out to do, and there was some fearful moments along the way.

I saw two things when the computer was apart.  First, there was some goo under the fan.

But, also very clearly, the logic board was clean.  The fan is attached to the logic board, but is completely separate and easy to remove on its own.  The goo was not on the logic board.  Close, but not there.  There was a bit of goo under, but not touching the logic board on the keyboard:

The logic board mounts on stand-offs and is not in physical contact with the black plastic you see (the fan is in physical contact with that plastic, which is why some goo got on the fan itself.

Let me be really clear about this:  I will be charitable, and say that the tech was confused about what had been affected by the spill and/or was extremely overly cautious.  There is room for opinion in everything.  But it is my strong opinion that they overestimated the extent of damage, and that overestimation was in the direction of greater profit for Apple.  They were wrong in their recommendation.  You can ask, who am I to suggest my opinion is better formed than Apple's? Read on, and you decide.

I cleaned the goo off the fan with some rubbing alcohol, and set it aside.  Since the problem was with the keyboard, I thought that, perhaps, the problems were caused by some lingering dirt or goo in the keyboard itself.  I removed all the keycaps, and bathed the keyboard in alcohol and contact cleaner, and reassembled the computer.

Unfortunately, the behavior was the same.  It ran fine, but the same keys that either didn't work, generated multiple keystrokes, or auto repeated still did.  On the plus side, I had done no damage to the computer.  But on the minus side, well, no progress had been made.  So much for the cheap, labor-only repair.

I ordered a new "top case" (the keyboard and the aluminum top to the computer are an integral unit) from the web.  It took a few days to arrive, and then I once again tore my computer apart.  As it turns out, all the parts of the Air mount into the top case, so it's not quite like you're just swapping out a part.  You are basically taking your computer completely apart, and then assembling all those parts into your new top case.

As before, I was worried about some new damage in the process.  The connectors and sockets in the computer are fragile and not built for repetitive removal and insertion. I didn't want to push them to the breaking point.  I did my best to be careful.

When I was done, I pressed the power button.  It booted just fine.  I gave the keyboard a quick test, and it all appeared to be working.  All the problems were fixed.

And so here I am.  The MacBook Air works perfectly, and I am typing this blog post on it.  Just in case I've missed a letter: The quick brown fox jumped over the lazy dog.  Perfect.

Total cost? About $225 for the top case.  Or, about $800+ less than I would have paid Apple to replace the logic board and the top case.

There are few real lessons I've learned from this.

First, the "Genius Bar" is neither.  You should treat their recommendations for repairs the same way as you would as the guy who repairs your car or fixes your plumbing.  Most of the time, they're going to be right.  But if it seems like they are asking to do a lot more work than you expected, it's time for a second opinion.

Second, Apple isn't perfect, and that hubris we saw in Steve Jobs extends down to the lowest levels.  It's not pretty, and I hope that Tim Cook manages to bring some humility to Apple.  They definitely need to get rid of the arrogance about repairs.  I've been around computers longer than these kids have been alive, and I know one thing: you can never be certain of being right until the computer works.

Third, I'm going to be more active and less passive the next time I have a repair issue.  I should have pushed back hard.  I should have demanded to talk to a manager.  I didn't feel like I was on solid ground because I did not (and, at that point, could not have) see the extent of the damage myself.  Now, if I hear something that seems wrong, I will demand proof.

Fourth, the Genius Bar is just a "swap shop".  They replace parts, they do not repair them.  Something that could be a $20 repair but is a $200 swap is going to cost you $200 because they don't do real repairs.  Maybe I could have sent it off to Apple somewhere or somebody else and gotten a better repair, but I wasn't willing to trust Apple at that point.

Apple still is, in my mind, the best PC on the market, and I still recommend them to anyone.  But Apple's not perfect, and we need to keep that in mind. I will be caveating my recommendations to everyone from now on with those observations.

Finally, I recommend people keep all drinks at least foot away from their computer...