Data Science: The Revolution Lacks Useability

37193508I spent the week in San Francisco at GraphLab, a data science and machine learning summit.

The interest in the field of data science is simply amazing,  growing rapidly (last year’s GraphLab had ~300 attendees, this year’s GraphLab had something like 500+)…and well justified. Companies like Pandora and Netflix who have made Machine Learning and Data Science techniques work have conquered the world…but capturing and recreating their magic remains an elusive task. The field is fraught with problems.

Data Science is a field which is deeply entrenched in it’s academic roots and the number of people who can claim any sort of expertise are few and far between (and often already rich/highly valued and therefore impossible to hire). Even rarer are large enterprises willing to invest in obtaining data science expertise.

The problem is part supply and demand. A Data Scientist is basically a software engineer who can do statistics. Most tech companies have a difficult enough time finding normal software types, now they have to find ones who dig stats. Such a person will be as rare as the mythical “Developer-Designer-corn” or hybrid developer / designer.  Such a person would be so intelligent and talented that they would likely not bother working for your company and be off starting their own.

The next part of the problem is communication: How do you actually get this mythical Software Engineering Statistician Unicorn to both turn up useful business insights and then convince management types to act on these insights?

In my experience, having business insights is often much, much easier than actually getting bandwidth to implement any insights outside of the realm of critical path tasks. An organization which is willing to proceed through the use of Data Science probably can’t exist unless the founder built the company around such an approach.

In short: Most current existing companies are very unlikely to suddenly sprout machine learning competencies where they had none before and the supply of people able to claim the expertise is nearly non existant. Therefore, most large enterprises are going to be better off getting some serious external help in that department. Otherwise, the only way I see data science becoming a focus is by starting your tech company with it as a cornerstone and proceeding from there.

The final hope for Data Science finding its way into main stream technology companies is to expose functionality in a way that BI, data analysts and online marketers can actually run exotic Graph based queries with simple interfaces (see: Google Analytics). Such interfaces have not been invented yet…will they ever be?

Graph based data does not really lend itself to analysis via traditional Microsoft Excel spreadsheets, a new set of tools will be needed.


Game Over, Man


Like a silent anaconda slowly wrapping it’s coils around a sleeping jungle wildebeest, Apple is slowly dropping the last pieces of it’s all-conquering gaming strategy into place.

Apple is not rushing to market….this is slow, methodical and almost sinister in it’s measured progression.

The first steps involved iOS, the iPad, the iPhone, the iPod, the App Store and then software features like Game Center and then micro transactions for games. At this week’s WWDC, we saw a few new glimpses of the freight train that is coming: SpriteKit and SceneKit….two brand new game centric tool sets for 2D and 3D games. Possibly of equal importance will be updates to OSX which allow for completely independent windows to be spawned on Apple TV’s using AirPlay…just like the Wii-U!

The strings are all in place but loosely tied…if Apple is half as intelligent and methodical as they appear to be, all of these elements will be strung together and suddenly yanked into a tight noose around the entire game and console market which will leave little room for escape….But something is missing.

Where is the final piece?

Changing topics for a moment, lets examine the new Mac Pro:


Just look at this thing. Have you ever seen anything like it? Me neither. Just imagine walking into a pristine, white Apple store filled with airy, ultra thin silver and white devices and then coming across this monstrosity…it sticks out like a sore thumb.

There are only a couple market niches where black, powerful devices for sophisticated high-end users abounds: High end work stations and game consoles.

If Apple can produce a Mac Pro that looks like this thing and is 1/8th the size of the original Mac Pro…imagine what they could do with an Apple TV Extreme…perhaps controlled, via AirPlay or Mifi game controller.


Part II: Mobile Software Is Eating Game Consoles


It’s easy to forget watching the celebrations going on at E3 around the launch of new game consoles that the console market has sprung a leak.

Mobile gaming is the elephant in the room…not used game sales, not DRM, not services and certainly not new console hardware.

The Nintendo was the first to fall…the broad casual market which Apple dominates through their app store turned out to be low hanging fruit. No one wants to pay $50 for a game when the alternative is either a free download game or a $0.99 game from the app store.

Microsoft and Sony will be next. It is likely that mobile may not, by itself, ultimately kill the console market…but downloadable, cheap games will likely be the most important factor controlling for all other variables.

Congratulations Sony, you have won the distinction of being king of the sand castle.  Just hope the wave takes two more years to arrive.



Software is eating the world, and mobile software is eating software

Brace YourselvesMark Andreeson wrote a famous column titled “Software is eating the world” which cautioned strongly that high tech has grown into an unstoppable, job destroying, disruptive machine. His overriding message is simple: If what you are doing can be automated by software, you had better start strapping on a life jacket because you are going in the water and the water is cold.

Of all of that disruptive software, mobile is, by far, the fastest growing segment…mobile is growing 12x faster than the original Internet and that growth is accelerating on an exponential curve. In short, if what you are doing can be done by mobile software, strap on the life jacket because you are going in the water and the water is cold.

Mobile is hard…incredibly hard and not for reasons of complexity, but for reasons of simplicity. Most corporations are incapable of cutting back on features as a result of group think and product management creep…mobile devices are tiny and space is limited so cuts are required.

As a result, many corporations will not survive the mobile shift.

Mobile is about doing fewer things, personalization, limited data entry and timely, smart data which you don’t even have to go digging for. Mobile knows what you want and gives it to you automatically because searching for data on a tiny screen is painful for end users.

Every enterprise software space will be impacted by mobile…if that software is not altogether supplanted by mobile, users will almost certainly expect lighter weight mobile versions of that same software.

In short: The clue phone is ringing off the hook with regards to mobile.


Simple, Different, Fun…Not Always “Premium”

2012-12-13-bondi-blueApple has not always been “just” a premium brand…this is a label that has been applied to them many times over the years due to their consistant strategy of producing the “best” mobile device: The iPhone and the “best” PC: The MacBook Pro (or AIR), but it has not always been the case….

It doesn’t take much of a history lesson to uncover several notable situations where Apple chose not to deliver the “premium” product in a given category….the original iMac is probably the best example. 

At the time the iMac was introduced, it was by no means, the “best” computer on the market. It was a very simple, somewhat affordable, playful all-in-one device which fit the bill for college students who had never owned a first computer. It wasn’t the iPhone of it’s time….there were plenty of other, more powerful, “better” devices on the market which could be purchased for a similar price.

It was, however, an “Apple” device: Simple, different, fun.

I think it wouldn’t necessarily be wise to assume Apple is incapable of rolling out products which use different marketing positioning and pricing strategies based on the competitive landscape…a cheap iPhone, for example, wouldn’t necessarily be a bad idea given that India and China (where many may not be able to afford the higher end iPhone) are much bigger growth markets than the United States….





PC Gaming And Console Gaming Will Converge…Soon

This thing is still too slow, hot and expensive

This year, the two worlds of PCs and Consoles will start to converge. Depending on how well Valve executes the Steam Box hardware and content roll out, it may happen almost overnight and with shocking speed.

Game consoles in the current generation (aside from the Wii-U) have been designed to prevent gamers from accessing their games and limit the power of their owners, however, the release of new, high powered, mobile gaming chips including Intel’s Haswell chip and NVidia’s Kepler may prove to be the tipping point into a new realm of high powered, console-like, PC’s.

Here are some laughable characteristics of the current crop of “hardcore” game consoles which leave them ripe for disruption:

  • You can’t play games you bought for past console generations
  • You must subscribe to nonsensical “services” in order to access features like internet browsers, Netflix, Hulu and online gameplay
  • You cant / are limited (aka charged money) in your ability to share games with friends
  • You cant play used games
  • DRM is strictly enforced and requires an “always on” internet connection
  • Indie game developers are suppressed or required to jump through onerous hurdles in order to publish or update games on these platforms
  • There are limited integrations with mobile devices or limited streaming from computer or mobile tablet capabilities
  • Games still cost $70

If the SimCity and Diablo III release debacles are not warning enough, the outcry against the XBox One release is the canary in the coal mine for what is about to happen to the console market: sudden death followed by soaring rebirth.

From what I have so far learned, the Steam Box will have all the positive aspects of PC gaming and none of the negative aspects of console gaming. In one word: Freedom.


The Console Market Is Looking Like The Phone Market In 2006


The new XBox reveal event was today.

Basically, it is the exact same as the old XBox except larger, the interface has been made five times more complicated by the inclusion of a range of hand waving and obscure voice commands, it now requires that you have a plugged-in Kinect at all times, won’t play any old XBox 360 games and…something something something TV.Not just any tv…regular, boring network cable television! Oh and lets not forget…it won’t play used games.

After the debut of the PS4 (yet another large black slab device) and the failed Wii-U, it is fairly safe to say that innovation in the console market is now dead and that either Valve or Apple should now feel welcome to steam roll the current field of mediocrity with something new and different.



Smart Collection View Cells

I was recently introduced to a neat way for UITableViewCells to automatically wire their own XIBs and register themselves with their parent table view. This nicely moves the responsibility for doing this out of table view and into the table view cell itself…makes for a generally cleaner table view controller “.m” class in general, especially when dealing with tables that can have multiple different cell types.

The same is true of UICollectionViewController. Here is a handy class to help do the same:

To use them, just put something like this in –
– (UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath;

RestaurantCollectionViewCell *cell = [RestaurantCollectionViewCell cellForCollectionView:self.collectionView forIndexPath:indexPath];

@interface SmartCollectionViewCell : UICollectionViewCell

+(id)cellForCollectionView:(UICollectionView*)collectionView fromNib:(UINib*)nib forIndexPath:(NSIndexPath*)indexPath;
+(id)cellForCollectionView:(UICollectionView*)collectionView forIndexPath:(NSIndexPath*)indexPath;


#import "SmartCollectionViewCell.h"

@implementation SmartCollectionViewCell


    NSString *cellIdentifier = [self cellIdentifier];
    [collectionView registerClass:[self class]
    [collectionView registerNib:nib forCellWithReuseIdentifier:cellIdentifier];
    UICollectionViewCell *cell = [collectionView dequeueReusableCellWithReuseIdentifier:cellIdentifier
    return cell;

    return [[self class] cellForCollectionView:collectionView
                                       fromNib:[self nib]

+ (NSString *)nibName {
    return [self cellIdentifier];

+ (NSString *)cellIdentifier {
    [NSException raise:NSInternalInconsistencyException format:@"WARNING: YOU MUST OVERRIDE THIS GETTER IN YOUR CUSTOM CELL .M FILE"];
    static NSString* _cellIdentifier = nil;
    _cellIdentifier = NSStringFromClass([self class]);
    return _cellIdentifier;

    NSBundle * classBundle = [NSBundle bundleForClass:[self class]];
    UINib * nib = [UINib nibWithNibName:[self nibName]
    return nib;


The Ins and Outs Of AutoLayout

Interface Builder Is A Bad Guesser

Interface Builder attempts to guess which constraints you “meant” to install whenever you move anything around in IB. 60% of the time, XCode guesses wrong….enough to make it cumbersome.

An equally painful problem is attempting to arrange sub-views on top of backing views. For example, in some cases I want a label to be on top of a view but not *in* that view. Interface Builder defaults to automatically sticking your sub-view into the view below it when dragging and dropping things.

Dynamic Layouts Are A Major Pain

Simple struts and springs are vastly easier for laying out interfaces containing many views which can change in size dynamically. Autolayout structures interfere with the process of dynamic layouts in bizarre, often unpredictable ways. Recently I had to put together a table view containing many different labels which could have many lines and needed to update the height of the table cell as well as the contained labels. It was a massive nightmare to do this.

The best you can really do is remove all the height constraints of the individual views in your dynamic layout and then add horizontal width or height distance spacers and then pray that, if the sizes of these views change, the layout will be smart enough to adjust itself.

Hiding Bad Autolayout Constraint Placements

Many times I will hit an Autolayout crash which is directly caused by constraints which don’t actually show up visually in Interface Builder unless I select the parent view and manually examine every constraint until I find the bad one. These hidden ones are often automatically inserted while shifting views around.

Forcing Irritating Constraint Requirements

Massive annoyance I encountered today was that Interface Builder demanded that a “bottom” spacer be inserted below a button. This bottom spacer was ruining the entire layout (inside a scroll view) and causing havoc when attempting to properly size the scroll view. There was no way to get rid of it unfortunately. I wound up scrapping the view controller design and putting it into a custom table view instead.

Adjusting Existing Constraints Is A Huge Pain

Sometimes you might want to adjust a constraint manually. This is generally discouraged, but possible. If Autolayout were a perfect solution, it might be less common. However, in order to manually tweak a constraint, there is no way to simply extract a specific constraint associated with a view….you have to iterate through a flat array of them and then try to scan for the one that interests you. If you want to modify the constraint, in many cases you must destroy it and then replace it in order to change the value.

Useless Crash Messages

Autolayout crashes essentially consist of a large list of constraints which might be the problem ending with one of them being arbitrarily broken in order to stop the application from crashing outright. To me, this is a sign of a fundamentally broken system – it is almost as though the XCode designers / developers threw their hands up in dispair after discovering how AutoLayout was leading to non-stop, impossible to detect in advance crashes and their best solution was to just arbitrarily guess and then snap a given constraint just to make it work.


In summation: The promise of AutoLayout was a lofty one, but it’s current existing implementation leaves a lot to be desired. I seriously hope this topic is revisited during WWDC and iOS 7.