Category Archives: Opinion

The Limitations of AI from the Horse’s Mouth

I wrote an opinion piece a long time ago on the absurd amount of hype around AI, both optimistic and pessimistic.  But I’m just a guy on the internet.  Andrew Ng, on the other hand, is a world-renown expert on all things AI.  Not only does he teach the famous Coursera Stanford Machine Learning class, he was the founding lead of the Google Brain team and currently leads Baidu’s AI team.

His article is similar to his teaching: concise, clear, and enlightening.  Give it a read:  What Artificial Intelligence Can and Can’t Do Right Now.

A Python Data Science Workflow using Remote Servers

A good toolset and workflow can be a big productivity gain.  Unless you’re working exclusively on your personal computer for the whole pipeline, development workflows can quickly get messy.  I recently found a Python IDE that has great support for work on remote servers along with many other nice features:  PyCharm.  I’ll briefly walk through some of the basics that have provided a great relief to me.

Read more

New Metallica review

Metallica is a very polarizing band.  When I was 12 it was unquestionably cool to be a Metallica fan.  Then…Napster.  Almost overnight it became severely uncool to like Metallica.  In the public consciousness they quickly became the rich guys that were past their prime, complaining that they’d only make 10 instead of 100 million.  Former loyal fans were less likely to come to their defense perhaps because many felt betrayed by their evolving sound, which was a significant departure from what made them famous to begin with.  And although it’s almost unfathomable upon reflection, they caught a lot of shit for cutting their hair.  Seriously, it was a big deal in the 90s.

I think there’s at least one more reason that helps explain the vitriol seen in the comment section of every article pertaining to Metallica.  The band has been around so long, many original fans are now much older with tastes that have significantly changed.  They got real jobs, had kids and just don’t listen to metal.  That’s fine, but they mistake their nostalgia of youth as evidence that Metallica has significantly changed.  Is Death Magnetic‘s “My Apocalypse” really that different from Master of Puppet‘s “Battery”?  Sure, there are differences, but it’s definitely in the ballpark.  People seem to get really creative in finding reasons to hate Metallica.  The big criticism for the totally excellent Death Magnetic was that they used too much compression in the mix.  That was the first and last time I had heard anyone except recording engineers even talk about compression, much less use it as a reason to not like a record.  Exhausting.

But despite the band being able to do no right in the public’s eye, they’ve continued to deliver uncompromising, excellent material and Hardwired …To Self-Destruct is no exception.  Similar to Death Magnetic, they have cherry-picked the best aspects of the many sonic eras the band has created to result in a highly polished behemoth of creative output.  Although there are many examples of high speed thrash metal riffs and guitar harmonies that define their early material, they have brought in more elements from their later material in this album.

Specifically, the slower groove that is perhaps unique to 90’s Metallica.  Think Load‘s “2 x 4”, Reload‘s “Carpe Diem Baby”, or the black album’s “Sad But True”.  “Now That We’re Dead” would be at home on any of those albums, but the punctuation of the more complex prog parts would have made it stand out.  “Dream No More” and “Am I Savage?” are perfect examples of dropping into the groove of that era and making the most of Hetfield’s singing ability that didn’t seem to appear until Load.

Another album I find myself making many connections to is Disc 1 of their cover album, Garage Inc.  The sort of Irish-inspired sound of “Whiskey In the Jar” shows up in a few places.  Riffs similar to their Mercyful Fate tribute are sprinkled throughout.  And most pleasing to me, the Sabbath groove of “Sabbra Cadabra” is front and center on my favorite album track:  ManUNkind.

I love riffs that are both in an odd time signature and can still groove hard.  This song hangs in a time signature I still haven’t figured out, but is executed with a skill that allows the listener to bob their head uninterrupted.  Like Clutch, Isis, or Tool at their best, on this track in particular Metallica strikes that elusive balance of technically challenging and extremely listenable.

Aside from this song having all the elements I personally like, I think it also represents something very important: a truly original sound.  It is incredibly difficult to create an original vibe.  Metallica did it in the 80’s with their combination of orchestral melodies and punishing riffs that came to be known as the best example of thrash.  They miraculously did it again in the 90’s by creating a sound that I don’t think has ever been coherently named or recreated by any band since.  I don’t know what genre “Ain’t My Bitch” is, but it’s not thrash and I’ve never heard anything like it before or since.  And while you can argue that putting together these elements in different combinations is original, I think the accomplishment of “ManUNkind” is something different.  It would not fit any other album in their catalog, which is a hallmark to its originality.

But generally, Death Magnetic sounded like a backlash against those saying they didn’t have the technical chops that helped define their legendary albums.  It’s as if much of the 90’s was spent trying to convince themselves that they could do more than just thrash, but then a decade later they had to remind themselves that they still could.  Death Magnetic sounded like the answer to the question, “Remind me again why we’re avoiding our thrash history?”

Hardwired…To Self-Destruct, on the other hand, sounds like the band has finally gotten comfortable with what they created in the 90’s.  They seem at peace with not only living in the shadow of their own legend, but with the various twists and turns they made trying to escape it.  Instead of being confined by their legend from feeling pressure to either recreate or avoid it, they are finally able to just ignore it and make the best album they can, by their own definition.  Hardwired sounds like an album created without the burden of constraint.  It marks a new chapter in the long, awesome book of Metallica and I’m already looking forward to the next one…

Pattern Recognition and Machine Learning

A quick book recommendation for those trying to strengthen their machine learning knowledge:  Christopher Bishop’s Pattern Recognition and Machine Learning.  I have found it incredibly helpful in adding depth to sources like Andrew Ng’s Coursera class.  It’s apparently a staple in the Computer Science world, but I was never exposed having come from physics.

The topics you would expect are there with great depth and clarity.  However, it is also focused on providing the Bayesian perspective.  If you’re new to Bayesian ideas applied to Machine Learning this is an excellent text.  It does a great job of both contrasting frequentist vs Bayesian approaches and showing their connections.  It should be on every Data Scientist’s shelf.

An Open Letter to Open Letters

Dear Open Letters,

Although you likely won’t read this since Open Letters are likely never read by their stated target, I’m writing it anyway because I’m actually just looking for a pedestal to shout from.  Not because I feel particularly strongly or that I think any actual change will result from this, but rather because it makes me feel good about myself to take zero risk while pretending to make a bold, public gesture.

This isn’t about you, Open Letters, this is about me.  You see, I need to feel important.  However, it’s hard work to actually be important.  I’d rather pen a pointless letter that takes an incredibly uncontroversial opinion but then write it as if it were aggressively brave.  Then I will unleash it on social media because social media asks the absolute minimum of others to show their support:  a movement of a finger by mere millimeters.  But this inane, inconsequential gesture is enough to stroke my ego, and maybe even those on the other end of the tapping fingers.  It validates that the trivial amount of effort was worth it and then I’ll sleep better knowing that I can say, “Well at least I raised awareness.”  SJW 101 stuff.

Consider what you actually are, Open Letters:  the world’s most efficient ego-stroking machine.  Just consider: if it only takes a person 5 minutes to read this, but it goes viral to 10M people, that’s 83,333 man hours.  That’s like 2,083 people working full time for a week, all for me.  And what did I have to do?  Spend 15 minutes at a keyboard.  So lest you think it’s about you and your content, Open Letters, be assured that it’s not.  You work for me.


A lazy blogger.

Being More Effective in a Technical Job

Due to some mentoring from a friendly colleague I’ve taken away a simple, but valuable lesson about managing a resource constrained technical problem within an organization.  For those of you that have become frustrated trying to solve a problem while also “managing your managers”, this may add perspective.

Background:  I’m currently tasked with delivering a high-profile, but uncertain R&D solution where, like everything else, it was needed yesterday.  It has the attention of several levels of managers with varying degrees of technical understanding and there are many indirect paths of communication within the organization.  In these scenarios, keeping communication coherent is a project itself.

As such, the frustration came in having to spend as much or more time in meetings answering questions, talking about plans, reassuring interested parties…etc than actually working to solve the problem.  I’m sure I’m not the only person that’s wanted to say, “I would have this answer for you, but I’ve been too busy talking in meetings!”  The conversation with my colleague helped me understand that the perception of how the project is going is separate than the reality.  I thought of a sort of truth-table to illustrate the point, shown below.

Perception graphic

Obviously we all want to be in Square 1, where the project is going well and managers perceive it as going well, and want to avoid Square 4, where it’s all going poorly.  So which of the other two are preferable?  This question is relevant because you may be confronted with a choice on whether you prioritize spending time on aligning communication or on solving the technical problem.

Consider the dynamics of being in each of these two squares.  Put another way, if you’re in Square 2, are you more likely to end up in Good/Good or Bad/Bad?  If the perception is bad but the project is going well, you’re likely to get intervention from managers since they see the choice as either stepping in to fix the problem or doing nothing while the project fails.  Obviously a self-interested manager would do the only thing they can, which is step in to help.  This may be a good move if the project is actually going poorly, but if the project is doing well in reality, this is likely to hurt the chances of success since presumably you are the expert on solving the problem at hand and were given the task for a good reason.  This means you’ll be spending your time trying to justify your plans or explaining why alternative plans are not as good instead of doing real, productive work.

Alternatively, in Square 3 you have real work to do on the problem but the managers trust your vision and do not intervene disruptively.  This essentially buys you time to redirect into Square 1.  In the previous case, you’re spending more and more time convincing people of your vision and fielding technical nuances, which takes away your time to actually solve the problem.  In this way the communication is very important for keeping the project on track because it has practical consequences on resource allocation.

This understanding of communication have shifted my perspective on priority.  Before, as a technically minded person, I would have said Square 2 was better because once you get traction on the problem you can sort out the communication.  I now better understand the adage “Perception is reality.”

Speculation/Opinion: A Call for Calm Amid A.I. Fears

It seems there’s been a recent surge of famous smart people expressing their fears about the future of artificial intelligence and the ramifications of passing the so-called singularity, which is where machine intelligence surpasses humans’.  Voices like Stephen Hawking and Elon Musk seem to be the loudest.  A recent book by philosopher Nick Bostrom warns of so-called Super Intelligence, which is a speculative concept on intelligence that is orders of magnitude beyond human capability.

First, let’s assess where we are currently:  nowhere remotely close.  Some great computer scientist have gotten computers to do some really cool things, but very little of it could be mistaken for an intelligence of anything except the programmer.  Let’s consider a couple examples.


Obligatory Brain Picture

The classic:  playing chess.  Quite simply, the machine is good because it has the ability to simulate the outcome of billions of possibilities and to choose the move that maximizes the probability of winning at each stage.  Humans are incapable of this amount of computation yet can still compete at a high level by a mechanism I won’t pretend to know how to describe.  In this case, the technology that enables the computer to compete has little to do with creating intelligent software and everything to do with the hardware that the computers use to do their relatively simple calculations very quickly.

Another illumination of this idea is presented by Jeff Hawkins in his book On Intelligence and a similarly themed Ted Talk.  I will paraphrase.  Imagine coming home from work, walking up to your house and going through the door like you do every day.  However, on this particular day someone stuck a small piece of sandpaper on the bottom of the door knob.  The moment your hand touches the knob it would likely get your attention as being abnormal.

The current way to mimic this with software is similar to how a machine is used in chess:  brute force.  Your robot would climb the stairs (something incredibly difficult to get a robot to do, although toddlers learn it with ease while chewing gum and having a conversation) and then need to run down the list of all the things to check.  If the programmer didn’t have the foresight to to check the texture under the knob as it was grabbed, then the machine won’t detect the abnormality.  Even if the programmer were clever enough to hard-code this routine there’s a huge space of other possibilities that could occur and the programmer would always be playing catch-up to account for new curve balls.  Whatever it is humans are doing, it seems it doesn’t include ruling out all conceivable possibilities.

These are fundamentally different approaches.  Computer scientists have found great ways of finding patterns in data and getting machines to be able to find (some) cats in videos using the best engineers and most powerful systems, but the approaches remain in the category of “huge numbers of calculations” and the results do not allow easy translation to new types of problems.

Getting a robot to demonstrate the flexibility of motor skills of a 3 year old is beyond the state of the art by a large margin.  But a computer can multiply numbers far better and faster than the most trained mathematician.  Finding clever ways to use mathematics and billions of fast computations to get a computer to approximate something humans do with ease is not anything like machine intelligence.  The fact is we know very little about building intelligent systems and to suggest we’re something on the order of a decade or two away from building a technology that surpasses human ability is as fanciful as thinking we would be driving flying cars in the year 2000.  Barring a radical new approach that enables more progress in a decade than has been accomplished in 50+ years, it just isn’t going to happen.  Arguments that quote Moore’s Law fail to see the distinction between powerful computation (which we’ve already figured out) and intelligence, where we’re clueless.

Hardware differences

A major difference, I speculate, that accounts for a portion of the discrepancy is that in information processing architectures.  Although exceptions certainly exist, the modus operandi of a computer is having a task, computing the outcome, and then going to the next task.  This is easy for us to overlook because computers complete their simple tasks very, very quickly.  Let’s say you want to model the movement of a school of fish.  The typical approach is to stop the clock, cycle through each fish and calculate what they will do next given some simple rules and the current configuration that’s frozen in time, let one tick on the clock click, stop the watch again, move the fish to their new positions, then recalculate for each fish again and repeat this loop many times.  In reality, each fish is worrying about themselves and there’s no master book-keeper tracking each and every movement.

Similarly, even if we “train an artificial neural network” so that it can detect faces in an image, the master book-keeping computer still has to sequentially compute how each pixel is affecting each artificial neuron.  In the retina/brain, the magic is largely happening in parallel.  Like the fish, the neurons are reacting to their particular environments and it doesn’t need to wait in line to carry out its action.  Understanding what each fish is doing individually is only part of the story—the collective behavior of many simply interacting fish is the emergent behavior of interest.

There are of course technologies that take a computer program and break down its tasks into smaller pieces for parallel computation if the problem at hand lends itself to such parallelization, but it’s far from the scale of what is happening in the brain.  Having a cluster of 10 powerful computers breaking down billions of computations and re-assembling the results is qualitatively different than a billion simple things making simple calculations simultaneously with ambiguous “inputs” and “outputs”, with many signals being sent among units on a complex network structure.  Statements that suggest that the problem is that we don’t have the computational power to simulate billions of neurons I think again miss the mark:  these implicitly assume the approach should be the status quo—serial computation (or parallel computation to speed up a largely serial task).  I would guess advances in AI research will be as much limited by hardware development as software and that doesn’t mean more clock cycles per second.  However, developing the tools to control and measure such a massively parallel system are challenges themselves.  I think the real meat of the problem is in emergence phenomena of complex systems, but of course, being a person studying in this field, I would think such a grand thing about it.

Simulating the dynamics by stopping the clock and making incremental changes may help us understand mechanisms and be useful exploratory tools, but there’s no reason to expect the best way to build a “school-of-fish-like” machine is with serial processing.  The fact that a lot of the AI community and the institutions that fund them are focused on traditional software-on-computers types of solutions suggests not only are we far away from realizing this “real” Artificial Intelligence, we’re not even in the right paradigm.

The Fear

So where does the fear come from?  I won’t pretend to have an answer to such a question, but it seems peculiar that people largely regarded for their intelligence fear intelligence that’s beyond their own.  Perhaps they’re fearful that their strength will be marginalized?  Regardless of their personal motivation, it’s all too common to see the public influenced by sensational reporting.  The fact that you likely understand the Hal 9000 and Skynet references not only shows this fear of advanced AI isn’t a new one, but also that their “distant future” predictions (2001 and 2029 respectively) tend to massively underestimate just how far away radical technological developments are to their present.  Although there’s little harm in needlessly speculating on how to prevent a Super Intelligence from destroying the human race (it feels absurd even typing it), there could be harm in taking some of the recommended actions, like creating institutions and legislation to fix the problems before they’re problems.  If we’re completely in the dark on how to build intelligent machines because we have a poor understanding of intelligence it seems reckless to speculate on the potential powers and “intentions” of such technologies.  Suggesting ways of dealing with these speculative powers and intentions appears, to me, quite absurd.  To imagine that we’ll stumble upon the technology to create a Super Intelligence and then make a mistake that allows it to super-manipulate us into it super-destroying our entire race vastly underestimates the challenges we’re facing.  Thinking we should start crafting ideas for controlling it seems to me like drafting traffic laws for flying cars in the 1970s.  Before we start building the Faraday cages to suppress the beast, let’s first see if we can get that voice recognizer built with the power of Google to do a little better than this on my voicemail:

“Ohh. Hi. This is Tony for maim client, she. I’m calling about steps 32 in the C R bye hey you know I’mtalking about once a step 30 to Okay bye”