Cognitive biases in enterprise IT

I have a theory that almost all inefficiencies, problems, or destructive behaviorisms in enterprise IT settings can be classified and connected to some type of cognitive bias. I realized this as I was reading through "Cognitive Biases - A Visual Study Guide". At a macro level you can easily see how these apply. Things like:

• The Consultation Paradox, and how it baffles you that your advice is ignored but when a consultant comes it and says the same thing, it's take up like no one has recommended it before
• How System Justification Effect causes infrastructure or development teams to maintain the status quo rather than modernize, seek improvements, adapt to needs of others, etc.
• How the False Consensus Effect causes you to over-estimate how much change you think you've influenced but haven't (and you're surprised when you find an island of teams who completely disagree with what change you're trying to make)
• When you try an new technique to speed up a deployment but it has a bug, and the Negatively Bias causes people to irrationally recommend against improvements for a period of time (related to Von Restorff effect as well)

There's classifications for biases you knew existed but never knew they had a proper name. Read through them yourself and I bet you can think of an example scenario for each bias if you've worked in enterprise IT long enough.

I even see some of my own work-place behaviors couched in some of these cognitive biases. Understanding each one and being conscious of them might help to avoid the error.

Now, how to remember each bias, recognize when they present themselves, and defuse or counter-act them, that's the hard part.

A way of many ways to gauge continuous integration maturity

In an enterprise setting, for those involved with pushing your organization to adopt continuous integration and strive for continuous delivery, you're familiar with the obstacle filled trail you have to take (keep your head up; the fruits of change are rewarding).

As you watch other teams pick up CI, you hear them talk about where they are in their journey, and when they ask you "Am I getting close to good CI maturity?" (no one actually says it that way BTW), you can say, "You know you're doing it right when your continuous integration bottleneck isn't an manual organizational delay anymore but instead you find yourself improving technical bottlenecks like: how long it takes to bootstrap chef on a fresh VM, the time it takes to spin up a VM on OpenStack, or the duration of running test cases, code coverage, static code analysis, etc."

Blogging about technology at Target Corporation

I think most people know this, but I write technology career oriented blog posts for Target Corporation's "Pulse" blog. I've been doing it for over a year and I actually enjoy it because I'm given the freedom to talk openly about the technology work we do at Target (I'm especially proud to talk about the things I'm personally involved with and work on). In return it gives Target an opportunity to highlight its increasingly visible technical brand.

If you haven't read any before, you can read my posts here: https://pulse.target.com/author/dan/

Talking to 6th graders about careers in technology

For the final week before summer vacation, Washington Middle School teacher Daniele Albrecht invited me and several others to Career Week for an opportunity to speak to 6th graders about what we do for a living. The chance to do this excited me because I often reflect on my own childhood at the point when I was beginning to think about technology as a job; having accessible people around willing to share their experiences, guidance, and knowledge can be a powerful motivator.

I did two sessions, and the students were top-notch in both of them (the teachers Bill Spradley and Bill Ethridge were excellent hosts as well). They really seemed to be interested in what it takes to do what I do; I could tell because we went over allotted time with more questions than I could answer. We talked about things like:

• "What's your typical day look like?"
• "How much do you make?" - one of the first questions both times, but I didn't mind giving a ballpark figure; mainly because I remember hearing as a child how much doctors made; that left an impression on me and motivated me to get a good job actually
• "What can I do to learn and start programming?"
• "Can I code games at home?" - a good question actually; that's how I got started a Commodore 64!

I stuck to a simple narrative: there are companies who want to pay people to use technology to make more money, and I laced it with examples about making video games, building robots, creating mobile apps, etc (things they could relate to). It worked out really well I think. I could see a little bit of me in some of the student's reactions; their interest in my words captured their attention and you could see I got through to a few of them.

The key things I wanted them to walk away with were:

• Start tinkering and self-learning now
• Pursue a degree
• Most of all, apply yourself

Lastly, Jimmy Jacobson's (@jimmyjacobson) post today in /r/programming reminded me to blog about this. In hindsight it would have be useful to have fun illustrations to share while I talked. Jimmy sets a creative example to follow; fork and modify for yourself if you ever have the opportunity to do this!

Thanks to the teachers who organized this and to the students who listened and wrote me this nice thank you card!

What a radioactive leak can teach us about avoiding blame culture

The Verge published a notable article today titled "Radioactive kitty litter may have ruined our best hope to store nuclear waste". It's a well published story by reporter Matt Stroud (@ssttrroouudd) about the New Mexico Waste Isolation Pilot Plant (WIPP) and how a seemingly banal procedural human error has resulted in the shutdown of the site and jeopardized the future of radioactive waste disposal for the facility.

More interestingly is the teachable moment in all this about avoiding blame culture. It's easy to quickly react and suggest that the human who made the error should be punished, perhaps heavily fined, fired, or worse (some of the comments in the article suggest just that, and even Jim Conca a PhD and ex-geologist at WIPP who Matt interviewed suggested the same but later backtracked). Matt jumped in to reply to a comment suggesting the offender be jailed with an insightful rebuttal from Per Peterson, a professor at UC Berkeley’s Department of Nuclear Engineering who Matt exchanged emails with for the article. In the comment Matt mentions what Peterson had to say about this:

"The natural tendency in events and accidents is to focus on assigning blame and punishing human errors. This approach is generally ineffective because human error happens. The critical issue for safety is to design systems which are tolerant of human error and which encourage reporting of problems and errors and effective corrective action."

He's absolutely right about this. And it's applicable to so many other industries like health, construction, banking, and more, but its especially relevant to me as it applies to my line of work: IT. It's not uncommon that major mistakes happen in software development, that a small coding error brings down a system or an incorrect infrastructure config causes downtime in the middle of the night. It's hard not to react with blame top of mind when these moments happen.

Peterson suggests we eschew the natural reaction and instead design processes that account for the possibility of human error AND that we promote feedback loops allowing for process improvement. In IT that can manifest itself as infrastructure-automation (Chef, Puppet, etc.), continuous integration with good test coverage, etc. for the former, AND things like DevOps culture, blameless post-mortems, etc. for the latter. Making this the default mindset in a company really comes down to culture and how this type of behavior is rewarded and encouraged. I know I don't always practice building a blameless culture myself, but stories like this and advice like the kind that Peterson gives reminds me of the importance of doing it.

Speaking about enterprise APIs at AppsWorld 2014

I’m speaking at AppsWorld (Moscone West, San Francisco, February 5th - 6th). Well not speaking exactly, more of a discussion panel. The session is titled: "Panel: Launch and management of your API". We're going to be talking about:

• Publishing, promoting and overseeing your API deployment
• What tools are available to help you launch and manage your API and how do you use them?
• How to achieve seamless integration with enterprise systems
• Monitoring the lifecycle of your API and assessing its effectiveness at meeting developer and application needs
• Security considerations

Having spent nearly 3 years developing the API platform at the company I work for, I think I'll have a lot of useful protips to share.

If you see me there, stop me and say hi OR if you want to talk enterprise APIs over beer you can reach me on Twitter @pmotch.

My experience converting from Spotify to Google Music

Yesterday I spent a few hours converting from Spotify to Google Music. I had been waiting for this moment for some time (I'll explain in a minute), so when Google announced at I/O that they were rolling out a subscription music service, I jumped on it right away.

Google is my choice for a consolidated digital locker right now; that was a major influencing factor why I switched. When the subscription service arrived, it filled the gaps that kept me from using Google Music exclusively.

What Google does well:

• Web client is dead simple, beautiful, and fully intuitive in my opinion
• Android client is excellent too (with the latest update that is)
• Well organized catalog of music
• Music recommendations and "radio" are on par with Spotify (if not better on occasion)

Where Google separates themselves from the competition:

• I can buy music to augment where subscription content is not available
• I can upload my own music (homemade recordings from friends, music absent from Google's catalog, or music purchased from other places)
• No software install required (for playing music)
• Software for syncing my local music to Google Music just works (works perfectly on Linux too!)
• Integrated with Google Play, which is integrated with Google Wallet; dead simple purchasing
• Google had ~5 songs that were absent or greyed out in Spotify for some reason (see below about a few missing songs though)

The bad:

• Converting playlists by hand took a few hours; I'm sure it's a matter of time before scripts or an official conversion tool comes along
• Hit ~5 "Try again." errors; nothing serious. Trying again did the trick. With any new service I would expect small bugs like that.
• Out of a large list of music, there were about ~10 songs that weren't in Google's catalog. I'm hoping in time they seek to land deals with the smaller labels.
• Google is the master of search; there were a few things that didn't turn up in results like I would expect, but with the right phrase it would. It'll get better I suspect.

I started with Grooveshark, moved to Spotify, now I'm on Google Music. I'm happy with the consolidation; I'm just hoping this lasts for a while so I don't have to switch again.

Our presentation at API Strategy & Practice Conference

It's not so often the occasion where so many of people you follow on Twitter or whose software you use, can you see face to face, have a conversation with, and talk about a mutually enjoyable subject (as much as APIs can be enjoyed) -- this happened last month over two days at the sold out API Strategy & Practice Conference in NYC.

Kin Lane and 3scale did a great job bringing everyone together; enthusiasts, competing vendors, speakers, developers, etc. There was a wide breadth of sessions: companies talking about their API programs, start-ups showing off their stuff (and debating each other!), developers discussing techniques, and more. There was something for everyone. I think the general consensus was it was a success. I hear they're talking about a second one later this year.

We had the opportunity to speak at the conference on day two. Out presentation was about "Logging and Monitoring APIs" -- it was pretty well received I think. Like many large companies, we've got lots of APIs (and more coming down the pipe), and if you want to keep a confident pulse on the health of APIs, you have to log and monitor them in creative ways; we think we do a pretty good job at that, so we decided to share what we've learned with others. You can find the presentation here on SlideShare.

A visit to Amazon's AWS re: Invent conference

Last November I was in Las Vegas giving a talk at a conference that happened to be at the same time as Amazon's AWS re: Invent conference. It's not often I'd get a chance to hear Jeff Bezos speak in person so I was determined to make my way over to see what was up.

Some observations:

• Overheard someone say, "Amazon budgeted for 1200, told vendors 2000, and 6400 people showed up." If you were keeping track of the live stream views, it hit 13k+
• If the vendor floor size and number of booths was an indicator of how sprawling and established the ecosystem of AWS products and partners is, then I'd say they've easily captured the attention of vendors and customers by a large margin (but we all knew that already, right?)
• Super high concentration of big name talent in one place was pretty awesome
• Competitors sent people to scope things out among other things. Lots of Googlers, Microsoft people, etc.
• No one was talking about competitors, let alone comparing them (save the S3 vs Google Cloud Storage price war that was happening during the conference)
• Some notable tidbits from the Bezos "fireside chat" with CTO Werner Vogels:
• All upper executives at Amazon take 2 full days of customer training in the trenches every other year.
• Invest in "flywheels" -- things that are NOT going to change. In retail these things are: price, speed, selection. If you're in retail then you've got to see the talk for yourself to understand the description of this. Consider it mandatory; know you competitor.
• Watch the talk here: http://www.youtube.com/watch?v=O4MtQGRIIuA

Below are some photos I managed to take while I was there.

Presenting at API Strategy & Practice Conference

In February +Eric Helgeson and I will be heading to New York to present at the API Strategy and Practice Conference. We're going to be giving a talk about monitoring APIs and how it can be used for anything from troubleshooting to extracting business intelligence.

If you're doing work in the API space, consider going to this one. The speaker line up is looking good and I bet many attendees will be practitioners, so there should be lots to share and learn.

If you want to grab a beer with us while we're there, just holler our way. We'd be happy to talk APIs!

Ubuntu 12.10 fix for boot blank screen with [OK] and mouse cursor

I decided to do a fresh install of Ubuntu 12.10 given I had done so much experimental tweaking with 12.04; figured it was good idea to start over. The installation went super smooth (better than past versions I can recall, and I've been using Ubuntu as my primary OS since Breezy Badger); the first boot however didn't go so well.

Here's the steps I witnessed: the BIOS steps clear just fine, GRUB loads up and defaults into Ubuntu after a countdown (I dual boot), the Ubuntu progress meter behaves normally, then it transitions to a black screen with a residual [OK] at the top of the screen (something left over from the splash and quiet boot message suppression) AND a functioning mouse cursor. Really odd IMO. I'm pretty sure the OS is functioning just fine at this point, but something isn't graphically working correctly.

Here's the fix I tried with success:

1. While at the blank screen with the mouse cursor, hit Ctrl+Alt+F1. This should bring up a terminal.
2. Log in with the credentials you set at install
3. Type: sudo apt-get update
4. Type: sudo apt-get upgrade (given this few of days since the release, I saw a few updates for Compiz)
5. Reboot. Type: sudo shutdown -r now
6. Problem should be fixed

I suspect it had something to do with LightDM and Compiz not behaving correctly, and the Compiz updates fixed the problem. Good luck!

Presenting at Splunk .conf2012

At Splunk .conf2012 this year I'll be presenting, along with Eric Helgeson, how at Target Corporation we used Splunk to monitor end-to-end our internal and external web service APIs we developed, and how we managed to do it all within 3 months. We'll cover the problem we faced, how we used Splunk to solve it, show lots of examples, cover the challenges and lessons, then talk about the future where we're planning to take it next. If you're in the business of monitoring the APIs you created for your business, this talk will be right up your alley.

If you happen to be there, come see us present September 12th @ 4:15 PM, and look for the session titled "Splunk All the Things: Our First 3 Months Monitoring Web Service APIs". If you have questions after our talk, don't hesitate to approach us.

After the conference, I'll post the presentation on SlideShare.

A short ebook on creating RESTful APIs

I've had some downtime recently to get through my list of "read it later" bookmarks. This InfoQ article from last March gives an overview of a short ebook published by Apigee's Brian Mulloy called "Web API Design - Crafting Interfaces that Developers Love" (downloading the ebook requires signing up for an Apigee account - free of course). It's a quick read at 38 pages and covers the basics really well. The book also does a good job of avoiding too much theory by using API examples from companies like LinkedIn, Facebook, Twitter, Foursquare, etc (the Digg examples were awkward though). Like me, chances are you already know most of this stuff, but it was still useful to revisit the basics and validate that the stuff I'm building at the company I work for fits most of the best practices outlined in the book. I even managed to learn a few new things.

For beginners, I highly recommend. For veterans, it's still good, if only to remind yourself of the basic best practices.

My first open source project

It's called wikislurp. It's not much really; just a python script that, as my README says:

"...calls to the Mediawiki API, only grabs the recent changes from the last call, denormalizes the output, and outputs it in a log like format."

It's useful for things like Splunk, log tools, or data visualization/analytic tools that prefer to consume things in a denormalized deterministic log format.

It's poorly written I should confess, but done is better than perfect (feel free to improve a fork and pull request). I've always wanted to release something as open source. In the past I've contributed to open source via docs, submitted bugs, beta tested, etc, but I've never contributed code back to the community. It's time I finally did this.

Technology Club for 5th graders

Last year I asked my wife if the grade school she worked at had any after-school technology related clubs. I remember getting an early start on computers and technology at home, so I was naturally curious what the schools were doing to intersect kids with technology these days. They didn't have one, and after talking about it, we thought it would be a good idea to try (especially given the emphasis on STEM these days).

We faced two significant barriers: the school didn't have after-school programs and there wasn't a lot of resources or existing stories from other teachers that we could find on Google.

To get the program we needed to plan the ideas and square away a few logistical things. First we started by writing a proposal for the technology club with a set of simple goals in mind that needed approval from the principle. Also, we had to come up with scheduling, cadence, structure for duration of a single meeting, permission forms for parents to sign, etc. We limited the scope to just 5th graders, and no more than 10 students (in the end, anyone who was interested was allowed to join) We had about 15 students. Also, we determined that six sessions every other week for an hour on Thursdays was the right amount of time.

We found stories of grade schools with lots of woven technology into regular studies, but not much on afterschool programs. That forced us to think about content. What are kids interested in, what's too complex, what are good gateway topics to get them more interested in other technologies, etc. Also, we didn't have much of a budget (it came out of our pockets), so purchasing materials and such was limited. This forced us to get creative and make this happen on a shoestring. Here are the topics we covered:

• 3D rendering with Blender - It's open source and free, easy to run from a laptop and projector, and the students just loved setting scenes, creating animation, etc. Because it was free and open source, we were able to point the kids to the website where they could download it at home and try it for themselves (and a handful of them did!)
• Take apart a computer - we had an old computer we brought in, we tore it down to the major components and explained how each one worked together (this is where creative analogies helped convey concepts). We even took apart the hard drive showing the apature arm, spinning disks and all (they really thought this was amazing).
• Created a simple webpage - together as a group we created a basic HTML static webpage using Notepad++ and simply loaded it into Firefox. They total got the concept that the HTML markup translated to colors, paragraphs, images, and font sizes in the browser.
• Sound editing with Audacity - it too is open source and free. We recorded audio, applied effects, created layers merging them together into a song, etc. The student participating in this one was very high. No one was bashful to speak into the mic. The highlight of this activity was teaching students to say their names backwards and then reversing it. They loved it and thought it was hilarious!
• Renewable green energy - we bought a small science kit that explained wind, solar, and hydro energy. We spent 2 sessions assembling each set for the types of energy, doing experiments, and we explained the tradeoffs for each and why you would use one over the other.

Lessons we learned:

• 80% will show up, each week a different set of students are present; it's good that each topic from week to week was not dependent on the previous week
• There will always be a handful of students each session who parents will be late picking their kids up
• Expressing technical explanations through analogies they would clearly relate to is important
• Draw illustrations to explain things
• I'm certain this would work for 4th graders. I think you could go as low as 3rd graders with this format.
• Be hands on as much as possible; it holds their attention
• It only takes one student to misbehave and distract the whole group; find creative ways keep everyone engaged and distraction free
• They're smart; don't underestimate their abilities to learn quickly, even on programming topics!
• Kids like guest speakers (that was me each week!)
• If you can get a budget, it will help with candy, kits, supplies, etc.

If anyone has more questions about this, please let me know in the comments. We'd even be willing to chat, share documents we created, etc.

Kids and technology are the future.

My first patent was issued today

Today I was notified by the company I work for that a patent I was an inventor of and applied for a few years ago has been issued. It's a software patent dealing with knowledge management and social media:

Identifying Opportunities for Effective Expansion of the Content of a Collaboration Application, Patent No. 8,073,861, issued on December 6, 2011

I have two others pending as well; another dealing with social media and the other a technical security/encryption protocol I co-invented.

New eBoy Rio poster

Rio by eBoy

I noticed on Boing Boing today that eBoy released a new poster on Rio. This follows a theme of city scapes that eBoy does. I noticed two things:

1. I get the impression this one is a little more tame than the others I've seen before (and Rio of all cities!).
2. These city posters usually capture slices of major landmarks and scenes of the city. Are the favelas represented in the bottom right?

The colors are pop'n as usual! Looks great.

Took the Cloud Foundry tutorial finally

Back in May 2011 I signed up for a Cloud Foundry beta account so I could give the new PaaS service a whirl. It's taken me this long to get around to it (better late than never).

The tutorial begins by installing ruby and gems (I did this on Ubuntu 11.10), install vmc (the tool you use to interact with Cloud Foundry) via gems, "push" deploy some sample hello world Rudy code, and finally check it out in a browser. Making changes to what you deployed is easy: simply make changes to the code, then "update" the application; this re-deploys the application and restarts the running application using the new code. Refresh the browser and there it is. Pretty straightforward.

A couple of hiccups to watch for if you follow the same steps as me:

• When I tried "vmc target api.cloudfoundry.com" I was getting "invalid date format" errors. Turns out there's a version issue with the Ubuntu supplied Ruby packages. To fix it, I had to install Ruby 1.9.2 from RVM: https://rvm.beginrescueend.com/
• After a few clicks deep I found the "Getting Started..." guide. It's in a support forum and it's a PDF which is kind of odd. I was expecting a one-click away wiki-like resource.
• For getting started, I suspect the Youtube videos > PDF

I'll spend some more time with Cloud Foundry and post updates if they're interesting enough.

Replying to automated company emails

Ever notice the tiny grey text at the bottom of automated emails from companies saying something like:

"Please do not reply to this email address. This is an automated email notification and we will not reply."

That's kind of frustrating if you need to reply and there's no obvious way to respond in the email or on the website. I experienced this myself recently. It would have been easier if I could have just replied. Then I got to thinking, why don't companies, with all their automated emails, enable this and forward all replies to their customer service department that responds to normal email correspondence? Or offer a helpful automated exception response with more information specific to the topic of the original automated email. I realize the tradeoffs when this kind of thing is opened up, but it's so much more simple and satisfying for the customer.

I also wonder if it's always done this way because that's how everyone else does it or because it's a case of "that's how we've always done it." If so, maybe it's worth revisiting and reinventing.

Seems like the right thing to do.

Migrating to Blogger

I'm trying Blogger out. It's not as hot a tumblr, but it's under the Google empire; they do a pretty good job of unifying their properties (which makes it convenient for me). Over the coming weeks I'll merge and import posts from previous blogs dating back to 1999 (before the word "blog" was common) across different platforms: custom rolled PHP blog framework I created circa 2001, Typepad, etc. More to come I hope.