So what’s my motivation for moving to a new blogging platform? Was what I was using before broken? Not exactly, but I didn’t enjoy the blogging experience that much because I found the experience to not be a lot of fun. I want to start writing more and feel that removing any barriers to this activity will get more content out there. I also didn’t like having to rely on a server side application server. For a website that changes around 30 times a year, a database backed content management system was overkill. Also, the website was hosted on a friends server that I had no backups for, and didn’t want to spend much effort creating a backup/restoration strategy.

After some searching, I found jekyll, wich I love for its simplicity. I was going to use bare bones jekyll but then I was directed to Octopress which is a framework to make working with jekyll a lot easier.

Since I was on Windows my install was a bit different than the documentation. Luckily I have created a tool to make ruby installation simple on Windows. Here’s everything I did to go from nothing to having a working blog hosted on github!

1. Installed yari, and got my environment using ruby 1.9.2
2. Followed the octopress setup instructions
3. Followed the octopress deployment instructions

I was blown away when I saw my blog appear when I went to scottmuc.github.com.

Migration

Unfortunately my previous blog engine’s export functionality is broken so migration had to be manual. Every post is going to be copy and pasted and re-linkified in markdown syntax. I wanted to get this new version up and running first so I made a few compromises.

1. I decided not to migrate every post. Some of my old content doesn’t really fit with the rest of the content so I decided to abandon the posts. I’m keeping the old blog up and running so that people can still find the posts there if they really want them.
2. Keeping links working is extremely important to me. Octopress provides the ability to set the permalink on an individual post which made this element of the migration super easy. Every blog post has “front matter” which is a little bit of YAML meta data. Adding a permalink value will make the post accessible from that URL path:

   permalink: /foo/bar/post

3. I went live with an incomplete migration. I only have about a third of my posts migrated. I did this so I can get this live and sort out issues right away before everything is migrated. Also, it’s a way to motivate me to get everything into the new blog ASAP.
4. Added a custom 404 to guide those who want to see non-migrated content to the old blog.
5. Added “scottmuc.com” to my CNAME file. I followed the instructions here. Note that this is enabled for non-paying users now! I use gandi.net for my DNS and updating the A record and CNAME (for www) was a snap.

Other Stuff

When setting up the blog I had an online discussion about how url design. I decided to leave dates out of my url and to include them in the title slug only when appropriate. I came to the conclusion that url slug design is extremely important and doesn’t always have to be word for word related to the post title. Because Octopress is so easy to configure, the configuration setting for this is simply this value in _config.yml:

permalink: /:title/

I’m now using other 3rd party services like feedburner, disqus, and gist.github.com. With all of these things taken care of for me, authoring a post is a call to rake new_post[“post title”], edit in vim, and preview using rake preview to view the changes before I deploy them.

The one thing I would like to change is learn how to create a custom theme. It’s quite common to see a lot of octopress based blogs use the same default theme that I’m using now.

Summary

Octopress is awesome! I love the workflow for authoring new content. I much prefer markdown over using some WYSIWIG editor over a web connection. The ability to do everything local means there’s significantly less effort for me. I believe Octopress is an excellent example of how an excellent deployment mechanism can shift how work can be done. If pushing brand new code to production is that easy, it removes the need for complex server side code.

Lastly, if you would like to see what a raw blog post looks like, take a look here.

Reading another colleague’s post a while ago inspired me to write a little retrospective of my own. Thanks Rose, you are wise beyond your years!

ThoughtWorks has been a place I wanted to work for since I attended DevTeach in 2007. I met some really smart people who were excellent at articulating their perspective. Also, I observed many of the best blogs I read at the time were written by ThoughtWorkers. Yeah, I was also a BFF (Biggest Fowler Fan), but now my experience has shown me that there’s so much more.

So far I have worked on 3 large projects. This scale of work wasn’t exactly what I was expecting to do when I joined. I’m now thankful I got to work on them because they’ve all provide unique experiences.

• Project 1 - This was a pretty hostile environment where I learned how ThoughtWorkers band together and deliver.
• Project 2 - I worked on this one for over a year on the largest project at the time. Here I met my sponsor (here’s him in a monkey suit at the ThoughtWorks North America Away Day)who has become an awesome friend in a very short time. The work done there will inspire me for quite a long time. It’s the reason why I yap about PowerShell so much. Project 3 - This project was even larger than the previous one. Over 400 people distributed across the globe. Here I learned how to be patient, consistent, understanding, and observant. Large systems have so many variables that coming in complaining and pointing out the flaws gets you nowhere. For 8 months we chipped away and I think we made a difference. It wasn’t the most interesting project from a technical standpoint. I don’t think I’m mature enough yet to understand all the I’ve learned during this time. Plus I got to work with @manderbabble!

Here are some other take-aways from the last 2 years that I did not expect:

Be Organized

I’m pretty organized in general but have realized a lot of people aren’t. I got the feeling that this ability to be organized help instill confidence in those that depended on what I did. I personally know that it’s a relief to have people that you can depend on. However, I did feel that I had to nag people on occasion. In large projects there’s already many different ways to become distracted and I think my organizational skill helped me keep a cool head no matter what kind of perceived chaos was ensuing.

Be awesome to your office manager! He/she runs the joint and when you need that private meeting room, or need lunch ordered for a meeting, they are your go to people. Also figure out logistical tasks like your client email, userid, and network access so soon as you can. Everyone else in the organization will have an expectation on you to have all that stuff sorted out.

Lastly, get a guy (for taxi service). While working in Bellevue I made friends with a father son taxi company (call Harry (206)-724-9525). They reduced the stress of travel big time. I could text them when I needed a lift and they would monitor my travel itinerary and be ready to pick me up at the airport. Their service was absolutely top-notch, thanks Harry and Gill!

Know Your Org

Understanding the organizational structure is hugely valuable. As much as we like to complain about how the structure holds you back, that effort is just a waste of energy. Learning the structure and how to use it is key when working with larger organizations. Changing that structure requires lots of time, and the will to build up a lot of trust. This can be, unfortunately, beyond what we can offer as technical consultants.

It’s funny how often technical help is seeked but the problem is usually human and requires a much different angle to work with. It’s sad that often I see people try to solve human problems through technical measures. Sometimes it doesn’t appear to be clear to people that we use computers and work with people, not vice-versa.

Travel

For the last 8 months I’ve been on the road, and I probably only slept in my own bed for about 15 nights in that entire time period (except the Christmas break). This is really fun and exciting (and one of the main draws of ThoughtWorks for me). Here are a few things that it has made me realize:

• Home is where the laptop is. It’s amazing how no matter where I traveled to, I would feel quite comfortable as long as I have my laptop and an Internet connection. At all times I’ve been able to connect with friends and family.
• Autonomy is important. No matter where I am, I always try to keep my life autonomous. Some days I would hang out with friends and go with the flow, but many times I would consciencly be sure to break off and do things on my own.
• Ok, maybe home isn’t where the laptop is. Over the holidays I’ve come to realize that Calgary is truly my home right now. The level of comfort I got from my own bed, couch, diner, and friends was a warm reminder to slow it down every now and then. I can see how traveling makes it hard for many people to stick with ThoughtWorks. I’m lucky to be at a point in my life where this amount of chaos is exciting. As much as I enjoyed my week of over the holidays I was ready to get back on the “road”.

When You Don’t Know

The biggest thing I’ve learned is how to ask for help. I used to pride myself on being able to do things on my own. Unfortunately I cannot scale, and doing things on my own severely limits my capabilities. Asking for help has been the best thing I’ve learned. It’s enabled me to venture into territory that I know little about, but do so without much fear because I know I have awesome colleagues to fall back on.

Even in the personal space I’ve learned to ask for help. ThoughtWorkers make amazing friends too and provide a support structure that is beyond comprehension. I don’t know where I would be if it wasn’t for some of the push and support they’ve given me.

Saying “I don’t know” amongst a group of ThoughtWorkers is an invitation to be taught without being scoffed at which is better than the snickers and scoffing you get amongst other technical crowds.

Validation

This one is a little selfish but working at ThoughtWorks these last couple years have helped me become more confident in my intuition and analysis. Many strategies and tactics I’ve done earlier on in my career were actually on a correct path, albeit not executed as well because of my naiviety at the time. Before ThoughtWorks I had 8 years of IT work, and I feel even better about the work that I did. Of course I wish I could have done it all differently, but that’s with todays knowledge.

I now feel even more confident that I can take on problems and solve them even when I really don’t know exactly what I’m doing. I’ve proven to myself that I can solve problems. Hey, if we knew the solutions ahead of time, they wouldn’t be problems now, would they?

Also like to wish Tyler a happy two years! We started on the same day back in 2010. He’s managed to put up with my humour longer than any ThoughtWorker and that’s a pretty good accomplishment.

I’m now realizing this sounds like a pitch to get people to join ThoughtWorks. It didn’t start out that way but having more awesome people around doesn’t sound like such a bad idea! If you’re interested please check out join.thoughtworks.com.

And now for something completely different (the best image from the last 2 years):

Reflections on 2011 and carry over from 2010:

• Mouseless Computing (2010) - I think the practice is paying off as a colleague of mine told me “You make Windows fly!”, and he’s a savvy keyboard only Mac OSX user.
• OSS participation (2010) - I started the following projects:
  • Pester - I’ve blogged about it and it’s getting some traction out there in the wild. It’s a spec based testing framework for Powershell. The first implementation was written on a hung-over new years day!
  • yari - A Windows ruby version manager. Currently being used by a very large client. I wrote this on a whim, because I thought I could do better than what was previously implemented. Turned out the client liked this a lot more and decided to adopt it right away.
  • YDeliver - The beginnings of a Continuous Delivery framework for .Net projects.
  • Misc - Submitted a patch to OpenMRS, a bug report to Puppet, directory additions to PSGet
• Learn VIM (2011) - This went really well. I’m definitely no expert at this point but I’m continuoully trying to learn new things. I’ve setup a github project for my vimfiles which is a fun way to share my tweaks.
• Develop an Android App (2011) - Nope, didn’t do anything here. It’s actually dropped off my radar as it’s not really something I’m interested in anymore
• Learn a server provisioning automation tool (2011) - I barely get a passing grade on this one. I’ve yet to use any of these tools (chef, puppet, etc…) on a project. In my spare time I’ve tried them out and understand the gist of how they work. I do question how necessary they are though. Unless you’re doing massive scaling it seems like they just add extra overhead for simple scenarios.
• Move this blog (2011) - This will be my last post on this blog engine. I’ve currently setup Octopress and have my new blog ready on github. I just have the task of migrating over all posts (and deleting some). I have more to say about Octopress soon.

Development Goals for 2012:

2012 is going to be different for me development wise. I’m finding myself less focused on technical issues and more on system, culture, and people goals. I’m striving to become more of a leader (clear sign of post-technical syndrome) and I think that’s reflected in my resolutions for this year.

Stop reading comments

Often when I read comments online I want to get those precious minutes of my life back. I favour a discussion over pointless online kibitzing any day. Now of course every scenario is different. The probability of reading comments on Youtube or on some news website is very low. On a blog post about some technical issue, I’ll likely read the comments.

No client work when away from client

Being away from the client is not a place in space problem, it’s a frame of mind issue. I want to develop some skills in removing myself from the client mentally. This is purely for my own sanity and likely for the sanity of those around me. I’m finding myself disconnected from my personal goals and more focused at facilitating the goals of whom I am working for. One thing I’ve learned by being on the road is that ME time is harder to come by. As an introverted individual, that ME time is how I recharge.

Complete YDeliver

Hey, an actual technical resolution! By the end of 2012 I want to know that at least 5 projects are using YDeliver. It’s incomplete right now and I need to get off my arse and complete some of the “last mile” stories to get it out there. I also don’t want to be the single committer to the project. This is a call for help for anyone interested in participating! I have some very strong opinions on how some of the pieces should work together and I will be happy to provide guidance.

Summary

Yeah, so this is probably less interesting than previous years. I’m actually very happy on how my previous resolutions have developed and are continuing to get better and better. In previous years I mentioned developing something on a completely different stack (like Ruby on Rails). Next year I’m starting a new project working on a small application that involves iOS and ruby development. So work is going to force some of my previous resolutions on me.

Introducing Yari

yari is a tool (implemented in Powershell) I’ve made to simplify both of these operations. Right now it’s hard coded to only install the 2 ruby versions I care about right now.

• 1.8.7 so I can play with Puppet
• 1.9.2 to be bleeding edge

Installation is described in the readme on the github page. Once installed you can simply run yari from anywhere in the command prompt. Also, you can pass a -InstallMachine argument to yari and it will permanently configure your PATH for the ruby version selected.

So far this tool has made my ruby management in Windows a lot simpler and hopefully it can help you out too. If you find it useful but can use a few changes please submit issues to the projects github page. Or better yet, fork it and submit a pull request.

After installing, here’s what you’ll be able to do (and installing things that require native extensions will work):

C:\Users\smuc> ruby -v
'ruby' is not recognized as an internal or external command,
operable program or batch file.

C:\user\smuc> yari 1.9.2

C:\user\smuc> ruby -v
ruby 1.9.2p290 (2011-07-09) [i386-mingw32]

C:\user\smuc> yari 1.8.7

C:\user\smuc> ruby -v
ruby 1.8.7 (2011-02-18 patchlevel 334) [i386-mingw32]

C:\user\smuc>

I keep on looking at the ruby community with their amazing tools like rvm, ruby gems, and bundler to make ruby scripting highly testable and repeatable. The formalization also means that you don’t see people writing ruby code with random imports and magical includes that only work in very specific scenarios. By creating a community around gems and some basic standards, it’s forced developers to factor their ruby code accordingly and modularize when necessary.

So far I’ve found the following package management tools for Windows:

• Nuget - conversations online have lead me to understand that nuget isn’t meant for powershell
• OpenWrap - still under evaluation
• Chocolatey (by Rob Reynolds) - wrapper around nuget. Seems to be about config management like Puppet and Chef.
• Chewie (by Eric Ridgeway) - bundler for nuget. Could have some input on how to manage PS module packages. Potential to be the front end for managing suites of modules.
• Web Platform Installer - not even going to go there
• PS-Get (by Andrew Nurse) - built around nuget but Andrew has the MS connection to make it popular.
• PS-Get (by Mike Chaliy) - my favourite but needs some growth in the community. A showcase for simplest thing that can possibly work.
• Nugit - almost forgot about this late entry. Don’t know anything about it yet.

Here’s my wishlist (that none of the above can fulfill):

• Ability to deploy multiple versions of the same package
• Different levels of isolation (System | Profile | Project ) levels
• A management tool so that projects can declare versions and packages that they need along with dependency chaining (ala bundler and ivy)
• Packages can be stored in many different locations (Nuget.org, github, some url pattern)

History has shown that pretty much all languages need their own package management system (which can then be wrapped by a system level package manager). I vote that some of us in the community have a round-table and discuss how to move forward with this. There’s so much to learn from what already exists to get Windows out of the scripting ghetto.

Also, I want to thank Manoj Mahalingam and Martin Aatmaa for their feedback and contributions!

* the video starts off pixelated but it fixes itself in a few seconds

Mike Chaliy has made probably what could be called the simplest thing that could possibly work, and it works well! It’s called PsGet (not to be confused with another project called PsGet). Take a look at the homepage and bask in its simplicity! Installation is copying/pasting the bootstrap code. Then install modules via name or absolute url to a powershell module.

Take a look at my post about Pester for an example of setting it up.

I have a couple features that I would like to bake into it. Hopefully I won’t add too much complexity to it as I do really appreciate it’s simple approach.

• Ability to deploy a specific version (using git tags) eg: https://github.com/scottmuc/Pester/archives/1.0.3
• Something similar to the concept of gemsets in rvm for easy switching between module versions

Inside Pester there’s already a trivial Calculator example but it’s not really the best way to demonstrate the app. Pester itself is tested using Pester. In fact it’s being tested by the version of Pester that’s under test (although Martin pointed out some uncovered areas). Still, probably not the best way to show how it all works.

What I’m going to go through here is the beginning of what could possibly be a real world scenario for a person assigned to write deployment code.

Here’s the story:

Initech has had issues where their .Net web applications have been deployed on production servers with the debug compilation flag set to true. This has made production support people irritable because they now manually tweak the web.config every single time they do a deploy. Michael Bolton has decided he’s going to automate this step but wants to right it test first. He doesn’t want to repeat the debacle of his previous attempt at being clever.

First step is to setup a project. Mike (as he now likes be known as) decides to call his project of tools IDeploy and creates a folder of that name where he does is development.

Setting up Pester is simply a matter of following the instructions at PsGet (author Mike Chaliy has tons of great PowerShell modules)then running Install-Pester. Then running Import-Module Pester anytime you open up a PowerShell session where you want to use Pester. The console should look like the following:

� MIKE-PC {C:\d\IDeploy} (new-object Net.WebClient).DownloadString("http://bit.ly/GetPsGet") | iex
Downloading PsGet from https://github.com/chaliy/psget/raw/master/PsGet/PsGet.psm1
PsGet is installed and ready to use
USAGE:
    import-module PsGet
    install-module https://github.com/chaliy/psurl/raw/master/PsUrl/PsUrl.psm1

For more details:
    get-help install-module
Or visit http://psget.net
� MIKE-PC {C:\d\IDeploy} import-module PsGet
� MIKE-PC {C:\d\IDeploy} install-module Pester
Module Pester was successfully installed.
� MIKE-PC {C:\d\IDeploy} import-module Pester
� MIKE-PC {C:\d\IDeploy} Get-Module

ModuleType Name                      ExportedCommands
---------- ----                      ----------------
Script     PsGet                     {Get-PsGetModuleInfo, Install-Module}
Script     Pester                    {It, Describe, New-Fixture, Invoke-Pester...}

Pester includes a helper function called Create-Fixture. Calling the function with no args looks like the following:

� MIKE-PC {C:\d\IDeploy} Create-Fixture
invalid usage, please specify (path, name)
eg: .\Create-Fixture -Path Foo -Name Bar
creates .\Foo\Bar.ps1 and .\Foo.Bar.Tests.ps1

Create-Fixture wants to know what the path of your function is going to be and what to call it. I personally like having my tests next to what I’m testing so Create-Fixture sort of enforces this convention. Note that Pester can be used without ever using this function. I just never remember how to create my fixtures so this saves me a bit of copying and pasting.

Armed with a quick way to create fixtures Michael runs his Create-Fixture to scaffold his feature. He decides he wants it to be called Ensure-AspNetDebugIsFalse and places it in the Deploy\Functions directory of his project.

� MIKE-PC {C:\d\IDeploy} Create-Fixture Deploy\Functions Ensure-AspNetDebugIsFalse
Creating => Deploy\Functions\Ensure-AspNetDebugIsFalse.ps1
Creating => Deploy\Functions\Ensure-AspNetDebugIsFalse.Tests.ps1

Wanting to see some red he runs the tests by running Invoke-Pester which loads all files that match *.Tests.ps1 recursively in the current directory.

� SMUCS-PC {C:\d\IDeploy} Invoke-Pester
Executing all tests in C:\dev\IDeploy\Deploy
Describing Ensure-AspNetDebugIsFalse
does something useful
Tests completed
Passed: 0 Failed: 1

As you can see Pester by default makes a failing test. Now it’s time for Michael to update the test to make it more meaningful. Here’s his test file after he’s setup his expectations.

Wow, it failed! Why is that? By default Pester will generate a fixture that is silly and won’t ever pass. So what should we do with this broken function? As it stands now it’s totally empty. Let’s update our specification (aka Test) and do something useful.

So Michael has written a test that didn’t require a lot of code, but is actually doing a few cool things. I’ll try to go line by line and explain what’s going on:

Ensure-AspNetDebugIsFalse.Tests.ps1 contents:

$pwd = Split-Path -Parent $MyInvocation.MyCommand.Path
$sut = (Split-Path -Leaf $MyInvocation.MyCommand.Path).Replace(".Tests.", ".")
. "$pwd\$sut"
. "$pwd\..\..\Pester.1.0.1\tools\Pester.ps1"
 
Describe "Ensure-AspNetDebugIsFalse" {
 
    Setup -File "inetpub\wwwroot\testsite\web.config" `
                "<configuration><system.web><compilation debug='true' /></system.web></configuration>"
 
    It "switches debug attribute to false for a web.config in a given website path" {
        Ensure-AspNetDebugIsFalse "$TestDrive\inetpub\wwwroot\testsite"
 
        [xml] $xml = Get-Content "$TestDrive\inetpub\wwwroot\testsite\web.config"
        $xml.configuration."system.web".compilation.debug.should.be("false")
    }
}

Ensure-AspNetDebugIsFalse.ps1 contents:

function Ensure-AspNetDebugIsFalse($websitePath) {

}

Here are some details on some of the lines in the test.

1. We obtain the directory of the test script because we need to base all other paths off of it

2. Here’s how we ensure our code/test conventions. This means that the code in Foo.Tests.ps1 will automatically include Foo.ps1 (the code under test) in line 3

3. Create-Fixture automagically resolves the Pester path no matter what version of Pester you have. This directory will remain static. I might implement an Upgrade-Fixture script so that as you update your Pester version you can update your tests as well.

4. Pester has the ability to do filesystem based setups. This line is creating a file in isolation in what Pester calls the $TestDrive. The $TestDrive is disposed of after every Describe context. This allows you to perform filesystem related tasks without having to maintain separate test files. This line has created a file called web.config in the inetpub\wwwroot\testsite path in the $TestDrive and the 2nd argument is the content of that file.

5. We execute the code we want to test

6. We assert that the debug attribute has been set to false.

Let’s make this bad boy pass! Here’s the function fleshed out and the test passes!

function Ensure-AspNetDebugIsFalse($websitePath) {
    $webConfigPath = "$websitePath\web.config"
 
    [xml] $webConfig = Get-Content $webConfigPath
    $webConfig.configuration."system.web".compilation.debug = "false"
    $webConfig.Save($webConfigPath)
}

and the execution of the test:

� MIKE-PC {C:\d\IDeploy} Invoke-Pester
Executing all tests in C:\dev\IDeploy\Deploy
Describing Ensure-AspNetDebugIsFalse
switches debug attribute to false for a web.config in a given website path
Tests completed
Passed: 1 Failed: 0

Now Michael has an extra tool at his disposal to call when he’s performing deployments. If only he had taken this approach when he wrote his money transfer code. He wouldn’t have missed all those decimal places!

Hope this helps get you running with Pester. There’s a lot more I would like to add but I’m really only adding stuff as I see the need for it. Pester was driven out by my desire to refactor another Powershell library I wrote called PowerYaml. Take a look at the project page to see how I took the untested code, wrapped tests around it, then refactored.

Feedback is greatly appreciate and I hope this helps your teams make reliable Powershell scripts.

Operations people have two numbers they care about. The number 9 and how many of them they have (99.999% uptime is commonly known as five nines). This is a great metric for a system that doesn’t change, but in a world where new requirements are constantly conjured up and need to be live yesterday, the number of 9’s you have becomes harder to keep stable. A new operational metric needs to be created to handle this new kind of ever changing system.

Historically operations people are stereotyped as grumpy curmudgeons who will only let you deploy something in the 1 minute downtime window that they provide for you on the 3rd full moon after each solstice on a Sunday night at 2:29am, and you better get it done before the backups fire up or else you’re screwed! Developers on the other hand are eager to get their newly written code live so that people can see the fruits of their labour. It’s pretty obvious that these mentalities are in complete opposition.

This clash of values can be lessened if operators adopted a new bragging metric. The Mean Time to Recovery (MTTR). An operations person that can fix a problem fast is much stronger than someone without that kind of debugging ability. To get ops to fix problems that means they need to have strong knowledge of the applications they are deploying and strong relationships with the teams that developed them. The hard part about this metric is that it needs to be exercised. You can’t measure your MTTR unless things break.

Another great metric was created by the Lean way of thinking where minimization of completed features (an inventory) waiting to go live is important because that backlog is considered wasteful. This is still a great system wide view of shortening the time from concept to market, but I want to focus a little on the mind of an operations person.

One of the problems with software development is a general sense of hubris when it comes to the stability and functionality of an application. Stats like the number of unit tests and code coverage are often leaned on with the implicit understanding that it means the code is working. That’s when concepts like Mutation Testing are great for truly determining the value and confidence that the test suite brings. Tools like Jester perform a little bytecode magic behind the scenes to determine how brittle your tests are.

Operations can suffer from the same kind illogical confidence. How are the 9’s measured? I’ve often seen uptime measured by how long heartbeat.html is still up, but how confident would that make you believe that the entire system can still process an order? A server that’s up and running serving up broken software should not count as being “up”. This is where a common agreement of the term “done” is important in bridging the gap between software development and operations.

It’s too bad that ops can’t perform activities like TDD where the developer intentionally fails a test in order to drive out the code to implement a feature.

At a few conferences I’ve met a few people who deploy to production rapidly in massive systems (IMVU, Facebook). They don’t know 100% the code they are deploying won’t break in production. By deploying to partitions they can use system wide health metrics to determine whether or not things are working. Are orders still being processed in the partition that the new code was deployed to?

In my previous work experience I deployed code to production all the time (and broke things). Every time I broke something was a learning experience, but I didn’t stress about it because I knew how to fix things reasonably fast. I didn’t realize at the time how valuable that was. Periodic failure was welcome because it re-assured us that we knew how the system worked. Now of course mission critical applications have much different sensitivities to failure. I’m not talking about software that launches the space shuttle here.

In conclusion I would like to see how we can allow operations to fail without such a negative impact. If a deployment fails because someone forgot to chmod a file with +x, that’s a great lesson learned and hopefully someone will update the deployment script. After all, we’re all human and we make mistakes. By recording the operational MTTR we can grow a team that updates production frequently and fixes errors fast, rather than a team performs massive batch releases to production and slowly figures out the errors. So let’s figure out how we can help our ops people fear downtime a bit less and empower them with the ability fix the software we build. On the flip side lets write software that is easily deployable and doesn’t require pages of instructions to make live so that operations can feel more comfortable tweaking and redeploying with confidence.

I don’t propose any solutions to this mindset but would love to hear some feedback. Would measuring MTTR be important in an operations group?

• Test Driven Design - I barely coded at all so that was a total flop. I also believe that this is something that’s hard to do on your own. This discipline really needs a pair in order to stay on track.
• Mouseless Computing - This is probably my most successful resolution. I’m still no keyboard ninja, but I’ve improved over last year and even wrote a fun negative re-inforcement application.
• OSS Participation - I’m going to defer this to this year as I have a couple things planned for the near future.
• Complete small project in different dev stack - Total flop. I attempted to do something in Rails and I couldn’t get going. I think I’ve been in the .Net world for too long. New frameworks completely twist the way I do things and I find I’m not good at learning the conventions of these new technologies.

Development Goals for 2011:

Learn VIM (building my lightsaber)

I’ve tried to find the perfect editor for ages. I’ve sort of come to the conclusion that I’m going to be a perpetual generalist. Sometimes I find myself working on all sorts of OS’s (Ubuntu, Win7, Mac OSX), and I don’t want to learn the best editor for each (GEdit, Notepad++, TextMate). VIM works on all of them so why not learn that? Historically VIM has been labelled as a difficult but powerful editor and with good reason. I’m writing this post in VIM and discovering how much learning I have to do to get to the level of VIM experts.

To make it more fun I’m treating VIM like a lightsaber. It’s the weapon that every Jedi must construct themselves. Part of the lightsaber is the crystal which is imbued with the force and resonates to produce the blade. To me, the .vimrc file is my version of my set of lightsaber crystals. Hopefully with some study and careful construction (and vimcasts) I can craft my very own coding lightsaber.

Develop an Android App

I’ve always wanted to dive into the mobile development world but I don’t own a Mac so iOS was out. Windows mobile development didn’t seem that much fun, and Blackberry was definitely out of the question. PalmOS was too niche as well so I passed on that too.

Now that I have an Android based phone and I have an idea for an application (using SongKick), my goal is to release something to the Android market sometime this year. At the same time I want to play around with Ubuntu as my development environment while I’m at it.

Learn a server provisioning automation tool

This year I learned about the term DevOps, and sort of got to work under that kind of role. I didn’t do such a great job of it as I’m more a dev than I am an ops, but I want to change that. Tools like Chef, Puppet, Babushka, Vagrant all look really interesting and I would like see how I can use these tools to solve some interesting problems. Unfortunately I can’t put them in my problem solving toolkit until I actually know how to use them.

Along with these tools, I’ll probably train myself on VM technologies so I can re-provision servers and play around with these automation tools.

Move this blog

At some point this year I need to get this blog off of this application. This old version of Graffiti CMS uses VistaDB for it’s persistent datastore and the BlogML export functionality is broken.

That’s it! Hopefully I’ve kept the scope small enough that I can actually accomplish all of this. It’s strange looking at my resolution posts and realizing how little I achieve of the small goals that I set for myself. Just goes to show how fast a year goes by and how your experiences can be steered in so many directions along the way.

I would categorize myself as relatively new to unit testing. My first hands on testing was when I took the Nothing but .Net bootcamp with JP Boodhoo but he emphasized BDD style tests. I saw the benefit of BDD but didn’t feel I had the know how to implement them properly. About 3 months ago I started writing tests in a more unit test like fashion and things started clicking.

This book felt like an Assertion of what I’ve learned over the last year. As I read the book I kept finding stories I could relate to. In my attempts to apply unit testing to my own code, I experienced nearly everything Roy talks about in this book.

Chapter 5 was great a explaining the differences between the terms Stub and Mock. What I liked was how he provided samples of manual stubs and mocks to give the reader a peak under the curtain as to what a mock (err faking, Roy’s personal word preference) framework does for us. I can understand how the work mock is overloaded now, and can appreciate the proper terminology and it helps when explaining these concepts to others.

For me Chapters 6 and 7 are where I got the most out of the book. Through all my reading of testing on-line, I found it quite simple because the scope of the demos were so small. I felt these two chapters armed me with some good principals on organizing tests and treating them just as important as my production code.

All in all this was one of the easiest programming books I’ve ever read. Roy’s tone was fun (especially his little jab at Ayende), but wasn’t so opinionate that it sounded preachy. In fact I felt comfortable making my own opinions that extend his foundations. Roy presented the material in a way that seems like he assumes you’re not an idiot. There wasn’t a lot of “hand holding” which can be found in other entry level books. He provides ample resources for those that want more instruction on certain topics which creates a good pace for the reader.

As for the negatives: I wish it was a little longer. I don’t know why, but I felt that there was more to say on a lot of topics. Perhaps more examples, which could seem a bit repatitive, but could be skipped by those who got the message. I also didn’t like how TypeMock was being pushed heavily and the disclaimer that Roy worked for TypeMock didn’t come in until near the end.

I would reccomend this book to anyone who is new to unit testing. I think I read the book at almost a perfect time. I knew enough that I could see past the trivial example to the message he’s delivering, but not enough to skip all the material because it was so obvious. A seasoned unit tester might not get as much value from this book, but I would urge them to get it anyways so they can loan it to those they work with who are new to testing.

A great prelude to reading this book is to watch Roy review the test suites for the following projects:

Nerd Dinner
Asp.Net MVC
Microsoft Unity

Be warned, you’ll be watching a grumpy Roy at 2am :-). Roy also has a great blog and is active on twitter as well.

“Atkinson [ed: Bill] had just completed rewriting a portion of the Quickdraw code, making it more efficient and faster. The new version was 2000 lines of code shorter than the old one. What to report? He wrote the number -2000” – Rosenberg

I couldn’t help but smile when I read that. I have felt that some of my best coding successes are when I eliminate lots of code. This probably pushes Test Driven Development (TDD) as a great practice even further. Lately I’ve been a bit stricter by doing TDD. This means that I’m writing a lot more code, but as I get my tests passing I start refactoring and remove much of the code that’s written. For every couple hundred lines of code I write, I eventually remove 50-80% of it as I glean knowledge from all the code I previously wrote.

It appears that LOC Rollercoaster development is an artifact of TDD and it’s a great ride!

Ever since I’ve begun using NHibernate the number one thing that’s caused me a lot of headaches is learning how to properly map my domain objects. Even the most basic mappings I wrote had bugs simple because I overlooked trivial items. This made me realize that I could save a lot of time and hassle if I unit tested by data access layer.

I’m a neophyte when it comes to unit testing, but over the last six months I’ve started to understand the benefits of testing and where it fits in with my development cycle. In order to test my mappings I needed a simple way of placing dummy data into my database. I also wanted to use SQLite as my test database implementation for simplicity, and speed reasons. When a new developer jumps on board, I think it would be cool if they can run the database unit test suite without having to setup SQL Server, or any other database application server. My research led me to a project called NDbUnit. It allows a simple way to populate a database with data from an Xml DataSet. This is exactly what I wanted, but it did not support SQLite. I took the plunge and implemented a SQLite provider and even got it added to their codebase! I’ve already blogged about that tangent here.

With tools in hand I started creating an integration test suite for the CBC Radio 3 website codebase. The website is currently undergoing a re-write where I am migrating from my hand written data access layer to an NHibernate based DAL (Data Access Layer). Oh how do I wish I wrote it with NHibernate from the beginning :-). The current database has 92 tables and I’ve only just begun mapping them to proper Domain objects. So far I’ve written about 20 Domain objects and I’ve hit enough walls with the mapping files. I realized that I needed a testing foundation to assert the persistence of my domain objects.

Enough about my objectives, here’s how I plan on making a database testing suite that doesn’t get bogged down by the size of the domain or the number of repositories I have.

I’ve attached an NDbUnit demo code base that should be fully runnable within Visual Studio if you care to try this out for yourself. Note that I’ve used a lot of thirdparty tools to assist with this setup:

• Castle.Windsor
• Rhino.Commons.Binsor
• Castle.Facilities.NHibernateIntegration
• NHibernate

The first step in setting all this up is my first test which does not have any assertions. It’s existence is purely to have a simple way of creating the SQLite database file with the schema populated from the Domain.

To elaborate briefly, the first line gets the Windsor container from a Boo config file. Next, it obtains the NHibernate Configuration object from the container, then exports the schema. There’s a lot of magic being done in these few lines of code and I would like to thank all those who worked on these projects for making my life a lot easier. My TestConfiguration class is just a wrapper around some hard coded string:

The test setup is inspired by Chris Canal’s post on “Blistering Fast Integration Tests with NHibernate and SQLIte”. My Boo config files are nearly identical to his, so I’ll save the bits and not paste them here.

Next I create a RepositoryTestBase to encapsulate the construction of my container and expose the ISessionManager to all my tests.

So that’s pretty much all the underlying infrastructure for my database tests so far. Since I have a lot of tables to test, and NDbUnit requires that I model them using Xml DataSet schema definitions I didn’t want to put them all in one file. I eventually came to the realization that I could segregate my XSD files to contain only the tables that I needed to test certain Aggregate Roots (although I barely know if I’m using that term correctly). When testing my IArtistRepository, I didn’t want to have to put Blog tables in my XSD and vice-versa. I eventually came up with a directory structure to deal with separating the concerns of my database tests. Each repository would get its own directory/namespace for all the classes that it’s concerned with.

I ended up designing the system so I have three repositories that returned a BlogPost, User, and Profile. So in my integration test suite I have directories for { Blog, Profiles, Users } to group repository functionality.

In each repository directory I create a custom test context class as a base class for all other fixtures in the directory. This test context will be used to expose the repository that’s under test, and to populate the dummy database using NDbUnit. Here’s an example of the UserRepositoryTestContext

The XSD for this section contains a dataset that for the Users, Roles, and UserRoles tables and the dummy data looks something like the following:

Now I can write succinct unit tests leveraging all the infrastructure that I’ve put in place:

The User domain object is quite simple and that’s why I also have tests for a simple BlogPost class that has a child collection of Comments. Using a similar structure I can write painless tests like the following:

With this kind of segregation between the repository tests I don’t have to worry about creating XSD DataSets for my entire database in one file. I can pick and choose the tables I want to test directly using NDbUnit. For my Blog tests, I didn’t want to worry about User persistence even though it’s mapped as a property on my BlogPost class. By leveraging lazy loading I don’t need to worry that I don’t have a valid User object in the database because my tests never access the User property, therefore I don’t get any exceptions when the NHibernate generated proxy attempts to obtain that entity from the session/database.

That concludes my one week analysis of using NDbUnit to testing my NHibernate persistence layer. Please leave a comment if you have suggestions on this setup. So far this setup hasn’t caused me too much grief, but it’s only been in use for a few days. On the surface it seems clean to me, but take it all with a grain of salt.

Because these tests are meant to be automated, I wanted to be as easy as possible to get the test framework into a consistent state. I found tool called NDbUnit and it allowed me to dump data stored in Xml straight to your database. It was looking great until I found out it didn’t support SQLite which is the database engine I want to use for the testing because of speed reasons. At least the tool is extensible and I took some time to write a SQLite portion to the codebase.

Anyways, enough about what lead me to this post. During the development of SQLite integration I came across a weird bug. A feature of NDbUnit is to allow a refresh of the data, so you can merge/update/insert new data without having to completely remove everything from the test database. One of the parts of this code is to compare the data set from the xml file to the data in the database. To find out if an UPDATE needs to be done versus and INSERT is by the return value of this method:

protected bool IsPrimaryKeyValueEqual(DataRow dataRow1, DataRow dataRow2, DataColumn[] primaryKey)
{
    if (primaryKey.Length == 0)
    {
        return false;
    }

    for (int i = 0; i < primaryKey.Length; ++i)
    {
        DataColumn dataColumn = primaryKey[i];
        // Primary key column value is not equal.
        if (!dataRow1[dataColumn.ColumnName].Equals(dataRow2[dataColumn.ColumnName]))
        {
            return false;
        }
    }

    return true;
}

It simply goes through all the keys of the table, and if they are all equal, then we’ve found a tuple that needs to be updated. Seems pretty straightforward, and not a method that I would presume would cause any problems. Strangely enough, here is where some strangeness started to occur. This method would always return false! I hooked up the debugger and inspected the rows and everything seemed normal. Until what seemed like the 30th time; I noticed something. The row from the XML had 32 bit IDs while the row from the SQLite database had 64 bit IDs.

and here’s what that DataRow object was being compared to:

I didn’t want jump to conclusions as to whether or not I found my problem so I wrote a test. Sure enough, the following test fails (and for good reason).

[Test]
public void One_should_equal_one()
{
    object _32bit1 = (Int32) 1;
    object _64bit1 = (Int64) 1;

    bool result = _64bit1.Equals(_32bit1);
    
    Assert.IsTrue(result);
}

Since SQLite always uses 64 bits for its integers I simply made a copy of the XSD file (which defines how the XML data relates to the database schema) and replaced the xs:int id declarations with xs:long data type.

The End.

Not wanting to over do it, I made goals that are easily attainable. Hopefully upon completion I’ll get more ambitious the next time around.

Development Goals:

Lately I’ve been spending a lot of time thinking about “the build”. Strangely enough, I get a lot of satisfaction out of a useful build environment. When I can checkout some code and be able to build and run a local instance of an application with a single command line execution, I feel great sense of value from the work put into my build scripts. These thoughts have heavily influenced my technical aspirations for 2009.

Learn A New Build System

I Love NAnt, but I’m really starting to hate XML. It’s not that I want to ditch NAnt because I know it well, and it does the job. It’s just that I think it’s could practice to see how the same thing can be done in other ways.

• Boobs Bake - Written by the great Ayende and uses the Boo language as its syntax.
• PSake - Written by James Kovacs and is written in PowerShell. I’ve yet to get into PowerShell and a build system seems like a natural place to start learning it. The one downside is that I don’t believe it’ll work in a Linux environment (or any environment without PowerShell for that matter).
• Rake - Ruby is another language I would like to learn and build scripts is yet another applicable task for it. Rake has a fairly large user base and it’s cross platform so it’s a strong contender.

I don’t really need to learn any of these considering I’ve accomplished quite a bit with NAnt (You can compile and deploy the CBC Radio 3 site in one command). Still, if I can get away from the XML build files tasks would be more concise and easier to extend.

Implement a Continuous Integration Setup

I’ve mentioned how much I love the one line build execution. What I hate is when that one line build execution breaks! I don’t know a whole lot about CI, but just the ability to have an automated build health check is reason enough for me to implement CI. Automated testing, and deployment are just bonuses. The only one I have some knowledge about is CruiseControl.Net written by the folks at ThoughtWorks. If it’s good enough for Martin Fowler’s team, it’s good enough for me.

Change Version Control System

Subversion has changed my life. I wish I had used it (or any versioning system) when in University. In fact, that’s the first thing I would recommend any new Comp-Sci student to learn is some sort of version system. It’ll save you so much hassle in the long run, and you’ll be versioning ninja once you get out into the workforce.

I’ve been reading about distributed version systems and like the idea of decoupling my repository from a server. I recently found a great article comparing other systems and there are two other systems that I want to try:

• Git - Written by Linus Torvalds, this distributed versioning system is quite powerful and has lots of mainstream support. I’ve seen discussions where some say Git is a bit too difficult, but once understood is extremely powerful.
• Bazaar - A distributed version system with a focus on usability. The one interesting note on their website is that their code base has over 10,000 unit tests! Whether or not I use it, I’m sure the code base would be an interesting read in how to write python applications.

Life Organization:

Bookmarks

I still have great pain in maintaining my browser bookmarks. I use Delicious, and love the FireFox integration, but now that doesn’t matter now that I’m using Chrome now. I’m still brainstorming ideas and may post later about how I’m going to move forward.

Calendar

One of my favourite features of my phone is how it syncs with my work Outlook calendar. The problem is that I’m not at work all the time (wait, that’s a problem?). I want an ubiquitous calendar. When I refer to a calendar, I mean the one calendar to rule them all and in darkness… oops, I mean… When I say I’m doing something on a particular date and time, I want my announcement cascaded to all calendar applications I use. Notice I didn’t say how I announce an appointment. I don’t want to limit myself to adding events in Outlook, my phone, or my Google apps calendar. So I’m on the lookout a calendaring strategy that allows me to add events to my “abstract” calendar. Good luck with that, eh?

Test First:

The BDD style of testing is inspired (and pretty much copied) from JP Boodhoo’s Nothing but .Net class.

Here’s the code:

That was my first go at it. I have a feeling I might generalize the class to be a delimited string builder. Which would be nice because then the method won’t be static. Anyways, just thought I would share something that I think is concise but will be used a lot.

1 - URLs must be as beautiful as possible

I really don’t need to say much about this because Mike Schinkel has already said it all. We’ve been contemplating the use of SWFAddress as a method of deep linking because the new Radio 3 website is wrapped in a FrameSet therefore you don’t see the URL change in the Address bar of a web browser. I know how well deep linking solves this problem, but I still have a hard time looking at a URL with the hash (#). I personally would never use it for a personal website because that 1 character of noise is so distracting to me.

Secondly, I believe urls should be guessable and hackable. Stackoverlow has some clean urls but they aren’t perfect. They are somewhat hackable, but what’s with have the database key in the url? this makes urls impossible to guess. I really wish my profile URL was: http://stackoverflow.com/users/scott-muc instead.

2 - Web pages must be constructed with semantic markup with no display decoration

Seperating the design from the markup can be difficult but when done correctly it is a thing of beauty. The introduction of micro-formats has made me see a huge potential in the markup as data and not about display. I’m extremely impressed by Corkd because you can actually use their website markup as a database for their wine reviews because of their usage of the hreview microformat.

Along with that I believe the markup should contain zero inline CSS and javascript. Easier said than done, but I simply believe that CSS should belong in the stylesheet files (or in the page header in the worst case). Inline boot strapping javascript I find ok, but using inline javascript to handle click events is really messy.

3 - No browser specific CSS and/or hacks

The first time I learned about the Star Hack I was in heaven. I finally felt I could make IE 6 work. Unfortunately this hack came with the cost of making CSS maintenance difficult. It takes more time, but having a design that requires the minimal amount of IE 6 only hacks the better. Or better yet, don’t do any IE 6 hacks and just use progressive enhancement and don’t fret about IE 6 looking bad in some situations. It’s simply not worth the time because we all have better things to do than support a nearly 10 year old web browser.

4 - No flash or objects on the page

Flash creates a load delay that’s an order of magnitude slower than a regular text only page load. I do like flash and a lot of the features it provides, but I think the best usage of flash is to only load it when the user wishes to use a feature that requires flash. The embedable music players and the embedded youtube videos are the worst offenders in my humble opinion. These can easily be handled by loading of the flash on click events.

5 - Don’t let your server side application framework take over your markup

The majority of my web experience is with Asp.Net and I really don’t like how some of the controls create horrible markup. I’m a big fan of the Repeater and ListView controls simply because I have 100% control over the markup they create. I can’t say I’ve ever used GridView in a production web site.

Here’s some code from around 3 years ago:

Here’s what the code I used to replace it:

Setup was very simple. I created a JS Build project in my build directory. Inside it I reference the .js files I want to include/minimize/concatenate. I select the folder containing the scripts and attempt to run the build.

My first attempt results in the following error:

An unhandled exception occurred while building:

System.Exception: Error: JSMIN unterminated string literal: 10
 at JSBuild.JSMin.action(Int32 d)
 at JSBuild.JSMin.jsmin()
 at JSBuild.JSMin.MinifyToString(String src)
 at JSBuild.JavascriptFile.get_Minified()
 at JSBuild.JavascriptFile.MinifyTo(String target)
 at JSBuild.ProjectBuilder.Build(Project project, String outputDir)
 at JSBuild.ProjectBuilder.Build(Project project)
 at JSBuild.MainForm.Build()

I tried to google the error but could not find anything. To troubleshoot I added 1 javascript file at a time and found that there were a small group of files that caused this error. It turns out that JS Build chokes on multiline strings that use the ‘' character to have the string declared on multiple lines in the source code.

Not a big deal, I did multiline string concatenation to build the strings instead (probably implement a string builder in the future). After that, everything built fine. During the research I came across an interesting article regarding the validitiy of the syntax of strings using the ‘' character.

Now I have a simple NAnt task that executes JS Build that constructs a single javascript file for a complex javascript audio player for the future CBC Radio 3 website (kudos to Phil Rabin for the code). Next step is to figure out how to deal with thirdparty javascript libraries that the project depends on.

I’m of the camp that likes to have a pristine desktop. I personally think the desktop should never be seen, and if it is then you’re not managing your windows properly. When things go on my desktop I naturally try to clean them up. I don’t need any application launchers because I use Launchy for that.

In the past I used to use a dedicated Downloads folder. What has always happened was a convoluted mess of files that I had no idea what they were or not sure if I even needed them. Since the folder was sort of out of the way I let it get messy. Since I already know I’m crazy about keeping my desktop clean I thought it would be interesting to make it my default download location. The result is that I’ve been diligent in following GTD actions with the files! If I need the file once I’m quick to delete it. If it’s important for something later I put it into a hold folder, and the last case is that I store it for reference.

This probably all sounds obvious, but reading GTD is like reading a design patterns book. Instead of everything looking like a singleton or a factory, I see things in their context, project, and next action.