Category Archives: Octopus

TFS Build With Octopus Deploy (Part 2)

In my previous post, I talked about Paul Stovell’s Octopus tool.  In this one I’ll just a add a little bit of code and instruction for those of you using Team Foundation Server for your builds.

I’ll begin by saying that there are probably better ways to do almost anything I show here.  The intent is to give you a starting point if you would like to use Octopus deploy in conjunction with TFS.  I know I’m not the Iron Chef of build coding.

We’ve designed our environment to run QA deploys, acceptance tests and function tests nightly.  To that end, we have two TFS builds:  the continuous integration build, using the DefaultTemplate.xaml, and the nightly build derived from the LabDefaultTemplate.xaml.  I won’t go into great detail about how to set up your build environment using TFS.  The subject is covered much better in other places.  If you are a book kind of person, I like Inside the Microsoft Build Engine Using MSBuild and Team Foundation Build Second Edition (despite the ridiculously long title).  If you prefer videos, the MSDN “Using and Managing Team Foundation Server” series is actually a pretty good resource.

There is nothing special about our CI build, but our lab build has some nice little additions.  I’ve added a sequence to the workflow that looks like this:

 

<Sequence DisplayName=Deploy Using Octopus sap:VirtualizedContainerService.HintSize=860,1441>

      <sap:WorkflowViewStateService.ViewState>

        <scg:Dictionary x:TypeArguments=x:String, x:Object>

          <x:Boolean x:Key=IsExpanded>True</x:Boolean>

        </scg:Dictionary>

      </sap:WorkflowViewStateService.ViewState>

      <sba:CreateNuget DisplayName=Create Nuget for Database sap:VirtualizedContainerService.HintSize=464,22 NugetFailed=[NugetFailed] NuspecPath=[PrimaryDatabaseNuspecPath] OutputDirectory=[NugetRepositoryDirectory] TimeoutMilliseconds=[Timeout] VerboseLogging=False />

      <sba:CreateNuget DisplayName=Create Nuget for Site sap:VirtualizedContainerService.HintSize=464,22 NugetFailed=[NugetFailed] NuspecPath=[PrimarySiteNuspecPath] OutputDirectory=[NugetRepositoryDirectory] TimeoutMilliseconds=[Timeout] VerboseLogging=False />

      <sba:CreateNuget DisplayName=Create Nuget for MTA Scheduling Service sap:VirtualizedContainerService.HintSize=464,22 NugetFailed=[NugetFailed] NuspecPath=[MtaSchedulingServiceNuspecPath] OutputDirectory=[NugetRepositoryDirectory] TimeoutMilliseconds=[Timeout] VerboseLogging=False />

      <If Condition=[NugetFailed] sap:VirtualizedContainerService.HintSize=464,309>

        <If.Then>

          <Sequence DisplayName=Nuget failed sap:VirtualizedContainerService.HintSize=279,208>

            <sap:WorkflowViewStateService.ViewState>

              <scg:Dictionary x:TypeArguments=x:String, x:Object>

                <x:Boolean x:Key=IsExpanded>True</x:Boolean>

              </scg:Dictionary>

            </sap:WorkflowViewStateService.ViewState>

            <mtbwa1:SetBuildProperties DisplayName=Set build status failed sap:VirtualizedContainerService.HintSize=200,22 PropertiesToSet=Status Status=[Microsoft.TeamFoundation.Build.Client.BuildStatus.Failed] />

            <TerminateWorkflow sap:VirtualizedContainerService.HintSize=200,22 Reason=Nuget creation failure />

          </Sequence>

        </If.Then>

      </If>

      <sba:CreateOctopusRelease DeployEnvironment=[DeployEnvironment] DeployFailed=[OctopusReleaseCreationFailed] sap:VirtualizedContainerService.HintSize=464,22 OctopusServerUrl=[OctopusServerUrl] Project=[Project] TimeoutMilliseconds=[OctopusTimeout] />

      <If Condition=[OctopusReleaseCreationFailed] sap:VirtualizedContainerService.HintSize=464,309>

        <If.Then>

          <Sequence DisplayName=Deploy failed sap:VirtualizedContainerService.HintSize=279,208>

            <sap:WorkflowViewStateService.ViewState>

              <scg:Dictionary x:TypeArguments=x:String, x:Object>

                <x:Boolean x:Key=IsExpanded>True</x:Boolean>

              </scg:Dictionary>

            </sap:WorkflowViewStateService.ViewState>

            <mtbwa1:SetBuildProperties DisplayName=Set build status failed sap:VirtualizedContainerService.HintSize=200,22 PropertiesToSet=Status Status=[Microsoft.TeamFoundation.Build.Client.BuildStatus.Failed] />

            <TerminateWorkflow sap:VirtualizedContainerService.HintSize=200,22 Reason=Octopus release creation failure />

          </Sequence>

        </If.Then>

      </If>

      <sba:DetermineDeploymentState ConnectionString=[ConnectionString] sap:VirtualizedContainerService.HintSize=464,22 OctopusProject=[Project] PingInterval=[OctopusDeploymentStatePingInterval] Succeeded=[OctopusDeploySucceeded] TimeOut=[OctopusTimeout] />

      <If Condition=[Not OctopusDeploySucceeded] sap:VirtualizedContainerService.HintSize=464,309>

        <If.Then>

          <Sequence DisplayName=Octopus Deploy failed sap:VirtualizedContainerService.HintSize=279,208>

            <sap:WorkflowViewStateService.ViewState>

              <scg:Dictionary x:TypeArguments=x:String, x:Object>

                <x:Boolean x:Key=IsExpanded>True</x:Boolean>

              </scg:Dictionary>

            </sap:WorkflowViewStateService.ViewState>

            <mtbwa1:SetBuildProperties DisplayName=Set build status failed sap:VirtualizedContainerService.HintSize=200,22 PropertiesToSet=Status Status=[Microsoft.TeamFoundation.Build.Client.BuildStatus.Failed] />

            <TerminateWorkflow sap:VirtualizedContainerService.HintSize=200,22 Reason=Octopus deploy failure />

          </Sequence>

        </If.Then>

      </If>

    </Sequence>

 

The idea here is to:

  • Create three Nuget units of deployment (using the “CreateNuget” activity).
  • Create an Octopus release and deploy it (using the “CreateOctopusRelease” activity).
  • Wait for the Octopus release to finish and add any errors to the build log (using the “DetermineDeploymentState” activity).

If any of the activities fail, the build will report the failure and exit.  You will feel sadness, but at least know what happened.

I’ve positioned this new sequence between the portion of the lab build responsible for creating the lab environment (look for DisplayName=” If Restore Snapshot”) and the portion of the lab build traditionally responsible for deployment (look for DisplayName=”If deployment needed”).

Now to talk a little about the activities themselves.

All of the activities in this sequence are created using the Code Activity workflow template found in Visual Studio.

You might want to set up a solution that looks something like this:

If you set up your builds as links (as is shown above), you can edit them in the context of your solution.  This allows you to more easily use source control and see your custom activities in the toolbox while you are working on your build workflow.  To save you some time looking, I found the Microsoft.TeamFoundation.Build.Workflow DLL on my machine here: C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies.

To save hassles, the build server has a utilities directory which contains the executables called by these activities (Nuget and Octopus API).  The directory has been added to the machine path.  If you don’t like this, obviously you can alter these activities to pass in the path of the executables.

CreateNuget Workflow Activity

using System;

using System.Activities;

using Microsoft.TeamFoundation.Build.Client;

using Microsoft.TeamFoundation.Build.Workflow.Activities;

using System.Diagnostics;

 

namespace SST.BuildTasks.Activities

{

       [BuildActivity(HostEnvironmentOption.All)]

       public sealed class CreateNuget : CodeActivity

       {

              public InArgument<string> NuspecPath { getset; }

              public InArgument<int> TimeoutMilliseconds { getset; }

              public InArgument<bool> VerboseLogging { getset; }

              public InOutArgument<bool> NugetFailed { getset; }

              public InArgument<string> OutputDirectory { getset; }

 

              protected override void Execute(CodeActivityContext context)

              {

                    string nuspecPath = context.GetValue(NuspecPath);

                    string outputDirectory = context.GetValue(OutputDirectory);

                    var buildDetail = context.GetExtension<IBuildDetail>();

                    string buildNumber = buildDetail.BuildNumber;

                    string logging = context.GetValue(VerboseLogging) ? “-Verbose “ : “”;

                    string args = string.Format(“pack \”{0}\” {2}-Version {1} -OutputDirectory \”{3}\””, nuspecPath, buildNumber, logging, outputDirectory);

 

                    try

                    {

                           using (Process nugetProcess = new Process())

                           {

                                  nugetProcess.StartInfo.FileName = “Nuget.exe”;

                                  nugetProcess.StartInfo.Arguments = args;

                                  nugetProcess.StartInfo.RedirectStandardError = true;

                                  nugetProcess.StartInfo.RedirectStandardOutput = true;

                                  nugetProcess.StartInfo.UseShellExecute = false;

                                  nugetProcess.StartInfo.CreateNoWindow = true;

                                  nugetProcess.Start();

                                  nugetProcess.WaitForExit(context.GetValue(TimeoutMilliseconds));

                                  context.TrackBuildMessage(nugetProcess.StandardOutput.ReadToEnd());

                                  if (!nugetProcess.HasExited)

                                  {

                                         throw new Exception(string.Format(“Nuget creation for {0} timed out.”, nuspecPath));

                                  }

                                  if (nugetProcess.ExitCode != 0)

                                  {     

                                         throw new Exception(nugetProcess.StandardError.ReadToEnd());

                                  }

                           }

                     }

                    catch (Exception ex)

                    {

                           context.SetValue(NugetFailed, true);

                           context.TrackBuildMessage(string.Format(“Nuget args: {0}, args));

                           context.TrackBuildError(ex.ToString());

                    }

              }

       }

}

 

Essentially, this kicks off Nuget with a set of arguments passed into the activity from the workflow:

  • NuspecPath: the path to the .nuspec xml file used to configure your Nuget unit of deployment.
  • OutputDirectory:  we made this the shared server path where Octopus pulls Nuget files.
  • VerboseLogging: probably want to keep this false most of the time.
  • NugetFailed: set by this activity to inform the workflow if the activity has failed.
  • TimeoutMilliseconds:  what it sounds like.

CreateOctopusRelease Workflow Activity

using System;

using System.Diagnostics;

using System.Activities;

using Microsoft.TeamFoundation.Build.Client;

using Microsoft.TeamFoundation.Build.Workflow.Activities;

 

namespace SST.BuildTasks.Activities

{

 

       public sealed class CreateOctopusRelease : CodeActivity

       {

              public InOutArgument<bool> DeployFailed { getset; }

              public InArgument<int> TimeoutMilliseconds { getset; }

              public InArgument<string> OctopusServerUrl { getset; }

              public InArgument<string> Project { getset; }

              public InArgument<string> DeployEnvironment { getset; }

 

              protected override void Execute(CodeActivityContext context)

              {

                    string octopusServerUrl = context.GetValue(OctopusServerUrl);

                    string project = context.GetValue(Project);

                     string deployEnvironment = context.GetValue(DeployEnvironment);

 

                    var buildDetail = context.GetExtension<IBuildDetail>();

                    string buildNumber = buildDetail.BuildNumber;

                    string args = string.Format(“create-release –server={0} –project={1} –version={2} –deployto={3}

                                                 , octopusServerUrl, project, buildNumber, deployEnvironment);

 

 

                    try

                    {

                           using (Process nugetProcess = new Process())

                           {

                                  nugetProcess.StartInfo.FileName = “octo”;

                                  nugetProcess.StartInfo.Arguments = args;

                                  nugetProcess.StartInfo.RedirectStandardError = true;

                                  nugetProcess.StartInfo.RedirectStandardOutput = true;

                                  nugetProcess.StartInfo.UseShellExecute = false;

                                  nugetProcess.StartInfo.CreateNoWindow = true;

                                  nugetProcess.Start();

                                  nugetProcess.WaitForExit(context.GetValue(TimeoutMilliseconds));

                                  context.TrackBuildMessage(nugetProcess.StandardOutput.ReadToEnd());

                                  if (!nugetProcess.HasExited)

                                  {

                                         throw new Exception(string.Format(“Octopuse deploy for {0} timed out.”, project));

                                  }

                                  if (nugetProcess.ExitCode != 0)

                                  {

                                         throw new Exception(nugetProcess.StandardError.ReadToEnd());

                                  }

                           }

                    }

                    catch (Exception ex)

                    {

                           context.SetValue(DeployFailed, true);

                           context.TrackBuildMessage(string.Format(“Nuget args: {0}, args));

                           context.TrackBuildError(ex.ToString());

                    }

              }

       }

}

 

This code is nearly identical to the Nuget code.  It kicks of Octo.exe (the Octopus command line API) with a set of arguments passed in from the workflow:

  • DeployFailed: set by this activity to inform the workflow if the activity has failed.
  • TimeoutMilliseconds: what it sounds like.
  • OctopusServerUrl: what it sounds like.  Url is usually in the format: http://YourOctopusServer:8081.
  • Project: The project you are deploying as defined in Octopus.
  • DeployEnvironment: The environment as defined in Octopus (for instance, Dev, QA or Staging) that will receive the deployment.

DetermineDeploymentState Workflow Activity

using System;

using System.Collections.Generic;

using System.Data.SqlClient;

using System.Text;

using System.Activities;

using System.Threading;

using Microsoft.TeamFoundation.Build.Client;

using Microsoft.TeamFoundation.Build.Workflow.Activities;

 

namespace SST.BuildTasks.Activities

{

 

       public sealed class DetermineDeploymentState : CodeActivity

       {

              // Define an activity input argument of type string

              public InArgument<int> PingInterval { getset; }

              public InArgument<string> ConnectionString { getset; }

              public InArgument<int> TimeOut { getset; }

              public InOutArgument<bool> Succeeded { getset; }

              public InArgument<string> OctopusProject { getset; }

 

              private const string Sql =

                    @”SELECT [State]

       FROM 

       [Octopus].[Octopus].Project P

       INNER JOIN 

       [Octopus].[Octopus].[Release] R

       ON P.Id = R.Project_Id 

       INNER JOIN 

       [Octopus].[Octopus].Deployment D

       ON R.Id = D.Release_Id

       INNER JOIN 

       [Octopus].[Octopus].Task T

       ON D.Task_Id = T.Id

       WHERE Version = @Version

       AND P.Name = @ProjectName”;

 

              private const string ErrorSql =

                    @”SELECT [ErrorMessage]

       FROM 

       [Octopus].[Octopus].Project P

       INNER JOIN 

       [Octopus].[Octopus].[Release] R

       ON P.Id = R.Project_Id 

       INNER JOIN 

       [Octopus].[Octopus].Deployment D

       ON R.Id = D.Release_Id

       INNER JOIN 

       [Octopus].[Octopus].Task T

       ON D.Task_Id = T.Id

       WHERE Version = @Version

       AND P.Name = @ProjectName”;

 

              // If your activity returns a value, derive from CodeActivity<TResult>

              // and return the value from the Execute method.

              protected override void Execute(CodeActivityContext context)

              {

 

                    var buildDetail = context.GetExtension<IBuildDetail>();

                    string buildNumber = buildDetail.BuildNumber;

                    int pingInterval = context.GetValue(PingInterval);

                    string connectionString = context.GetValue(ConnectionString);

                    int timeOut = context.GetValue(TimeOut);

                    string project = context.GetValue(OctopusProject);

 

                    try

                    {

                           using (var cnnc = new SqlConnection(connectionString))

                           {

                                  cnnc.Open();

                                  using (var cmd = new SqlCommand(Sql, cnnc))

                                  {

                                         cmd.Parameters.AddWithValue(“@Version”, buildNumber);

                                         cmd.Parameters.AddWithValue(“@ProjectName”, project);

                                         int count = 0;

                                         bool keepPinging = true;

                                        while (keepPinging & count < timeOut)

                                         {

                                                object result = cmd.ExecuteScalar();

                                               if (result != null)

                                               {

                                                      string state = result.ToString();

                                                      switch (state)

                                                      {

                                                             case “Success” :

                                                                    context.SetValue(Succeeded, true);

                                                                    keepPinging = false;

                                                                    context.TrackBuildMessage(string.Format(“Octopus deploy {0} for {1} successful.”, buildNumber, project));

                                                                    break;

                                                             case “Failed” :

                                                                    context.SetValue(Succeeded, false);

                                                                    keepPinging = false;

                                                                    cmd.CommandText = ErrorSql;

                                                                    result = cmd.ExecuteScalar();

                                                                    context.TrackBuildError(result.ToString());

                                                                    break;

                                                      }

                                               }

                                               // Wait one interval

                                               Thread.Sleep(pingInterval);

                                               count += pingInterval;

                                         }

                                  }

                           }

                    }

                    catch (Exception ex)

                    {

                           context.SetValue(Succeeded, false);

                           context.TrackBuildError(ex.ToString());

                    }

 

              }

       }

}

 

This code is a little bit of a hack.  It polls the Octopus tables to determine if the deployment has finished.  If the deployment fails, it send the error to the build log.  My guess is that this will be replaced by an Octopus API call in the future.  (Update below)* If the table structure used by Octopus changes in future releases, obviously the SQL in the activity will need to be altered accordingly.

Arguments passed in by workflow:

  • PingInterval:  How often the code polls the database.  I am impatient so I’ve set this to 1000 (milliseconds).
  • ConnectionString: This is an ADO connection string used to access your Octopus database on your SQL Server.
  • Timeout: in milliseconds.
  • Succeeded: set by this activity to inform the workflow if the deployment has succeeded or not.
  • OctopusProject:  The project as defined in Octopus.

I think that should wrap it up for now.  To summarize, this post is intended to aid teams who wish to integrate Octopus deployments into their TFS builds.  It covers the workflow changes needed in the LabDefaultTemplate.xaml, and the code for several workflow activities which enable Octopus integration into the build.

– Update: Paul confirmed that the Octopus API will be updated

“This (build status) information is available via the API – I’ll extend Octo.exe to let you pass a –waitForComplete flag. It can even return an error code if the deployment fails.”

Thanks Paul!

TFS Build With Octopus Deploy (Part 1)

I’m using Paul Stovell’s shiny new deployment tool called Octopus on my current work project. I think it’s safe to say, for deploying a site to IIS, it beats the crud out of anything else I know . If you haven’t heard of it, the basic idea is to use Nuget packages as units of deployment. A site on IIS receives instructions to deploy releases from either a web page or an API.  A machine receiving the deployment (a tentacle) receives a push of the Nuget package(s) containing your application. The tentacle runs the Nuget package(s), deploying your functionality.

For proponents of Continuous Delivery, this is one of the missing pieces for getting a TFS shop to one touch deployment.

I’ve been using this technology in conjunction with the TFS continuous integration server, and I have to say that I’m very excited about the results so far.  To be clear, from the moment we brought TFS in house I’ve had a love/hate relationship with it.  I love that EVERYTHING is integrated and can be audited and automated when using TFS.  I can trace every workitem to the code that resulted, to the tests that were created and to the build that fixed it. …I hate just about everything else.

I’m definitely not a big fan of the Team Build process.  The way I see it, Microsoft took MSBuild which doesn’t totally suck, and wrapped it with Workflow Foundation – which does.  I’m sure many would disagree, but I they probably haven’t been scrambling for the past week to pull together bits and pieces of Team Build instruction from all over the internet.  And they might not have had their IDE crashing constantly and inexplicably when trying to edit their build workflow.  They also might not be big fans of TDD which, as far as I can tell, is close to impossible when writing Team Build code.

In the interest of saving others a bit of that heartache, I’ll post some helpful tidbits of my own in the next blog post.  I’ve created a Team Build code activity to initiate an Octopus release creation and deployment.  I’ve created another which will monitor the deployment and incorporate the results in your build.

Sound helpful?  Read on: next blog post.