TeamCity VCS trigger gotcha

You’d typically attach a VCS trigger to your build to cause it to queue automatically when changes are detected via a VCS root, however, the trigger itself might not always fire as expected. Here’s why:

Triggers are actually more like events and therefore won’t fire for changes that were committed prior to the point the trigger was created or enabled.

Whilst this might seem obvious to most, it becomes less so when you’re disabling and re-enabling a trigger. Consider the following example:

  1. Bob pauses triggers on their build configuration to make some changes to the config; meanwhile:
  2. Fred pushes a commit to the remote, sits back and waits for his build to run
  3. Bob finishes his work and enables triggers on the build configuration once again
  4. Fred checks TeamCity and finds no build despite his change being listed as a “Pending Change” on the build config
  5. Jim pushes a new commit and suddenly a build queues up with both Fred and Jim’s changes

As you can see, it’s relatively easy for teams to get into a situation where their builds don’t trigger, but they can’t immediately see why.

In these situations the only options really are to queue up a build manually, or wait for a future change to fire the trigger. Given this, I’ve started checking pending changes after activating a build’s triggers again to see if I need to manually kick off a build to avoid confusion.

Git: Zombie tags from hell

I love Git. I was very much a TFS purist up until the last year or so, but after using Git for a while now, I’m hooked. Whilst TFS certainly isn’t bad, I love the flexibility and the fact I can muck about to my hearts content in a DVCS without affecting anyone (unless I choose to). That doesn’t mean I love everything though, one of my biggest headaches recently has been tags.

Tags in Git are simply textual markers against specific commits, very much like branches (although tags don’t move). Generally they’re used to mark points in your commit history (i.e. versions etc), but they can – being just text – be used to mark anything really.

So what’s the problem? Well, the issue isn’t with tags themselves, it’s more with the fact they’re very, very annoying to kill. I purposefully say kill and not delete, because deleting a tag is actually very simple, it’s making sure that tag stays deleted/dead that’s the problem. Once a tag has been pushed to a remote, it is automatically retrieved during any subsequent pulls, at this point the tag exists in the user’s local repository until they choose to delete it. But it also means that said user can also then re-push a single tag – or even worse, all their local tags – to a remote when doing a push.

To illustrate this, consider the following example:

  1. Bob creates the tag ‘foo’ against a commit and pushes this to a remote
  2. Fred pulls from the remote to update his local copy of a branch he’s working on with Jane
  3. Jane does the same as Fred, meaning both Fred and Jane now have Bob’s tag ‘foo’ in their local repos
  4. Bob realises that ‘foo’ is a terrible name for a tag and wants to delete it, so he deletes the tag locally and pushes the delete to the remote
  5. Fred has written some more code he wants to share with Jane, so – using his handy Atlassian SourceTree app – he pushes his changes. But; Fred doesn’t 100% understand what he’s doing, so he checks the “Push all tags” checkbox when doing his push, thus re-pushing the tag which Bob just deleted!
  6. Bob does a further pull later that day, and lo and behold, his poorly named tag ‘foo’ has reappeared in his local repository!
  7. Jane pulls again and also pulls down the once deleted tag, much to Bob’s annoyance

Alright, so the answer here is probably to give Fred some guidance about how to use Git, but it does illustrate the real life problem I had recently, the only difference was that in my situation there were lots more developers involved.

So how do you solve this? Well, the Git documentation states that – by default – tags are not pushed unless you specify otherwise (using the “tags” switch). So actually, Git does a pretty good job of preventing people from automatically sharing all their tags, but tools like SourceTree, which provide a seductive “Push all tags” checkbox when pushing – are clearly too tempting for developers to pass up:


Another option – and probably a better one than just relying on your developers to not push their tags – is to implement a pre-receive server side hook to check whether tags are being pushed and reject the push it the user doesn’t have permission. Unfortunately, if you’re using a hosted Git repository like GitHub or Bitbucket, you’re out of luck as they don’t currently provide the ability for you to edit hooks and unfortunately don’t allow you to apply permissions to tags. This is a real shame as the functionality is already there to control access to branches.

Of course, the final option is just to accept that tags are not really meant to be deleted after they’re pushed, which is all well and good if you and your developers understand this from the start, not so good if not!

TeamCity Artefact Dependency Woes

A key feature of TeamCity is the ability to define properties in a build. TeamCity implements a form of inheritance for properties. If you add a dependency to another build, you can reference properties on the referenced build just as if they belong to the build you’re configuring. This is extremely handy when you’re implementing any sort of pipeline as you can allow the source build (i.e. the build which triggered a build chain etc) to define a set of properties and then simply re-use these properties throughout your subsequent builds.

There is a gotcha in all this however, and it’s as follows; your dependency configuration is absolutely critical to being able to reference inherited parameters. By this I mean, if – when you’re configuring your dependant build configuration – it cannot resolve a suitable build at the time you’re configuring it, it won’t allow you to reference the dependency parameters. This is probably best illustrated with an example:

Say you’re defining a new build configuration for your new application Foo:


You define a couple of properties on your configuration, one to store a semantic version number and another to store the build configuration (e.g. Debug or Release):


Lets ignore the fact that these would probably be better placed in a template which your configuration implemented. Whilst good practise, it’s not all that relevant for this example.

Now lets say you want to define another build configuration to actually deploy your builds of Foo:


Now, lets say you don’t want all builds of Foo to be deployed, only the ones which the developers have marked up with a tag called “stable”. So you configure an artefact dependency which picks up the last successful build with that tag:


I realise this isn’t an entirely realistic scenario, but not outside of the realms of possibility. For reference; a better solution is to have the developers use the “Promote” functionality to promote the build they’d like to the deployment build configuration.

At this point you realise that you want to reuse the semantic version property you defined in your Foo build configuration so that your deployment builds of Foo show which version was deployed. Since you’ve already defined an artefact dependency, you try and reference the property using the normal syntax (e.g. “%dep.” etc):


Now, what you’d probably expect to see at this point is a list of properties which you could reference, but instead you get the dreaded “No suggestions found” tooltip. Odd. For the sake of the example, lets assume that this might just be a UI thing and manually type out the property as you expect it to be referred to in the dependant build:


However; now when you move to the properties of the deployment build configuration, you notice this:


This happens because TeamCity doesn’t understand how to resolve the property you’ve referenced so assumes you want to define it in this build! This is actually a very handy feature of TeamCity that means you can define all your build steps without having to think about all your properties first, but in this instance it just confuses things.

So what’s going on? Well, despite properties being defined on build templates/configurations, as far as dependencies are concerned properties actually exist on builds themselves (as in, a “run” of a build configuration). In our fictitious scenario, there aren’t actually any builds with the “stable” tag, so it doesn’t have anything to interrogate to work out which properties are available.

Lets run a quick build of Foo and mark it as stable:


And now, if we go back to the Deploy build configuration and try and resolve the property again:


Everything is good in the world!

When I first encountered this, I was utterly stumped as to why my properties weren’t showing up. It took a long time and many hours of frustration to work this behaviour out. I was so convinced that this wasn’t intentional that I raised a bug with JetBrains. The response was that this was “by design”. Obviously this is JetBrains’ prerogative, but – in my opinion – it doesn’t make a whole lot of sense. I understand that properties aren’t all defined in the build configuration or template itself and that some are only actually present once you run a build, but it seems like at least displaying the list of properties which are defined for the user to select would have been a half-way-house and avoid this confusion.

Exploiting Config Transformation For Non-Web Applications

I’m currently working in a team which is building a new website for a client comprising of a couple of ASP.NET MVC/Web API applications, a handful of Windows services and some SQL Server databases. The application is being developed in an enterprise environment, so naturally various aspects of application configuration need to be adjusted depending on whether we’re releasing to a development environment, or something further down the line (i.e. staging, production etc). The team uses a relatively large number of different environments, which is not uncommon in this sort of scenario, and we needed a way of being able to adjust configuration easily without the need for manual intervention. Everything is being pushed out to the various environments using MSDeploy/Web Deploy, so it was important to find a solution that played well together.

Because this is a system built predominately using a Microsoft stack we had the ability to make use of the extremely handy Web.config file transformation syntax built into the Web Application project type in Visual Studio. If you haven’t used this before; essentially it allows you to define a sort of delta (i.e. the transform file) which gets applied to your Web.config file at compile time and allows you to change/add attributes and elements. Each transform file is tied to the project configuration, so essentially you can have as many transform files as you do project configurations. Sound good? It is; but – and this is a big but – it’s only supported by the Web Application project type! Meaning any other project type (Windows Service for example) can’t make use of the feature. Although other project types also use configuration files using the same .NET config file schema, they seem to have been ignored. Because the system being developed consists of a number of Windows services, this presented a problem because it meant we would need an alternate approach for modifying configuration for non-Web Application projects.

Before we continue; it’s important to note that this actually isn’t a new problem and a number of people have already posted solutions online. One of the notable ones being from the .NET Web Development and Tools Blog with their post XDT (web.config) Transforms in non-web projects. There are some gaps (noted by the author) however, and in general I wanted a solution more in-keeping with the approach taken for the Web Application project type, which also includes some handy features such as auto parameterisation of connection strings etc.

Almost all project types in Visual Studio are created and processed as MSBuild project files. Because of this, it actually makes them relatively easy to unload, edit and hook into to adjust the behavior your project exhibits when building and packaging etc. All it takes is a simple import of a custom targets file:

  <!-- Snip -->
  <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
  <Import Project="$(SolutionDir)\.nuget\nuget.targets" />

  <!-- Pull in our targets file here. -->
  <Import Project="..\..\..\Targets\WindowsService.Package.targets" />

Web Application projects make use of Microsoft’s Web Publishing Pipeline targets which – like most of the Visual Studio targets files – are extremely complicated and more than a little overwhelming at first. The nice thing about all the Microsoft targets files however, is that they are very extensible. Almost everything has hooks so you can execute your own targets at pretty much any point in the pipeline, this is very good from our perspective because it means we can adjust properties and item-groups at the right time to achieve our goal.

So what exactly is our goal here? Well, we want to essentially be able to import the Web Application targets and make use of the WPP to transform our configuration file in exactly the same way as a Web Application would. The first step in this process it to add our App.config file (the non-Web Application equivalent of the Web.config file) to the list of files to be processed:


<!-- The renamed App.Config file is not included as a file for packaging by default, so we need to add it. -->
    <FilesForPackagingFromProject Include="$(ProjectConfigFileName)">

By default transform files are picked up by the WPP from the bin directory, however since we’re not dealing with a Web Application, there is no bin directory, and the transform files will exist in the same root directory. Because of this, we need to adjust the paths a little to allow the transforms to be processed:

<!-- By default the transform will be looked for in the bin directory, so this needs to be picked up from the project root instead. -->
<Target Name="AdjustTransformFilePaths">

Obviously this is just a naked target at the moment, it won’t get hooked into the process yet, but we’ll deal with that later.

Now for the meat and potatoes of the process:

<!-- Parameterisation will only occur for the App.Config file, so we need to copy the transformed file to this first. -->
<Target Name="AdjustFilesForPackagingFromProject">


    Move the transformed config file (through a copy and delete) to match the project config file name
    so it's picked up by the parameterisation process down the pipeline.
    <Copy SourceFiles="%(FilesForPackagingFromProject.FullPath)"
          Condition="'%(FilesForPackagingFromProject.Filename)%(FilesForPackagingFromProject.Extension)'=='$(TargetFileName)$(_ProjectConfigFileExtension)'" />
    <Delete Files="%(FilesForPackagingFromProject.FullPath)"
            Condition="'%(FilesForPackagingFromProject.Filename)%(FilesForPackagingFromProject.Extension)'=='$(TargetFileName)$(_ProjectConfigFileExtension)'" />


        <!-- Include the transformed (and now renamed) config file so it will be picked up for parameterisation. -->
        <TransformedRenamedConfigFiles Include="$(TransformedRenamedConfigFile)">
        <FilesForPackagingFromProject Include="@(TransformedRenamedConfigFiles)" />

        <!-- Now remove the incorrectly named config file from the pacakge list. -->
        <ConfigFilesToRemoveFromPackage Include="@(FilesForPackagingFromProject)" Condition="'%(FilesForPackagingFromProject.Filename)%(FilesForPackagingFromProject.Extension)'=='$(TargetFileName)$(_ProjectConfigFileExtension)'" />
        <FilesForPackagingFromProject Remove="@(ConfigFilesToRemoveFromPackage)" />

        <!-- Move all the packaged files (excluding the configuration file) back from the bin directory for packaging. -->
        <FilesForPackagingFromProject Condition="'%(Extension)'!='$(_ProjectConfigFileExtension)'">


To break down what this target is doing, it:

  1. Moves our transformed App.config file to the transformed folder so it’s picked up for auto-parameterisation later on in the pipeline
  2. Includes the newly transformed config file in the list of files to be packaged
  3. Removes any incorrectly name config files (i.e. App.Debug.config) from the files to be packaged
  4. Alters the path for each file to be packaged to pull it back from the bin directory which the WPP insists on using

Finally; we need to import the Web Application targets and hook in our targets at the correct points in the pipeline:

<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v$(VisualStudioVersion)\WebApplications\Microsoft.WebApplication.targets" />



And there we have it. A project file which will now apply config transformation files as if it’s a Web Application.

A word of warning; at the moment this implementation is very geared towards applying configuration prior to packaging an application for deployment using Web/MS Deploy. As such, you’ll only actually see these config transformations taking place when executing the Package target of your project. While I admit this might be a limitation for some people, in this instance we make heavy use of MS Deploy to push the applications  out to our various environments, so it doesn’t actually cause a problem. I’ve made the targets file available for download, so please feel free to use it and let me know if it helps.