Pair up now

I fully believe we can deliver faster projects, with better code, exposing less risk to the company and in the end be more profitable if team members pair program.

The world was fine before

The world and businesses worked for thousands of years, even decades since computers was invented before agile methodologies took over how projects and companies was run. Projects was delivered and companies didn’t go bust.

Workable products got shipped before testing was done thoroughly, before unit testing was common and TDD was even heard of.

At the same time many teams worked well, developers wrote good code, meeting deadlines and shipping products before pair programming was a thing.

Sort of..

Were we also fine before cars replaced horses, computers replaced pen and paper, modern medicine replaced witch doctors?

Before Agile projects was delivered often late, not meeting the updated requirements, and all the other benefits with Agile. But some was delivered on time with features as desired and to budget. Just fewer and a lot slower.

And products were shipped with many bugs over and over again, with costly fixes over and over again.

Why not pair?

Double the resource cost

Many companies and managers can not see past the issue of using two people on one problem. Basically doubling the resourcing cost per story/task/whatever.

And some (not many) are aware of the teachings of the Mythical Man Month that adding more resources to a problem is not a linear reduction in time to finish the problem.

My space - my own thing

Many people and developers are also not interested in working with others on a task, especially all the time. They feel their private space violated and interfering with their work habits. I/they/most people do like to think and experiment and solve a problem on their own. And be trusted to do so.

Being forced to sit and watch, to talk to another person about every step of solving a problem is a big change in work habits and social skill requirements.

So why pair program?

3 reasons: Code Quality, Knowledge Share and Feature Speed.

Code quality

Mini plan

If two people sit together to work on a task they are quite likely to discuss the issue and basically perform a mini up-front architecture analysis of the problem. They might even apply TDD approach to solving it. In essence it should deliver a slightly more thought out solution than simply jumping in the deep end and hacking out a solution.

Human rubber duck

Explaining the problem and solution to a human rubber duck also unveils the root cause and the better solution much quicker.

Less rabbit holes

Two people are less likely though not guaranteed to go down a completely wrong rabbit hole in solving the problem. And at least quicker to realise that they are and pull out out sooner.

No short cuts

If you sit with someone you are less likely to cut corners and take short cuts in your approach and implementation. This is a very good reason for why pair programming results in better code quality. You are for example less likely to skip writing tests if someone sits with you.

Less mistakes

Simple mistakes and syntax errors are quickly spotted and corrected. Human compiler and typo spotter is often what happens but that is not really a bad thing, just don’t be offended by it.

Clean readable code

By having two people agree on the implementation then the readability and approach to the code should be cleaner and more maintainable as you both have to agree and understand the delivered implementation. This basically negates the need for extensive code review process later on as covered in my "Pair with people you like and code reviews with people you don't like" blog post

Ninja code

A lone developer often so called ninjas may also try to write smart code. Code that none else can understand nor maintain when that person leaves. A pair would prevent writing code that is unnecessarily smart. And share understanding of elegant code that is actually useful.


In the end per finished feature the code is on average more likely to be of higher quality, with more tests, more maintainable and properly solves the initial problem.

Knowledge share

One obvious benefit of pair programming is that now two people know everything about that task. And two or more now know about the product / system that the task involves.

No single point of failure

This reduces project, product and company risks and costs a lot. If one person goes on holiday or is sick there is no stoppage in working on that task or using that system as the other pair half can continue on his own.

And worse if a person leaves the company or is hit by a bus there is no panic as multiple people know about that system or feature. Ideally if a person is off for longer period or permanently a new pair is formed quickly to remove the new temporary single point of failure threat.

Best practices and showing scars

Another good knowledge share benefit of pairing up is that the two people start to share best practices in approaches to implementation or even develop new ones. They share stories of previous scars of bad practices. This should make them both better developers.

Rotate pairs - spread knowledge

If you also continuously or with some frequency rotate the members of each pair you gain even more knowledge share.

Product and system knowledge is now shared across the entire team. The risk of loosing knowledge is reduced further.

Best practices are spread around the team. Members pick up new skills and enjoy sharing their experiences. The knowledge and quality increases, and team morale should also increase.

Feature speed

Initially the feature tasks and project speed may be slower if your team members pair program. In a mathematical sense it could be halved as it ties up two members per task. And the benefits of pair programming are often more longer term.

But some implementations will go quicker as less likely to waste time on the wrong rabbit holes etc.

Speed per feature != initial implementation

In the longer term a great increase in benefit is visible. If you count bugs, planning, etc as part of the actual total feature speed, a pair programmed feature is much more likely to be of higher quality and less likely to come back over and over again with bugs. And it is bug fixes that take time.

The original implementation is often a low fraction of the total effort and time cost per feature. As Olaf describes in his "Pair programming Economics" post, implementation is perhaps only 10% of the time spent on a task.

Clean code

In addition with good clean implementation the code is more maintainable making future extension and fixes much easier and quicker.

Less procrastination

There will also be less procrastination with dealing with issues, with talking to other people, etc as two people should drive each other on to finish a task.

Less wasted time 

When working on your own you may often spend a frustrating long time trying to figure out what is the actual problem, and how and where you should solve it. Sharing that with another person often cuts that waste down as it is quite likely one knows where to look already.

Quicker ramp up and less resource bottlenecks

If knowledge of a system of feature is already known by several team members if not all then starting a new related task can be ramped up quickly compared to starting on a feature or system you have never touched before. Shared knowledge also means there is no virtual bottleneck waiting for a person to be available to work on that issue if he is the only one that knows about it.

Not religious but no tricks 

Fake agile

Many companies and teams say they are Agile but in reality they are not. “Being Agile” is not a black and white thing, it is a grey scale from non agile black to an unreachable white. Most projects are unfortunately quick dark grey even if they think and say they are Agile. Simply just using Atlassian’s Jira, having a daily standup, even some sort of iteration does not make you Agile. It is a lot more of fully adopting the practices and continuously try to come more agile.

Fake pair programming

The same goes with pair programming. Many say they do some sort of pair programming. Simple occasionally sitting together for a little while is not enough when most of the time they work by themselves.

You need to commit to pair programming being the default convention for every tasks, for everyone. Set adaptable conventions on how to organise pairing. And when exceptions are acceptable, not the other way round.

Shared responsibility

Agile processes means not delegating and assigning tasks to individuals, but in the end team members pull tasks from the queue and make themselves responsible for that tasks. If they pair program that assignment is for both in the pair not one person. If it goes well, both get praised, if there is a problem both stand responsible. Similar to how the team shares praise and criticism.

Personal space and time

Another important step is to allow for personal space and time away from the other half of the pair.  I don’t believe 100% pairing from 9am to 5pm is a good thing, no need to be in each others arm pits all day. That would cause friction and stress on the team. There is no need to be religious about it. Its a convention not a rule.

Let people take breaks from each other. Much as Pomodoro Technique allows frequent breaks from work, pairs should take breaks from each other.

They could spend that time on researching the tasks, perform mini hacky experiments. Watching another person google around for an hour is not that productive nor fun.  Allowing people to catch up with IM chats and emails, make personal calls etc will allow for a more happy and productive pair when they pair up again.

Non pair tasks

Some tasks does not need to be paired up for. Simple research, monitoring, tiny typo fixes can be done by just one person. Having a nice balance of the majority of the time is spent on important paired tasks with a few smaller non paired tasks is probably a good idea.

Effective onboarding

Taking on new team members, recruiting junior members still have a great benefit from pair programming. Sarah Mei write a great article on the benefits of "Pairing with Junior Developers". Leaving new team members to plod along fixing bugs is definitely the wrong way to onboard them.

Who to pair

Probably out of scope and can cover an entire blog post but a tip is to mix pairs. Senior with juniors. Front end with back end focused skills. Testers with developers if appropriate. And also seniors with seniors.

Remote challenge

If the team is distributed pair programming does become a challenge. However it can still be achieved. Pragmatic Bookshelf have a great book on “Remote pairing” by Joe Kutner. And a lot of tools and advice are available.

Not for everyone but for most

As I have shown here, I believe the pair programming is essential in a modern project. Sure, not pairing has worked for a long time and will still work. And for some specific teams and people it is still fine not to pair.

But pairing works better with most teams. It reduces risk, it builds team culture, increases actual velocity, makes the team enjoy their work and in the end deliver much better products.

Just approach pair programming with some common sense. It is not a factory, but don’t trick yourself into a halfway house either.

Paper Cuts and Broken Windows

It is the little things that matter

We are often reminded of the facts of keeping focus, deliver a minimum viable product, lean startup principles, only invest in what returns business value and so on. Whilst all these are definitely true, but to some extent they are also false.

Delivering business value, concentrating on one task is a virtue I do insist on, however we should not abandon all other tasks and common sense. Secondly it is actually very important we do the low priority and the little ROI tasks as well. Why? Because of Paper Cuts and Broken Windows.   

Note: This may apply to other professions and aspects of life in general but I am only thinking of IT development projects.


TL;DR: Fix your bugs asap, evolve systems continually, keep code clean and take pride in it

The problem: Paper Cuts

Many little bugs that in total hurts a lot.

A paper cut is a simple bug or an outstanding minor feature that is deemed not important enough to fix/implement right now. Some may have best intentions of implementing it soon/eventually but we all know from experience that it is most likely to never happen. 

Over time these paper cuts adds up, soon everything you touch is already full of bugs, painful procedural steps or legacy code, and in the end your team and project may start to venture close to death by a thousand paper cuts or at least feel like it. 

The problem: Broken Windows

If something is already broken, it does not matter if you break something else

Similar to paper cuts, broken windows is when a system/project/code base has some or many things wrong with it so that the current maintainers no longer feel any pride in it. It is then simple to not fix things, to ignore bugs, poor code etc.

If a continuous integration server notification of a broken build or a Nagios alert occurs for that system, people are more concerned about getting rid of the notification than actually fixing the root cause.

In an application suffering from broken windows any new feature will not be done to their best possible standard or refactored for cleaner more maintainable code.  It will not be fully tested, in fact may introduce a few known bugs and limitations that will be ignored.

Also if some epics are never completely finished, it may leave an impression that you do not need to finish stories and epics properly and as such broken windows of missing features start to be an accepted practice.

Combined problem

Essential paper cuts and broken windows are about the same. Paper cuts are how much it is hurting you by slowing you down and making people resistant to do anything with it. Broken windows is about lack of pride and willpower to make an application better as it feels already partially broken.

Or worse, if your customers start to have the same paper cuts and broken windows experience then they will simply stop using the application or even your company.

Derivative problem: The costly rewrite

A system suffering from Paper Cuts & Broken Windows will eventually be so despised by the developers that they will campaign for it to be rewritten from scratch. Which leads to a long period of delivering no business value, just pure expense. Refer to my post: Do Not Rewrite. 

But the new system will eventually also quickly suffer from paper cuts and broken windows if the processes are the same.

Solutions?

There is no right and wrong way to do prevent this. Eventually it is inevitable that a system will suffer from paper cuts and broken windows. However there are several ways to minimise the risk of it happening quickly, increase morale to reduce its spread and significantly delay the rot until the system has been used for so long that there is so little business value left and that it can be shut down.

Prevent death by paper cuts

Fix bugs ASAP

You cannot fix every bug and implement every feature as you will be trying to reach the impossible perfection which would cost an unlimited amount of time and money.

However the cost (in time and therefore money) of fixing a bug increases by magnitudes the longer you leave it, due to the costs of context switch, environment setup, remember data model/logic flow, knowledge share etc. A bug fixed straight away due to a very short feedback loop is invaluable, a bug fixed the same day or next day does take a little time but not much at all. Any longer than that it does become a real exponential costly tech debt.

So try and just fix bugs straight away or at least whilst still on the same specific task. It will make life so much more comfortable. Pushing out buggy features should be deplored.

For minor tasks and bugs that are still skipped or found at a later date they will still cause paper cuts.  A thorough automated and manual QA procedure will reduce this but they will still occur. 

Fix old bugs continually

However I have found continually picking up a few of these left over bugs and tasks will slowly reduce the amount of them on the backlog and therefore help prevent too many paper cuts.

When finishing a large complex story, try to pick up 1 or 2 bugs and minor tech debts before the next big story is started. If you finish another feature an hour before lunch, meeting or end of the day then instead of half starting another story fix a quick bug instead. This continuous self healing done in what is usually mostly lost productivity time will be invaluable over time. It is a beneficial minor procrastination.

If it turns out that bug/feature is a much larger task than you have time for now then simply just add that information to your issue tracker so that at least you have done some backlog grooming that makes it future planning easier. It may then be a valid candidate for one of the few paper cuts that is never fixed, but at least now with more data.

Development principles

Another way of preventing paper cuts is obviously to never introduce them in the first place. A bug free system does not exist, but you can reduce the frequency and impact of them by applying good procedures, architecture and code style. 

Design a system as simple as possible and writing code that is clean ie KISS and SOLID principles, so that a method, class, application or system only does one thing and is very easy to understand. This significantly reduces the risk of introducing unwanted secondary effects and greatly enhances the ease of maintaining it. 

Applying functional programming techniques such as avoiding mutability of objects and removing state where it is not needed will also reduce risk of unwanted secondary effects or architecture issues when scaling a system.

Designing systems and features using TDD reduces risk of bugs, ensures test coverage and also avoids implementing unnecessary features.

Introducing continuous integration delivers a very quick feedback loop so secondary effects are found whilst the developer are still aware of the context. Avoiding time spent on unintegrated feature branches is also advisable (ie max hours, never days). 

Prevent lack of pride due to Broken Windows

Reducing paper cuts will prevent the broken windows feeling. Applying the principles mentioned above (KISS, SOLID, immutability, TDD, CI, etc) will reduce the risk of broken windows. And by insisting on the same principles of good code standards and sensible processes people will take more pride in their work.

But broken windows will happen. Due to unpreventable technology evolution, staff turnover and lack of knowledge transfer or just plain mistakes. 

How do you fix these “windows” and when?

However windows can be fixed. And with more and more windows fixed people take pride in their systems and their work again.

Embrace automation and continuous delivery. If most parts of the process of maintaining a project is automated then any barrier to take the time and energy to fix a broken window will be very low and much more likely to happen voluntarily. 

I wrote a post about a project I was on where we transformed most of what we routinely did into one button clicks on our CI server. The returned value of that investment was great, most tasks was no longer a chore, removed many bottlenecks, the risk of process typos or forgotten steps was minimised and business value feature delivery time was reduced immensely.

A good practice is to apply the same principle of picking up and fixing a broken window issue as you do with bugs, ie as soon as possible or just after finishing another story.

Another practice is to fix issues as part of story, especially if it touches the same areas. If for example a deploy often fails on a server, when you deploy to that server just fix it properly. Do not continue with your head in the sand. 

With these you will slowly mend your most of your broken windows.

Little known time sinks  

It is the little used and little known applications and features that becomes major time sinks when changes are needed. And as such is also the parts the team will avoid to fix. Try to prevent these from becoming this by always evolving architectures and killing features.

Evolve systems

Create your architecture from the start or bit by bit of an existing system into modularised projects and components. That way you can refactor and improve one small part at the time without big and long costly rewrite committal periods of no business value, as detailed in Do Not Rewrite. 

Do not leave old systems to rot with outdated technologies and ever shrinking pool of possible maintainers. Keep evolving it, especially their integration points by for example by applying Micro Service Architecture and/or Strangler Application pattern.

Trim the fat, kill features

If an existing feature is no longer really needed, then be quick to kill it. Euthanasia to unimportant or historic features will make maintenance much easier, reduce risk and speed up delivery of new business value features.

Kill entirely an old application if nearly all its original features have already been migrated away or removed.

Bandwidth

To be able to do this you need the time to do this.

You need a strong tech lead to insist on the team to fix bugs and remove tech debt continuously, and can protect his team if they are pressured to cut corners.

You need non blinkered, smart product owner and/or project manager that understands the long term value of not doing the short term features only. A PO/PM that values the velocity of feature counts only and accepts bandwidth is also used on tech debt to achieve this. The PO/PM should not micromanage tasks and tech debt, only priorities at the epic and story level.

Henrik Kniberg’s Lean from the Trenches book describes the value of feature count above any other metric, and whether you solve tech debt (or not) and how much is up to the team and no-one else.

Summary

To reiterate the TL;DR at the top: Fix your bugs asap, evolve systems continually, keep code clean and take pride in it.

Need to ask what each member of your team is doing? Then your team is too big!

In daily (or more frequent) stand ups that is common in agile/scrum based teams many practices the process of asking each and every attendee: what did they do yesterday, will be doing today and if there is any impediments. Many do this by default without considering alternatives.

I have previously blogged that I trust my team members and do not need to know in detail what they did yesterday. I firmly believe it is more productive to focus on the tasks on the board rather than the individual. If you do not trust every member of your team then you have bigger problems.

However task focused only works with a small “1-2 pizza” team. A small team where everyone knows what everyone is doing, where the scrum master / project manager have full understanding of what tasks is currently in progress if not already obvious from the board.

Once the team grows too large you tend to have too many tasks on the board and people are all working on different perhaps even unrelated  tasks. The other team members, and especially scrum master, cannot keep up with all individual and task statuses so you need to do the token ring quiz.

The ring quiz takes time, and tend to let people’s focus drift away from the stand up and not really listen particularly well to what the others have to say after a while (I am guilty of this frequently), especially if some tasks never involve certain other members.

Also the project comes more inclined to individual tasks and not pairing and swarming together on the same tasks. The risk is then some people can disappear between the cracks or loose focus and your project start to become less efficient and agile.

So if you find you stand ups are leaning towards individual “what-you-did what-you-will-do any-impediments” interrogations then it is a strong smell of having a team that is too big.

(“1-2 pizza” is an analogy of how many can share large pizzas together. In my case 3-5 members is a good size, 8-9 is too big, and more than that is simply not agile)

Continuous Deployment via Stack Overflow

I do have an interest in Continuous Deployment/Continuous Delivery. Continuous Deployment goes beyond Continuous Integration and automatically deploys builds to production all the time, not once in awhile. It goes well with agile thinking.

With frequent deployments the delta that can go wrong is very small, the feedback is very quick and the knowledge is fresh and active with the developers so fix turnarounds are immediate. By relying heavily on automated integration testing and DevOps that provide automated deployment the result is quick and painless.

This week I answered a difficult question on Stack Overflow. The question by Emwee was for advice on how to use Continuous Deployment with multitude of inter-dependant systems.

It is a tricky question and I did not have an exact answer. My answer was more along the line of how to facilitate an easier deployment by making the dependency and coupling looser and roll-outs smoother.

I referred to Duck Typing, Feature toggles, evolution scripts, version tables, symlink stage & deploy, and referenced a Hacker News discussion on how Amazon deploys their systems, a video on how Netflix builds releases in the cloud and how at IMVU they deploy 50 times a day.

I also referred to Humble and Farley's book on Continuous Delivery.

There was another reply as well by EricMinick referring to his previous answer to a similar question. Eric does in detail describe scenarios of using promotion builds into different isolated test environments and suggest solutions that his company UrbanCode provides in relation with Jenkins.

In the end Continuous Deployment is a great evolution. But with large enterprises you need to keep your tongue straight. But it is worth the investment.

Agile project tools for personal/open source projects

Been briefly assessing some online free tools for agile task planning for a few personal FOSS projects.

A physical task board is perhaps the suggestion from the agile purists, however not useful for me (nor my family :)).

At work I often have to use the awful Quality Center. It is good for planning functional testing, but not much else. The user interface is painful, and only works on windows with IE.

But most project I have been on eventually drop it for the more developer friendly Jira by Atlassian. Its UI gets cleaner and cleaner. And is great for Scrum projects since the intergration of GreenHopper. It is however very feature rich which is good and bad, and sometimes quite slow. I recommend Jira for distributed larger organisations. It is however an overkill for my needs.

I have been using Pivotal Tracker for some of my projects for a few years. It is a great tool. For scrum projects it is the tool I would recommend the most. They recently started charging but it is still free for public projects. It is however very iteration/scrum centric and as such not useful for my more Kanbanish time irrelevant requirements.

So I started to look at more tools (and revisit some previous ones).
My requirements are:

1. Free, as in beer or near enough. $9/month and similar is too much for personal projects unless heavily used.


2. Agile task board simulation


3. Not time iteration based


4. Simple functional UI, but not ugly


5. Icebox feature for storing tasks/ideas not yet ready for the backlog


6. Pivotal like Feature, Chore and Bug classification


7. Limiting WIP


8. Kanban queues


9. Simple T-shirt or fibonacci estimates

Not all requirements have to be met.

Here are my initial impressions:

Pivotal Tracker

Time iterative centric.
Looks nice. Clean interface.
No WIP limit.
No kanban queue.
Got Icebox feature
Got Feature-chore-bug classification.
Fibonacci estimates.
Unlimited free public projects.

AgileZen

Kanban style flow.
Looks nice. Clean interface.
Columns can be renamed.
Got WIP limit.
No icebox. Can rename backlog icebox and rename another column backlog.
No estimates
Only 1 project on the free price plan.
FOSS projects can apply for free usage.

Kanbanery

Kanban style flow.
Looks nice. Clean interface.
Columns can be renamed.
Got WIP limit.
Got Icebox feature
Got Feature-chore-bug classification.
T-shirt estimates.
Only 1 project on the free price plan.
No FOSS free plan.

Kanbanpad

Kanban style flow.
Clean interface.
Little confusing UI.
Got Kanban queues.
Got WIP limit.
Got Icebox (the "backlog").
No estimates.
Unlimited projects.
All plans are free.
Permissions are strange. No member can edit and public can only view. Either member view and edit with no public access, or public(anonymous) can view and edit!!

Leankit

Kanban style flow.
Seems very feature rich. Perhaps too many features.
UI a little cluttered.
Tasks seems too much like post-it notes.
Only 1 project on the free price plan.
No FOSS free plan.

ScrumDO

Scrum focused.
Looks nice.
Feature rich.
UI a little confusing.
No Icebox.
No WIP limit.
No kanban queue.
Fibonacci and t-shirt estimates.
10 project on the free price plan.
No FOSS free plan.

Flow

Kanban style flow.
Tasks seems too much like post-it notes.
No Icebox.
Got WIP limit.
Only 1 project on the free price plan.
FOSS projects can apply for free usage.


I may update this in the future when I get more impressions of the ones I use and if I find other tools.


My recommendations depends, but currently they are:

• For large commercial projects Jira offer features and reports. And can be installed inside your firewall.


• For Scrum projects Pivotal Tracker offers the most complete package.


• For Kanban projects, the it depends on your own requirements and taste, but my current favourites are Kanbanery and AgileZen. Kanbanpad's no restrictions on number of projects is also tempting

Do not rewrite

Just don’t do it

Instead evolve

Many (99.99%) of developers continually insist that whichever application they work on needs a rewrite. As in scratch everything, redesign the architecture and technology choices and rewrite every component because the current choice and state is not ideal. I used to feel and say this as well.

Any code I have written (and even worse if written by others) which I need to update/maintain will look bad after only 6 months. After 2 years it smells of bad architecture design and logic. After 5 years the code base feels like a mess, totally unmaintainable.

This is slightly because technology has moved on, and my skills or rather preferences have evolved. But it is mostly due to that over time any application has been updated due to bug fixes and new features, and the maintainance might have been handed over to other people several times, with different skill level or understanding of the architecture. This bloats the code structure and exceeds what the original and probably quite tidy structure was intended for.

Any further maintenance is more and more costly as it takes longer and/or more people to make any changes to the application. And any innovation is muffled as inspiration is dampened.

Many companies makes a good living when other companies outsources the costly maintenance of these systems.


I used to feel the need to rewrite in every new assignment but not anymore. My advice now is: Please do not rewrite


Why are rewrites a bad idea?


No value

If you undertake a rewrite of your core application it will be very costly for your business, with no initial return on investment. Take 6 -12 months of pure losses and in the end the customer (internal or external) has not received any extra value. Most likely they will have less features as scope was reduced, but rarely any new features. Future performance gains and maintainability is a plus but not of value there and then. So it is a huge hit on fragile budgets or even liquidity of smaller companies, with little to no benefit.

This is the main reason not to rewrite, the business side gains nothing from it, it is a pure expense. Any reasonable business person would not approve it. And any future relationship will quite likely be very damaged.


Never finishing

Another risk is that the rewrite takes too long and either never really finishes or is cancelled. Making it an expensive task with no value now nor in the future.

Also rewrites that span a long time usually suffers from that the decisions made at the beginning start to look outdated and wrong before it is even in production and before any future value is gained.

Evolve

However I do not mean you should not evolve your application. Do not keep status quo either. If it is an important core application, you should evolve it, but not perform a total rewrite.

Instead do it in smaller steps while adding value and features for the customer. This way your product is cleaner and up to date, but the business side does not take a huge hit. This is also much more predictable and less volatile. Which make is less likely the project will be cancelled or worse for smaller companies which could go bankrupt.


How do you go about evolving and not rewriting? This very much depends on your type of application.

(If it is a tiny application, go ahead rewrite it. Just make sure the dependant systems are aware of this).


Refactor

On a minor level, continual process of refactoring, with larger refactoring a frequent acceptable task. This should slow down code rot, postponing the need for larger changes a while.


Modularisation

Eventually your application will need a drastic change. If by chance your original design was quite modular or less coupled with other systems you made evolving smoother. If not, try to modularise your current design.

By being modular you can still perform rewrites, but on much smaller scale, taking one module at the time. So that it is not a 12 months loss, but perhaps 1 month loss. While normal work on other modules can still continue. Much less likely to kill your project/company.


Parallel systems

Instead of a hard one day switch over from the old system to the new system, instead run two systems in parallel for awhile, the old legacy system and the new clean system.


Skinny interface adapter layer

If modularised or not, a cleaner way to evolve and replace sections of your application is to introduce an adapter layer for your interface, and keeping it skinny.

This interface adapter only relays between your system/application and other systems. When other systems only talk via the adapter you are free to change your system without affecting external systems.

More importantly you can now split your system into two systems and relay certain calls to the new system instead of the old, without external systems are aware nor affected by this.

The adapter layer is not always possible, or people are resistant to introduce yet another layer/system, but it really makes your much more adaptable in changing your own architecture, you can move elements to a new system and roll back without external costs.


No logic in adapter layer

Do not ever put any logic in interface adapter. Otherwise you end up with yet another system to maintain. Unfortunately this happens quite often when bad planning and management leads to shortcuts which adds logic in the adapter layer. And then you have just multiplied your maintenance costs. And believe me that logic is then rarely temporary.


ESB

Do not interpret this adapter layer requirement as a reason for introducing an ESB. ESB tend to just camouflage the spaghetti integration code, and often introduce above mention unwanted logic. However if you already have an ESB in place it can perform the adapter layer requirement.


Legacy system becomes adapter layer

Another method implementing the adapter layer is to use the existing system to relay calls to the new system instead of handling them themselves. Internally in the existing legacy system you have to do this anyway on each element migrated to the new system, so you can also allow external system to do this.

It is however cleaner to have a separate adapter layer. This will be easier to maintain as probably designed on newer technology and platforms. And allows you switch off the legacy system when you can. But also makes so less likely for people to just decide to use the legacy system instead.


Topic / feature

An even better rewrite strategy/evolution than per module is per topic / feature. This is an even smaller grained change and less risky. As part of more agile strategies you can move one feature at the time to the new clean system.

With an adapter layer this switching is smooth, but not restricted to an adapter layer. Without the adapter you just have more administration of changing every dependant system for each feature moved.

Feature toggles might be part of this strategy.



New feature -> new system
Update feature -> new system

Every new feature request is naturally going to be implemented on the new system. But a good method of choosing which existing features to move is choosing a feature which there is a requirement to modify. Then not touch it on the old system but instead rewrite and update it on the new system. This is then a good carrot/stick way to ensure the migration is performed. And ensures that the company receive some value for each rewrite.


Do not update old(legacy) system/application

Another very important rule is not to update the old legacy system at all when each feature/module is migrated off it. It is so tempting to make a shortcut and update both system as perhaps you have not introduced an adapter layer or instructed enough external systems to use the new system. This will kill your migration/rewrite.

For this step the leadership of your team needs to be firm about and insist on. Do not end up maintaining two systems (or 3 with the adapter layer).


Ensure legacy system delegates

The legacy system will undoubtedly internally refer to the rewritten and migrate module or feature. You need to ensure the legacy system now delegates internally to the new system. This is the only update you should do on the old system. Otherwise again you will need to maintain two systems, and run the risk of the legacy system and the new system executes slightly differently for the same task, leading to people insisting on using the legacy system.


Kill the legacy system

You need to plan and ensure the old system is eventually switched off. Otherwise you will still need to keep people and skills on how to maintain that system. It may be tempting and erroneously to leave some elements behind that should be on the new architecture.

There may be some elements not needed on new system or unrelated that is left on the old system. But probably better to even move these to another new system than keep the old system lying around and increasing the cost of maintaining that small element as well.


Kill the adapter layer?

Once you killed the legacy system, do you want to keep the adapter layer? Keep in mind you might want to move on from the new system when it starts to rot as well. However it may be tidier and less complicated to kill the adapter layer as well. I would kill it and if needed in the future reintroduce it instead.


Do not stop half way

If for some reason the migration to a new system is stopped, either due to reprioritisation, lack of progress etc, then you are stuck with maintaining not 1 but 3 systems. Many companies end up burned by this and is mostly down to not strong enough leadership/management.


So my point is to rewrite small topics/features then modules, but never the whole application. This way value is introduced along the way without a budgetary black hole.


References

Many, but some from former colleague Anders Sveen

Remove deadlines, increase productivity

In a recent project the tasks were always delegated and pointless deadlines set on everything. This seems so counter productive and ineffective compared to my past 5 years of agile based projects (inc maintenance).

As for task delegation I will need to evangelise more about the benefits of self organising and ownership through bottom up delegation, WIPs, velocity projection, prioritisation benefits etc.

My current beef is about noisy deadlines. Deadlines are for product owner / project managers. The team should never need to deal with deadlines. Especially when not meeting them has no consequence, so why have them at all? The team deals with prioritised tasks and if needed time boxed tasks. The tasks are divided up so that they are small enough so that a slip of 300% from estimation has no really impact as you are only talking about hours or days. However if the project velocity overt time does not seem to meet external deadlines then the scope of the backlog need to change, not to push developers/testers harder.

In agile teams they should not confuse deadlines for an estimation of how much work is left. If your current task may seem like another 2 days worth of work, do not set a deadline in two days. You may have interfering meetings or get stuck on a small issue that extend that tasks into next week. (Or hit an coding-happy-zone or eureka moment and finish in a few hours). Setting a deadline do not make you finish earlier and pushing developers/testers harder does not improve code quality. A task based deadline is only noise and stress.

Product owners can plan, communicate, etc with dates to customers, company boards or steering groups etc but they should be soft, movable dates and based on current project velocity, and no minor deadlines. If the velocity slows down, then reduce the scope of the backlog, or move the dates. Do not put pressure on the team with more micromanaged deadlines. It will only be noise and slow down development. The team does not really need to know specific dates apart perhaps from general roadmap (lavalamp?)for big releases.

True, some dates can not be moved, such as Christmas promotions, larger entities' deadlines etc, but with enough balls you would be surprised how many customer, senior management deadlines can be moved with good communication, even completely removed. Good project transparency and communication, and thus project velocity visibility is usually a much bigger relation and delivery benefit.

I agree that some pressure of delivery should exist but no cutthroat stress inducing deadlines. An agile process of pride in delivering tasks may quickly vanquish need for deadlines.

• agile-commentary.blogspot.com/2009/06/are-deadlines-important.html


• developers.slashdot.org/story/06/09/28/1242257/Good-Agile-mdash-Development-Without-Deadlines


• steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html


• www.slideshare.net/jurgenappelo/agile-management-authority-delegation


• www.noop.nl/2009/10/why-we-delegate-the-darkness-principle.html


• www.informit.com/articles/article.aspx?p=1675546&seqNum=6


• blog.adsdevshop.com/2010/02/09/task-delegation-has-no-place-in-agile/


• stackoverflow.com/questions/63100/scrum-process-management-tips-pitfalls-ideas

Scrum bashing - natural iterative evolution

Not throwing my weight into a current Scrum bashing trend that erupted this week, as I'm a nobody when it comes to agile. But I tend to agree some of the points and disagree with others.

It all started off with Uncle Bob responding to Chris Brookins whom was looking for some of Scrum shortcomings to present at his work.

That raised a large number of responses on the thread and in the blogsphere. People such as Jeff Anderson stated that it was time for Scrum to evolve.

Others such as Jurgen Appelo came in defense of Scrum by insisting people should stop pissing on Scrum.

Issues raised

• No technical rules, ie no prescribed methods of insisting on CI, TDD etc.
  I dont agree with limiting Scrum to development only, but suggestions are fine, which they kind of already do.
  


• Sprint lengths are too long.
  They are. But again agile carrots of emphasising e.g. 2 weeks are better than bashing teams that need longer sprints.
  


• Scrum masters assume or are assumed to have to much power.
  That does happen too ften, but coaching of team members and external chickens are probably the solution.
  


• CSM, Certified Scrum Masters.
  Yup the title does not help, I even got it....,
  but dropping it may result in very few taking the courses all together. Time will give a solution to this I think.
  


• Backlog structure.
  Not sure this can be solved, because so many different type of organisations use scrum for a variety of usage, the items in the backlog are very different.
  Partially the problem here also lies in the pedantic use of manual physical task boards,which translate into difficulties in transferring stories/tasks.
  Also the tools available, e.g. Jira+Greenhopper, have poor support for Stories and Themes.
  


• Anti management.
  Scrum and its adapters have a reputation of Anti management.
  This may be true as team members often would like to introduce Scrum for the empowering of the team in the decisions.
  But once implemented it is not so much the case.
  I would actually say it is a bit pro management, once they see the results and a clearer vision of their future. Also Scum increases management due to the team's own micromanagement and constant status reporting.
  


• Automatic tests.
  It's Uncle Bob, Mr FitNesse. True, an essential part, but not always the rule.
  


• Multiple teams.
  Yes, this is one point that Scrum struggles with.
  

Future of Scrum

I think this bashing of Scrum now is a natural evolution.

The early adopters, the evangelical Scrummers, now need a new fix, and are quite vocal in Scrum's negative points. The early sceptics, also realises Scrum does not solve everything and would like to move on.

But it is natural. RUP, then XP, Lean etc are not perfect, they are just an iterative history of continual evolution and improving methodology of teams and projects.

Scrum is not a bible to follow for the next 2000 years. But now, or rather 5 years ago it was the best thing around. It has helped a huge amount of organisations become agile. Not perfect, but a lot better than before.

We should not always jump to the new shiny thing, but it is probably time to evolve, to continually improve and never rest at least not too long.


I quite like ideas suggested by Kanban. Kanban practices more common sense. As Henrik Kniberg writes in his minibook about scrum and Kanban.

But again it is not the final evolution. And everyone should adapt their needs as appropiate. There is no need for this bashing of Scrum. But neither should we not try to improve it.

Don't like Scrum. But will not use anything else! Part 4

This is "Don't like Scrum. But will not use anything else! Part 4". But really it should be: Love "Scrum", but it is not perfect. Yet.

Right, I am now a Certified Scrum Master, so I should know more about the pros and cons of Scrum, and how to reduce some of the cons/risks.


Shorter sprints. Definitely something I have learned, so 2 or 3 weeks seems ideal. I have been involved in 4-5 weeks sprints, and they work, but not so well. Longer sprints have more scope, more risk, and can end up with several developers idle for many days at the end. Shorter sprints cuts this to perhaps max one to two days if any. It is also makes you pick only a few stories so they have more focus, instead of 5 stories which end up spanning many sprints due to impediments etc.

Each developer should pick only one task, and try to share tasks. Avoid bottlenecks and teams overdependent on one person.

Use electronic tools for task board, but have it showing clearly in stand up room, ie on projector or large screen.

Dont be a jobsworth police, but try and get the group interested to be on time for standups. And for people to say nearly only 3 sentences, no need to explain in detail what they did, so less pressure to have a "plan" ready and also much quicker.

Arrange discussions for later/afterwards, if anything pops up, do not solve them there.

Encourage another voluntary standup/chat in the afternoon if standup is in the morning or inverse. This to further encourage rubberduck chat and share knowledge and problems.


As for the Scrum Master certification, I did a two day Mike Cohn course here in Oslo. He is very good teacher/lecturer, as he knows the subject, have a well prepared agenda/curriculum, can answer all questions clearly and with authority. So I can recommended the course very much.

Don't like Scrum. But will not use anything else! Part 3

I like ranting I do.... :)

Now to my third instalment(1,2) on this subject, which will regurgitate a lot that has been said before.

As mentioned before I have major issues with the Scrum methodology. BUT it is also the only methodology I would use! (At the moment).

Scrum and XP, which are being advocated by developers as the new messiah. But I think developers fail to see it will have a derogatory affect on themselves.



First of all: Stress: The constant status demands every morning, all the time on notes or JIRA tasks etc. The constant planning of your day and defending every minute used. Mostly due to the stand-ups.

By iterative development, you minimise the stress at the end of a major release, but introduce others.

True, for business, this means more effective developers in the short run. For managers, it is a good way of keeping oversight of how things are proceeding, and a good whip to keep people in line.

But for developers, yes you are more effective, but eventually you are also getting very stressed. The need to perform at your maximum all day every day will wear and grind your psyche. If you had a bad day, where not much was actually done on a specific task, you have to defend that the next morning, or simply lie. You can not cover the slack by concentrating a bit more for the next few days.

For the perfect A4 cubical worker, ala German, Norwegian, Japanese (anyone else I can offend by stereotyping?), then this rigid life will work well, but for more creative, individualistic, "agile" worker this can cause friction.

(Not sure I've mentioned I don't like the standing up in "standups"...)

But again, I can not see how you can not have morning stand up meetings, as the benefits outweighs the eventual stressed programmers....



Second: Pair programming.

Unless you are the A4 type programmer without a personality, I can not think any self respecting programmer would welcome pair programming? It is intrusive, violates your personal space, stops any creativity, lack of trust and is just smelly.

The benefits of sharing of code knowledge, the idea bouncing while creating classes and methods are great. The banter if personalities match can be good. And also distracting?

The smirks as girlfriend/wife sends embarrasing emails/IMs that pop up in the previews...

The concentration of working on only one specific sub task can be good for velocity. But also sometimes you are stuck and switching context, briefly or for longer periods can also help enormously. But can you with someone sitting next to you? Or just drag that awful day further by staring at the screen?


But again, the benefits outweighs the violation of the individual. But I'd much prefer a more limited pairing, shorter periods, perhaps not always sharing desks, etc.


Also forgetting/ditching all knowledge learning from previous methodologies seems quite frivolous. RUP may be tedious and over extended, but many bits can assist and enhance your project. Don't just ditch everything , because you have converted religiously to something else, something "new" and "exciting".


But again. Scrum works well, brings a lot of benefits, and I recommend it!

Don't like Scrum. But will not use anything else! Part 2

Following on from my general rant about Scrum, Part 1, here is Part 2.


The main beef I have about Scrum, are
* the constant status updates at stand up meetings
* the use of non electronic tools
* inflexibility on tasks

These are elements that are easily changed and is probably mostly interpretation. My dislike for these can also reflect perhaps badly on me, if taken from a superficial view.

I don't like the morning stand ups. For several selfish reasons.
* I don't like mornings.
* I am not in early. (Especially now living in Norway where developers start work 4 hours earlier than in the UK even with only 1 hour time zone difference. My last job in Manchester, developers where not in till 11am. Here they start at 7am... )
* I don't like standing up.
* My memory is terrible.
* My plan for the day have not yet been thought of.
* I don't like standing up.
* I prefer to work from home in the morning, thus actually getting something done, instead of being a zombie at a meeting.
* I don't like mornings.
* I don't like the micro-management of it.
* It is a forced meeting.
* I don't like standing up.


It is not that I don't agree with status meetings and I can see standing up keeps meeting briefer.

Many of my work places, many of the companies' main issues have been solved in frequent 5min fag breaks (the English sigarette meaning of the word) even when no-ones smokes at the place. As a non-smoker I would always attend them, and discuss a bit rubberducking about things Im working on and stuck on. So informal meetings are very usefull.

Regarding standing versus sitting, I just hate standing. And not able to fully listen to anyone else's issues.

That it is just a quick run around of what you did yesterday, what you are going to do today and if any impediements, I just don't see the value for the developer. This should be clear in JIRA or any similar tool. Sure, it is a face-to-face status update for the management, so they can micro-manage everyone's day, but really mature developers should be trusted to deliver. That you can not discuss issues, devalues any development benefit from the meeting.

There are also some cultural issues to consider. Standing is meant to make it quicker, but Norwegians once a meeting is finished its timeslot will stand up and leave the room even if nothing has been resolved, or even when senior management or customers are present.


As a technologist and computer geek, the use in Scrum of non-technology such as post-it notes, sticking notes on boards, face-to-face meetings, is a sore point. Again I see the value, but it just goes against the grain of a technologiest.


Also the focussing on only the tasks on the board, post-its on your desks, and status updates on these every dawn is quite blinkered. True, it will keep people in check and some is usefull to make people consentrate on the tasks delegated.
But also have negative effects. Such as me, whom may help too many people, which is discouraged in Scrum as you should only focus on your own task.


And don't get me started on pair programming! That I should share my days with some other smelly developers odour is quite repugant. But again I see the value of writing code, solving problems together, sharing application knowledge. I just prefer frequent fag breaks and my private space! :)


BUT let me reiterate, I would only use Scrum!

Don't like Scrum. But will not use anything else! Part 1

As the title says, I don't like the Scrum (the project methadology). But I don't think I would use anything else!

It is like Churchill or Rooseveldt said something along the lines of: "Democracy is flawed, but nothing else works!". Which is also true. Democracy is the rule of the mob and pop culture, but all other governing styles leads to chaos, elitism or despotism.


I do like a lot of the ideas behind Scrum, the agile thinking is great, the XP ways do work. Everyone seems to jump on the Scrum bandwagon taking every element as gospel, and defending it religiously. But Scrum, has introduced many elements I don't like. I can see why, and what they can achive, but some I really detest.

Unfortunetly, all other project managent styles have more flaws, so I think "Scrum matured", or some better Agile methodalogies in the future is a better solution.


For Scrum and agile there are 3 sides to view from the pros and cons to its benefits.

* Management
* Developers
* Customers

It is mainly been developers who having been pushing Scrum as it will be better for customers, hence in the end for management as they are more profitable. However the bits I don't like is mostly where the benefits are purely for the management.


For management the pros are:
* Constants status updates
* Up to date status
* Future cost projectability
* Focused costs
* Low risk of wasted development


For customers the pros are:
* Ability to direct and change requirements
* Cost transparency
* Feature and status transparency
* Final release is as required


For developers the pros are:
* Low up front documentation
* Task sharing
* Modern and new, therefor interesting
* Management and customers are open to ideas
* Iterative development, of which one benefit is you dont have to solve everything immidietly


There are other pros, but they are not specific to Scrum, more that Scrum project are Agile and open to new technologies, and other methoods etc. Such as Continuous Integration, Wiki, JIRAesque.



But the cons are for developers
* Stress, due to constant pressure to perform every day
* Loosing individuality
* Turning into factory lines
* Constant status updates
* Perfect planning every day
* Orvelian supervising
* Loss of trust
* Low priority of refactoring
* Stand ups
* Back to low technology


Cons for management
* Stress and morale of staff
* Projects fix specific problems, but leave all else untouched, increasing rot and risk of general purpose tasks
* Distributed development is tricky


Cons for customers
* Probably not that much!
* Must trust supplier
* Difficulty in estimating accurate final cost


So you can see the pros does outway the cons. For Customers there really are no cons. It is just pros. For the management, once informed and convinced, they are also mostly pros.


It is just for the developers there are real cons, and they are the most noisy Scrum advocates! I am afraid as the idea behind Scrum gets older, developers will wain of it when they realise some of the consequences. But perhaps by then "agile" people will have forseen this and adapted a more mature and compatible "Agile 2.0" methodology and processes.

Planning Poker

Attended a estimation seminar yesterday morning. Some fluff, but being exposed to the theories and practicalities of Planning Poker was worth the time.

Used it immitietly in the afternoon for a project that we were still struggling to define the tasks needed. Very effective, in just getting a discussion going, grouping tasks and assigning estimation units to each task/story.

Well worth a try, even if it made us look like we were playing poker all afternoon, especially as we sat in a tiny glass cubicle at a client.

Agile estimation book by Mike Cohn