Why we moved from Scrum to Kanban

When I joined my current team, we were a full on Scrum shop: three roles (product owner, scrum master, team), three ceremonies (sprint planning, spring review, retrospective), three artifacts (etc), the whole shebang.  And, it mostly worked, in that things got done, software got shipped.

Sprint Planning Hell

Squeezed Testing

Stories Over-running

Problems with Traditional Scrum

It assumes that everybody can do everything

It assumes that developing stories is a single task

Why Kanban Works For Us

Our First Kanban Board

WIP Limits

Advantages of Kanban

Visualisation

Continuous vs Batch

To me, Scrum often felt like a ‘batch process’ model of software development – you do these things, heave them over the wall (into production) and then do the next things.  Everything had to fit the two-week sprint cycle.  With Kanban, I feel like we’re now using a continuous-flow process – small stories scurry across the board quickly, and can be shipped when ready; larger stories take longer.  I think it’s no accident that this is a good fit with the ‘continuous delivery’ model of software delivery.

No software engineering manager at a tech company should spend less than 30% of his or her time coding. Whether managing a team, a division, or all of engineering, when managers spend less than 30% of their time coding, they encounter a significant degradation in their ability to execute their responsibilities.

My claim stands in stark contrast to what I see as the expected path for software engineers who become team leaders. At each promotion, the engineer is expected to spend drastically less time coding, with a particularly steep drop-off when they go from a “lead” to a “manager” title. At that point, they’re expected to maintain passing familiarity with their codebase. A director’s coding is done, if at all, as a hobby.

This happened to me about a year ago as more of my time became absorbed in other things, such as recruiting, managing, and strategizing; I found then that I had crossed a threshold where my effectiveness as a technology leader suffered out of proportion to the amount of coding time lost. I wrote a short post on my blog that presented my thoughts following that experience, but without much concrete detail. Here, I’d like to expand that into a more developed thesis.

Why You Should Keep Coding

Many people believe that managers should step back and concentrate entirely on strategy and management. It makes sense that managers are expected to spend the majority of their time on these things. But as an industry, we pay too high a price for our managers when we allow or demand that they barely spend time coding. Once a person stops coding for a significant portion of time, critical connections to the concerns of developers atrophy. When that happens, decision-making, planning, and leadership suffer, undermining the entire basis for promoting engineers to management positions.

Estimates

One of the most important skills in an engineer’s toolkit is estimation. Strategic planning is quite simply not possible without the ability to accurately estimate. And yet we engineers are, as a class, notoriously bad at it. So bad, in fact, that we are advised to just double whatever number we come up with when asked to estimate something. In general, it’s easy to fool oneself into thinking that things will go optimally, but if we use the concept of “estimate traction,” code appears to have a particularly slippery surface. Because there are so many ways to implement features, when you lose familiarity with the details, your estimates become even more unreliable.

Technical Debt

Another thing that engineering managers need first-hand exposure to is the impact of technical debt. These days, that term has been popularized enough that when you have to debate the priority of a new feature versus. refactoring, you have a good chance of having that debate with people familiar with the concept. Engineering managers need to be more than familiar with the concept — they are the ones directly responsible for making the judgment call as to when technical debt needs to be paid down. A manager who codes regularly will have much better information on when and how to make that decision.

Continuity of Understanding

I haven’t chosen the number 30% arbitrarily. I chose it based on my experience because it is a simple heuristic for enough time to keep up with the changes that happen in a codebase under active development. With less time, it’s easy to lose the thread of development; and once that thread is dropped, I will need to ramp up all over again to retrieve it, thereby incurring an extra time penalty.

Parity with Responsibility

As a leader, you definitely should not be making all the decisions for your team, or approving all the decisions, but you need the context and the knowledge to facilitate all decisions. In the end, you are responsible for the outcome, and your ability to sensibly make choices should match that responsibility.

Your Team Respects You For Loving Code

Let’s be clear: To be successful as a manager, you must facilitate your team members’ efforts, foster their development, and ensure they are fulfilled by their work. I’ve been writing about how to diagnose and repair issues with poor managers on my blog in a series called Debugging the Boss. But to truly excel at managing software engineers, you had better love code. Because your team does, and they will respect you all the more if you lead by example.

Obstacles to Reaching 30%

Despite of my best efforts, I have run into many obstacles trying to maintain my coding time at 30%. These include the following.

Actual Responsibilities: At a startup, there is always more work to do than there is time to do it, and even as a company scales and matures, being as effective as possible is always an exercise in managing competing priorities. An engineering manager has many responsibilities, which should take up 70% of their time. Here are a few:

• Leadership and Team Tending: This responsibility is the first to appear in an engineering manager’s career. You are no longer just responsible for your work, you are responsible for enabling your team to produce their best work. It takes time to mentor your team, resolve disputes, and think about how to optimize their environment for happiness and productivity.
• Strategy: As the level of responsibility grows, an engineering manager is required to spend more time contributing to strategic planning at various levels. Initially, this will be limited to tech strategy, but later, both product development and competitive strategy will play a large part.
• Recruiting: Managers, directors, VPs, and CTOs need to build their teams, sometimes quite rapidly. While a good recruiting staff is a help, there is no substitute for a strong leader who actively seeks out new connections and sells every good engineer they meet on how great it is to work on their team.
• Customers: As engineering managers gain responsibility, they will often become more external-facing. This means they’re brought in “pitch meetings” with high-value prospects and called on for firefighting when important customers are unhappy.
• PR: Senior tech managers devote time to public speaking engagements, writing blog posts, and (of course) articles in prestigious tech journals. No matter how much help you have with these tasks, it takes time to write, edit, rehearse, travel, and present.

Avoidable Time Losses: The responsibilities I just discussed are what an engineering manager should be spending time on. These next areas are pitfalls I’ve experienced that have been my undoing when trying to maintain a bare minimum of 20% of my time spent coding, and which still stand between me and the 30% I am fighting to return to.

• Not Saying No Enough: Achieving great things means working hard; however, growth has to be sustainable, and one of the most crucial responsibilities of an engineering manager is to say “no” when their team is over-committed or on the verge of becoming so. When you don’t say no, other people begin to dictate your schedule and time commitments.
• Meetings: An entire cottage industry exists to give advice on how to meet effectively, and justifiably so. I have wasted more time in meetings than any other single activity in my career. This is especially debilitating when you have fallen behind in hiring, and are attending meetings that really should be run by team leaders.

Failing Strategies

In my quest to regain my coding time, I have tried a number of things that have not worked.

• Sleep Less: While surprisingly alluring for me, sacrificing sleep doesn’t work. Your brain stops working and you become unpleasant to be around and much less effective.
• Read Headers Only: I thought this was promising, but in practice, reading only the headers of C++ code commits gets you very little of the benefit you need for management.
• Overspecialization: Knowledge of only one project in your overall codebase is appropriate for a team lead, but not a director or above — you need familiarity with everything you are responsible for.
• Delegating Too Much Too Early: It’s easy to make more work for yourself by delegating recklessly when your reports actually need careful mentoring.

Successful Strategies

In spite of numerous dead-ends, I have managed to uncover some successful strategies:

• Time Blocking: The percentage of time on my calendar that is not allocated weeks in advance is minuscule. It seems obvious in hindsight, but I needed to allocate blocks of time specifically for coding. In practice, these blocks are frequently rebooked, but having even 8 hours blocked out per week makes an enormous difference.
• Delegating: Delegating is tricky, especially when you have very strong opinions about how tasks should be done and the ability to do them if you had the time. There are many reasons why managers resist delegating, but every reason has to be viewed as a problem to be solved, rather than an insurmountable barrier. Nothing frees up your time for coding like handing off running a meeting to someone you trust.
• Office Hours: Something I’m planning on instituting in the near future is office hours. This technique should help a lot with the random interruptions by consolidating them into discrete time windows, during which I can work on the many tasks I have that do not require committed, long-term focus.

Final Tips

Here are a few points of practical advice for managers finding themselves approaching, but not getting across, the 30% threshold:

• Learn how to read code. It’s a different skill than writing it.
• Commit to a meeting structure and hold your organization to it. Do not attend a single meeting that does not have a defined agenda.
• Get a real machine to work on. That MacBook Air you love for meeting hopping? Not it.
• Know how to access a dev environment and test a change fast.
• Understand if you’re the kind of person who can use five 20-minute blocks. If you need an hour, then put it on your calendar.
• Remember, 20–30% is a heuristic I’ve come up with for myself. Your mileage may vary. So measure yourself (Estimate how long it would take you to try out a hot fix; Can you list the most indebted areas of your codebase? Pick a random code review and see if you understand the conversation and the choices. If you don’t, you need to dig in more).
• Categorize your work by when you can work on it and what you need to accomplish it, rather than by topic. (Advocates of Getting Things Done (GTD) will recognize this as the essential basis of their productivity technique.)
• Finally, I’ve lately become fond of getting paper homework. As backwards as it seems, printing out a spec, a list of features to prioritize, or even a block of code is often a nice balance to spending large amounts of time looking at a screen.

I hope these tips are useful. If you have other techniques that have worked for you, please leave them in the comments area.

I read this excellent article by Richard Perfect and wanted to save it, in case it get memory-holed.  When I read #12, I laughed because I have used it for quite a while

12. Explain the problem to your dog. It really doesn’t matter who you explain the problem to it’s the act of explaining that gets your brain to analyse the problem from different angles.

Here is the article

60 Problem Solving Strategies

by Richard Perfect

Our lives as programmers are a never ending series of one problem after another.

If we are lucky, our daily slog at the keyboard is occasionally punctuated with brief glorious moments where it all “just works” and we feel like gods.

If we are really, really lucky these serendipitous interludes also happen to coincide with events like “product demos” and “production releases”.

In between these glorious moments we toil through the frustration of one problem after another. We are not programmers, we are just hacking out the first solution that works.

This post will describe a bunch of different problem solving strategies. You might not have time to read it all now, so bookmark it for later.

Share this post with your team if you find it useful.

The Prime Directive

1. Don’t write defects in the first place. The best defect to fix is the one that you never wrote to start with.

• Nicely unit tested code
• Enforce database constraints
• Use an input validation framework
• Avoid unimplemented “else” conditions
• Understand how to use something in isolation before you use it in the main application
• Pepper your code with exceptions for situations you don’t expect to happen

Logging

2. Print Statements. The ye-olde print statement is still as useful after thirty years of programming as the first time I used one in high-school. Think carefully about the problem and often one or two lines of extra output will help to isolate the problem.

3. Switch to Fine Logging. Sometimes when an underlying library or product is not telling you why it is failing then increasing the verbosity of the logging can uncover additional clues.

4. Search the Logs. If there’s too much logging to read easily, search through the log files for keywords or error codes.

5. Wrap on, Wrap off. Controlling the logging viewed with word wrap on or off can also be helpful. Sometimes you want it on and sometime you want it off.

6. Search different logs. The main server log might not be the only useful log to search through. Java application servers often produce other log files that might be useful.

7. Windows Event Log. Another source of log files might be from the operating system itself.

8. Craft useful logging. Sometimes if you are not getting any useful logging then you might need to write it yourself. For example if you have a complicated data structure to deal with you might include carefully crafted “dump” statements at strategic points in order to get the visibility you need to solve the problem. This can also be quite handy when dealing with multi-thread problems. Sometimes half the problem is just “seeing” what’s actually happening.

Talk to Someone

9. Ask someone who might know. If the problem is something that you think someone else on the team might know about then ask them. When you join a new project or team there are going to be dozens of things that you just won’t know about and someone else on the team will. Don’t be afraid about looking dumb. It’s far better for the team as a whole to get the work done efficiently and get you up to speed as quickly as possible. The only sin is to keep asking the same question over and over, so when someone tells you the answer that you make a note of it somewhere and don’t ask the same question twice.

10. Ask dumb questions. You might think they’re dumb questions but they probably are not. If you don’t know the answer then it probably isn’t a dumb question.

11. Explain the problem to a teammate. They might know the answer or be able to suggest things that you haven’t thought about.

12. Explain the problem to your dog. It really doesn’t matter who you explain the problem to it’s the act of explaining that gets your brain to analyse the problem from different angles.

Writing

13. Write a problem statement. Write the most accurate and precise problem statement you can. By being precise you are challenging your brain to accurately describe the problem which in turn helps you to think of possible solutions.

14. Write a problem diary. Create a text file with a bunch of notes to yourself about the different things you have tried. Include snippets of code or configuration settings and also any errors produced

15. Keep a record of your problems and solutions. Have you ever had a problem where you know you’ve solved it once before but you can’t remember how? Once you have a problem and then solved the problem, document the solution somewhere that’s easy to search (preferably across your team) like a wiki, defect tracker, or just email it to yourself.

Support

16. Read the FAQ. Check the Frequently Asked Questions before submitting your support request if there is one.

17. Submit a support request. If you have support available for a product or library then use it. Often the support desk is in a different timezone so when you get to the end of the day put together a support request and let someone else work on it overnight for you.

18. Before you Click Send. What you will also find is that the act of writing the support request will prompt you to think about the problem again and you often solve the problem or come up with new things to try before you even hit the send button.

19. Support. Breakthrough 1st and 2nd level support and talk to the real developers directly and preferably in real-time, chat/skype/screen sharing.

Move Away from the Keyboard

20. Go for a walk. Taking a break from solving a problem is a good way of generating fresh ideas of things to try. Walking by yourself is a good way to put your brain into neutral and let it swirl around the problem.

21. Sleep on it. Likewise sleeping on the problem is also good for letting your brain come to grips with the key aspects of a problem.

22. Reset the Priority. The other good thing about taking break from a problem is that it lets your reevaluate how important a problem is. The issue might be a CSS/Layout issue that just simply isn’t worth spending 16 hours on solving, so be careful that you are spending your time effectively.

23. Ignore the problem. If you ignore the problem what you find happens is that you will be hypersensitive to related keywords for the next week or so. What will happen is when you are reading something like JavaLobby all of these keyword related articles will suddenly jump out you and might contain useful information.

Isolation

24. Which line of code. Are you sure you know which line of code is causing the problem. Sometimes it helps to temporarily insert additional print statements between each line of code with nothing more than a “line x” statement. Watch carefully and you might be surprised that sometimes the flow through the code isn’t what you thought it was.

25. Break out into an example program. If you are having problems with a library or product then sometimes it helps to break out the related code into a program separated from the main application. This might take a little time to setup but often dealing with an isolated example program is easier to deal with than your project’s overall build process. Once you’ve got “something” working then it gives you a basis for comparison with the main program.

Change the Code

Even if you don’t know how to solve the problem changing the code anyway can be an effective technique for problem solving.

26. Write New Unit Tests. Since you have to spend time with this code anyway, write some new unit tests. It’s helpful to get your brain thinking about the code in the same way that the computer is.

27. Refactor it. Problem code is often messy code. Tidy-up the code by applying simple refactorings like renaming variables, or unwrapping nested if/then/else blocks.

28. Find Bugs. Another code tidy-up technique is to look at any “Find Bugs” reports you have for the related code and start working with those first. Get the code into a tidy state first because;

It’s an easy cost-effective way of getting your brain focused on the code The problem may become more apparent

29. Rewrite It. One technique is to dump all of the related code and rewrite it from scratch. A fresh perspective may avoid the problem altogether.

30. Comment out unnecessary code – or at least what you “think” is unnecessary. You may discover that your understanding of the flow through the code isn’t quite what you thought it was.

31. Experiment. If you are not sure how the underlying product or library is working then it can useful to introduce little experiments particularly around boundary conditions.

32. Periodically go back to a clean state. If you have been making lots of different changes either in the code or with configuration settings it’s important to periodically go back to a clean slate. Otherwise side-effects from experiment number “3” might affect the “right” answer and so you never actually discover the right solution.

33. Switch Technologies. If you are having problems with one particular technology it might be worth dumping it and switching to a different technology.

Products

Products and libraries can be great when they work and head-banging frustrating when they don’t. Here are some product specific strategies.

34. Upgrade to a Later Version. Maybe the problem you are dealing with has already be found and fixed. Try upgrading to a later version of the product.

35. Downgrade to an Earlier Version. Maybe the problem is being caused by incompatibilities with other products/libraries you are using and needs to be downgraded to something more compatible.

36. Patch the product. Have you been able to find a fault in the product, or maybe you just need it to behave differently for your application, patch over it with the problem. More useful for Open Source project, be careful with commercial products as you might violate the license or support agreements.

37. Download and attach source code. It’s a good investment of time at the beginning of the project to download all source code (that you can) and attach it to the classpath in your IDE. Since we all know how accurate the documentation is the best documentation you have is the source code itself.

Documentation

38. Read the Manual. Most developers would probably consider this a low-probability strategy but hey, you never know maybe the answer is contained in the documentation.

39. Read the Right Version of the Manual. Be careful that you are reading the right version of the manual.

40. Is the Manual Right? Of course often even it’s the right version of the manual that might be the problem since the code has been updated and the manual hasn’t.

Debuggers

For an article that’s all about debugging, there’s not a lot to say about debuggers. Personally I find they have limited value since they are slow to use. In the time it takes to step through the debugger a couple of times I could have written a new unit test that will be an “asset” into the future rather than the “expense” of just using the debugger.

41. Learn the Keyboard shortcuts. Hands-up if you know what the keyboard shortcuts are for stepping through the code? Hmm… I thought so.

42. Step Backwards. If you do step over the wrong line of code then look for the debugger feature that lets you step back through the code. Of course this is typically called something useful like “Drop Stack Frame” rather than “step backwards”.

43. Write code for breakpoints. Debuggers will let you have conditional breakpoints but I tend to find it easier to just drop in a few lines of temporary code to catch the condition I want to break into the code at.

44. Break on Exception. One somewhat useful feature of debuggers is the ability to catch a particular exception class from wherever it’s thrown.

45. Use a specialized tool. There are lots of other kinds of debugging tools available now. For example;

• Plumbr
• AppDynamics
• Chronon
• Wireshark
• HTTP profilers; Fiddler2, Charles, Live Http Headers

Source Control

46. Defect Number Hooks. Have you ever had a problem where something gets fixed one way and then a few weeks later becomes a bug that gets fixed a different way (by someone else). Sometimes the issue might flip-flop between two “right answers” until eventually someone has fixed it often enough that they actually remember something is happening here. The solution to this problem is to tag each source code change with the related defect number and keep a more detailed description of why something was changed and who was involved in the decision.

47. Blame. To aide in find out who changed a particular line Subversion has a great little feature called “Blame” which tells you the username of the person to last change a particular line of code.

48. Git bisect. Git has an interesting command called “bisect” that will automagically perform a binary search through your commit history looking for the first commit that introduce a fault.

Searching for Answers

49. Search Google. Searching Google can be effective, the trick is to know when, and also to not spend too much time on it. If you have some nice juicy keywords like an unusual Exception message or error code it can be quite effective to run a quick search. Open the first five or so links that look related and quickly scan them to see if they are on topic or not. But don’t spend hours digging through Google until you have tried some of the other strategies first.

50. Google Bundles Forums Posts. One thing to watch out for with Google is that it tends to bundle multiple posts from a forum into a single search result. If you only click on the first link you might not know that there are other posts in the forum that might be more useful.

51. Search product specific forum. Google can’t see into all forums, so sometimes you need to search through a product specific forum directly.

52. Search Stack Exchange. Google tends to search stack exchange anyway, but sometimes it’s better to search it directly.

53. Create a Stack Exchange Question. If you can’t find the answer then maybe you can ask it.

Other

54. Hire an expert. It might be cost effective to bring in an expert for a short period of time. For example if the bulk of the team are not web designers then hire a Web designer for a short period and have them work on multiple CSS issues at the same time.

55. Give it to an Intern. The opposite of hiring an expert is to hire a newbie. Sometimes beginners full of enthusiasm can solve problems just because they come at it from a different angle.

56. Change the requirement. If you can’t fix the defect then change the requirement. Your client might change their mind about fixing IE6 defects when you explain how much it’s going to cost them.

57. Change an upstream/downstream system. If you can’t find why a particular input file is causing a problem maybe you don’t allow that kind of data into the system. Maybe the defect becomes a “business validation” error before it’s accepted into the system.

58. Don’t run before you can walk. When adopting a new technology understand the basics before jumping into the advanced stuff. Practice makes Perfect.

59. Check your configuration by making it break. Change key configuration values and make sure it breaks in the way that you think it should because that way you can be more confident (though not absolutely certain) that the configuration settings you do have, are possibly right.

60. Be methodical. Sometimes you have to get three or four combination of things right at the same time. If you find yourself dealing with multiple variations of something at the same time then keep a careful note of which combination of things that you’ve tried and be sure to try every combination (if you think it’s required).