Transcript

[00:00:00] Christopher:

I've been employed in tech for years, but I've Almost Never Worked is the title of an article that a colleague sent to me. And I'm going to read a little bit from that article. So, reading from the article, after being employed in the tech sector for years, I have come to the conclusion that most people in tech don't work. Look, I don't mean that we don't work hard. I mean we almost don't work at all. And when we do get to do some work, it often brings low added value to the company and its customers. All of this while being paid an amount of money that some people wouldn't even dream of. What is happening right now in tech may be one of the greatest market inefficiencies or even deceptions in history. I know my statement may sound a bit hyperbolic. How could people be constantly paid a lot to do close to nothing? Surely that can't be right. Well, let me share some examples from my own experience. Five months ago, I was hired as a software developer by one of the world's most prestigious investment banks. Since the beginning of my employment, I've worked for around 3 hours in total, not counting non focused Zoom meetings, which I attended without paying much attention. When I first joined the company, I was excited. However, since I joined, they've only given me tasks that were exceedingly easy to complete in just a few minutes, but allocating days or even weeks to them. At first, I wanted to speed things along. I genuinely wanted to build a cool product. So I connected with people across the organization to ask questions about our intended users, their needs, and how our product would satisfy them. But it was soon made clear to me a few times that I shouldn't do that. One person told me, I don't want to tell you not to ask too many questions, but and she basically told me not to ask too many questions. I soon realized that the project was overstaffed and most people were pretending to work. And I also realized that was the job I was hired to do. My job was to pretend. If this had been the only time this ever happened to me, I would consider it an anomaly. Unfortunately, this has been the case with almost every tech job I've had in years. Consider the case of my previous tech job in which I was hired as a data engineer for one of the world's largest telecommunications companies. In the year and a half that I worked for them, there was only one two week period during which I worked at full capacity. Other than that, I did almost nothing at all for 18 months. Outside those two productive weeks, most of my work involved attending irrelevant meetings, performing small tasks to pretend that a broken product worked well, and even generate fake results. It felt ethically compromising, and it was boring. I only worked half an hour, a week or so. Non focused meetings aside, this isn't just me. All of the people I know who work in tech seem to be going through the same thing. One of my former colleagues, for example, told me all he does at work is watch coursera courses. He's considering resigning after the company sponsored coursera subscription ends. Another former colleague was hired a year ago as a data scientist for a large oil company. All she does is prepare a PowerPoint presentation every week and she's utterly bored. Another one of my friends was hired two years ago as a quant for one of the world's most important investment banks. The interview process for that kind of job is among the hardest you can imagine. Brain teasers, differential equations, graph algorithms. He was very excited at first, thinking he'd be building cutting edge technology. However, while people always seem to appear busy from the outside, in reality he does almost nothing and is horribly bored but well paid. It's hard to speak with others about this. For a lot of people, the idea of being paid a lot to do nothing sounds like a dream come true. However, while we may not do almost any real work, we do have to constantly pretend that we do. That can be extremely frustrating and soul draining. And that was taken from an article written only last year by a software developer. Jesse, what do you make of that?

[00:04:58] Jessy:

Yeah, it's first reaction can't be surprised.

[00:05:04] Christopher:

I can't say I've gone through years like that, but I've definitely had work where if I hadn't generated my own work I could have been doing that.

[00:05:13] Jessy:

Yeah, that's kind of the status quo. These organizations and talking about the big ones here, everything just takes so long. So if you have simple tasks you can just say it's going to take you a long time and they won't even question it. So if you want to make it easy for yourself then that's so doable. So that makes stories like these very unsurprising.

[00:05:36] Christopher:

Yeah well you touch already on an interesting point as to whether software developers are taking advantage of the inefficiencies of large organizations. I suspect that's not the primary problem but we'll come back to that I think. Let's read on to see what else the author has to say. Back to the article business people love predictability. They like to estimate in advance how much a project would cost and how long it would take. This can be surprisingly difficult in some disciplines. Even when building a physical thing like a railway, projects often overrun significantly. Imagine how much worse the situation gets with intangible products like software, where it's hard to define exactly what it takes to deliver the product and even what the product is. So in the early two thousand s a philosophy called Agile, which sought to improve software development as a process, became really popular in tech. In Agile, software is developed in very short cycles, as short as two weeks, and the result is validated and goals realigned between cycles. Every couple of weeks the team gathers to plan the tasks for the next cycle. But in this planning phase, something really strange happens. It has become the standard practice to highly exaggerate the efforts required to perform a task which we'll call Task Bloating. In the planning session, hands on people, managers and other stakeholders all agree that each task takes an abnormally high number of hours to complete and requires an abnormally high number of employees. Let me share a couple of examples so you can see the extent of the problem. One of my recent tasks at the investment bank was to analyze what a couple of software code templates provided by Microsoft could be used for. Anyone familiar with software development would be able to do this in a couple of hours at most. However, in our planning session it was collectively considered that this task required many days of work and two people. The difficulty of tasks is chosen by vote in this company and in this case, the collective wisdom decided that the task was relatively hard. I voted that it was easy, which didn't match the majority vote. When they inquired about this, I conceded that maybe it was more difficult than I thought. After all, it is hard to disagree in those situations because it seems you're going against the collective wisdom of the team. Also, when you notice that every single task is bloated this way and no one says anything, you don't want to challenge the system. As two of us were assigned to this task, we ended up splitting it into two even easier parts, one for each. I completed my part in a few minutes and pretended it took much longer. In the following cycle, I was assigned the task of writing a paragraph summarizing the results of the previous task. I voted for a difficulty score of one, the lowest possible, as it was a ten minute task. However, my colleagues didn't agree. They voted that my task was harder than it looked and worthy of several days of work. While the mechanism of estimating tasks in this way will sound familiar to most, I think, but this tendency to overestimate, or task Bloat, as he called it have you ever seen that, Jesse?

[00:09:41] Jessy:

Every refinement, yeah.

[00:09:44] Christopher:

I've never been convinced of, or perhaps never fully understood the rationale behind story points or this obfuscation of time. I mean, we're trying to estimate that's the purpose somebody has said it's important to get estimates and story points make it more obscure.

[00:10:10] Jessy:

It kind of reminds me like when you're at a festival and you buy, like these tickets rather than money and then you go out spending these tickets but you kind of lost the value of what you're actually spending. It kind of feels like that in a way.

[00:10:21] Christopher:

I know what you mean. It's an abstraction.

[00:10:23] Jessy:

Yeah. And that's the thing, you're spending an abstraction now, so nobody really knows what that means. And then three story points, it can be a day or two days or three days.

[00:10:34] Christopher:

Anyway, story points are just a detail. There's a real underlying issue here which I think is best described in this next part of the article. So reading from the article again, work must be divided into clearly defined cycles, usually called sprints. And we must try to predict the difficulty of tasks to make sure they fit within a sprint. The structure encourages. Task Bloating. Employees want to deliver on the promises they make for the sprint, so they end up bloating the tasks to be sure to complete them. Productivity is sacrificed in the name of predictability, I take it that's recognizable.

[00:11:20] Jessy:

If you want us to be predictable no matter what, then I'm going to make really sure that the work I do will actually fit within this period. But that's not your benefit in terms of what you'll get.

[00:11:34] Christopher:

We should be aware that that's the choice we're making.

[00:11:37] Jessy:

Yeah, I was just going to say as long as you're honest about it, then it's fine. If you want predictability, that's okay. But that's just an expensive thing to ask for in terms of you pay productivity to gain predictability.

[00:11:54] Christopher:

Yeah. Another way of looking at this is that if you observe that your development team consistently hits sprint deadlines, sprint goals, consistently delivers everything that they say they're going to deliver, then you can almost be guaranteed that they are, I'm not going to say unproductive. It's still possible that they produce a good amount of work, but at any rate, they are less productive than they could be.

[00:12:31] Jessy:

Exactly.

[00:12:33] Christopher:

All right, perhaps we've somewhat established that this business of estimating tasks and then fitting them into a two or three week sprint cycle doesn't always yield the desired effects and perhaps even has some unintended side effects. But why are we doing this anyway? What's the purpose of it? Yeah, well, it's as the article said, business people love predictability. Now, they don't just love predictability for the sake of it. Everyone loves predictability. We can plan then. What if we've got some real world event that cannot be changed for whatever reason? Well, then we have a deadline that matters. So if you find yourself responsible for a significantly sized software project and your commitment is to deliver that project on time to a specific deadline, well, apart from being in an uncomfortable position and a challenging position, your focus is not going to be on the process. While you will need a process, even the most capable and willing team will fail if operating in complete chaos. But the process is not the thing that you're going to be relying on to deliver on time. That's really going to hinge on your ability as a leader and building a narrative around this mission that you're on. The challenge will be to get buy in from everyone working on the project and that means their full commitment. Your deadline needs to be their deadline.

[00:14:26] Jessy:

But how would you do that in a situation where somebody has a company and just wants his product to go out as soon as possible or whatever but there's no actually real world event, he just wants to have it done quickly.

[00:14:40] Christopher:

Well, this is the case where you want productivity, right? So it's again not about throwing away all process but the part that doesn't help us in terms of productivity is where we go and estimate little tasks and try and fit them into a two or three week period. We've already established that that won't help productivity so we can eliminate that aspect of things. And to be clear, it really is the combination of estimating and then fitting it into a two or three week period. I don't really see a problem with having a bunch of tasks, estimating them and then making decisions based on those estimates. Plenty of useful things can come from doing that like let's deprioritize this because it's going to take too long or we can't estimate this at all because there's just too many unknowns. Those are useful things and that exercise can be completely separate from anything else. Similarly, this idea of coming together every three weeks for feedback and updates and checking the direction, that's a good thing, there's no reason to throw that away. In fact you'd be encouraged to do that. But the idea of combining these two things, as we've said, doesn't yield the results you would expect. It's almost as though the process is now leading and we've forgotten what our objectives are and the fact that we have people involved in this process who are going to respond to incentives that you put out there.

[00:16:30] Jessy:

Yeah, but that's what makes the original manifesto, agile Manifesto, pretty good. Like there's real nuance in there, it's good to focus on people but we're not saying process is wrong. Correct. But if you need to choose, we're suggesting you let the compass needle of your decision point towards directing the people.

[00:16:53] Christopher:

Yeah, indeed. And speaking of the nuances of the Agile Manifesto, we should also realize that this way of working that we've been talking about has the added disadvantage of creating a certain unwanted dynamic between the business and the developers. And that's the dynamic where the developers kind of feel like they're being checked up on and the people who have set the tasks feel like they have to check whether the tasks have been done or not. I mean, it makes sense to they've been committed to upfront, it doesn't make sense not to check that. But that's exactly the dynamic that Agile tries to warn against. It's the contract negotiation part of the manifesto and there's just no getting around it. Each sprint is or at least can descend into something like a mini negotiation, a mini contract and that is definitely not the dynamic that you want. I said earlier, in fact, when you've got a project where you need to hit a deadline that the most important thing is to get the commitment of everyone on that project. Well, the same is true even when you don't have a deadline, you still want the commitment of the people. But what happens sometimes with scrum is that the process becomes so mechanical that there is nothing to commit to in an emotional sense. There's nothing to get behind. There's nothing to motivate other than completing tasks on a list. And the worst thing about that is you probably did have a rather motivated and committed team because software developers tend to be that way. They don't like to sit around and be lazy and do nothing. They want to have a challenge. They like solving problems. And it's sometimes the case that the process ends up robbing them of any intrinsic motivation that they had. Okay, I'm going to finish up here by going back to the article and reading what it has to say. In summary, the principles behind agile software development are commendable and much more suitable for the job than old school ideas. So many organizations have strived to be agile. However, they've done so by adopting agile recipes which are a step by step guide supposed to make a team agile. The adoption of a recipe results in a box ticking exercise that makes companies believe they become agile just by strictly abiding by a set of inflexible rules. The effect is the opposite. The most common characteristic I see behind truly productive tech work is that the people involved prioritize building the right product above everything else. They build it in small cycles and validate it, which follows the agile philosophy. But they don't limit themselves to a strict agile recipe. They sometimes borrow some elements of an agile recipe, but only because it helps them build the right product. The title of this article says I've almost never worked when employed in tech. But that doesn't mean I've never worked. In fact, I've worked really hard and been part of some amazingly productive teams which deliver top notch products in record time and clients love them. There's actually more in this article that I think is a good idea to revisit on another time, on another day, Jesse. But for now I think we'll wrap it up. Thanks a lot for coming. Thank, thanks for giving your input. Yes, you're welcome and thank you for listening.