Reuse

One thought that's bounced through my head a lot is this concept of reuse of our code. Through my career, how much has my stuff be reused? It's one thing to build a system that sees a lot of use within the container it was built for, it's another to build a system that sees use outside of that container.

I've certainly written code that has found re-use, but those tend to be the exceptions rather than the norms. A lot of stuff feels like special snowflake code. One-off implementations that exist to map data from one object to another, or perform some calculation on a data set that will never live outside of the application it was built for. In spite of all of this, I value the concept of reuse. If this is so, why does it feel like it happens so infrequently?

Code reuse is thwarted by coupling. When you have a Foo that has to convert to a Bar, you're generally left with a piece of functionality that only works when your other systems also use a Foo AND a Bar. Chances are, your system will be lucky to reuse one, let alone both. Foo was built to satisfy some need of your original problem, not the one you're working on today. If we could just operate on strings or ints all day reuse would be easy, right? It's easy to write functionality that does some calculation with numbers or performs some manipulation with strings and use it again and again all over the place. How many of us really work that way though? We're told that we need to use Cats and Dogs, not ints and strings. Our frameworks and toolsets have enormous effort invested in keeping us thinking about Cats and Dogs. Ints and strings aren't useful because ints can represent any kind of number. They don't readily indicate a Cat's age or the number of a Dog's teeth. Not alone anyways.

Perhaps we can write small stuff that yanks out cat ages and dog tooth counts. Then we're working in those basic types that are easy to manipulate, right? I think there's some value here. We've decoupled some data from the objects that store the data. Wait - we weren't told simply having an object with data in it is a form of coupling. I thought it thought it was just when we take we have a scare() method on a Dog that accepts a Cat as a parameter and then sets the Cat's setScared property. What we really want is a ScareAdapter, right?

Well if objects can be coupled to the data they hold, how is this done? One thing I've noticed is that once I've made a Dog able to scare a Cat, even with the adapter we're still talking about Cats and Dogs here. What if I later need a Dog to scare a Dog, or even a Mouse to scare an Elephant? Is there something intrinsically unique about the way in which a Dog must scare a Cat? Do the properties of scaring and being scared have to be strictly defined for each animal? If wide-spread reuse is a thing we really can get to, the answer must be no.

The thing I'm starting to realize is that objects, which we should really think of as data with behavior attached, couple themselves to the data they hold via that behavior. What is scaring really? You take an animal and set its scared property to true. Now it doesn't matter who is being scared. Dogs, Cats, Mouse(s), Elephants, etc all all able to be scared because all of those things are just buckets of data in which your scare() functionality can work its magic on.

It turns out this is the tree that Functional Programmers have been barking up for a long time. I've long thought there are things I like about FP but ultimately I don't see how you write practical applications with it. If you can't have mutable state, then how does the machine do anything besides get warm when your software runs on it? You need to write to buffers and streams and databases in order to accomplish some real work. At some point, the tires must hit the pavement, right? This isn't helped that historically the examples for FP have been super abstract math stuff. Ok great, you can write code that does operations on lists of numbers. I don't have an application that speaks that domain language. My applications deal with shopping carts, shipments, rentals, mailbox messages, game entities, etc. Number manipulation isn't so hard - and so showing me trivial examples doesn't really impress to me how an application structures itself and how I go hunting down a problem when somehow a shipment didn't get all of the data it needed. I can manipulate numbers easily with my current code using my current tools. So where's the benefit?

I've seen a lot of smart people gravitate towards FP or even make a living using it, so I was hesitant to totally dismiss it, but I did need some kind of ambassador from the land of FP to lay things down for me in terms of the language I speak. Well if you've ever had to tutor for software engineering before, you'll find it's damn hard. You basically have to teach another person how to think. How the hell do you even start with that? Is there a book "How to Think for Dummies"? "Learn to Think in 24 days"? Let me know when you find it. I don't blame the FP folks for having difficulty teaching their stuff. I've heard it explained that all applications have to have some concept of math, or at least we all know how to work with ints and ints aren't really specific to a domain, bet it driver development, games, web stacks, etc. As I learn more about FP, I think working in basic stuff like numbers is where the simplicity of FP comes from. It's not a matter of saying "Everyone has to do some kind of math in their software at some point so we'll all just collectively teach at this level". It's because that's really the place where they spend a lot of time. You write some functionality that knows how to operate on ages or counts and then you apply them to cats and use them to tally teeth on dogs. This form of decomposition is very fundamental to how FP works, and I think FPers at times can take that for granted as we do object dereferencing.

There's a lot more to FP than just figuring out how to work your way down to basic types and then going nuts, but I think accepting that this is a huge way in which the problems are approached is a huge factor in the success in learning it. With that in mind, maybe we can find some learning material that helps emphasize these approaches, even if it's from a high level document with pretty boxes and lines.

If you've done functional programming and came from imperative land, what helped it click for you?

Changing the Culture of Apathy

Generally speaking, I try not to point out specific things about specific places I've worked. It feels like mud slinging. This post will talk about my current employer. It won't be flattering. It won't be mudslinging though. My hope is to identify some problems and seek a solution. Ultimately, that's what we do, right? Some problems are computer problems, and some are human problems. Being a software engineer often means being able to tell to the two apart, but I know that I personally overlook the human aspect quite a bit. I'd like to change that.

The culture at my workplace is one of apathy. Engineers don't stress code quality, they don't fight for things they feel are important, and they tolerate gross impediments to their output. This is shown in the tools employed (or lack thereof in many cases), processes used, and the scrambling many teams do just to stay afloat.

The level of despair brings down other engineers who otherwise would be fighting the good fight. Not all is lost, however. At some point I pushed against my manager and got myself a spare laptop (VMs can involve prohibitive amounts of red tape). This laptop is now our CI/CD server. It has a sticky note on it saying "Do not shut down, doing servery things" - this is because it was shut down during a standup by a well-intentioned employee. This isn't what gives me hope though. What gives me hope is that in a sprint delivery demo, I gave a brief showing of the code coverage report. The demo was hijacked by the coverage report. The CTO/CIO (the exact title eludes me) is the one who hijacked it. "We need more of this" and "How do we get other teams doing this?" are sentences that were exchanged. This is what gives me hope.

My damage is that I like a good challenge. While I bet I could go find a sweet gig at New Relic, which is a couple blocks from my current location, there's something really interesting about showing up at a place and seeing a software shop that has dysfunction, but wants to get better. I'm in a position to help with that. Do I have all of the good ideas on how to handle that? Of course not.

Yesterday I told my team I was going to get it such that our turnaround for a feature into production in 5 minutes. Currently it's 60 days at best. It's a tall order, and there's much to do to get there. I can see the path we need to traverse to do it. The problem I have is that I'm just one engineer at a shop who must have 200 people in IT. Doing this alone while still keeping up with my daily obligations feels daunting. So what do I do?

I've thought about getting more involved in the hiring process, but I think pulling in the 'right people', whatever that is, will be a fool's errand. While any shop should make an effort to guide its culture, I think changing the culture by bringing in different people isn't the best way to go. I think anyone wanting to bring in some good practices is probably going to avoid a place like this. While it's worth putting some energy into this, I believe the majority should go towards fixing the problems within. There needs to be a way to motivate engineers to want to be better, learn more, and stop putting up with a broken process.

Materials on inspiring engineers to do better feels sparse. Maybe that's because this is more generic managerial stuff. Here's some ideas that come to mind:

1. Set aside resources for engineers to improve. This means time to explore tech and money set aside to go to conferences and the like. This also means engineer time isn't 100% devoted to some product. It means they get to find ways to improve their workflow without having to put in OT to do it. Brown bags are great, but they aren't a real investment on behalf of the organization. If you want to retain people who are invested in you, you need to invest in them.
2. Stop punishing failure. Innovation means you're going to take risks and risk means you're going to make mistakes. If you punish those mistakes severely then nobody will try out new things or do anything amazing - these are inherently risky things. Mistakes will happen regardless of how hard you try to minimize it. The better route to go is having a means of recovery when mistakes do happen.
3. Treat impediments like the plague. On of the most demotivating thing I see a behemoth process standing in my way of making any kind of improvements. Need access to the documentation wiki? Submit a ticket. Need access to the company chat service? Ticket. Need to be able to view the logs for the set of servers your team uses (be it development or production)? Ticket. We have a similar story with deployments, which is probably why we have an entire release management department. These impediments snuff out any hopes that an engineer can make a difference and that their work has impact.
4. Stabilize our environments. My initial thought is to say no future development on features until our nasty fire-starting bugs are fixed. Even then, we need to be on lockdown until we have a release pattern that doesn't encourage dysfunction. This one I feel less confident about, since it's a bigger ask. I'm not sure how else to do this though. People who are willing to help make things better often are not in the place to do it because of how crazy it can be just to keep up.

Any other ideas on what to do? I'll gladly accept reasoned speculation, but hearing some success stories would be great too.

Explaining computer programming

Have you tried explaining what you really do at your job all day? Saying that you're implementing features and fixing bugs is kind of like saying a writer writes down sentences and will remove some that don't make sense anymore. It simply doesn't do the profession justice.

Simplify

Being able to explain something simply is said to help one's understanding of a topic. We could all understand software a little better, right? I've tried a few different things, one of them being that I get to make stuff up all day. Have I ever created a ClaimValidationProcessorFactory before and know what that is? Nope. I just made it up. And it's going to production. This explanation is a start, but I don't think it captures the scope very well. Understanding the depth behind being able to make stuff up, and having built off of others' made up stuff is vitally important to getting a good gist of what software engineers fight with on a daily basis.

Calvin Ball

Enter Calvin Ball. This is a simple to learn game that is very complex - you just make up the rules as you go. It's interesting because each new rule has to coexist or build off of the previous ones. If you have a rule that says you must hop on one leg and another rule that says you must hop with two, well that doesn't make much sense does it? The trick is extrapolating that a bit. In a non-trivial game of Calvin Ball (aka your app), the number of rules can easily be in the 100s. Knowing all of them at once is infeasible, but that's the demand that's put on you. Mess it up and you have moments where your players are told to do contradictory things. In software engineering we call that a bug. To help us out, there's certain patterns and conventions. You don't make _anything_ up. We know ClaimValidationProcessorFactory is a Factory, which carries a heavy set of assumptions on what a CaimValidationProcessorFactory does. If it doesn't, well, the original engineer needs their hands slapped. It happens far more often than you'd think.

Suppose

In the software world, things are composed of many layers. Each layer is a kind of thought that starts with "Suppose...". Suppose we have a state that has two possible values: on or off. Suppose we have many of these states in a series. Suppose these state series are used 4 at a time. Suppose we have a means of looking at one of these groupings of states and deriving an action from it. Suppose off off off on means "take the next two groups of 4, and add them together - assume them as numbers, and place the result in a known place". Suppose our series looks like this: "off off off on off off off on off off on off". You might say "oh I see off off off on a second time, so it adds again!", but that would be wrong. You must treat them like numbers. We supposed that already, right? We just described how machine language runs on a computer, which all of our software is based off of.

What it's not

Computer programming is not construction/manufacturing, despite construction related analogies we do all of the time (build, construct, factory, etc). If it does feel like construction to you, well, you're doing it wrong.

Ending thoughts

It's still considered cool to not be a "computer whiz" just like it's cool to not be able to do basic arithmetic to balance your checkbook... somehow. Taking some of the mystery out of what we do by giving a simple explanation without actually having to teach someone how to code could help us rope in more bright contributors and expand the industry as a whole. How do you explain it? Are there other things you see people equating it to that are simply incorrect?

Coding in Anger - Adventures in Quality of Life

During a tech meeting at work, my manager started off an explanation with "It's been a while since I've coded in anger, but...". This simple sentiment really resonated with me. Often times I've made the argument for particular choices that ultimately enhanced quality of life, but I've lacked the ability to properly articulate why those choices were beneficial even if the surface value seemed to be that it would make my life easier. That simple phrase has given me a lot to think about in terms of framing the justification here.

I've heard the argument that your user doesn't care about your test suite, or how easy it was for you to write something. While I think there's merit in such an argument, the black-and-white perspective of it simply makes it untrue. There's an implication that we should consider ourselves lucky when we get to do things like test drive, pair program, use good tools, and make good architecture. It's like we as engineers are compulsively driven to want to put things in nice and neat little boxes for organization's sake. When we can't do that or something disrupts the organization, we throw a tantrum because we're simply children. While I believe there are certainly cases where this applies, it's blatantly incorrect to paint all of us under such a stroke.

The thing is we get frustrated and angry at our cools, codebase, process red tape, etc because it hinders our ability to perform. If I'm too busy jumping through the hoops of broken framework X then that means I'm not delivering awesome stuff. Instead it usually means I'm writing harder to maintain code. When something is hard, it doesn't get done. Or at least it doesn't get done as much or as well. Sure, you can build entire empires on PHP, but by that reasoning it means the next grand project akin to the Pyramids of Giza can be done without any modern trappings, right? If logs, rope, sweat, and blood were good enough for them, then it must be good enough today, right?

A software shop should fight tooth and nail to stay on top of the curb. Does that mean we have to go back and rewrite? You betcha. But if our technical debt is reduced, if the engineers are able to get closer to the features or pull out 3rd party modules to assist in development, then isn't that a big win? Avoiding rewrites is a lost cause. How many banks are glad they are still running on COBOL today? Sure it was probably great at the time, but that time has passed. Everything we use now will eventually become archaic and dated. We may as well embrace the idea that code is a living document that needs small and big adjustments both over time. Does that mean we should jump on every shiny new framework the moment it comes out without evaluation? Of course not, but this entire "suck it up and get back to typing" attitude needs to go. Your user might not care if you're using punch cards to enter your code, but they certainly are going to feel impact of such a choice.

Something to consider.