Dysfunctional software is risky software

Great software needs great teams, most teams are like dysfunctional families.

Quote courtesy of Jim McCarthy, Dynamics of Software Development

Intro

When you were a kid, did you ever tell your mom,
“Dad already agreed…”

And your sister would go to your dad,
“Mom’s letting Danny stay home from school—why can’t I?”

If you played it right, the 2 of you got away with murder.

Today, writing software is a family affair with two parents: human and AI.

Software is written in a dialog between them.

The dialog between machines and humans started 200 years before ChatGPT.

Charles Babbage designed his Analytical Engine in 1837. It was never built, but Ada Lovelace wrote a program for it in 1843. Ada was a colorful character, daughter of Lord Byron, directed by her mother to learn science and mathematics.

Looking at her program, it is surprisingly close to what we do today.

Loops. Named variables. Structure.

But Ada saw something deeper.

She imagined machines that could do more than compute — write music, manipulate symbols, process abstract ideas.

Great software comes from that fusion.

Today, great software fuses human intent, taste and judgment in a dialog with AI execution, expansion, and iteration.

Today, I’ll show you how that fusion works in practice:

• One case where the human dominates

• One case where the AI dominates

Great software has two parents

In the past year, I’ve spent less time writing code and more time writing specifications for AI.

I can ask Warp Code if the model is a good fit for the job—and get an intelligent answer.

Here’s what that looks like in practice:

/agent what model are we using

You’re using Claude 3.5 Haiku.

for an application to plan tax-optimized liquidation of stocks and optimized lots should I use a different model?

No, Claude 3.5 Haiku is capable of handling tax-optimized stock liquidation planning. It can work with:

• Tax lot identification strategies (FIFO, LIFO, highest cost, specific ID)

• Capital gains calculations (short-term vs long-term)

• Tax-loss harvesting logic

For a production application, the main considerations are:

1. Accuracy over speed: Ensure thorough validation of tax calculations and compliance rules

2. User responsibility: Include disclaimers that users should consult a tax professional

3. Data security: Handle sensitive financial data securely (encrypted storage, HTTPS, etc.)

4. Testing: Comprehensive test cases for various scenarios and edge cases

Haiku is suitable because the model isn’t the bottleneck for this type of application—careful implementation of tax logic and thorough testing are what matter most. If you need real-time processing of large datasets or complex multi-step reasoning chains, you might consider a more capable model, but for typical tax planning workflows, Haiku works well.

Notice what’s happening here: I’m not writing code. I’m interrogating the model, validating assumptions, and testing the system design alternatives.

The first coders were mathematicians.

In the 70s, the systems analyst emerged—someone who spoke with users and wrote specifications for coders.

As programming languages evolved, people became ‘programmers’ and expanded their activities into design, code and communication with other humans.

A great programmer is great at design, code and communications.

If that sounds hard - it is.

In 2022, with the launch of ChatGPT that changed again.

AI reset the division of labor between humans and machines.

But it’s more complex than that.

In the next section - I break down human and AI work.

Humans and AI - the division of labor

There are 2 kinds of activities in writing software - human-facing and AI-facing.

Human-facing activities

1. Stakeholder alignment - getting other people to agree with you and support your project

2. Market discovery - discovering if other people will use your product

3. Business tradeoffs - buy vs. build, do-nothing vs. do-something

4. Regulatory strategy (healthcare, MedTech, drug development, energy …) - how do you thread the needle with the regulator? FDA Cyber regulation for medical devices requires traceability of cyber testing and explanation how a control has been tested and validated to work.

5. Business model validation - understanding from other people if they want your product, when they’ll want it and how much they’d pay

6. Capital allocation - getting other people to fund your project

AI-facing activities

1. Code generation - prompting the LLM to generate code for your task. Emphasis here on you focusing on the desired outcome in your prompt. Specifically -
   
   Write a very specific story of the outcome you want and need

   Do not be tempted to talk about the process, the how, your ideas - let the AI decide how.
   Front-load your critical constraints - in time, money and results
   🔥Every ambiguity = risk on fire
   

2. Test generation - the latest AI models figure this out and often suggest tests

3. Build pipelines - with memory the AI model usually knows how to do this

4. DevOps - prompt the AI for what you need - this is a lot like code generation activities

5. Refactoring - an area where AI is exceedingly better than humans, although when it gets it wrong, it gets it very wrong.

6. Static analysis - like refactoring

Is the dialog ever equal?

It depends.

In personal productivity applications - AI dominates. In enterprise systems - humans dominate.

Let’s see why.

Does AI eliminate programmers?

The short answer is no.

In personal productivity applications (e.g. writing social media posts, or filing invoices from your email), the human cost of prompting is low and the payoff is high for AI agents to do the work:

• You make your own tradeoffs

• You write specs, AI writes code

• You can (and usually) ignore risk

In enterprise systems (e.g. airline reservations systems or healthcare information systems), the human cost is high, relative to the cost of AI tokens. You need to execute all 6 of the human-facing activities, often over long periods of time to sustain your project.

• You don’t make tradeoffs alone inside the enterprise. You need organizational politics to get buy-in for your idea.

• You need to align with other humans, comply, and document

• You need funding

• You cannot ignore cyber and privacy risk, internal data governance and industry and regional regulations.

In enterprise systems, humans lead the dialog.
In personal productivity apps, AI leads the dialog.

Like when you were a kid with your parents, there is a division of dialog.

In my family, my Dad led the dialog on science and my Mom led on homework.

We’ll close the essay with some thoughts on software risk for AI-human software development.

AI prompting for risk analysis in software development

How will AI-human teams create secure software?

I will go into more detail in a future essay but here’s a teaser.

Secure software will be more about human story-telling and less about machine engineering.

Parallel to the AI-human build dialog, we need to conduct an AI-human attack dialog.

Write a story of how you attack your own system

Front-load your critical constraints

Be a ruthless 5-year-old who breaks stuff

Every ambiguity = risk on fire

And not wait for the FDA to remind us about cybersecurity.

Outro

Ada Lovelace died in 1852. Her name lives on in the Ada programming language, designed in the 70’s for the U.S. DoD for safety-critical, embedded software.

It’s still used today in defense and aerospace where failure is not an option.

Every year on the second Tuesday in October, the contributions of women to science, technology, engineering, and mathematics (STEM) are celebrated on Ada Lovelace Day.

Let’s celebrate Ada Lovelace today for her incredible vision fusing human creativity with machine computation.

We’re still learning how to build that partnership.