If there is only one thing that you’re willing to invest your time on design thinking, it has to the Double Diamond by the Design Council. Other than explicitly calling out the “Problem” and “Solution” domains in which the two diamonds sit respectively, the council’s presentation of the double diamond couldn’t be more clearer. I only add anecdotal examples of the mistakes we make when we don’t intentionally practice design thinking.

Solving the wrong problem

We finetuned Flan-UL2 with contents in our 50,000 big Q&A documents store to help call center agents to better answer customer’s questions. Because of the peculiar way of our fine-tuning, special characters are present in the generated text to represent bullet points. Because we want to avoid costly and meandering generations with a “max_tokens_to_generate” parameter, the tail end of the generated text may contain incomplete sentences. Both calls for post-processing, to remove the special characters and the incomplete sentences.

Our LLM Ops provider, one of the big three cloud providers, was engaged, for months on end without satisfactory solution. A meeting with their LLM model serving engineering team was scheduled and I needed to understand where the impediments are and what’s going on. Here it is casted in the double diamond:

So, the two collaborating teams were solving the wrong problem! We’re not interested in custom way of generating the text (custom serving), rather all we need to do is to take the generated text and apply some post-processing logic to it. The ideal experience for us would be a set of purposely built lifecycle hooks in forms of Python decorators with which we may easily declare and define our logic and count on the LLM model server to execute them at the right time.

As all “impact (a.k.a. expediency) driven” engineering team goes, our cloud provider acknowledges “post-processing” life cycle hooks are the right problem to solve, but offered hacks inside their custom serving handler as the solution anyway.

Our response? Screw you. We moved up one level in our stack and implemented the post-processing logic in the Generative AI (GenAI) Gateway layer, hoping our provider eventually turn around implementing those lifecycle hooks so we can rip the post-processing logic out of the gateway and put them back where they belong: the LLM endpoint.

Solution posed as problem

On my first day of work at a health insurance company I was pulled into a meeting titled “Segmentation collision avoidance”. Let’s unpack before we continue with our story:

• Segmentation. Our medical business unit instituted various patient outreach programs for different patient groups: those with high total cost, high risk for a specific costly disease, low medication adherence… each identified by an AI/ML segmentation model.
• Collision. When a patient is simultaneously identified by multiple segmentation models, redundant and (from the patient’s point of view) annoying outreach actions would be triggered: the patient would receive a USPS mail on medication adherence at the end of the day, get a phone call in the morning the next day, followed by a nurse visit in the afternoon on the same day. Not to mention other outreach channels such as email and text messages.
• Avoidance. The medical team asked us to adjust the decision threshold of various segmentation models to avoid the repeated outreach actions.

“$150K”, “No, $200K!”, as the medical and AI/ML teams debated the reasonable threshold for each model, a double diamond emerged in my mind:

And immediately followed is the question: are we solving the right problem? Of course not:

Avoiding collisions among segmentation models is not the problem, it’s merely one possible solution to the real problem of avoiding redundant patient outreaches. Even as a solution approach, it’s a bad one: for those patients who are identified by multiple models, we want to reach out to them about all the concerns!

The double diamond shown above helped me to convince the medical team to build a patient outreach calendar and a reconciliation logic which consolidates multiple outreach actions across multiple channels to the most personal one. For example, if there is a nurse visit recommended by an AI/ML model and planned for a patient, all other less personal modes of outreaches within the vicinity of a two-week time window would be consolidated into that visit. The patient would get a single encounter packed with all topics pertinent to her health.

I had a great first day of work, thanks to design thinking.

The social aspect of design thinking

What design thinking truly is? Categorizing design thinking as a way of problem solving is like treating Ballet as a kind of body movement. It’s not wrong but with the over-generalization, all the nuances and gist of design thinking are lost.

Just like Total Quality Management and Six-Sigma movements which engaged people on the shop floor to improve quality in manufacturing, design thinking is a social technology that encourages people to do higher or different level work than they usually were asked to. What Design Thinking trying to say is simply: Non-designers, design more! Designers, think more!

Different organization may have different realizations of design thinking that caters to their cultural needs. While almost every leading player has a www.foo.com/design page, that’s just their UX guidelines for third party developers, not design thinking.

If you want to learn one company who actually practices design thinking, read up on IBM’s Enterprise Design Thinking and search how they eat their own dog food internally and monetize it externally.

If you want just one article on design thinking, read “Why Design Thinking Works”. If you want just one book on design, read “The Design of Everyday Things”. Then you come back and think: How can your organization apply design thinking to improve our bottom line?

In my humble opinion, for any analytics shops:

Design thinking is about priority: User First

We must shift away from the engineering-driven “features-first” ethos towards a more “user first” mentality.
A user-first mentality “start with the user you want to serve. Then, specify the outcome you want to enable them to achieve, and the differentiator that will make your solution worth their while. We refer to these three elements as the Who, the What, and the Wow.”

In a “user first” culture, a tester’s usability complains should not be pushed back by the developer with a “What do you know?” disdain. Yes, testers may know less technically but so do our users! What our testers complain today will be the things our users complain tomorrow.

Design thinking is about culture: everybody @ your organization

“Design is everyone’s job. Not everyone is a designer, but everybody must have the user as their north star,” writes IBM’s designated chief design evangelist in the company blog.

I challenge executives not just sit there and be demoed. Carve out two hours of your time, participate in a Beta-lab and try to carry out the assigned task with to-be-shipped product. I promise the experience will be eye opening, if not jaw dropping.

I challenge anyone who frowned upon the experience of using your own products, whether you’re a developer, tester or consultants. Don’t let it go, file a user experience defect and mark it “NO SHIP”.

Design thinking is about cutting-edge

Designers must know engineering. Otherwise, a great design risks being pushed back by development as impractical. Development tend to use mature technologies, which limit their imagination about what’s possible.

AX researchers serve as the guinea pig trying out cutting-edge front-end development technologies to gauge their practicability and capabilities. In this sense, AX designer and researchers serve as the “R” for our front-end developers.

AX Designer/Researcher

While everybody in an analytics organization need to practice design thinking, the AX designers and researchers are full time evangelist whose focus are on the design and research of analytical user experiences.

A sample of tasks AX designer/researchers can take on comes to mind:

• Foster the design-driven culture. Turn employees into design thinkers with education programs. Train employees on design thinking techniques such as empath maps, as-is scenario maps and storyboarding. See for example the IBM design camp from a marketing perspective.
• Ensure design and code reuse with analytical components. Identify and curate commonly used analytical components. Define the look, feel, behavior and syntactic interface. Provide reusable implementation that can be plugged into multiple solutions.
• Research on front end development technologies. Bleeding edge frond end development stacks can be used to build internal systems such as the Multi-Vendor Benchmarking system or the defect system. Best practices learned are then applied to production offerings.
• Advocate for external analytical user communities. Social media is now the front door to user loyalty. Social media platforms have become the most direct channel to get honest user feed-backs.

References

1. Intuit’s CEO on Building a Design-Driven Company.
2. IBM is gearing up to become the world’s largest and most sophisticated design company.
3. IBM design thinking.
4. Why design thinking works, Harvard business review.