Hi there, I’m back from vacation and am trying to get back on the “Sam writes stuff that he finds interesting” kick, so here goes. But first, an aside: I’m probably the only person who is tired of seeing AI-generated clipart on every single LinkedIn / Substack / marketing email. So for my sake (and to annoy you) I’m going to take what I would have used as a prompt and just draw it myself.
A modestly agile proposal:
I’ve been thinking more and more about what it would look like if legal non-profits actually adopted an “agile framework” to product development. I wrote briefly about it last month, and suggested I might subject you, dear reader, to a lengthier post at a later date. Today’s your lucky day.
Three symptoms of non-agile thinking:
In my mind there are three main symptoms of non-agile thinking:
Gravitating towards the heaviest possible solution for any problem.
Non-profit legal tech is especially prone to this: for whatever reason, tech solutions that meet the needs of private industry and law firms aren’t considered for our use cases.
A default to perfectionism.
Also known as the “making the perfect be the enemy of the good” syndrome.”
As pointed out in this great podcast (and more about this in a later post) at about the 7 minute mark, the default position for over 90% of people is no help.
The “special snowflake” condition.
This is related to the first point, but deserves its own listing. Non-profit legal tech has this intractable idea that all of its use cases are so unique, so special, and so mission-critical, that normal off-the-shelf tools cannot possibly be good enough. For example, instead of using WordPress for an informational website, orgs will go and get a grant for a custom-made one-off platform.
Trying to be a bit more agile, and the MVP idea:
The agile development method centers around the idea of the “MVP” - Minimum Viable Product. Let me give you an explanation by way of an example:
Let’s say that in Anytown, USA, a large percentage of people aren’t showing up to their court hearings. This is causing some bad effects - when people miss their hearing, they automatically lose their eviction case (even when they have good defenses) or have a warrant issued for their arrest. Anytown Legal Aid has approached you, a legal tech company, about building some kind of tech solution to help alleviate this problem, and they have some compelling ideas:
Idea 1: Build a legal information portal site that shows people the potential consequences of missing a court date.
Idea 2: An online portal where people can input their case number, case type, and Zip code, and get a customized & personalized report (in the form of a PDF download) of the potential consequences for missing court.
Idea 3: An online form builder for people to create a court form that requests an extension of their court hearing.
Pretty cool stuff, right?
But there’s big problems with all of these approaches. None of them fit into the actual process that people experience when they have a court case. None of them were designed with input from the actual users (people with court cases). None of them were created using an Agile framework, and here’s how we can tell:
Instead of jumping into creating Products that Sound Cool and Get Funded, the Agile framework starts by trying to get to the fundamental problem, i.e., why aren’t people showing up? Agile takes the actual problem - people not showing up to their court hearings - and ask the simple question of why.
So let’s ask why. There could be a many reasons people aren’t showing up:
People simply don’t want to show up;
People can’t find parking and are late;
People don’t know when or where they have to show up;
People can’t find child care or get time off from work; and / or
Other reasons we haven’t discovered yet.
These are our hypotheses: the theories that we need to test. The testing step has to come as early as possible here. Each hypothesis can lead us down a very different direction - the solution to not enough parking would be very different from the solution to people not actually wanting to show up.
So to test these hypotheses, we need to see what’s actually happening. Go to the court house on a hearing day. Talk to the people that show up. Did they have trouble parking? Did they know where to go? What was hard for them? Talk to the judge. Talk to the judicial assistant. Talk to the lawyers. Get data and information, and you’ll find out things like there’s plenty of parking, but people had trouble figuring out which courtroom to go to (I’m foreshadowing here).
After you test these hypotheses, one stands out as a common theme: people don’t know where or when to show up. It sounds like people need better information about the where and when of their court hearing. This might actually be the problem that needs to be solved.
The next step is to ask “what’s the lightest-weight thing we can do to see if we’re on the right track?” This is the MVP. Here’s the best part: it doesn’t have to be a web portal! It doesn’t have to be an iPhone / Android app! It can be something as simple as redesigning the paperwork people get.
Like I said above, I was foreshadowing, so now I’ll give away the ending. A program in NYC used this type of approach for this exact probelm, and got some great results. They found that simply re-formatting the paperwork people got to be more readable reduced the rate of people not showing up by 13%. Adding a text-message reminder on top of that reduced it even further, by 21%.
Granted a redesigned paper form and simple text-message reminder service aren’t a web portal, and probably doesn’t fall under the rubric of Products that Sound Cool and Get Funded. But do we, as non-profit legal tech people, want to do things that matter, or are we just creating web portals?
In other news:
Maybe ChatGPT lawyers aren’t isolated incidents:
This is a quote from a Florida Bar News article about the new AI ethics guidelines is interesting:
“You can’t imagine the phone calls we get on this issue,” he said. “Every place I go, a DCA judge comes up to me and says a lawyer filed a brief using ChatGPT and there were errors in it.”
Seriously? I think a lot of people were convinced that the use of ChatGPT or whatever in legal filings was isolated to a few notable examples, but my hunch is that it’s way more widespread. Lawyers probably aren’t going to advertise the fact that they’re using it, but I think they’re using it. Maybe alongside ethics rules we need to do some guilt-free lawyer education about how to use the tools? Just a crazy thought.
Anthropic’s terms of use require human-in-the-loop for legal advice to consumers:
I saw this in Anthropic’s Acceptable Use Policy and thought it interesting:
If your business is using or deploying our tools and services as part of providing legal, medical, or financial advice to consumers, we require that you implement the additional safety measures listed below:
Human-in-the-loop: any content that is provided to your consumers must be reviewed by a qualified professional in that field prior to dissemination. Your business is responsible for the accuracy and appropriateness of that information.
Disclosure: you must disclose to your customers that you are using our services to help inform your decisions or recommendations.
Finally, if your business is using or deploying our products as part of an automated service where your external customers or users interact directly with our products, for example chatbots, you must disclose to your users that they are interacting with an AI system rather than a human.
I do wonder how Anthropic will enforce this, but I agree with it.
Editor’s note: all typos are irrefutable evidence of my moral failings as an individual.
Our doctors and dentists have worked out a great system of reminders for appointments.
They text you before and even request a reply.They have at least two incentives, avoiding no shows which waste time and money and provide better healthcare if is stuff like teeth cleaning.
Would typos be evidence of not exclusively using ChatGPT?
Sam you are way too practical. I'm sending you a dog and pony show..