It has been a tumultuous holiday week for OpenAI and the entire AI ecosystem. We’ll catch you up. Also, Happy Birthday to Ctrl-Alt-Operate! We turn 1, and we’ll reflect on how the world has changed in the past year.

Table of Contents

📰 The News: The Wild, Wild West Coast: OpenAI implosion
🤿 Deep Dive: Ctrl-Alt-Operate turns 1 ⭐️

Share

Subscribe now

📰 The News: Trouble in Paradise at OpenAI

Odds are the followers of this newsletter are up to speed on the drama at OpenAI. Even if you are, so much has happened that its worth going over:

A week ago, Sam Altman was fired as the CEO of OpenAI. Why would a board fire the chief executive of what is widely touted as a possibly humanity-altering company? That’s a great question and precisely what everyone was thinking. OpenAI's board cited a “lack of candid communication,” but the ambiguity of their statement only deepened the mystery and speculation.

What followed was something that I hope Michael Brown’s team, and the executives of Netflix, are already planning to make a bestseller and blockbuster. The impact was immediate and dramatic: key figures like Greg Brockman, OpenAI's president, resigned in a show of solidarity or discontent, signaling a deep rift within the organization.

The plot thickened when rumors of Altman's potential reinstatement surfaced, hinting at a tug-of-war between OpenAI's nonprofit roots and its profit-driven ambitions, fueled by major investors like Microsoft. As the saga unfolded, Microsoft CEO Satya Nadella announced that Altman and Brockman would lead a new AI division at Microsoft, a move that seemed to mark the end of their OpenAI tenure. However, in a stunning reversal, OpenAI declared Altman's return as CEO, alongside a reshuffled board, culminating a week of high-stakes corporate drama. This tumultuous episode at OpenAI underscores the intricate challenges of balancing ethical considerations and business objectives in the frontier of AI technology, leaving all of us, and all of Twitter, eagerly watching what comes next.

By the end of the week, Altman and Brockman were back at OpenAI. Oh, and Larry Summers (yes, former Secretary of the Treasury under Clinton, former president of Harvard) is now on the OpenAI board.

What does this all mean for healthcare, healthcare builders and those interested in AI?

In the short term, not much. With the key leadership now stable at OpenAI, I don’t expect the pace of innovation to change. That being said, this week shows how fragile the current ecosystem is. If Altman did leave, and employees left with him, that would create a power vacuum that leaves the whole industry vulnerable.

If you are a builder in healthcare AI, reducing reliance on one specific model or company, making your platforms modular, and being adaptable to the inevitable shake ups of the future is key.

Deep Dive: Ctrl-Alt-Operate Turns One!

Just over a year ago, we sat down and decided to start writing in public (FYI, this was before chatGPT came out). 35 issues later, we are still here for our (mostly) weekly look at the latest, greatest and generally most interesting obscure factoids from the tech world that physicians and surgeons need to know about.

It’s amazing how far we have come since our first issue, and also how little has fundamentally changed. So let’s indulge in a brief trip to the internet wayback machine to take stock of how far we have come, and how far we still have to go.

So here’s our first newsletter:

We identified three early themes that would go on to make waves over the course of 2023:

1. Generative AI is extremely flashy, but difficult to put into high-reliability applications.

2. ML advances are rapidly lowering the entry costs for physicians to get in the trenches of software and ML development

3. The need for a central node to connect clinicians and computer scientists around health care, improving collaboration and accelerating joint projects.

In large part, all of these stories were the central story of 2023, but there were a few things that we didn’t fully discuss (but should have).

1. Healthcare operations will be the first site of AI penetration as this is most similar to existing consumer technologies (e.g. business operations)

2. Efforts to implement HAI within the enterprise will face incredible degrees of resistance from existing stakeholders. Successful efforts combine internal champions, data sensitivity, and successfully address business goals

3. The vast majority of the work remains undone, even at one year in. This will be a 10-15 year transformation project.

But, the investment in healthcare AI has never been bigger; see pitch from VC giant Sequoia. And large healthcare systems like Kaiser and Emory are already partnering and rolling out documentation systems using AI. So we are making progress.

As we approach one year of Ctrl-Alt-Operate, we’d like to put out a call to action to think bigger. The ones who are behind the drive of AI often call it a once in a generation technological change, and use analogies like… the invention of electricity. The power of this cannot be overstated. How do we use this technology to fundamentally change how we practice medicine, how we deliver care, and how we operate on a daily basis for the betterment of patients? This is our challenge, and why we keep on writing.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Share

Subscribe now

Leave a comment

Table of Contents

📰 The News: The News: Some New Tools, Words of Warning, and Wearables
🤿 Deep Dive: Report from Digital Neurosurgery 2023

Share

Subscribe now

📰 The News: Some New Tools, Words of Warning, and Wearables

The clinical documentation ecosystem is getting frothy…

As nicely pointed out by @rikrenard on X, there seems to be a looming shut down of Orthopedics AI documentation company, Robin Healthcare. Yet Simultaneously, Kaiser is doubling down on their AI investments- they announced a partnership with Nabla while simultaneously investing in AI startup Abridge.

Interesting play by Kaiser - perhaps a thesis that there will be many players in this world, it’s tough to predict who will come out on top, so best to hedge.

(As a brief aside, we built dotphrase.ai, yet another AI clinical documentation tool, to be super easy to use and wildly cost effective. But we are biased 😉)

How many hours of your life have you spent editing surgical video for a talk or journal submission? What if AI could give you hours back in the future. Descript's AI video tool looks too good to be true. They’ve already changed how people create video - automatically scrubbing any “uh” or “umms” in the audio. But now they allow you to edit the words you’ve said (via text), and it automatically changes what is said in the video. Next level.

Always a good read, Eric Topol with some words of caution for physicians: shape AI or someone else will. This is a belief Dan and I have held for years. AI is coming - just look at this survey of hospital CIOs/CTOs. Who should decide how AI is used in clinical care settings? If we don’t ensure providers (and patients) get a say, they won’t.

Similarly, the New England Journal of Medicine published a piece on how medical statistics might change in the era of AI.

Two new AI wearables to talk about. The first is Rewind Pendant, which would (selectively) record and transcribe your life, for you to reference or chat with in the future

.

The second is Humane’s AI Pin which premiered at the Paris fashion week.

Is healthcare the perfect domain for wearables? We frequently have both our hands in use, and especially in the inpatient setting are often away from a computer but needing clinical information. Something to think about…

A bit more on the amusing front, Meta launched generative AI stickers, allowing users to create their own little emoji’s to attach to Messenger conversations. Unfortunately the screening on these wasn’t great, and led to some questionable AI generated stickers.

Lots to go over this week. AI moves fast. We help you keep up.

Deep Dive: Report from Digital Neurosurgery 2023

If you’re wondering why the newsletter has been sparse over the past three months, let me explain. We recently undertook a 10-month-long process to found a new meeting, which we called “Digital Neurosurgery” (www.digitalnsgy.com). This meeting, with over 150 attendees and speakers, was a resounding success and created a series of impactful opportunities to reimagine how neurosurgery, and surgery writ large, interacts with the broader AI/ML community.

We will post some of the talks shortly - from luminaries such as Craig Mundie and Sebastien Bubeck $MSFT, leaders at , medical strategics, and ,

I want to a take a few minutes to share the inside story of why we thought yet-another surgical meeting was needed, how we went about creating this meeting, and what we hoped to achieve. Lastly, I’ll outline what will follow from the meeting.

Why create a new meeting?

Our mutual interest in all things surgery, data, and computer vision should be no surprise to readers of this newsletter. Yet, even after dozens of abstracts, papers, awards, and so on, we felt that publishing another paper in a surgical meeting wouldn’t move the needle. There is a great deal of phenomenal work happening in surgical data science, “surgical AI”, and many other “slashies,” but translating that work into impact remains challenging. We wanted to shake things up.

A few focused meetings bring clinicians into discussion with computational advances, such as MICCAI (and CLINICCAI), while remaining rooted in academic discourse. In order to impact the direction of our field, we have to look beyond existing structures. So, we sought to aim bigger: including venture, big tech companies, medical device manufacturers, and others who are typically left out of the conversation. Furthermore, we recognized that simply enlarging our view within medicine wouldn’t do justice to the vibrant community of computational innovation that has already brought products to markets in selected use cases. So, we invited voices from outside of medicine: sports, self-driving cares, and military/defense. These areas are farther along the innovation curve - we wanted to solicit and internalize their experiences.

How did we do it?

Well, the simplest answer is that we created a team - including one of the foremost members of our field, Dr. Sam Browd, an accomplished neurosurgeon scientist innovator and entrepreneur who has been walking the walk for two decades. We brought in other neurosurgeons with defined expertise, interest, and reach. We formed a small planning committee with a few kind souls who donated their time and effort to make sure every detail went off without a hitch (Caroline, Rick, and Amanda - much love to y’all). Oh, and we got some financial sponsors, without whom the event wouldn’t have been possible. Then we beat the bushes for attendees, speakers, interesting folks from all walks of the world. We did the logistics and legwork (many thanks to Stanford Neurosurgery, and many others for supporting us along the way). Speakers cancelled at the last minute, venues changed pricing, and all the usual joys. We used a couple technology elements that I would recommend (eventbrite, though others may be cheaper), and an app (Whova) that was excellent. We worked our tails off for the last four months to get everything into shape and used every resource and connection we had. It truly took a village

How did it go?

In one word, transformative. Having leaders from Neurosurgery (department chairs, AANS/CNS/Washington Committee leadership, etc.), engineering/industry, and business/venture in one room was powerful. The fact everyone was discussing the future and the ability for AI to change how we treat our patients was even more powerful. But don’t take our word for it. Here is some feedback:

“I have attended these types of meetings for 20+ years. This has been the best yet”

“I came here as a blank slate just ready to learn, and I have learned more in 3 days than I ever thought possible”

“You must do this again. It would be a tragedy not to”

“Everyone raves about the meeting. It must continue.”

Collaborations were started and new colleagues introduced. Even a year ago, when Ctrl-Alt-Operate started, people didn’t really care about A.I. How things have changed!

Rest assured, when tickets for 2024 open up, Ctrl-Alt-Operate readers will be the first to find out.

Now what?

Time to capitalize on the momentum from the meeting, both inside neurosurgery and beyond it. When we wrote our prior editorial we laid out our plan for this year, including the post meeting period. We want to achieve several aims with the consensus built around the meeting:

• 1. Catalyze “Organized Surgery” around the topic of AI. We are discussing the creation of a joint section / task force on AI to facilitate data standards and best practices and speak with a united voice. We aim to connect these efforts across surgical societies and other organizations. Lastly, we aim to bring hospitals and payers into the dialogue.

• 2. Accelerate multi-institutional research structures. We have founded one of these, the Surgical Data Science Collective (SDSC, www.surgicalvideo.io), oriented around surgical video data, but additional efforts are needed. We firmly believe that research organizations that can operate outside of the identity and data management space of a single hospital or academic institution will be the engines of progress. We demonstrated the progress of SDSC at the meeting, in the hopes of encouraging les autres.

• 3. Sharpen future conferences to accelerate cross-disciplinary innovation in ML and computer vision for surgery. We wrote that “new meetings can galvanize progress and create challenges by using a [Common Task Framework] to focus the attention of the field on clinically impactful use cases within neurosurgery… Neurosurgeons should partner with scientists to define [Frameworks] that are clinically impactful… and operationalize innovations in our field.” Expect this to be a major theme of our work in the next twelve months.

• 4. Increase awareness, adoption, and analysis of surgical video. In essence, we aim to persuade surgeons that they *should* be recording and managing their precious video data, as well as other preoperative data sources. One opportunity that we haven’t yet tapped into is the partnership with patients, who may be our best advocates for surgical quality and care improvement.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

The past two weeks have been monstrous for artificial intelligence developments. Like most things AI, they come to the software developers first, then to the general public, before making their way to healthcare. Consider this newsletter a heads up so when you see these making their medical rounds, you’ve got some context to go off of.

Table of Contents

📰 The News: Hello multimodal models!
🤿 Deep Dive: Milestones. Where are we in the surgery x AI journey?

Share

Subscribe now

📰 The News: New Models Galore

This has been one of the craziest weeks in AI. That being said, I feel like I say that every other week at this point. But, it continues to be true. Here’s what you need to know.

In OpenAI’s own words: ChatGPT can now see, hear and speak.

This is a huge update, and allows chatGPT to engage with the world using images. We’ve talked for months about computer vision, or the ability for computers to analyze pictures and video. Combine that with the logic and reasoning that chatGPT allows and you have a very powerful system. Here it is “debugging” how to adjust the seat on a bike.

The implications in radiology, and image processing are fantastic. Imagine preparing for a case and chatting with the PACS to get lengths and widths of relevant anatomy - guided and directed by you the surgeon. What if reviewing video was as easy as chatting with chatGPT, asking it to “Find a 30 second segment of video showing the anastomosis.” We aren’t there yet, but can be soon.

Not wanting to be outdone by themselves, OpenAI Announced DALLE-3, which has some pretty amazing text capabilities. Here’s a very sad looking Mario after breaking up with Princess Peach.

It’s not unreasonable to start planning your postoperative scans. Example: given pre-op scans of a patient with bad spine deformity, it’s not outside the realm of possibility to start seeing how different constructs might affect coronal or sagittal imbalance, postop. Now, they would be generative, and not based in reality, but would still be quite remarkable for patients to see.

RayBan and Meta have partnered to create Smart Glasses .

The possibilities here are endless. Walking in a patients room and having their scans, meds, or notes loaded up on your glasses in case they need to be referenced. The possibilities in the operating room, with analytics and navigation require their own deep dive. This is cool technology, and we should keep an eye on it.

Bringing it back to the healthcare world, Oschner Health to integrate generative AI into patient care, while Abridge partners with Emory and Epic . This gets back to the ultimate question: will enough people be willing to pay for Generative AI tools that these companies see profits? The jury is still out, but at least there are a few large systems willing to put their money where their mouth is and try it out. For the sake of the AI ecosystem, we should hope they do well.

🤿 Deep Dive: Milestones and Where Surgery x AI is Today

I would strongly recommend checking out our new meeting, Digital Neurosurgery, www.digitalnsgy.com, an inperson-only event from October 12-15, 2023. This is a new opportunity and forum for surgeons, technologists, venture and academics to come together in one room and work on critical issues for surgery, neurosurgery, AI and medicine. Get the last tickets today at www.digitalnsgy.com

In case you’ve been creating a gen AI startup for medicine, starting a (nonprofit) company, creating a new meeting around surgery and AI, running a research lab, being a surgeon, or enjoying your summer in other ways, you might have missed the innovations we’ve described in the section above. But few could miss the Lasker Prize (“the American Nobel for biomedicine”), awarded to the creators of Alphafold - perhaps the first major prize in biomedicine to be awarded for AI (Eric Topol did a terrific post on this). As our days shorten, I wanted to take a retrospective look back at what we’ve learned about building in AI - the enduring lessons, the good and inspiring stuff. Then its on to the new news for next week.

Three lessons we learned from building in the first wave of AI

1. The journey from idea to first prototype has never been shorter - but the devil is in the healthcare details. Repeat after me: Healthcare is not a consumer technology product. The SAAS webapp tech stack has never been better - a single full stack dev can create an impactful MVP in 10 days. The art (for subject matter/product/strategic types) is picking out a real clinical and business problem and distribution channel for the technological solution. For example, many folks (including us) have identified medical note writing as a multibillion dollar TAM ripe for disruption, a real clinical and business problem in medicine. But the precise, accurate, and timely problem statement has never been more important. Are we solving for physician time? Staff coordination? Patient satisfaction? Accurate capture of billing codes? These are all real clinical issues that stakeholders want to see solved - but which stakeholders are willing to pay for them? How do we do business with the hospital institutions? Private practices? Individual doctors all around the world? Each of these issues requires recognition, planning, and execution at a level that is quite different from throwing up an AI avatar photobooth app (remember those??). And a level of commitment to the cause that isn’t commonly found in hackathon level products. Now, in Fall 23, we are starting to see the fruits of some impressive labor over the preceeding year, but it takes a year or more to see these results.

1. AI is still the mind virus of 2023. The most common questions we are being asked at our speaking engagements are still defensive and predictive - “I’m scared of this new thing I don’t quite understand, how will it ruin my 2024?”, “What are the trends going to be for AI in healthcare in 2024”, and so on. I don’t have a crystal ball, and predictions are hard especially about the future, so I generally demure from these requests. Here’s what we do know, as of fall 2023:

   • AI is minimally implemented in medicine outside of a few small niches

   • Hospitals can remain technologically incompetent longer than any individual company can remain solvent - we are still using pagers.

   • Business and billing functions drive adoption of enterprise systems, clinicians drive adoption of innovative clinical products.

   • AI is stuck in the middle: not clinically useful outside of a few small niches, and business functions are slow to change

   • The adoption of AI across clinical domains has been uneven and siloed - like medical hardware devices, not like consumer software (when the cardiology department upgrades its ultrasound machines, the ICU doesn’t necessarily get new machines).

   • Every major medical institution is scrambling to create partnerships, innovation centers, and take a “leadership role” in AI - but few want to suffer the consequences of being early adopters of failed technologies.

   • Major unanswered questions about data and algorithm ownership, access rights, co-development and so on have not yet been answered or solved. You could focus on solving these in your domain (as we are working in surgical video)

1. •”Guerilla adoption” of unsanctioned AI products in the hospital (like Windows Copilot) is inevitable and a potential distribution channel for small companies. For example: clinical decision support apps like MedCalc (but with AI), Glass Health, and others may never be sanctioned but may infiltrate medical schools and early year residents.

   Most of these points are unchanged since Fall 2022 (when we started writing), but there are some fundamental opportunities

   • Attitudes towards AI are being defined as positive, even envious. When consumer grade AI significantly outpaces that of medical AI (a fact that is obvious and will become even more glaring in 2024), we are poised for a disruptive break in medical practice.

   • AI human teaming is still not regarded as a fundamental area of research (You should work in this area). It isn’t impressive to say ‘chatGPT can write a medical note’, but it is impressive to say ‘here is a system for teaming up humans with AI to write notes, here are the experiments we tried, and here is how we will plug-n-play the next AI that comes along

2. The brightest days of healthcare x AI are still ahead of us - and you’re still early. We are getting to the point where visiting lecturers on chatGPT pack auditoriums at our hospital grand rounds - whether they should or shouldn’t - and there are more AI x medicine “symposia” than you can shake a stick at (sorry). But no one has come up with an answer or the answers to any of the questions that we ask in this newsletter, which means you can join in, get your hands dirty, and make an outsize impact.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Share

We had a little summer hiatus as we prepared for the academic year and settled into some newer roles. But, we are back. And while we’ve been gone, AI hasn’t stood still. Let’s walk you through the major milestones, including Med x AI announcements from the major players in the field.

Table of Contents

📰 The News: Big Tech makes Big Bets on Medical AI
🤿 Deep Dive: Human-AI Teams in Practice

Share

Subscribe now

📰 The News: A Recap of the Past Few Weeks

Over the past few weeks, all the major movers and shakers in the world of AI had major announcements. We’ll go through them, highlight the ones which might affect your clinical life, and talk you through what to expect in the future.

First, Teladoc has announced in integration of Nuance’s (owned by Microsoft) DAX AI clinical documentation system into its platform. Teledoc has over 50 million paying users, and their Solo platform is enterprise ready and allows health systems to engage with tele medicine without re-vamping their entire infrastructure. If there’s any system that can get AI into as many hands as possible, having Nuance, Microsoft and Teledoc on the same page is one of the best ways to do that.

Microsoft also announced a collaboration with the pathology AI company Paige, to make the world’s largest image based AI model for cancer. What exactly that looks like and how it assists in improving… anything … is left to be seen.

More recently, 3M (yes, that 3M) and Amazon Web Services (AWS) announced a partnership to bring AI documentation to 3M’s M*Modal speech-to-text software. Next to Dragon’s nuance, this is definitely the most high-profile speech to text system, now partnering with one of the big boys of tech. Let the games begin.

But perhaps all that glitters is not gold. A recent WSJ article looked into the AI note-writing startup DeepScribe and found that while most of its users believed their technology was using AI to write notes, much of the work was actually done by actual scribes.

Not to be left out, Apple is allegedly creating a chatGPT competitor, called Ajax. Combined with the new VisionOS, there is no reason Apple shouldn’t be in the AI cage match. Meta has recently announced Llama 2, which is an open-sourced, commercially licensable LLM. This is a major departure from the chatGPT and Bards of the world, which are completely closed models. This should hopefully allow builders of all shapes and sizes, including clinical teams, to start building robust infrastructure on top of these models. Maybe DeepScribe should take some notes…

Google had a few massive announcements related to medicine. They published their MedPaLM paper in Nature, with the inflammatory title: “Large Language Models Encode Clinical Knowledge”. Their model achieved 67% on the USMLE, and were judged to be helpful to laypeople 61% of the time. While these models are impressive there’s plenty left to wonder about how it can be rolled out in a useful manner. They also launched a multimodal model called MultiMedBench which can tackle a wide range of data:

This is a natural extension of current capabilities of these models. Medicine is inherently multimodal, and so the ability to capture and encode a wide variety of data types, including images, is exactly what we expected would happen. Not to mention, everyone forgets OpenAI’s GPT-4 is actually multimodal as well. But to our knowledge, no one besides a few pre-selected partners have access to that system.

Hollywood and AI continue to be at odds, with writers and screen actors strikes painting a dystopian future for what A.I. could mean for work. The idea of using someone’s AI generated likeness is of course infinitely cheaper than actually paying for their time.

Question for the clinicians:

Assume there was an AI model that could answer initial triage questions with comparable outcomes to an urgent care. And assume patients were okay with seeing this A.I. clinic, and were actually going there. How would you as a clinician feel? Would there be an element of feeling like your territory was being taken from you?

For those who think there’s no clinical evidence of AI being helpful, those times may be changing. A new study in Lancet Digital Health showed that human-AI teams are noninferior to human-human teams in breast cancer detection in screening mammograms.

I think in some way we as clinicians feel that there is a uniquely human element to what we do, but these questions need to be addressed and dealt with head on. A.I. isn't replacing doctors anytime soon, but like with all technologies, its progress will likely disappoint both optimists and pessimists.

🤿 Deep Dive: Human + Medical AI teams Make it to Mainstream Journals

As discussed above, a recent publication in Lancet Digital Health by Dembrower et al., showed that human + AI teams performed with non inferiority to human-human radiologist teams in detecting breast cancer in mammography. The idea of human-AI teams are an important concept which will help determine the ability for AI systems to work alongside clinicians. It’s worth a deep dive.

The study was conducted by a Swedish group, who enlisted an AI system to work in parallel to all radiologists at one hospital in Sweden looking at over 55,000 mammograms. The aim was to test the detection and false positive finding rate when the gold standard workflow (a read by two radiologists, blinded to the others response), versus a radiologist plus AI, versus two radiologists plus AI.

If any one of the three “readers” flagged a scan, it was sent to consensus by human radiologists, and then down the various diagnostic pathways until a final determination was made (via tissue, further imaging, etc).

The takeaways:

1. 1 radiologist plus AI as a second reader performed identically to the gold standard. In fact, with AI and 1 radiologist, 11 more women had breast cancer identified than with 2 radiologists alone, a 4% increase in screen-detected, true cancers.

2. Single reading by AI was also non-inferior to double reading by two radiologists.

3. There were 19 cases where the AI was the only positive screening reader, prompting consensus conversation and subsequent diagnosis.

4. From the paper: “Double reading by one radiologist plus AI caused a 21% (868/4104) increase in the number of examinations with abnormal interpretation (following consensus by humans)”

Let’s unpack this.

The Radiologist + AI team had the exact same performance as a Radiologist + Radiologist team. This has massive implications. As the authors suggest:

This would be a considerable time saver given the demands for precision diagnostics and increased imaging performed worldwide.

Another interesting point was that the AI + Rads group had a higher number of scans sent to consensus compared to Rads + Rads. This means that there may be a slight difference in how the AI is interpreting these images compared to the formal training a radiologist may have. Perhaps different systems with different training data may see different results.

This study did a great job of actually following through on the suggestions made by AI. All AI screened scans had to go to consensus. As a result, 19 women’s cancers were detected when they would have otherwise fallen through the cracks. The authors also test how AI would have done alone.

They come to the conclusion that it actually performs as well as the gold standard (two radiologists). However, they make a point which I think gets to the core of why human+AI teams are the way forward, versus AI alone.

This sentiment is unsettling amongst physicians, and likely amongst the public as well. It creates important questions around reliability, public perception, and in the US, liability.

The Shortcomings

While the study was quite robustly designed, it was not randomized. The true litmus test would be to randomize scans into either Rads+AI, Rads+Rads, or Rads+Rads+AI cohorts, and determine the outcome. Knowing it Rads+AI is non-inferior, it is reasonable to then go forward with a larger, prospective, randomized study.

One important consideration is the significant testing which went into choosing an AI system in this paper. It is worth questioning what criteria you would choose when deciding what AI system you would want to work with. We’ll leave it up to you:

Best of Twitter:

Maybe one day Elon’s feud with substack will be over. What should we replace this with? Let us know via email.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

We’re refueled and recharged to focus on our mission of delivering you what you need to know about the state of surgical AI in 2023.

We can now announce registration is open for the Digital Neurosurgery 2023 Conference, in Palo Alto, CA from October 13-15, 2023.

www.digitalnsgy.com

Leaders in AI and surgery are joining forces to chart a course for the future of our field. Sign up today since our room blocks are going fast! If you’re a student or trainee and need assistance with registration or have other logistical questions, email info@digitalnsgy.com

Our deep dive continues from the prior week to define the landscape of AI-enabled tools in the operating room in 2023. These are actual companies shipping actual products that you might decide to use in your cases next week. We’ll give a brief overview of each market segment this week and then go into the specific players in the weeks to come.

Table of Contents

📰 The News: Wild, Wild West No More? Enter: Government
🤿 Deep Dive: State of the AI Landscape in Surgery
🪦 Best of Twitter: RIP

Share

Subscribe now

📰The News: Governments Getting Involved with AI

This week in AI news, we discuss a growing trend of governments and governmental agencies dipping their toes into the AI landscape. And unlike the usual oversight committee hearings (I will again, link the hilarity that was the TikTok CEO testimony before Congress), agencies are actually taking out their checkbooks and committing real resources to A.I.

The NHS announced a £21M initiative for improving healthcare efficiency using A.I. for diagnostic tooling. They aim to start with Chest XRs and stroke detection - well known AI use cases that have seen significant clinical adoption and validation.

…Is it just me or does 21M pounds seem like how much it would cost to implement any initiative at a handful of hospitals? Considering Inflection.AI just raised $1.3 BILLION, I am skeptical as to the scale this will enable technologic adoption.

Back on this side of the pond, The New York Department of Financial Services is exploring purchasing a supercomputer and hiring AI experts to run it, to better understand AI and use it to predict bad actors in the regulatory sector. Similarly, the FTC pre-announced a national rule making surrounding data security, specifically outlining how advanced computational techniques are used in highly regulated industries like healthcare.

Why does this affect you? With the clip that AI is moving, we can rest assured that the agencies that govern clinician’s lives, from the FDA to CMS, to even the AAMC / ACGME / American Academy of {insert specialty here} will soon weigh in AI, and start to regulate how and where their members can utilize it. Maybe this is okay? It’s becoming increasingly clear that chatGPT accounts are being compromised. We’re always trying to balance innovation with safety. I hope these guardrails don’t preclude the innovation we expect when it comes to these tools.

At the end of the day, we need to focus less on the mechanics of AI systems, and more about how they can actually be deployed. An example recently highlighted is Insilico, one of the many AI-powered drug development companies. They recently got FDA approval for an AI-discovered drug targeting idiopathic pulmonary fibrosis, and are in Phase II clinical trials having just dosed their first patient. They also recently announced a transformer-based platform which can assist in identifying factors associated with aging.

Many specialties, particularly surgical ones, suffer from the small data problem. Diseases are too rare to make hyper-specific inclusion/exclusion criteria for clinical trials, and resulting trial outcomes tend to have obfuscated results or come with a laundry list of caveats. The ability to generate new drugs, or even simulate trials with any sort of validity may allow us to get to larger scale trials faster, or allow us to design better trials knowing the next trial is only months, not years, away.

Finally, we’d like to remind everyone that if you’re reading this newsletter, you are so far ahead of the curve.

A new study by The Verge found that >40% of people haven’t even heard of chatGPT. Forget about the rest of the field - pretty much an afterthought. Not to mention physicians themselves are a rather tech-backward bunch. You are ahead of the curve, and you must continue to keep up with the tools to stay there.

Interested in building something in healthcare x AI? Need some help to get started? Shoot us an email with your idea / project and we’ll send you some resources to get you going.

Send us an email

🤿Deep Dive: State of the AI Landscape in Surgery (Part 1: Creating the Roadmap)

Last week, we gave you an overview of AI in the hospital and medical records. This week we turn our focus to our favorite place: the operating room. Rather than being forward-looking or visionary, as we often want to be, we’ll now focus on the goods that can be delivered today. This practical tour through commerically available solutions will be valuable to all of our readers who want to know more about the technologies they can deploy today.

Wait, What is AI in Surgery?

Artificial intelligence is generally defined as computational systems that have the capability to produce human-level performance across reasoning tasks. The subset of artificial intelligence we're most interested in is machine learning, which are systems that are capable of improving their performance without explicit instruction. Almost any piece of software that contains the ability to intake information and output information is rapidly becoming an artificial intelligence system. Even the cautery device goes to an electrical system with a circuit board - as soon as a company figures out how to monetize the “AI electrocautery”, it’ll be sold to us. (Side Bet: this will happen before Q1 2025) The real question will soon be: what isn’t AI in surgery?

Thus our mission: to define where we are today, looking at common threads of use cases and scenarios you might find yourself in. You’ll find that most of these companies rely on sensors and output devices, commerical platforms (hololens, Meta Quest, common robotic or navigation systems) and other similar elements. A major difference is the software and specific optimizations for defined use cases. Let’s dive in!

Here is the roadmap for our survey of the AI landscape in Surgery for July, 2023.

Each week, we will turn to one of the following areas of AI in surgery and review the current state of the art affecting the operating room today. Many of these examples come from our experience with neurosurgery and orthopedics, and we welcome insights from our audience about other commercially available AI tools. Initially note, we aren’t going to discuss AI-radiology or AI-pathology, even though those are two of the most common use cases in the Software as a medical device (SaMD) marketplace today because of our focus on perioperative care.

1. Training and Education (Examples: OssoVR, PrecisionOS, Figure1). Although most platforms focus on virtual environments and data sharing, we believe these organizations will be the focus of AI-generated assessments in the months and years to come. If they have anything to say about it, “See one, do 100 in the virtual” might become the new training paradigm. From an education standpoint, image and data-sharing platforms are poised to become important AI model training grounds even as they continue to train human surgeons.

   

2. Preoperative planning and decision support (Examples: Nuvasive iGA, Medtronic UNID / Medicrea, Atlas Pathfinder). Surgeons are looking for preoperative plan development for their most challenging cases. Difficult problems can have multiple, potentially correct solutions. Surgeons consider different approaches, such as the use of a larger or smaller incision, the specific trajectory to navigate to a target area, or the specific maneuvers to perform once the target is reached. Creating a plan before the first incision is made and predicting its downstream effects could reduce the likelihood of poor outcomes or treatment failures.

   

3. Pre-/Intraoperative Visualization Assistance (Examples: Augmedix, Novarad, ImmersiveTouch, SurgicalTheater, Medivis, OpticSurg, SentiAR, BrainLab, Medtronic Stealth, Zeta Surgical). During surgery, surgeons mentally integrate a variety of imaging modalities including radiographic images and the actual visual data collected during surgery to create an minds-eye view of the patient. Performing the equivalent of multiple 3-D mental rotation task while manipulating delicate and vital structures can lead to predictable human error. These organizations create patient-specific representations that can be displayed before or during surgery on novel or existing displays.

   

4. Intraoperative imaging (Examples: ActivSurgical, Proprio, Advanced Scanners, InveNIO, Medtronic GI Genius, Olympus x Odin Vision). Although preoperative imaging can be quite useful, what about all of the new data seen at surgery? These tools use the data from existing sensors such as endoscopes and laparoscopes or provide new sensors allowing the perception of the previously imperceptible. But we just said that surgeons might be troubled by the myriad of information already available to them, how could they handle more? You guessed it - these systems deploy computer vision and artificial intelligence methods to streamline and integrate novel sensor types into surgical workflows and displays.

   

5. Intraoperative Effectors aka Robots (Examples: Intuitive, Stryker Mako, Medtronic Hugo + Mazor, CMR Versius, Globus ExcelsiusGPS, Vicarious, ROSA, Moon Surgical, Asensus, Corindus(?spinning down) and many more). The most classic use case for surgical technology in the operating room are robotic effectors. Now, these aren’t robots in the classical sense, like the roomba you have cleaning your house. They don’t (generally) move without specific user direction, but they translate the surgeon’s movements into the movement of the arms. That statement is about to change with the introduction of semi-autonomous and supporting robotic tools, including the Moon Surgical assistant as well as the orthopedic surgery robots that plan and execute cuts during arthroplasty (joint replacement surgery). Many new technologies are coming to the market in this high-cost, high-growth area

6. Postoperative decision support: (Examples: none). This is an area ripe for exploitation by surgeons and surgical data scientists. Carrying forward intraoperative data into the immediate postoperative period to minimize adverse effects, reduce costs, and improve long-term patient outcomes represents a major opportunity. There’s a lot of exciting research being done in this area, and yet…

   

7. Postoperative analysis of performance (Theator, C-SATS, Hutom, Proximie, Caresyntax, Medtronic x Touch Surgical). This is an area near and dear to our hearts. Suffice it to say that this will be a major focus of our series, so we don’t want to spoil things too much :)

So there you have it, that’s our summer newsletter roadmap for surgical AI. Of course, new events will come up and certain segments may have to be put on hold, but we are very excited for what’s to come.

🪦 Best of Twitter

Any suggestions for what you’d like to see here instead?

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

This week, medicine x AI was featured in front-page articles in the NYT and WSJ, which we will recap below. We’ll even give you a small mission statement based on some of our earlier lessons learned in AI and ML. Our deeper dive builds on the news updates with a more incisive look at AI use today.

Table of Contents

📰 The News: From the front page of the NYT to the reference list of Nature, AI is everywhere
🤿 Deep Dive: …OK so Where is AI in the hospital?
🪦 Best of Twitter: RIP

Share

Subscribe now

📰The News: From the front page of the NYT to the reference list of Nature, AI is everywhere

Here’s a quick rundown of the top articles in mainstream media this week in AIxmedicine:

Are physicians using chatGPT to be more … humane? AI doesn’t get tired and doesn’t have the same counter-transference problems as humans (but it very well may have its own hidden biases). Caveats abound

On the other hand, hospitals are misusing AI to overrule human clinical decisions. It should surprise NO ONE that healthcare IT systems’ implementations of AI are at high risk of making things worse for patients, caregivers, and hospital systems. Here are the underlying laws that we should consider:

• Goodhart’s Law (Once metric becomes a target, it stops measuring the underlying)

• Smith’s Law (“Murphy was an optimist”)

• Arthur C Clark’s three Laws (especially #3, advanced tech is indistinguishable from magic)

• the Dilbert/Peter principle (at any moment, all managers in an organization are incompetent)

• Brandolini’s law (the effort required to refute “BS” is an order of magnitude greater than that required to produce it … and it took all the GPUs to create the latest shiniest LLM)

• Shirky’s principle (“Institutions preserve the problem to which they are the solution.”)

So where does this pessimistic view take us? Healthcare AI will provide magic-sounding solutions that are flawed but irrefutable, attracting minimally competent leaders & mis-incentivized organizations that are targetting useless metrics, and everything that can go wrong, will.

Ever wanted to know how much doctors make? A guy with no coding experience built a data aggregator from physician postings in one month using chatGPT. Click here to learn how he did it. Is it accurate? No. Is it useful? YES! This is a demonstration of going from 0 experience → web app, and an example of the power of aggregating information only present in disparate corners of the web! You could do this (better).

The eyes are a mirror: From high-quality images of eyes, neural nets can rebuild the reflections that the subject is looking at behind the camera. Watch out

!

The Citation Question - what you need to know in June 2023. With ChatGPT inching towards mainstream adoption, major journals and publishers have begun to develop policies, style guides, and references regarding the use of generative AI in text, images, audio, and video.

Generative AI (such as large language models (LLM) and Generative Pretrained Transformers) is a family of algorithms that create a “most likely” output to follow the inputs. The output might be more text, an image, an audio file, a video, or some other type of data, depending on the underling algorithm. For example: If you type a question (called a “prompt”) into chatGPT4(3.14.2023,no plug-ins), it will read your prompt and generate a series of characters in response. Owing to the process through which the algorithm was created, it is overwhelmingly likely that you, a human, will recognize its output as a response to the question.

Nature says: “no” to images and video, “yes” to text. In a recent position paper, the Nature publishing group says that it will not publish images and video created using generative AI. Text, for now, is permitted as long as it is appropriately identified as such.

APA provides a style sheet. For those of you using the APA publication style, here’s how to cite LLMs. And here’s the AMA policy.

What about Grammarly, gMail, and MS Office? We’ve all lived in a world where spell-check and grammar-check are baked into our word-processing systems, and all of these products are increasingly incorporating generative AI. This article, along with most of what I write, is written using Grammarly. But do its recommendations come from a generative model or a “different” AI? And is it so obvious that one is acceptable, and the other isn’t? Furthermore, these proprietary details are obscured in the user interface. There is a separate AI product that is being sold for explicit text generation (“Write an email to Mike telling him he’s fired, be polite, reference the Noodle Incident and the Salamander Incident.”) Should I report the entire “tech stack” to Nature in my next submission?

🤿Deep Dive: How is A.I. Being Used Today?

We, including on this newsletter, oftentimes discuss A.I. as a monolith that will one day “arrive”, as if it is a space shuttle we expect to land somewhere. The reality is it can be treated more like a class of drugs, like immunologic therapies. Advances in technology / research enabled the development of AI (like immunotherapy). As more therapies are developed, some will be duds and others ultra mega hits which become mainstays of treatment from here on out. And as the years go, immunotherapy (and AI) will get integrated into the care continuum so it’s no longer a question of “is it here yet?”

But, for now, let’s dive into where we are and discuss how AI has actually been used in the hospital setting. We're kicking off with a study from our neurosurgical colleagues at NYU, published in Nature, which is perhaps one of the best AI in medicine papers we’ve read.

In this study, the researchers harnessed the power of unstructured clinical notes from electronic health records. They took their own patients’ notes, and pre-trained and fine-tuned tuned a large language model across a number of operational tasks: including predicting 30-day readmission, in-hospital mortality, and length of stay.

The team then not only evaluated the model’s performance, but subsequently deployed the model into their EMR, and in front of real clinicians for a trial period. The beauty of this work is that it isn't just about building a model; it's about creating a framework that can be developed and deployed. This takes effort and significant institutional buy-in.

Now, not all that glitters is gold, and the positive predictive value of the model was quite poor (meaning: if the model over-called things). But, as we have argued time and time again, the standard should not be 100%, it should be the ability of clinicians today. How good would you be at predicting whether a patient would be readmitted just by reading their notes? What about if you took care of them?

Impressively, the team then deployed this model prospectively, and attained qualitative feedback from clinicians regarding the readmissions the model caught. From the figure above it’s clear that a significant portion of these were predicted, unplanned, and penalizable (by Medicare) readmissions.

The paper also hits on an interesting concept of bias- that a model trained on data may not be representative of the data it is deployed on. In this case, the trained : deployed data could not be more 1:1. They are all patients within one health system. Is this a better model for training? Creating frameworks and structures but training small, non-generalizable models?

But what about the Operating Room? A.I. has already made strides here as well. Recently, SAGES announced an extended partnership with Theator, an A.I. computer vision company providing data analytics in the OR. They're striving to enhance surgical outcomes by providing real-time insights and identifying areas for improvement. The ambition? More precise surgeries, reduced surgical times, and ultimately, better patient outcomes. What is left to be seen is how these insights are not only delivered back to the ones in the OR, but actually taken up. As we’ve seen, things we do for no reason is a common refrain in medicine, and changing practice may take much more than simply data.

Separately, Proprio recently received FDA clearance for their spine navigation system. They are leveraging computer vision to circumnavigate the typical constraints of navigation techniques. Their systems enable real-time visualization of complex anatomical structures, reducing variability in navigation systems once the surgery begins.

But, the integration of these tools is not without its own tough questions. A few for you to consider:

Do we ever mandate the use of AI?

How do we reconcile differences when seasoned clinicians disagree with A.I. recommendations?

The recent piece in the Wall Street Journal discussed the conflict between nurses looking after patients and an AI model telling them the patient was septic. This raises these questions, highlighting the dichotomy of decision-making in a health crisis: human versus machine.

Many physicians might baulk at the idea of allowing an AI model to sway their decision-making process. But as always our radiology colleagues are leading the way in this field, and have shown that at all skill levels, we are biased towards believing the A.I.

If anything, the sepsis algorithm vs nurse story tells us that not all A.I. is created equal. It’s widely known that the Epic sepsis alerts are at best horribly annoying but actually likely detrimental to care. But, it’s also widely known in the community that the landscape of A.I. in 2023 is starkly different from when the sepsis algorithm was introduced. Would a sepsis alert 2.0, built on the backs of GPT-4 (or NYUTron), do better?

Ultimately, we must answer two fundamental questions:

1. At what point (Positive Predictive Value, Specificity, Sensitivity) do we allow A.I. models to enter the patient care pathway?

2. At what higher threshold do we actually mandate their use?

A.I. isn’t a fad or some far-away concept which one day may “arrive.” It has been slowly working its way into our workflows for the past few years and will continue to do so with more force.

As described in our mission above, it’s our job as those in the arena to demystify and adjudicate and ask these tough questions. We owe this to our colleagues, our profession, and to our patients.

🪦 Best of Twitter

Any suggestions for what you’d like to see here instead?

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Today we’ll be going over Apple’s announcements from their worldwide developers conference and how it could impact medicine, and break down a new government program meant to mirror the defense research organization DARPA.

Table of Contents

📰 The News: Apple Enters the AI game…with a Headset
🤿 Deep Dive: ARPA-H for Surgeons interested in AI/ML
🪦 Best of Twitter: RIP

Share

Subscribe now

📰The News: Apple Drops VisionOS

Apple had their annual WWDC (worldwide developer conference) this past week, and they launched their fist major product since the iPhone… a $3500 headset known as Vision Pro

Imagine your Mac or iPhone but displayed in your real world. That’s the sell. The product is set to ship in “early 2024”, but developers got a chance to test it out at the conference. The results were overwhelmingly positive. Some of the quick hits included:

1. Eye and hand tracking were nearly perfect (more on that soon)

2. Latency was near zero (more on that soon)

3. The ability to interact with your normal environment + Vision Pro’s environment was key

Check out this wild thread on how Apple used what they describe as neurotechnology to correlate eye tracking movements and finger movements with predictive analytics to be able to pull/load/predict the app you’re going to use before you even start reaching for it.

At a $3500 price tag, it’s clear the market for these will not be the everyman - but that’s what happened with the iPhone at its original release as well. Lets quickly jump into some medical use cases and times our mind was blown, thanks to Apple:

1. Finger, hand and eye tracking was perfect. Are you kidding me? Apple placed 12 sensors/cameras around the goggles and can pick up subtle movements. This is a game changer for predictive analytics, surgeon movement tracking, and the entire field of surgical data science - if we can get our hands on the development kit.

2. This might be an early version of a consumer oriented Brain Computer Interface. Taking pupillary movements, likely saccade-like and before even voluntary motor function has been enabled, is clearly moving along the brain-computer-interface timeline. Who know Apple would be the one to productize “the eyes are the window to the brain”?

3. What will be interesting to see is how the visual information is displayed relative to the real world. This could have enormous implications in the OR - if data could be provided to a surgeon, without being obtrusive to the field of view (but still visible) … they might actually use it. I am hopeful.

4. I hope the world of inpatient rehabilitation is overhauled with VR applications. The ability to gamify, transport patients to places they want to be, or worlds that don’t exist, could hopefully be an incentive to get up, moving and out of bed.

Interestingly enough, Apple did not mention “AI” at all during the WWDC. Instead they brought up machine learning, a subset of AI, plenty. But the reality is different: everything from face recognition, to background erase, to neurotechnology has a large machine learning backbone powering its intelligence. This is in classic Apple fashion - disguising the highest of technical excellence as simply Apple Magic.

Bottom Line:

The new AR/VR headset by Apple defined an entirely new product category. Despite the hype (and the hefty $3500 price tag), everyone who tried it on almost unanimously said it was fantastic. The use cases in medicine for really good AR/VR are strong. We’ll keep you updated as we learn more.

What do you think? Will you be buying one…For science?

Other News:

Carbon health announced an AI-powered ambient listening service which they’ve rolled out to over 500 providers, which reduces the time taken to write notes by 60%. We recently dipped our toes into this arena as well, and launched dotphrase.ai, a safe easy and fast dictation and documentation service powered by A.I. (Disclaimer - we built it). You can try it out here.

Google released their model playground for Google Cloud. This is a big win for enterprise players, as it means small teams (or single developers) can play around and fine tune Google models on their own proprietary data that’s stored on Google Cloud.

This was a common complaint amongst hospitals who couldn’t let their IT build on OpenAI - now any GCP clients can go ahead and do so quite freely and (importantly) with small teams.

The playground also offers some handy features like making it easier to choose what format you want the output in, and a place to give the model examples of how you want a conversation to go (this concept is called few shot learning which we will get into next week)

🤿Deep Dive: ARPA-H for Surgeons interested in AI/ML

For this week’s deep dive, we’re going to take a closer look at a new federal research agency devoted to exactly the kind of research we talk about every week in the substack, the Advanced Research Projects Agency for Health. This is a new funding mechanism with at least $1B in appropriations (could be as high as $6.5B, or more). We’ll cover why it exists, what it might do, and how you might want to consider interacting with it.

Backstory: The other ARPAs

ARPA-H is the newer, less self-confident sibling in the ARPA family. The oldest and best known member of the family is the Defense Advanced Research Projects Agency (DARPA). DARPA was founded in 1958 by President Eisenhower with an explicit mission of regaining supremacy in military technology through high-risk, high-reward scientific research. With space research largely carried out by NASA (created the following year) and the respective military branches, DARPA’s initial funding of $520M ($5.46 billion in 2023 dollars) was devoted to transformational activites. DARPA’s work led to the creation of the systems that would eventually become the internet (ARPANET, hypertext), global positioning (NavSat), ballistic missile defense, hypersonic flight, and many other systems. In addition, DARPA programs and contractors have access to a significant fraction of the ~$80B classified “black budget” through BRIDGES and other programs. Honestly, the history of DARPA is fascinating and could occupy many volumes of this newsletter, if its tales could ever be openly discussed.

But DARPA isn’t the only ARPA out there. There is an ARPA-Energy, a Homeland Security ARPA, an Intelligence ARPA, and a number of other federally funded R&D centers that are supported by these and other (perhaps, even more secretive) agencies.

Why ARPA-Health?

Given this plethora of existing agencies that often fund biomedical research, and the existence of explicit science and medical funding agencies such as NSF and NIH (disclosure, I’m funded by NIH NIBIB and planning to apply for funding with ARPA-H), why in the world would we need a new ARPA? And isn’t this just a drop in the $245 billion bucket of US public and private biomedical research?

Unless you’ve been sleeping in a Biohazard Safety Level-4 secured facility for the last three years, you’ve probably noticed that a sweeping global pandemic has catalyzed efforts to create working health care projects from basic research.

This time, we got lucky: moderna’s mRNA vaccine technology needed for COVID-19 vaccines was partially funded by DARPA (P3, ADEPT) in the mid-2010s, waiting to be used.

Most of us who work in the peer-reviewed, grants-funded game know that the system is fundamentally broken. It took me five years to be ready to apply for an NIH K23, three months of almost 100% time to write the application, a further eight months to receive notice of funding, and it’ll take me four years for my project. That’s 10 years. And I got lucky! I was successful on my first NIH application, an extremely rare feat.

In addition, the framework of the NIH avoids proposals that have high probability of fundamental technical challenges, are novel and thus likely to fail. But if they succeed…

What is ARPA-H and how does it differ from the rest of the NIH?

A new agency under the Department of Health and Human Services, requesting $6.5B (~ 14% of the NIH budget), ARPA-H has received $2.5B in appropriations with the first $1B available at the end of FY2024. The first of three ARPA-H offices are located in Washington DC, and by statute, are not affiliated with the NIH campus. The director of ARPA-H reports directly to the secretary of the HHS, not the NIH director. However as a “start-up” agency, ARPA-H will utilize existing NIH infrastructure to some extent.

Unlike the NIH, which typically issues grants, the program managers will likely issue contracts and transactional agreements that have specific milestones and deliverables, with greater accountability and importance of the interaction between the program manager and the contracted research performers. The explicit goal at the end of the program is to graduate contracts to partner companies and organizations to increase scale and implementation.

What will ARPA-H do?

Simply put, ARPA-H occupies the gaps in the existing federally-funded and corporate biomedical research landscape. Ideal ARPA-H projects might be: high risk, eexpensive, require complex coordination among multiple parties, have an applied focus rather than a basic research focus, and have a broad scope that no company can realize full economic benefit. ARPA-H projects are designed to have an explicit exit, “transitioning” these projects into implementations.

The first four focus areas (there might be up to eight, total) are led by autonomous program managers to address challenges that are not easily solvable through existing research and have measurable outcomes:

ARPA-H has adopted a version of the “Heilmeier Catechism,” a set of questions that DARPA uses, to assess program proposals for ARPA-H investment. The questions include … “How is it done today? What are the limitations of present approaches?” and “What is new about your approach? Why do you think you can be successful at this time?” New questions specific to ARPA-H include, “To ensure equitable access for all people, how will cost, accessibility, and user experience be addressed?” and “How might this program be misperceived or misused (and how can we prevent that from happening)?”

The individual program managers and director have tremendous autonomy

There’s certainly a lot of white-space for this team to utilize. As we learn more about who joins the agency, we might get a better sense of how the individual programs will function. The current director, Dr. Renee Wegrzyn, is an applied biologist with extensive experience in both industry and biosecurity at DARPA. Most recently, she was the VP of business development at the controversial synthetic biology firm Gingko Bioworks . The current deputy director, Dr. Susan Monarez, also has extensive experience in national biosecurity initiatives as well as international cooperative agreements. Her prior focus on innovative technologies to improve global health provides an interesting angle. Much of the agency remains to be filled in, but at least five program managers have been named with experiences ranging from the NCI to the US Navy.

“Transitioning” - a key difference between ARPA-H and NIH projects

Unlike the NIH, the end goal of an ARPA-H project isn’t another grant, rather the establishment of a commercially viable and operating implementation. This will require close coordination with FDA, CMS, VA, CDC, and many other alphabet soup agencies that regulate medical innovation. Perhaps, ARPA-H might be able to shelter its innovations from the ‘valley of death’ that bogs down successful projects in regulatory and commercial market challenges preventing their ultimate productization and adoption.

Would you like to know more?

For a complete summary, read this Congressional Research Summary report, upon which this post is largely based. And of course, the website is found at https://www.arpa-h.gov

🪦 Best of Twitter

Any suggestions for what you’d like to see here instead?

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

This week, Time magazine led the doomer charge with its cover (“The end of humAnIty”, but we told you to expect this in our last issue. hit $1Trillion market capitalization. released a new version of photoshop that “magically” fills in or replaces features of images using generative AI. And much more…

This week we will get deep into generative AI, from fake celebrity videos to synthetic training data for health care data. It’s a critical area that is all the rage - and we’ll take it on. We’ll also take a look back at the state of surgical AI in our deep dive.

As always, we are grounded in our clinical-first context, so you can be a discerning consumer and developer. We’ll help you decide when to bring A.I. into the clinic, hospital, or O.R.

Share

Subscribe now

Table of Contents

The News: 📰Another mindboggling week in AI

Deep Dive: 🤿The state of surgical AI research - an update

The News: Another Mindboggling Week in AI, zooming in on Generative AI

Many things happened this week in AI (AI-powered discovery of new antibiotic against acinetobacter might be particularly cool), but I want to start to draw a line through one common thread of progress: generative AI. There are many ($billions of venture funding) reasons to think more about generative AI in medicine, and there are many exciting use cases, but we have to start from the basic definition:

Generative AI is a family of neural network models that generate (synthesize) output based on a given input sequence. Unlike traditional neural networks, which report back a prediction or classification of the input sequence, generative models output a new sequence . Generative models are modality specific (text/image/video/audio), but can have different input and output modalities (translate text to image, etc). Examples of this include GPT’s, Clip, DALLE, stable diffusion, midjourney, and so on.

Here’s why generative AI models are so important, right now, in medicine:

Synthetic Data Is All the Rage in Biomedicine. Machine learning algorithms are data-hungry. Most published models outside of medicine rely on billions or trillions of data points to create their models: GPT-3 was trained on billions of websites, each with hundreds or thousands of individual text tokens. And nobody knows the training dataset for GPT-4, but we can believe it would only be larger. Collecting biomedical data is hard: data holders are reluctant to give up their data but often lack the competency to study it. Few marketplaces for medical data exist - and when they do, they are seldom truly open. But what if we could create more training data for models, without waiting to accrue patients or perform costly diagnostic tests?

Enter the world of Generative AI. You’ve heard about generative AI: fake videos of will smith eating pasta, political leaders saying things they never said, and of course GPT itself. But what if these videos are good enough to train AI models, at least at a first approximation (pre-training), that will get close enough to allow a smaller dataset to do the rest of the work (fine-tuning)?

In a recent paper, Dolezal et al created synthetic pathology slides of tumors using a neural network design called a conditional generative adversarial network (cGAN), a type of image generator that uses the image labels as part of its training process. Their network generated realistic imaging ( below), but most interestingly, that these synthetic images could be used to train pathologists to improve their skills in a 1 hour training scenario (72.9% to 79% classification accuracy on real images). Of course, their skills improved most when looking at real images that were similar to those where the network was “best” at generating. This creates a fascinating educational paradigm where AI can be used to augment education by providing an increased number of relevant examples of rarely seen pathology, helping pathologists “hone in on” the features that the network finds “most relevant” to its image generation. In essence, the AI is “learning” what visual elements best differentiate between tumor types, and then “teaching” that insight back to the pathologists.

And similarly, for EHR data, Li et al did similar work with time-series data. Time series data, comprised of multiple patient measurements across a time interval, is traditionally difficult for GANs. A space very much worth watching

Generative AI can read your mind. We are huge fans of the medarc research organization concept. One of their first papers generates viewed images from brain activity. Yes, that is what it sounds like: you watch an image in the fMRI scanner, the neural network sees your fMRI changes in blood oxygenation level dependent-signal, and it predicts the image you are looking at with > 90% accuracy. Next up (already in the works): scalp electrode recordings.

And if generative AI can see the world as you see it, what use might that be?

Well, a world-beating team led by friend of the substack and Caltech Prof. Anima Anandkumar recently connected an AI model trained to play the videogame Minecraft to GPT-4.Voyager (minecraft). The AI now travels farther, is better at creating objects, and performs significantly better at playing minecraft. It can troubleshoot its own code much better using GPT-4 compared to GPT 3.5. Other models (STEVE-1) can see images from the video game, interpret them, and act on text based instructions (“go chop some seaweed”).

What does this mean for surgeons? Imagine if, instead of creating a minedojo (giant training library of all human knowledge about minecraft), we created a surgical video training ground for AI models. Sure, you’d need an underlying physics engine for surgery itself, which is the obvious missing link here, but with games like “surgeon simulator” and an increasing industry of serious games for healthcare, how much farther away could this be?

And what would happen if we took a real moonshot at creating a $250M “virtual surgery world” that simulated a single speciality like brain surgery or abdominal surgery, similar to the cost of the most expensive video games ever created. There’s every reason to think it might costs much less (especially if the new Unity AI engine is as good as promised). And even if it did cost us $2B to create accurate simulations covering the vast majority of surgery. That’s only 0.05% of the US healthcare spending for one year. There’s never been a reason to do this - until right now. Today, if we had a virtual surgical universe, we could change the face of surgery forever.

Let’s get building.

🤿 Deep Dive: The State of Surgical A.I. Literature

This week, our deep dive looks at the latest issue of Neurosurgical Focus, a monthly publication from the Journal of Neurosurgery with article submissions all focusing on one theme each issue. This month's issue is, unsurprisingly, machine learning.

For our non-neurosurgical audience, fear not, this will not be a deep dive into the world of clinical outcomes research in spine surgery, or different mRNA markers in glioblastoma. But, we will use these articles to highlight where surgical A.I. research is (or was at the time of submission), and we’ll highlight some common threads and insights that might be true in your field as well.

Machine Learning as a Predictive Tool

One common thread is the use of machine learning as a predictive tool. Park et al., explored the use of ML models to predict postoperative improvements in pain in patients with cervical myelopathy. Another article highlights the development of nonlinear k-nearest neighbors classification algorithms to identify patients at high risk of increased length of hospital stay following spine surgery. A team from Jefferson found that in meningioma, radiographic features from MRI could be used preoperatively to predict Ki-67, a postoperative histopathological marker which portends poor prognosis. Knowing a patient’s Ki-67 beforehand could guide patient-specific treatment strategies, surveillance plans or timing of surgery.

Understanding and Interpreting Complex Biological Systems

One of the best use cases for A.I. in my book has to be the consolidation and processing of incredibly large amounts of data. I’m talking human genome level volumes of data and finding meaning out of it in an automated process. One study delves into the metabolomic differentiation of tumor core versus edge in glioma. The implications for taking any N number of metabolic compounds and determining their correlation to a particular functional tumor status are non-intuitive to humans by default. Separately, another article explored how interhemispheric connections maintained language postoperatively- addressing the issue of postoperative aphasia in glioma patients. This is a) an important problem and b) again a perfect example of how problems with lots of complex inputs are well suited for using ML models. The concern we always run into, however is: how do we know we’ve captured all the right inputs.

Bridging the Gap: From Predictive Models to Holistic Clinical Solutions

The potential of machine learning in neurosurgery, as demonstrated in the articles from the JNS Focus, is undeniably powerful. The ability to accurately predict surgical outcomes, interpret complex biological systems, and identify nuanced patterns in patient data is revolutionizing the field. But to borrow a phrase from the world of tech startups, we're still very much in the "proof of concept" phase. We're demonstrating that machine learning can solve isolated problems, but the real magic happens when we integrate these solutions into the larger clinical workflow.

To make machine learning truly transformative, we need to transition from proof of concept to proof of value. This involves not just predicting surgical outcomes, but also demonstrating how these predictions can inform clinical decision-making, tailor patient care, and improve health outcomes. As researchers and clinicians, we should focus on developing comprehensive systems that embed machine learning tools within the clinical environment. These could take the form of decision-support systems that provide real-time insights to clinicians or patient management tools that personalize care plans based on predictive algorithms.

How you ask? I think the answer is simple: clinicians need to become builders. The more we have surgeons and clinicians tinkering with software, taking models and putting them into a user interface they can actually interact with, or just playing with the model code to see how different parameters or datasets influence outcome, the closer we get.

If you don’t have a technical background, that’s okay. A.I. tools like chatGPT, Ghostwriter and Copilot can give you all the help you need to take you from zero to a working proof of concept. Surgeons have a rich history of invention and tinkering. Just because software requires a new set of skills, doesn’t mean that culture of innovation should change.

Rapid Research Dissemination: Embracing the Pace of Innovation

Another takeaway: artificial intelligence is advancing at an unprecedented pace. New algorithms, models, and applications are being developed every day. But while the field is evolving rapidly, the process of disseminating this research is not keeping up. Traditional academic publishing does not keep up.

Before you all start defending journals and their need for peer review, ask yourself: what’s the longest you’ve waited between getting a manuscript accepted and its publication. For me it was about 11 months.

I can not even imagine how the world of A.I. will look 11 months from now. Why would any research I publish be relevant then?

The solution? We need to create new avenues for rapid, yet rigorous, research dissemination. Medrxiv and pre-print services are fine, but oftentimes adversarial reviewers make papers better, and plenty of times things just don’t deserve to be published in the first place. How do we create a system where reviewers are 1) incentivized to review, 2) incentives to review quickly, 3) there is a high bar for entry but 4) there is rapid turnaround with near zero time from acceptance to dissemination.

Is anyone working on this problem? Let us know at ctrlaltoperate@gmail.com

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Subscribe now

Share

Leave a comment

Table of Contents

📰 The News: Neuroscience is cool again!
🤿 Deep Dive: Doomers, Medical Education, and
🪦 Best of Twitter:

Share

Subscribe now

📰The News: Neuroscience Back on the Cool Throne

Congrats to everyone who had in their portfolio this week, shoutout to you- it is clear that despite a hazy macroeconomic picture, the A.I. train and business need for computational power remains undeterred. Also highly recommend this podcast which dives into NVDIA’s CEO, Jensen Huang. My favorite part of the podcast: his parents immigrated to the US and meant to send him to boarding school, but instead sent him to a remedial school where he tutored his ex-juvy roommate in math in exchange for learning how to fix up Harley Davidsons. Awesome stuff.

Onto the news.

The world of AI, Neuroscience and functional neurosurgery went mainstream this week. A team out of Switzerland developed a brain-spine bridge to help a man walk after a motorcycle accident left him paralyzed in 2011. The subtleties of the technology are the interesting parts here, and the full paper is here. Brain-computer-interfaces have exploded in popularity and funding- where electrodes implanted on cortical surfaces can record, transmit and even interpret electrical signals. Simultaneously, in the spinal cord injury world, epidural electrodes that send signals to lower motor neurons to activate muscles (via a backpack-like mechanism), have been able to allow spinal cord injury patients to take steps. However, a machine deciding to make your muscles move was a reasonably unnatural experience.

The Swiss team bridged these two technologies, and decoded BCI-based movement patterns and paired them with LMN electrical impulses to simulate a much more “natural” walking feeling. This is a step (no pun intended) towards definitive care for spinal cord injury patients who are otherwise generally young and healthy.

In more neuroscience news, Elon Musk announced his BCI company Neuralink has received FDA approval to start clinical trials. For what? For whom? With what endpoints? No details disclosed.

For some more research-y news, a team out of Stanfords AI Lab has launched meerkat, which enables foundation models (think chatGPT, large scale vision models, generative image models like DALL-E) to be applied to unstructured data.

Impressively, the team leverages foundation models to reason over images, using text to search for characteristics without these images ever being labeled. Here, they look for pacemakers in CXRs. Conceptually, this makes sense. If a model can reason through what a pacemaker looks like, where it is inserted, etc. it can reasonably find CXRs with an artifact on the L side of the patient (plus any small number of images with pacemaker labels).

This appears to be a powerful tool for data scientists, particularly in medicine, where so much of our data is 1) unstructured and 2) multimodal.

One of the faculty building meerkat, James Zou, was on the Vital Signs podcast, which you can listen to here.

In other news…

The Pew Research Center found that while a majority of Americans have heard of chatGPT, few have tried it themselves. This is a good reminder for those in the A.I. echo chamber, of which we are squarely guilty of being a part of, that this is still brand new technology. Specifically, only 14% of Americans have actually tried ChatGPT.

I guess now is a good time to bring back this classic from 1995:

🤿Deep Dive: Doomers, a pre-mortem for AI in Medicine.

As chatGPT swept the front pages of our popular press, a minority of technologists publicly argued that technological advances expressed in AI systems were extremely dangerous. These AI doomers, issued a public call to pause all AI research which attracted signatories such as tech leaders (Musk, Wozniak), former presidential candidate Andrew Yang, intellectuals (Yehoshua Bengio, Gary Marcus, Eliezer Yudkowsky), and 31000+ other compatriots. Yudkowsky argues that design flaws and evolution over time could lead to unforeseen risks, and even the end of humanity itself. Central to this concern is the idea of "intelligence explosion" or "foom," where recursively self-improving AI systems rapidly transition from subhuman general intelligence to superintelligence, potentially outpacing our ability to control or understand them. A misaligned AI might make a series of internally consistent decisions that have catastrophic outcomes (such as optimizing the number of paper clips in the world by turning all matter into paper clips and fighting humans). While this seems farfetched to us, and most people who have ever attempted to design any computational system, several of the underlying concepts such as alignment, bias, and human-AI teaming effects, are very real concerns for the deployment of AI systems in medicine, and worth diving into at length today. This is our opportunity for a pre-mortem: an exercise to imagine darker alternative futures so that we may avoid them.

First, let’s talk about alignment. All actors, whether AI or human, may have different preferences. When actors attempt to cooperate, one of the trickiest parts of the interaction is successfully managing these differing preferences (in economics, this has been studied for decades as the Principal-Agent problem). For example, if I hire a nurse to perform home blood pressure monitoring, I should design the rewards and engagement structures (salary, others) to make it more likely that they will do the work I intended. Similarly, if I ask an AI to write code, I should ensure that it is only “rewarded” (programmatically, with its favorite 10010101’s) when the work is successfully completed. Otherwise, it might output the score from the Dodgers game and call it a day. The doomers worry that solving Principal-Agent problems with superintelligent and highly connected AI may be impossible (since it’s smarter than us), and the pace of AI superintelligence may accelerate (called foom) faster than we can manage. Before we can pull the plug on our AI, we will all be turned into paper clips, or so the theory goes.

What does alignment mean for medical AI systems? Frankly, I’m less afraid of superintelligent foom and doom healthcare AI’s, but far more afraid of misaligned AI. First, defining human preferences is devilishly tricky. Patient satisfaction measures are notoriously fraught: repeated studies show that the most satisfied patients are more likely to die and less likely to receive care in some settings.

Secondly, even if we can define valid preferences for patients, the patients are not going to be creating the AI models that are deployed clinically. Both capital and regulatory structures dictate where these models will be created: large insurers, hospital systems, or even governments. It would be illogical to believe that this historically misaligned sector will do better with AI than it has with communication, financing, or care provision. How do we weigh the desires of an insurer (reduce costs), patients (live longer, usually), hospitals (run a business, serve a community), and governments, in deciding what recommendations we should make? Even “simple” diagnostic algorithms have an alignment challenge: whom should the AI algorithm designers ask about the relative risk of overdiagnosing cancer versus missing a lifesaving opportunity? Or should we aim for the most “accurate” diagnostic, pretending that these scenarios are all equally valuable? Unless we undertake a massive, burdensome and unfinanced effort to tackle our incentives problem, AI can only replicate our existing challenges.

Bias is similarly important, but it may not mean what we are used to thinking about in the DEI-sense of the word. Addressing algorithmic performance across definable categories of persons is difficult, but possible. We should know to look for bias in our data and our policies (although we must be better at this). But there’s no reason to believe that the multidimensional interactions within an AI model will neatly map onto our existing protected classes of persons in a human way. Bias might occur due to characteristics of hospitals, physicians, procedures, subtle geographies, or any number of a myriad of interactions between underexplored, machine-generated, and potentially uninterpretable features (variables). And what do we do with uninterpretable and biased systems? It’s not as simple as some sort of restorative justice effort where we “add-in” a factor to “correct” for race or orientation. Algorithmic bias could be worsened by efforts to “suppress” it using human-interpretable categories.

Bias and alignment also have important interaction terms. When we can’t articulate our preferences, can’t measure our biases, and consistently build systems that have unclear behavioral properties, the final result may be worse than the sum of these challenges. On a more superficial level, we can instruct large language models to be “better” than the observed behavior of physicians: “be empathetic”, “explain at the level of a 7th grader”, “don’t be racist”, etc. But who decides, how those decisions are made, how are decisions represented to stakeholders, and how are they validated? These are difficult questions that we have never shown the ability to grapple with as a society and sector.

For example: An AI chatbot is deployed to interact with patients by an insurer. Maybe it starts as a GoodBot that is found to actually drive up costs (accidentally, since the behaviors of systems are difficult to predict), but then is modified over time to meet the incentives of its creator (minimize costs). You could similarly imagine an EvilCo creating an EvilBot that was instructed to always provide highly satisfying, kind and compassionate results that also reduced the likelihood of patients’ requesting highly expensive tests. Given the opacity of AI systems, and the underlying brokenness of health care, it would be hard to imagine any other path forward.

Where does Medical Education fit into this puzzle? Historically, MedEd is an (almost) unfunded mandate within medicine. Both students and trainee physicians are valued far more for their cheap sources of labor, and educational endeavors are loss-leaders for most MedEd institutions. As we integrate AI systems into medicine, who is going to pay attention to the way that these systems interact with the cognitive process of human learning? Entire realms of competence may be lost if medical professionals opt out of those vital, messy, mistake-prone, and difficult experiences that form our identities.

Is medical AI more like a “spellchecker” or an “autopilot”? How do learners learn “with” AI? When these costly decisions, in the most likely scenario, are made in an opaque and abstracted design phase (by industry) and purchased (by payers and healthcare systems), who is watching the store? How do we center the human-AI interaction on improving the human side of the system, while simultaneously improving the quality of the dyad, as best we can measure?

The learning behaviors of humans in human-AI dyads are under-examined and unstudied. How we meet this challenge will be a critical determinant of the success of the medical enterprise overall and of our adoption of human-AI systems.

In technical and procedural disciplines such as our beloved neurosurgery, we are working to build a short-term future where AI can help teach and explain surgery in a way that improves human health worldwide.

But what about our more cognitively inclined brethren and sister-en? If we can learn to think “better” through interactions with AI, we may dramatically improve human health worldwide. However, if we learn to think “less”, we may never recognize the incredible costs of lost human performance, creativity, and future innovation.

Unlike the doomers and their foom, we can’t pause these developments, nor should we. Our patients demand better - it would be unethical NOT to improve medicine with AI.

These care improvements require clinicians, particularly educators, to get in the driver’s seat.

Who’s coming with us?

Best of Twitter - RIP Elno.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Subscribe now

Share

Leave a comment

Sam Altman testified on The Hill, Google announced a new medical LLM, and some big bets are being made that A.I. will revolutionize medicine. We’ll unpack all of that in The News this week.

In our Deep Dive, we break down the difference between model performance and team performance, and why only one matters.

Table of Contents

📰 The News: Where There’s Smoke, There’s Fire… Right? Big Healthcare Bets
🤿 Deep Dive: We Need Team, Not Model, Benchmarks
🪦 Best of Twitter: :(

Share

Subscribe now

The News: Big Healthcare Bets with AI

Sam Altman, CEO of OpenAI, testified before Congress this week. Hearing our representatives discuss technology feels increasingly more painful the more I hear it (here’s your reminder that the Tik Tok CEO testified earlier this year). A few important kernels here, including confirmation that Altman actually has zero equity in OpenAI, that he encourages regulation of the space, and importantly:

“My worst fear is we cause significant harm to the world"

Yikes. Small stakes. The regulatory space is good to keep an eye out for us clinicians. We say this because it’s becoming increasingly clear healthcare is becoming a big bet for A.I. moguls…(keep reading)

Google announced Med-PaLM 2, a large language model fine tuned on medical data which scored an impressive 87% on the MedQA dataset. Physicians also rated MedPaLM 2 answers as more high quality than their own colleague answers to the same questions!

An important note though, as these models continue to improve and improve, from Peter Lee (research exec at ) tweeted:

Getting the “right answer” is only one part of the clinical problem. And considering how many problems in medicine have multiple answers or no clear answers, getting the right answer may not even be the most important part!

In another big bet, Silicon Valley VCs invested $50M in a seed round to hippocraticai, a company focused on large language models for healthcare. They’ve built a state-of-the-art model which claims to outperform GPT-4 on a wide variety of medical examinations, and is pre-trained on evidence-based content. A 50M seed round is a large bet in one company, and without public availability of the models or any real use cases published yet, the jury is still very much out.

In other news, Apple banned chatGPT at work, citing proprietary data claim information concerns. And a startup out of Japan launched a robotic arm attachment which, quite honestly, reminds us all of the Spiderman villain, Dr. Octopus. But, the abilities for these types of cyborgs to have actual implications in the operating room can’t be understated. Even something as simple as holding a tool nearby would be an immediate win.

🤿Deep Dive: We Need Team, Not Model, Benchmarks

The A.I. hype train is in full throttle mode, and clinical practice is not immune to potential disruption. Clinicians are a funny bunch- we lament at the lack of change or progress in healthcare, yet rarely work with any new technology enough to allow it to grow into its role.

This becomes important with AI models. With Med PaLM2 achieving spectacular scores on a standardized question bank, and hippocraticai.com raising 50M in seed funding, it appears the models are getting all the attention. This is reasonable, given the rise and fame of chatGPT and GPT-4. However, let’s not lose sight of the bigger picture: the point is to have real, clinical significance. This involves a clinic/hospital, nurse, physician, EMR, patient portal … the list goes on.

We must focus on the ability for A.I. to improve the delivery of care, not on its ability to outperform itself on a test.

This introduces the concept of human-A.I. teams. This is of particular importance in healthcare, where even human-human teams have a complex hierarchy, liability structure, and ultimately responsibility. Adding a non-human, A.I. component to the fold is a very hard problem.

Take the following paper which showed employee performance improved more with A.I. than with human feedback, but fell once it was disclosed as coming from A.I. In fact, the best performance delta was when AI feedback was said to be coming from human managers. (h/t Ethan Mollick)

Clearly, there is a complex interaction between what is said, who says it, and how it is received by the end-user. And others have echoed this claim - even as far back as 2019, DARPA had an active call for projects looking for human-AI teaming efforts.

Just because a model gets a series of questions right, does not mean it deserves to be trialed in the real world. We must spend more time adjudicating how it fits with the environment, how the users of the model interact with it, and how it influences their decision making.

This was a fantastic slide that made the rounds following RSNA this year - very experienced, moderately experienced, and inexperienced radiologists were all influenced by A.I. decision making - even when the AI system was incorrect (part of the study design)

As we have postulated here before, inexperienced users were more likely to follow incorrect A.I. guidance than experienced ones. Hospitals are struggling with this too

Yes! The A.I. is one piece of the clinical puzzle, and we as a clinical society must decide how it fits in, and in what scenarios. If you follow this newsletter, odds are you’re bullish that there are many scenarios where it fits in - so let’s find them.

So, this begs the question - how do we test these teams? Of course, the randomized controlled trial cohort of academicians will scream for RCT’s for every AI tool. But an autonomous A.I. chatbot that triages patient acuity (high risk) versus an A.I. chatbot which just simplifies medical jargon, for example, should likely have different thresholds to meet.

Here’s a three part framework which we might want to use as we evaluate these tools moving forward. The mechanisms by which these should be dissected (randomized trials, pilot studies, etc.) are likely use case-specific. But this framework should carry us through most of their adjudications.

1. Define Target Point of Interaction

Who are the people that are going to be interacting with the A.I. model, where are they interacting with them, and in what format? Is it in the EMR, on a screen during a surgical case? Requiring a surgeon to scrub out to rotate a 3 dimensional image may not be the best implementation practice, for instance.

This is where even in the design phases of products, clinical input is necessary. The end users are ultimately the ones delivering care to patients. These clinicians are crucial to the delivery of the right data to the model and the delivery of timely insights to the team.

2. Define Behavioral Change A-priori

This, I think, is an under-described aspect of human-A.I. teams. A.I. is meant to modify human behavior - either by changing decisions, or removing humans from tasks entirely.

Predefining the behavioral change we expect to see is crucial to the trust and safety of these models. Take our radiology example, for instance. If the goal of the model was to increase the efficiency of radiologists, and that the radiologist would only use the model for triaging, then this appears wonderful, as all levels of radiologists benefited from the model.

But, if a change in the read was the behavioral adjustment, then it’s clear that there’s a gradation in effect between experience levels. This is not inherently good or bad - simply another factor to consider.

3. Create Clinical Endpoints

At this point, it's more than reasonable to discuss clinical endpoints. This gets back to medical school and our Phase 3 clinical trial paradigm, where effects on provider efficiency, patient outcomes, clinical efficiency, provider burnout, etc. should all be quantified prior to the implementation of an A.I. system, and following. We have the tools to accomplish this.

Whenever people bring up the “black box” nature of A.I., I like to remind them we still do not know exactly how Tylenol works. But, we have so much data on its efficacy, it doesn’t even matter.

That’s where we need to be with A.I.

🪦Best of Twitter: Hiatus

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Subscribe now

Share

Leave a comment

headsets are coming (to the OR?). ‘s I/O day (mother-of-all-press-releases) was headlined by a new LLM reveal, PaLM 2. Scale.ai announced enterprise-grade models with DoD and deployed them with the XVIII Airborne Corps: is medicine next? Hollywood writers are on strike to prevent AI from taking their jobs (striking to avoid replacement is not a smart long term strategy), a canary-in-the-coal-mine moment. In medicine, when AI writes our textbooks, will we be overjoyed or horrified? Education will be similarly disrupted, and we’ll talk about what that means for surgeons and patients. Our view is that physicians and technologists should partner to lead the coming changes in AI, avoid AI-is-evil alignment problems and job-loss disruption, so we will keep you focused on leading from the front edge of AI and ML in medicine.

In our deep dive, we’ll talk about chatGPT’s new rival from Google, Bard. It came with a lot of fanfare, but is it really a contender?

Share

Subscribe now

📰The News: From headsets to GPT for your cells

I reviewed the first surgical video narrated by an AI this week, and you know what? It wasn’t half bad. These Consumer grade products could help surgeons communicate naturally, worldwide, accelerating innovation in science. Douglas Adams would be proud.

is edging closer to a probable XR headset release this July, which seems half-baked after >7 years of development, ‘s Hololens failure and other lost causes. The struggles with XR headsets in consumer applications primarily stem from human factors limitations: people don’t like wearing bulky headsets. However, surgery is not a consumer application. There is a real opportunity to replace our existing vision enhancers (loupes, headlights) in the operating room, if only someone would seize it. There are only ~ 100,000 surgeons in the US but we might pay $10K each, which gets us to the mythical $1B TAM (total addressible market) everyone craves. All it would take is an app store integration :)

Education is over, your kids will be taught by AI, and they’ll be smarter for it. The role of the teacher in the classroom of 2025 will be more like conductor than violinist. Every child will have a custom AI tutor. Existing first-gen AI’s look like Khanmigo’s chatbots, and are already replacing far-worse alternatives that propagated in the murky depths of the cheating ecosystem (overseas paper writers, cheating forums). Here are some screenshots from learn.xyz ‘s platform for a custom lesson about brain development aimed at young kids. It reminded me to start at the basics (What is the Brain?), includes some mildly terrifying AI-art, and adds practice questions at the end to improve retention.

Doctor-patient education is getting better too: (paper link) One of the core use cases for GPT’s is translation, and friends-of-the-sub Rohaid Ali, Ben Johnston et al. validate the use of GPT4 to change literacy levels of consent forms. This is a great first step to escape the hellscape that is patient-doctor communication. A second step would be for a medical translation company like Martti (bought by , although upHealth seems quite troubled). What we really need is a scalable infrastructure for validating machine translations in medicine . Any takers?

Lastly, lets nerd out on GPT for your cells. (link to paper) If cells communicate, and GPT’s are fundamentally competent at predicting the next token in a language communication, could we use GPT’s to advance cell biology in the way that other GPT-based models accelerate advances in protein folding? In a nice result, they show that a GPT-based approach beat other benchmark methods for cell classification. GPT’s enable computational experiments at scale across many domains, so stay tuned for more from this domain.

Quick Hits From the World of AI:

HF released a transformers-as-agents API, where text input is evaluated and routed to the appropriate downstream tool (Colab demo here). We’ve covered this previously on the transformers as controllers / Toolformers side of things. But an API sure is dandy!

AI startups are so hot, venture capitalists are coming to them! Maybe we should get in on the action ;)

Meta released a new multimodal model, ImageBind. This is a huge deal but we will get to this shortly…

AI in medical ethics: better than humans?

Palantir AIP: AI as a controller … for weapons systems?

What does it mean for the future of human knowledge when LLMs cannibalize all value from the organizations that maintain human knowledge repositories? Yes, stack overflow has a terrible UX, but it figured out how to get people on platform sharing knowledge. New structures are needed - and medicine had better be ready for this revolution.

🤿Deep Dive: We Examine Bard, Google’s chatGPT competitor

Google’s I/O event was met with huge fanfare (and subsequent stock jump by 11%). Part of what they launched was Bard, their chatGPT competitor, but with Google-app integration and internet connectivity built in, for everyone. This contrasts chatGPT’s slow, premium-user-only rollout of these much requested features.

If you use chatGPT, you might be wondering whether Bard is better for you at this point. I did too. So, naturally, I put it up to the test so you don’t have to.

tl;dr: Bard is a step-up from its old version, but poor performance with the Google brand associated with it leaves a sour taste.

I started off with something easy… even Bing could do a quick search and find out who I was - I have a LinkedIn after all. Bard struggled.

“Maybe once I’m an attending, then large language models will know who I am”, I thought foolishly, as I searched for Dan.

Okay, I’ll give it a toss-up, I thought- and I tried Harvey Cushing.

For the non-physicians, Cushing was the founder of modern neurosurgery, and one of the pioneering physicians of the modern era. Great, looks like that’s the bar for Bard to know about me.

But, Google’s whole aim was that Bard was connected to the internet. I wanted to test these capabilities. I started with general tasks, then moved to medicine specific ones. I started with asking for financial data on , which it did quite well.

Okay, we’re getting somewhere! This is promising. Let’s try something medical. I asked if it could look something up in the Greenberg Handbook of Neurosurgery, which it claimed it could.

Okay, now I’m excited. As a soon-to-be on-call resident, this would be incredible. So I asked it to look up the Hunter-Hess scoring system for subarachnoid hemorrhage. Layup, I thought.

Wrong.

I imagine this is how those “too-good-to-be-true” deals feel when you discover it really was, too good to be true. This was disappointing to say the least. Because the Google brand is associated with it, it does make the disappointments feel even more disheartening.

But, not to be deterred, I wanted to see how well Bard could code. The chatGPT code-interpreter plugin is a game changer for data visualization. If Bard can access the internet, then it should be able to perform pretty complex tasks. I asked it a relatively simple one- show me a graph with the number of physicians in the united states over time

I expected a Python script, and some data source which may or nay not be correct. But what I got was simply a description of the number of physicians. Again, Bard falls flat.

I then moved on to information retrieval. I wanted to see if Bard would pull up relevant literature when prompted. I asked it to find mainstay trials for intra-arterial treatment of ischemic stroke.

On the surface, this looks fantastic. And if you only read the first output, it’s great! These are all real trials!

But sadly, the citations for 3/4 trials listed were incorrect, and the summarizations for 2 trials were incorrect. For example, the DAWN trial was actually published in 2018 (not 2015), and only had 200+ patients, not 2000.

This phenomena, known as hallucination, continues to be a problem. When you Google a question and the answer appears in the OneBox (see below) - you assume it is correct. This is the same standard I felt myself applying with Bard, but clearly I should not be.

Let’s try something ChatGPT got wrong a lot- billing. I found an operative note online for an C3-5 ACDF (for non-physicians, that’s a very common surgery of the cervical spine in your neck).

I asked Bard to provide the CPT codes for it. I even gave it some help, and provided Nuvasive’s billing guidelines to use as a guide. For the non-physicians, CPT codes classify procedures and their components, ultimately to be used for billing and hospital/physician reimbursement.

I am impressed. I’m not even here to debate how accurate this is, as there are a few inconsistencies. But, these are real codes, pulled from a source URL, and all quite relevant. Any small changes like: how to bill for multiple levels, instrumentation, etc., can be figured out quite easily with some prompting and few-shot learning. Good job, Bard!

Finally, I asked it to perform a task I had all the confidence it could pull off - write a prior authorization. To my surprise, Bard filled in details I did not provide, like the patient’s symptoms, the sidedness, etc. In many ways it needed to do this, otherwise it couldn't perform the task successfully. But, simply knowing it’s making things up is a little unsettling.

All in all, Bard is powerful. It can absolutely do some things chatGPT cannot do, and importantly, it's plugged in with the Google Ecosystem for easy exporting and downstream tasks. But, there are fundamental tasks (code interpretation, citation of sources) which need to be met in order for me to make the switch, personally.

That being said, maybe this is good enough for people? If you never used chatGPT, this is incredibly accessible, and by a brand you trust. What do you think? Will you be a Bard user?

🪦 Best of Twitter (On Hold)

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

This week (seemingly like all weeks) was filled with news, but in a much more political, he-said-she-said, kind of way. We get into the news, and how that even remotely brings us back to the clinic. Oh, and we’ll bring you the newest Glaucomflecken video which takes on AI!

Remember when we talked about how chatGPT could be your data intern? That time has come. Dan got access to chatGPT’s new code interpreter - this is an incredibly powerful tool that abstracts out exploratory data workflows, especially quantitative analysis. He’ll go through a first-person POV using code interpreter as his data intern in our deep dive.

Table of Contents

📰The News: A.I. Gossip, Big Bets, and Some Science

🤿Deep Dive: Putting our new data intern from chatGPT, Code Interpreter, to the test!

Share

Subscribe now

The News: A.I. Gossip, Big Bets, and Some Science

First, some humor. AI has gone mainstream and social-MD guru Dr. Glaucomflecken shares his take on Ortho Bros powered by AI. Enjoy:

If you haven't watched the latest episode of The Tech Bros of Silicon Valley, don’t worry, we’re loyal fans so can fill you in on all the billion dollar corporate gossip. Now, admittedly, some of this may not be directly applicable to clinical or surgical life. But, they involve the most powerful companies on the planet who dictate our A.I. futures, so we think it’s worth knowing.

A leaked memo from a Google researcher has been making the rounds, with the bold statement:

The concept of a moat comes from Warren Buffett, who described it as a company’s durable competitive advantage, protecting it from competitors as its name implies. Their moat in the AI world has thus far been that Google and OpenAI’s models have just been, flat out, the best. The open-source world is closing that gap, and fast.

The CEOs of OpenAI, Microsoft Google and more met at the met at the White House to discuss A.I. policy (including $140 Million in National Science Foundation funding!). Two key missing persons: Elon Musk and Mark Zuckerberg. While chatGPT and language models are all the rage, Tesla and Meta (formerly Facebook) have built a legacy of computer vision advancement (see prior issue of Ctrl-Alt-Operate).

Finally, the “Godfather of AI” Geoffrey Hinton, senior author of one of the seminal computer vision papers, has left Google so he can speak freely on the dangers of A.I. This tracks for Hinton: he was a lifelong academician who even left Carnegie Mellon because he was uncomfortable with taking Pentagon dollars. He then had a decade long stint at Google, which he was only in because he also left tens of millions of dollars on the table so he could work for Google instead of the Chinese tech-giant Baidu (this is an insane story). He is worried that because of the fierce competition and pressure to innovate, we could create dangerous A.I. systems at a rapidly alarming pace and only realize it too late.

In totally unrelated news, Palantir announced their A.I. for Defense platform, where troop movements and analysis can be performed using natural language. PwC also announced a billion dollar investment in the next three years in AI, where it sees agents writing notes, performing billing duties, and more (sound familiar?)

Now to return to some more true A.I. news, OpenAI continues to impress and dropped a “Text to 3-D Shape” paper. Input words, output a 3D rendering (rotatable) shape. My favorite one is “Chair that looks like an avocado” that also looks like a chair in every hotel lobby.

I think this allows us to double down on our thought process that multi-modal models, that is- models that incorporate text, speech, image analysis, image creation, etc. will represent the next “wave” of A.I. technology. From a surgical standpoint, as we move towards more hyper-personalized implant technology, this could be a fantastic way to correlate plans with real prints- guided by natural language.

Finally, ElevenLabs released a voice-cloning model which can change your voice into a number of different languages. Their demo is scary good - take a native Spanish speaker, and watch their own voice start speaking English, with the appropriate accent and everything. What if we had these buddies (sync’d with Google Translate) in the hospital? Where our voice (and the concern, empathy, lightheartedness…) could just be said in the language our patients most comfortably speak?

🤿Deep Dive: Code Interpreter, your new data intern from chatGPT.

Although most MD’s don’t write code, anyone who publishes a paper has to analyze data using computational methods (raise your hand if you’re still using a slide rule and transparencies - nope, didn’t think so).

This section is for you - whether you’re a student or seasoned prof.

Let’s take the latest secret tool from OpenAI, Code Interpreter, for a quick spin and see where things stand in May 2023.

First, some background. Since its earliest inception, chatGPT and similar tools from major IDEs (Copilot, replit) have accelerated code writing. These tools started as simple auto-completes, but are now able to write functions and even entire programs, moving ideas into production-ready webapps in hours instead of days.

Here’s where I think Code interpreter stacks up today.

1. Data Visualization: Can I use CodeInterpreter to explore a dataset faster and more easily than writing r or python scripts? (My Grade: B, limited but does well within its limitations)

2. Data Cleaning: How well does CodeInterpreter handle messy datasets? (My Grade: B+, needs supervision but can do simple jobs)

3. Data Analysis: Can we build toy models, and understand them? (My Grade: F, the interface is too problematic to use effectively).

What is Code Interpreter: Code interpreter is a large language model (GPT) interface to a jupyter notebook that runs python code in a sandbox environment to which you can upload files and data. You can find the full list of libraries here.

It allows you to create and run python code using natural language.

Let’s see how this does on real data. We’re going to use our public dataset of surgical performance, called SOCAL, which you can download here. SOCAL was created from a nationwide training exercise where surgeons attempted to control bleeding from an extremely lifelike perfused cadaveric simulator, and supported a series of publications from 2017-2022 where we described our earliest steps into automating surgeon performance analysis.

First, we will upload two csv’s that contain the data about our surgeons and how they performed. Since codeinterpreter uses standard python packages (pandas, numpy, scikit-learn, matplotlib, seaborn) and jupyter notebooks, it will default to those methods.

Our data files are formatted strangely, since one file contains all of the data about the participants, including their trial #1 and trial #2 performance, but the other contains a list of trials and their outcomes. Let’s see if we can guide code interpreter through this process.

Looks good. Now let’s make some nice figures using seaborn. Not too bad.

Now lets give it a real task.

Now here’s the neat part, it went through a series of errors (just as I would have), troubleshot its own mistakes through several iterations, came up with a solution, failed, tried again, failed, and voila:

Not the most visually appealing, lets see what happens if we ask it to do better:

Whoops! Like any intern, it misinterpreted what I asked, and gave me a bunch of bar plots instead of nice looking tables, so we’ll try to get it to try again. Unfortunately the current build of code interpeter doesn’t have the right python package or permissions to install it, so this is the best we will get for now :-/ But it will write the correct code so that I could execute it in my python environment with a copy-paste.

See what happened? It picked the wrong variable to use in the logistic regression. We have to be careful about supervising our intern otherwise the analysis might not make any sense! After a few iterations of this, it generates a model that allegedly has some pretty good performance characteristics (but on closer inspection … needs some help)

What about some other fun exercises? Well, it turns out that we monitored the heart rates of many of our surgeons. Let’s see if we can generate a model that predicts their baseline heartrate. Were more experienced surgeons more likely to have higher or lower heartrates? What about surgeons who had more or less aerobic training?

On our first attempt, the intern does something cheeky: it builds a model using all of the other heart rate data to predict the baseline data. Clearly good thinking, but not useful for us. Once we tell it not to do that, it isn’t nearly as successful.

Oh, and here’s how it handles errors:

In summary: Code Interpreter is a limited playground tool that can automate creation of python scripts and run them against data in a sandbox. It has limitations based on the restrictions placed on its permissions as well as its computational environment. The less specification you give it, the more you need to police its work. It can troubleshoot some simple errors.

Overall, it didn’t speed up my work and there are some areas of its output that were problematic. Personally, I would prefer working in a more open, pair-programmer interface like copilot or even using copy-paste chatGPT to provide a better experience than StackOverflow, and I didn’t like how obfuscated some of the design decisions and code samples were.

I know that there is more to come on this front, but for now, I don’t think Code Interpreter will replace me or my (imaginary, human) intern. With a few tweaks to the front end, and a better ability to interface with existing IDE’s, the underlying concept will be baked into everything we use next year. So take it for a spin when you get access :)

🪦 Best of Twitter (On Hold)

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

This week’s newsletter is brought to you by AI Radio (language warning, most songs are pop-hip hop).

For us non hip-hop experts, these might sound eerily real, but they are all made without a real artist singing. We’ll guide you through these and other controversies below.

This week, as always, we bring you all that’s happened in A.I. this week and a deep dive into one topic. If you’ve been with us before, send this to one clinician who considers themselves “techie” - those are our people 😁

Share

Subscribe now

We launched a few months before our faithful friend chatGPT took the world by storm. Since then, we’ve been flooded with friends, colleagues, and new friends from all over the world asking how they can get involved with A.I. in surgery or more broadly in medicine. Today, we try to answer some of these questions in our deep dive. Let’s get started.

Table of Contents

📰 News of the Week:
🤿 Deep Dive: ChatGPT, MD?
🪦 Tweets of the Week: On Hiatus

News of the Week: Non-Medical Grab Bag

A Pew research poll found that most americans beleive that AI with affect jobholders, and only 7% favored using AI in making final hiring decisisons. What about in screening applications - like residency applicants? (many job applicants are using GPT to write their personal statements) Reading many hundreds of applications is extremely time consuming and typically done by clinicians (read: Dan) with numerous competing demands on their time. We can use GPT to read those voluminous ERAS packets and letters of recommendation, and pull out the gems that are most likely to sway our decisionmaking. By 2026, maybe our interviews will be between an applicant’s GPT agent and the program’s GPT. Regulators are getting hip to this, so stay tuned. Yes - we all worry that GPT’s will make horrible errors, but having been on the other side, I think we need to understand the human baseline rates of these problems as well.

Harvey, the GPT for JD’s startup, recently raised from Sequoia and is becoming increasingly integrated into BigLaw.

The freelance content writing market is thoroughly disrupted, with everyone* using GPT, the emergence of seccondary freelancers who clean up other freelancers’ GPT outputs, and some poorly thought out GPT bans. These debates are important to follow, serving as bellweathers for larger markets including medicine.

The AI from the movie “Her” (ScarJo/ Joaquin Pheonix / Spike Jonze) is now available as an app on your iPhone (ScarJo voice coming soon). I’m not a big fan of the impact of these apps on humanity writ large. But, no one is listening to me.

LLMs, including Khan Academy’s KhanMigo, are augmenting classroom performance. This private tutor-model could dramatically improve educational achievement. Will we see this in medicine, this year, as part of Amboss’s pivot from a flashcard company into a $1B unicorn?

A recent Semafor article highlighted tech folks who have transitioned from CS to healthtech/biotech, including friends and college classmates of ours. It’s a good time to be recruiting!

released Rapsai, a no-code ML tool that allows the generation of custom ML pipelines in 15 minutes. It’s like figma and colab had a baby. These tools are a big part of the commodification of ML implementation.

Meanwhile, in Medicine …

health is expected to release a machine learning-enabled health coach in September that uses your data to provide customized insights to improve your health. You can get a sneak preview of what this might look like at the HealthGPT Github repo, which provides chat functionality against your own personal iOS data.

ChatGPT got access to the internet (covered in much greater detail in the deep dive) , the front page of the WSJ, and has continued to captivate public imagination. Admittedly, the performance in today’s build leaves much to be desired. But with these technologies and chain-of-thought prompting reaching consumers, be prepared for another explosion of “advances” in GPT performance this summer.

Structured light navigation is coming to an operating theater near you. Proprio Vision, led by friend-of-the-substack entrepreneurial maven Dr. Sam Browd, recently announced it received FDA 510K clearance for a structured light-based computer vision-powered surgical navigation system. These camera based systems help surgeons “see through” the body to accurately place implants and identify critical structures. Navigation techniques register a three dimensional map of the surgical field to preoperative imaging, and there is a whole host of structured-light based technologies that are currently available (7D Surgical) and in development (Advanced Scanners, among others).

Urology leads the way. The annual AUA urology conference is ongoing this weekend, heavily featuring AI topics.

https://twitter.com/MSLee_MD/status/1652300764637790209?s=20

Remember More? Transformer-based neural networks might be getting longer memories. I won’t get too far into the nerdery of neural network architectures but it is important to understand that memory is one of the most important limitations on the current transformer based architectures that underly GPT, many of the computer vision models we use, and much else (see our prior newsletters). The problem is that for every increase in context of n tokens into your vanilla self-attention network, your compute requirement increases by n^2. You should care about memory and context length because most of the really “cool” things happen over long contexts - models get much “smarter”, can “remember” longer conversations, etc.

🤿 Deep Dive: Can ChatGPT answers patient questions better than doctors?

Run a little thought experiment with me:

Premise 1: Your neighbor has a health concern, but they don’t know you well enough to ask you.

Premise 2: Your neighbor will post on the internet about their problem.

Question: Would you prefer your neighbor receive advice from a random internet doctor, or from chatGPT?

Loading...

This week’s deep dive is on an article published in JAMA Internal Medicine that was released on Friday (hot off the presses here at Ctrl-Alt-Operate), and has gone pretty viral on Twitter. The is Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum by Ayers et al., led out of UCSD.

a.k.a. “chatGPT and MDs answer questions from patients - who does better?” The results suggested chatGPT robustly outperformed clinicians, which even got a shoutout from Greg Brockman (OpenAI President):

So… did chatGPT beat out MDs? 🤔

Sort of, but not in the way you’re thinking. The study took 200 questions from a Reddit’s r/AskDocs - here, users can submit anonymous healthcare questions, and verified clinicians can answer them. Users are verified through an internal mechanism, display their qualifications (Doctor, EMT, Nurse, etc) in their bios, and likely represents an anonymous grab bag of physicians from around the world.

The study compared 200 questions answered by Reddit doctors to chatGPT, in answering questions like “What do I do if I swallowed a toothpick” and “I have a lump on my penis”

Sounds like a dinner party where all of a sudden someone shows you their new rash…

The group then examined chatGPT’s answers to the answers delivered on r/AskDocs by verified physicians, and asked a group of nurses and doctors to evaluate which responses were better, and score them in terms of Quality and Empathy.

ChatGPT was the preferred output in 79% of evaluations. ChatGPT outperformed doctors on Quality of response (3.6x higher prevalence of good or very good responses) and on Empathy (9.8x higher prevalence of empathetic or very empathetic responses).

The results from the abstract are pasted below

Now, as all things in life, the internet took this article by storm as a chance to 1) disparage physicians, 2) AI-doomsday + fear-monger, and 3) talk about how A.I. will replace every job.

First off, kudos to the team - getting discussions on A.I. x medicine out into the open is a commendable accomplishment, and one our profession and society will deal with in the coming years.

The study design was obviously limited- they used internet doctors verified via an internal Reddit verification system answering questions submitted in an anonymous forum to simulate a clinician answering an in-basket message. This is of course not a real clinical scenario, and thus cannot be generalized as such. The authors clearly state that in their discussion (but who reads the paper, anyway).

But their study design was smart. It allowed a pseudo-control group (internet doctors) without actually putting patients in the loop.

Why is this key? As we keep building these systems faster than we can write IRBs, we need to find a way to test A.I. systems without necessarily deploying them in real clinical scenarios.

Some questions to think about:

• Given these results, would you be comfortable with your institution running a clinical trial evaluating the outputs of MD-inbasket messages vs chatGPT?

• What if another member of your team double checked its work?

• What if I told you your team answered questions incorrectly 8% (made up number) of the time? How might that change your perspective?

• What if only as a triaging tool?

Some more takeaways:

It’s clear people are turning to the internet for healthcare advice. How do make sure that advice is good? The solution cannot be “go see a doctor” when patients are increasingly dissatisfied with their care and the cost.

People continue to discuss chatGPT’s tendency to hallucinate information (i.e. “sometimes wrong, never in doubt”) - this is true and guardrails must be placed. However, I’ve yet to see an analysis of correct vs incorrect information delivered by real care teams, versus by chatGPT (either in a clinical scenario or a simulation). This is necessary.

If anything this study forces us to ask: how much of our advice is ignored by patients because responses lack empathy? Do your in-basket responses look more like chatGPT’s or the physician responding? How much better could we be if our responses were augmented by these capabilities?

If anyone is conducting research in this area or running a pseudo-trial, please let us know.

Best of Twitter (paused until mom and dad stop fighting)

We’re torn because we love both of our social media parents equally, but the formatting and maneuvering has sucked the joy out of this section. So I’m leaving it on perma-pause.

https://twitter.com/eli_cole_/status/1652700392583729152?s=20

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Subscribe now

Share

Leave a comment

This week, as always, we bring you all that’s happened in A.I. this week and a deep dive into one topic. If you’ve been with us before, send this to one clinician who considers themselves “techie” - those are our people 😁

Share

Subscribe now

We launched a few months before our faithful friend chatGPT took the world by storm. Since then, we’ve been flooded with friends, colleagues, and new friends from all over the world asking how they can get involved with A.I. in surgery or more broadly in medicine. Today, we try to answer some of these questions in our deep dive. Let’s get started.

Table of Contents

📰 News of the Week: Healthcare is front and center
🤿 Deep Dive: Our most common question: How do I get involved with AI?
🐦🏆 Tweets of the Week

News of the Week: Healthcare Front and Center!

This week was quite wild, so buckle up 👷‍♂️

We’ll start with the clearly-healthcare news first. Microsoft and our favorite EMR overlord, Epic, announced deeper integration of OpenAI’s GPT-4 into their systems to automatically draft messages, and (unsurprisingly) assist in backend data curation analytics pipelines.

On one hand, this is encouraging! I love that these technologies are being used to target the bane of all clinicians’ existence - the inbasket message. On the other hand…

… water continues to be wet, and revenue optimization will likely continue to supersede clinician quality-of-life.

Other news, the venture capital group Andreessen Horowitz (a16z) recently published a fantastic piece on why tech should be building in healthcare. I think this should be mandatory reading not just for the tech sector, but for clinicians. Understanding the language of venture capital, how startups are ideated, funded and governed is key to helping bring these technologies into the clinic.

Notice the distinct lack of patient story, of clinic pain point, and of pathology. Also notice the distinct non-emphasis placed on the clinical perspective. I would encourage clinicians to avoid the holier-than-thou mentality we so often employ, and instead approach it with the following mindset: if the solution to our problems must come from these avenues, how do we bridge the gap so that VCs, technical builders, and the end-deliverers of care (clinicians) can work together?

Okay, some more news to get to:

Greg Brockman (OpenAI President) did a TedTalk where he uploads a spreadsheet into chatGPT and watches as it performs data analysis (makes tables / figures) for him in real time. Let me not mince words:

The ability for chatGPT to write code and interact with files you provide it will fundamentally change how we do research.

Spoiler: if you’re thinking of hiring a data analyst, I’d try chatGPT Pro for $20/mo first.

Meta has gone all in on the computer vision boat, allowing the other tech giants to fight for the chatbot/large language model scraps. They released DINOv2, a better and more generalizable image segmentation model. This is on the heels of Segment Anything, which allows users to (as the name suggests) segment almost any image with surprisingly good accuracy- even hip radiographs (see here). This has clear implications for radiology, but can absolutely be extrapolated as a preoperative planning tool for the O.R., possibly an educational tool for residents on call, etc.

We finish the news off with a demo by Humane (Twitter links still broken…), showing the future of wearables + A.I. models which can speak to you. Imagine a non-intrusive wearable which gets updates from your post-op patients, so you can monitor for complications before they arise. Or even simpler, what if after a long case, your wearable A.I. could just tell you everything you missed while your phone was down? We hope this is what’s coming.

Deep Dive: Our Most Asked Question: “How Do I Get Involved?”

One of the most common questions that we get when we're writing this is how can I get involved? So for this week's deep dive, I’ll answer that question by describing strategies for both clinicians and technical/scientific practitioners across the spectrum of training and practice. We’ll specifically address the following audiences:

• Medical trainees

• Technical trainees

• Medical practitioners and educators

• Technical practitioners

• Early learners

First, I wanted to open the floodgates for your questions or requests for collaboration. You can write to us at ctrl.alt.operate@gmail.com, tweet @ddonoho or @dhirajpangal

Second, here’s a question for you:

For the clinical learner (medical student, resident, etc…)

First, for clinical trainees. Initial prerequisites for getting involved are lower than you might think. You'll have a lot of time available to you even though it doesn't seem like it. Believe me, life only gets more complicated when you're a faculty. So the first step for you is to have a curiosity driven, honest, self-aware approach to this novel area. You'll have to think about what's important to you, what you value, what your skills are, and what you enjoy doing, not just talking about.

That could take you many different places. For example, you might design a curriculum using chat GPT to learn simple code in Python or you might enroll for the fast.AI course. While taking a massive online open course, I would have an LLM running in the background as an interlocutor: you can pause the lecture, ask questions, and even potentially design your own practice questions to improve knowledge retention. You can even trial the LMM's performance on the current exercises in the course to see how that LLM performs writing code itself. Understanding how to partner with AI to accelerate your learning is an evergreen skill that no one will teach you.

Basic statistical principles never go out of style. It's critical that you understand how to conceptualize and design experiments, and how to evaluate the results of those experiments. That'll be true whether you're writing a clinical paper, designing a drug trial, evaluating any surgical technique, doing a meta analysis, or anything else.

You’ll do most of your best learning alone, but most of your best work will be done in teams. Medicine in 2023 is too big, too complex and changing too rapidly for any individual to make a major impact. You can stay motivated and engage with relevant groups at your institution who are also working on similar problems. If they don’t exist, I would encourage you to create them. This is a new field, and we would be happy to mentor you from a distance.

For the scientific/technical learner (CS/EE major, masters/PhD student)

As a research scientific trainee, the answers are a little bit simpler. Achieving technical mastery in your chosen domain, along with broad foundational competence, will allow you to take on tasks that improve health.

As we suggested for the clinical trainees, gaining even modest experience in medical problem domains is extremely valuable. Practical hands-on experience solving one medical problem in one setting will dramatically increase your understanding of what it means to work in health care space. There are significant challenges in healthcare, including working with regulated data, managing teammates with diverging incentives/demands, dealing with sensitive patient information that has real world implications and understanding model performance in those contexts.

Your expertise in setting up a computational environment for scientific rigor will be valuable to the remainder of your team. In fact, many of the medical research settings you might find may have deficiencies in this area (including ours), and you’ll be the relative expert in best practices of code versioning, unit tests and many other relevant considerations.

One of the byproducts of a healthcare focused team is you may be operating at a significantly elevated role relative to your level of experience.This is not something that everyone wants. But for many, it's a unique and interesting opportunity.

There's also the side benefit of working on a project that can directly improve human health if it's implemented. But just like many ML and AI research projects, a large number of medical AI research projects are essentially executed as research objects for a paper publication, and then that's it. They never go anywhere.

So this is a place where the technical scientific collaborator can put on their industry hat and bring the prototypes into deployment. Technical collaborators who can draw from industry experience (theirs, or their colleagues) to implement products, be it in technology, life sciences, or otherwise, can vastly enhance the impact of innovations in the health domain.

For the medical practitioner (MD/DO/NP/PA/PT/OT/RN/Health PhD)

As a practicing clinician, you have access to a larger number of potential collaborators, a larger quantity of data, but also significant responsibilities and time constraints. You'll understand the clinical questions better, but achieving technical domain, competence is going to become increasingly difficult due to competing demands on your time, changes your level of interest, and the pace of development in this field, which can seem frustrating to clinicians who are used to learning skills such as, for example, biostatistics, and then being able to lather rinse and repeat using those same methods for every paper. That's in some ways true for ML and AI research.

The key tasks for clinicians are:

• identify high-priority problems (work on what matters)

• define a clinical problem in solvable terms (even if you don’t know the solution)

• organize and curate data for ML applications (you’ll know the health data best)

• build and co-lead teams (many funding sources are in the health domain)

• learn (formally or “on the job”) from your technical colleagues

• validate clinical relevance of solutions

• design solutions for implemented functions, not research objects

A hallmark of ML/AI research is the influence from industry hegemons in creating, developing, and deploying the most advanced models. In practice, you may develop a working model on your data and then next week, the model's underlying architecture could be made obsolete through previously unknown development in industry that had been created by the legwork of an army of a hundred PhDs and multimillion dollar research budget for model training. Working in a space where the technical innovations are largely not occurring in the setting of academic research labs is a very different feeling for clinicians. We are used to doing cutting edge work in their lab and then seeing that work brought to scale and to the bedside through productization, but here the products often come first, and then the application to health care comes later. A world where clinicians focus on adopting and adapting scientific advances to the health domain has implications for funding, for publications, for what it simply feels like to work in this field. The concept of “competing” with industry is likely not a winning pattern. Work in health ML/AI has to be more collaborative.

For technical practitioners (academic/industry CS, Eng)

For technical domain experts, on the other hand, the process is a little bit simpler. The ask from clinicians is often “Can you just do x on my data?” And the potential downside is that's not going to result in an ICML or a CVPR paper. Because those institutions don't necessarily care as much about applications, focusing on innovations and exploration in the technical domain (even if said “innovations” have zero relevance and a demonstrated, functioning application would save lives today). On the other hand, clinicians care only about applications and don't particularly care about the latest extremely large parameter model trained on an obscene amount of data nor some clever innovation in the architecture that enables the model to be increasingly performant. They care about whether that moves the needle for an individual patient. So managing those dual expectations from the technical side is critical to getting your graduate students into jobs, a key to achieving funding from your lab.

The key tasks for technical practitioners are:

• collaborate on problem identification and definition from the beginning

• teach your clinical colleagues how to organize and curate data for ML applications (you’ll know what the models “need” and what matters)

• build and co-lead teams (MD’s have a tough time retaining CS talent)

• learn (formally or “on the job”) from your medical colleagues

• seek technical innovation amidst clinical use cases

• teach MD’s to solutions for implemented functions, not research objects

We believe that creating long-term technical-clinical partnerships is a path to success. So in summary, I think that there are 4 different sets of challenges faced by both trainees and clinicians at the training and senior scientist level. I've outlined how to manage some of those tensions with examples of what we feel are some successful projects and some strategies for getting involved in this field.

For the early learners (high school, undergrad/postgrad)

Lastly, I wanted to address a group of folks for whom I have great admiration, the undifferentiated students. I’m talking to our high school students or undergraduates who haven't yet committed 10 plus years into a career path or training pipeline. Two years ago, I wrote that my economics training provided me with phenomenal background for health care. And I would certainly insist all the more that that is true today.

Explicitly acquiring a background in a computational method, whether that's a particular programming language or software design principles or hardware design principles or having structured interactions with engineers, scientists, and clinicians is going to become increasingly important whether you're a poet or a surgeon. In our increasingly digitized world, most of human activity over the next 10 years will be in reaction to that digitization in some way. Whether you're for or against it, inside or outside of it, you're going to have a relationship to digital disruption. So I think it's important that however you view your path, whether it's in health or science or the humanities more broadly construed, understanding what these digital technologies are about will pay outsize dividends.

For example, if you are a poet and can program, you are an extremely rare intersection occupied by a handful of people in the world. And your opinions may be quite interesting. Ditto for ethnography or romance languages or Japanese art history. The folks who can have some flexibility across domains are going to increasingly find themselves in a rare space with rare opportunities. Two years ago I argued that economics was a foundational major for health care, and I still believe that to be true today. Where I went to college, we had other foundational requirements: languages, distribution of courses, and even a swimming test. We might wish to consider whether having some computational exposure in software, hardware, engineering, or even in the interaction of humanities and ethics with technology should be considered pre-requisite requirements. And realizing this - before your peers are required to realize it - could be a real source of opportunity.

Best of Twitter

Twitter previews are still broken on Substack… we’ll pause these until a workaround or resolution exists. Anyone have one?

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Subscribe now

Share

Leave a comment

The discussion about artificial general intelligence (AGI) and autonomous AI agents has reached a fever pitch, and this week we will dive in to the medical and surgical implications of AGI and autonomy. Unsuprisingly, calls for regulation have also heightened. We’ll tell you what to watch out for, and why some of the calls might be more self-serving than they seem. We’ll also cover press releases from and others that have broad application to the medical domain. Lastly, our tweets of the week describe a groundbreaking study using human-AI teams to read ICU chest x-rays for acute respiratory distress syndrome (ARDS), a life-threatening condition that requires modification of mechanical ventilation strategies to improve survival. So, who should read the chest X-ray first, a human or an AI? Let’s get into it.

As always, we’ll do the heavy lifting, keeping track of the who’s who, what’s what, and bringing it back to the OR, clinic, hospital, or wherever else you find yourself delivering care.

Table of Contents

📰 News of the Week: Regulators, Mount Up!
🤿 Deep Dive: AutoGPT, What’s the Worst that Could Happen?
🐦🏆 Tweets of the Week

🔥📰News of the Week: Regulators, Mount Up!

The Japanese government announced that they are considering integrating AI into government functions. Coming on the heels of Italy’s “ban” on chatGPT, and growing calls to regulate AI in the United States, the diversity in governmental responses to AI should be closely followed. Based on other heavily regulated industries such as medicine and finance, we can predict an initially incoherent mish-mash of attempted regulation and an ultimate convergence of regulatory regimes necessitated by the increasingly interconnected nature of software and internet technologies. Ezra Klein argues that the industry itself is clamoring for regulation:

Among the many unique experiences of reporting on A.I. is this: In a young industry flooded with hype and money, person after person tells me that they are desperate to be regulated, even if it slows them down. In fact, especially if it slows them down.

Government regulation can be profoundly anti-competitive, aiding market leaders to calcify their position. Much more to come on this front.

AWS finally returned fire in the AI PR wars, dropping a new foundation model-as-a-service tool, Bedrock. If you’ve got your data on AI, and a bunch of lambda functions and integrations running, now you can plug in a foundation model with ease (as long as you don’t want to use GPT, midjourney, Llama, or other leading models). Remains to be seen if they can compete with Azure marketplace

announced another exciting foundation model press release that further increases medical domain generative text performance in response to clinical queries. As with all google products, it remains unclear how this will be productized, validated and ultimately deployed based on the scant press release, but the hype wheels are turning again.

AI Breakfast interviews Zak Kohane:

It's also clear that students are using these tools whether we guide them or not and furthermore, many of our patients are doing the same. Therefore, embracing and understanding this new player in the patient, doctor relationship seems important and worthwhile.

Caption anything: a logical extension of Segment Anything Model + chatGPT to create image labels, brought to you by Chinese mega-media conglomerate Tencent’s research labs. You can try it out in an HF space here. The implications of these technologies for visual question answering and model assisted labelling are only beginning, starting a new wave of tools that unlock visual data for quantitative analysis.

Oh, but you don’t have enough visual data, you say? Not to worry, here comes Internet Explorer: a self-supervised web-connected agent for training data acquisition. You provide a (small, inadequate) model training dataset and the agents scour the web for images that progressively improve the performance of your classification model. For example, say you want to train a model to classify folks who are wearing masks properly (PPE detection was a hot topic for obvious reasons). You provide the model with some images of mask wearers, with labels, and see that your model is performing poorly. Voila:

How does this technology work, you ask? Sounds like it’s time for a primer on autonomous agents

🤿 🤖 The Next, Next New Thing: Autonomous Agents

This week's deep dive is on a topic that has exploded in popularity in the past two weeks - autonomous A.I. agents. Unlike traditional chatGPT models, Autonomous A.I. agents work without human intervention. These agents can perform tasks, generate new tasks based on completed results, and prioritize tasks in real-time. Imagine an early version of Jarvis from the Iron Man series. This week’s series of autonomous agents use the popular OpenAI GPT-4 models to access internal systems, communicate with other GPT-4 models, the internet, etc. These "agents" can iteratively communicate with each other, creating tasks, completing tasks, and reprioritizing tasks until the ultimate job is done.

The idea is: if you want a coffee ordered, the autonomous A.I. agent would be able to create a task list that includes: 1) knowing your current location, 2) knowing the nearest coffee shops with delivery, 3) accessing the coffee-shops online ordering system, and 4) placing the order for pickup or delivery.

This is a simple example, but can be scaled to complex tasks which require the creation of task lists, and iteration on that list depending on the results: market research analysis, event planning …

The kickstarter for the Autonomous Agent trend was a paper/model release called BabyAGI (or Baby-Artificial General Intelligence), and it has been the top trending repository on GitHub for the past two weeks. A team out of Google/Stanford also released this paper (image above), where they created a Pokemon-like game simulation and had multiple AI models communicate with each other within the game. The end result: the AI’s independently planned a birthday party. Even crazier, some of the agents said they couldn’t attend! Since then, there have been dozens of product demos that have shown the ability to run these AI agents on your desktop, browser, having them write and execute code, etc. Needless to say, this is just the beginning

The original tweet can be found here.

Somehow, we think these agents will make its way into the healthcare space sooner than expected (our take is +/- 1 month). Accordingly, we wanted to bring it to you, clinician/surgeon/healthcare enthusiast, so you can be prepared for when it inevitably does splash.

In the healthcare space, autonomous A.I. agents have the potential to bring transformative changes, and ultimately reduce the administrative burden placed on the system and on clinicians.

Imagine an autonomous AI agent as your “sidekick” that anticipates your day and identifies problems before they reach patient care.

For surgeons, AI agents could streamline preoperative planning. If the agent knows the types of cases surgeons do, what is contained in each tray, it can communicate with an AI model specializing in operating room scheduling, ensuring all necessary surgical equipment is available for a given case, and with particular circulators/scrub techs. The AI agent could then speak with another AI model responsible for dealing with prior authorizations, negotiating with insurance companies, and scheduling patients. The important principle is that these are all directed by the human at the top level, but the underlying “tasks” are carried out somewhat autonomously.

For clinics, autonomous A.I. agents could help manage in-basket messages and other administrative tasks, allowing them to focus on patient care. An AI agent could communicate with different AI models to sort, categorize, and prioritize incoming messages, and even draft responses or delegate tasks to other staff members. This coordinated approach would help clinicians manage their workload effectively and improve overall productivity.

Improving patient experience is another area where autonomous A.I. agents could make a difference. AI agents could communicate with different AI models to coordinate appointments, and provide real-time updates on test results and treatment options. We all want to answer every question our patients have; AI agents fine-tuned to your clinic workflows and protocols could allow your patients to have access to a triaged version of you, without burdening you or your staff. By streamlining communication using various models, patients would receive a seamless experience that reduces stress and enhances their overall satisfaction. Not to mention the administrative burden which is forcing more and more clinics to shut down, or sell to private equity.

We can use autonomous agents to test novel methods of health-care delivery. The agents themselves can create new patterns and systems.

For example, you could set a policy for these agents that might task them with minimizing clinic wait times, or maximizing imaging accuracy, and place them in a simulated model of your hospital environment. Want to see what happens if you move your clinic from the 4th to the 1st floor, or add an extra scrub tech to the OR rotation? The autonomous AIs would then interact as simulated patients, doctors and nurses to evaluate your change. Some of the AI agents might even be permitted to discover and implement their own optimizations in the “digital twin” simulated environment. Maybe they would be able to come up with useful solutions, just as AI has often (but not always) beaten humans at Go, Starcraft, and countless other games.

We firmly believe clinicians need to lead the deployment of these AI innovations. AI agents can create new challenges in data privacy, model accuracy, and ethical considerations. The less oversight, the more dangerous these models. And of course, the question of clinical accuracy always (rightfully) comes into play. Also, when does this require any sort of FDA oversight?

These models aren’t close to deployment yet. But, if the past few months have taught us anything it’s that the gap between “not there yet” and “there” can be closed at lightning fast speeds. It’s our job at ctrl-alt-operate to keep you well informed, so when an healthcare AI agent shows up in your inbox, you’re primed for what it might (and might not) be able to do.

🐦🏆 Tweets of the Week:

In a clear winner of our coveted golden tweet award for this week, Farzaneh et al describe their method for teaming AI with human reviewers for chest X-rays for the indication of ARDS. Their best paradigm was to send the XR to the AI first, and then if the AI has intermediate confidence in its classification of ARDS or not ARDS, having a physician over-read the study. Similarly, for images where physicians were not confident, the AI outperformed humans. What we really need, of course, is a foundation model for chest-XR that is capable of multi-domain performance rather than a single model trained on a single disease state at a single institution. Stay tuned… https://twitter.com/npjDigitalMed/status/1646546253650558976?s=2

https://twitter.com/sheng_zh/status/1645459835071385605?s=20

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Share

Subscribe now

This week we will dive deeper into the use of computer vision models for surgical performance assessment by explaining the strengths and limitations of the models and methods, at a level that surgeons and clinicians can understand. We’ll give you a rundown of our trip to Montreal for Surgical Data Science sessions and a visit to the virtual reality and simulation lab at MNI. The war of words surrounding the potential risk that AI poses to human existence has been heating up, the AI hype cycle is going full speed ahead, and regulators are taking notice.

As always, we’ll do the heavy lifting, keeping track of the who’s who, what’s what, and bringing it back to the OR, clinic, hospital, or wherever else you find yourself delivering care.

This week, you’ll notice that some twitter links are a bit off due to the substack-twitter feud. Ultimately, the feud may even be good for our newsletter as it will encourage us to focus on topics more interesting to our readership, but the twitter images will be less “clickable”. So you’ll see a URL beneath them. We will keep you posted.

Table of Contents

📰 News of the Week: Trip Report to Canada and Quick Hits
🤿 Deep Dive: More Than Meets The Eye: Vision Transformers for Surgical Performance Assessment, Part 2.
🐦🏆 Tweets of the Week

🔥📰News of the Week:

Trip Report: Montreal 2023

Last week we took a trip to Montreal for a Surgical Data Science symposium. This organization, led by Lena Maier-Hein and others, has been bringing folks interested in surgery, technology and AI to meet to discuss progress and outline future opportunities in the field. Our meeting focused on the needs assessment of surgeons for developing AI systems in the OR, and gave us some new connections to other global leaders in surgical AI.

We then visited the Montreal Neurological Institute labs, including Prof. Rolando Del Maestro (Neurosim Lab), Prof. Louis Collins (Neuroimaging and Surgical Technologies lab), and their teams for a research presentation and tour. We presented our recent updates on surgical performance assessment and learned about Prof. Del Maestro’s process for evaluating VR simulators and building an autonomous surgeon feedback system in VR, and Prof. Collins’ pioneering work in neuronavigation. Dan got his surgical technique graded in both VR and in an ex vivo calfs-brain model of epilepsy resection (he did well, they said). Here’s an example of the recent work from the Del Maestro lab, from first author Recai Yilmaz MD PhD.

Quick Hits

Google’s CEO claims that they will start integrating chat into Google search. Feels like a defensive move since OpenAI plug-ins and Perplexity have made significant inroads into the search market for the first time in years.

Meanwhile, ChatGPT sturm-und-drang continues with real developments, anti-hype and hype. Anti-hype: ChatGPT was banned in Italy for “privacy concerns”, and since EU follows a common standard, additional regulatory action (but not necessarily bans) will follow.

Voices calling for a “pause” in AI research are growing louder, and so are the counter-voices. This is an issue that touches on technology, regulation, domestic and even international political concerns. Here’s Tyler Cowen, polymath-economist (marginalrevolution.com is a daily read of mine):

Stanford released their “State of AI” update. As readers will know, industry is developing ML models at a scope, scale and pace 10x greater than academia. This is due to cost and hardware requirements, as well as the need to respond to increasing demand for business use cases (instead of academic CS research).

Here’s a take-home chart from Eric Topol showing the rising cost and computational requirements (Floating Point Operations / second) of recent models (GPT 3, chat and GPT4 are not pictured because those details were not released)

BloombergGPT press release: Bloomberg developed an in house, “small” 50B parameter model (1/3 the size of GPT3, GPT4 is unknown). The model was trained on Bloomberg’s financial data over 40 years (web data, news, corporate filings and press releases) as well as public datasets, requiring 1.3 Million GPU hours (A100s, ~$4/hour if purchased on spot instances, they paid less) over 53 days. For a “small model”, it performed well on both general tasks and outperformed many existing models on financial question-answering (no surprise, given the training data) and financial headline generation.

Meanwhile, smaller models with excellent performance are being released daily. Vicuna, a 13B open source model that can be trained for $300 (wouldn’t even make it onto the graph above), achieves 92% of chatGPT performance and can run on desktop hardware. It’s trained on user-shared conversations from chatGPT and uses GPT-4 to judge its output.

The hype cycle is increasing around “Autonomous GPT systems. ” In one version of this system, the user defines an identity and goal for the chatbot, gives it access to the internet, and allows the chatbot to prompt itself continuously. In this case, a “Chef GPT” is asked to discover an upcoming event and create a new recipe for that event in order to increase net worth, grow social media accounts, etc. Here’s the final recipe (for Earth Day). It’s underwhelming today, but these technologies are going to be used beneath the surface for many applications in the months to come.

In health-GPT, we saw demos like petvet.ai, which purports to provide both diagnoses and treatments for our furry friends in a chat-like interface. I do not endorse this use case and would not trust it with my worst enemies. Physicians and surgeons (human and veterinary alike) will increasing experience patient reports of interactions with these systems due to the largely-unmet demand from patients for information and control over their health information. This trend is only accelerating.

Segment Anything: Facebook AI Research Provides an Open-Source Foundation Model for Image Segmentation

This week, released a foundation model for image segmentation, called "Segment Anything". Image segmentation is the process of classifying one or more pixels within an image as part of an object, like a person, tool, or anatomic feature. Usually, when we are developing models for object detection or segmentation, we have to provide many hundreds or thousands of labelled images to the neural network in order to "train" it what each specific object looks like. Assembling that "training set" requires vast amounts of time, human labelling effort, and money. And then every time you have a new project - you need a new training set. 's model performs object detection and segmentation automatically across any type of image without specific training. That's why it's called a "foundation model": the model's training dataset is so large, and the model is so capable, that it can be applied to new uses with zero- or minimal retraining. You can see an example of this process below, where I used the model to segment previously-unseen surgical video from our publicly available dataset of cartoid artery injury simulations, MRI scans, and other biomedical image sources.

https://twitter.com/visishs/status/1643705449743540231?s=20

🤿 🤖More Than Meets The Eye: Vision Transformers for Surgical Performance Assessment, Part 2.

Our deep dive continues our investigation into the use of computer vision tools for surgical performance assessment. In part 1, we explained the need to 1000x global surgical care, described our rationale for developing a Surgical Artificial Intelligence System, and gave a brief overview of three papers from the larger USC-Caltech group that illustrated a use case of this AI system. In part 2, we delve into the “how”, introducing a higher level understanding of vision transformers so that you can have a general idea when computer vision models might be successful, when they might struggle, and what would be needed for you to get involved in this field.

Ok, that’s nice Dan, now tell me:

• What does this system require? (from surgeons and physicians)

• When might it fail?

• What else can it do / how do I build my own?

• How does a neural network work? (without putting me to sleep)

Away we go!

What does this system require: 1. Lots of video. This system was trained on 78 videos divided into 4,774 samples from 19 surgeons. 2. Labelled video. Right now, we need to provide the system with labelled data for training and testing. This requires labels for steps, actions (also called subphases), gestures (surgical instrument movement type) as well as skill assessments (high skill, low skill). This includes hundreds of gestures manually labelled. While the model is now capable of generalizing to unseen video from related procedures with related modalities, it is reasonable to expect a data annotation pipeline to be a critical part of at least validation, if not training the AI system. 3. New use cases both within urology and other disciplines. 4. Continued validation with short and long term clinically relevant endpoints. 5. Pipeline for production. Although not a requirement upon the physicians and surgeons, the current model is a research object that requires design and implementation testing prior to implementation in a clinical workflow.

When might it fail: 1. All AI systems have a risk of bias, and we outlined strategies where we can use training with explanations (TWIX) to ensure that the model and human experts remain aligned. 2. SAIS can ingest very long videos, but the current tasks that it performs are executed against short clips. Long sequences of video are broken down into shorter clips (seconds to a few minutes) for certain tasks within this model. There are challenges and potentially unreasonable computational requirements to ingest videos of many minutes or hours in length. The core belief is that transforming video to sequence / action data enables us to unlock longer surgical procedures. 3. Anatomical understanding. The model doesn’t take patient anatomy or anatomical features as an input. 4. Moving camera perspective. The model was trained and validated on robotic surgery data. We believe that the underlying architecture should work well with other video types, but that’s a testable hypothesis. 5. We don’t know that this model generalizes to all surgical procedures, and there are some surgical procedures which are so variable as to make structured understanding more difficult.

What else can it do / how can I build my own: The research here is in its early days, and there are some interesting possibilities with this architecture. Dani Kiyasseh, the lead machine learning scientist, plans to release the model architecture and code in the coming weeks in his github repo at danikiyasseh.github.io . If you want to build your own… keep reading…

I’m still reading, and I want to learn more about how neural networks function.

Here’s how vision transformers work. First of all, vision transformers are a specialized class of transformers, a neural network that uses a process called self-attention to identify the “most relevant” components of an input (text/image/EEG). Let’s start by understanding the basics of the most common neural network, a multi-layer perceptron, and then understand how that building block fits into a video vision transformer.

Neural Network Basics: Neural networks can accept any type of input data, whether it’s images, text, EEG waves, since it’s all 1s and 0s. These sequences of 1’s and 0’s require specific methods of processing before they are presented to the neural network. This is sometimes called “feature extraction”. Image data might be processed by using methods for detecting edges, textures, colors, or simply by raw pixel values. In a transformer, words are processed into tokens, which are numerical representations of “chunks” of the sequence. Tokens and input sequences have a deterministic relationship - if you know the token, and the tokenizer function that created it, you can always recover the same input from the token.

Here’s an example of character and word tokenizers. The little numbers below the letters or words are the token values, or what the model wants to see.

There are several other steps that have to occur (vector embeddings, etc). Now that you’ve got your data “ready”, lets take a look at the neurons in the neural network itself. This kind of network is called a multi-layer perceptron, or MLP.

Neural networks are made up of many (billions, sometimes) of neurons that are organized in layers. Layers in between the input and output layers are typically called “hidden” layers, and they do most of the “work” for the network.

Neurons in one layer are connected to neurons in other layers, but usually* not to neurons of the same layer. Unlike in the human brain, we are most interested in the connections between neurons, not the neurons themselves. Neurons in an AI network typically don’t perform any functions on the data within the neuron. As data passes along the connection between neurons, a mathematical function is performed on the data. This function along the connection between neurons typically contains two “parameters”, or learnable features: a weight and a bias. When data passes from an input neuron along a connection to a receiving, or output, neuron, it is changed by this mathematical function that includes a weight and a bias. The weight and bias are modifiable during the model training process.

So, when you hear that GPT-3 had 175 billion parameters, that means that it had about 87.5 billion connections between neurons, and about 40-80 billion neurons. For reference, the human brain has about 80-100 billion neurons. However, the human brain has a much more complicated set of connections between neurons than most neural networks, with thousands of connections (synapses) from each neuron. In total, the brain has over 100 trillion synapses- 1,000 connections or more per neuron. Once we get to models with hundreds of trillions of parameters, we will truly be at human-scale endeavors. Maybe next year…

You can try this example of an MLP here in a playground.

So how do we train this thing? In a common training method, called supervised learning, you’ll give the network pairs of input data and labels, like a picture of a tool and it’s name (for an object detection task), or sentences with words deleted and the missing words (for NLP tasks). The network then “learns” to output the label based on the the input it “sees” through a training process that adjusts its parameters to improve its performance. In the beginning, the connections between the neurons (weights and biases) might be random or have pre-assigned starting points, so it won’t make very accurate predictions. Let’s train them!

After a pre-specified amount of data (batch) passes through the network (forward pass), the outputted predictions (such as dog, cat) are compared to the true labels (such as dog, dog) using a loss function. The results of that process are then passed back through each layer of the network (backward pass/backprop) in a process that alters the values of the parameters (connections between neurons) to improve the network performance on the next batch (hopefully…).The new parameter values are used for the next batch. Then another batch goes forward through the model, the outputs are scored, and the parameters are updated again in the back prop. Training then continues thusly through the entire data set (epoch) or longer, until any one of a variety of stopping conditions occurs.

Once you have a trained model, you can put it to work by providing it with unlabelled data, and it will provide outputs (like numerical predictions, classifications, words, etc). This is called “inference”. Because the model is only running “forwards” and does not need to update itself, inference requires fewer mathematical operations than training. Thus, inference can be performed on less powerful computational devices than training.

This all sounds very nice, but there’s one problem.

Images are really big compared to text data. Videos contain many images. Even if we had a 100B (or 100T) parameter model, and really good feature extraction, we might not be able to capture the “right” relationships between pixels. So an MLP, in and of itself, won’t get us the right kind of predictions we want. Fortunately, a variety of different types of algorithms and data processing steps have been built around the MLP in order to augment its ability to handle visual data. One of these that we we think will work well is a transformer.

What’s a Transformer, Anyways: The classic example of a transformer is a type of encoder-decoder architecture. Meaning: some data → encoding function → decoding function → some output. Transformers perform a series of mathematical operations on input data in a step called an “attention” layer before the data reaches the “multi-layer perceptron” (MLP) we described above. In an attention mechanism, the elements of an input sequence are compared to understand their relatedness, the relevance of each element to the current sequence, and the importance of each element. The “relatedness” or “importance” of certain elements of a sequence helps the model respond more strongly when it sees those elements. The outputs of the attention layer are then fed forward into an MLP (like above), and an output is then passed into the decoder architecture that provides the model output (a word, object presence, numerical value). There’s a lot more going on here, but the high level understanding is that transformers have the ability to use the relationships between data in a sequence to provide better predictions of subsequent elements in a sequence, and that they inherently utilize this “relatedness” or “expectedness” in their operations. This is referred to as “attention” because someone people believe this is similar to how humans “focus” on the most relevant stimuli to the exclusion of less relevant stimuli. There are many different types of attention (and self-attention), particularly in multidimensional inputs. Which brings us to …

What’s Different about Vision Data: Images are both larger (more 1s and 0s) than text data, but they also contain two spatial dimensions that are important instead of one-dimensional sequence data like words. So, we have to manipulate the 1s and 0s in a particular way to both preserve these relationships and utilize them within the network, in ways that aren’t required for text. Rather than tokenising an image into a short number, we have to perform additional manipulations. For example, we divide each image into different pieces, or “patches” that may be 16 x 16 pixels in size, and place them in a sequence (flattening, linear embedding), then feed them into the network, while keeping track of where they came from (positional encoding). We also have to think about relatedness, for transformer architectures, in multiple dimensions. This is important because transformers quadratically (x^2) increase the number of computational operations based upon the sequence length.

What’s Different about Video Vision Data: Video is the most challenging and resource intensive type of data to work with because it not only has positional (image patches) data but also temporal data. Thus, naively you might need exponentially more mathematical operations for model training and inference, the longer the video, the worse it gets. But people are developing methods to work around this. A good example is the TimeSformer architecture by FAIR, but there are many other/better methods. Here’s their cartoon, you’ll note the patches, some are highlighted in red (indicating that the temporal attention network is giving them higher importance), or into blue/green (indicating greater spatial attention), and the potential interaction between the two. We didn’t use this architecture, but we have some related references below.

What are some examples of video vision transformer models you used in this paper?

https://openaccess.thecvf.com/content/ICCV2021/papers/Caron_Emerging_Properties_in_Self-Supervised_Vision_Transformers_ICCV_2021_paper.pdf

https://github.com/facebookresearch/dino

https://arxiv.org/abs/2104.14294

https://ai.facebook.com/blog/dino-paws-computer-vision-with-self-supervised-transformers-and-10x-more-efficient-training/

🐦🏆 Tweets of the Week:

https://twitter.com/npjDigitalMed/status/1644251301809913857?s=20

https://twitter.com/DanStoyanov/status/1642962951823826976?s=20

https://arxiv.org/abs/2303.17651 (ht:Lior tw@AlphaSignalAI)

https://twitter.com/SurgicalAI_Lab/status/1642271032055611393?s=20

https://twitter.com/mattrly/status/1642005132475867136?s=20

See you all next week.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Share

Subscribe now

Our deep dive this week goes behind the scenes at three Nature-family publications released this week 🔥🔥 by our extended research family at Caltech-USC where we taught computer algorithms to “see” surgical video and provide rapid, actionable feedback to improve surgical outcomes. If you want to catch up on computer vision, we feature it heavily in our writing (here, here, here, here) because we believe it is the most impactful tech coming to surgery.

As always, we’ll do the heavy lifting, keeping track of the who’s who, what’s what, and bringing it back to the OR, clinic, hospital, or wherever else you find yourself delivering care.

Table of Contents

📰 News of the Week
🤿 Deep Dive: Surgeons Using A.I. to Get Better
🐦🏆 Tweets of the Week

🔥📰News of the Week:

While this week didn’t have quite as much pomp and circumstance as a few weeks ago, there’s still plenty to talk about. Fortunately with less Earth-shattering news, we can hone in on some A.I. news in medicine and surgery.

A.I. has officially become mainstream in medicine - the New England Journal of Medicine (NEJM) recently published a new series outlining the uses of A.I. This is a great primer and perfect for your colleague who says things like “I don’t believe this A.I. stuff” (i.e. the same one who said EMRs would never replace charts). NEJM even has an article dedicated to ChatGPT/GPT4 and how it can be useful in the future for helping patients understand their own conditions, and helping ease administrative burdens. It’s a perfect appetizer for a ctrl-alt-operate subscription 😉

Two papers in the neurosurgical literature were published this week, showing chatGPT performed impressively well on the neurosurgical written boards, and how GPT4 shows significant improvements, getting 83% of questions correct - well enough to pass. For context, the mean score of this neurosurgery specific Q-bank is 69%.

One of the more underrated pieces of news this week was the release of HuggingGPT, a system which uses LLMs (i.e. chatGPT) as a controller to execute different A.I. tasks across different functions. This is an extraordinarily powerful concept. It allows intelligent language models to: intake a goal, select the appropriate models from a menu of options, and interpret the results for you. Study this image, we think it’ll come back again in the future…

🤿 🤖More Than Meets The Eye: Vision Transformers for Surgical Performance Assessment

As astute substack readers may know, both Dhiraj (undergrad, med school) and Dan (neurosurgery training) spent formative years at the University of Southern California. Over the past decade, we placed the first digital planks in the foundation of a surgeon-scientist-industry partnership that is now driving the digital transformation of surgery. 

This week, we dive into the research resulting from the leadership and vision of Dr. Andrew Hung, Dr. Gabi Zada, and Prof. Anima Anandkumar, the innovation and skill of scientist Dani Kiyasseh, and the hard work of dozens of teammates in Los Angeles and across the world. We would need a book to do justice to this overnight success more than ten years in the making but we’ve only got your attention for another 83 seconds, so let’s get into it.

We should develop technology to make surgery safer for all people worldwide.

Lack of feedback critically limits surgical improvement. After the completion of formal training and certification, surgeons receive approximately zero performance feedback during the remainder of their careers. Despite living in data-poverty, surgeons work diligently to improve their skills through decades of practice, relying upon their memory, experience and intuition. Without quantitative data and feedback, the individual ability of surgeons to improve is constrained, delayed, and inequitably distributed.

Improving surgical skill is a life-or-death imperative for each of the 300 million patients undergoing surgery each year. Improving surgery is a global health priority: perioperative death is third-leading global cause of death according to WHO.

A 1000x expansion of global surgical care is a human rights imperative: the 1000x disparity in surgery-per-capita between the most limited (Ethiopia) and most available (Colombia) nations results in over 143 million needed surgeries that are not performed, costing millions of lives and trillions in lost GDP and over the next decade.

In order to safely and effectively 1000x global surgery, we require the ability to systematically measure, understand, and improve the basic constituent steps and actions of surgery at scale.

We can fundamentally transform human health by addressing this deep human inequity and trillion-dollar market opportunity using existing technology.

Look at all of these potential data sources. Today, their data fade into the ether.

In past years, we published public surgical video data to train machine learning systems and developed a system that beat human experts in predicting when surgeons would be unable to control life-threatening bleeding during brain tumor surgery.

We have a simple message:

Video of surgery contains life-saving data.

Please stop throwing video away or locking it in a desk drawer.

Here’s what our team can do with surgical video data:

This week, Dr. Hung, Prof. Anandkumar, lead scientist Dani Kiyasseh, and team published results of their deep learning, computer vision-based model that can watch and evaluate the most minute actions of surgery, called gestures, using video data from a global network of hospitals and across several procedures. By studying Dr. Hung’s surgical videos of surgeons performing urologic surgery, Kiyasseh and team developed a Surgical Artificial Intelligence System (SAIS) that can watch new videos it has never seen before, detect steps and gestures, and assess surgeon skill.

Across three papers in Nature-Portfolio journals, Kiyasseh et al. describe our arc of research:

1. Develop a unified surgical artificial intelligence system capable of reliably detecting surgical steps and gestures, evaluating how skilled the surgeon was at suturing, and generalizing across unseen procedures and hospitals. This unified system can be used to create quantitative representations of what happens during surgeon, qualitative assessments of how well it was done, and can even explain why it assigned a grade (down to the video frame and movement type). This kind of system can address the lack of feedback and assessment in surgery. (Nature Biomedical Engineering)

   

2. Improved the surgeon feedback provided by the AI by making it more fair. The AI’s skill explanations were not consistent across all institutions, patient characteristics and surgeon types. We developed a training methodology (TWIX-training with explanations) that explicitly taught the AI which frames were most important using the explanations given by human experts. TWIX keeps the model focused on the parts of surgery that matter, and doesn’t allow it to pick up on extraneous or confounding visual data. TWIX improved model performance and reduced bias compared to a standard attention (vision transformer) architecture. Knowing that the AI’s judgements were more closely aligned with how humans would have viewed a surgical scene improves the trust humans have in the AI’s judgements. (Nature Communications Medicine)

   

3. We took a closer look at bias and our proposed solution, TWIX, in the deployment of SAIS to other hospitals in order to detect bias and evaluate our mitigation strategy. We also evaluated whether TWIX actually made the model predictions better, finding that in some cases TWIX led to marked improvement while in others, the overall prediction improvement was small or slightly worse. Our work highlights the importance of looking for bias at the patient, surgeon, hospital and procedure level and developing mitigation strategies (npj Digital Medicine).

   

I encourage the readership to visit https://danikiyasseh.github.io/blogs/index.html, where lead author Dani Kiyasseh and team maintain general-audience focused narratives of this work.

What this means for surgeons, students and scientists:

We have the tools we need to build the “surgical report card” where you can track your progress and see your improvement over time. We can overcome the feedback gap. We can deploy these technologies at a distance, with fairly low cost, to any surgeon across the world.

There are real challenges in this work, primarily geared around the requirement for large datasets and high quality annotated surgical video. We are developing methods to reduce our reliance on experts, but keeping AI aligned with humans requires a significant amount of interaction and assessment.

We still don’t know the best ways to present this data to surgeons and represent the surgical episode. Will we need a video reconstruction a la HawkEye in MLB? Or is text narrative more efficient? Probably a mix of both…

Lastly, we are actively looking for collaborators in this work. Surgeons can join this effort from any surgical discipline. We welcome all collaborators and contributors. The only thing you need to bring is your available and accessible data.

Sneak Preview of Part 2:

In Part 1, we covered “why” we do this work and “what” it is that we do. Next week we will delve into the “how”, introducing a higher level understanding of vision transformers so that you can have a general idea when computer vision models might be successful, when they might struggle, and what would be needed for you to get involved in this field. Until then …

🐦🏆 Tweets of the Week:

Physicians…if you don’t know how to program and don’t think you can - here’s some inspiration:

And in normal news…

See you all next week.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com

Share

Subscribe now

AI showed some early signs of general intelligence that seem closer than ever

What does it mean for MD’s to have language models that can beat 90% of US medical students on USMLE steps 1 2 and 3 and are showing behaviors that some are calling “sparks of artificial general intelligence”?

Will these new AIs come take our jobs?

Will we be begging them to take away our legacy medtech EMR products that are still killing patients?

This week continued the trend of breathtaking (and breathless PR) reports of developments across the AI/ML world at unprecedented scope, scale and pace.

As always, we’ll do the heavy lifting, keeping track of the who’s who, what’s what, and bringing it back to the OR, clinic, hospital, or wherever else you find yourself delivering care.

Table of Contents

📰 News of the Week
🤿 Deep Dive: Plug-ins
🐦🏆 Tweets of the Week

🔥📰News of the Week:

Last week we said there aren’t too many weeks where things “feel this different”.

This week, the AI/ML world said “Hold my beer.”

The digital health tech market is valued at $1.3T. Large Language Models are more creative than 90% of humans. GPT-4 got connected to the internet via plug-ins and browser, which is a major development that we will discuss in greater detail. Many of the earlier problems with language models and simple computational tasks seem to be surmountable by allowing the language models to call previously developed, more specialized functions (a la Toolformer).

(stephenwolfram.com)

Bill Gates wrote an influential editorial where he compares the development of AI to the development of the graphical user interface (GUI) as a fundamental method of interacting with computational systems. This is a long, but worthy read.

Here’s what he has to say about health care, much of which aligns with what we have been saying here since our inception:

AIs will... help health-care workers make the most of their time by taking care of certain tasks for them—things like filing insurance claims, dealing with paperwork, and drafting notes from a doctor’s visit. I expect that there will be a lot of innovation in this area.

Other AI-driven improvements will be especially important for poor countries, where the vast majority of under-5 deaths happen. For example, many people in those countries never get to see a doctor, and AIs will help the health workers they do see be more productive. (The effort to develop AI-powered ultrasound machines that can be used with minimal training is a great example of this.) AIs will even give patients the ability to do basic triage, get advice about how to deal with health problems, and decide whether they need to seek treatment.

…

In addition to helping with care, AIs will dramatically accelerate the rate of medical breakthroughs. The amount of data in biology is very large, and it’s hard for humans to keep track of all the ways that complex biological systems work. There is already software that can look at this data, infer what the pathways are, search for targets on pathogens, and design drugs accordingly. Some companies are working on cancer drugs that were developed this way.

The next generation of tools will be much more efficient, and they’ll be able to predict side effects and figure out dosing levels. One of the Gates Foundation’s priorities in AI is to make sure these tools are used for the health problems that affect the poorest people in the world, including AIDS, TB, and malaria.

In fact, OpenAI has already been making inroads as a tool for drug discovery and clinical trial administration.

Google released Bard, its chatGPT competitor and Twitter responded by dunking on it in every way imaginable. For a $1.4T market cap company, Google’s underwhelming press releases are an interesting institutional failure.

NVIDIA had a major release this week, rolling our their PR campaign for their new H100 chipset (“12x faster in inference”) and DGX cloud compute services. became a services platform. Makes sense: if you have all the GPU’s, who better to train the models?

Speaking of GPU’s, would you like to know where some people think they are? Notably missing: OpenAI/$MSFT (allegedly "tens of thousands of A100's", (how many M3 chips ...), (admittedly not A100s), and governmental TLA’s and their overseas equivalents. Oh, and said it would buy 16,000 H100s (hi cerner!).

And what if you don’t have all the GPU’s? Don’t worry…

And what if you only have $100? A version of your trained LLM’s can run on minimal hardware as well.

What can we do in images and video with all of this computing power? The world of generative AI images is really hitting its stride. Imagine the ability to create videos, 3D scenes and elaborate 2D images from text, and now put that ability into a product that everyone knows and loves already. With more than 30M paying Creative Cloud subscribers, Adobe is one of the most overlooked software ecosystems in medicine. We might want to start paying more attention, because this week Adobe Firefly dropped its demo of generative AI tools. If you can describe or sketch, Firefly can bring it to life. and the biggest impacts will come for those who already are skilled and have design sensibilities, because those folks can now go from prototype to finished product in minutes or hours instead of weeks.

Bing Image Creator brings DALL-E generative AI art into search and chat, as we predicted weeks ago.

Instruct-NeRF2NeRF highlights the ability to use text-based instruction to transform 3D neural radiance field - stitched 2D images. Want more snow in your skiing videos? Or to join a party video when you were really sitting at home working on your newsletter?

Yep, that came out this week as well in Runway’s Text-to-video AI demo. If you can describe it in words, you can get a reasonable visual reproduction in hours or minutes.

Now, should we really be so focused on language as the only relevant AI capability? The Turing test, where a machine can produce conversational responses indistinguishable from humans, has long been an aspirational goal of AI systems. Large language models blew past this landmark last year (in some respects…), leaving some of our neuroscience community feeling a bit left out. After all, conversation is a rare behavior in animals, and intelligent biological systems exhibit a myriad of physical behaviors that are highly relevant to potentially intelligent artificial systems. NeuroAI, a consortium of neurosciences AI researchers, released a manuscript detailing a new version of the Turing test, which they dub the “embodied Turing test” that incorporates behavioral assessments. In the embodied Turing test, the candidate system should have observed behaviors that are indistinguishable from the predicate biological system. So, if a rat-bot moves and interacts with an environmental scenario at a performance level indistinguishable from a biological rat, the rat-bot would be said to pass this embodied Turing test. The test enables the creation of common task frameworks and catalyzes new collaborations and reproducible research in the neuroscience, AI and engineering communities. Onwards!

🤿 Deep Dive: the new ChatGPT is more powerful than ever?

Usually we try to stick to our academic roots, but today we have to double click into OpenAI’s newest bag of tricks, and how it supercharges chatGPT, and hopefully how it’ll soon help out your practice. This week, chatGPT was upgraded with three new features - each of which we’ll go into separately: 1) plug-ins, 2) browsing and 3) code interpretability.

Plug-In’s

This may be the early signs of an AI ecosystem type app store, where companies host their integrations with chatGPT. In essence, this allows users to complete a litany of functions across a wide variety of apps - from OpenTable to Zapier - all within chatGPT.

“Find me a Thai restaurant reservation with a happy hour, next Tuesday for six people, on the West Side” → ChatGPT can not only execute this function on OpenTable, but you don’t even need to tell it to use OpenTable, it’ll just figure it out based on the request.

Epic is planning to integrate GPT-4 into their systems. While I have no faith that there will be large scale clinical integrations anytime soon (happy to be proven wrong), what if this type of a system was how our EMR’s worked? Simply open a patient's chart, and query or command away:

• “Make NPO at midnight” → Order placed, note updated. One might even imagine a friendly patient facing assistant, politely letting them know of this change in the evening (instead of when their breakfast gets skipped)

• “Pull up most recent and comparison CT scans” → PACS opens up automatically with appropriate CTs queued up.

• …the list goes on

Browsing

What’s the worst part of using MDCalc or UpToDate? Inputting the data, and actually searching for the content.

With browsing functions turned on in ChatGPT, this (might) get easier.

There are two aspects of this update which are important for clinical deployment. First, it should greatly reduce the hallucination of chatGPT- particularly if you point it in a particular direction. “Show me the most updated GDMT guidelines from the American College of Cardiology” should ideally allow browsing in a directed fashion. In medicine, we often know exactly where the information we need is located, but navigating to it takes up precious minutes which compound across 20+ patients a day.

The second key innovation here is in the screenshot above, which is that now, chatGPT should be able to return the sources by which it retrieved information. This is critical as it greatly reduces the “black box” aspect of large language models in deployment, but also allows for AI to be seen as a true “assistant” versus something which does the work “for us”. We already browse the internet for resources, this just helps us do it faster.

Code interpretability

ChatGPT has always been excellent at producing code. But now, it can actually run the code. Here is the OpenAI president (a must-follow, BTW), showing how the new chatGPT can shorten a video by just asking. Their demo shows the ability of chatGPT to plot graphs, manipulate data, and produce figures. Data Science 2.0 is here.

I think this brings about a new way of not just developing software, but developing ideas, products, and tools. We must all become much better at translating our thoughts and our vision into words. Soon, whoever can do that in the most efficient and effective manner will have unprecedented access to the worlds best software engineer.

Imagine the implications for research. We can execute experiments, reconfigure data, generate figures simply by asking. This is a huge leap forward, but the scientific and medical community must get better at communicating what we want. We may also need to rethink how we work with data in the first place.

Is the adage of hypothesis driven experiments outdated in the era of billions of data points and instantaneous data analysis? Are we leaving too much on the table by not performing more broad, exploratory works, perhaps even led by the machine and facilitated by the experts?

Imagine this scenario: there are two PhD students with the same backgrounds, skillsets, and access to the same large corpus of data, and tools (including chatGPT). One student proceeds to create new knowledge via the traditional route of reading the literature (with AI assistance), coming up with a reasonable hypothesis with their PhD advisor, and running experiements (in this case via large datasets) to validate it. The second student on the other hand, spends a month with chatGPT and the data, simply exploring relationships from first principles. Asking open-ended questions, and seeing where the data takes them.

What might the results be? Whose data would you trust more? Who might find the bigger breakthrough?

I don’t have these answers. But I think it’s time we start having these discussions.

🐦🏆 Tweets of the Week:

Share

Subscribe now

It was the start of the March Madness Tournament this week, and not the college basketball contest. No no, this week we saw nearly every A.I. behemoth - including , , OpenAI, Anthropic, and more release new models, products or applications. This matters for the MD of today, and matters even more for the MD of tomorrow.

This describes our week quite accurately:

Lucky for you - we’ll do the heavy lifting, keeping track of the who’s who, what’s what, and bringing it back to the OR, clinic, hospital, or wherever else you find yourself delivering care.

Table of Contents

📰 News of the Week
🤿 Deep Dive:
🐦🏆 Tweets of the Week

🔥📰News of the Week:

If you think A.I. is real, you need to read this edition. There aren’t too many weeks where things “feel different”. This is one of those weeks. Albeit, those weeks are happening more and more often. Exciting times!

Apple ( ) , who have been characteristically slow and methodical in their release of technology, may be coming out of hibernation. Allegedly, they will be launching a new Siri + a new AI Headset in June. Being able to augment your reality in real time is depicted here for interior design, but what about preoperative planning?

What about helping a patient visualize outcomes, either for motivation or realistic expectations? If built from a clinical-principles first mindset, these tools can be powerful tools in the clinicians and surgeosn armamentariam.

Okay, augmented reality was the appetizer. On Tuesday, OpenAI released the much anticipated GPT-4, the newest upgrade to their line of Large Language Models (of chatGPT fame). GPT-4 is fucking awesome.

We hate making you do work, but you owe it to yourself to watch Greg Brockman (President and Co-Founder of OpenAI) demo GPT4. Amongst the crazy sh*t he did:

1. Copied and pasted a section of the tax code and had it do his taxes.

2. Copied and pasted and entire websites worth of code documentation and had GPT-4 create its own chatbot (discord)

And then the craziest part,

He drew a website on a physical piece of paper, took a picture of it, sent it to gpt4, and it created the website for him 🤯

This will have implications for everyone. But we’re just getting started. Let’s round up the news and then get to the implications at the end.

Not to bury the lead, but Google also showed their newest clinical large language model achieved >85% on the USMLE… uhh…yeah thats really good, and probably puts the AI somewhere in the 260’s...

Separately, Google announced their entire workspace would be powered by AI. We’ve all been there - long day in the OR and now you’re 12 emails behind on a thread; let Gmail summarize A.I. it for you no problem. This, amongst many other features.

Ok lets keep going:

Because Microsoft could not be undone by Google, they announced Office 365 would be powered by an AI copilot.

Academics, Surgeons et. al.,

Save your clinical duties, where perhaps A.I. must meet a significantly higher bar for it to be implemented, or there need to be institutional strongholds lifted, etc. Nonetheless, these tools can help you become a better administrator, teacher, academician, leader and innovator.

Here’s on example: we all have data sitting on our computers, messy, waiting to be analyzed. We can now chat with a bot which can write a script to clean that data for us. Never done it before? Ask how to do it. Don’t know which windows to open, which buttons to click, which applications you need? All you have to do is ask.

🤿 Deep Dive: Learning with AI

For those of you living outside of the bizarre job non-market of medicine, this week was “Match Week” for final-year medical students in the US. After months of interviewing at dozens of hospitals for privilege of being absurdly underpaid for 3-7 years, these soon-to-be MD’s approach the Hogwarts sorting hat of the National Residency Matching Program algorithm and are promptly assigned into fictional houses (called residencies) that will forever change their adult lives. This is a high stakes match: putting a Slytherin into House Hufflepuff could be disastrous. Meanwhile, third year medical students are gearing up for influential fourth year rotations. Second year students are taking Step 1 of the USMLE licensing exam. Spring is a stressful time for all medical students, so if you see one of them out there, give them a hug. And, before you ask, Dhiraj did great!

Here at Ctrl-Alt-Operate, we are all learners, whether in formal degree and training programs or in self-directed and lifelong study. This week I’ll ask you to invest 10 minutes of your time thinking about how we can use AI to improve knowledge and technical procedural learning in medicine and surgery.

First, I should specify some contours of the clinical learning environment that affects all physicians from medical students to practicing physicians.

1. Students of medicine (at all levels) are adult learners. Adults learn differently than children, with unique needs and characteristics. Some of these principles include self-directed learning, the need for relevance and applicability of new knowledge to the learner's life or work, and the use of experience and prior knowledge as a foundation for new learning. Adult learning theory also recognizes the importance of a learning environment that fosters collaboration, feedback, and reflection.

2. Across domains of human performance, expertise and mastery are best developed through deliberate practice. The definition of deliberate practice, developed by K Anders Ericsson is the oft-forgotten prepositional phrase of the “10,000 hours rule” popularized by Gladwell and others. Expert performance results from a specific style of purposeful and focused practice. Deliberate practice involves identifying and working on specific weaknesses, receiving feedback, and constantly pushing oneself beyond current skill levels. Through deliberate practice, individuals can continuously improve their performance and achieve mastery in a particular domain. Although the “10,000 hours rule” is well known, most forget that “10,000 hours” is a noun adjunct, not the thing itself. Deliberate practice is the thing we care about. Clinical medicine is lacking here…

3. Clinical medical practice does not provide deliberate practice. In clinical care, the primary focus is on treating patients. Learning may occur as a byproduct of repeated high stakes performances, but playing Carnegie Hall every day is not the same thing as rehearsing. Deliberate practice requires timely, tailored feedback but busy professional often lack the time, training or incentives to provide guidance, coaching and skill-building. Deliberate practice requires structured sub-task repetitions, like playing scales on a piano, but the variety and unpredictability of medical situations rarely permit “pausing” a patient’s care to repeat a sub-task until it is mastered. Moreover, overt mistakes are often subsumed into a culture of silence and shame resulting in missed opportunities for improvement.

Might we be able to harness AI to deliver deliberate practice to adult learners in clinical medicine? Two key facts.

1. AI’s are being baked into major software platforms. Our prior section showed how both Gsuite and office are receiving deployments of their organization's respective LLMs.

2. Large Language Models (particularly multimodal vision-language models that we covered last week) are likely to achieve expert human performance on tests of medical knowledge. Although press releases leave a great deal to be desired, the trend is undeniable. Models are sequentially approaching human expertise in the limited publically available test data (allegedly segregated from the training dataset).

But our question this week isn’t “how good are LLM,” but “how can we work with LLM to get better,” so let’s keep diving further. What should adult learners in search of deliberate practice do with a medical knowledge expert, text generator and parser, and chat-interface in their pocket?

1. Start by understanding the observed behavior of March-2023-caliber foundation models. As always, Cowen and Tabarrok have been leading public intellectuals in this corner of the internet, and their tutorial-spotlight on how novice LLM users can use AI in learning economics is influential.

2. Having understood the general observed behaviors of LLM, here is what doctors need to know in order to learn clinical medicine using AI assistance. Some of these strategies can be performed using chatGPT today, others need building (attention: computer science team mates!)

   1. Accelerate conventional knowledge management and spaced repetition strategies using chat interfaces. Use AI to generate questions for an article you are reading, since answering questions improves knowledge retention. Want a mnemonic for something you need to remember? Ask chat GPT for the mnemonics and the cloze deletions that you can copy-paste or reformat into .csv for direct import.

      

      Multiple mnemonics can be generated but the end-user has to decide if the underlying medical facts are true, false, useful or useless.

   2. AI for medical information support: Inline comparison of literature based on clinical medical notes. Imagine a note-writing interface the presented the most clinically relevant guidelines to the medical note you were writing, based on blue-ribbon information sources. Having trouble remembering the components of the CHADS2-Vasc score or when to use it? What about those latest American Diabetes Association guidelines ? Want to save those guidelines for later review, or future use? Information support as opposed to “decision support” (a truly horrific euphemism) represents the best of AI-human systems, and can be turned into knowledge acquisition and retention systems. Save the articles you rely upon, and then use the tools from part a. above to turn those articles into questions. Too difficult? Summarize the articles and tie them into the cases you are actually seeing using the generative interface.

   3. Exploring the edges with AI: Let’s say you’ve identified an area where you want to improve, but don’t exactly know where to go. For example, pediatric TBI. You might want to test your knowledge with questions, but where can you find them? Consider using LLM-generated questions as a starting point for further exploration, recognizing that the LLM is not a gold standard reference text, but a probabilistic representation of language.

      

      

      Now, you can’t ask LLM’s for references and trust them directly, as this is a major source of confabulation/hallucination. But you can use elicit.org (screenshot below), Google Scholar or the venerable PubMed search engine.

      

   4. Customized Knowledge Course Creation: You can then ask the LLM to generate knowledge content to brush up on topics that you want to understand better. Again, the key is to view these as possible questions rather than the gospel of medical management. However, it is a reasonable starting point.

      

   5. Generating Learning Apps with no-code AI assistance. We will touch on these techniques in upcoming newsletters. For now, you should know that innovations that can dramatically reduce the time and expertise required to build your own learning and knowledge management tools are already in production.

For any educators in the audience, Ethan and Lilach Mollick have a nice guide here

🐦🏆 Tweets of the Week

Share

Subscribe now

Remember chatGPT, that little AI chat interface that changed public sentiment about (almost) everything in AI? Well, it’s about to happen again. Supposedly as soon as next week, is going to release GPT-4, and it will have some crazy new bugs disasters capabilities that you should anticipate. So this week, we dive deeper into what you need to know about multimodal large language models to keep you on top of the news before it comes out. Whether a release actually occurs next week or not, it’s a good idea to understand this technology at least at a high level.

We’ll keep an eye on current developments but remain focused beyond the horizon to spot the next upcoming wave of innovation, disruption, and enthusiasm.

Table of Contents

📰 News of the Week
🤿 Deep Dive: AI assistants for surgical education.
🐦🏆 Tweets of the Week

Share

Subscribe now

📰 News of the Week

GPT-4 is coming. According to Deutschland, this could happen as soon as next week. The highlight of GPT-4 will supposedly be it’s multimodal nature, meaning that it won’t be limited to text input and output. Although no one knows what the consumer facing aspect will look like, the underlying model will likely be incorporate insights from our deep dive feature on Kosmos-1, Microsoft’s new multimodal LLM.

Several companies have set their sights on the age old EKG interpretation “problem” and at least one new entrant is poised to deliver their FDA-approved algorithm for ~$10/EKG. We aren’t holding our breath.

All large databases can be queried using natural language via GPT’s. Here’s an example of the US census data. Go ahead, ask it your weirdest questions. Due to data access requirements, it’s not clear if this will ever be delivered for AHRQ HCUP datasets (like the nationwide inpatient sample), but we can dream…

And it’s no surprise - the first AI-generated voice scams are out there. Beware if you get a call from that long lost relative that sounds exactly like them …

And on a somber note, the startup world was rocked by the collapse of Silicon Valley Bank, one of the premier lending institutions for tech and life sci. SVB rose to the top 20 US banks due to its mission to provide cash flows and banking services to startup founders who otherwise might not rate them based on credit histories and balance sheets (for example, by lending against projected future earnings). Many retrospectively-horrifying takes described the institution as "growth-y in an industry that isn't growth-y", and indeed its deposits tripled in the last 3 years to est. ~ $220B (a fraction of et al.).

This week, the bank announced $2B in losses due to a treasury bill strategy that failed in the current high interest rate climate. The announcement caused a bank run of about $40B in withdrawals (~20% of deposits) that couldn't be met, the bank couldn't raise funds to make itself and its depositors whole, and voila! Since the vast majority of depositors held much more than the FDIC $250K limit, unless a creative solution unfolds this week (which it probably will), depositors face a months-long resolution process that might only return a fraction of their balances.

The downstream ripples of this failure could rock the startup community. As many as a third of startups won't be able to make payroll. And not that anyone cares about crypto but the USDC “stablecoin” (allegedly pegged to the US dollar) couldn’t access its accounts, creating a drop of over 13% until Coinbase froze trading.

📚Deep Dive: Language Is Not All You Need

Okay, medicine geeks. Time to fill you in on a computer science geek inside joke. In 2017, Google Brain’s paper, Attention is All You Need described an architecture called a Transformer which just so happens to be the T in things like…chatGPT. That paper has been cited a whopping 67,000 times since publication and has since seen pithy (but very real) paper titles such as Money is all You Need (referring to the cost of machine learning research) and the paper below.

This past week, dropped another bombshell with their multimodal language model, called KOSMOS-1.

What is the difference between a multimodal large language model and a “regular” LLM ?

Kosmos-1 can input both language and images, learn in context, reason, and generate output

Is it a little amusing the computer knows the boy is sad because his scooter broke? Yes. But this is a big deal because artificial general intelligence requires a convergence of language, multimodal perception, action, and world modeling. KOSMOS-1 aligns visual perception with language models, opening up new possibilities for machine learning in high-value areas such as multimodal machine learning, document intelligence, and robotics.

The team trained KOSMOS-1 from scratch on web-scale multimodal corpora, including text, images, and image-caption pairs. They evaluated the model's performance on various tasks, including language understanding, generation, perception-language tasks, including multimodal dialogue, image captioning, visual question answering, and vision tasks - pretty wide variety, though not necessarily too deep in any domain. It makes for some cool graphics though.

So what does this mean for surgeons? Think about the OR for a second. Think of the multimodal data approaches there. Speech happens once and never again, differences in force and pressure are applied constantly, different people have different vantage points. All this is data that may be impossible to capture, but it may not be. These types of models at least enable us to start analyzing the data if we capture it.

We discussed the OR Black Box a few weeks ago. Most surgeons do not like it. But what we could decode actions using these types of MLLM models?

Second, MLLMs provide a real step forward towards some of the constraints behind chatGPT. A big sticking point is that chatGPT is text-based only. And a big sticking point with a lot of visual A.I. technology is there is very little room for nuance. We've all heard of radiology AI that gets things wrong. But what if it a radiology AI had the conversational component of chatGPT as a “friend” to help the overnight reader? Or for the on-call surgeon? With MLLMs, we could potentially create more comprehensive and accurate AI models that could bridge the gap which currently exists.

And what about the implications of MLLM for the world of general AI and “human-level” performance across a variety of tasks. These models might be the key to unlocking one of the greatest challenges in AI that no one likes to talk about: we are, order of magnitude, running out of textual training data. I know, it sounds crazy to say that humans haven’t digitized enough text in our history, but we do have some early data that could suggest that case. There is a fantastic review of AI scaling “laws” based on what we have learned from existing LLMs over at dynomight.substack.com and citing Villalobos et al Considering price, compute and training data, we will need improvements in all three in order to use current technology to reach dramatically lower levels of model loss. However, of these three, only price and compute seem to have the possibility to scale dramatically. The quantity of textual training data hasn’t dramatically changed over the past 10 years, roughly 10^14 text tokens on the potentially accessible internet (although humans might generate 10^16 tokens/year globally across all media, how can we get those?). However, the quantity of available visual data is pathetically small (10^9 or 10^10) compared to the quantity of acquired visual data each year (10^12 - 10^16 depending on how you think about video). And, what’s more, it’s not immediately clear that one image would only generate one token. A lower bound of 10^1 (upper bound unknown, 10^3? more?) tokens might be generateable from each image, describing the subject, context, style, metadata and many other features. And since video is recorded at 24+ fps, each second of video represents another source of ≥ 10^3 tokens.

Watch for this phenomenon to explode 🚀 like crazy the next 6-12 months. Oh, and don’t think that it’s just Microsoft …

Best of Twitter

It hardly seems fair to give a tweet 🏆 trophy to @StanfordHAI but this one is a worthy long read. If you’ve got a nice fire going and are feeling a bit tired on this laziest of sundays (daylight savings time, for those outside the US, is totally insane), take a look.

Yes, we do love google’s models, but we never get to see the code, or the data, or even use them. So umm, nice press release again…

This is a close runner up and may get a bigger feature in subsequent weeks. Giving clinicians validated AI risk assessments of patient mortality doesn’t make patients live longer (duh), or help clinicians at all. So, umm, can we get some actionable information? But if it provided useful info, the model might be accountable…

A two-fer of awesomeness from our general surgery colleagues.

Feeling inspired? Drop us a line and let us know what you liked.

Like all surgeons, we are always looking to get better. Send us your M&M style roastings or favorable Press-Gainey ratings by email at ctrl.alt.operate@gmail.com